Codewebs: a Pedagogical Search Engine for Code Submissions to a MOOC

A knowledge of computer science is increasingly becoming an essential career skill in today's world. This demonstration showcases the Codewebs system, which we are developing for leveraging a massive database of code submissions to an online programming intensive course in order to deliver high quality feedback to students. For this demonstration, we will run the Codewebs system using a million code submissions to a machine learning course offered through Coursera.

Under the hood, Codewebs can be viewed as a search engine for efficiently querying a massive collection of code submissions that all try to implement the same functionality. With so many submissions of the same assignment, we are able to obtain a dense sampling of the solution space, allowing for submissions to be meaningfully linked into a network connecting highly related solutions. The majority of erroneous solutions even in such a large dataset fall into a relatively small number of clusters that are made evident by the network. Human instructors can then evaluate one or a few assignments from each cluster, and their comments can be diffused along the network to provide specific feedback to a large number of student solutions. One of the novel features of our work is that we can compare code both by syntax and semantics.

See also: http://www.stanford.edu/~jhuang11/research/pubs/moocshop13/codeweb.html