Clockwork Raven steps in to do what algorithms can't: it sends your data analysis tasks to real people and gets fast, cheap, accurate results from Mechanical Turk.
Clockwork Raven is a web application that you set up for yourself or your organization. You upload your data, build a form to send out to reviewers, and then send it off to Mechanical Turk to have real people answer your questions. Then, you can review the results in Clockwork Raven or export them to a number of formats and use whatever tools you like.
Here at Twitter, we use it to get large number of subjective judgments (“How relevant is this Tweet to this user?”), complete tasks that require human intelligence (“Find 10 Twitter users that relate to this topic”) and evaluate algorithms (“Here's what topic we think this tweet belongs to. Is that right?”). Use it to analyze data, evaluate your algorithms, train machine learning models, or anything else you'd normally have to do yourself.
Clockwork Raven is simple to use. Use the drag-and-drop form builder to build complex analysis tasks without any knowledge of HTML. Import and export data from CSVs, which work with everything from spreadsheet programs like Excel to statistical analysis tools like R.
Clockwork is free to use and the source is available on GitHub. Missing something? Find a bug? let us know, or fix it yourself. You can do whatever you need to make Clockwork Raven work for you.
Clockwork Raven has a master list of every evaluation you've launched. Send links to teammates so that can see your results, and copy others' evaluations to use as a starting point for your own. Clockwork Raven even integrates with LDAP so users can log in with their existing credentials.
The drag-and-drop form builder allows you to design forms to send out to Mechanical Turk with no knowledge of HTML. Easily add items from your uploaded data to create dynamic tasks. Create reusable components to display commonly-used items: here at Twitter, we have components for tweets, timelines, and search queries, which makes launching rich tasks dead simple.
The integrated analysis tools displays your data and lets you slice it across multiple variables. See the results of any multiple-choice question you asked the Mechanical Turk workers, and segment that data by metadata fields in the data you uploaded. You can also see individual results and search through responses to your free-response questions.
Follow @clockworkraven to get updates about Clockwork Raven.
Join the mailing list to ask questions and get advice from other Clockwork Raven users, and to discuss future development of Clockwork Raven.