I’ve only recently started using Amazon’s Mechanical Turk (Mturk), the 6-year old brainchild of the brilliant Jeff Bezos, which allows you to assign tasks that can’t be automated through programming and requires a bit of human ingenuity. It prove that humans haven’t been completely replaced by machines and software. You can get those pesky questions answered regarding your survey, find the correct category for a product listing or even identify product images, discover e-mails, addresses, and phone numbers, and pretty much whatever your mind can come up with. There are some restrictions which limit you from doing tasks such as having people sign-up for things, submitting your website to social media sites, and collecting people’s personal information(from your mturkers).
Before you go out there and attempt to create a business from the survey results you got from Mturk, you’ll have to consider some of the downfalls of it as well. The main one being QUALITY! Since you’re paying people a few cents to do the task, you can probably assume that they won’t be putting their best effort in it. So you’ll have to have some design considerations when creating your task which I’ll provide examples of further on in the article. Second consideration is understanding how the users of mTurk search for tasks, which through studies has shown that the most recently posted and most available tasks get the most attention. Another consideration is demographics of the population on mTurk, there have been a few studies on the subject. In which they found that the constitution of mTurk is as follows:
- United States: 46.80%
- India: 34.00%
- Miscellaneous: 19.20%
If you’re like me and just started using mTurk, you’ll most likely miss all the developments that have occurred through it’s use and around it. It seems MIT students have taken a special interest in understanding and optimizing mechanical Turks ability. An example of this is Turkit made by Greg Little, which does the following.
TurKit is a Java/JavaScript API for running iterative tasks on Mechanical Turk. You can safely re-execute TurKit programs without re-running costly side effects on Mechanical Turk, like creating new HITs, but still write your program in a straightforward imperative manner—there is no need to unravel the program into a state machine.
If you’re like me, that probably made you even more curious about full functionality of the Iterative Task software, and a demonstration of Turkit as well. There have been some interesting developments using this framework, such as Soylent, which is a crowd-sourced word processor.
Another cool creation adding human intelligence to software is VizWiz which aids vision-impaired individuals answer visual environment oriented questions such as who sent you the letter, and distinguish various items in your pantry. Some of these questions are answered as quick as 59 seconds.
Also for those of you that stuck around till the end, and were wondering why it’s called an “Experiment Post.” I actually outsourced parts of this article to mTurk, but I haven’t gotten a hang of the Find-Fix-Verify algorithm yet(If anyone knows more about it, please let me know), so I’m doing much of it manually at the moment. I will describe the process (including scripts, templates, etc…) that I used to make this article in my next post.
