Thursday, November 30, 2006

The Sheep Market: Two Cents Worth

Aaron Koblin has published his thesis paper on the rather intriguing Sheep Market project.

To be honest, it is a little over my head, but still a good read. He brings up the very valid point about how, things like his Sheep Market, are clearly not worth the wage provided. Yet Turkers complete the tasks. The only thing truely comparable to his project is the YRUHRN book. Another half-experiment/half project that used the Mechanical Turk.

So just what does motivate the Turkers? What motivates someone playing a game? Seriously, the video game industry is huge and the sole purpose of most of the games is simply to get to the next level. Now, some do have intriguing storylines and other stuff of that sort. But if gamers simply wanted that, they could go watch a movie.

The high point of my Turking, and all of the old Turkers was the Image Adjustments. Semi-mindless tasks that we did over and over for 3 cents (in November of 2005). We did however make decent wages (6-12 bucks depending on your setup) But what was the point? Was there any pride in doing it? From my standpoint, no, it was just about the money.

Now, you could put HITs on mTurk into a two groups.

Group A: Mindless tasks that one couldn't take pride in, essentially tasks with right and wrong answers. Like "Is there a person in this picture? Yes | No"

Group B: Mindful tasks, like drawing a sheep, transcribing a podcast, giving advice, Ranking Your Top 3 of something. These are basically HITs with an very large or infinite number of right answers. In fact in reviewing of HIT results you see the personality of Turkers with these sorts of tasks. To the point that you sometimes recognize who did some work without checking the Worker ID.

Now Group B tasks degrade into Group A tasks when they become repetitive. For instance, if all of the sudden, I am drawing 100 sheep a day, 7 days a week, then sheep drawing has become a mindless task for me.

And so, I think this is a solution that Automated Webshoring can give us. In life there will always be repetitive tasks. Most lower paying jobs (and many higher paying jobs) become just a sea of mindless work for employees. Now you learn to live with it and technology has ridden us of some of it (copy machines!). But what if we "Websource" unskilled labor, by breaking down the tasks we give workers the ability to choose what they do, on a momentary basis. So one minute I'm a Fast Food worker taking an order, the next I'm helping a child with there homework, then I'm typing a license plate number from a picture taken by a red light camera, and then I'm typing out a small chunk of a podcast.

Now granted, some of this is down the road....some might not even be on the road. But to me, it seems like a bright future. It seems like it's something that could change the world, perhaps for the better, perhaps for worst. But one thing is for sure, our systems will keep getting smarter, faster, and more efficient by whatever path companies find.

via AWS Blog (Yes I know it took me a week to finish writing the post)

Wednesday, November 29, 2006

Contracts and Mortage Lookups are Flooding my Window!

Lately whenever I visit mTurk (which is often), the top of it is flooded with a bunch of HITs that I have no interest in. And most Turkers don't have any interest in them either or they'd be disappearing at a decent rate.

Now you've all seen the "Mortage Lookups" from Enrovia Research. The issue with these is they take a long time and pay next to nothing.

And then there's the newer but similar HITs, "Contract details" from JB (JB is actually an Amazon Employee, note the @amazon.com email address)

For those unfamiliar with these two HIT types, in a nutshell you go to another website, find scanned records, and then extract information from those.

Now I personally think both of these HITs should pay more, so do all of you. So does everyone. We've already established that for all HITs. But business is business, you pay the smallest amount possible to get your work done.

The issue I have isn't the pay, the issue is that these HITs could be simplified with programming. They have workers manually go to a site and lookup documents that could be scrapped and put into an HIT, or the Requesters could release a tool to quicken the process (remember those Greasemonkey scripts). Or even simply make these into External HITs (HITs not hosted on mTurk but appear to be) and inside the HIT have a frame that takes users to the site so they don't have to deal with having multiple windows/tabs.

Something that many Requesters seem to miss is that the trick to mTurk isn't high pay, but rather simplification. Break tasks down as much as possible, into very small, very quick, 1 or 2 cent HITs. Even if the only way to break them down and simplify is through additional HITs (which of course can raise costs)

So Requesters do yourself a favor, stop for a moment, look at your HITs and brainstorm out how to simplify them, even if it means a week worth of programming to automate it. The time investment is worth it if you are going to be sending out thousands of HITs.

Simply put, make the task easier, and you can pay less and get results faster.
(This conclusion is based off my personal experiences as a Turker and Requester and from what I've seen on mTurk)

Labels: , ,

Monday, November 27, 2006

Blank CAPTCHAs! Freeze your Turking!

Ok, so I was innocently just plowing through a bunch of Survey Verification HITs, and then BAM! I get a blank CAPTCHA. I was using Firefox....and was stunned, I had read about this issue before on Turker Nation. But I figured it had since been resolved.

I'm unsure still of what causes this issue, but I hope they address it soon because it sucks when you are on a roll and then stuck.

The fix is simple, go into IE, and accept an HIT, and the CAPTCHA will be visible, you simply type it in. Below I've attached the images to prove that one is blank and I'm not blind (also so that the mTurk developers that read this blog can look at them if this is an unknown issue)

Thursday, November 02, 2006

Can You Speak Italian?

von Kempelen sure hopes you can! 12 HITs have shown up that pay varying rates. Exactly how much per paragraph I have no idea, because you can't see these HITs without accepting them and you can't accept them without an English to Italian L1 Qualification Score of 50 or more. Granted my Italian is far weaker then my English (aka me no speakie italianie), but I do love pizza. Does that count von Kempelen? Come on....please? No? Darn.

But seriously it's nice to see these sorts of HITs on mTurk. Paying for translation work is expensive, but there's tons of people out there that are bilingual (most of Europe last I heard). And if you are actually fluent in two languages, it seems to me like it'd be a semi-effortless job (yes?...no?). Or say you are trying to become fluent and you have a pretty good grasp, you just need some experience, translation HITs would be a great way to get it. And pass the time and make a little cash while you are at it. Then once you got the hang of 7 languages or so, go get a nice 6 digit job with some company.

Happy 1st Birthday mTurk!!! WOOT!!!!

So today, mTurk has turned the big 1. That's right 1 year ago, on November 2nd mTurk launched. The net went crazy..."Wait...I can sit here in my underwear and get paid? No way." But it was true, there was so much excitement about mTurk that the servers were down half the time for the first week after launch. Granted mTurk has now gotten off to a slow start, the Turkers all hang out at Turker Nation and there's not that many of them, but it's been a fun year. Hopefully this next one will bring us many many more HITs, filled with joy and gooey stuff.

Now I had wanted to throw together like a scavenger hunt or something fun for today, but that's a lot of work, and I'm lazy. But I did make this HIT, where you can go get a whooping 5 cents (count em, 5!) for drawing a nice b-day thing for the Turk!

I am posting all the decent ones here.

And here's my favorite so far
From mTurk B-Day

Little of this, a little of that

So I've had bad updater ship lately. Sorry!

First off there's a nice article on Turking by the Christian Science Monitor.
Which states that the reason AskForCents stopped was because of NowNow and Askville, afraid of some corporate competition?

Secondly, von Kemplen is actively back into mTurk, they've had some weird test HITs up, that allow you to draw right in mTurk (using VNC on EC2!!) As well I've been seeing some language translations up from them.

The YRUHRN book is great, I am somewhat disappointed in all of the Turk references, which makes it less interesting to my non-turker friends. But as a Turker it's a wonderful book that gives you insight into what sort of people the Turkers are.

That pretty much wraps things up. I'll try to get back into my regular cycle of posting like I should.