Monday, February 26, 2007

Edit Assistant - Transcript Editing Tool

Edit Assistant is an automatic filter and speaker identifying tool written by me.

I wrote this tool to help those who perform CastingWords Edit HITs, but it is also helpful for editing your own transcripts before you turn them in to get a higher score (which you want for bonus based HITs to get paid more and plus higher grades mean you get a higher score which will allow you to do those elusive Expedited HITs which pay way more then normal HITs).

I hope those of you who edit find it useful. But remember you still need to listen through the audio, this just helps with some of the mindless work.

via my post on Turker Nation

Friday, February 02, 2007

You Can Help Find Jim Gray (from home)

Repost from Werner's blog

Computer science icon Jim Gray mysteriously disappeared after a solo trip with his sail boat outside San Francisco Bay. The coast guard has been searching for 4 days but has not been able to locate anything, not even debris. On Thursday 3 private planes searched through the coastal areas and they also returned unsuccessful.

Through a major effort by many people we were able to have the Digital Globe satellite make a run over the area on Thursday morning and have the data made available publicly. We have split these images into smaller tiles that can be easily scanned visually and stored into the Amazon S3 storage service. We then created tasks for reviewing these images and loaded then into the Amazon Mechanical Turk Service.

This is where you come in. We need your help in reviewing these images to see whether you can locate Jim’s boat in any of these images. Please go to the Amazon Mechanical Turk site and help us find Jim Gray.

The weather conditions were not ideal as some areas were cloudy, but we can still look for him in those places where there is a somewhat clear view. We hope to get more satellite data in the coming days of a wider area. The current images are panchromatic with a 0.82m, and Jim boat would be about 6 pixels in size. Please visit the Amazon Mechanical Turk site for more details.

I have to stress that many individuals and companies are to thank for making this possible; many academics friends relentlessly worked around the clock to get access to the data, many industry friends of Jim functioned as connectors to hook up officials and individuals, and people from NASA, Digital Globe, Microsoft, Google, Oracle, Amazon and others worked hard get to the data collected and available on a very short time scale. The Mechanical Turk team worked deep into the night to make this work.

Now it is your turn, go find Jim Gray.

If everyone completes a few of these HITs, we will get through all of them. And possibly find something that will help find him.

Monday, January 29, 2007

Turking for a Living

Imagine working any hours you please.

And never being late, just simply working when you feel like it. Plus you can take vacation whenever you wish and as much as you want without having to give notice. Heck you can even live anywhere with a decent internet connection.

This could be the life of someone that Turks for a living.

Of course I wouldn't dare try this without some money saved up. But if you are currently doing a minimum wage job and the prices on mTurk went up a little. Why not? Even if for no other reason then to say "Hey I Turk for a living!".

Of course I definately wouldn't go out on such a limb quite yet. Perhaps give mTurk another 6 months and you could. Maybe a breed of Turk Bums could be born? Highly skilled workers that have figured out how to do some steady task on mTurk super-fast and so they only work a few hours a day. That'd be a sweet life.

Let's keep dreaming.

Inspired by Jonathan's Comment on this thread

Thursday, January 18, 2007

Janurary Status (This post is mostly a rant)

So I haven't posted in quite a while. I had a few things I felt like talking about or making pretty graphs to make, but I haven't found the motive (I mean the time!)

Amazon recently sent out an email to all registered Turkers (we believe), meaning a number of people who had long forgot about mTurk have come back and seen the new HITs.

And really mTurk's HIT offerings are a bit different. Geospatial Vision continues to stun me with excellent flash powered HITs though for 50 images 8 cents seems a little cheap. But that's with all HITs.

CastingWords continues to post it's HITs with only slight changes over the months. (And some new types ever once in a while like Assess Audio Quality)

Von Kemplen seems to be quiet, though when they do post HITs it's often under a different name.

So all and all mTurk is quiet. Still no major new use but a worker could certainty make a few bucks an hour in their free time on it (as opposed to playing a game or going out and spending money to fill one's free time)

Personnaly....I think that how much one could make an hour could go up quite a bit with a specialize "client" that would have plugins for different HITs, and do a number of things to increase the speed of work. But scrapping the info is a pain and Amazon has yet to release the "Worker API".

But I'm confident that this year we be one of growth for mTurk. Though probably still not as energetic as the energy behind Amazon's S3 and EC2.

Sunday, December 24, 2006

Merry Christmas Turkers! (And Requesters!)

So it's Christmas! (Well tomorrow is) I hope you've got all your shopping done (I still need to start my wrapping!)

As far as mTurk lately, we've had thousands of Tagging HITs from Amazon which basically just fix it's product labeling system (Awesome use of mTurk). And then of course the Unspun HITs have been steadily entering the stream. And of course there's been little bit of this and that.

But let's forget about work! And just enjoy a Merry Christmas with family (Get along with them! I know it can be hard!)

So, I'm simply rambling here, but hey, have a Merry Christmas, and a Happy New Year. And of course during this busy time of year, remember the holidays are about fun with friends and family, not stressing out over having enough time to get everything done. Make cuts if needed and smell the flowers. And in worst case scenario post HITs to get your work done! Heh.

Have a good one. :-)

Monday, December 11, 2006

Get Paid to Pirate!

Here's a new use of mTurk that I haven't seen before from Nat. He'll pay you a whole 15 cents to:

Find entire The Handing Down by Ed Gerhard in MP3 Format and upload it here.

So, I tried checking to make sure the song isn't in public domain and I don't see any evidence that it is. In fact, here you can go get the album it's in, for 15 bucks.

So Nat, I like your creativity. But if you want to pirate some songs, go get Limewire and leave the Turkers out of it.


Thursday, November 30, 2006

The Sheep Market: Two Cents Worth

Aaron Koblin has published his thesis paper on the rather intriguing Sheep Market project.

To be honest, it is a little over my head, but still a good read. He brings up the very valid point about how, things like his Sheep Market, are clearly not worth the wage provided. Yet Turkers complete the tasks. The only thing truely comparable to his project is the YRUHRN book. Another half-experiment/half project that used the Mechanical Turk.

So just what does motivate the Turkers? What motivates someone playing a game? Seriously, the video game industry is huge and the sole purpose of most of the games is simply to get to the next level. Now, some do have intriguing storylines and other stuff of that sort. But if gamers simply wanted that, they could go watch a movie.

The high point of my Turking, and all of the old Turkers was the Image Adjustments. Semi-mindless tasks that we did over and over for 3 cents (in November of 2005). We did however make decent wages (6-12 bucks depending on your setup) But what was the point? Was there any pride in doing it? From my standpoint, no, it was just about the money.

Now, you could put HITs on mTurk into a two groups.

Group A: Mindless tasks that one couldn't take pride in, essentially tasks with right and wrong answers. Like "Is there a person in this picture? Yes | No"

Group B: Mindful tasks, like drawing a sheep, transcribing a podcast, giving advice, Ranking Your Top 3 of something. These are basically HITs with an very large or infinite number of right answers. In fact in reviewing of HIT results you see the personality of Turkers with these sorts of tasks. To the point that you sometimes recognize who did some work without checking the Worker ID.

Now Group B tasks degrade into Group A tasks when they become repetitive. For instance, if all of the sudden, I am drawing 100 sheep a day, 7 days a week, then sheep drawing has become a mindless task for me.

And so, I think this is a solution that Automated Webshoring can give us. In life there will always be repetitive tasks. Most lower paying jobs (and many higher paying jobs) become just a sea of mindless work for employees. Now you learn to live with it and technology has ridden us of some of it (copy machines!). But what if we "Websource" unskilled labor, by breaking down the tasks we give workers the ability to choose what they do, on a momentary basis. So one minute I'm a Fast Food worker taking an order, the next I'm helping a child with there homework, then I'm typing a license plate number from a picture taken by a red light camera, and then I'm typing out a small chunk of a podcast.

Now granted, some of this is down the road....some might not even be on the road. But to me, it seems like a bright future. It seems like it's something that could change the world, perhaps for the better, perhaps for worst. But one thing is for sure, our systems will keep getting smarter, faster, and more efficient by whatever path companies find.

via AWS Blog (Yes I know it took me a week to finish writing the post)

Wednesday, November 29, 2006

Contracts and Mortage Lookups are Flooding my Window!

Lately whenever I visit mTurk (which is often), the top of it is flooded with a bunch of HITs that I have no interest in. And most Turkers don't have any interest in them either or they'd be disappearing at a decent rate.

Now you've all seen the "Mortage Lookups" from Enrovia Research. The issue with these is they take a long time and pay next to nothing.

And then there's the newer but similar HITs, "Contract details" from JB (JB is actually an Amazon Employee, note the email address)

For those unfamiliar with these two HIT types, in a nutshell you go to another website, find scanned records, and then extract information from those.

Now I personally think both of these HITs should pay more, so do all of you. So does everyone. We've already established that for all HITs. But business is business, you pay the smallest amount possible to get your work done.

The issue I have isn't the pay, the issue is that these HITs could be simplified with programming. They have workers manually go to a site and lookup documents that could be scrapped and put into an HIT, or the Requesters could release a tool to quicken the process (remember those Greasemonkey scripts). Or even simply make these into External HITs (HITs not hosted on mTurk but appear to be) and inside the HIT have a frame that takes users to the site so they don't have to deal with having multiple windows/tabs.

Something that many Requesters seem to miss is that the trick to mTurk isn't high pay, but rather simplification. Break tasks down as much as possible, into very small, very quick, 1 or 2 cent HITs. Even if the only way to break them down and simplify is through additional HITs (which of course can raise costs)

So Requesters do yourself a favor, stop for a moment, look at your HITs and brainstorm out how to simplify them, even if it means a week worth of programming to automate it. The time investment is worth it if you are going to be sending out thousands of HITs.

Simply put, make the task easier, and you can pay less and get results faster.
(This conclusion is based off my personal experiences as a Turker and Requester and from what I've seen on mTurk)

Labels: , ,

Monday, November 27, 2006

Blank CAPTCHAs! Freeze your Turking!

Ok, so I was innocently just plowing through a bunch of Survey Verification HITs, and then BAM! I get a blank CAPTCHA. I was using Firefox....and was stunned, I had read about this issue before on Turker Nation. But I figured it had since been resolved.

I'm unsure still of what causes this issue, but I hope they address it soon because it sucks when you are on a roll and then stuck.

The fix is simple, go into IE, and accept an HIT, and the CAPTCHA will be visible, you simply type it in. Below I've attached the images to prove that one is blank and I'm not blind (also so that the mTurk developers that read this blog can look at them if this is an unknown issue)

Thursday, November 02, 2006

Can You Speak Italian?

von Kempelen sure hopes you can! 12 HITs have shown up that pay varying rates. Exactly how much per paragraph I have no idea, because you can't see these HITs without accepting them and you can't accept them without an English to Italian L1 Qualification Score of 50 or more. Granted my Italian is far weaker then my English (aka me no speakie italianie), but I do love pizza. Does that count von Kempelen? Come on....please? No? Darn.

But seriously it's nice to see these sorts of HITs on mTurk. Paying for translation work is expensive, but there's tons of people out there that are bilingual (most of Europe last I heard). And if you are actually fluent in two languages, it seems to me like it'd be a semi-effortless job (yes? Or say you are trying to become fluent and you have a pretty good grasp, you just need some experience, translation HITs would be a great way to get it. And pass the time and make a little cash while you are at it. Then once you got the hang of 7 languages or so, go get a nice 6 digit job with some company.

Happy 1st Birthday mTurk!!! WOOT!!!!

So today, mTurk has turned the big 1. That's right 1 year ago, on November 2nd mTurk launched. The net went crazy..."Wait...I can sit here in my underwear and get paid? No way." But it was true, there was so much excitement about mTurk that the servers were down half the time for the first week after launch. Granted mTurk has now gotten off to a slow start, the Turkers all hang out at Turker Nation and there's not that many of them, but it's been a fun year. Hopefully this next one will bring us many many more HITs, filled with joy and gooey stuff.

Now I had wanted to throw together like a scavenger hunt or something fun for today, but that's a lot of work, and I'm lazy. But I did make this HIT, where you can go get a whooping 5 cents (count em, 5!) for drawing a nice b-day thing for the Turk!

I am posting all the decent ones here.

And here's my favorite so far
From mTurk B-Day

Little of this, a little of that

So I've had bad updater ship lately. Sorry!

First off there's a nice article on Turking by the Christian Science Monitor.
Which states that the reason AskForCents stopped was because of NowNow and Askville, afraid of some corporate competition?

Secondly, von Kemplen is actively back into mTurk, they've had some weird test HITs up, that allow you to draw right in mTurk (using VNC on EC2!!) As well I've been seeing some language translations up from them.

The YRUHRN book is great, I am somewhat disappointed in all of the Turk references, which makes it less interesting to my non-turker friends. But as a Turker it's a wonderful book that gives you insight into what sort of people the Turkers are.

That pretty much wraps things up. I'll try to get back into my regular cycle of posting like I should.