Embracing "Confirm Artist Name" HIT's

Well I've never been a fan of the 2 cent "Confirm Artist Name" HIT's, but today I learned to appreciate them. Most can be done faster then Image Adjustments and some of the album covers are pretty entertaining.

Here's my setup, first off I am using Firefox 1.5RC3, with the extension Greasemonkey 0.6.3. Then I am using the Greasemonkey script TurkOp. Next I am using "3 in 1 Magic Joy Box" to hook up one of my Gamecube controllers to my PC, and then I have JoyToKey running, so that I can control my browser using the gamepad. So I set the keys up how a I want them and BAM! For about 75% of "Confirm Artist Name" HIT's I can simply press a few buttons, and it goes super fast. But then part of them the correct answer isn't there and I have to type it out. Which is a slowdown...But must be done. Tomorrow I might time myself a bit and see if I'm actually making over minimum wage on these now. I think if I can just get in rytham with having to type them out part of the time I might be able to make more per an hour then I was with making Image Adjustments.

The End of Image Adjustments?

Well today's been boring, I got 100 image adjustments done this morning. Since I haven't seen anything but Confirm Artist HIT's, which I have no taste for.

So is this a drought of Image Adjustments? Or are we done with them for a while? If I said they were over, 10 minutes later 10,000 would show up, and if I said they will be back soon, they wouldn't. My point being.....I control the weather....Just kidding.

Anyways have a nice day, and try to keep your sanity without your Image Adjustments, my sanity already left hours ago.

A Dry Day

Today has been a very dry day....Almost no HIT's except "Confirm Artist Name"'s

Which tend to have a very high rejection rate for most people. Personally I credit the dryness of this weekend to the holiday, since it's been 4 days since a normal work day, I suspect there are many bots running around eating up HIT's and the banhammer has yet to come down on them.

In other news, the new wrapper is nearing release...it was scheduled for tonight but alas the lack of Image Adjustment HIT's has kept it from final testing....so we have something to look forward to tomorrow.

MIA HIT's

Users are reporting that yesterday's rejected + approved + pending HIT's do not equal the submitted amount of HIT's.....off by hundreds

My own is off by about 20, I assume this is merely a dashboard display issue like the negative pending's. So nothing to worry about....hopefully.

Podcast HIT's

I haven't gotten to try these myself, and there's only been a few, but Podcast HIT's are here. Basically you type what is said. Pretty basic. I've heard they pay 2 cents or 10 cents. I assume the company is testing them and plans on paying a regular amount once they are done with testing.

In other news the servers are running well, and I just ordered myself a new computer monitor with MTurk money (to replace my 7 year old one I am using). As well I bought an adapter on ebay, so that I can try Turking with a console controller (I hope it works with my wavebird)

Servers at a Crawl

As we enter Thanksgiving Day, the servers are at a crawl. I'm not sure why, since it's late at night, when the load is usually low. Maybe more people are up since they don't have to go to work tomorrow? Anyways Happy Thanksgiving.

And keep up the Turking (btw it's rumored that a kickass new Image Adjust Assitant will be coming out in the next few days, not MTurk Wrapper....which is nice, but lacking....something awesome)

First HIT from 3rd Party

Over at Turker Nation, mino spotted the first HIT from a 3rd party. A Website evaluation, only paying ten cents. But then the creator of the HIT popped up and had this to say

Hi,

We are the creators of that HIT.

We are just starting to test that our application can submit HITs to Amazon, so the payment is set low. Once we start processing real surveys, we will be increasing the payment.

Of course, we'd love to hear any feedback on what you think an appropriate payment would be for that type of survey. It will vary depending on the length and complexity of the questions.

Personally I'm very excited that MTurk may finally become more then just image adjustments, as it is intended to be.

Money Problems

Today posts about issues with negative account balances have filled forums and been the subject of few blog posts.

Apparently people are trying to transfer money to a bank account, only to have that money bounce back, they then try again, only to have both transactions be successful. Leaving them with a negative balance. Defiantly not a good thing for Amazon, especially not if someone catches on, repeats a transfer of a large amount several times, then abandons the account. Though hopefully Amazon could contact the bank and get their money back (like they need it).

Anyways just wait a few days after a transfer bounces just to make sure it doesn't end up going through.

Thank You Chris Thomas

So today I decided to have another look at Chris Thomas's "Amazon Mechanical Turk Quicker Image Adjustment" GreaseMonkey Script, and it's come a long way. Still a few annoyances, like I can't set it to full page, and the yellow pages link are annoying, and I don't like the one click submit instead of select. But overall it has a nice set of impressive features, like thumbnail zoom, keyboard only, and auto-accepting of HIT's. So go over and check it it out. (btw he somehow managed to fix the memory leak issue)

Personally I am now using a self-modified version of the script in which I changed it so that clicking an image only selects it, instead of selecting and submitting.

All HIT's Approved Overnight

Wow....I'm very impressed, every single one of my HIT's from yesterday was either approved or rejected overnight.....Though I am left with a pending of -2.....Maybe I was suppose to do 2 more HIT's yesterday?

But this is great compared to having to wait a week or so before getting approved.

Some Enlightenment from Jeff Bar

Here's a few tidbits from posts by Jeff Bar (on Turker Nation), a member of the Amazon Web Services Developer Relations Team.

"In the future there could very well be special types of HITs that would be easier to do through some type of client than through a web interface. Things like listening to some music clips, manipulating 3D objects, watching movies come to mind. The MT concept and the HIT model are both pretty general, and all of these are within the realm of what can be done."

Jeff really shows us how wide of a variety of HIT's could show up, heck you could even be drawing cartoon characters...which would rock. MTurk could become this massive sort of place with HIT's for people with all types of skills (unlike the current Image Adjustments which are basically mindless work). And some of that sounds pretty fun to do, heck I watch movies and listen to sound clips for entertainment....heh.

Then we have

"The response to the Turk concept has been gratifying. While I can't go into any specifics, we are talking to a number of organizations about getting HITs of all types into the Turk over the course of the coming weeks, months, and years."

Personally I've been very curious as to whether we are going to be seeing any HIT's from someone other the Amazon, because the system is really built to work with mostly 3rd party HIT's rather then just one's from Amazon. From this I get the feeling some new HIT's will be appearing soon from 3rd parties and they might rock.

Now to be fair, let's remember this is someone who works in Amazon's Web Services, employees tend to get overexcited about the possibilities of a new product, and often times many things in the vision never happen. But even if MTurk dies, I can see things like it being created by many other companies. Like Google, Yahoo, and small startups. Really the MTurk setup is a much better way to use extra time then mindlessly wandering the web. And perhaps if it stabilizes, could be the equivalent of a full time job.

Confirm Artist Name HIT Type

Ok granted this HIT type sucks. It pays only 2 cents and takes a little longer then Image Adjustments, usually....and requires research far too often...and it has been around for a while but has been rather elusive, so here's a screenshot

No HIT's ToDo

Hmm first time in a while that there's been no HIT's......hopefully some will appear before too long. It's been about an hour so far with no HIT's......I am addicted.....and now I'm dying with no HIT's!!! Heh

Back Up

MTurk is back up after a 24 hour downtime or so.

In this time there were tons of approvals....though I'm not sure of the accuracy of these newly approved HIT's......I have negative 1 pending on one day...is that possible? heh

Long Downtime

MTurk is having really long downtime today.....I assume something went wrong....Perhaps some major vulnerability that they have to fix or perhaps some server issues.......But ya 12 hour+ downtime...

Getting your Money if your Not in the US

Ok....Now Amazon has been rather inconsiderate, and done what us Americans tend to do, totally ignore the rest of the world. Obviously there are English speakers all over, that want to make some spare cash in their free time. But currently you can only transfer money to a US bank account, and when buying off amazon.com, shipping can kill you.

So the solution (brainstormed up on Turk Nation, thanks sandoz)

Simply find an item on eBay that is going for about the same as it costs from amazon.com

Let's say a recent video game, Mario Kart DS

Here's a listing on ebay and here's one on Amazon. So once you have enough money including shipping, transfer it over to Amazon. Then make a listing on ebay for whatever item you decide would work best. Tell bidders that it will be shipped new directly from Amazon perhaps even explain why (make sure to target US bidders) Then once someone bids and the auction ends, purchase it on Amazon. Put their address in the shipping and bam! You just got your money via ebay which has a lot more payment options.

Problems with this is that your item could sell for less then it costs you, losing you MTurk money. As well this is kinda a pain comparable to just directly transferring it to your bank account. But Hey! It works. Until Amazon supports other payment options....which could be a while.

Link Attack

Here's all the sites that I've found that are dedicated to MTurk, if you know of anymore please post them in the comments.

Turking.com - Nice blog/forums, first site that I've seen that someone actually put a little money and some work into the building of.....Though it feels a little bloated

MTruk - Humorous parody site

Turk Lurker - Nice blog on Mechanical Turk that puts thing into perspective from someone out of his youth

Turk Toiling - Blog with typical updates and some nice insight

Turker Nation - The goto place for forum discussion on MTurk

Another MTurk Blog - So-so Blog on MTurk

The Knight's Tour - Very nice, sleek blog on MTurk

Also there's Mechnical Turk Photoblog which is great....But seems to be dead this morning...Perhaps the address changed

HIT's Auto-Accepted After 30 days

Finally we can be at ease that we won't have HIT's pending for the rest of our lives. According to an email from Amazon, HIT's still pending after 30 days are automatically accepted (from AAI posted on Turk Nation)

"Generally there is 30-day time period in which if a requester has
not approved a result it will be automatically approved."

Not many other updates, things have been pretty quiet, approvals not going as fast as everyone would hope, the servers seem to be becoming more stable, there's no clear sign of any bots, and the newness of the Image Adjustments is wearing off.....People are getting less and less HIT's done, as this becomes a "normal" everyday part of Turkers lives. Personally I've been less strict with myself and dropped down to 900 HIT's today....Pathetic...But I was pushing myself too hard...Money isn't everything but it sure is nice to have..heh.

As well Chris Thomas's "Quicker Image Adjustment" Greasemonkey script has been updated. Personally I've gone back to just the auto-accept because of the memory leak issue (Chris's script crashes most people's browsers after 30 minutes.) The new version let's you choose which style of scrolling and has auto-accept built in.

Someone's Listening

Hahaha....I believe it was Sandoz who said instead of a black box for image adjustmetns they should use a watermarked image....well he got his wish.
It's definatly going to be a lot harder for bots to ID this then that black box.

By the way, for some reason the black box is still on many of the Image Adjustment HIT's but I assume it's going to be replaced by this city image with text on it eventually.

Rejection Rates Down

Well every since the new "black box" appeared rejection rates have dropped at least several percent for most people. As well, today recently completed HIT's were being approved faster then in previous days. (I did about the same today as yesterday but already have twice as many approved). Stability was so-so today. But pretty much a good day....I lost my taste for Image Adjustments towards the end but I still reached my goal...Hopefully my motive will come back tomorrow.

Message Board + New Script

Hatcher has now setup a proboards board to talk about MTurk on, be sure to check it out.....

As well there's a new script by Chris Thomas that should speed up Image adjustments, here. Personally I think he has some really nice stuff in this script and some unnecessary junk....like for instance I am on a low resolution moniter because I'm too cheap to go buy a new one (this one is from 1998) so I can't see all of the images with his script. As well it just adds some junk that I don't like...and doesn't have auto-accept built in...but on the good side it adds keyboard shortcuts like going forward though pictures and back, and accepting via enter which is awesome. So I think I may modify this script to meet my owns need better and I might post my modified version.

Update: Ok I just modified it a little to work for me (not a good modification so I'm not going to post it) But I realized I was rather harsh....This is a wonderful script that will save us all time and keep us from wearing out our scroll wheels....I applaud Chris Thomas...but I hope this is an early version and that future ones will have less muck....But seriously this is a great script

Bots and Zombies Vs. The Black Box

Well everyone who has MTurked recently has noticed the new black box. And it seems to annoy the heck out of most of you.

First off let me say this the black box is good for us...It's good for everyone who isn't running a bot and who isn't a zombie. How so? Obviously by killing current bots (and trust me there are a number running) and it reduces the rate that HIT's disappear...Although HIT's haven't appeared to be in short supply. Secondly for those who aren't zombies when you do ID the business properly, you have a much better chance of it getting approved and not rejected because of "none of the above" votes from bots. And thirdly the system has stopped lagging so bad since the black box appeared. Fourthly it helps us all sorta slow down and make sure we complete the HIT correctly.

Of course this modification isn't hard for a good bot coder to get around. Simply scan the images for a large amount of black....Or randomly pick a radio button and hope for the best. But the scanning the images uses more computing power, time, and bandwidth. And randomly picking a radio button would drop profits for them considerably.

So Yes the black box was a good decision for Amazon, for right now, and perhaps for us in the long run. But I hope this is a temporary thing, there are many other ways that Amazon could stops bots....For instance simply suspending someone who chooses "none" 200 times in a row. Or just noticing the timing and how rhythmic it is. Two things I assumed they already had in place though I haven't heard anything on. But nevertheless the black box is a minor annoyance and if MTurk had it when it came out I wouldn't have minded....So let's try to be open minded and maybe pass a few better anti-bot measures along to Amazon via feedback

Let's remember too....The main reason nothing quite like this has been successful before....Is that bots and zombies have killed them....Let's bear with MTurk as it tries to rise above that. Oh ya and don't quit you day job....Not yet, at least.

Some detailed Speculation on Approvals

I just wanted to draw attention to drekmonger's comment on how the HIT's are approved or rejected which has some very nice speculation and a few bits on the API

"drekmonger said...

I've already scanned through the API. The mechanism for approval is entirely up to the requester...

There's a call to retrieve a completed HIT, reject a completed HIT, and approve a completed HIT. Whether or not to approve a HIT is logic that is coded by the requester.

From my observations, the critia for approval and disapproval of the A9 HITs is evolving. As of late Saturday, early Sunday, the HITs seem to approve if X (possible 3 or 4) people click the same answer to a single HIT. Otherwise, the HIT is reguritated for a tie-breaker.

The system doesn't seem to keep track of who's done what HIT--at least it doesn't seem as such based on my quick read of the API. And HITs can be set to done multiple times.

So, it's possible that a single person (or bot) to receive the same HIT 3 or 4 times. If you answer it the same way each time, then you'll get that many HITs accepted.

I've noticed HITs I've completed end up in what I believe are the "tie-breaker" requests. These are the HIT groups for a city that contain just a few dozen HITs. When I complete such a HIT idetically to a prior HIT, I've noticed the number of HITs accepted in my dashboard increment by 2.

The trick would then be to guess how other people would answer the HIT, rather than try to answer the HIT itself. Unfortunately, I think there's a lot of people zipping through the HITs and not paying close attention--so even if a single person correctly identifies a business, he could be outvoted by 3 zombies clicking "none of the above"

Change to Image Adjustments

Click on the screenshot and if you can't figure it out the change...stop turking
Update: Btw This appears to be an anti-bot change (trying to keep bots frombeing able to trick mturk)
Update 2: Wow this is annoying....just a tip always click the black box on your way down it's easy to later on down click another one then have to go back up and find the black box. Secondly what is sucking is I keep getting ones with no black box only options.....so the actually place isn't in any of the photos so I have to give them one that is wrong (I usually just choose the best looking photo) Though the server just went down again...so maybe they changed their minds...lol

I'm Officially Addicted

This morning I woke up out of a dream of me MTurking. No joke I was MTurking in my sleep, too bad I wasn't sleepwalking...Then I could make money in my sleep...Which would own. Anyways some updates.

• MTurk has been slower today then usual (Sunday), and while I've been on has gone down three times....I expect this is a sorta second wave of high traffic. Usually new sites will have the first big rush and then a second one that will also take it done but it won't be hit as hard....Or maybe they are just updating the servers.
  
• I've personally managed to get above 8000 HIT's now....Which feels awesome. As well I have accumlated more then $100 in earnings (tons more pending too), though the 2 dollar bank transfer I tried has yet to go through. But my bank still hasn't been open since I tried so hopefully that's the problem.
  
• Finally I have had one day about a week ago approve or reject all of the HIT's that I did that day. So they do eventually all get off pending....hopefully amazon will fix it so they get approved or rejected faster.
  
• A few interesting trends, that I've been noticing. I keep doing the same Image Adjustments over and over....I swear I've ID'ed a few businesses like 5 times. I work mostly in the same cities because I become familiar with them and I can get more HIT's done once I am familiar....But still are they just trying to keep us busy or why the repeats? I can't be confirming my own can I? Especially not over and over.
  
• As well I've noticed whenever the HIT's get low (only around 5k or so left), many new groups show back up but never new towns just the same old ones. Are we in some pointless cycle? (well pointless for Amazon not for us as long as we are making money) maybe a glitch of sorts? Or maybe they want an active force to be ready for some future projects....Or perhaps this is a demonstration to other companies that there system is capable of doing work for them....I really need to download the API and see if it can give me insight on how the approval process and other such works....

Anyways Good Luck out there MTurking.

More Down Time

Hmm MTurk was seeming pretty stable....But now I am getting this error "Http/1.1 Service Unavailable" even on the homepage. Anyways I have some more photos to share with you from my MTurk adventures

First off Watching Big Brother, the two men here are setting up a security camera.
Then we have The Reason the Press is Full of It, they are all drunk. Seriously this store is called "Press Liquor" and is located in the National Press Center building in DC...
Then we have Man from Matrix, I have to admit he does look cool.
Then we have De Place, I'm not sure what it's "De Place" for....Maybe drugs? That explains all the colors.

Anyways....I gotta go finish up MTurking my way to victory.

A Little More on How they take the BlockView Pictures

Well I finally found what I've been looking for on BlockView thanks to a comment by Fan Wang on Turk Lurker. This page has a nice video on how they take the pictures (which is a pretty basic system, gps+cameras) I am very surpised that they have the cameras on top of the vechicles, I had assumed that they were inside so as to not have the creepy affect (big brother is watching)....as well in reflections of the vechicle I've never seen the cameras that are obviously there in these pictures. Perhaps they even changed the setup, determining that the creepyness factor was worth the beter pictures. Or maybe I'm just blind. Lastly it looks like BlockView is an active project, in fact you can even apply. That'd be a pretty cool job actually, just drive around and see a city, sure you have to setup the system but then it's just smooth driving.

Musings

Well yesterday I got to exactly 3,000 HIT's in one day....Which was ownage. Of those 1,014 have been approved, I really don't understand why some are staying on pending forever while others approve in 10 minutes or so...And it's annoying.... Anyways another nice MTurk blog to check out is Turk Lurker. And after you check that out be sure to see this picture(from BlockView)...That's not photoshopped....I except it's the result of bored people with a crane but I'm not sure. And while I was MTurking my way to victory, I found a forgotten friend, the Segway...A few years ago these things were all over TV.....Then when they finally came out the price tag killed them...heh. This may even be a demonstration of the Segway since there are two, but I'm not sure.
And lastly I'm just going to leave you with this picture of our nation's history....

What takes the BlockView Photos?

One of the most intriguing things about BlockView is that the people have no idea they are being photographed. Which makes one wonder what the vehicle taking the photos looks like? In reflections it's been shown to be an SUV, here's a good picture of it
I've also seen what appears to be a grey SUV, and a black one but they may have simply been white ones in the shadows. From the pictures it appears that most were taken about a year ago. I looked up A9 BlockView and it launched in the beginning of 2005 (see it here). And the BlockView adjustments we are doing appear as though they are for Amazon's Yellow Pages. Personally I'd like to know if these SUV's have been driving around all year and have tons more cities that haven't been posted on BlockView yet, but I can't find any actually information on this. The way things look right now, by the end of November we will have eaten through all of the BlockView Image Adjustments, hopefully some other companies will catch on and start using the MTurk API by then and work will still be plentiful. We'll see.

Auto-Accepting HIT's with Greasemonkey

For all those who haven't heard of Greasemonkey it's an extension for Firefox that basically allows you to totally customize pages (download it here) I'm not much of a coder so I went over to the greasemonkey mailing list and asked for them to whip me up an Auto-Accept Script, Jeremy Dunck was kind enough to do so. What the script does is searches for a link with "accept" in it. Anotherwards when your doing A9 Blockview adjustments you don't have to keep clicking accept every freakin time (which is so annoying). Which for me has increased how many HIT's I can get done by a lot (as I mentioned earlier)

Anyways to get the Auto-Accept Script right click here and click "Install User Script" if you have greasemonkey. Btw when you want to stop doing HIT's just click on "HITs Assigned To You" and do the last one that it auto accepted and then your out.

Good Luck.

Actually getting your Money!

Ok so today I got up to $39 bucks, so I decided that I better make sure I can actually access this money. First I tried transferring $2 last night to my bank account, on the Amazon side everything said it went fine....But my bank account has yet to show the deposit (it's been 24 hours)...I assume that my bank has to manually approve it, and it hasn't been open since I deposited it (today's a holiday)

Then today I tried transferring some to an Amazon gift certificate balance. It was there instantly and I used it to buy Mario Kart DS, which I paid nothing for out of my pocket, everything from my Mturk money, which feels great. Basically it just cost me time that I would've spent pointlessly talking and surfing the web. And hopefully there's more to come.

Good Luck Mturking your way to victory.

Down for System Maintance

Hmm I was mturking my way to victory when the pages started not loading...then only halfway loading and looking different like they updated.....or something

Then I get this message

"Amazon Mechanical Turk will be unavailable for the next hour for system maintenance. To read about Amazon Mechanical Turk and its web services APIs, go to www.amazon.com/webservices -Amazon Mechanical Turk Team "

So there goes my night.....but anyway mturk has been a bit buggy still even with a lot less traffic so hopefully this will fix it all up....now I gotta figure out what to do for the next hour (also hopefully this will fix the problem with some HIT's taking forever to be approved.)

btw it's 11pm Pacific Time that they went down for maintance....wish it was later I was in a groove....

New Type

A new type of HIT showed up today in which you look at the album art of a cd and write in the name of the artist....only 3 cents though...so not really better then the A9 Blockview adjustments.

As well a friend of mine wrote a greasemonkey script to auto-accept A9 Blockview HIT's for me...so that sped up the process a bit...I got in 831 HIT's today so far (gonna do a few more tonight)

Bringing me to a Total of 1885...which is awesome

Starting to think that this wouldn't be bad to do regulerly forever, If I roughly make 10 a day that's 300 a month which would be a nice chunk of extra money to have lying around.

Anyways I gotta go mturk my way to 2k.

Funny Pics

As I go through Adblock image adjustments I can't help but notice how funny some of them are...for instance....

What is this guy's Problem? Looks like he's about to kill her or something.
Here with have some proof of the Sox and Cubs rivalry

Then we have china town...which looks awesome

Mechanical Turk's Launch

Wednesday November 2nd as a great day, Amazon's Mechanical Turk launched, essentially artificial artificial intelligence is finally being pushed by a company with some money to make it work. So I started up this blog, to keep track of activities on the Mechanical Turk.

After Thursday the Turk went into an unstableness, from too many people checking it out. Today it has mostly recovered. And there are literally 10's of thousands of HIT's for us little humans to complete. The only ones posted so far are automative description writing and feature points. And the wonderful Blockview adjustments, which are just plain fun to do, despite being only 3 cents per an adjustment.

HITS: 515