Google Image Labeler

In the latest example of human powered solutions, Google recently launched Google Image Labeler. This was licensed from The ESP Game by Professor Luis von Ahn of CMU. Two people are matched and then are each shown the same sequence of images. Each player types a word that describes the image, hoping to match the other person. The players continue to type words until there is a match. The players can pass if they feel stuck. Occasionally words are “taboo” - most likely because they have already matched frequently. The players objective is to achieve as many matches as possible in the alotted time. The objective of the overall project is to label images using people.

There are many applications where this kind of human activity can do a better job than, or enhance, computer algorithms. Specifically, the idea of having people identify information - images, Web sites, video, music - is enticing. I’m curious to see how well this particular method works. Having played a bit I observed a few things:

When you’re on a timer, you don’t enter thoughtful descriptions. My Flickr tags are far more thoughtful than what I entered into Google Image Labeler. With Flickr, I copy and paste, or look up spelling, including names, etc.

The game starts to bias toward a set of “unwritten rules” very quickly. This is a “double blind” test, right? No information, other than the guesses, is transmitted between players, right? What if information can be transmitted within the guesses? Since people match off against different people with the same objective they learn from one another. You soon learn what will make a match. For example, here are a few simple rules:

• if there’s a person, simply type “man” or “woman” to start - doesn’t matter if it’s Bono or Larry King or Howard Stern or Apolo Ohno. Don’t bother. The guy on the other end either doesn’t know these people, or he knows that typing “man” is a sure win.
• Go for the color. Doesn’t matter what the image is of, you have a good shot at a match just by picking the prominent color. Seems to me software could do that easily.
• If it’s words of any kind, just type the first word. If it’s taboo, then type the second word, etc.
• And so on…

Lowest common denominator. Well that’s to be expected. Ever make friends with a Danish traveler who speaks little English or Japanese in a restaurant in Tokyo and try to figure out what fish each sashimi is made of? That’s what this is like. Half the time I don’t know who or what the image is, and the other half of the time I’ve entered the person’s name 3 different ways and I can’t believe the other guy doesn’t recognize Bono. (So we agreed on “man”) Perhaps this filtering is the brilliance of this exercise and professor Luis von Ahn is carefully sifting the good matches from the junk. So even though I and my partner only managed to label a photo of Apolo Anton Ohno rounding a winning lap in a speed skating final at the Olympics as “man”, some out there were able to label it “skater”, “speed”, “Olympics” and even “Apolo” or “Ohno”.

There’s a site Captcha, that lets you search 30,000 images that have been labeled on the ESP Game. There are a few entries for skating (none of them Apolo), but no surprise, there are 9,558 images labeled “man”

I’d like to post some example images and the guesses that went with them, and compare them with Flickr photos and their tags. Perhaps I’ll have time tomorrow.

There are a few questions that come to mind:

1. Which will work better, Google Image Labeler or Flickr Tags?
2. How well does this approach work on other objects such as video and Web Pages
3. How long before Professor von Ahn is on the Google payroll?

Cheers,
Michael

Trackback URL: http://blog.wink.com/2006/google-image-labeler/

Tags: Wink, Google, ESPGame, ImageLabeler

4 September 2006 | observations | Comments