Lecture 26: Vision for the Internet CS6670: Computer Vision Noah Snavely

Announcements Wednesday’s class cancelled Final project status reports due Wednesday Office hours this week: Tuesday 9:30 – 11am

Vision and the Internet

How is the Internet useful for Computer Vision? 1.As a source of data 2.As a source of people with too much free time

How is the Internet useful for Computer Vision? 3.As a source of new applications

The Internet as source of labor Next slides courtesy Luis von Ahn

Mechanical Turk

Mechanical Turk – Demographics United States76.25% India8.03% United Kingdom3.34% Canada 2.34% Age distribution Motivation

LABELING IMAGES WITH WORDS MARTHA STEWART FLOWERS SUPER EVIL STILL AN OPEN PROBLEM Slides courtesy Luis von Ahn

IMAGE SEARCH ON THE WEB USES FILENAMES AND HTML TEXT

TWO-PLAYER ONLINE GAME PARTNERS DON’T KNOW EACH OTHER AND CAN’T COMMUNICATE OBJECT OF THE GAME: TYPE THE SAME WORD THE ONLY THING IN COMMON IS AN IMAGE THE ESP GAME

PLAYER 1PLAYER 2 GUESSING: CARGUESSING: BOY GUESSING: CAR SUCCESS! YOU AGREE ON CAR SUCCESS! YOU AGREE ON CAR GUESSING: KID GUESSING: HAT THE ESP GAME

© 2004 Carnegie Mellon University, all rights reserved. Patent Pending.

4.1 MILLION LABELS WITH 23,000 PLAYERS THE ESP GAME IS FUN THERE ARE MANY PEOPLE THAT PLAY OVER 20 HOURS A WEEK

SAMPLE LABELS BEACH CHAIRS SEA PEOPLE MAN WOMAN PLANT OCEAN TALKING WATER PORCH

REVEALING IMAGES REVEALERGUESSER CAR PARTNER’S GUESS BRUSH BRUSH CAR CAR

How can we choose a set of representative images? Statistics over large numbers of people Flickr results: “Pantheon”

How can we choose representative images?

Scene summarization [Simon, Snavely, Seitz, ICCV 2007]

Scene summarization – tags

Typical Flickr tags

Scene segmentation [Simon and Seitz, ECCV 2008]

Scene exploration

Mapping the World’s Photos [Crandall, Backstrom, Huttenlocher, Kleinberg, WWW ‘09]

Mapping the World’s Photos [Crandall, Backstrom, Huttenlocher, Kleinberg, WWW ‘09]

The Internet as a source of applications

Using social networks for recognition [Stone, Zickler, and Darrell, Workshop on Internet Vision 2008]

Using social networks for recognition Useful information: Which pairs of people are friends? Who are the photographer’s friends? Which people have appeared together in the past? Who appears most often in the photographer’s photos? [Stone, Zickler, and Darrell, Workshop on Internet Vision 2008]

Using social networks for recognition [Stone, Zickler, and Darrell, Workshop on Internet Vision 2008]

Camera calibration "Priors for Large Photo Collections and What They Reveal about Cameras," S. Kuthirummal, A. Agarwala, D. B Goldman, and S. K. Nayar, European Conference on Computer Vision, 2010

Questions? See you on Tuesday, May. 17, 2 – 4:30! Look for regarding signup times