1. CS115/MAS115: Computing for The Socio-Techno Web
Welcome to CS115 also listed as MAS115.
Welcome to the Wellesley Human Computer Interaction Lab.
In case you’re wondering, I am Prof. Orit Shaer – can call me Orit.
I’ll be teaching the lectures on Monday and Thursdays. Lectures will take place in this room
image source:
twitter.com
CS115/MAS115: Computing for The Socio-Techno Web
Multimedia Search and Human Computation

2. Multimedia Search Multimedia vs Hypermedia
Multimedia: Beyond text communication
Stored in a variety of formats
Audio
Today most popular formats:
Image
Animation
Video
Today’s most popular formats:

3. Searching for Audio Tagged audio files Raw audio Tune recognition
News organizations (e.g., NPR, BBC, …)
iTunes etc services
Raw audio
Signal processing
To Speech recognition processing
To Text (inaccurate)
Perform text search
Tune recognition
Pattern matching
E.g., midomi.com
In npr.org: “palin attacking pakistan”
In midomi.com:
Whistle “yesterday”
Hum “bridge over the Kwei river”
Also, service hsearch.nayio.com seems out of commission

4. Searching for Video Tagged by humans MIT’s Lecture Browser
News organizations
YouTube.com
TED.com
MIT’s Lecture Browser
Lectures have additional challenges [ Glass+ Recent Progress in the MIT Spoken Lecture Processing] Project
Extract audio track
Convert into text
Analyze text to reveal structure
Segment based on structure

5. Searching for Images Use surrounding text Tagged by humans
Filenames, html, nearby text
Tagged by humans
Expensive! Slow!
Computer Vision techniques
Still a tough problem
Tagged through games!
The ESP game (now part of gwap.com)
The Peekaboom game (newer version called squigl)
Interesting to try to find it as the ESP game in the URL – lots of spammers appear (e.g., theespgame.com, espgame.com,…)
Now it is called gwap.com – Have account with password the usual wellesley one
Peekaboom is now called Squigl and the rules are a bit different.

6. January 28, 2009

7. Inspiration: Harness “human cycles!”
Captcha: program that can tell whether its user is a human or a computer
Can generate and grade test that
Most humans can pass
Current computer programs will fail
Re-Capcha
Newer work on ReCAPTCHA: