CS115/MAS115: Computing for The Socio-Techno Web Welcome to CS115 also listed as MAS115. Welcome to the Wellesley Human Computer Interaction Lab. In case you’re wondering, I am Prof. Orit Shaer – can call me Orit. I’ll be teaching the lectures on Monday and Thursdays. Lectures will take place in this room image source: http://www.igenii.com/blog/Social%20Media/social-media/ twitter.com http://www.worthofweb.com/blog/case-study-this-revolution-will-be-tweeted/ http://foxwoodonlinemarketing.typepad.com/my-blog/social-media/ CS115/MAS115: Computing for The Socio-Techno Web Multimedia Search and Human Computation

Multimedia Search Multimedia vs Hypermedia Multimedia: Beyond text communication Stored in a variety of formats Audio Today most popular formats: Image Animation Video Today’s most popular formats:

Searching for Audio Tagged audio files Raw audio Tune recognition News organizations (e.g., NPR, BBC, …) iTunes etc services Raw audio Signal processing To Speech recognition processing To Text (inaccurate) Perform text search Tune recognition Pattern matching E.g., midomi.com In npr.org: “palin attacking pakistan” In midomi.com: Whistle “yesterday” Hum “bridge over the Kwei river” Also, service hsearch.nayio.com seems out of commission

Searching for Video Tagged by humans MIT’s Lecture Browser News organizations YouTube.com TED.com MIT’s Lecture Browser Lectures have additional challenges [ Glass+ Recent Progress in the MIT Spoken Lecture Processing] Project http://web.sls.csail.mit.edu/lectures/ Extract audio track Convert into text Analyze text to reveal structure Segment based on structure

Searching for Images Use surrounding text Tagged by humans Filenames, html, nearby text Tagged by humans Expensive! Slow! Computer Vision techniques Still a tough problem Tagged through games! The ESP game (now part of gwap.com) The Peekaboom game (newer version called squigl) Interesting to try to find it as the ESP game in the URL – lots of spammers appear (e.g., theespgame.com, espgame.com,…) Now it is called gwap.com – Have account with password the usual wellesley one Peekaboom is now called Squigl and the rules are a bit different.

January 28, 2009

Inspiration: Harness “human cycles!” Captcha: program that can tell whether its user is a human or a computer Can generate and grade test that Most humans can pass Current computer programs will fail Re-Capcha Newer work on ReCAPTCHA: http://www.youtube.com/watch_popup?v=Aszl5avDtek&vq=medium#t=30