Applied COIN Lab Anatole Gershman, Alan Black, Louis Van Ahn TA: Mehrbod Sharifi August 26, 2008

Who are we? Alan W Black Luis von Ahn Anatole Gershman Mehrbod Sharifi

Why are we teaching this class? This class is about inventing Applications People don’t use computers – they use applications In your future jobs you’ll be building applications

What kinds of applications will we see in the future? Future applications will: Sensethe world through networks of sensors; Thinkby making sense out of massive amounts of information; and help us Actin the real world Example: Your smart phone senses that you are at the airport; it checks your schedule and concludes that you are there to take flight 123 to Chicago; it acts by displaying the departure gate

Is a smart phone a product or a service? The distinction between physical products and services will largely disappear: most services will be provided through physical products most physical products will become conduits for services Applications will combine physical products and services They have to be designed together

What will you be doing in this class? You will be inventing and building applications that combine physical products and services and can sense think and act You will work in small groups or individually We will provide project ideas, tools and advice We expect you to submit: a project proposal a project design and mock-up a working demo a project report We expect to hear from you on a weekly basis

Class Schedule 8/28/08 First Class 9/4/08 Proposal + Groups (1-3) 9/11/08 Mockup (Specs and PPT) 9/18/08 9/25/08 Final Plan and starting work 10/2/08 10/9/08 10/16/08 10/23/08 10/30/08 Mid Point Milestone 11/6/08 11/13/08 11/20/08 11/27/08 No class - Thanksgiving 12/4/08 Project presentations 12/11/08 Project presentation + final report

Deliverables 1. Project proposal (1-2 pages) due next Thursday What will the prototype do? Why is this interesting and important? What kinds of tools and equipment will be necessary? Who will be the team members? 2. Mockup and preliminary specs of the system due 9/11 A sequence of Power Point slides depicting a demo scenario Preliminary architecture and specs Preliminary selection of tools and equipment Preliminary assignment of roles and responsibilities 3. System specs and work plan due 9/18 Architecture and design decision justifications Tools and equipment Roles, responsibilities, schedule

Deliverables (cont.) 4. Weekly progress reports One page summary of the last week accomplishments and goals for the next week 5. Final working prototype A 10min demonstration, explaining what it does, why it is important and how it does it 6. Final report due 12/11 A 5 page paper describing: Project goals System design Use scenario Potential applications Lessons learned

Project ideas: Smart Online Cabinet How would the cabinet know what’s inside? RFID tags and reader How would it recognize the user? camera, voice? What other sensors should it have? What services should it provide? How do we communicate with it? When and what should it say and when should it remain silent?

Project ideas: Smart Mobile Assistant Your smart phone or PDA knows where you are and can record and play back snippets of information in the right context For example, while meeting someone new, you might record a snippet of the introduction when you say: “Hi John, nice to meet you” Later, you would be able to ask: “Wednesday, noon, cafeteria” Your assistant will play the snippets recorded at that time in that location

Project ideas: Smart Objects If this plant could talk, what should it say? What services might it provide? What about the door to your house or your office?

Project ideas: Interactive Picture Remember pictures on the walls of Hogwarts School of Magic? We already have the software and the hardware to track your position in front of the picture What applications would be interesting and useful?

Project ideas: Tricorder Or the remote control to the world Scan the environment to see what’s controllable Provide standard control interfaces Example: Point the tricorder at an interesting car in the street, click the “copy” button then find the nearest big screen, get a search engine and “paste” the car into it.

Project ideas: Speech Eliza Joseph Weizenbaum, MIT Q A.L.I.C.E.A.L.I.C.E. and AIMLAIML Chatterbot Dr. Sbaitso ELIZA effect Jabberwacky List of Chatterbots Loebner prize PARRY Racter Simulated consciousness Turing test Virtual Woman

Project ideas: Smarter Virtual Worlds A school yard in Second Life Robotic kids have internal states and behaviors governed by rules They can sense each other location and behavior and engage in conversations Verbal combat between characters

Tools and equipment RFID kits PDAs Smart Phones Bluetooth beacons Laptops Webcams Headsets and speech kits Sensor kits

Questions? Coin.scribblewiki.com All course information will be on Scribble Wiki 1. Project proposal (1-2 pages) due next Thursday What will the prototype do? Why is this interesting and important? What kinds of tools and equipment will be necessary? Who will be the team members? 2. Mockup and preliminary specs of the system due 9/11 A sequence of Power Point slides depicting a demo scenario Preliminary architecture and specs Preliminary selection of tools and equipment Preliminary assignment of roles and responsibilities