Henry Lieberman MIT Media Lab AI in End-User Software Engineering Henry Lieberman, MIT (presenting) Raphael Hoffmann, Michael Toomim, U. Washington, Seattle

Henry Lieberman MIT Media Lab Roles for AI in End-User SoftEng Supply knowledge end-user doesn't have Recognize user intent Plan generation and plan recognition Automate common patterns of use Generalize & Specialize Manage multiple representations Error checking and sanity checking

Henry Lieberman MIT Media Lab U. Washington SoftBots & more Planning-based software agents Version-space Programming by Demonstration Sequence prediction via Logic & Probability Recommendation from software libraries

Henry Lieberman MIT Media Lab SmartEdit - Lau

Henry Lieberman MIT Media Lab Version Spaces

Henry Lieberman MIT Media Lab Linked Editing Alternative to content abstraction facilities Powerpoint Master Slides Style Sheets "Symbols" vs. instances in Flash Web templates, components User stays within WYSIWYG paradigm

Henry Lieberman MIT Media Lab Linked Editing End-users "underuse" abstractions Linked Editing User edits concrete example Generalizing over content rather than actions Finds other instances of description Performs "analogous" edits Program code, Web pages, Vector graphics, Sheets

Henry Lieberman MIT Media Lab Linked Editing

Henry Lieberman MIT Media Lab Linked Editing

Henry Lieberman MIT Media Lab Assieme - Customization Recommendation Goal: Recommending scripts for application extensions: Emacs, Firefox Collect keystrokes, file names, commands Provide Web service for recommendation Collab filtering, learning classifier

Henry Lieberman MIT Media Lab Assieme - Customization Recommendation

Henry Lieberman MIT Media Lab The crazy dream: Programming in English Programming languages are too difficult for human beings People can express procedures in English (or other natural language) Why not program in natural language? That was the original idea behind Cobol (whereas Fortran tried to make programming like math)

Henry Lieberman MIT Media Lab Childrens description of Pac- Man

Henry Lieberman MIT Media Lab But what can you do if you can only partially understand English? Need to make the interaction fail-soft Rely on dialogue to resolve ambiguities and errors Maybe dont eliminate code entirely Emphasize correspondence between language and code An "outliner" for code Application: Programming MOOs

Henry Lieberman MIT Media Lab New resources for Common Sense Reasoning Open Mind Common Sense knowledge base ConceptNet Semantic Net MontyLingua/LangUtils

Henry Lieberman MIT Media Lab Metafor Interface

Henry Lieberman MIT Media Lab Programmatic Semantics English to scaffolding code Roughly, Turns nouns into classes and/or instances; Verbs into functions Adjectives into instance variables Resolves anaphora Untangles conditional and loops Propagates context

Henry Lieberman MIT Media Lab Narrative stances I want to make a bar with a customer. (1 st person programmer) There is a customer in the bar. (3 rd person objective) I am a customer sitting on a stool. (1 st person program) The bartender said, Here is a customer (3rd person subjective)

Henry Lieberman MIT Media Lab Refactoring There is a bar. (atom) The bar contains two customers. (unimorphic list) It also contains a waiter. (unimorphic wrt. persons) It also contains some stools. (polymorphic list) The bar opens and closes. (class / agent) The bar is a kind of store. (inheritance class) Some bars close at 6pm. (subclass or instantiatable)

Henry Lieberman MIT Media Lab Shameless Plug