An Agent Capable of Learning to Create and Maintain Websites Anthony Tomasic, Ravi Mosur Alex Rudnicky, Raj Reddy, John Zimmerman Carnegie Mellon University 18 April 2003

Outline Project vision Problem –Assumptions –Inputs –Outputs Missing capabilities Our approach Impact Evaluation Conclusion

Project Vision – Honeydew Honey – An agent –Learns by observation –Obtains advice and consent –Creates and maintains public project websites

Project Vision – Honeydew Honey Can Perform Tasks –Organize, manage and update a complex project website –Delegate tasks –Generate periodic briefing folders extracted material and online documents with planning and summarization capabilities –Respond to specific information queries –Extract relevant information WWW & mailing lists –Perform teaching –Communicate with other Honeys and EPCAs

Problem Assumptions Scope of a “project” website is predefined and not learnt: –Publications –Presentations –Milestone status –News updates –Demonstrations –Links to collaborators –Software releases –Documentation –FAQs

Problem Assumptions Things that Honey will not do –System administration Capacity planning –Graphic design Font selection Site design Some layout design possible … –Content creation

Inputs to Honeydew messages with updates to website –Volunteered and solicited information Minutes from project meetings –Tracking project participants and events Queries from external sources –Inferred information needs –Click sequences Publicly visible events, not explicitly provided to webmaster –Conference appearances, news stories, etc.

Inputs to Honeydew Sequences of UI actions performed –Receive request to add paper to WWW site –Extract title, author, abstract, publication forum, funding agency –Think up file name –Copy attachment to conference paper directory –Update WWW page with info and link –Notify user of change –React to advice from user about change

Expected Outputs Project websites (5 subprojects, 1 project) Report generation –Overviews of activity over time (e.g., quarterly reports) Briefing generation –Overviews of current project activities Question-answering agents –Google-like search, summarization in response to specific queries Semi-automatic FAQ generation

Expected Outputs Shared knowledge base of learned tasks Toolkit for rapid construction of new assistants Assistant Monitoring and Interaction GUI Requirements for “Assistant Aware Applications” Stream of papers Stream of masters and Ph.D. students

Missing Capabilities – What Identify significant webmastering events Represent webmaster activities through generalizable descriptions Create consistent and complete task representations Formulate key clarification dialogs Adapt to errors in task execution

Our Approach Ethnographic study of Webmasters WoZ system for domain definition Human webmasters with Honey observing activities Information-sharing among EPCAs

Our Approach In-line data labeling by humans Interactive clarification of human actions Lightly-supervised learning Generalizable and sharable representations of activities Learn by being told Enthnographic study of Honey users

Impact Relieve the user of routine maintenance tasks associated with web pages Illustrate portability by using in other WWW task domains –HCII web page –Pittsburgh Post Gazette web page –Workflow systems

Impact Dramatic reduction of human effort in construction and maintenance of web sites –Improve time productivity by 100 to 500% –Little or no loss in quality of site EPCAs that can learn the skill of cooperatively structuring and managing information Dramatic reduction in the cost of construction of assistants –Reduce size of backroom knowledge engineers Assistants become trainers of new Web masters

Evaluation: mid-term and finals Honey performance to be compared with 5 human subjects –5 other human coaches to be used in providing data and inputs needed for the Honey to learn from Experience –Operational Version 0 in 3 months Evaluation – 6 months –Honey performs 50% of tasks correctly Evaluation – 12 months –Elapsed time from to WWW update improved by 1.3 –Elapsed time to assemble report improved by 1.3 –Honey performs 75% of tasks correctly –Quality of WWW site and reports comparable to human

Infrastructure Input Users Architecture Functional Specification Text –Filtering –Summarization –Extraction Quality Assurance Evaluation Knowledge Representation Dialog

Conclusion Key emphasis –Learning –Coaching –Retargetting Potential huge impact to Webmaster job Large amount of shared infrastructure Many similar problem domains Many, many research problems

Research Issue Architecture Monitor/Do Event Stream Matching Recommend Execution GUI Server File System Editor Command LearningSummarization KB Monitor/Do GUI KB GUI Coaching GUI User GUI