SAB ReviewFebruary 2004Pervasive 2004April 2004 Using an Extended Episodic Memory Within a Mobile Companion Alexander Kröner, Stephan Baldes, Anthony Jameson, and Mathias Bauer German Research Center for Artificial Intelligence (DFKI) GmbH Stuhlsatzenhausweg 3, Saarbrücken, Germany BMB+F IW C03 (SPECTER)

SAB ReviewFebruary 2004Pervasive 2004April 2004 About this Talk About Specter –Goals, example, challenges About the personal journal –User interface Adding value to the journal –Binding services to journal entries

SAB ReviewFebruary 2004Pervasive 2004April 2004 What is Specter? Specter is about… –… context- and affect-aware personal assistance –… in instrumented environments –… using a long-term memory Major issues –Extension of perception –Learning about behavior and affect –Augmentation of decision making and effecting –Reflection and introspection

SAB ReviewFebruary 2004Pervasive 2004April 2004 Example Scenario: Shopping Assistant Technology –RFID enabled shelf and basket –RFID-labeled Products –Integrates software of projects Specter and REAL (Saarland University) Logging low-level actions –Product in shelf, removed from shelf, etc. Applying logged information –Proposing alternative products

SAB ReviewFebruary 2004Pervasive 2004April 2004 Obtaining Data from the Environment Stream of raw, low-level sensor data UM Personal Journal decision support episode retrieval situation classification motion data classification phys. data classification plan recognition object classification location classification action recognition affect recognition motion profiling abstraction of data... reflection/introspection

SAB ReviewFebruary 2004Pervasive 2004April 2004 Obstacles in Obtaining Data Automatically retrieved input –Sensor input may be incorrect –Sensor input may be completely missing –Inference mechanisms are not perfect user input required Manual (user) input –Lack of motivation manual input has to be minimized input value has to be maximized –Recorded information may be sensible user's trust has to be strengthened

SAB ReviewFebruary 2004Pervasive 2004April 2004 Design Principles Input has to be acquired collaboratively [addresses: manual input, lack of motivation, trust] –filling the journal automatically –asking for additional information if required –providing control of information and bound services Provision of diverse, multiple benefits [addresses: maximizing value] –providing the option of binding various services to journal content Support for diverse forms of collaboration [addresses: lack of motivation] –providing various ways of entering data –enabling input/feedback in varying scenarios

SAB ReviewFebruary 2004Pervasive 2004April 2004 Accessing the Personal Journal The personal journal user interface

SAB ReviewFebruary 2004Pervasive 2004April 2004 Browser/Viewer Approach Browser Goal: browsing the journal in a similar fashion like the Web Adopted concept of Web browser Browsed documents are generated on-the- fly by data viewers Browser and/or viewers can be exchanged Viewers

SAB ReviewFebruary 2004Pervasive 2004April 2004 Navigating the Personal Journal Content requests –History (forward, backward,…) –Bookmarks –Hyperlinks Reminder Points –Created by the user during interaction with the instrumented environment –Indicate the need to inspect points within the timeline during introspection

SAB ReviewFebruary 2004Pervasive 2004April 2004 Adding Content Annotating entries –Free text comment, references, categories, ratings Grouping entries –A group indicates that some entries are related together –For manual organization, and input for Specter

SAB ReviewFebruary 2004Pervasive 2004April 2004 Collaborative Learning of the User Model Binding services to the personal journal

SAB ReviewFebruary 2004Pervasive 2004April 2004 Example: EC Card Purchase

SAB ReviewFebruary 2004Pervasive 2004April 2004 Example: Assisting an EC Card Purchase Goal: checking bank account automatically before payment is required Binding a service –Identifying situation where check is required Identifying relevant conditions, e.g., shop size, total prize of wares contained in basket –Binding a service “account check” to that situation To be stored as trigger in the user model Evoking a service –Comparing given situations with triggers from the user model

SAB ReviewFebruary 2004Pervasive 2004April 2004 Binding Services: Workflow User or Specter chooses a service that should be triggered automatically Specter collects attributes from (potentially) relevant journal entries User discards irrelevant attributes and/or adds relevant ones Specter computes a rule and shows it to the user User accepts or changes the rule Specter adds the new rule as service trigger to the user model

SAB ReviewFebruary 2004Pervasive 2004April 2004 Defining Decision Trees EC Card Purchase Store Price EdekaHela  117,5€ > 117,5€ Price –+ DayOfWeek –+ TimeOfDay –+ –+ SoSa...  112€ > 112€  5:25pm > 5:25pm  1:07pm > 1:07pm

SAB ReviewFebruary 2004Pervasive 2004April 2004 Customizing Decision Trees EC Card Purchase Store PriceDayOfWeek EdekaHela  117,5€ > 117,5€SoSa... userdef #1 TimeOfDay otherwise [12am,2pm] +– TimeOfDay otherwise [12am,2pm] +– TimeOfDay otherwise [12am,2pm] +– TimeOfDay otherwise [12am,2pm] +–

SAB ReviewFebruary 2004Pervasive 2004April 2004 Creating a Service Trigger

SAB ReviewFebruary 2004Pervasive 2004April 2004 Conclusion and Outlook Specter –Overview –Personal journal user interface –Binding services to journal content Next steps –Journal user interface Queries Setting up plans Additional viewers –Identifying additional services