1. Conceptual Level Interaction Johnny’s Big Half-Baked Idea Sept 2001

2. My Approach...

3. Very Hard: Video & Audio “Tom is giving a presentation”

4. My Approach... Not as Hard: “Tom is giving a presentation” Hold Error Messages/Alerts Silence Mobile Phone

5. My usage of “Conceptual”

6. The Concept: “I am giving a presentation.”

7. My usage of “Conceptual” At the moment: –User States Available Busy In Meeting Giving Presentation Etc...

8. My usage of “Conceptual” At the moment: –User States Available Busy In Meeting Giving Presentation Etc... –Data Manipulation/Management Tasks “Check email from Tom” “Find the cheapest hotel in Boston”

9. My usage of “Conceptual” Much harder stuff: –Ideas or Visions –Abstract Thoughts –Other stuff I’m missing

10. Details… “So, where’s your project idea?”

11. Things to try... Two Ideas –User State Interface/Modeling –The Apprentice Underlying Concept: –Work with the data that’s already there –Monitor the Desktop Computer

12. Things to try: User Modeling Applet on the desktop –Interface for user to provide a state “I am in a meeting” Just collects data, doesn’t actually do anything –Monitors user activity Acknowledging Keyboard/mouse activity (not recording) Keep tracking of open/active applications –Tries to guess user state My hypothesis: People will more likely to correct the applet than constantly volunteer the info. Learning algorithm or data collection for later analysis.

13. Things to try: User Modeling User state modeling applet on the desktop –Interview users to gain a list of states, or allow them to create new states. –Probably some literature on this Example:

14. Things to try: Apprentice Collect data on user tasks –Ask user what they want to do upon returning to computer (after 1 min. idle) –Records any following activity and pairs it with user provided command. –Analyze the data offline by hand for patterns or segmentations Example:

15. Things to try... User Modeling –Data collection task –Will enable “appropriate action” –If model fails, still have a “user state” interface. The “Apprentice” –Data collection task –Mine for patterns/chunks –High level goal: to allow users to use a phrase recognition interface to command information on the computer.

16. Conceptual Level Interaction Questions & Comments