Context-Aware Computing John Canny HCC Retreat 7/5/00

Consequences of Ubiquitous Computing and Calm Technology: We want to have constellations of devices working for us, somehow inferring and supporting our activities.

Mining Tacit knowledge from activity: Activity mining involves content and context: the inter-relatedness of people and information objects.

ABC: Activity-Based Computing Activities are clusters with: –Users –Documents –Tools Realizations can be –GUIs –speech, etc. Nearness encodes awareness.

Recent studies (Gruen ‘96) support the value of the activity model, but: –activities are usually not distinguishable instantaneously, they interleave and overlap. –Activities can persist over different time frames and occur in different places. ABC: Inspiration Based on “activity theory” from psychology (Vygotsky, Leont’ev, Engestrom, Kutti). Activities are high-level behaviors directed toward some end, usually with others.

ABC: What is it good for? Streamlining interaction: disambiguation, menu customization. Prefetching files, prestarting devices. Awareness (who else is working on the project now) Attention management - graded awareness of other activities. Pro-active sharing in group work. Infering document utility. Human expertise location. Re-contextualizing documents: authorship, roles, backgrounds, discipline-specific vocabulary...

ABC: A lo-fi prototype (Danyel Fisher)

ABC: Contextual data sources Who: –Direct communication: 1-1 , phone, F2F. What: –Topical discussions, forums, F2F meetings. –Document writing, reading, search, markup. When: –The current time; time windows for activities. Where: –A “place” which has meaning for the users activities.

ABC: Representation User 1 Document5 Device7 Program12 User 2 User 3 Document3 Mail Read Start Markup Write Project Algorithms are SVD and other pattern analysis schemes

Knowledgescapes (Heyning Cheng) Knowledgescapes is a search engine that uses activity logs only - it doesn’t look at document content: Activity data is used in knowledgescapes to infer document relevance/quality. Results: –Performance is fair with text queries –Performance is very good with related document queries In progress: Infer user expertise from document selections.

Knowledgescapes prototype Information Need Query (terms) Search Engine Document1 Document2 Document3 Document4 Rankings from reading time. Treat as probability of interest or E(interest)

Another inspiration: LSA Word 1 Passage 1 Passage 2 Passage 3 Passage 4 Word 2 Word 3 This structure is resolved by SVD into latent semantic categories which better model the document content.

Another inspiration: LSA Word 1 Passage 1 Passage 2 Passage 3 Passage 4 Word 2 Word 3 Decomposition of the linear may from words to passages into two linear maps, with latent concepts in between. Concept 1 Concept 2 Concept 3

Content and context mining Many successful content analysis schemes are based on LSA (Latent Semantic Analysis). Widely used “context” analysis schemes (from social network analysis) use similar algorithms. We are developing a latent-variable method which combines evidence about content and context to model activities. We will use it for: –Document prefetching for unseen documents –Document authority/quality estimates –Naming activities –Building personalized thesauri

Context and content analysis User 1 Document1 User 3 Document3 Read Markup Write Use document access data and LSA and Factor both maps into latent categories User 2 Document5 Term 1 Term 2 Term 3

Sensors for the HCC lab Sensor networks: –Microphone arrays –Camera arrays –Small scanners Output –Wall projectors, PDAs, the UPM Networking using USB and IrDA Sensor proxies for outside access via Jini or HTTP