Mik Kersten Thesis defense December 15, 2006 Focusing Knowledge Work with Task Context

Problems Information overload Many knowledge work tasks cut across system structure Browsing and query tools overload users with irrelevant information Context loss Tools burden users with finding artifacts relevant to the task Context is lost whenever a task switch occurs Users to waste time repeatedly recreating their context

Thesis A model of task context that automatically weights the relevance of system information to a task by monitoring interaction can focus a programmer's work and improve productivity. This task context model is robust to both structured and semi-structured information, and thus applies to other kinds of knowledge work.

Approach Memory Episodic memory: one-shot, only single exposure required Semantic memory: multiple exposures required Our approach Leverage episodic memory, offload semantic memory Tasks: episodes Context: weighting of relevant semantics to task

Related Work Memory Episodic memory: one-shot, only single exposure required Semantic memory: multiple exposures required Our approach Leverage episodic memory, offload semantic memory Tasks: episodes Explicit tasks (UMEA, TaskTracer): flat model, lack of fine-grained structure Context: weighting of relevant semantics Slices (Weiser) and searches (MasterScope): structure only Interaction-based (Wear-based filtering): no explicit tasks

Model & Operations

Interaction Task context Degree-of-interest (DOI) weighting Frequency and recency of interaction with element Both direct and indirect interaction Model interest

Task context graph Edges added for relations between elements Scaling factors determine shape, e.g. decay rate Thresholds define interest levels Topology [l, ∞] Landmark (0, ∞] Interesting [-∞, 0] Uninteresting

Operations Once task context is explicit Can treat subsets relevant to the task as a unit Can project this subset onto the UI Perform operations on these subsets Composition See context of two tasks simultaneously Slicing Unit test suite can be slow to run on large project Find all interesting subtypes of TestCase c d b c b a T T

More operations Propagation Interacting with method propagates to containing elements Prediction Structurally related elements of potential interest automatically added to task context Only interaction stored

Implementation: programming

Implementation: knowledge work

Validation

Questions Does task context impact the productivity of programmers? Does it generalize to other kinds of knowledge work? Problems Knowledge work environment hard to reproduce in the lab No evidence that non-experts are a good approximation of experts Measure long-term effects to account for diversity of tasks Approach Longitudinal field studies Voluntary use of prototypes Monitoring framework for observation

Study 1: feasibility Productivity metric Approximate productivity with edit ratio (edits /selections) Programmers are more productive when coding than when browsing, searching, scrolling, and navigating Subjects Six professional programmers at IBM Results Promising edit ratio improvement

Study 2: programmers Subjects Advertised study at EclipseCon 2005 conference 99 registered, 16 passed treatment threshold Method and study framework User study framework sent interaction histories to UBC server Baseline period of 1500 events (approx 2 weeks) Treatment period of 3000 events, to address learning curve

Study 2: results Statistically significant increase in edit ratio Within-subjects paired t-test of edit ratio (p = 0.003) Model accuracy 84% of selections were of elements with a positive DOI 5% predicted or propagated DOI 2% negative DOI Task activity Most subjects switched tasks regularly Surprises Scaling factors roughly tuned for study, but still unchanged

Study 3: knowledge workers Subjects 8 total, ranged from CTO to administrative assistant Method and study framework Same framework as previous, monitor interaction with files and web No reliable measure of productivity, gathered detailed usage data

Study 3: results Task Activity Users voluntarily activate tasks when provided with task context Activations/day ranged from 1 to 11, average is 5.8 Task context contents Long paths are common Density over system structure is low Tagging did not provide a reliable mechanism for retrieval Task context sizes Non-trivial sizes, some large (hundreds of elements) Many tasks had both file and web context Model accuracy Decay works, most elements get filtered

Summary

Contributions Generic task context model Weighted based on the frequency and recency of interaction Supports structured and semi-structured data Weighting is key to reducing information overload Capturing per-task reduces loss of context when multi-tasking Task context operations Support growing and shrinking the model to tailor to activities Integrate model with existing tools Instantiation of the model For Java, XML, generic file and web document structure Can be extended to other kinds of domains and application platforms Monitoring and framework Reusable for studying knowledge work

Conclusion Tools’ point of view has not scaled Complexity continues to grow, our ability to deal with it doesn’t Task context takes users’ point of view Offloads semantic memory, leverages episodic memory Impact on researchers University of Victoria group extended it to ontology browsing Users “Mylar is the next killer feature for all IDEs” Willian Mitsuda Industry “…it’ll ultimately become as common as tools like JUnit, if not more so.” Carey Schwaber, Forrester analyst