Collaboration via the Info. Utility John Canny Endeavour Mini-retreat 1/20/2000

Consequences of Ubiquitous Computing and Calm Technology: As computing becomes ubiquitous and invisible, it also ceases to be the focus of activity. The focus shifts instead to people and the tasks they are working on. This implies a Human-Centered Computing approach (HCC) to designing applications. Our work involves collaborations with several faculty outside CS who are expert in aspects of human behavior, and especially group behavior, via a HCC center.

Three takes on collaboration Design media for natural (physical) interaction: Desks, PRoPs, Bearables. Model behavior: Activity-Based Computing (ABC): –uses tacit data to infer structure in group members’ activities. Learning as collaboration: –Large classes create opportunity via group cooperation and student perspectives to help other learners.

Three takes on collaboration Design media for natural (physical) interaction: Desks, PRoPs, Bearables. Model behavior: Activity-Based Computing (ABC): –uses tacit data to infer structure in group members’ activities. Learning as collaboration: –Large classes create opportunity via group cooperation and student perspectives to help other learners.

Designing Media: Distributed Cognition Theory People use artifacts in the world to share their thoughts (Norman, Hutchins, Pea). Computer-enhanced artifacts have richer behaviors and be shared across distance.

A desk that loses entropy... The UPM (Universal Planar Manipulator) is a desk that can move many objects placed on it. It is a tangible input/output device.

Applications of the UPM Teaching Chemistry: A tangible simulation of planar organic chemistry. Colored disks for atoms, simulation of bond forces and reactions, movement of atoms by the desk. Urban planning. Phycons of buildings moved by initially by hand, then aligned by the desk according to a set of rules. Other layout problems: web site design (moving postit-like notes).

UPM principle of operation

PRoPs: Proxys for remote collab. There is still a huge gap between face-to-face and videoconferencing. First difference: Physical cues: gaze, posture, proxemics... are missing or distorted. Much collaboration involves “being in” a remote space. You need to have a body there...

PRoPs: Proxys for remote collab. PRoPs are internet-controlled robot avatars that allow you to have a second presence somewhere. The user sits at a computer console watching video from the PROP, and controlling it with mouse and joystick.

Theoretical underpinnings Right now, much of the theory driving PROP design comes from outside CS: Non-verbal communication (D. Keltner psychology). Theory of interpersonal psychology (G. Mendelsohn). Spatial reasoning and spatial memory.

Bearable computers The laptop can serve as a “bearable” computer, linked wirelessly to its owner. Laptop in briefcase with wireless TV and keyboard TXs. Pocket-worn portable LCD TV Double-mirror element in glasses Heads-up virtual image Wireless chordal keyboard or Palmpilot in pocket for input.

Advantages of Bearables vs. Wearables Short term: –Leverage higher performance of laptops and mini- TV displays compared to wearable components. Long term: –No wires between components. –No power source to the head. –The laptop display is preferable sometimes. –Some optical benefits for augmented reality.

Three takes on collaboration Design media for natural (physical) interaction: Desks, PRoPs, Bearables. Model behavior: Activity-Based Computing (ABC): –uses tacit data to infer structure in group members’ activities. Learning as collaboration: –Large classes create opportunity via group cooperation and student perspectives to help other learners.

ABC: Activity-Based Computing How do you build a UI when the computer is invisible, e.g. a smart space? What is context and how do you use it? How does ABC help collaboration?

ABC: Activity-Based Computing How do you build a UI when the computer is invisible, e.g. a smart space? Build a conceptual model instead. What is context and how do you use it? How does ABC help collaboration?

ABC: Activity-Based Computing How do you build a UI when the computer is invisible, e.g. a smart space? Build a conceptual model instead. What is context and how do you use it? Use Activities as the context. How does ABC help collaboration?

ABC: Activity-Based Computing How do you build a UI when the computer is invisible, e.g. a smart space? Build a conceptual model instead. What is context and how do you use it? Use Activities as the context. How does ABC help collaboration? Pro-active document sharing; Understanding roles; Document context;

ABC: Activity-Based Computing Activities are clusters with: –Users –Documents –Tools The model can be implemented as a GUI, or with speech, etc. Nearness encodes awareness.

ABC: Motivation Based on “activity theory”, a rich psychological theory of human behavior, especially collaboration. AT Emphasizes on tools (such as computers) as mediators of activity. Includes the role of symbol systems (language, documents) as mediators. Ties together development (learning) and everyday activity. Rich models of context.

ABC: Uses Context (disambiguation) of person, object, tool. Pro-active sharing of information. Skill mining (tracking user expertise). Document visibility and access rights. Understanding individual’s roles in activities. Managing work relationships. Inferring quality of documents/tools. Sensemaking for documents. Perspectives on a document or meeting. Attention management/awareness.

ABC: Under development now Context (disambiguation) of person, document, tool. Pro-active sharing of information. Skill mining (tracking user expertise). Document visibility and access rights. Understanding an individual’s roles in activities. Managing work relationships. Inferring quality of documents/tools. Sensemaking for documents. Perspectives on a document or meeting. Attention management/awareness.

ABC: Tacit data sources Direct communication: 1-1 , phone, F2F. Topical discussions, forums, F2F meetings. Document access, reading, doc. markup. Document search and browsing, who, when.

Knowledgescapes: Info Retrieval with tacit data Information Need Query (terms) Search Engine Document1 Document2 Document3 Document4 Rankings from reading time. Treat as probability of interest or E(interest)

Info Needs and tacit rankings Info need1 Document1 Document2 Document3 Document4 Info need2 Info need3 Document5 P(D1|N1) P(D5|N3)

Start with a query string and use all the information need nodes recorded for that query. Return highest weighted docs. First use: Most popular hits Document1 Document2 Document3 Document4 Info need2 Info need3 Document5 Query

The IN is usually more specific than the query, and this is encoded in the user’s tacit rankings. Tacit knowledge in user queries Document2 Document3 Document4 Info need1 Info need3 Document5 “Airlines” Info need2 Document1 West to East Coast US to France

Tacit user ranking while browsing boosts most probable info needs, improving recall. Tacit knowledge in user queries Document2 Document3 Document4 Info need1 Info need3 Document5 “Airlines” Info need2 Document1 West to East Coast US to France

Tacit user ranking while browsing boosts most probable info needs, improving recall. Tacit knowledge in user queries Document2 Document3 Document4 Info need1 Info need3 Document5 “Airlines” Info need2 Document1 West to East Coast US to France

Tacit user ranking while browsing boosts most probable info needs, improving recall. Tacit knowledge in user queries Document2 Document3 Document4 Info need1 Info need3 Document5 “Airlines” Info need2 Document1 West to East Coast US to France

Current testbeds: Berkeley DigLibs (environmental) collections. Knowledgescapes on Dlib is running at: IMDB Philip Stark’s statistics course. NEEDs: the ME courseware database. Future: My groups web and campus library access.

Three takes on collaboration Design media for natural (physical) interaction: Desks, PRoPs, Bearables. Model behavior: Activity-Based Computing (ABC): –uses tacit data to infer structure in group members’ activities. Learning as collaboration: –Large classes create opportunity via group cooperation and student perspectives to help other learners.

Education as collaboration: Opportunities of scale. In lecture format classrooms, attention is the critical resource (Norman, Papert...). Attention is best gained by interaction: with artifacts (the LOGO model) or with other students (the Stanford TVI model).

An augmented reality classroom Virtual image with notes, questions, private chat space Physical space Glasses to rotate the laptop image

Livenotes in the classroom Students work in groups of 4; communicate silently via pen or keyboard chat. Each group has one main note-taker; others add their own comments or questions to the transcript. Students can mark up a group transcript, the lecturer’s notes, or a non-archived window. One student per group works as facilitator or TA, posing questions to the others.

The group transcripts include notes, plus student comments and questions. Remote live participants should be equally engaged in the note-taking. Using Livenotes

4 Students used Livenotes in a grad course in F99 on IBM laptops running Netmeeting on a wireless net. Reactions: –Overlay touch screens were bad, everyone used keyboard chat. –Difficulty in listening and chatting simultaneously only in first lecture. –After that, attention level higher. No chance of falling asleep. –Many notes: two parallel threads, the note-taker and the group chat. Group chat periodically comes back to lecture content as new notes appear from the note-taker. Initial feedback

Move to Vadem Clios with wireless: –Support both keyboard and pen note-taking. –Cheaper, lighter, batteries last all day. Develop custom software based on feedback: –Support conversation threads. –Include hyperlinks (or hyper-ink). –Include timestamping to allow synchronization with MM transcript of the lecture. –In-class “lecture rewind” was proposed as a useful feature. Next Step

Offline students lose interaction, but gain through access to live student perspectives (their notes). By organizing live student notes according to expertise and a few dimensions of perspective, we can provide offline students with an ideal “scaffolding” perspective. Further along...

The End..

PROPs and Bearables are complementary A user with a bearable display can see a video overlay of the PROPs pilot.

PROPs and Bearables are complementary A user with a bearable display can see a video overlay of the PROPs pilot. This display can cover a much larger area than the PROPs built-in display, and provide body language cues.

PROPs and Bearables are complementary A user with a bearable display can see a video overlay of the PROPs pilot. This display can cover a much larger area than the PROPs built-in display, and provide body language cues. In this display, the pilot is standing up, implying he is controlling the PROP with a wearable computer.