September 29, 2002Ubicomp 021 NIST Meeting Data Collection Jean Scholtz National Institute of Standards and Technology Gaithersburg, MD USA

September 29, 2002Ubicomp 022 Goals / Approach Goals (Intermediate) Support evaluation efforts for speech community (Future) Support evaluation efforts for context- aware/ubicomp community Approach Provide rich/diverse pool of audio video corpora for advanced recognition research Provide infrastructure for integration and evaluation Provide flexible architecture for incorporating and annotating activities in sensor –rich environment

September 29, 2002Ubicomp 023 NIST Data Collection Laboratory Typical meeting space equipped with standard meeting equipment Instrumented with –200 microphones, 5 cameras – all synchronized with NIST SmartFlow system across 13 processors –several disk arrays handle huge storage requirements Monitor/Review workstation in adjoining room –operator can start/stop data streams, select video views, audio channels, and manipulate cameras –participants can review meeting recordings and deselect excerpts from public distribution Relational database log with user interface

September 29, 2002Ubicomp 024 Entrance Ramp Whiteboard Modular Conference Table 13’ x 5’ NIST ‘SmartFlow’ synchronization NIST Meeting Data Collection Laboratory 7’ x 7’ screen Ceiling Projector E-Whiteboard mic omni mic quad mic omni mic COTS Mic Data Collection System Wireless Mic Receivers omni mic 3 Mic Array Data Collection Systems Mic Array 5 Video Data Collection Systems

September 29, 2002Ubicomp 025 Meeting Review/Monitor Workstation Used to both collect and review recordings LINUX-based, interfaces with Smartflow live or archived data User can select video views and audio channels User can control camera view/movement with joystick

September 29, 2002Ubicomp 026 Scenario Ontology Scenario Type Scenario Example Dimensions Tasks TimeParticipants Duration (minutes) Local vs. Remote Same people Different people Org. pre vs. adhoc #of participants Domain expertise Moderator planningBrainstormingNegotiation Decision makingCompetitive performance Dissemination of Information Different Focus group discussion Place Local605-8noneadhoc Round Table discussion: May include expert to present info and moderate discussion Staff meeting Local455-8adhoc Participants discuss real technical issues, brainstorm ideas and make decisions. Planning Local455-8noneadhoc Event planning: Plan the different tasks that each member will carry out, timelines and coordination. Interactive game playing Local603-5nonepre Information exchange and decision making Information gathering and decision making Local adhoc prenone Office Furnishing: An expert will help participants to choose office furniture, carpet, etc. for an office. Participants engage in a storytelling games, e.g. Once upon a time, Settlers of catan. Shop on line: Participants search the internet and collaborate to purchase computer hardware.

September 29, 2002Ubicomp 027 Pilot Corpus Design 20 hours of meetings are being collected –~60 GB per hour uncompressed data rate –data stored/distributed on large hard disks Varied forums –focus groups, game playing, interacting with experts, real working group meetings, event planning Varied meeting lengths –15 minutes to 1 hour Varied number of participants –3 to 8 participants

September 29, 2002Ubicomp 028 Sample of Meetings Collected TopicMeeting TypeDateLength (min) Native M / F Non- native M / F Participants IAD New Year Party Planning Planning12/12/01600 / 20 / 13 IAD Open HousePlanning3/5/02561 / 41 / 17 ITL Party PlanningPlanning1/11/02283 / 20 / 16 BioterrorismFocus group11/15/ / 10 / 04 Office Furniture Layout Interaction with expert 2/14/02551 / 30 / 26 Settlers of CatanGame playing1/25/02600 / 03 / 14 Once Upon a TimeGame playing3/4/02581 / 12 / 04 Real Working GroupStaff meeting2/13/02693 / 21 / 06 Shop On-lineCollaborative design 3/19/02601 / 03 / 04

September 29, 2002Ubicomp 029 Use of Architecture Streaming Sensor Input: video, audio, images, location tracks, application interactions Distributed Services: Smartflow, Device management sw, Distributed data management, Archival Storage, Service/ device discovery MetaData Architecture: ATLAS, Annotation/review stations, Annotation ontology, Visualization, Browsing, Annotation generation, Annotation tools, Logging tools for COTS products, sensor data annotation Context Architecture: Annotation integration and abstraction, Realtime annotation maintenance, Context entities (people, activities, artifacts) Component Technologies: Context abstraction components, simulations for context-aware apps CorporaComponent test bed Context-Aware apps Non-aware apps Component test bed Test bed Architecture Component Technologies: Recognition components (ASR, Speaker ID, Face Recognition), Location recognition, Visualization of sensor data Architecture Can incorporate component technologies, log interactions, evaluate Can incorporate context widgets, log interactions, evaluate

September 29, 2002Ubicomp 0210 Future Plans Extend to distributed meetings/activities Provide test bed capabilities to allow developers of applications to incorporate new component technologies and capture user interactions for user-centered evaluations. Develop annotation ontology for user interactions to facilitate user-centered evaluations of meeting room applications. Extend this capability to context-aware applications allowing developers access to sensor inputs without having to install those sensors in their environment. Create community of researchers interested in using this technology