CSCW – Evaluation P. Dillenbourg & N. Nova Evaluation & Exam

CSCW – Evaluation P. Dillenbourg & N. Nova 4 research positions Project 1. –D2: Visualization of group organization –D3: Statistical analysis –D4: Interactions analysis Project 2 –D5: Table design ( ) –D6: Video analysis (  PPT) Agenda –11.01 Presentation D3 & D4 + Video Analysis –18.01 The role of virtual & physical space in collaboration –25.01 Knowledge Managment + Course evaluation –01.02 Synthesis, Questions –??.02 Exam 30 min: Team presentation of Project 2 (15 min) plus questions 3 X 20 min individual questions on projects1 & 2 Language: English or French, up to you. Mark –30%: (written) Deliverables D2,D3,D4 –30%: (oral) Presentation of project 2  Bring your PPT on laptop / USB key –40%: (oral) Four questions on project 1 & 2 (  50 question examples on the web site) –You can bring all documents and your laptop with you

CSCW – Evaluation P. Dillenbourg & N. Nova

CSCW – Evaluation P. Dillenbourg & N. Nova

CSCW – Evaluation P. Dillenbourg & N. Nova Hitachi

CSCW – Evaluation P. Dillenbourg & N. Nova

D6: Video Analysis: Questions 1.Does the table support group participation? –Does the leader occupy a more central position? –What’s the position of the ‘left over’ if any? –Are some chairs moves away from the table? –Do they exchange objects? –What’s the orientation of bodies and chairs? –Do they sit where you expected them to sit? 2.How laptop usage interact with other activities? –Do they move often their laptops for re-organizing their own space –Do they move their laptops or fold its screen to facilitate interaction? –How often do their turn their laptop to show its screen on the same-side partner or to another-side partner ? –How much time is spent looking at the laptop versus looking at each other ? 3.How do they use the table space? –Is there some ‘dead’ zone versus an ‘interaction’ zone’? –What are the objects present on the table, which one are often used? –Do they share or exchange some objects? –What do they draw on the table? Where do they draw? Who draws? –Where do they put the other objects on the table (documents, mugs,..) ? –Is there useless space? Is there a ‘focus’ area? –Are there different phases in using the table space?

CSCW – Evaluation P. Dillenbourg & N. Nova Video Analysis Methods 1. analytical (usability studies/ experimental research) 2. synthetic 3. critical events

CSCW – Evaluation P. Dillenbourg & N. Nova Results Segmentation Coding Counting Transcript (annotated text file) Units (utterances, turns, content units,…) Categories (speech acts, grounding acts,…) Transcription Dialogue (audio file) 1. Detailed interaction analysis

CSCW – Evaluation P. Dillenbourg & N. Nova Who draws where ? Colour = person Intensity = erase & redraw Where are the laptops? A snapshot every 5 minutes 2. Synthetic usage maps

CSCW – Evaluation P. Dillenbourg & N. Nova Who talk to who ? Social Network Analysis in a spatial context Position & direction of the chairs? A snapshot every 5 minutes 2 2. Synthetic usage maps

CSCW – Evaluation P. Dillenbourg & N. Nova 3. Critical events It’s a subjective method, used in ethnography, that aims to detect specific episode that help better to understand the dynamic of team processes than neutral statistics Salient events –Main reorganisation of space –Conflict about space (territories) –“Perspective” episode (turning documents or laptops) –Loss of participation –… Significant events –Emergence of convention on how to use space, implicit or explicit –Convergence towards a visual focus zone –…

CSCW – Evaluation P. Dillenbourg & N. Nova D6: Video Analysis: Deliverables Maps of Table Usage “Critical events” (Interesting episodes) 3.Conclusions 1.Pros and cons regarding to your table design 2.Suggestions for design a CSCW table (included groupware) 8 slides