1. Multimodal reformulation during shared synchronous note-taking : potential pedagogical consequences for teachers and students Kristine Lund*, Gregory Dyke°, Jean-Jacques Girardot° *ICAR Research Lab, CNRS, University of Lyon °RIM Department, Ecole des Mines de St. Etienne

2. 2 Motivation  Note-taking and multimodal reformulation The corpus  A series of tool-supported meetings – tutoring computer programming projects at l’Ecole des Mines de St. Etienne Tatiana (Trace Analysis Tool for Interaction ANAlysts)  Synchronization of multi-source data, analyses and visualizations Results  Form and content / pedagogical consequences ? Synthesis and perspectives Overview

3. 3 Note-taking Listen, understand and assimilate, choose what to record, study (Suritsky & Hughes, 1991)  Modify definition for our context…(e.g. also reading, but no trace of studying) Look for patterns in note-taking and determine efficiency in relation to students’ results  The value of detailed notes rises with time between the time notes are taken and the test (Williams & Eggert, 2002) ;  A strong correlation between the quantity of notes taken and exam performance (Nye, et. al. 1984). Note-taking as a tool for transforming knowledge and not as data collection (Castello & Monereo, 2005)  We focus on the process of note-taking

4. 4 Multimodal interactions (and reformulations) Modes  Abstract resources “non-material” that help us to make sense – touch, hearing, smell, taste (senses), – gesture, speech (means of expression) Media  Specific material forms in which modes are expressed – The human body, a mechanical body, computer (physical apparatus) Example of a mode  Manipulate an object to write Example of four media Kress & Van Leeuwen (2001) ; Bellik & Teil (1992)

5. 5 Reformulation Collaborative writing / note-taking context An oralo-graphic situation, articulates two different semantic modes  Apotheloz (2001) Cooperation in this task consists in “showing in a continuous way, each step of the way, the manner in which what is formulated articulates with what has already been formulated”

6. 6 A multimodal corpus (talk, writing) First year computer programming projects at Ecole des Mines in St. Etienne  33 tutoring sessions, lasting from 30 to 60 minutes a piece (PPF --> 12 transcribed)  9 groups  3 project types, each chosen by three groups.  These 9 groups were tutored by 2 professors. – Computer traces of note-taking and transcribed dialogue – The computer program / final product – Final reports – The presentations – Grades

7. 7 Exploratory research questions What are the nature of the multimodal reformulations ? How can they demonstrate conceptual transformations ? ?

8. 8 Tatiana - Trace Analysis Tool for Interaction ANAlysts Synchronize multi-source data  Computer traces of human interaction  Videos and their corresponding transcription (dialogue, etc.) Replay in order to understand activity Define analysis units and categorize facets of the interaction Visualize the categorizations on a temporal line Regroup, re-categorize, re-visualize Share

9. 9 Synchronize the multi-source data, replay and think about units of analysis / Research Q Transcription of talk Log file Shared text editor where notes are taken (replayed) Video

10. 10 Create the note-taking writing units (done manually) Log file Note-taking units

11. 11 Categorize the reformulated - reformulation pair (done manually) Categorize the transcription of talk Categorization schema Categorize the note-taking units

12. 12 Create the links between the reformulated - reformulation pair (manual ) Categorized transcription of talk Categorized note-taking units Directional link between the reformulated and the reformulation

13. 13 Visualization (automatically created) Transcription of talk A’s note- taking B’s note- taking The reformulatedThe reformulation Synchronize multi-source data Replay in order to understand activity Define analysis units and categorize facets of the interaction Visualize the categorizations on a temporal line Regroup, re-categorize, re-visualize Share

14. 14 Tatiana - example analysis of corpus Log Data (1) Writing units (2) Transcription (3) Tool replayer (5) Video (6) Visualization (4) Highlights are synchronized Remote control (7) Transcription Writing units Reformulation

15. 15 Results - session 1

16. 16 A writing conflict provokes a spatial organization Results - session 2

17. 17 Chronological time-lag between monologue and note-taking Results - session 2

18. 18 Differences in the structure of reformulation Results - session 2

19. 19 Reformulation of tutor dialogue written and co- elaborated by two students Results - session 2

20. 20 Tutor reaction upon seeing note-taking Results - session 2

21. 21 Synthesis, interpretations and questions Multimodal reformulation, co-elaborative written reformulation of dialogue  Not yet seen oral reformulation of co-elaborative note-taking From written to oral  Response to a student question;  Incite students to take note-taking further (conceptual deepening) – Is this still reformulation ? From oral to written  Anticipation of teacher talk ; do such students understand content better ?  How to understand the choices students make when reformulating ?  What is the linguistic nature of these reformulations ?  How do the characteristics of the shared text editor influence note-taking and co- elaboration? (technical affordances of resources) – Collaborative note-taking ?  For what type of content is there a time-lag between teacher talk and note-taking ?

22. 22 Perspectives More closely study the nature of these multimodal reformulations How can they illustrate knowledge transformation ?  We don’t look to measure student performance in terms of exam results as a result of note- taking.  Rather, we want to quality the transformation of conceptual knowledge through characterizing reformulation.  What are the pedagogical consequences ? How can the shared text editor characteristics influence note-taking ?  More or less co-elaboration ? Can note-taking be understood as a sort of formative (vs. summative) evaluation ? What about auto and hetero-regulation ? And vicarious learning?

23. 23 Thanks to the PPF ! Transcription of dialogue from videos Communication between TATIANA and CARTE  Collection, activity Analysis and Regulation based on Traces Enriched (Christophe Courtin, Stéphane Talbot / Chambery) – to facilitate automating certain analysis tasks (e.g. definition of note-taking units, categorization) – to highlight relevant sequences of events in large numbers of TATIANA traces – …  CARTE supplies the following software tools : – convert TATIANA traces CARTE traces and vice versa – import/export CARTE traces to and from a corpus of traces – Analyze CARTE traces by means of rules (pre-defined patterns)

24. 24 You can download Tatiana http://code.google.com/p/tatiana/ gregory.dyke@emse.fr kristine.lund@univ-lyon2.fr jean-jacques.girardot@emse.fr steven.collins@emse.fr