1. VMT CSCL Workshop June 2004

2. VMT CSCL workshop Evaluation & analysis

3. VMT Workshop June 2004 3 Math Forum research theories Interest and motivation research Social networks and the social construction of community Imagination of community Activity theory and cultural psychology Socio-cognitive approaches to culture Information society/ globalization theory

4. VMT Workshop June 2004 4 Evaluation in the VMT project VMT Project overall(macro level) VMT Working groups(meso level) Math collaboration(micro level) Face-to-face: what is happening? Transfer to virtual: what is different? Virtual explore: what is happing? Virtual support: computer support?

5. VMT Workshop June 2004 5 Analysis methods 1. Quantitative content analysis 2. Ethnographic analysis 3. Conversation analysis

6. VMT Workshop June 2004 6 1. Quantitative content analysis In quantitative content analysis the communication is segmented (if this is applicable), coded, summarized and frequencies/ percentages are used for comparisons and/ or statistical testing.

7. VMT Workshop June 2004 7 Exploring chat Quantitative content analysis Segmentation – define the units of utterances Threading – define the references between Coding – categorize along multiple dimensions Conversation (based on Beers et al., 2004) Social (based on Renninger & Shumar, 2002) Problem-solving (based on Polya, 1957) Mathematics (plus math accuracy and progress) Support (i.e. automated entries and facilitation)

8. VMT Workshop June 2004 8 Coding scheme

9. VMT Workshop June 2004 9 Reliability: Beyond dyadic chat Threading affects reliability Conversational threading (C-thread) Problem solving threading (P-thread) Preliminary reliability C-thread Conversation Support Social

10. VMT Workshop June 2004 10 Preliminary C-thread reliability

11. VMT Workshop June 2004 11 Preliminary reliabilities

12. VMT Workshop June 2004 12 Coded chat excerpt (1)

13. VMT Workshop June 2004 13 Coded chat excerpt (2)

14. VMT Workshop June 2004 14 Coded chat excerpt (3)

15. VMT Workshop June 2004 15 Ethnographic analysis An interpretative method where data is produced through an encounter between the ethnographer and the ‘Other’. The analysis provides ‘thick descriptions’ of social situations. Uses theoretical frameworks combined with interpretative methods in order to understand the ‘natives’ point of view with a social context.

16. VMT Workshop June 2004 16 Ethnography in VMT Using a modified grounded approach Traditional grounded theory is inductive Modified is more theory driven This work will be developed more this summer

17. VMT Workshop June 2004 17 Grounded analysis of chat Looking for connections across lines of text Grounded analysis is relational where coding is discrete Coding gives an important quantitative picture of what people are doing Grounded analysis allows us to think about what the actors are thinking at particular moments

18. VMT Workshop June 2004 18 Analysis of video clips Looking at how people communicate with their bodies in space and time Contrast with online world where extra- linguistic cues are different

19. VMT Workshop June 2004 19 3. Conversation analysis A qualitative, explicitly interpretive analysis of the social practices or interaction methods used by the participants in a conversation to accomplish what they are doing. Typically stresses the sequentiality of conversational moves and how shared meaning is created and negotiated through tacit practices. Ex.: yesterday’s data session interpreting the video clips of taxicab geometry problem solving.

20. VMT Workshop June 2004 20 Conversation analysis in VMT Bridges the content analysis and ethnographic approaches Looks closely at the micro-cultural interactions Allows us to link the specific discursive processes of meaning production with larger social issues, i.e. mathematical thinking or resistance to mathematics

21. VMT Workshop June 2004 21 Future research in VMT Design research: consecutive iterations to improve overall design Understand the practices of online math collaboration Specify the requirements for software design and pedagogy Design problems, software and service Evaluation in PoW-wow! context