VMT CSCL Workshop June 2004

VMT CSCL workshop Evaluation & analysis

VMT Workshop June Math Forum research theories Interest and motivation research Social networks and the social construction of community Imagination of community Activity theory and cultural psychology Socio-cognitive approaches to culture Information society/ globalization theory

VMT Workshop June Evaluation in the VMT project VMT Project overall(macro level) VMT Working groups(meso level) Math collaboration(micro level) Face-to-face: what is happening? Transfer to virtual: what is different? Virtual explore: what is happing? Virtual support: computer support?

VMT Workshop June Analysis methods 1. Quantitative content analysis 2. Ethnographic analysis 3. Conversation analysis

VMT Workshop June Quantitative content analysis In quantitative content analysis the communication is segmented (if this is applicable), coded, summarized and frequencies/ percentages are used for comparisons and/ or statistical testing.

VMT Workshop June Exploring chat Quantitative content analysis Segmentation – define the units of utterances Threading – define the references between Coding – categorize along multiple dimensions Conversation (based on Beers et al., 2004) Social (based on Renninger & Shumar, 2002) Problem-solving (based on Polya, 1957) Mathematics (plus math accuracy and progress) Support (i.e. automated entries and facilitation)

VMT Workshop June Coding scheme

VMT Workshop June Reliability: Beyond dyadic chat Threading affects reliability Conversational threading (C-thread) Problem solving threading (P-thread) Preliminary reliability C-thread Conversation Support Social

VMT Workshop June Preliminary C-thread reliability

VMT Workshop June Preliminary reliabilities

VMT Workshop June Coded chat excerpt (1)

VMT Workshop June Coded chat excerpt (2)

VMT Workshop June Coded chat excerpt (3)

VMT Workshop June Ethnographic analysis An interpretative method where data is produced through an encounter between the ethnographer and the ‘Other’. The analysis provides ‘thick descriptions’ of social situations. Uses theoretical frameworks combined with interpretative methods in order to understand the ‘natives’ point of view with a social context.

VMT Workshop June Ethnography in VMT Using a modified grounded approach Traditional grounded theory is inductive Modified is more theory driven This work will be developed more this summer

VMT Workshop June Grounded analysis of chat Looking for connections across lines of text Grounded analysis is relational where coding is discrete Coding gives an important quantitative picture of what people are doing Grounded analysis allows us to think about what the actors are thinking at particular moments

VMT Workshop June Analysis of video clips Looking at how people communicate with their bodies in space and time Contrast with online world where extra- linguistic cues are different

VMT Workshop June Conversation analysis A qualitative, explicitly interpretive analysis of the social practices or interaction methods used by the participants in a conversation to accomplish what they are doing. Typically stresses the sequentiality of conversational moves and how shared meaning is created and negotiated through tacit practices. Ex.: yesterday’s data session interpreting the video clips of taxicab geometry problem solving.

VMT Workshop June Conversation analysis in VMT Bridges the content analysis and ethnographic approaches Looks closely at the micro-cultural interactions Allows us to link the specific discursive processes of meaning production with larger social issues, i.e. mathematical thinking or resistance to mathematics

VMT Workshop June Future research in VMT Design research: consecutive iterations to improve overall design Understand the practices of online math collaboration Specify the requirements for software design and pedagogy Design problems, software and service Evaluation in PoW-wow! context