Research in Statistics Education Some reflections on the field and approaches of inquiry Simon Goodchild

Fields of inquiry Intended curriculum Interpreted curriculum Communicated curriculum Received curriculum Enacted or effective curriculum Beliefs and values of curriculum planners. Beliefs and values of teachers. Approach, resources used, etc. Beliefs, attitudes and values of students. Knowledge and skills available to students outside the classroom.

Three perspectives on Statistical Education Mathematician Statistician Lay-person Context for applying and exploring mathematical techniques and concepts. The appropriateness and behaviour of statistical measures – using statistics to test theory. Interpretation of information communicated through statistics A naïve characterisation …

‘average after-tax income’ for households of couples aged 45 to 54, with children (in 1998) was Kroner. ‘Statistics Norway’ (Statistisk sentralbyrå 2000)

Table 258 gives population data for ‘Persons aged years, by sex, marital status and socio-economic group. Annual average’. … the total population in 1995 was males, and females.

Examples Field of inquiry Choice of method Content of the instrument Design of the study Interpretation of the data

Choice of method Test Questionnaire Interview Observation with intervention Observation without intervention diagnosing interpreting probing challenging perceiving METHOD characteristic Confirming theory Developing theory

Methods – factors to consider Objectivity Generalisability Reliability Replication Comparability Cost and resources Communicability Possibility of aggregation and analysis

Design of the study Context of fieldwork – location, time, relationships Schedule of fieldwork Fieldwork and data analysis tools Pilot study and development of ‘instruments’ - operationalisation

Interpretation of the data What does the data ‘say’? What explanations are possible? What use is it? Where does it lead to next?

Example 1 School students’ understanding of average. - Exploring the enacted curriculum.

Motivation Pollatsek, Lima & Well (1981) wrote: … for many students, dealing with a mean is a computational rather than a conceptual act. Knowledge of the mean seems to begin and end with an impoverished computational formula … This remark was made about US college students. In the UK students are taught about the mean in secondary school, the remark motivated me to explore whether it applied also to students in the UK secondary school context.

Semi-structured interviews I wanted to probe understanding – hence I chose to interview students. Oral Repeats Rephrase Timing Probing Order of Qs Variations of Qs and branching Planned Motivates responses Affective response and stress Uniformity Comparability Misleading Communication between subjects proscons

Context of questions It is when we are able to interpret ‘average’ in a real-world context and anticipate its application in a non-routine problem that we can can claim the concept is understood. Responses to routine questions, as experienced in the mathematics class, do not provide any insight into a student’s understanding.

Design: to explore and expose … Interviews followed a planned schedule of questions. Each interview lasted from 20 to 30 minutes. Students were instructed not to discuss – so that they did not give an unfair advantage, students were asked if they had any idea about the questions before the interview stated. Students were provided with pencil, paper and a calculator. They sat at a table with the interviewer, a tape recorder was also on the table, recording the interview Two phases, each took place over one week with one week (holiday) separating the two phases. The interview schedule was modified between phases.

Problematic issues: Responses ‘embedded’ in the perception of the context of packing matches. Responses ‘embedded’ in the context of mathematics. Interpretation of language used. Inappropriate cues.

Implications for the practice of statistical education Need to develop students’ understanding of average in terms of representativeness and expectation as well as location. Need to give students opportunities to make inferences about data from given summary statistics as well as calculating the statistics.

Example 2 School students’ goals in mathematics classroom activity. - Exploring the received curriculum

Motivation A growing awareness of the unpredictability of students’ responses to teacher’s actions. A desire to explore the received curriculum and how this might relate to the communicated curriculum.

Conversations I wanted to explore and expose the goals towards which students were working in the course of their regular activity. Thus I needed to observe and intervene in that activity. Flexible: follow the student. Open: not pre-determined by theory or pre-existing ideas. Responsive: do not impose special requirements or conditions. Natural: observe the student in regular activity. Immediate: do not rely on retrieval from LTM or transfer between STM and LTM.

Conversations Challenging & probing Concurrent observation Minimal disturbance Flexible Open Responsive Rephrasing Natural Immediate Preparation Unsystematic Inefficient Prolonged engagement Volume of data Summarisation pros cons I wanted to explore students’ mind-set as they engaged in regular classroom activity.

The lack of structure in a conversation leaves it open for challenging and probing that enables the researcher to test his/her own concurrent interpretations and the robustness and resilience of students understanding. The informality of conversations creates opportunities for rephrasing questions and reduces the level of stress experienced by the student. Concurrent observation enables the collection of other data such as communication between students and interactions between teacher and students. Creates minimal disturbance in students routine.

Impossible to prepare for situations that arise. Unstructured might be interpreted as unsystematic. Impossible to ensure efficiency and effectiveness of fieldwork. The approach requires a great commitment of time, it leads to a huge volume of data to be analysed but the rewards appear quiet modest. The trustworthiness of the research relies on ‘thick description’. The result is qualitative data, it is difficult to summarise and its accessibility makes it appear commonplace. New knowledge ought to look different and difficult to understand!

Generalisability Sample to population generalisation is not possible. Analytic generalisation – that is testing consistency with theory and other published research is possible. Case-to-case generalisation is possible for independent readers of the research.

Context of questions Conversations were prompted by questions that arose in the regular activity of students. The tasks set by the teacher, and the students’ response to those tasks set the context.

Design: to explore and expose An ethnographic-style, part grounded theory study. Attending every mathematics lesson of a Year 10 Mathematics class for nearly one complete year. Observing and audio-recording episodes in which the teacher interacted with the whole class. Interacting with individual students in their activity – observe – engage. Moving between students, revisiting students after a period of about 5 weeks. Collecting documentary evidence: photocopies of pages from student work books, records of achievement, test papers, printed texts, scheme of work, etc. Pilot study

Analysis of data Use of NUD.IST Stages of analysis Testing the interpretation

Problematic issues: The approach demands a deep knowledge of the mathematics studied and types of students responses to the tasks set. Given this the researcher cannot enter the field as ‘ignorant stranger’, that is as an anthropologist approaches ethnography. My response to this challenge was to complement my knowledge of the mathematics classroom with a theoretical framework that was constructed from different accounts of cognition – the framework required the adoption of different theoretical positions.

Problems experienced Researcher’s familiarity Researcher as ‘teacher’ Disturbance of the arena Weariness

Implications for the practice of statistical education: The tasks in which students engage should appear meaningful and realistic. The topics should be perceived as purposeful and useful. Need to develop approaches that make probabilistic reasoning meaningful for students.