1. Ask no Questions Get told no lies! ……………. Comparative Q’s Jim Hogan Marc Paterson Informed by doctoral thesis work by Dr Pip Arnold

2. Rationale – why this topic? Moderator comments strongly suggested that good investigative questions led to good analysis and good student answers. They also reported (2014) that the lack of good questions in moderated work, approved by teachers, persists. This.ppt is only about creating good questions and only for year 11,12,13. The year 11 information could apply to junior ages as well. And …

3. Dr Pip Arnold (doc thesis) Statistical investigations is a major strand of the New Zealand statistics curriculum. A preliminary study identified the initial problematic situation that some year 10 students (ages 14–15) were unable to complete a statistical enquiry because their question, which the teacher had approved, was unsuitable for the multivariate data set they were investigating. An understanding about key concepts underpinning statistical questions seemed to be lacking. A subsequent review of the literature failed to find criteria for what makes a good statistical question. Hence the purpose of her study…and…

4. Doctoral Thesis is online at: https://researchspace.auckland.ac.nz/handle/2292/21305 The main findings from the research were: identification of the criteria for what makes a good question… explication of the conceptual infrastructure that students need for investigative questions, making a call… …especially developed learning materials designed to build concepts such as variable, sample, population, sampling variability,… 249 pages of reading! Go for it. See last slide…

5. Plan Level 1 clarifications Example Questions Very good people Do Maths Framework Which example investigative q’ are fit for purpose Level 2 clarifications Example Questions Very good people Do Maths Framework Which example investigative q’ are fit for purpose Level 3 clarifications Very good people Do Maths Framework Which example investigative q’ are fit for purpose What have we learned?

6. Level 1 – Lets start here.

7. AS91035 (clarifications) Problem Students need to pose their own investigative question for this standard. The investigative question must involve a comparison, include the variable to investigate, specify the groups being compared, the population and the direction of the comparison. A suitable question would be ‘Do New Zealand year 11 boys tend to be taller than New Zealand year 11 girls?’ NZQA link

8. Possible Investigative Questions AS91035 I wonder if 15 year old girls are shorter than 15 year old boys in the 2015 NZ C@S database? I wonder if NZ Year 11 boys are taller than NZ Year 11 girls? I wonder if boys have longer right feet than girls? I wonder if boys tend to be taller than the girls in the 2015 NZ C@S database? I wonder if 16 year olds have longer hair compared to 15 year olds? Which of these are not fit for purpose and why?

9. Here is a Framework to use Very Good People Can Do (3) (Maths)

10. Here is a Framework to use Very – Variable identified Good – Groups, 2, clear People – Population clear Can – Comparison tendency Do (3) – Direction (Difference) (Maths 2,3) - Medians

11. Apply the frame VGPCD(M) I wonder if 15 year old girls are shorter than 15 year old boys in the 2015 NZ C@S database? I wonder if NZ Year 11 boys are taller than NZ Year 11 girls? I wonder if boys have longer right feet than girls? I wonder if boys tend to be taller than the girls in the 2015 NZ C@S database? I wonder if 16 year olds have longer hair compared to 15 year olds? VGPCD

12. Level 2 – Let’s move on

13. AS 91264 (clarifications) Problem The investigative question that is posed must involve a comparison and needs to include the variable, the population groups being compared, the population parameter that the inference will be about and the direction of the comparison.

14. Possible investigative questions AS 91265 I wonder if the median right foot length for Year 11 boys tends to be longer than the median right foot length for year 11 girls in the 2015 NZC@S database. I wonder whether women in NZ have less salary/wages than men in NZ. I wonder if the arm span of Year 12 boys is longer than the arm span of Year 12 Girls in the 2015NZC@S database? I wonder if there is a difference between the median income of a highly educated person in NZ and the median income of a lowly educated person in NZ Which of these are OK and which are not? Why?

15. Reminder Very – Variable identified Good – Groups, 2, clear People – Population clear Can – Comparison tendency Do (3) – Direction (Difference) (Maths 2,3) - Medians

16. Possible investigative questions AS 91264 I wonder if the median right foot length for Year 11 boys tends to be longer than the median right foot length for year 11 girls in the 2009 NZC@S database. I wonder whether women in NZ have less salary/wages than men in NZ. I wonder if the arm span of Year 12 boys is longer than the arm span of Year 12 Girls in the 2009NZC@S database? I wonder if there is a difference between the median income of a highly educated person in NZ and the median income of a lowly educated person in NZ VGPCDM

17. Level 3 – Moooving ON

18. AS91582 (clarifications) Posing an appropriate comparison investigative question using a given multivariate data set Sufficient time needs to be allocated for students to research the context and acquire appropriate contextual knowledge. For all levels of achievement students need to identify a purpose and pose an investigative question which is informed by this contextual knowledge. The question needs to be comparative and needs to refer to the population and the parameter under investigation. [Remember quantify the difference…]

19. Possible Investigative Questions AS91582 What is the difference between the median number of text messages sent per day by adults in New Zealand and the median number of text messages sent per day by teenagers in New Zealand? What is the difference between the median iron levels of athletes who play ball sports and the median iron levels of those who play a non-ball sport? Comparing of the median % of body fat difference in female and male. (from different sports of AIS)

20. Reminder Very – Variable identified Good – Groups, 2, clear People – Population clear Can – Comparison tendency Do (3) – Direction (Difference) (Maths 2,3) - Medians

21. Possible Investigative Questions AS91582 What is the difference between the median number of text messages sent per day by adults in New Zealand and the median number of text messages sent per day by teenagers in New Zealand? What is the difference between the median iron levels of athletes who play ball sports and the median iron levels of those who play a non-ball sport? Comparing of the median % of body fat difference in female and male. (from different sports of AIS) VGPCDM

22. Key Ideas at each level. Level 1 is about a tendency of numeric variable being greater or less between two groups. Informal inference of the population based on visual comparison of data. Level 2 is about asking a tendency for the median of one group being greater or less between two groups. Informal Inference of the population is based on use of the overlap of the informal confidence intervals. Level 3 is about quantifying the difference between the medians. A formal inference of the population is made from a bootstrapped confidence interval.

23. Have I answered my investigative question? Does my investigative question allow the inference to be made at the right level? ……….. See C@S

24. Good learning We sample to make a guess about a population parameter. We use the “sample variation VIT tool”. We prove 1/√n We empower students to own formula. We use iNZight We regularly visit SSTLG

25. Criteria Considerations for teaching and curriculum development The variable(s) of interest is/are clear and available The variable being described for a summary situation is clear. The variable being compared in a comparison situation is clear. The variables being looked at for a relationship are clear. The variable(s) has been correctly identified from the actual survey question that was asked; see the example given on pages 107–108 about students rating their ability in maths. The variable(s) is/are available. For example, students posed a question around favourite place to live when the variable given was the region that students live in. The variable favourite place to live was not available, only the variable where students live was.

26. Criteria Considerations for teaching and curriculum development The population of interest is clear Watch student use of the words a or the in front of the population or group they are using. For example, students write about “a boy”, “the boys” or simply “boys”. The difference between these three is that “a boy” indicates an individual, “the boys” suggests the boys in the sample, whereas “boys” most likely indicates that it is about the population. In addition students sometimes refer to “these boys”, also meaning the sample rather than the population (see pages 104–105). The use of language is often the downfall of students who inadvertently refer to the sample rather than the population, but students can also pose their question directly about the sample rather than the population, missing the point about posing questions about the population. An example of this is: “I wonder if the boys sampled in the 2007 NZ CensusAtSchool have a longer arm span than the girls sampled?” (2008 student, post-test)

27. Criteria Considerations for teaching and curriculum development The population of interest is clear (continued) Students must identify both population groups. Students often identify one of the population groups and then revert to an abbreviated form for the second group, and as a result do not clearly identify the second population group; for example, referring to the first group as year 10 New Zealand boys and then girls for the second group. Watch the use of age groups as a possible population group that is represented by the data when the data is from a single year level. The issue here is that it might be a biased sample as only the oldest, or the youngest, of a particular age group would be represented in a single year group. For example, in year 10 most students are 14 years of age but there are also 13 year olds and 15 year olds. The 13 year olds tend to be students in their second half of their 14th year and the 15 year olds tend to be students in the first half of their 16th year.

28. Criteria Considerations for teaching and curriculum development The intent is clear From the question it needs to be clear if it is a summary, comparison or relationship question. The intent of the question is critical as it indicates the relevant analysis that will be undertaken for the variable(s). “Is the question clear and unambiguous?” (Wild & Pfannkuch, 1999).

29. Criteria Considerations for teaching and curriculum development The question can be answered with the data The problematic situation first arose around this criterion (see chapter 5). The question is not able to be answered as the population selected for summary/relationship questions or the population groups for comparison questions result in the sample available being too small for any sensible analysis to be made. This can also be an issue when a sample is given and then students sample from this sample, resulting in too small a sample. The question needs to be specific, so that it is answerable from data – questions that are too vague and general are harder to answer (Graham, 2006).

30. Criteria Considerations for teaching and curriculum development The question is one that is worth investigating, that it is interesting, that there is a purpose The information obtained by answering the question will be useful to someone, i.e. there is a purpose for the investigation. The question should be personally interesting to you – not only will this bring greater motivation, but also common-sense knowledge about the context should help to ensure that the investigation proceeds along sensible lines (Ben-Zvi & Amir, 2005; Burgess, 2007; Graham, 2006; Hancock et al., 1992; Ridgway et al., 2005). If neither of these points is relevant, then one would need to ask: Why we are investigating this situation?

31. Criteria Considerations for teaching and curriculum development The question allows for analysis to be made of the whole group The question is referring to the aggregate picture rather than an individual case (Bakker, 2004a). For example, questions that ask about who is the tallest or who has the longest arm span are not suitable. Figure 6-3. Criteria for posing investigative questions (Arnold, 2013, p. 111)

32. From Albert Einstein "The important thing is not to stop questioning. Curiosity has its own reason for existing.”

33. That’s all folks