1. Robert H. Carver Stonehill College/Brandeis University June 12, 2007
Students’ Ambiguity Tolerance as a Success Factor in Learning to Reason Statistically
Robert H. Carver
Stonehill College/Brandeis University
June 12, 2007

2. Quick Outline Genesis of this Research Current project
Classroom experience
Literature review
JSM 2006 presentation
Current project
Invitation to participate
Q&A

3. Genesis of the Research
Some observations from the classroom…
Learning statistics is difficult in many ways
Intro Stats can activate profound emotional responses
“but usually I like/I dislike math classes…”
Stat Ed literature
Focus on variation as a central theme
Studies on activities, techniques, topics
Relatively little work on variation among learners

4. Learners Vary! Variation among learners Prior coursework
Level of effort—motivation, capacity, etc.
Aptitude
Attitudinal orientation (Schau, et al.)
Myers-Briggs (BTI)
Other personality/emotional characteristics

5. Ambiguity Tolerance
Frenkel-Brunswik, Else (1948)
Ambiguity Tolerance Construct:
Some are stimulated by ambiguity, some are threatened
Personality trait vs. preferred process
Stable personality attribute vs. context-dependent
Relationship to rigidity, uncertainty tolerance, openness

6. The inner conflict Low ambiguity tolerance
Per Frenkel-Brunswick:
Low ambiguity tolerance
 conflict & anxiety in ambiguous situations
 rigid adherence to preconceived ideas
 failure to process contrary evidence

7. Statistical Thinking
Statistical thinking requires simultaneous consideration of variation within one sample and among possible samples.
Statistical methods provide a means of making decisions in inherently ambiguous situations, relying on incomplete information.
Inference requires a leap of faith—a ready embrace of ambiguity

8. Contrast with ’Ambiguity’ in Decision Theory
Ambiguity as a property of the situation or state of knowledge
Ambiguity as property or proclivity of the thinker

9. Ambiguity Tolerance Measurement Scales Budner,1962
Rydell; Rydell & Rosen 1966
MacDonald, 1970
Norton, 1975
McLain, 1993

10. Questions
Do students with high AT have an advantage in learning to think statistically?
Do students with low AT tend to “shut down” when presented with instruction in inferential reasoning and techniques? OR
Do students with low AT welcome statistical thinking as a way to cope with ambiguity?

11. Methods
Sample:
85 undergraduates enrolled in 4 sections over 2 semesters
Differences among sections
Technology: Minitab vs. SAS
Normal, Learning Community, Honors
Informed consent
Credit & incentives
Course-embedded data collection

12. Methods Dependent variable:
Score on Comprehensive Assessment of Outcomes for a first course in Statistics (CAOS) post-test
Developed by Web ARTIST Project (U.Minnesota and Cal Poly) team
Pre- and Post-test
40 items

13. Purpose of CAOS test
The CAOS test was designed to provide an instrument that would assess students’ statistical reasoning after any first course in statistics. Rather than focus on computation and procedures, the CAOS test focuses on statistical literacy and conceptual understanding, with a focus on reasoning about variability.
ARTIST project, University of Minnesota

14. CAOS post-test
Illustrative question:
Researchers surveyed 1,000 randomly selected adults in the US. A statistically significant, strong positive correlation was found between income level and the number of containers of recycling they typically collect in a week. Please select the best interpretation of this result.

15. CAOS post-test
We cannot conclude whether earning more money causes more recycling among US adults because this type of design does not allow us to infer causation.
This sample is too small to draw any conclusions about the relationship between income level and amount of recycling for adults in the US
This result indicates that earning more money influences people to recycle more than people who earn less money.

16. CAOS post-test
We cannot conclude whether earning more money causes more recycling among US adults because this type of design does not allow us to infer causation.
This sample is too small to draw any conclusions about the relationship between income level and amount of recycling for adults in the US
This result indicates that earning more money influences people to recycle more than people who earn less money.

17. CAOS post-test
A study examined the length of a certain species of fish from one lake. The plan was to take a random sample of 100 fish and examine the results. Numerical summaries on lengths of the fish measured in this study are given.
Mean
26.8mm
Median
29.4 mm
Std. Dev.
5.0 mm
Minimum
12.0 mm
Maximum
33.4 mm

18. CAOS post-test 26.8mm Mean Median 29.4 mm Std. Dev. 5.0 mm Minimum
Maximum
33.4 mm

19. CAOS post-test 26.8mm Mean Median 29.4 mm Std. Dev. 5.0 mm Minimum
Maximum
33.4 mm

20. CAOS post-test

21. CAOS post-test
Improvement

22. Measuring AT McLain’s 22 question instrument 7-point Likert Scales
Independent Measures & variables:
Abiguity Tolerance:
McLain’s 22 question instrument 7-point Likert Scales
Max score for extreme tolerance = 74
Min score for extreme intolerance = - 58
Reliability: Cronbach’s alpha = 0.897

23. Measuring AT Selected items:
I don’t tolerate ambiguous situations well.
I’m drawn to situations which can be interpreted in more than one way.
I enjoy tackling problems which are complex enough to be ambiguous.
I find it hard to make a choice when the outcome is uncertain.

24. Covariates Other explanatory factors and controls tested:
Score on CAOS Pre-test
Section controls
Cohort (55% 2006; 45% 2007)
Gender dummy (49% female; 51% male)
First-year student dummy (61% 1st year)
Math SAT
Prior Stat Education (37% had some)
Course cumulative average
Attendance

25. Findings: CAOS Pre-test
Variable
Coeff
Signif
Constant
9.07
0.438
Female dummy
-1.13
0.638
AT scale
0.048
0.537
First year dummy
-5.581
0.028
Prior course dummy
5.256
0.032
Math SAT score
0.063
0.001 F
4.89
Adj R2
21.3%
A.T. did not have a significant main effect on Pre-test scores

26. Findings:CAOS Post-Test
Variable
Coeff
Signif
Constant
33.374
0.000
CAOS Pre-test score
0.559
AT scale
0.110
0.079
First Year dummy
-3.726
0.072
Prior course dummy
-3.406
0.099 F
12.29
Adj R2
37.0%
AT score has a significant (p < 0.10) effect on Post-Test reasoning score

27. Findings:CAOS Post-Test
Variable
Coeff
Signif
Constant
-2.529
0.751
CAOS Pre-test score
0.437
0.000
AT scale
0.117
0.039
Course Cumulative Avg
0.473
Prior course dummy
-3.946
0.035 F
19.46
Adj R2
48.9%
AT score has a significant (p < 0.05) effect on Post-Test reasoning score

28. Summary of Key Findings
AT non-significant in predicting pre-test scores
Suggests that the pre-test does not measure ambiguity tolerance
Significant findings re: prior coursework, academic preparation (though not much explanatory power), Math SAT

29. Summary of Key Findings
AT is significant in predicting Post-Test scores
Also significant
Pre-Test score
Prior statistics coursework (but negative)
First year dummy
Course results
Not significant
Gender, cohort, section, MathSAT

30. Discussion Main Findings: Caveats:
Ambiguity Tolerance may have a positive main effect
Low A.T. likely to be surmountable
Caveats:
CAOS scales measure several aspects of statistical thinking
Small sample
Substantial unexplained variance
Measurement issues: effort, engagement

31. Discussion Implications:
An individual’s orientation toward ambiguity can affect his/her success with statistical reasoning.
Tolerance of ambiguity construct may provide a motivation for success
Course pedagogy may address A.T. directly
Note: Course averages not explained by AT

32. Discussion/Invitation
Research directions:
Can these results be replicated, especially in larger samples?
Would the results hold up with different measures of statistical reasoning?
Do other personality or personal style variables shape success in statistical reasoning?
How can we structure pedagogy to address personality variation among learners?
Does A.T. affect application of statistical reasoning in practice?

33. Q&A/ Discussion Join me! rcarver@stonehill.edu rcarver@brandeis.edu