1 An Application of the Action Research Model for Assessment Preliminary Report JSM, San Francisco August 5, 2003 Tracy Goodson-Espy, University of AL, Huntsville M. Leigh Lunsford, University of AL, Huntsville Ginger Holmes Rowell, Middle TN State University

2 Overview Action Research Model In the Context of a Collaborative Project Showing Results from a Specific Example

3 Action Research Model* What is the problem? I.e., what is not working in the classroom? What technique can be used to address the learning problem? What type of evidence can be gathered to show whether the implementation is effective? What should be done next, based on what was learned? * R. delMas, J. Garfield, B. Chance

4 Teaching Experiment Cycle Teaching Hypotheses; Curricular & Instructional Choices Class Implementation & Feedback Instructors’ Reflections and Curricular Modifications

5 Action Research Model What is the problem? I.e., what is not working in the classroom? What technique can be used to address the learning problem? What type of evidence can be gathered to show whether the implementation is effective? What should be done next, based on what was learned?

6 What is the Problem? Math majors who take only one course in “Probability and Statistics” are exposed to very little statistics Student understanding of complicated concepts –CLT, CI’s, Combinatorics, Baye’s Theorem,... Increase reasoning and thinking instead of memorization Better preparation for careers

7 Action Research Model What is the problem? I.e., what is not working in the classroom? What technique can be used to address the learning problem? What type of evidence can be gathered to show whether the implementation is effective? What should be done next, based on what was learned?

8 What Technique can be used to Address the Learning Problem? NSF DUE A/I Collaborative Research Award –Adaptation & Implementation of Activity & Web- Based Materials in Post-Calculus Introductory Probability & Statistics Courses –PI’s Tracy Goodson-Espy, University of AL, Huntsville M. Leigh Lunsford, University of AL, Huntsville Ginger Holmes Rowell, Middle Tennessee State University Big Picture *This project is partially support by the National Science Foundation. The project started in June 2002 and continues through August

9 A Collaborative Approach Athens State Univ. M. Leigh Lunsford Middle Tenn. St. Univ. Ginger Holmes Rowell Univ. of Alabama, Huntsville Tracy Goodson-Espy A&I Materials into Post Calculus Prob/Stat Courses Provide Objective Independent Assessment of A&I

10 The Materials for A&I “A Data-Oriented, Active Learning, Post- Calculus Introduction to Statistical Concepts Methods, and Theory (SCMT)”“A Data-Oriented, Active Learning, Post- Calculus Introduction to Statistical Concepts Methods, and Theory (SCMT)” A. Rossman, B. Chance, K. Ballman NSF DUE “Virtual Laboratories in Probability and Statistics (VLPS)”“Virtual Laboratories in Probability and Statistics (VLPS)” K. Siegrist NSF DUE

11 What Technique can be used to Address the Learning Problem? Increase statistical content –Change course description & course number Understand complicated concepts and increase “thinking” instead of memorization –Change course materials used Better preparation for careers –Integrate technology, group work, report writing Specific Techniques

12 Action Research Model What is the problem? I.e., what is not working in the classroom? What technique can be used to address the learning problem? What type of evidence can be gathered to show whether the implementation is effective? What should be done next, based on what was learned?

13 What Evidence can be Gathered to Show Implementation is Effective? Student’s self perception of learning of concept (survey results) Teacher perception of student learning –In-class student feedback during the activity (continuous monitoring) –Follow-up in-class quiz –Student reports –Test/Exam questions Student attitude survey results

14 Evidence Continued: One Example – Central Limit Theorem “Sampling Distributions of Sample Means” Computer Laboratory Simulation Activity –A. Rossman, B. Chance, K. Ballman “A Data-Oriented, Active Learning, Post-Calculus Introduction to Statistical Concepts Methods, and Theory (SCMT)”“A Data-Oriented, Active Learning, Post-Calculus Introduction to Statistical Concepts Methods, and Theory (SCMT)” –Spring 2003: used 3 of 4 examples from this activity, students wrote reports, question on test

15 Evidence (continued) CLT Example Students’ self perception of learning –80% (n=25) of students remembered it as an “activity that aided learning” (Fall 02 – instructor lecture & 1 SCMT example: 23%) –“My understanding of the Central Limit Theorem” Survey question of their self reported knowledge on a scale of 1 (low knowledge) to 5 (high knowledge) Mean response = 4.6, stdev =.64, n = 27 (Fall 02: mean = 3.8, stdev =.90, n = 25)

16 Evidence (continued) CLT Example Teacher’s perception of student learning In-class monitoring –Overall minimal difficulty with computer lab –Needed feedback on their understanding of the concept Gave in-class (no credit) unannounced quiz at beginning of following class Not enough students had gotten far enough on the activity to provide conclusive evidence

17 Evidence (continued) CLT Example Teacher’s perception of student learning Reports –Structure (could work in groups) Introduction, Explanation of the 3 examples in the lab activity, Explain method used, Conclusions –Results (graded for big picture concepts and conceptual details) Three “wrong” responses (n=27) 1) “I did not know how to do that [example 4].” 2)Mean gets “smaller.” 3)An incomplete conclusion.

18 Evidence (continued) CLT Example Teacher’s perception of student learning Exam questions –Students completed Example 3 (uniform lunch times) from the Course Pack activity on the test Not previously assigned No indication that it would be on the test The “Uniform” subcommand was new This portion of the test was taken in the lab Worked individually

19 Test Question Output Variable N Mean StDev Sample means n= Sample means n= Sample means n=

20 Evidence (continued) CLT Example Teacher’s perception of student learning Results –Gave “hint” (minus 1 point) with Uniform command Some found Uniform in the pull down menu Biggest mistake – 1/3 of students used Uniform instead of Uniform 0 60 –Filled in a table like they had done previously –Wrote a “paragraph” summarizing their findings. Mean score 2.6 out of 3 points (st dev = 0.7)

21 Action Research Model What is the problem? I.e., what is not working in the classroom? What technique can be used to address the learning problem? What type of evidence can be gathered to show whether the implementation is effective? What should be done next, based on what was learned?

22 What should be done next, based on what was learned? Improve my writing assignment Improve my evaluation of student learning on this concept –Pre/post test –Use delMas/Garfield/Chance instrument (?) –Coordinate with Lunsford on this unit Incorporate more report writing in my class Keep using and evaluating these activity- based, discovery learning materials.

23 Resources delMas, R., Garfield, J., and Chance B. (1999), A Model of Classroom Research in Action: Developing Simulation Activities to Improve Students' Statistical Reasoning, Journal of Statistics Education v7, n3, Hollins, E. R. (1999), “Becoming a Reflective Practitioner,” in Pathways to Success in School: Culturally Responsive Teaching, eds. ER Hollins and EI Oiver, Mahwah, NJ: Lawrence Erlbaum Associates. Hopkins, D. (1993), A Teacher’s Guide to Classroom Research, Buckingham: Open University Press. Noffke, S., and Stevenson, R. (eds.) (1995, Educational Action Research, NY: Teachers College Press.

24 Contact Information