1. Authentic Discovery Projects in Statistics GCTM Conference October 16, 2009 Dianna Spence NGCSU Math/CS Dept, Dahlonega, GA

2. NSF Grant Project Overview Grant Title: “Authentic, Career-Specific Discovery Learning Projects in Introductory Statistics” Project Goals: Increase students’...  knowledge & comprehension of statistics  perceived usefulness of statistics  self-beliefs about ability to use and understand statistics Tasks:  Develop Instruments  Develop Research Constructs and Projects  Develop Materials and Train Instructors  Measure Effectiveness

3. Instructional Model: Discovery Learning in Statistics Authentic Research Projects  Experiencing the Scientific Method  Discovering Statistical Methods in Context Design of Research Question Definition of Variables Demographic Data Representative Sampling Issues Data Collection Appropriate Analyses of Data Interpretation of Analyses

4. Project Format Linear regression  Variables student selects often survey based constructs  Survey design  Sampling  Regression analysis t-tests  Variables may use data previously collected  Designs Independent samples Dependent samples  Hypotheses

5. Interdisciplinary Team Disciplines Represented  Biology/Ecology  Criminal Justice  Psychology  Sociology Tasks of Team Members  Identify authentic research constructs  Define instrument/measurement of construct  Suggest simple statistical research projects  Nursing  Physical Therapy  Education  Business

6. Online Resource: Instructional Materials Homepage Instructor Guide  Project overview Timelines Implementation tips Best practices  Handouts for different project phases  Evaluation rubrics  Links to student resources http://radar.ngcsu.edu/~rsinn/nsf/

7. Online Resource: Instructional Materials Homepage Student Guide  Overall Project Guide Help for each project phase  Technology Guide  Variables and Constructs http://radar.ngcsu.edu/~rsinn/nsf/

8. Common Questions How long will the projects take? How do I fit this in with the content I am supposed to teach? How should I go about organizing and facilitating these projects? What are the student “deliverables”? How do I assess the students’ work? How much should this count in determining student grades?

9. Project Phases Form Teams Generate Research Ideas Develop Constructs and Variables Develop Surveys or Other Instruments Project Proposals Data Collection Data Analysis Project Report Team Presentations

10. Finding Time for the Project Make the projects the vehicle through which students learn course content Selecting projects to use as class examples  current students’ projects (work in progress)  example project(s) you have made up  former students’ projects (when you have them) Leverage sample projects  Gives students an idea what should be included in their project  Helps students connect course content and put it in context

11. Aligning Course Content with Project Phases PhaseClass Topic Define variablesIndependent/dependent variables Develop surveysTypes of bias Data collectionSampling methods Data analysisAppropriate statistical analyses Example: Exploring slope of regression line “One team examined the relationship between number of vegetable servings consumed in a week and number of hours spent exercising in a week. Their regression equation was ___?___. What is the slope and what does it mean?”

12. Considerations and Options: Forming Teams Team Size Assigned Members vs. “Pick Your Own” Grouping by Common Interests Giving Team Members Specific Roles

13. Considerations and Options: Project Proposals Formal vs. Informal IRB or Similar Entity Other Permissions Required? Require instructor approval before data collection begins!

14. Considerations and Options: Data Collection Dialog About Sampling Strategies  Random  Stratified  Convenience Dialog About Representative Samples Assist Students with Organization and Data Entry

15. Considerations and Options: Data Analysis Linear Regression Projects  Scatterplot  Correlation Coefficient r  Regression Line  Regression Equation  Slope  R 2 and Explanatory Value of Model

16. Considerations and Options: Data Analysis t-Test Comparison Projects  Appropriate Design Two Independent Samples Dependent Samples/Matched Pairs  Appropriate Hypotheses One-tailed (and which tail) Two-tailed  Significance Level  Interpretation  Implications of Type I & II Errors

17. Considerations and Options: Project Report Content Requirements  Outlines  Templates  Sample Reports  Scoring Rubrics (Students can use as checklist) Other Requirements  Writing Standards  Submission of Technology Files  Reflections

18. Considerations and Options: Team Presentations Presentation Guidelines  Content and Scope  Aesthetics  Pace and Organization  Time Limit & Enforcement Audience Accountability  Evaluations  1-2 Sentence Recap of Points

19. Assessment Rubrics  Advantages Consistency Manageability Communicate expectations  Encompass All Project Components Grade milestones along the way  Explicit vs. Holistic  Resources for Rubrics Use one of ours Customize your own

20. Assessment Team Member Grades  Accountability of Individual Members Shared Team Grade Individual Contribution Other “Tricks” Weight of Projects

21. Exploratory Study Fall 2007 Instrument Validation and Concept “Trial Run” Based on 10 sections of Introductory Stats 4 experimental sections  Used authentic discovery projects  n=113 participants out of 128 students 88% participation rate 6 control sections  Did not use authentic discovery projects  n = 164 participants out of 192 students 85% participation rate

22. Exploratory Results: Content Knowledge Instrument  21 multiple choice items  KR-20 analysis: score = 0.63 Results  control mean: 8.87; experimental mean = 10.82  experimental mean 9 percentage points higher  experimental group significantly higher (p <.0001)  effect size = 0.59 Instrument shortened to 18 items for full study

23. Exploratory Results: Perceived Usefulness of Statistics Instrument  12-item Likert style survey; 6-point scale  5 items reverse scored  score is average (1 – 6) of all items  Cronbach alpha = 0.93 Results  control mean: 4.24; experimental mean = 4.51  experimental group significantly higher (p <.01)  effect size = 0.295 Instrument unchanged for full study

24. Exploratory Results: Statistics Self-Beliefs Beliefs in ability to use and understand statistics Instrument  15-item Likert style survey; 6-point scale  score is average (1 – 6) of all items  Cronbach alpha = 0.95 Results  control mean: 4.70; experimental mean = 4.82  difference not significant (1-tailed p =.1045)  effect size = 0.15 Instrument unchanged for full study

25. Full Study: Pilot of Developed Materials 3 institutions  1 university (6 undergraduate sections)  1 2-year college (2 sections)  1 high school (3 sections) 5 instructors Quasi-Experimental Design  Spring 2008: Begin instructor “control” groups  Fall 08 - Fall 09: “Experimental” groups

26. Pilot Results Varied by Instructor Overall results given here Instrument  Perceived Usefulness Pretest: 50.42 Posttest: 51.40 Significance: p = 0.208  Self-Beliefs for Statistics Pretest: 59.64 Posttest: 62.57 Significance: p = 0.032**  Content Knowledge Pretest: 6.78 Posttest:7.21 Significance: p = 0.088*

27. Attitudes and Beliefs Statistics Self-Beliefs  Self-beliefs improved significantly overall Significant Gains –for regression techniques ( p = 0.035 ) –for general statistical tasks ( p = 0.018 ) Little or No Improvement –for t-test techniques ( p = 0.308 ) Perceived Utility for Statistics  Student perceptions of the usefulness of statistics improved slightly but not significantly  No sub-scales on this instrument Overall Perceived Utility ( p = 0.208 )

28. Performance Gains Concept Knowledge: 3 Components  Regression Techniques Moderately Significant ( p = 0.086 )  T-test Usage Moderately Significant ( p = 0.097 )  T-test Inference No gain

29. For more information Project Website  http://radar.ngcsu.edu/~djspence/nsf/ http://radar.ngcsu.edu/~djspence/nsf/ Instructional Materials Home  http://radar.ngcsu.edu/~rsinn/nsf/ http://radar.ngcsu.edu/~rsinn/nsf/