Discovery Learning Projects in Introductory Statistics PI: Dianna Spence Co-PI: Brad Bailey Overview NSF DUE-1021584 Objective: Develop materials to help instructors combine real-world, discovery-learning projects in introductory statistics courses; measure the impact on student performance and beliefs. Design of Pilot Test: Eight participating instructors (nationwide) test the methods and materials developed to facilitate discovery projects in their classes. CONTROL GROUPS: Each instructor teaches one or more sections without using the methods and materials designed to facilitate projects. (Completed AY 2011-2012) TREATMENT GROUPS: During subsequent academic terms, each instructor teaches one or more sections using the methods and materials developed. (Completed AY 2012-2014) Curriculum Development Materials (Online/Print) Student Guide Instructor Guide Technology Guide Contents include Scoring rubrics Proposal forms Guidance at each phase Condensed and extended project timelines Resources for developing variables/constructs Student Project Description Students generate their own research questions, define their own variables, and draft a research proposal. Upon approval, students collect data; organize, analyze, and interpret results, and report findings in a formal paper and an oral presentation. Below are project types and examples. Linear Regression – Examples: NBA Player Salaries and Points per Game Car Engine Horsepower and Average Miles per Gallon t-Tests – Examples: Comparing Males and Females: Attitudes about Tattoos Comparing Northern and Southern States: Divorce Rates Instruments: Student Outcomes and Subscales Content Knowledge (CK) – 17 multiple choice items Statistics Self-Efficacy (SE) – 16 Likert scale items Perceived Usefulness of Statistics (PU) – 11 Likert scale items. Subscales of Student Outcome Measures Mixed Methods Research Qualitative Inquiry Five instructors participated in qualitative data collection. Quantitative Analyses Control vs treatment: compare student outcomes For individual instructors For overall groups All 8 instructors (N = 353 Control, 441 Treatment) Preferred protocol only: 5 instructors (N = 198 Control, 344 Treatment) Instructors omitted for protocol exceptions: Treatment section taught in summer mini-mester format Target student population substantially different in treatment group Treatment section was different course than control Repeated treatment sections after initial trial: 3 instructors (N = 122 Control; N = 122 / 103 / 43 in 1st, 2nd, 3rd Treatments) Multivariate models: predict student outcomes by treatment, instructor variables, other factors (esp. for instructors with repeated treatment sections) Results (Qualitative) Each theme is explored with respect to codes (left). Participants are compared, with commonalities, differences, and specific manifestations noted. Sample Observations Nature of interaction between instructor and students is changed when projects are incorporated (Engagement/communication) Student misconceptions are easier to identify earlier in learning cycle when carrying out projects (Conceptual learning/project purpose) Students appear more invested in the outcome of studies they have designed themselves (Real-world application/student dispositions) Instructors vary widely in how they choose to guide students conceptually through projects (Conceptual learning/Instructor pedagogy) Results (Quantitative) Control vs. Treatment – Overall Group Gains: All Instructors (Overall group declines: none) Control vs. Treatment – Overall Gains: Instructors within Protocol Significant Gains and Declines: Individual Instructors (*Protocol exceptions; omitted from table directly above) The Experience Factor: Instructors Repeating Treatment Sections Guiding Questions How do instructors vary in their implementation of the discovery statistics projects? How do the instructors’ teaching philosophies pervade the project implementation? What are the implications of the observed instructor differences for student learning? Data Collection Written prompts Course syllabi/materials Phone interviews Classroom observations Analysis Code and Triangulate (team process) Identify and describe themes Synthesize with quantitative findings Scale Group N Mean (% of max) p Content Knowledge – Subscale: Identifying Analysis C T 353 441 1.33 (44.3%) 1.54 (51.3%) .001 Self-Efficacy Main Scale 78.51 (81.8%) 80.20 (83.5%) .021 Self-Efficacy – Subscale: Hypothesis Testing 23.41 (78.0%) 24.43 (81.4%) .002 Self-Efficacy – Subscale: Data Collection 19.12 (79.7%) 19.98 (83.3%) .000 Scale Group N Mean (% of max) p Content Knowledge Main Scale C T 198 344 7.52 (44.2%) 8.63 (50.7%) .000 Content Knowledge – Subscale: Linear Regression 2.50 (35.7%) 3.12 (44.6%) Content Knowledge – Subscale: Identifying Analysis 1.23 (41.0%) 1.61 (53.7%) Self-Efficacy Main Scale 76.00 (79.2%) 80.13 (83.5%) Self-Efficacy – Subscale: Hypothesis Testing 22.08 (73.6%) 24.38 (81.3%) Self-Efficacy – Subscale: Data Collection 18.86 (78.6%) 20.04 (83.5%) Primary Coding Categories Emerging Themes Conceptual Framework Project Purpose Instructor Pedagogy Student Dispositions Communication Engagement Real-world Application Conceptual Learning Scale #1 #2 #3 #4 #5* #6* #7 #8* CK – Content Knowledge (Main)  CK Subscale: Linear Regression  CK Subscale: Hypothesis Testing CK Subscale: Identifying Analysis CK Subscale: Sampling SE – Self Efficacy (Main) SE Subscale: Linear Regression SE Subscale: Hypothesis Testing SE Subscale: Data Collection SE Subscale: General Scale / Subscale #1 #2 #7 Group Content Knowledge  CK: Linear Regression CK: Hypothesis Testing CK: Identifying Analysis CK: Sampling Self Efficacy SE: Linear Regression SE: Hypothesis Testing SE: Data Collection SE: General Contact Us PI Dr. Dianna Spence djspence@ung.edu Department of Mathematics University of North Georgia Co-PI Dr. Brad Bailey Dahlonega, GA 30597 bbailey@ung.edu Phone: 706-864-1805 Project Website http://faculty.ung.edu/DJSpence/NSF/ Content Knowledge Statistics Self-Efficacy Perceived Utility Linear Regression Hypothesis Testing Identifying Appropriate Type of Analysis Sampling Data Collection General (Learning and Understanding Statistics) Real-World Relevance Personal Benefit to Understanding N = 122 N = 122 N = 103 N = 43 Experience with Treatment: Significant Gains