Mastering’s Powerful Diagnostics: Analyze Student Performance and Use Results to Improve Learning with Scott Hildreth

Mastering's Powerful Diagnostics: Analyze Student Performance & Use Results to Improve Learning! Scott Hildreth - Chabot College

Mastering's Powerful Diagnostics: …and detect cheating… Scott Hildreth - Chabot College

4 A bit about me… Teaching Physics & Astronomy since 1989 Teaching Physics & Astronomy since 1989 Chabot College, CSU East Bay, UC Berkeley Chabot College, CSU East Bay, UC Berkeley Astro sections ~ 50 students (3 hours/week) Astro sections ~ 50 students (3 hours/week) Physics sections ~ 30 – 45 students (7 hours/week) Physics sections ~ 30 – 45 students (7 hours/week) Scott Hildreth - Chabot College - April 2015

5 Ways I use Mastering… In Physics In Physics Pre-lecture “intro” assignments ~ 1 hr/week Pre-lecture “intro” assignments ~ 1 hr/week Traditional end-of-chapter HW ~ 3 hrs/week Traditional end-of-chapter HW ~ 3 hrs/week Group Work in class ~ 1 hr/week Group Work in class ~ 1 hr/week Exam Review ~ 5 hrs/week before exams Exam Review ~ 5 hrs/week before exams Remediation Remediation Scott Hildreth - Chabot College - April 2015

6 Ways I use Mastering… In Astronomy In Astronomy Weekly Reading Quizzes ~ 30 min/week Weekly Reading Quizzes ~ 30 min/week Weekly Homework ~ 1 hr/week Weekly Homework ~ 1 hr/week Animated Tutorials Animated Tutorials Reading Questions Reading Questions Ranking Tasks Ranking Tasks Exam Review ~ 1 hr/week before exams Exam Review ~ 1 hr/week before exams Scott Hildreth - Chabot College - April 2015

7 An “Aha” moment Scott Hildreth - Chabot College - April 2015 Another wonderful Gary Larson Cartoon went here! (But it is copyrighted, and can’t be distributed over the web.)

8 Scott Hildreth - Chabot College - Febrarury 2013 Mastering Captures Data Cannon, Annie Chandrasekhar Einstein, Albert Feynman, Richard Hawking, Stephen Herschel, Carol Klumpke-Roberts, Lightman, Alan Perlmutter, Saul Rubin, Vera Quickly see who is working

9 Scott Hildreth - Chabot College - Febrarury 2013 Mastering Captures Data Quickly what they miss & why

10 Mastering Captures Data Quickly see what they miss & why Scott Hildreth - Chabot College - April 2015

11 Types of Diagnostic Data Class-wide (scores, times, difficulty) Class-wide (scores, times, difficulty) Scott Hildreth - Chabot College - April 2015

12 Types of Diagnostic Data Class-wide trends Scott Hildreth - Chabot College - April 2015 Class-wide trends Click here…

13 Types of Diagnostic Data Assignment specific (bar charts & histograms of scores, times, difficulty) Assignment specific (bar charts & histograms of scores, times, difficulty) Scott Hildreth - Chabot College - April 2015

14 Types of Diagnostic Data Scott Hildreth - Chabot College - April 2015 Click here… Student Assignment Specific Assignment Specific

15 Types of Diagnostic Data Student specific Student specific Scott Hildreth - Chabot College - April 2015

16 Types of Diagnostic Data Scott Hildreth - Chabot College - April 2015 Item specific Click here…

17 Scott Hildreth - Chabot College - April 2015 Comparitive Data & Errors

18 What about “gaming the system” Student Compare Time & Score by Students Scott Hildreth - Chabot College - April 2015

19 Scott Hildreth - Chabot College – April 2015 What about “gaming the system” Compare Time & Score by Students

20 Scott Hildreth - Chabot College - April 2015 What can you do with the data? Use Diagnostic Data from HW Use Diagnostic Data from HW Change Lectures Change Lectures “Tune” delivery to key problems “Tune” delivery to key problems Focus on Common Errors Focus on Common Errors Change Class Format Change Class Format More peer learning with group work More peer learning with group work Correlated Group Work to Retention Correlated Group Work to Retention

21 Scott Hildreth - Chabot College - Febrarury 2013 Another AHA moment… Measuring and Correlating Data Another wonderful Gary Larson Cartoon went here! (But it is copyrighted, and can’t be distributed over the web.)

22 Scott Hildreth - Chabot College – April 2015 Mastering Enables RAPID Data Analysis

23 Scott Hildreth - Chabot College - Febrarury 2013 Mastering Enables RAPID Data Analysis

24 Scott Hildreth - Chabot College - Febrarury 2013 Retrieving and Using Mastering Data Download Grade Book to Excel Download Grade Book to Excel

25 Retrieving and Using Mastering Data Create total columns for assignments Create total columns for assignments Graph totals against one another Graph totals against one another Now I spend my time looking at data and coming up with new ways to try things… Scott Hildreth - Chabot College - April 2015

26 Scott Hildreth - Chabot College - Febrarury 2013 Fall 2012 Astro Data from Excel

27 Scott Hildreth - Chabot College - Febrarury 2013 Does HW improve Quiz Scores?

28 Does Homework Success Correlate to Exam Success? Fall 2014 Physics (Calculus-based mechanics_)

29 Does Homework Success Correlate to Exam Success? Scott Hildreth - Chabot College - April 2015

30 Does Homework Success Correlate to Exam Success? Scott Hildreth - Chabot College - April 2015

31 Homework Quizzes DO Correlate better with Exams Scott Hildreth - Chabot College - April 2015

32 Homework Quizzes DO Correlate better with Exams Scott Hildreth - Chabot College – April 2015 Spring st Midterm Physics 4A

33 Homework Quizzes DO Correlate better with Exams Scott Hildreth - Chabot College – April 2015 Spring nd Midterm Physics 4A

34 Homework Quizzes DO help identify students gaming the system Scott Hildreth - Chabot College – April 2015 Good scores on HW, Terrible on midterm

35 Homework Quizzes DO help identify students gaming the system Scott Hildreth - Chabot College – April 2015 Good scores on HW, Terrible on midterm Actually didn’t do the HW!

36 Ways I Use Mastering Now… Calculus-based Physics Homework Homework Introductory “prelecture” assignments for each chapter Introductory “prelecture” assignments for each chapter Weekly homework assignments Weekly homework assignments HW Quiz in class to ensure students working authentically HW Quiz in class to ensure students working authentically In-class collaborative group work (teams) In-class collaborative group work (teams) Lecture – doing problems together Lecture – doing problems together Exam Review Exam Review In-class competition (teams) In-class competition (teams) Outside of class preparation Outside of class preparation Extra Credit & Remedial Help Extra Credit & Remedial Help Scott Hildreth - Chabot College - April 2015

37 Ways I Use Mastering Now… Introductory Astronomy Homework Homework Weekly homework assignments with ranking tasks Weekly homework assignments with ranking tasks Reading quizzes Reading quizzes Links to additional reading, videos, and resources Links to additional reading, videos, and resources Lecture – doing online simulations together Lecture – doing online simulations together Exam Review Exam Review Outside of class preparation Outside of class preparation Extra Credit & Remedial Help Extra Credit & Remedial Help Scott Hildreth - Chabot College - April 2015

38 A quick caveat… You don’t have to do all of these at once! Try one new thing a term…but measure it!! Scott Hildreth - Chabot College - April 2015

39 Long Term Benefit: Data! Scott Hildreth - Chabot College - April 2015

40 Scott Hildreth - Chabot College - Febrarury 2013 Mastering Enables Data Analysis: Learning Outcomes

41 Personal Goals Track student usage easily Track student usage easily Take earlier, positive action Take earlier, positive action Students obviously struggling Students obviously struggling Those working hard, and… Those working hard, and… Those hardly working. Those hardly working. Scott Hildreth - Chabot College - April 2015

42 Current Data Analysis Projects Identify most-missed problems with diagnostic data Identify most-missed problems with diagnostic data Add messages, hints, YouTube to suggest and target student thinking Add messages, hints, YouTube to suggest and target student thinking “online office hour” nudges… “online office hour” nudges… See if students do better See if students do better Scott Hildreth - Chabot College - April 2015

43 Questions? Discussion? Check out the resource list and current research Check out the resource list and current research Login to my classes to look around Login to my classes to look around me with questions! me with questions! Images reprinted in the online presentation and available online were borrowed with permission from " .this.book", The Cartoon Bank, Alfred A. Knopf, NY, “The Far Side” comics by Gary Larson cannot be distributed online. Images reprinted in the online presentation and available online were borrowed with permission from " .this.book", The Cartoon Bank, Alfred A. Knopf, NY, “The Far Side” comics by Gary Larson cannot be distributed online. Scott Hildreth - Chabot College - April 2015