1. John Whitmer Updated: 1-25-2013 Research Findings Logging on for Higher Achievement Research

2. 1. CHICO STATE LEARNER ANALYTICS RESEARCH STUDY “Logging on to Improve Achievement” by John Whitmer EdD. Dissertation (UC Davis & Sonoma State)

3. Case Study: Intro to Religious Studies Redesigned to hybrid delivery through Academy eLearning Enrollment: 373 students (54% increase on largest section) Highest LMS (Vista) usage entire campus Fall 2010 (>250k hits) Bimodal outcomes: 10% increased SLO mastery 7% & 11% increase in DWF Why? Can’t tell with aggregated reporting data 54 F’s

4. Driving Conceptual Questions 1.How is student LMS use related to academic achievement in a single course section? 2.How does that finding compare to the relationship of achievement with traditional student characteristic variables? 3.How are these relationships different for “at-risk” students (URM & Pell-eligible)? 4.What data sources, variables and methods are most useful to answer these questions?

5. Variables

6. Clear Trend: Grade w/Mean LMS Hits

7. Scatterplot: Grade w/Mean LMS Hits

8. GenderFreq.Percent University AverageDifference Female23162%51%11% Male14238%48%-10% Age 0% 17226% 18-2130281% 22-30226% 31+10% Under-represented Minority No26471%73%-2% Yes10929%27%2% Pell-eligibleFreq.Percent No21056% Yes16344% First Attend CollegeFreq. No26872% Yes10528% Enrollment StatusFreq. Continuing Student21758% Transfer175% First-Time Student13937%

9. Correlation: LMS Use w/Final Grade Scatterplot of Assessment Activity Hits vs. Course Grade Statistically Significant (strong to weak)r% VarianceSign. Total Hits0.4823%0.0000 Assessment activity hits 0.4722%0.0000 Content activity hits 0.4117%0.0000 Engagement activity hits 0.4016%0.0000 Administrative activity hits 0.3512%0.0000 Mean value all significant variables18%

10. Correlation: Student Char. w/Final Grade Scatterplot of HS GPA vs. Course Grade

11. Separate Variables: Correlation LMS Use & Student Characteristic with Final Grade LMS Use Variables 18% Average (r = 0.35–0.48) Explanation of change in final grade Student Characteristic Variables 4% Average (r = -0.11–0.31) Explanation of change in final grade >

12. Combined Variables: Regression Final Grade by LMS Use & Student Characteristic Variables LMS Use Variables 25% (r 2 =0.25) Explanation of change in final grade Student Characteristic Variables +10% (r 2 =0.35) Explanation of change in final grade >

13. Smallest LMS Use Variable (Administrative Activities) r = 0.35 Largest Student Characteristic (HS GPA) r = 0.31 >

14. Regression r 2 Results Comparison

15. At-Risk Students: “Over-Working Gap” 15

16. Filtering Data – Lots of “Noise”; Low “Signal” Slides: http://goo.gl/DmT8z Final data set: 72,000 records (-73%)

17. Feedback? Questions? John Whitmer (jwhitmer@calstate.edu)jwhitmer@calstate.edu