John Whitmer Updated: Research Findings Logging on for Higher Achievement Research

1. CHICO STATE LEARNER ANALYTICS RESEARCH STUDY “Logging on to Improve Achievement” by John Whitmer EdD. Dissertation (UC Davis & Sonoma State)

Case Study: Intro to Religious Studies Redesigned to hybrid delivery through Academy eLearning Enrollment: 373 students (54% increase on largest section) Highest LMS (Vista) usage entire campus Fall 2010 (>250k hits) Bimodal outcomes: 10% increased SLO mastery 7% & 11% increase in DWF Why? Can’t tell with aggregated reporting data 54 F’s

Driving Conceptual Questions 1.How is student LMS use related to academic achievement in a single course section? 2.How does that finding compare to the relationship of achievement with traditional student characteristic variables? 3.How are these relationships different for “at-risk” students (URM & Pell-eligible)? 4.What data sources, variables and methods are most useful to answer these questions?

Variables

Clear Trend: Grade w/Mean LMS Hits

Scatterplot: Grade w/Mean LMS Hits

GenderFreq.Percent University AverageDifference Female23162%51%11% Male14238%48%-10% Age 0% 17226% % % 31+10% Under-represented Minority No26471%73%-2% Yes10929%27%2% Pell-eligibleFreq.Percent No21056% Yes16344% First Attend CollegeFreq. No26872% Yes10528% Enrollment StatusFreq. Continuing Student21758% Transfer175% First-Time Student13937%

Correlation: LMS Use w/Final Grade Scatterplot of Assessment Activity Hits vs. Course Grade Statistically Significant (strong to weak)r% VarianceSign. Total Hits0.4823% Assessment activity hits % Content activity hits % Engagement activity hits % Administrative activity hits % Mean value all significant variables18%

Correlation: Student Char. w/Final Grade Scatterplot of HS GPA vs. Course Grade

Separate Variables: Correlation LMS Use & Student Characteristic with Final Grade LMS Use Variables 18% Average (r = 0.35–0.48) Explanation of change in final grade Student Characteristic Variables 4% Average (r = -0.11–0.31) Explanation of change in final grade >

Combined Variables: Regression Final Grade by LMS Use & Student Characteristic Variables LMS Use Variables 25% (r 2 =0.25) Explanation of change in final grade Student Characteristic Variables +10% (r 2 =0.35) Explanation of change in final grade >

Smallest LMS Use Variable (Administrative Activities) r = 0.35 Largest Student Characteristic (HS GPA) r = 0.31 >

Regression r 2 Results Comparison

At-Risk Students: “Over-Working Gap” 15

Filtering Data – Lots of “Noise”; Low “Signal” Slides: Final data set: 72,000 records (-73%)

Feedback? Questions? John Whitmer