Jenn Shinaberger Lee Shinaberger Corey Lee Coastal Carolina University eLearnReady: A Free Assessment Tool for Determining Student Online Readiness Jenn Shinaberger Lee Shinaberger Corey Lee Coastal Carolina University Research funded in part by Coastal Carolina University Assessment Grant Program

Background Support Distance Learning (DL) initiatives at Coastal Carolina University Improve learning resources for online learners Design and build a sophisticated online assessment/advising system Prepare students to take an online class by determining their level of readiness and providing customized feedback

Desired Outcomes Improve student satisfaction and retention in online courses. Improve student readiness to take an online course. Identify factors associated with student success and satisfaction in online courses. Provide recommendations for faculty development and student-support resources.

www.eLearnReady.com

Factors Contributing to Student Success in Distance Learning Accessibility of course info and students’ varying abilities to process it (Sprague, Maddux, Ferdig, & Albon 2007) Comfort level with technology and ability to be self-directed (Pérez Cereijo, 2006) Adopt efficient learning strategies Develop a study schedule Interaction with classmates and instructors

Dimensions Measured Self-Motivation Self-Management Feedback Interaction Reading: Visual Text Reading: Visual Graphics Listening Technology Course Management System Factors of Student Persistence

Timeline Fall 2012 – Pre-Pilot Test Summer I – Pilot Test October 2013 – Site Launch Fall 2013 – Pilot Test Completed June 2014 – Results Published at ASCUE

Pre-Pilot Test Pre-pilot test conducted in Fall 2012 Measured the functionality and validity of the of the instrument 128 students participated Adjustments were to the instrument based upon a factor analysis

Pilot Test: Participants - Instructors Distance learning courses at CCU were identified for Summer I, Summer II, Fall 2013 Instructors were invited to participate via email and follow-up phone calls and emails Instructors emailed students or posted links 24 instructors (out of 50 total) Summer I 2013 13 instructors (out of 46 total) Summer II 2013 18 instructors (out of 69 total) Fall 2013

Pilot Test: Participants - Students 248 unique graduate and undergraduate students 174 females and 74 males Age of Respondents Under 18 1 18-25 167 26-35 45 36-45 18 46-55 13 56-65 4

Procedure and Questionnaire Students were asked to complete an online survey questionnaire 40 questions in the survey Forced completion

Customized Student Report Students enter their email address to receive a customized report Report provides strategies and tips Report links to resources

Customized Student Report

Student Report

Results and Analysis Correlation Analysis Multiple questions measure each dimension Correlation analysis used to test validity One question discarded (five questions remained that measured the chosen dimension) No remaining validity concerns. Low p value is significant, probability that the null hypothesis (no difference) is true,

Results and Analysis Factor Analysis Are the dimensions distinct? Concern about multicollinearity: n = 248 Anecdotally, require n>300 (or 400) Determinant < 1x10-6, multicollinearity is a concern Remove questions with exceptionally high correlation to other questions Eight additional questions removed, 6 factors found KMO = 0.78 (sampling adequacy is “good”) and Barlett’s Test of Sphericity = 0.000 (<0.001) indicated the test is appropriate

Results and Analysis Factors Identified: Technology and Course Management System Need for Interaction and Preference for Listening Self-Motivation Self-Management Reading (Visual Text and Visual Graphics) Need for Feedback

Performance and Satisfaction Survey

Performance and Satisfaction Survey Students were asked to take the Performance and Satisfaction Survey at the end of each semester Respondents from the initial survey were emailed directly 20 questions, forced response 22 students participated

Results and Analysis Confound: Low response rate to Satisfaction survey Satisfaction survey respondents have significantly higher Self-Motivation scores than the eReadiness survey respondents (p = 0.021) 20 of 22 Satisfaction respondents anticipate receiving an A (remaining grades: 1 B, 1 B+) Low p value is significant, probability that the null hypothesis (no difference) is true,

Results and Analysis Regression analysis performed to determine if the eReadiness scores could be used to predict Satisfaction results. No significant relationship found between eReadiness results and Satisfaction responses except: Q3: The support materials used in this course were helpful. Q17: I prefer this online course to traditional face-to-face classroom instruction.

Results and Analysis Significant Correlation with Self-Motivation Q3: The support materials used in this course were helpful. Significant Correlation with Self-Motivation p-Value = 0.0003 (significant relationship) Correlation Coefficient = 0.70 (strong positive correlation)   Interpretation: Students that are Self-Motivated are more likely to find the support materials are helpful.

Results and Analysis Q17: I prefer this online course to traditional face-to-face classroom instruction. Significant Correlation with Self-Motivation, Listening, and Need for Interaction p-Value = 0.0001 (significant relationship) Correlation Coefficient = 0.82 (strong correlation) Self-Motivation: Positive correlation Listening: Positive correlation Interaction: Negative correlation

Results and Analysis Q17: I prefer this online course to traditional face-to-face classroom instruction. Interpretation: Self Motivation: higher self-motivation score, higher preference for online courses Listening: higher listening score, higher preference for online courses Interaction: lower need for interaction, higher preference for online courses

Discussion Predictive factors of student satisfaction High self motivation Lower need for interaction Learning preference: listening Are these the best predictors of online success? Almost all students reported they would receive an “A” on the Performance and Satisfaction Survey

Next Steps Revise Performance and Satisfaction Survey Compare individual eLearnReady results to student performance http://www.eLearnReady.com

www.eLearnReady.com

Questions Questions

References Pérez Cereijo, M. (2006). Attitude as Predictor of Success in Online Training. International Journal on E-Learning, 5(4), 623-639. Sprague, D., Maddux, C., Ferdig, R., & Albon, P. (2007). Online Education: Issues and Research Questions. Journal of Technology and Teacher Education, 15(2), 157-166.

Contact Information Jenn Shinaberger Coastal Carolina University Assistant Director of Distance Learning and CeTEAL jshinabe@coastal.edu Lee Shinaberger Coastal Carolina University Lecturer E. Craig Wall Sr. College of Business Administration lshinabe@coastal.edu Corey Lee Coastal Carolina University Assistant Professor Spadoni College of Education clee@coastal.edu http://www.eLearnReady.com