Tracking Pathways to Success Identifying Learning Success Factors Across Course Delivery Formats Peter Usinger, State Assessment Meeting 2013

Student Success in College Environment High W/F As high as 50% attrition of FTIC cohort 45% of Withdrawal Reasons are Course Related Student Attributes Financially Limited Non- traditional Academically Underprepared Study Context Reviewing High F/W Engagement in College Prep F2F vs. Online Instruction Nursing Orientation SLS Course Support

Research Questions Q1 Delivery format based differences between self- directed behaviors and learning strategies. Q2 Similarities and Differences across subject domains/disciplines and their sensitivity to course delivery formats. Q3 Relationships among MSLQ constructs, and with student characteristics, course characteristics, and delivery formats. Q4 “Predictive” model to explain the relationships for developing support mechanisms and change curricular design for greater success/completion

(Online) Course Completion Student Characteristics Demographic variables Previous e-learning experience Self-Directed Learning Habits Metacognitive skills Motivation Self-discipline Autonomy Self-regulated behaviors

Sample drawn from… Mixed withdrawal/failure rates 136 Courses Humanities, English/Letters, Mathematics and Statistics, Sciences, Social Sciences, Nursing; Hybrids and Workforce Dev. excluded from analysis due to low frequencies. Delivery Online (216 Sections) Face to Face (931 Sections) >20,000 Students Population Invited via to complete MSLQ online (15 min.) 1,179 Course sections (total)

Sample by Academic Area 2,200 Participants:11% Response Rate DisciplineF2FOnline Humanities 4329 Letters Math Nursing Sciences Social Sciences 8564

MSLQ Scales and Reliability Motivated Strategies for Learning Questionnaire (MSLQ) Original MSLQApplied in Study Itemsα α A. Motivational Constructs 1. Intrinsic Goal Orientation Extrinsic Goal Orientation Task Value Control of Learning Beliefs Self-Efficacy for Learning & Performance Test Anxiety B. Learning Strategies Constructs 1. Rehearsal Elaboration Organization Critical Thinking Metacognitive Self-Regulation Time/Study Environmental Management Effort Regulation Peer Learning Help Seeking Total Items in Questionnaire8161

Motivational Correlations Pearson Correlation Coefficients (N=2,200) …between GradesPassing Online + …and Motivational Constructs 1. Intrinsic Goal Orientation 0.19***0.16*** Extrinsic Goal Orientation *** Task Value 0.24*** 0.19*** 0.04* 4. Control of Learning Beliefs 0.30*** 0.25*** 0.10*** 5. Self-Efficacy for Learning & Performance 0.46*** 0.37*** 0.07** 6. Test Anxiety -0.23***-0.15***-0.01 Note 1: *p <.05. **p <.01. ***p <.001. Note 2: + Method: F2F=0, Online=1.

Learning Strategies Pearson Correlation Coefficients (N=2,200) …between GradesPassingOnline …and Learning Strategies Constructs 1. Rehearsal ** 2. Elaboration 0.09***0.07** Organization * 4. Critical Thinking * 5. Metacognitive Self-Regulation * Time/Study Environmental Management 0.19*** 0.13*** 0.07** 7. Effort Regulation 0.27*** 0.21*** 0.06* 8. Peer Learning *** 9. Help Seeking *** (where *p <.05. **p <.01. ***p <.001.)

Due to the very minor statistical differences in the results for grades vs. course success, and the fact that a significantly higher proportion of C and D students across most discipline areas are subsequently failing, we’ve created a Student Performance aggregate variable that combines student’s Course Performance into three levels: Level 0: Failure or Withdrawal from Course Level 1: Course Grade Equals C or D Level 2: Course Grade Equals A or B This Performance variable is used in the student success analysis following. Data Aggregation Rules

Comparison by Delivery Type MSLQ Correlation w/ Course Performance OnlineFace to Face MeanCorrelationMeanCorrelation Intrinsic Goal Or *** *** Extrinsic Goal Or * Task Value *** *** Control Beliefs *** *** Self-Efficacy *** *** Test Anxiety *** *** Rehearsal Elaboration * * Organization Critical Thinking Metacog. S-Reg Time/Study Mgmt *** *** Effort Regulation *** *** Peer Learning Help Seeking N=3661,751

Comparison by Course Level MSLQ Correlation w/ Course Performance DevelopmentalCollege Level MeanR Coeff.MeanR Coeff. Intrinsic Goal Or *** *** Extrinsic Goal Or * Task Value *** *** Control Beliefs *** *** Self-Efficacy *** *** Test Anxiety *** *** Rehearsal Elaboration *** Organization Critical Thinking Metacog. S-Reg * Time/Study Mgmt *** *** Effort Regulation *** *** Peer Learning Help Seeking N=551962

Important Note by Author: Please note that the following slide has been corrected since it contained a transfer error due to a truncated spreadsheet field. Affected is one of the interesting findings with regard to the role of test anxiety in the course performance of Nursing students. The original presentation slides showed a positive correlation between Test Anxiety and Course Performance for the whole Nursing group in this sample. However, this is only true for the 2 nd year cohort of the RN program involved. Pre-admission, 1 st year, and BSN students show patterns similar to other academic areas. This is an extremely valuable research result since this correlation also goes along with a positive correlation between Test Anxiety and Rehearsal Strategies and Peer Learning. This indicates that our 2 nd year Nursing students were not negatively overwhelmed with anxiety, but had learned to offset their anxiety levels successfully by rehearsing the subject matter at hand, and working with their peers to master the course content. We will follow-up on these findings with an appropriate discipline-specific pathway model in the next phase of this longitudinal research project.

Correlations by Academic Area Pearson Correlation with Course Performance Humanities (N=75) Letters (N=416) Math (N=1043) Nursing (N=174) Sciences (N=276) Soc. Sci. (N=156) Motivational Constructs 1. Intrinsic Goal Orientation Extrinsic Goal Orientation Task Value Control of Learning Beliefs Self-Efficacy Learn. & Perf Test Anxiety Learning Strategies 1. Rehearsal Elaboration Organization Critical Thinking Metacognitive Self-Reg Time/Study Management Effort Regulation Peer Learning Help Seeking

Success Comparison in Math Example: Math Differences Failure/WithdrawalPassing GradeDifference Motivational Constructs Intrinsic Goal Orientation 58.3%66.8%8.5% Extrinsic Goal Orientation 79.6%84.0%4.5% Task Value 48.1%59.7%11.6% Control Beliefs 63.4%77.9%14.5% Self-Efficacy 50.0%73.8%23.7% Test Anxiety 67.6%55.6%-11.9% Learning Strategies Constructs Rehearsal 65.1%67.1%2.0% Elaboration 65.9%68.3%2.4% Organization 67.6%70.1%2.5% Critical Thinking 50.0%48.9%-1.1% Meta-Cognitive Self-Regulation 61.5%64.8%3.3% Time/Study Management 67.1%73.4%6.3% Effort Regulation 70.4%81.2%10.7% Peer Learning 36.0%37.3%1.4% Help Seeking 53.7%56.1%2.5%

Multivariate Regression I Multivariate Regression Analyses (Total Sample) Predicting Course Performance F-ValuePr > F R-Square Fit of Model 42.1< Parameter Estimates t-ValuePr > |t|St-B Intrinsic Goal Orientation Extrinsic Goal Orientation Task Value Control of Learning Beliefs Self-Efficacy for Learning 14.84< Test Anxiety Rehearsal Elaboration Organization Critical Thinking Metacognitive Self-Regulation Time/Study Management Effort Regulation Peer Learning Help Seeking

Multivariate Regression II Multivariate Regression Analyses w/ Performance Online Courses Face-to-Face Classes F-ValuePr > F R-Square F-ValuePr > F R-Square Fit of Model 7.1< < Parameter Estimates t-ValuePr > |t|St-Bt-ValuePr > |t|St-B Intrinsic Goal Orientation Extrinsic Goal Orientation Task Value Control of Learning Beliefs Self-Efficacy for Learning 5.36< < Test Anxiety Rehearsal Elaboration Organization Critical Thinking Metacognitive Self-Regulation Time/Study Management Effort Regulation Peer Learning Help Seeking

Multivariate Regression III Multivariate Regression Analyses w/ Performance DevelopmentalCollege Level F-ValuePr > F R-Square F-ValuePr > F R-Square Fit of Model 14.0< < Parameter Estimates t-ValuePr > |t|St-Bt-ValuePr > |t|St-B Intrinsic Goal Orientation Extrinsic Goal Orientation Task Value Control of Learning Beliefs Self-Efficacy for Learning 7.31 < < Test Anxiety Rehearsal Elaboration Organization Critical Thinking Metacognitive Self-Regulation Time/Study Management Effort Regulation Peer Learning Help Seeking

MSLQ scales: excellent reliability & context sensitivity Motivational factors clearly driving student success Learning Strategies (cognitive skills) show multi-faceted instructional improvement (learning) opportunities Peer Learning/Help Seeking: Underrepresented within cohorts, not an integrated/facilitated college activity Many underprepared students succeed largely via intrinsic motivation and confidence/self-efficacy, combined with hard work and study management The lack of exposure to meta-cognitive abilities and critical thinking skills nurtures course environments in support of compliant learners instead of self-directed learners! Conclusions

Study Impact / Next Steps Online: Assess Quality Matters impact across delivery formats and feed information back into process 1 st Year/EWS: Compare demographic differences associated with the various success patterns SLS: Evaluate impact with SLS courses when applying the MSLQ as formative assessment tool early in Student Success or Developmental Ed classes Overall: Conduct additional focus groups at college sites; seek expansion of study to inform interventions FCS: Seek study replication across service areas

Thank You!