VSS Pre-Conference on Research & Evaluation, 11/14/2009 Evolving Evaluation of a State Virtual School Tom Clark, PhD TA Consulting

Consulting Evaluation Team John Watson & Jennifer Ryan Evergreen Consulting Cathy Cavanaugh Drs. Cavanaugh/UF Meredith DiPietro UNC-Charlotte Tom Clark TA Consulting

Consulting MoVIP in Year 2 MoVIP Management Approach DESE oversight Separate instructional management vendors, K-5 & 6-12 in Y1/Y2 Bocavox SIS used across K-12 Year 2 Enrollments 15,810 enrollments in semester classes 8,478 - Fall ,332 – Spring 2009

Consulting MoVIP in F-TP-TTOTAL K-5: : TOTAL Unique Finally Enrolled Students

Consulting National SVS Enrollments, Source: Keep Pace with Online Learning (in press)

Consulting MoVIP Evaluation Plan 1. Are Courses and Teaching High Quality? 2. Are Equal Access and Enhanced Options Being Provided? 3. Are Infrastructure and Support Services Effective? 4. Is Student and Program Success Being Ensured?

Consulting Course & Teaching Quality Survey ItemParents 6-12 Facili- tators MoVIP offers high quality courses Spring 2009 Fall 2008 Spring % 90% 97% 100%

Consulting Course & Teaching Quality Survey ItemK-5 Students 6-12 Students Courses Challenging Spring 2009 Spring % 86% 83% 88% Courses too Difficult Spring 2009 Spring % 12% 34% 47%

Consulting Course & Teaching Quality Survey Item6-12 Students Facili- tators A Good Teacher Spring 2009 Spring % 91% 89% 94% Teacher Responds Spring 2009 Spring % 82% 84% 78% Teacher Helps Spring 2009 Spring % 78% 88% 85%

Consulting Equal Access and Options 40+ presentations statewide by MoVIP Director in Y2 Outreach to rural districts in Y2 School of Choice designation in Y2 High-enrolling urban districts (KC, STL) Minority and low income students Elementary division more diverse Good FRL representation in secondary 236 classes available; 89% increase

Consulting Meeting Curricular Needs Survey Item6-12 Facilitators MoVIP Benefits Students Spring 2009 Spring % 93% MoVIP Benefits District Spring 2009 Spring % 88%

Consulting Technology & Support Survey Item 6-12 Stu- dents 6-12 Parents 6-12 Facili- tators Computer prob- lems don’t keep students from learning Spring 2009 Spring % 62% 76%65% 60%

Consulting Student & Program Success Completion rates improving Higher completions by grade level, other factors State End of Course Tests English II –higher than state average Algebra I – NSD Biology –lower State MAP Tests Statistical comparisons yielded some promising results NCLB subgroup means often higher than state

Consulting Student & Program Success Grade/ Subject WhiteIEPFRLAll Students 4 th Gr. CA NSD Sign. Higher NSD 4 th Gr. Math NSD Sign. Higher NSD NSD

Consulting Student & Program Success The wide range of data available allowed analysis of success factors. Some early results: Time spent in the LMS influenced learning outcomes positively in 11 out of 15 high enrollment courses Lack of economic disadvantage Influenced outcomes in 5 courses Full-time status Influenced learning in 5 courses

Consulting Recommendations Continue to review internal policies Address legislative policy issues that may limit ability to meet mission Build a positive school culture Build a leading PD system Share course review process

Consulting Recommendations Improve rapport with facilitators Work with home study families Address special education concerns Address common tech problems Monitor key learning indicators

Consulting Thank You! Tom Clark, PhD TA Consulting Mobile: (217) On behalf of TA Consulting and Associates