Exemplary Practices and Your Evaluation Report Migrant Education Program (MEP) Webinar March 31, :00 pm – 4:00 pm EDT OME Mission To provide excellent leadership, technical assistance, and financial support to improve the educational opportunities and academic success of migrant children, youth, agricultural workers, fishers, and their families.

Webinar Logistics Please “mute” your phone when listening to the webinar. Please “un-mute” your phone when it is your time to share information with your colleagues. OME will monitor the “chat box” and respond to your questions at an appropriate time.

Objectives Participants will: Share exemplary evaluation practices, and Gain a deeper understanding of the MEP Evaluation Checklist requirements, so that… Participants may use the Evaluation Checklist and Toolkit to guide future, successful, MEP evaluations.

Today’s Agenda We will discuss MEP Evaluation requirements and share exemplary practices of: 1.Activities Before the Evaluation 2.The Evaluation Plan 3.The Written Evaluation Report 4.Additional Expectations

STATUS OF MEP EVALUATIONS 24 State MEPs: Currently Producing Evaluations for Review 13 Evaluations Under OME Review 6 Evaluations Are Under Revision by States 5 Evaluations Accepted

ACTIVITIES BEFORE THE EVALUATION

1. Activities Before the Evaluation Collection of Performance Indicator Data Collection of MPO Data

Collection of Performance Indicator Data: Georgia How do I collect data on? Percent proficient in Reading/LA and Mathematics, in grades 3-8 and HS Graduation rate Disaggregated by PFS, other migrant, and non-migrant

Collection of Performance Indicator Data: Georgia 3 rd GradeTotal TestedDNMMeetsExceedsTotal Percentage Meeting/ Exceeding Migrant % Non-Migrant126,03614,51970,64949, %

Collection of Performance Indicator Data: Georgia 3 rd Grade Reading 3 rd Grade Migrant3 rd Grade Non-MigrantGap 89.1%92.4%3.4%

Collection of Performance Indicator Data: Georgia, EOC High School 9 th Grade Literature and Composition Total TestedDNMMeetsExceedsTotal Percentage Meeting/ Exceeding Migrant % Non-Migrant133,83916,92861,84255, %

Collection of Performance Indicator Data: Georgia, 9 th Grade Literature 9 th Grade Migrant3 rd Grade Non-MigrantGap 67.3%87.4%20.1%

Collection of Performance Indicator Data: Georgia, 3 rd -8 th Grade E/LA 3 rd GradeTotal TestedDNMMeetsExceedsTotal Percentage Meeting/ Exceeding PFS % Non-PFS %

Collection of Performance Indicator Data: Georgia, Graduation Rate The 2013 graduation rate for all students in the state of Georgia was percent. Migrant 50 percent The 2014 graduation rate for all students in the state of Georgia was percent. Migrant 57 percent Non-Migrant 73 percent Migrant PFS 54 percent Migrant Non-PFS 58 percent While this two year trend shows an increased percentage of migrant students graduating high school, the exact impact of the MEP on the graduation rate is undetermined. However, the gap between migrant (PFS and Non-PFS) and non-migrant is significant and indicates the need for a continued focus on completing high school. This is important for the Georgia MEP as well as other sending and receiving states in the U.S.

Collection of Performance Indicators ResultsLarge N Assessed Per Grade Level (N>100) Median N Assessed Per Grade Level (N=72) Small N Assessed Per Grade Level (<30) Performance Indicator: Achievement Every Grade LevelGrade Span Bands (3-5, 6-8, HS) One Performance Indicator Per Content Area Performance Indicator: Graduation Yes – Provide Rate Provide Number of Graduates PI: Disaggregate by PFS and Other Migrant Yes No

Collection of Performance Indicators: Questions Please use the chat box if you would like to ask questions or share information in regard to performance indicator-related activities that occur before the evaluation.

Collection of MPO Data Goal: Proficiency in Reading/LA, Measured by Statewide Assessments The MEP will serve 75% of all Migrant age 3-5 children in preschool programs. 60% of all K-3 Migrant students will achieve phonics mastery in “x” assessment. 90% of all K-3 Migrant children will receive targeted Reading/LA interventions. 25% of all K-3 Migrant ELL children will attain English proficiency in “y” assessment.

Collection of MPO Data: Oregon How do I collect data on? Measurable Program Outcomes (MPOs) MPOs disaggregated by PFS

Collection of MPO Data: Oregon Oregon MPOs: 80% of summer school students show gains on reading assessment 80% of summer school students show gains on mathematics assessment 80% of summer school pre-school children will demonstrate an increase in developmental skills

Collection of MPO Data: Oregon Example MPO 1b: By the end of the school year, 80% of migrant students participating in summer reading programs at least half of the time will show gains between pre/post on a State-accepted reading assessment. As evidence for the above MPO, please see Appendix A for summer growth for in Reading at the summer schools in Oregon. The average in 2012 for reading was 77% which fell below our 80% goal. Many of the areas provided a yearlong assessment for the pre and the post. Students that had just left the regular year did fairly well on the pre assessment, whereas, a month later, did not do as well. One of the lessons learned during my monitoring visit that first year of measurement was to remind the summer schools to only include in the assessment what was taught during the summer school. In 2013, the average reading rate was 79% which was higher than the previous year, but still 1% below our stated goal of 80%. Last year in 2014, we achieved 86% growth in reading in our summer schools. This surpassed our goal of 80%. Our plan is to continue to improve every year in our instruction and expectations during summer school.

Collection of MPO Data: Oregon Summer School Appendix A, Reading Summer School Growth for Summers of Year  2012 Reading Growth 2013 Reading Growth 2014 Reading Growth School District # that Grew Out of% # that Grew Out of% # that Grew Out of% # that Grew Out of% # that Grew Out of% Beaverton SD %688085%151979%568070%719972%

Collection of MPOs ResultsLarge N Assessed Per Grade Level (N>100) Median N Assessed Per Grade Level (N=72) Small N Assessed Per Grade Level (<30) MPOEvery Grade LevelGrade Span Bands (3-5, 6-8, HS) MPOs Per Content Area and Other MPOs MPO: Disaggregate by PFS and Other Migrant Yes No

Collection of MPOs: Questions Please use the chat box if you would like to ask questions or share information in regard to MPO-related activities that occur before the evaluation.

THE EVALUATION PLAN

2. The Evaluation Plan Contained in the SDP, and specifies: How the State will collect data related to implementation results How the State will collect data related to performance results

The Evaluation Plan: New Jersey Service Delivery Plan Contained in the SDP, and specifies: How it will measure implementation results How it will measure performance results

The Evaluation Plan: New Jersey Service Delivery Plan – Reporting Implementation and Outcome Data “The evaluation will report both implementation and outcome data to determine the extent to which the measurable outcomes for the MEP in reading and math achievement, school readiness, high school graduation, and OSY achievement have been addressed and met.”

The Evaluation Plan: New Jersey Implementation Results Evaluation questions answered by implementation data include the examples below.  Were local projects implemented as described in their approved MEP applications? If yes, what worked and why? If not, what didn’t work and why not?  What challenges were encountered by the MEP? What was done to overcome these challenges?  What adjustments can be made to the MEP to improve instruction, professional development, and the involvement of migrant parents?  To what extent were the procedures used for identification and recruitment of eligible migrant students found to yield reliable results?  To what extent were MEP staff better prepared to help migrant students close the achievement gap?  To what extent did migrant parents report being involved with their children’s learning in reading and math and school readiness skills?

The Evaluation Plan: New Jersey Performance Results Evaluation questions answered by performance results data include the examples below.  To what extent did migrant students demonstrate proficiency on the NJ ASK and HSPA in Reading and Mathematics?  To what extent did 3-4 year old PK migrant children improve school readiness in migrant-funded preschool programs?  To what extent did secondary migrant students earn sufficient credits to remain on track for graduation?  To what extent did migrant OSY improve career-related and/or English language skills?

The Evaluation Plan: Questions Please use the chat box if you would like to ask questions or share information in regard to the evaluation plan.

THE WRITTEN EVALUATION REPORT

3. The Written Evaluation Report Documents the Evaluation Plan in a Written Report Performance Results Performance Indicator Results MPO Results Implementation Results Implications and Recommendations

Performance Indicators Performance Indicators disaggregated at tested grade levels (3-8, HS) by: PFS Other Migrant Non-Migrant

Measurable Program Outcomes Measurable Program Outcomes disaggregated by: PFS Other Migrant

Implementation Results The written evaluation report provides specific implementation results for fidelity to services and activities during: Regular Year (Instructional and Support Services) Summer (Instructional and Support Services)

Implications and Recommendations The written evaluation report provides implications and recommendations for improvement based upon: Implementation results Performance results …and done every two to three years.

Implications and Recommendations: Indiana Recommendations Increase the reporting on the results of services: Six of the 18 sub- grantees submitted data in the first year. Following the establishment of the regional model in , all six regions and the two local programs submitted data regarding progress on the MPOs. Standardize measurement of the impact of summer/fall programs in reading and math: During Indiana implemented the myON reader in all summer programs. The online platform includes assessments which establish baseline for and measure growth in student Lexile levels. This provided a basis for comparing implementation of reading programs across regions. Several sites implemented Math MATTERS in the summer programs, and all sites were able to provide data about student gains during summer programs.

The Written Evaluation Report: Questions Please use the chat box if you would like to ask questions or share information in regard to the written evaluation report.

ADDITIONAL EXPECTATIONS

4. Additional Expectations Conducts an annual performance results evaluation, in order to inform SEA decision- making. Upon results of the full evaluation, the State describes specific changes to the SDP and services, based upon the evaluation of performance and implementation results.

Additional Expectations Performance Results and Implementation Results Comprehensive Needs Assessment Updated Service Delivery Plan that Incorporates Results and Student Needs (update provided internally, working with LOAs)

THANK YOU! The Office of Migrant Education Edward Preeti Choudhary