1. 1 2009 MEPA Reporting Teleconference September 2009

2. 2 Presenters Kit Viator, Director of Student Assessment Dan Wiener, Administrator of Inclusive Assessment Bob Lee, Chief Analyst

3. 3 Today’s Agenda Welcome MEPA Overview and Updates 2009 MEPA Reporting: What’s New and Why? New Performance Levels and Reporting Scale Standard Setting: Setting the Cut Score What the Scores Mean –Making Reclassification Decisions –Planning Instruction Important New Resources Questions and Answers

4. 4 MEPA Overview The state’s English language proficiency assessment of LEP students in grades K-12, based on English Language Proficiency Benchmarks and Outcomes –MEPA-R/W and MELA-O Fall testing: Test only those LEP students in grades 1-12 who did not participate in spring 2009 (no fall testing for Kindergarten) Spring testing: Test all students in K-12 reported as “LEP” in October 2009 or March 2010 SIMS Sub-scaled scores in four tested areas –Reading: 0-30 points –Writing: 0-30 points –Listening: 0-5 points –Speaking: 0-20 points NOTE: Sub-scaled scores  Overall scaled score

5. 5 Fall 2009 MEPA-R/W Tests Grade SpanWhich Test & Sessions to Administer 1-2 (K-2 in spring 2010) Level A (one session for Reading and one for Writing) OR Level B (one session for Reading and one for Writing) 3-12 (grade span tests: 3-4, 5-6, 7-8, 9-12) Sessions 1 and 2 OR Sessions 2 and 3

6. 6 MELA-O (Listening and Speaking) QMT/QMA Retraining and Requalification By January 31, 2010, all QMT/QMAs must be requalified in order to continue to report MELA-O scores for MEPA Purpose: To improve consistency and reliability of student ratings Fall 2009 QMT Training schedule –Retrain current QMTs: October 13, Westborough –Train new QMT (if no QMT in district): September 21-22, Westborough November 3-4, Norwood

7. 7 Spring 2009 MEPA: What’s New? Redesigned tests for grade spans 3-4, 5-6, 7-8, and 9-12 New K-2 tests New test questions based primarily on authentic reading passages Wide range of difficulty of test questions, and more constructed responses Locator survey (K-2)/locator tests (3-12) New reporting scale: 400-550 New performance levels: Levels 1-5

8. 8 New Tests for Students in K-2 Student takes either Level A or B, depending on –Level of English proficiency and literacy –Results of K-2 Locator Survey What’s tested on K-2 MEPA-R/W: –Level A: (R) alphabet recognition; match high-frequency words with pictures; answer questions based on story read aloud; (W) write or complete a dictated word; write the word for a picture; write a two-sentence story –Level B: (R) match or write a word/sentence that matches picture; questions based on story read by student; (W) correct punctuation; letter writing to complete sentence; write a story based on 3 pictures

9. 9 MEPA Reporting: 2005-2008 Beginning 300 Early Inter- mediate Inter- mediate Transitioning 400 MEPA Performance Levels

10. 10 Why We Needed New MEPA Performance Levels 395 Many students begin to demonstrate proficiency on MCAS ELA tests 300 305 400 2005-2008 MEPA Beginning Early Inter- mediate Inter- mediate Transitioning

11. 11 MEPA 2005-2008: Distribution of Scaled Scores Students at low Beginning and high Transitioning levels were clustered into lower and upper ends of scale. 300320340360380400 1,000 2,000 3,000 4,000 5,000 # STUDENTS SCALED SCORE 0

12. 12 MEPA Reporting: 2009 MEPA Performance Levels Level 1 Level 2 Level 3 Level 4 Level 5 400 550

13. 13 New MEPA Performance Levels A student at: Level 1 cannot yet communicate in English Level 2 communicates using simple written and spoken English, with frequent errors Level 3 communicates using basic English with occasional errors Level 4 is generally fluent, with few errors Level 5 communicates effectively in English across all academic subjects

14. 14 Standard Setting: Summer 2009 Performance levels describe performance at five levels Panels of ELL and general educators spent 3 days last summer with ESE determining where one performance level ended and another began (“cut score”) Recommendations of panelists for Levels 1-4 were adopted. Level 5 was based on panelists’ recommendations plus other evidence, including –Percent of LEP students Proficient on MCAS ELA –Statewide reclassification rate …and was fixed at 500-550 on each test.

15. 15 Reporting Scale Expanded to 400-550 MEPA Scaled-Score Cut Points Grade Span Level 1Level 2Level 3Level 4Level 5 K-2400453466485500 3-4400432452474500 5-6400436456479500 7-8400443464486500 9-12400450464489500

16. 16 2009 MEPA Grade K-12 Overall Scaled Score Distribution 400 # STUDENTS SCALED SCORE 550

17. 17 Sample Parent/Guardian Report: Grade 7 Very similar in appearance to 2005-2008 P/G Report, showing: –student’s score –location within the performance level –likely range of scores if student took the test multiple times Level 1 Level 5 Level 2 Level 3 Level 4

18. 18 Sample Parent/Guardian Report: Grade 7 (Continued…) Relative strengths and weaknesses in four tested areas

19. 19 Score compared with other students who have been in Massachusetts for 1 to 5+ years –About 45% of students who have been in Massachusetts for 3 years were in Level 3 on this MEPA test, as was this student. Sample Parent/Guardian Report: Grade 7 (Continued…)

20. 20 Using MEPA Results to Make Reclassification Decisions Level 1 and Level 2: Not a candidate for reclassification Level 3: Not a likely candidate for reclassification Level 4: May be a candidate for reclassification, if other indicators are aligned* Level 5: Reclassification recommended, if other indicators are aligned* * MCAS scores, grades, local assessments, observations

21. 21 Available Now Guidance on Using MEPA Results to Plan Sheltered English Immersion (SEI) Instruction and Make Reclassification Decisions for Limited English Proficient (LEP) Students (2009) Guide to Interpreting the 2009 MEPA Reports for Schools and Districts (Both available at: www.doe.mass.edu) School “Drop Box” with pre-loaded student results for 2009 MEPA and 2008-2009 MCAS (Department’s Secure Internet Portal)

22. 22 Accessing MEPA Results Go to www.mcasservicecenter.com Click “MEPA” Then “MEPA Reporting” Then “Continue” Log In

23. 23 Accessing MEPA Results (Continued…) Choose your district from the dropdown menu Select your school or District Office from the school menu Enter your MCAS/MEPA five-digit password and login

24. 24 Select “All Grades” under Administration Check the reports you want to view; then “Download” If you have trouble viewing the zipped files, click the blue “here” link for help uploading free unzipping software Accessing MEPA Results (Continued…)

25. 25 Accessing “Dropbox Central” for Principals or their designees https://www4.doemass.org Log in with your Security Portal password Principals and those assigned the MEPA school role by your directory administrator can access “Dropbox Central” Select the “2009 MEPA and 2008-2009 MCAS” folder Download the.TXT file which can be opened using Excel or other spreadsheet software