1. Use and Impact of the Student Progress Monitoring System Mississippi Department of Education Office of Instructional Programs and Services Office of Student Assessment MASS January 10, 2006

2. 2 Assessment that is explicitly designed to promote learning is the single most powerful tool we have for raising standards and empowering life long learning. – Assessment Reform – Assessment Reform Group (1999) Group (1999)

3. 3 Student Progress Monitoring System (SPMS) is designed to offer a web-based approach to help districts track which students are mastering key objectives and which are not. Student Progress Monitoring System (SPMS) is designed to offer a web-based approach to help districts track which students are mastering key objectives and which are not. The ultimate goal of SPMS is that teachers use SPMS assessments to focus on reviewing, revising, and improving daily instruction through rapid assessment and feedback. The ultimate goal of SPMS is that teachers use SPMS assessments to focus on reviewing, revising, and improving daily instruction through rapid assessment and feedback. Through the daily/weekly review of student progress, districts will have a better way to track their expectations for meeting annual accountability requirements and for preparing students for national standardized tests. Through the daily/weekly review of student progress, districts will have a better way to track their expectations for meeting annual accountability requirements and for preparing students for national standardized tests. Overview

4. 4 District Use Participation is a district and school decision. Participation is a district and school decision. Districts registration for SPMS may include all schools or only targeted schools. Districts registration for SPMS may include all schools or only targeted schools. For the 2005-06 year, the state is funding the entire per pupil cost, and the MDE intends to fully fund SPMS for the 2006-07 school year. For the 2005-06 year, the state is funding the entire per pupil cost, and the MDE intends to fully fund SPMS for the 2006-07 school year. Implementation depends heavily on local control; therefore, districts can structure the use of the system. Implementation depends heavily on local control; therefore, districts can structure the use of the system. Some districts have chosen to use SPMS more for “benchmark” testing, while others are using the system for formative assessment data as well. Some districts have chosen to use SPMS more for “benchmark” testing, while others are using the system for formative assessment data as well.

5. 5 The item bank currently contains math, reading, and language arts items aligned to the “old” Mississippi Curriculum Frameworks. The item bank currently contains math, reading, and language arts items aligned to the “old” Mississippi Curriculum Frameworks. Coverage includes items for grades 1-12 in these areas. Coverage includes items for grades 1-12 in these areas. Coverage for science and social studies includes items for grades 3-12. Coverage for science and social studies includes items for grades 3-12. SPMS also supports writing at grades 4, 7, and 10 (English II) with automated scoring. SPMS also supports writing at grades 4, 7, and 10 (English II) with automated scoring. Tests can be built by teacher, school, or district. Tests can be built by teacher, school, or district. Application

6. 6 Individual and summary reports enable teachers and administrators to track, monitor, and disaggregate data by various demographic categories at the classroom, school, and district level. Individual and summary reports enable teachers and administrators to track, monitor, and disaggregate data by various demographic categories at the classroom, school, and district level. Student portfolios enable teachers and students to track progress in real time. Student portfolios enable teachers and students to track progress in real time. SPMS provides system-generated parent letters. SPMS provides system-generated parent letters. A new resource unit will be unveiled at the district level at the end of January and will allow users to use lesson plans, models, and activities already in the system and to add personal resources for all SPMS users. A new resource unit will be unveiled at the district level at the end of January and will allow users to use lesson plans, models, and activities already in the system and to add personal resources for all SPMS users. Application

7. 7 Reporting features: Reporting features: State test results are automatically loaded into the system. State test results are automatically loaded into the system. Summary performance reports provide results for curriculum competency and objective. Summary performance reports provide results for curriculum competency and objective. Item analysis and roster reports show performance for each test and for each student. Item analysis and roster reports show performance for each test and for each student. Application

8. 8 Status Comparison 2004-2005 2004-2005 Districts: 63 Districts: 63 Schools: 244 Schools: 244 Students: 133,000+ Students: 133,000+ 2005-2006 Districts: 116 Schools: 647 Students: 398,413

9. 9 2005-2006 State Users To date in 2005-2006, 247 schools across 78 districts are recording test results in the Mississippi Student Progress Monitoring System. To date in 2005-2006, 247 schools across 78 districts are recording test results in the Mississippi Student Progress Monitoring System. Over 70,400 test sessions have been administered to students in elementary through high schools in the subject areas of reading, math, writing, science, and social studies. Over 70,400 test sessions have been administered to students in elementary through high schools in the subject areas of reading, math, writing, science, and social studies.

10. 10 2005-2006 Users SPMS testing began in August 2005, with November and December being the heaviest testing months in the school year thus far. SPMS testing began in August 2005, with November and December being the heaviest testing months in the school year thus far. Testing continues as a brisk pace in 2006, with over 5,000 tests already recorded in January. Testing continues as a brisk pace in 2006, with over 5,000 tests already recorded in January.

11. 11 2005-2006 Test Sessions by Month

12. 12 2005-2006 Test Sessions by Month

13. 13 Relationship Between SPMS Usage and State Test Scores Little is known about the nature of the tests administered, the importance of the SPMS tests, and how the data from SPMS were used to drive instruction. Little is known about the nature of the tests administered, the importance of the SPMS tests, and how the data from SPMS were used to drive instruction. Investigations into the relationship between usage of SPMS and performance on state assessments show that the usage across the state in the 2004-05 year may not have been organized, integrated, or strategic enough in order to show major trends across the board. Investigations into the relationship between usage of SPMS and performance on state assessments show that the usage across the state in the 2004-05 year may not have been organized, integrated, or strategic enough in order to show major trends across the board. Case studies provide the most insight into the effectiveness of the SPMS. Case studies provide the most insight into the effectiveness of the SPMS.

14. 14 Case Study: Forrest County

15. 15 Forrest County Grade 3 MCT Math Gains from 2003-04 to 2004-05 for Grade 3 MCT Math mean scale score State gain: 2.4 scale score points 499.8 to 502.2 Forrest County gain: 10.9 scale score points 494.6 to 505.5 Gains for MCT Math from 2003-04 Grade 2 to 2004-05 Grade 3 % Proficient and Above State gain: 0.2% 91.7% to 91.9% Forrest County gain: 6.0% 86.7% to 92.7%

16. 16

17. 17 Forrest County Grade 3 MCT Math Black-White Gap from 2003-04 to 2004-05 for Grade 3 MCT Math, % Proficient and Above Forrest Co.State 2003- 04 2004- 05 2003- 04 2004- 05 Black71828788 White96 Gap251498

18. 18 Forrest County Grade 3 MCT Math Economically Disadvantaged Gap from 2003-04 to 2004-05 for Grade 3 MCT Math, % Proficient and Above Forrest Co.State 2003- 04 2004- 05 2003- 04 2004- 05 ED818889 Non-ED9596 Gap14877

19. 19 Case Study: Kemper County

20. 20 Kemper County Grade 8 MCT Reading Gains from 2003-04 to 2004-05 for Grade 8 MCT Reading mean scale score State decrease: 7.0 scale score points 563.6 to 556.6 Kemper County gain: 9.4 scale score points 544.5 to 553.9 Gains from 2003-04 to 2004-05 for Grade 8 MCT Reading % Proficient and Above State decrease: -4.8% 61.6% to 56.8% Kemper County gain: 14.2% 41.1% to 55.3%

21. 21

22. 22 Economically Disadvantaged Gap from 2003-04 to 2004-05 for Grade 8 MCT Reading, Percent Proficient and Above Kemper Co.State 2003- 04 2004- 05 2003- 04 2004- 05 ED37544744 Non-ED52607772 Gap1563028 Kemper County Grade 8 MCT Reading

23. 23 Kemper County and SPMS 2004-05 Vicki Parker is a retired Mississippi educator and certified Vantage trainer. Vicki Parker is a retired Mississippi educator and certified Vantage trainer. Vicki worked with Kemper County during the 2004-2005 school year. Vicki worked with Kemper County during the 2004-2005 school year. Vicki will also be a trainer for the SPMS Curriculum, Assessment, and Instruction (CAI) modules. Vicki will also be a trainer for the SPMS Curriculum, Assessment, and Instruction (CAI) modules.

24. 24 MS In-State Vantage Contact David Timmons, who taught in the Bay- Waveland School District, is a certified Vantage Trainer. David Timmons, who taught in the Bay- Waveland School District, is a certified Vantage Trainer. He is now serving as the MS Contact Person for SPMS. He is now serving as the MS Contact Person for SPMS. His e-mail is dtimmons@vantage.com. His e-mail is dtimmons@vantage.com.

25. 25 New for 2006 3-in-1 bubble sheets with significant cost reduction 3-in-1 bubble sheets with significant cost reduction Version 2.0 for district users Version 2.0 for district users MCT practice tests MCT practice tests Science practice tests Science practice tests MS-teacher reviewed items for social studies and science MS-teacher reviewed items for social studies and science SPMS Curriculum, Assessment, and Instruction (CAI) modules SPMS Curriculum, Assessment, and Instruction (CAI) modules

26. 26 Bubble Sheets for Off-line Testing

27. 27 3-in-1 Bubble Sheet

28. 28 Version 2.0 Version 2.0 will be ready for the 2006-2007 school year. Version 2.0 will be ready for the 2006-2007 school year. Vantage is making 2.0 available for users at the district level only at the end of January for “trial use”; 2.0 will not contain “live” files. Vantage is making 2.0 available for users at the district level only at the end of January for “trial use”; 2.0 will not contain “live” files. Version 2.0 will have a new resource site where districts users will be able to enter lesson plans or activities to the resource list already identified within the system. Version 2.0 will have a new resource site where districts users will be able to enter lesson plans or activities to the resource list already identified within the system.

29. 29 Version 2.0 Version 2.0 will have new functions to allow users to nominate items to be included in the SPMS database for all users. Version 2.0 will have new functions to allow users to nominate items to be included in the SPMS database for all users. Version 2.0 will have functionality to allow external test data (such as DIBELS) to be entered into the system. Version 2.0 will have functionality to allow external test data (such as DIBELS) to be entered into the system.

30. 30 MCT and Science Practice Tests Release set for late January for first practice test for grades 3 and 7, all three content areas Release set for late January for first practice test for grades 3 and 7, all three content areas Second release set for February for first practice test for all grades, all three content areas Second release set for February for first practice test for all grades, all three content areas Spring release for practice tests two and three for all grades and content areas Spring release for practice tests two and three for all grades and content areas Release in spring for practice science tests for grade 5, grade 8, and Biology I Release in spring for practice science tests for grade 5, grade 8, and Biology I (These are not the Riverside hard copy tests.) (These are not the Riverside hard copy tests.)

31. 31 MS Teacher Reviewed Items New science and social studies items New science and social studies items All items reviewed by Mississippi teacher committees All items reviewed by Mississippi teacher committees Items to be identified to distinguish them from original SPMS items Items to be identified to distinguish them from original SPMS items Mississippi courses new to SPMS Mississippi courses new to SPMS Science: Biology II, Physical Science, Anatomy and Physiology Science: Biology II, Physical Science, Anatomy and Physiology Social Studies: MS Studies, US Government, and Economics Social Studies: MS Studies, US Government, and Economics

32. 32 New Math and Language Arts Items Addition of math and language arts items to SPMS was delayed so that new items will match the revised frameworks. Addition of math and language arts items to SPMS was delayed so that new items will match the revised frameworks. Teachers will meet in late spring and/or early summer to review items. Teachers will meet in late spring and/or early summer to review items. Items will be added to SPMS by January 2007. Items will be added to SPMS by January 2007. New MCT2 will not be administered until spring 2008. New MCT2 will not be administered until spring 2008.

33. 33 SPMS CAI Modules

34. 34 SPMS CAI modules Districts selected for Series I or II were notified January 6. Districts selected for Series I or II were notified January 6. Ten teams were selected for Series I and for Series II based upon a rubric evaluation of the district’s application. Ten teams were selected for Series I and for Series II based upon a rubric evaluation of the district’s application. Letters regarding Series III and IV will be mailed to superintendents and SPMS contacts next week. Letters regarding Series III and IV will be mailed to superintendents and SPMS contacts next week. Districts that applied but were not selected for Series I or II will be able to submit a “Reapplication” to be considered for the next series. Blank boxes Non-users Series V and VI sites have not yet been selected, but one will be in south Mississippi, close to the coast.

35. 35 CAI Modules Each series consists of three days of training, each day separated by about two weeks. Each series consists of three days of training, each day separated by about two weeks. Districts are asked to send three-member teams. We suggest a central office person directly involved the district’s plan for student improvement, a school administrator, and a lead teacher/SPMS-user teacher. The same team should attend all sessions. Districts are asked to send three-member teams. We suggest a central office person directly involved the district’s plan for student improvement, a school administrator, and a lead teacher/SPMS-user teacher. The same team should attend all sessions. These series are not initial training sessions for users. All team members who attend these sessions should be active SPMS users. These series are not initial training sessions for users. All team members who attend these sessions should be active SPMS users.

36. 36 CAI Curriculum Objectives 1) Gain a clear understanding of the alignment between SPMS and the Mississippi Curriculum Frameworks 2) Develop a scope and sequence guide to assist with the integration of SPMS for instructional purposes 3) Develop a plan to use SPMS to ensure student growth in mastering Mississippi Curriculum Framework objectives as well as local district curriculum goals

37. 37 CAI Assessment Objectives 1) Create appropriate SPMS assessments for targeted objectives 2) Understand SPMS reports available 3) Utilize SPMS assessment data to inform instruction 4) Establish district priority goals for increasing student achievement 5) Design a district/school strategic plan and program of action with measurable outcomes and an evaluative component

38. 38 CAI Instruction Objectives 1) Use SPMS assessment results to review/critique instructional practices for targeted objectives 2) Explore SPMS reports to refocus instruction on targeted curriculum, including pre-requisite skills 3) Connect SPMS reports to the use of research- based instructional strategies 4) Examine SPMS reports to create learning opportunities and revise instructional strategies based on individual or group needs 5) Access resources through SPMS to assist in providing quality instruction

39. 39 Under Consideration For the 2006-2007 school year, the MDE and Vantage are working to have data available when school begins. For the 2006-2007 school year, the MDE and Vantage are working to have data available when school begins. The idea is to “promote” all SPMS students at the end of the 2005-06 school year to the next grade and/or next school. The idea is to “promote” all SPMS students at the end of the 2005-06 school year to the next grade and/or next school. This process would mean districts might have to “clean up” the data, but the process would allow districts access as soon as the 2006-07 school year begins. This process would mean districts might have to “clean up” the data, but the process would allow districts access as soon as the 2006-07 school year begins.

40. 40 SPMS Advisory Committee The SPMS Advisory Committee meets quarterly during the school year. The SPMS Advisory Committee meets quarterly during the school year. The committee makes recommendations and considers new functions. The committee makes recommendations and considers new functions. Currently, there are 21 members on this committee. Currently, there are 21 members on this committee. Michael Hood, assistant superintendent from Montgomery County, is a “super” user of SPMS and a member of the Advisory Committee. Michael Hood, assistant superintendent from Montgomery County, is a “super” user of SPMS and a member of the Advisory Committee.