1. Accessing and Reporting State Student Achievement Data for GPRA Purposes Amy A. Germuth, Ph.D. Compass Consulting Group, LLC

2. 2 Overview of Session The GPRA Performance System  Overview of the Government Performance and Results Act (GPRA)  MSP GPRA Performance Indicators Acquiring Student Achievement Data  Potential issues related to using student achievement data  Acquiring student achievement data Reporting Student Achievement Data  Goals and Issues

3. 3 The GPRA Performance Indicator System The Government Performance and Results Act of 1993 requires federally funded agencies to develop and implement an accountability system based on performance measurement. Agencies must: 1. state clearly what they intend to accomplish, 2. identify the resources required, and 3. periodically report their progress.

4. 4 The GPRA Performance Indicator System Expected outcomes:  Increased accountability for the expenditures of public funds.  Improved congressional decision-making through more objective information on the effectiveness of federal programs.  Promotion of a new governmental focus on results, service delivery and customer satisfaction. Above excerpted from: Demonstrating Results, An Introduction to Government Performance and Results Act, Spring 1999

5. 5 Annual Performance Report Information on program’s outcomes on GPRA indicators are collected via the Annual Performance Report (APR). Project impacts, as assessed via individual project evaluations and use of control/comparison groups are also collected via the APR. Together this information is used by Congress in determining future funding and thus it is critical that both are reported in as consistent a manner as possible.

6. 6 MSP GPRA Performance Indicators Under GPRA, the Department of Education must account for the impact of ED funded activities, usually obtained via grantees’ evaluations. The Education Department is also required to develop performance measures that focus on what’s happening within and among programs. Thus for MSPs the following two GPRA performance indicators have been adopted:  The percentage of students who score at the basic level or above in state assessments of mathematics or science.  The percentage of students who score at the proficient level or above in state assessments of mathematics or science.

7. 7 GPRA Data Needs From Grantees For GPRA purposes MSP projects are asked to report student achievement data aggregated at the project level. Please note - most projects will want data aggregated at the teacher/class level and possibly student-level data linked to teachers for their own evaluation needs. For GPRA, only the most recent year of data is reported, so new data is reported each year This means GPRA measures are not longitudinal measures and thus do not assess change / improvement over time; instead they provide a snapshot of the MSP program at a certain time point.

8. 8 End Overview of GPRA Questions?

9. 9 Issues with Use of Student Achievement Data Available state testing data for math and science based on NCLB requirements:  Currently all states should have an annual mathematics assessment for grades 3 through 8 and one given at least once in grades 10 through 12.  By the 2007-08 school year an annual assessment in science should be given at least once in grades 3 through 5, once in grades 6 through 9, and once in grades 10 through 12.

10. 10 Issues with Use of Student Achievement Data Family Educational Rights and Privacy Act – 1974 Requires that schools obtain written permission from students/parents before releasing individual educational records. Aggregate (statistical) data that contains no personally identifiable information about students does not require written permission from students/parents. The information all MSPs will be required to submit does not fall under FERPA and thus does not require parental permission.

11. 11 Issues with Use of Student Achievement Data Anonymity vs. confidentiality  Anonymity – person’s identity not known to data user  Confidentiality – person’s identity known to data user but when publicly reported no identifiers included The information all MSPs will be required to submit does not require that the data users know the teacher’s identity, thus anonymity is maintained.

12. 12 Data To Request On All Participating Teachers 1. Total number students taught in main subject (math or science) summed across all classes and grade levels. 2. Total number of students of participating teacher who were tested on state standardized test in math/science (summed across classes/grades) 3. Total number of students of participating teacher scoring at the basic level or above (summed across classes/grades) 4. Total number of students of participating teacher scoring at proficient or above (summed across classes/grades) NOTE: Math and science should be reported separately!

13. 13 Example Mr. Jones is a participating teacher and teaches the following: 7 th grade math: 30 students, all tested, 28 at or above basic, 24 at or above proficient 8 th grade math: 31 students, 29 tested, 21 at or above basic, 17 at or above proficient 8 th grade elective - science and civilization: 32 students The following numbers would be calculated for Mr. Jones (and similarly all participating teachers): Number taught: 30+31 = 61 Number tested: 30+29 = 59 Number at or above basic: 28+21 = 49 Number at or above proficient: 24+17= 41 These numbers would be summed across all participating teachers, schools and districts to represent project-level data, which is what will be reported on the APR.

14. 14 Notes on Data Requests There are various ways in which the districts might provide this data to grantees, and thus how grantees would need to transform that data into what needs to be reported.  In some cases districts are only able to report actual scores. In these instances grantees will need to convert scores to performance levels.  In other cases districts might provide the grantee with data aggregated to the classroom level. In these instances grantees will have to do the aggregation to provide project-level data.

15. 15 Removing Roadblocks It is important to know:  what you need to ask for,  in what form you should request it,  whom to ask for it, and  when to ask for it. Always keep in mind that obtainment of data for MSP GPRA indicators is both legal and required.

16. 16 Acquiring Student Data Identify who has access to the data needed (should be someone in district). Inform stakeholders and require maintenance of anonymity and confidentiality of data. For GPRA purposes only assessment data that cannot be linked to individual teachers or students is being requested. (However, for individual project evaluations this data may need to be linked…)

17. 17 Acquiring Student Data Find out district requirements for requesting data, including:  Whom specifically to request data from  Timelines for requesting data  Form(s) in which data are available  When data may no longer be available! Want aggregated data in electronic form not linked to individual teachers  Realize this may require a long lead time so request data early  Ensure cooperation during proposal design phase  May want to create a MOU to ensure access to data

18. 18 Selecting Teachers On Which To Aggregate Data Create a list of all teachers who participated in MSP at any time, regardless of when they participated. May want to also request same data for a comparison group of matched teachers who did not participate in the MSP project. Request data without IDs. Sum data across teachers, classes, schools, and districts to report project-level data

19. 19 Notes on Selecting Teachers In some instances districts have had to have lists of students’ names in order to access their achievement data as it wasn’t available by teacher.

20. 20 End Acquiring Student Achievement Data What have been your experiences trying to obtain student achievement data? What roadblocks have you faced? How have you overcome them?

21. 21 Reporting Student Achievement Data Goal: To report uniform student achievement data for all participating MSP teachers in the project. Reporting Issues  Data availability  Dealing with missing data - report on all teachers even if test data missing or not available - make note of number of teachers and students for whom data is missing

22. 22 What Should be Reported Information aggregated to the project level, including:  Number of districts  Number of schools  Number of MSP teachers  Number of students of those MSP teachers  Number of students with student assessment data in the relevant subject (math or science)  Number of students who scored at basic or above  Number of students who scored at proficient or above

23. 23 Wrap-Up What we covered: Understanding GPRA, PART, and the APR and how they relate to each other. Reviewing the MSP performance indicators and what data must be reported. Issues around obtaining student achievement data:  Knowing in what form to request the data  Knowing from whom, when, and how to get student achievement data How to report student achievement results

24. 24 Wrap-Up Continued Questions?