1. CALIFORNIA DEPARTMENT OF EDUCATION Tom Torlakson, State Superintendent of Public Instruction Training on the Use of the 2011-12 Academic Performance Index Calculator

2. TOM TORLAKSON State Superintendent of Public Instruction Agenda Overview of the Academic Performance Index (API) Calculator Review of the API Rules Practice Questions Entering Standardized Testing and Reporting (STAR) Program and California High School Exit Examination (CAHSEE) Assessment Results into the API Calculator Resources and Contact Information 2

3. TOM TORLAKSON State Superintendent of Public Instruction 3 Overview of the API Calculator What is the API Calculator? –An Excel spreadsheet designed for local educational agencies (LEAs) and schools to input STAR and CAHSEE student results to estimate their API scores.

4. TOM TORLAKSON State Superintendent of Public Instruction 4 Overview of the API Calculator (Cont.) Where is the API Calculator located? The calculator is available on the California Department of Education (CDE) API Web page at http://www.cde.ca.gov/ta/ac/ap/ under the bullet “Calculation Spreadsheets Base and Growth.” Be sure to select the calculator for the appropriate API cycle (2011 Base and 2012 Growth).http://www.cde.ca.gov/ta/ac/ap/ Save the spreadsheet on your computer before using it.

5. TOM TORLAKSON State Superintendent of Public Instruction 5 Overview of the API Calculator (Cont.) The API Calculator has 12 worksheets: 1.Instructions 2.What’s New and FAQs 3.Grade 2 4.Grades 3-5 Students with Disabilities (SWD) Only 5.Grades 3-5 Non-SWD 6.Grade 6 SWD Only 7.Grade 6 Non-SWD 8.Grades 7-8 SWD Only 9.Grades 7-8 Non-SWD 10.Grades 9-12 SWD Only 11.Grades 9-12 Non-SWD 12.Bridge (schools that cross grade/SWD span)

6. TOM TORLAKSON State Superintendent of Public Instruction Overview of the API Calculator (Cont.) What data will I need in order to complete the API Calculator? 1.STAR and CAHSEE (The results are sent to LEAs by the Educational Testing Service [ETS].) 2.California Longitudinal Pupil Achievement Data System (CALPADS) data 6

7. TOM TORLAKSON State Superintendent of Public Instruction Overview of the API Calculator (Cont.) Why do I need to use STAR/ CAHSEE results from ETS and CALPADS data? Because inclusion/exclusion and adjustment rules are applied as the first step to calculating an API score, it is important to use the STAR/CAHSEE results received from ETS and the CALPADS data in order to make specific determinations. (The STAR/CAHSEE results posted on the CDE Web pages do not contain all the necessary information needed to accurately apply the inclusion/exclusion or adjustment rules.) 7

8. TOM TORLAKSON State Superintendent of Public Instruction API Rules API inclusion/exclusion and adjustment rules must be applied prior to entering STAR/CAHSEE student results into the calculator. Full details of the API rules can be found in the 2011-12 API Reports Information Guide posted on the CDE API Web page at http://www.cde.ca.gov/ta/ac/ap/ (see pages 25-39).http://www.cde.ca.gov/ta/ac/ap/ 8

9. TOM TORLAKSON State Superintendent of Public Instruction API Rules (Cont.) Only test scores that meet the inclusion rules are included in the API; these scores are referred to as valid scores. 9

10. TOM TORLAKSON State Superintendent of Public Instruction API Rules (Cont.) The following inclusion/exclusion and adjustment rules are applied to determine when and how a test score is included in the API: –Mobility/Continuous Enrollment –English Learners (ELs) –Blank Test –Testing Irregularity –Accommodations –Modifications –Special Conditions –Adjustments to Performance Levels –Assignment of 200 –CAHSEE –Reassignment of Test Scores 10

11. TOM TORLAKSON State Superintendent of Public Instruction 11 API Rules (Cont.) How are the inclusion/exclusion rules applied in determining the number tested and valid scores? Generally, the process used in applying the inclusion/exclusion rules occurs in the order presented in the prior slide. Some variations may occur for student records where multiple inclusion/exclusion rules apply.

12. TOM TORLAKSON State Superintendent of Public Instruction API Rules: Mobility or Continuous Enrollment Mobility or continuous enrollment is defined as the student being enrolled at the same school and/or district from Fall Census Day to the first day of STAR/CAHSEE testing without having a gap in enrollment of more than 30 consecutive calendar days. Beginning with the 2011 Base API, continuous enrollment is determined using the CALPADS data. Any student who is not continuously enrolled will not have their test scores included in the API. 12

13. TOM TORLAKSON State Superintendent of Public Instruction API Rules: ELs EL students who have been enrolled in a U.S. school for less than one year are not included in the API: –March 15 is the cut-off date used to determine “less than one year.” –For example, if an EL student enrolled in a U.S. school on March 16, 2011, and took the STAR/CAHSEE in spring 2012, this student’s test scores will not be included in the API. 13

14. TOM TORLAKSON State Superintendent of Public Instruction API Rules: Blank Test If a student record does not have a performance level or shows that no items were attempted on any part of the California Standards Test (CST), California Modified Assessment (CMA), or California Alternate Performance Assessment (CAPA), the entire STAR student record is not included in API. 14

15. TOM TORLAKSON State Superintendent of Public Instruction API Rules: Blank Test (Cont.) Exceptions: If a student record shows “Tested but Marked No Answers” (STAR Special Condition Code Z) for any content area, the content area is included at the Far Below Basic performance level (200 points in the API calculation). 15

16. TOM TORLAKSON State Superintendent of Public Instruction API Rules: Blank Test (Cont.) Exceptions (continued): If a CAHSEE grade ten student census (February or March) record shows “Blank/Not Attempted” for one or both content areas, this record is included at the Far Below Basic performance level (200 points in the API calculation). –Blank records for grades eleven and twelve are excluded. 16

17. TOM TORLAKSON State Superintendent of Public Instruction API Rules: Testing Irregularity If a content area on a student record reflects a student or adult testing irregularity, the result is included in the Base API but is not included in the Growth API. Any adult testing irregularity results in a school not being eligible for any awards program in the current year. –If the adult testing irregularity affects over five percent of the students, the school’s API is invalidated (i.e., no online report is generated) and the school is ineligible for awards in both the current and future year. 17

18. TOM TORLAKSON State Superintendent of Public Instruction API Rules: Accommodations Test administration variations and accommodations do not result in adjustments being made to the student’s test results. 18

19. TOM TORLAKSON State Superintendent of Public Instruction API Rules: Modifications Student records indicating that a modification was used are included at the Far Below Basic performance level (200 points in the API calculation). 19

20. TOM TORLAKSON State Superintendent of Public Instruction API Rules: Special Conditions The following are the STAR special conditions codes that are considered in API calculations: –M: Took some tests but moved before remaining tests were administered. (The tests taken are included in the API at the school where the tests were taken.) –P: Not tested by parent/guardian request. (These tests are not included in the API calculation.) 20

21. TOM TORLAKSON State Superintendent of Public Instruction API Rules: Adjustments to Performance Levels Students in Grade Eight Through Eleven Who Take CST General Mathematics: –Grade Eight Students: Test performance level is dropped by one (e.g., advanced to proficient). –Grade Nine Students: Test performance level is dropped by two (e.g., proficient to below basic). –Grades Ten and Eleven Students: Test performance level is dropped to Far Below Basic (200 points in the API calculation). 21

22. TOM TORLAKSON State Superintendent of Public Instruction API Rules: Assignment of 200 Grades eight through eleven students who did not take an end-of-course (EOC) CST in mathematics are assigned 200 points (test weight of 0.10 compared to the test weight of 0.20 for students who take a CST in mathematics). Grades nine through eleven students who did not take an EOC CST in science are assigned 200 points (test weight of 0.05 compared to the test weight of 0.22 for students who take a CST in science). 22

23. TOM TORLAKSON State Superintendent of Public Instruction API Rules: Assignment of 200 (Cont.) Life Science results are used for grade ten students who take the CST in Life Science but do not take an EOC CST in science (i.e., 200 points will not be assigned). Grade ten students who take both an EOC CST in science and the CST in Life Science will have both scores included in the API. 23

24. TOM TORLAKSON State Superintendent of Public Instruction API Rules: CAHSEE Grade ten students who take the CAHSEE (February or March census) are included in the API: –1000 points for a student who passes –200 points for a student who does not pass Grades eleven and twelve students who take the CAHSEE are included in the API only if they receive a passing score. 24

25. TOM TORLAKSON State Superintendent of Public Instruction API Rules: CAHSEE (Cont.) A grade ten make-up record must be matched to a census record using the Statewide Student Identifier (SSID) for the test score to be included in the API. “Non-matching” records will not be included in the API. 25

26. TOM TORLAKSON State Superintendent of Public Instruction API Rules: Reassignment of Test Scores A reassignment of student test scores occur for grades two through twelve when students are referred to an alternative education program (AEP) school. An AEP school is defined as: –County Community School –Continuation High School –District Community Day School 26

27. TOM TORLAKSON State Superintendent of Public Instruction API Rules: Reassignment of Test Scores (Cont.) A reassignment occurs if a student: –Was enrolled at a non-AEP school on Fall Census Day, and –Was referred to an AEP school using exit reason code T167 within the CALPADS, and –Took the STAR/CAHSEE at the AEP school. 27

28. TOM TORLAKSON State Superintendent of Public Instruction Entering Assessment Results into the API Calculator After applying the inclusion/exclusion and adjustment rules and determining valid scores, the number of students scoring at each performance level can be entered into the API calculator. The API calculator has specific worksheets for “Non-SWD” and “SWD” for all grade spans except grade two. 28

29. TOM TORLAKSON State Superintendent of Public Instruction 29 Entering Assessment Results into the API Calculator (Cont.) Definition of SWD –Any student record that has a valid Primary Disability code (other than “000” or “999”), or –Any student who took the CMA or CAPA Definition of Non-SWD –The remaining students who do not meet the above criteria

30. TOM TORLAKSON State Superintendent of Public Instruction Practice – Grades 2-11 If a grade eight CST student record has a valid Primary Disability Code of “290,” where would this student record be entered into the calculator? a.The “Grades 7-8 SWD Only” worksheet b.The “Grades 7-8 Non-SWD” worksheet 30 Question 1

31. TOM TORLAKSON State Superintendent of Public Instruction Practice – Grades 2-11 (Cont.) Any student record with a valid “Primary Disability Code” would be entered into the SWD worksheet, including SWD students who take a CST. 31 Answer: a

32. TOM TORLAKSON State Superintendent of Public Instruction Practice – Grades 2-11 (Cont.) Three students took the Standards-based Tests in Spanish (STS). Under which column should their results be entered into the API calculator? 32 Question 2

33. TOM TORLAKSON State Superintendent of Public Instruction Practice – Grades 2-11 (Cont.) The STS results are not entered into the API calculator because the STS is not included in the calculation of the API. A list of the assessments used in the API can be found on page 5 of the 2011-12 API Reports Information Guide (http://www.cde.ca.gov/ta/ac/ap/docum ents/infoguide12.pdf).http://www.cde.ca.gov/ta/ac/ap/docum ents/infoguide12.pdf 33 Answer

34. TOM TORLAKSON State Superintendent of Public Instruction Practice - Elementary A student at Mountain Elementary School: –Enrolled on October 20, 2011, and –Participated in STAR in April 2012 Would this student’s STAR results be included in the school’s API score? 34 Question 3

35. TOM TORLAKSON State Superintendent of Public Instruction Practice – Elementary (Cont.) Because the student enrolled in the school after Fall Census Day (first Wednesday in October), the student is not considered continuously enrolled and the assessment results will not be included in the school’s API score. 35 Answer: No

36. TOM TORLAKSON State Superintendent of Public Instruction Practice – Elementary (Cont.) A student at Rocky Elementary School: –Enrolled on the first day of school in August 2011, and –Exited via CALPADS for 30 consecutive calendar days in January 2012, and –Re-enrolled at Rocky Elementary School and participated in the STAR Will this student’s STAR results be included in the school’s API score? 36 Question 4

37. TOM TORLAKSON State Superintendent of Public Instruction Practice – Elementary (Cont.) The student was enrolled at the school on Fall Census Day and was exited via CALPADS for less than 31 consecutive calendar days. Therefore, this student is considered continuously enrolled and the STAR results will be included in the school’s API score. 37 Answer: Yes

38. TOM TORLAKSON State Superintendent of Public Instruction Practice – Elementary (Cont.) An EL student entered into the U.S. and enrolled at River Park Elementary School on March 6, 2011, and participated in STAR in the spring of 2012. Would this student’s STAR results be included in the school’s API score? 38 Question 5

39. TOM TORLAKSON State Superintendent of Public Instruction Practice – Elementary (Cont.) The “cut off” date is March 15, 2011. Students enrolling in a U.S. school for the first time before March 15, 2011, will be included in the 2012 Growth API. 39 Answer: Yes

40. TOM TORLAKSON State Superintendent of Public Instruction Practice – Elementary (Cont.) If a grade eight student scored “Proficient” on the CST General Mathematics, what performance level would be entered into the API calculator? 40 Question 6

41. TOM TORLAKSON State Superintendent of Public Instruction Practice – Elementary (Cont.) The performance level would be adjusted from “Proficient” to “Basic” because the performance level of grade eight students who take the CST in General Mathematics is dropped one performance level for the API calculation. 41 Answer

42. TOM TORLAKSON State Superintendent of Public Instruction Practice – Elementary (Cont.) If a grade eight student took the CST in English-language arts (ELA) and scored “Proficient,” but did not take an EOC CST in mathematics, what performance levels, if any, would be included in the calculator? 42 Question 7

43. TOM TORLAKSON State Superintendent of Public Instruction Practice – Elementary (Cont.) Because this grade eight student did not take an EOC CST in mathematics, a score of Far Below Basic (200 points) will be entered into the API calculator. The CST in ELA “Proficient” score will also be included in the API calculator. 43 Answer

44. TOM TORLAKSON State Superintendent of Public Instruction Practice – High If a grade nine student scored “Advanced” on the CST in General Mathematics, what performance level would be entered into the API calculator? 44 Question 8

45. TOM TORLAKSON State Superintendent of Public Instruction Practice – High (Cont.) The performance level would be adjusted from “Advanced” to “Basic.” Scores for grade nine students who take the CST in General Mathematics are dropped two performance levels for the API calculation. 45 Answer

46. TOM TORLAKSON State Superintendent of Public Instruction Practice – High (Cont.) If a grade eleven student took the CAHSEE and passed the ELA portion, but did not pass the mathematics portion, would both results be included in the school’s API score? 46 Question 9

47. TOM TORLAKSON State Superintendent of Public Instruction Practice – High (Cont.) Only passing grade eleven and twelve CAHSEE results are included in the API. Therefore, the CAHSEE score for the ELA portion would be included but the score for the mathematics portion would not be included. 47 Answer: No

48. TOM TORLAKSON State Superintendent of Public Instruction Practice – High (Cont.) A grade ten student took the following assessments: –CST: ELA, Life Science, and World History – CAHSEE: ELA Mathematics Based on the above information, which assessments would be included into the API calculator? 48 Question 10

49. TOM TORLAKSON State Superintendent of Public Instruction Practice – High (Cont.) All CST and CAHSEE results would be included in the calculator as is (i.e., there would be no adjustments made to the performance levels). However, because this student did not take an EOC CST in mathematics, this student would be counted in the “Far Below Basic” cell within the API calculator for mathematics. 49 Answer

50. TOM TORLAKSON State Superintendent of Public Instruction Practice – High (Cont.) A grade nine student took the High School Summative Mathematics Test and scored “Basic.” What performance level would be entered into the API calculator? 50 Question 11

51. TOM TORLAKSON State Superintendent of Public Instruction Practice – High (Cont.) “Basic.” Adjustments are not made to performance levels for the High School Summative Mathematics Test. 51 Answer

52. TOM TORLAKSON State Superintendent of Public Instruction 52 Part I of API Calculator After the inclusion/exclusion and adjustment rules have been applied, enter the number of students scoring at each performance level into the cells in Part I of each appropriate worksheet. Calculation Spreadsheet for the 2011 Base API or 2012 Growth API For Grades 3-5 Students with Disabilities Only Part I - School Test Data California Standards Test (CST), California Modified Assessment (CMA), and California Alternate Performance Assessment (CAPA) Performance Level Weighting Factor (fixed) Performance Level Number of Valid Scores: English- Language Arts (ELA) Number of Valid Scores: Mathematics Number of Valid Scores: Science (Grade 5 Only) 10005 : Advanced000 8754 : Proficient000 7003 : Basic000 5002 : Below Basic000 2001 : Far Below Basic000 Total Valid Scores000 Verify that Part I data entry is complete (all cells = 0 or greater).

53. TOM TORLAKSON State Superintendent of Public Instruction Part I: Grades 9-12 Worksheet, Students with Disabilities 53 California Standards Test (CST), California Modified Assessment (CMA), and California Alternate Performance Assessment (CAPA) Performance Level Weighting Factor (fixed) Performance Level Number of Valid Scores: ELA Number of Valid Scores: Math Number of Valid Scores: Math 200 Pts.* Assignment Number of Valid Scores: Science (End-of- Course Tests) Number of Valid Scores: Science 200 Pts.* Assignment (End-of- Course) Number of Valid Scores: History- Social Science (Grades 9- 11 Only) Number of Valid Scores: Life Science (Grade 10 Only) 10005 : Advanced01N/A0 00 8754 : Proficient11N/A3 40 7003 : Basic17N/A1 86 500 2 : Below Basic36N/A10N/A60 200 1 : Far Below Basic3141130 Total Valid Scores8164151216 Verify that Part I data entry is complete (all cells = 0 or greater).

54. TOM TORLAKSON State Superintendent of Public Instruction Part I: Grades 9-12 Worksheet, Students with Disabilities (Cont.) California High School Exit Examination (CAHSEE) Performance Level Weighting Factor (fixed) Performance Level Number of Valid Scores: ELA Number of Valid Scores: Mathematics 1000Pass20 N/A 200No Pass11 Total Valid Scores31 54

55. TOM TORLAKSON State Superintendent of Public Instruction Part I: Grades 9-12 Worksheet, Students Without Disabilities California Standards Test (CST) Performance Level Weighting Factor (fixed) Performance Level Number of Valid Scores: ELA Number of Valid Scores: Math Number of Valid Scores: Math 200 Pts.* Assignment Number of Valid Scores: Science (End-of- Course Tests) Number of Valid Scores: Science 200 Pts.* Assignment (End-of- Course) Number of Valid Scores: History- Social Science (Grades 9- 11 Only) Number of Valid Scores: Life Science (Grade 10 Only) 10005 : Advanced2468N/A94N/A10159 8754 : Proficient176145N/A138N/A13668 7003 : Basic130214N/A141N/A7641 500 2 : Below Basic22149N/A33N/A316 200 1 : Far Below Basic13257192279 Total Valid Scores58754174252371183 Verify that Part I data entry is complete (all cells = 0 or greater). 55

56. TOM TORLAKSON State Superintendent of Public Instruction Part I: Grades 9-12 Worksheet, Students Without Disabilities (Cont.) California High School Exit Examination (CAHSEE) Performance Level Weighting Factor (fixed) Performance Level Number of Valid Scores: ELA Number of Valid Scores: Math 1000Pass201212 N/A 200No Pass74 Total Valid Scores208216 56

57. TOM TORLAKSON State Superintendent of Public Instruction Part II of API Calculator Part II of the calculator is automatically updated as Part I is completed. 57

58. TOM TORLAKSON State Superintendent of Public Instruction Bridge Worksheet The worksheet labeled “Bridge” combines the data from each individual worksheet (e.g., grade 2, grades 3-5, etc.) and automatically calculates the school and LEA API scores. 58

59. TOM TORLAKSON State Superintendent of Public Instruction 59 Bridge Worksheet (Cont.) Example: Calculation Spreadsheet for the 2011 Base API or 2012 Growth API For Bridge Schools that Cross Grade/SWD Spans API Calculation Grade Levels Total Weights API Scores Grade 20.000 Grades 3-5 Students with Disabilities Only0.000 Grades 3-5 Students with No Disabilities0.000 Grade 6 Students with Disabilities Only0.000 Grade 6 Students with No Disabilities0.000 Grades 7-8 Students with Disabilities Only0.000 Grades 7-8 Students with No Disabilities0.000 Grades 9-12 Students with Disabilities Only15.98558 Grades 9-12 Students with No Disabilities609.43846 Total Values 625.41839

60. TOM TORLAKSON State Superintendent of Public Instruction Resources “What’s New and FAQs” worksheet in the API calculator CDE API Web page: http://www.cde.ca.gov/ta/ac/ap/ http://www.cde.ca.gov/ta/ac/ap/ CDE STAR Web page: http://www.cde.ca.gov/ta/tg/sr/ http://www.cde.ca.gov/ta/tg/sr/ CDE CAHSEE Web page: http://www.cde.ca.gov/ta/tg/hs/ http://www.cde.ca.gov/ta/tg/hs/ ETS STAR Web site: http://www.startest.org/http://www.startest.org/ ETS CAHSEE Web site: http://www.ets.org/cahsee/about http://www.ets.org/cahsee/about 60

61. TOM TORLAKSON State Superintendent of Public Instruction Contact Information Betty Miura, Research Analyst Jenny Singh, Administrator Academic Accountability Unit aau@cde.ca.gov 916-319-0863 61