Information About the Accountability Provisions of No Child Left Behind California Department of Education Policy and Evaluation Division July 2003

2 Timeline of Events Accountability Workbook submitted by January 31, 2003 deadline Peer Review occurred February 26 Submitted State Plan and additional material for the Accountability Workbook on May 1 Discussions and negotiations continued with US Department of Education through June 6, 2003

3 Final Workbook Approval Final approval of workbook by US Department of Education (USED) on June 10 State Board of Education (SBE) approval of revisions required by USED on June 11 For information on California’s state plans:

4 Approved Without Additional Revision API as additional indicator CAHSEE as 10 th grade academic measure Subgroup size (100 or 50 if 15%) Intermediate goals for meeting annual measurable objectives Definitions of “mobility”

5 Significant Revisions Participation Rate – for grades 2 – 8, parent exemptions must be counted in “number of students enrolled” Graduation Rate – CAHSEE proxy replaced with NCES formula EL Subgroup – EL for 3 years of proficiency in ELA CST ASAM indicators replaced with AYP Small school AYP determination done by state

6 Timeline for Release of Reports July 2003: –2002 Base APIs for districts and ASAM schools –2002 Baseline AYP report (2002 testing data) Advisory to LEAs Videotape Information Guide Staggered Release

7 Timeline for Release of Reports August 2003: –Phase AYP report (AMO’s and participation rate) October 2003: –Phase AYP report (API and graduation rate) –2003 Growth API release December 2003: –Phase AYP report (updated data and application of “safe harbor”) –Certified 2003 Growth API report

AYP Baseline Report The CDE had hoped to release the 2002 AYP baseline information in late May 2003 With the approval of our Accountability Workbook, CDE will release this information in July This report provides a starting point for each school using the new AYP definition and metric (e.g., percent proficient) Districts can see which schools may be at risk for not meeting AYP when the August 2003 is posted

Adequate Yearly Progress

10 Adequate Yearly Progress (AYP): The Basics Based on English language arts and mathematics separately All students held to same high academic standards Goal is 100% proficiency by Inclusion of all students –95% participation on assessments –Accountability for all students

11 Components of AYP 1)Achievement of the statewide Annual Measurable Objectives (AMO’s) in both English language arts (ELA) and math “Percent proficient” 2)Achievement of a 95% participation rate on all applicable assessments 3)Achievement on the “additional” indicators API for all schools, and Graduation rate for high schools

12 AMOs in ELA and Math 95% Participation Rate API Graduation rate AYP

13 Annual Measurable Objectives (AMO’s) For Elementary and Middle Schools are based on: –The California Standards Tests (CSTs) in English language arts and math –The California Alternate Performance Assessment (CAPA) for students with severe cognitive disabilities For High Schools are based on: –Results from the Grade 10 California High School Exit Exam (CAHSEE) administration –The California Alternate Performance Assessment (CAPA) for students with severe cognitive disabilities

14 High School Data ETS completed the technical procedure to set three performance levels on the CAHSEE for NCLB purposes (see slides15 and 16) Starting points for high schools were set using the new CAHSEE performance levels and the methodology set by NCLB Annual measurable objectives and intermediate goals were set for high schools and approved by the State Board of Education (see slides 22 and 23)

15 Using CAHSEE to Generate “Percent Proficient” – High schools only NCLB requires that three performance levels (i.e. advanced, proficient, basic) be set on all assessments used for AYP Linked CAHSEE ELA to grade 10 CST-ELA Linked CAHSEE Math to grade 7 CST-Math Technical process done by ETS

16 Cut Scores on the CAHSEE for NCLB High schools only English Language Arts Advanced = 413 or above Proficient = Not Proficient = Below 387 Math Advanced = 417 or above Proficient = Not Proficient = Below 373 These Cut Scores are Independent of the CAHSEE Pass Score. The CAHSEE Pass Score will Remain Unchanged.

17 More On The AMO’s… Statewide goals are applicable to ALL –Schools, including alternative and charters –Subgroups –Districts –State NCLB requires –Annual goals –Intermediate goals (no more than 3 years apart)

AMOs for Schools ELAMath Elementary or Middle School 13.6%16.0% High School 11.2%9.6%

19 School and District AMOs Elementary/Middle Elementary District School AMOs AMOs Unified District, High District (7-12) AMOs High School High School AMOs District (9-12) AMOs

AMOs for Districts ELAMath Elementary School District 13.6%16.0% High School District (Grades 9-12) 11.2%9.6% Unified or High School District (Grades 7-12) 12.0%12.8%

21 Defining the Starting Point for the AMO’s USE THE HIGHER VALUE Option 2: Statewide % of students proficient in lowest achieving group: - -Economically disadvantaged - -Major racial/ethnic groups - -Disabled students - -ELL Students Option 1: Rank all schools by % proficient Count from bottom up to to reach 20% of total enrollment Percent of students at proficient at that school is the starting point

22 AMO’s: English language arts Elementary and Middle Schools and Elementary Districts

23 AMO’s: Math Elementary and Middle Schools and Elementary Districts

24 AMO’s: English language arts High Schools and High School Districts

25 AMO’s: Math High Schools and High School Districts

26 AMO’s: English language arts Unified Districts and High School Districts with Grades 7/8

27 AMO’s: Math Unified Districts and High School Districts with Grades 7/8

28 Participation Rates 95% required on any assessment used for AYP under NCLB The remaining 5% is the maximum allowable percentage of non- participants, including students who are exempted from testing at parental request.

29 Additional Indicators The API will serve as the “other” indicator for all grades –How would a school meet the “other” indicator? API above the “status bar”, OR Show growth of at least one point Graduation rate will be an additional indicator for high schools –Demonstrate a one-tenth of a percent increase up to 100%

30 The API “Status Bar’

31 Graduation Rate National Center for Educational Statistics (NCES) four year completion rate Progress = increase of one tenth of one per cent per year until the school reaches 100%

32 Graduation Rate High School Graduates, year 4 [ High School Graduates, year 4 + (Grade 9 Dropouts, year 1 + Grade 10 Dropouts, year 2 + Grade 11 Dropouts, year 3 + Grade 12 Dropouts, year 4) ] Four year graduation rate as required by NCLB:

33 Graduation Rate Example / ( ) = 90.9% Grad Rate 120 / ( ) = 91.6% Grad Rate Change in rate: 91.6% % =.7% Met requirement Must increase Grad Rate by at least.1% to meet requirement

34 Safe Harbor Definition Alternate method of meeting AYP if a subgroup is showing progress in moving students from “basic” to “proficient” “All students” is considered a subgroup If a subgroup or the school fails to make the AMO, they may make AYP if: –The percentage of students below proficient decreases by 10% over the prior year –The group has at least 95% participation –The group meets the “other” indicator

35 Safe Harbor Example Year 1Year 2 60% of the students are performing below proficient 54% of students are performing below proficient Met participation rate and other indicator Met AYP 10% of 60% is 6 percentage points +

Where Are Results Counted?

37 NCLB Student Mobility Rules Student was enrolled since CBEDS date Count in school accountability report Count in district accountability report Student was enrolled in more than one school in the same district since CBEDS date Yes No Count in state accountability report No

38 Mobility Definitions Full academic year = Enrollment from CBEDS date to first day of testing “Continuously enrolled” –The student did not withdraw or was not dropped from the school’s (or LEA’s) enrollment any time between the CBEDS census date and the first day of testing” 2002 baseline AYP report will use the current API rule (enrolled in district since CBEDS date) New mobility rules begin go into effect with the August 2003 AYP report (2003 testing cycle)

Other Issues

40 Subgroup Size Reporting will occur for groups with at least 11 students Schools will be held accountable for groups that have: –100 students, OR –50 students that comprise 15% of the student population This rule will apply to schools and districts CDE is pursuing legislation to align API with AYP rules for sub group size

41 Subgroup Definitions All racial/ethnic definitions will remain the same as with the API (collected via STAR) Socio-economically disadvantaged will be used per API definition Students with disabilities included if they have a disability coded on the STAR answer document

42 Subgroups African American (not of Hispanic origin) American Indian or Alaska Native Asian Filipino Hispanic or Latino Pacific Islander White (not of Hispanic origin) Socioeconomically disadvantaged NEW: Students with disabilities English Learners

43 English Learners All students designated on the student answer document as EL (English Learners) or as RFEP (Redesignated Fluent English Proficient). RFEP students will continue to be included until they have attained the proficient level on the CST in ELA for three years consistent with the federal definition of limited English proficient students in paragraph (25) of Section 9101 of Title IX of NCLB.

44 District Accountability Held to same AYP criteria as schools; will be held accountable for all students enrolled in the district for a full year (not just those who aren’t counted at the school level) Districts will receive a 2002 Base report; first AYP report in August 2003 Will be identified for Program Improvement (PI) in the same manner as schools The first year a district could be identified is proposed to be The CDE will provide additional guidance

45 Schools With Fewer than 100 Valid Scores CDE will assume responsibility for establishing AYP for schools with fewer than 100 valid test scores: –Step 1: Apply pairing and sharing for schools with grade spans outside the testing program –Step 2: Aggregate test results across years –Step 3: Apply statistical test to achieve a 95% confidence interval

46 Schools With Fewer than 100 Valid Scores School results with a small number of scores tend to fluctuate For these schools, California’s NCLB accountability plan requires that determination of AYP be based on statistical procedures to adjust for fluctuations These procedures are posted on CDE’s AYP web site at

Identification of Program Improvement Schools and Districts

48 AYP for Title I Schools and Districts  Applies to all schools and districts that receive Title I funds  Title I schools and districts must meet all four components of AYP  Percent of students proficient or above on statewide assessments  Student participation rate in the statewide assessments  API  Graduation rate (high schools)

Title I Schools Identified for PI Did not meet the annual measurable objectives (AMOs) in the same content area (English-language arts or math) in both and or Did not meet any one of the other components of the AYP.

50 Identification of Schools for Program Improvement 2002 data was used ONLY to exit schools from Program Improvement (PI) –See letter from the CDE dated February 6, data will be used to determine AYP for all schools and districts –New schools may enter PI –Schools may advance to later years under NCLB –Schools may exit –(Districts will not enter PI until after )

51 Identification of Schools for Program Improvement NCLB Requirement: – Schools enter PI when they fail to make AYP for two consecutive years In California –A school will enter PI only if the school fails in the SAME content area for two consecutive years (participation rate or AMOs)

52 Title I Districts Identified for PI Did not meet the annual measurable objectives (AMOs) in the same content area for two consecutive years for any significant subgroup or district-wide or The district did not meet any one of the other components of AYP.

53 AYP for School-wide Programs vs. Targeted Assistance For 2002, AYP was evaluated differently for TAS than for SWP schools –For SWP: AYP = meeting school-wide API target and targets for all numerically significant subgroups –For TAS: AYP = meeting API target for socio- economically disadvantaged subgroup only NCLB allows for differential treatment of TAS

54 Appeal Process for PI Schools and Districts A district may appeal on its own behalf or for a school. Appeal must be based on substantive or statistical error (to be defined). 30-days to file appeal and to receive final determination

55 NCLB PI School Requirements Year 1 Program Improvement Revise school plan. Use 10% funds for staff development. Provide school choice with paid transportation. District provides technical assistance (TA).

56 NCLB PI School Requirements Year 2 Program Improvement Continue –Staff development –Choice –District TA Add –Supplemental services/tutoring

57 NCLB PI School Requirements Year 3 Corrective Action Continue –District TA –Choice –Supplemental services Add –District corrective action

58 NCLB PI School Requirements Year 4 Corrective Action Continue –District TA –Choice –Supplemental services Add –Development of plan for alternative governance

59 NCLB PI School Requirements Year 5 Restructuring Implement alternative governance plan –Reopen as charter. –Replace staff. –Contract with external entity. –Takeover by state.

60 PI District Requirements Year 1 Program Improvement Revise LEA Plan. Use 10% funds for staff development. Target students not making AYP. Provide extended learning opportunities. Involve parents. Receive TA from state.

61 State takes one corrective action: –Reduce funds; –Institute new curriculum and staff development; –Replace personnel; –Appoint trustee; –Restructure LEA; Authorizes choice and one of the above actions. PI District Requirements Year 2 Implement Plan Year 3 State Corrective Action

62 PI Schools Identified Prior to NCLB Placed in Year 1, Year 2, or Year 3 for the entire school year, in accordance with NCLB.

63 Options for Existing PI Schools (Years 1 and 2) in Schools will exit PI status: –Made AYP in 2002 and Schools will remain in place: –Made AYP in Schools will advance to the next level under NCLB: –Did not make AYP in 2003.

64 Options for Year 3 Corrective Action Schools Identified for school year. –Made AYP in 2002 and 2003, will exit PI. –Did not make AYP in 2003, will advance to Year 4. –Made AYP in 2003, will remain in Year 3. Identified for school year. – All will remain in Year 3 for

65 After August 2003 AYP Report New PI schools must move immediately to implement school choice. Advancing PI schools must move immediately to implement new requirements. PI schools remaining at the same level must continue required activities.

66 After October 2003 AYP Report Schools that made AYP for August Report, but did not make AYP for October Report, must immediately move to implement the requirements of NCLB.

67 After December 2003 AYP Report Schools that made AYP for August and October reports, but did not make AYP for Final December Report, must immediately move to implement the requirements of NCLB.

Timeline for AYP Information

Baseline AYP Reports  Starting points for each school and district.  CDE will post on the Internet on July 22,  Districts and schools may determine whether they are at risk for not meeting AYP criteria when the 2003 report is released in August.

AYP Reports  CDE will post 2003 AYP reports on the Internet on August 15,  Reports will include the “percent proficient” and participation rates based on 2003 testing data.  Will be used to identify districts and schools that are not making AYP for 2003.

Reporting Timeline 2002 Base AYP Report December 2003 Final AYP Report August 15 July 22 October 2003 Growth API Report 2003 AYP Report September November

72 Notification of AYP Status Base 2002 AYP Report –Early summer –Districts can identify schools at risk for failing AYP in 2003 –Districts can plan and prepare for possible PI identification and implementation Phase 1 AYP Report: –August 15, 2003 –Attainment of AMO’s (i.e. percent proficient) and participation rate

73 Notification of AYP Status Phase 2 AYP Report : –October 2003 –2003 Growth API’s and high school graduation rates Phase 3 AYP Report (Final) : –December 2003 –Final AMO’s and participation rates, APIs for schools making data corrections, and application of “safe harbor” to all schools and subgroups

2002 Base AYP Results

75

Effects on Current Statewide Accountability System

77 How Will the API be Affected? Remain the same: –Statewide target (800) –Base-growth cycle –Calculation of the index and targets –Schedule of reporting –Timeline for inclusion of new assessments Changes*: –Addition of two new subgroups (ELs and students with disabilities) –Change in subgroup size –Increase in participation rate for high schools to 95% –Mobility rule *Subject to legislation

78 Are API Growth Targets Still Important? Attainment of API growth targets will affect eligibility for awards Still criteria for exiting state intervention programs (e.g., II/USP) YES!!!!

79 Importance of the CAHSEE Contributes 15% of the API weight for high schools The grade 10 census administration is the basis of AYP for high schools Need 95% minimum participation rate for AYP

80 How Will State Awards and Interventions be Affected? Legislation has been introduced to align the API methodology with the AYP requirements The CDE is in the early stages of planning to align state and federal interventions and sanctions Eligibility and priority for awards and interventions/sanctions will be based on making both AYP and API

81 For More Information Questions related to AYP –Evaluation Unit at (916) or at Questions related to API or the AYP Reports –EPIC Unit at (916) or at Questions related to Program Improvement –Title I Policy and Partnerships Office at (916) or

82 New AYP Internet Sites AYP Reports Letters, Memos, Informational Materials

83 This presentation is available on-line at: under “Presentations”