College and Career Readiness (CCR) A-F School Grading The A-F school grading system provides accountability on how schools and districts are meeting the needs of New Mexico’s diverse student population. The report card presenting each school’s grade provides a tool for educators, parents, and the public to see what is working well and what needs additional work. This module on college and career readiness is one in a series that explains the components of the School Grading system. College and Career Readiness (CCR)

What is College and Career Readiness (CCR)? 4-year graduation cohort members Participation and success 13 eligible programs A minimum of 3 records Shared accountability College and Career Readiness scores are determined by the percentage of 4-year graduation cohort members who show evidence of participating in college or career preparation, along with the proportion of those students meeting a benchmark. Evidence of participation and success is established through any of the 13 indicators available to high school students. CCR is computed for all high schools that have members of the 4-year graduation cohort for the assessment year. Cohorts must have a minimum of three student records to be included. Like the calculation of graduation rates, CCR calculations use a shared accountability method. The shared accountability model proportionally distributes a student’s CCR participation and performance across all the high schools that the student attended.

Eligible Programs AccuPlacer International Baccalaureate (IB) ACT PSAT ACT Aspire SAT Advanced Placement (AP) SAT Subject Tests Career Program of Studies/Career and Technical Education ASVAB TABE Concurrent Enrollment/Dual Credit WorkKeys SAM Schools The eligible 13 programs include AccuPlacer, ACT, ACT Aspire, Advanced Placement, Career and Technical Education, Concurrent Enrollment/Dual Credit, International Baccalaureate, PSAT, SAT, and SAT Subject Tests. Schools evaluated with the Supplemental Accountability Model, called SAM schools, are allowed to use additional indicators: ASVAB, TABE, and WorkKeys. All students enrolled in grades 9 through 12 are eligible for participation in one or more of these programs. We’ll take a closer look at these programs one by one at the end of this presentation.

Participation and Success Participation – 5 points Any single student attempt Single credit for a single student's participation SAM schools: ASVAB, TABE, WorkKeys Success – 10 points Any single student success Single credit for a single student’s success School grading cut points Derived in 2012 Held stationary since then Participation can yield a total of 5 points. Any single student’s attempt within the four years of high school is counted towards participation. Students may make multiple attempts, with multiple indicators, in multiple years. However, this results in only a single credit for participation. In school grading, up to 10 total points can be earned for successful performance on a CCR indicator. The success rate follows the same calculation as participation. Students who achieve any one or more of the benchmarks or higher are considered successful in the numerator, whereas students who attempt any program or assessment form the denominator. The success rate is the percent of participants that succeeded. Just like participation, students may achieve one or more benchmarks on multiple tests, but each student’s success is counted only once. Using participation and success together, CCR makes up a total of 15 points in the high school’s overall grade. The school grading cut points for participation and success were established in 2012 and have remained stationary since then.

School Grading Report Card A: 11.3 or above B: 9.7 to 11.2 C: 8.2 to 9.6 D: 6.7 to 8.1 F: 6.6 or below Now let’s take a look at the school grading report card. To earn an A on the CCR indicator, a high school needs to have 11.3 points or above. To earn a B, a high school needs to have 9.7 to 11.2 points. To earn a C, a high school needs to have 8.2 to 9.6 points. To earn a D, a high school needs to have 6.7 to 8.1 points. And finally, if a high school has 6.6 points or lower, it will receive an F on the CCR indicator. In this example, this high school received a 5 out of 5 for participation, and 6.8 out of 10 for success. Altogether, this school received 11.8 points, and therefore this school earned an A on the CCR indicator on the school grading report card.

School Grading Report Card The school grading report card also shows CCR participation and success rates for student subgroups: females, males, Caucasians, African Americans, Hispanics, Asians, American Indians, economically disadvantaged students, students with disabilities, and English learners. The dark blue shading indicates that a given subgroup had a CCR participation rate of 50% or higher. In this example, we can see that 70% of the African American students at this high school participated in the test AccuPlacer and 63% of them participated in Career and Technical Education. The medium blue shading indicates that a given subgroup had a participation rate between 20% and 50%. We can see that 48% of the American Indian students took ACT, and 29% of them participated in Career and Technical Education. Finally, the light blue shading indicates that a given subgroup had a participation rate that is below 20%. In this example, only 3% of the English learners at this school took AccuPlacer.

Count of Snapshots in School Count of Snapshots in State CCR Computation Student High School Count of Snapshots in School Count of Snapshots in State Fraction for that School CCR Attempted CCR Success CCR Participation Numerator Mateo Pine 4 16 0.25 Y Elm 8 0.50 Oak Mia Cedar 2 10 0.20 N 0.00 5 3 0.30 How are CCR points calculated? This example shows how two students, Mateo and Mia, contributed to the CCR rates for the multiple high schools they attended during their 4-year tenure. Note that the attempt, which is counted as participation, and success flags are distributed to all schools the students ever attended. This means that all high schools are held accountable for their students’ college and career readiness no matter the school’s grade configuration or the mobility of the student. Let’s take a look at how CCR participation and success rates are calculated for Pine High School. Both Mateo and Mia attended this high school. However, we can see that Mateo only spent one quarter of his time at Pine High School, because only 4 out of his 16 snapshots were associated with Pine High School. Because Mateo participated in CCR tests and passed benchmarks for success, he is counted towards both CCR participation and success. Mia, on the other hand, spent half her high school career at Pine High School, because 5 out of her 10 snapshots were associated with Pine High School. She participated in CCR tests but did not pass the benchmarks for success, so she is only counted towards CCR participation. To measure Pine High School’s CCR participation, we add its two students, Mateo and Mia, together. For the denominator, we will add Mateo’s fraction, 0.25, with Mia’s fraction, 0.5. For the numerator, we will also add those two numbers together, because both Mateo and Mia participated in the CCR test. That gives us 1, showing that 100% of Pine High School’s students participated in the CCR tests. The denominator for CCR success is the same, 0.75. The numerator is 0.25, this is because only Mateo passed benchmarks for success. When we divide 0.25 by 0.75, we get 0.33, which indicates that 33% of Pine High School’s students succeeded in the CCR tests. Note that multiple attempts or multiple successes do not change the student’s contribution to the calculation. The student’s single best effort is used and distributed to all high schools he or she attended. Pine High School Participation: 0.25 + 0.5/0.25 + 0.5 = 1 Success: 0.25 + 0/0.25 + 0.5 = 0.33

CCR Points Computation Standardation: StandardizedCCRRate z = (CCRRate - MeanCCRRate) / (SDCCRRate) Transform z into a probability that can range from 0 to 1: CCRRate = CDF(z) Multiply CCRRatej by the maximum number of points for the indicator (5 for participation and 10 for success) Participation: Points = NormalizedCCRParticipationRate X 5 Success: Points = NormalizedCCRSuccessRate X 10 Add together To translate CCR participation and success into points for school grading, the same statistical procedures are followed. Let’s use participation as an example. The first step is to standardize CCR participation rates by dividing the values by the standard deviation, which will give us a z score for each high school. If a school’s z score is 0, this means that the school’s CCR participation rate is the average rate of all high schools across the state. The second step is to transform standardized residual values, which are schools’ z scores, into probabilities ranging from 0 to 1 using the cumulative normal distribution function. If a high school’s z score is 0, then its z score will be converted into a probability of 0.5. The last step is to multiply the converted numbers by the maximum numbers of points for the indicator, which is 5 points for participation and 10 for success. Finally, points for participation and points for success are added together, and that’s the CCR points a high school will receive on the school grading report card.

Eligible Programs AccuPlacer International Baccalaureate (IB) ACT PSAT ACT Aspire SAT Advanced Placement (AP) SAT Subject Tests Career Program of Studies/Career and Technical Education ASVAB TABE Concurrent Enrollment/Dual Credit WorkKeys SAM Schools The rest of this module will introduce all of the 13 eligible CCR programs. Contact information for any questions about the CCR indicator on the school grading report card can be found on the last slide of this presentation.

AccuPlacer Computer-adaptive college placement test Placing students in appropriate courses Benchmarks for success College-Level Mathematics: 50 Elementary Algebra: 80 Reading Comprehension: 82 Sentence Skills: 83 WritePlacer: 6 AccuPlacer is a computer-adaptive college placement test offered by the College Board that helps institutions of higher education place students in appropriate courses. Questions are chosen for each student on the basis of their answers to previous questions. The Accuplacer assesses Reading, Mathematics, and Writing.

ACT College admissions examination Benchmarks for success Mathematics: 22 English Composition: 18 Reading: 22 Science: 23 ACT is another widely used national college admissions examination. The ACT yields scores in four areas, English, Mathematics, Reading, and Science, and offers benchmark scores that indicate college readiness in each.

ACT Aspire 10th graders Guide career options Predicts outcomes on the ACT Benchmarks for success Mathematics: 432 Science: 432 English: 428 Reading: 428 Writing: 428 ACT Aspire, an assessment for 10th graders, is designed to utilize a student’s achievement to guide career options and the remaining years of high school. ACT Aspire has the capability to predict outcomes on the ACT. The test assesses English, Mathematics, Reading, Science, and Writing.

Advanced Placement (AP) National qualifying examination Aligned to 34 college-level courses No credit for enrolling in high school AP classes Participation and success in the national exam Benchmarks for success: 3 AP, Advanced Placement, is a national qualifying examination aligned to 34 college-level courses. Most four-year colleges grant students credit, advanced placement, or both on the basis of the score on the AP exam for that subject. Students do not get credit for only enrolling in a high school AP class. They must demonstrate participation and success in the national exam.

Career Program of Studies/ Career and Technical Education High school courses Industry-recognized certification Complete all coursework with a C or better Graduate with a regular diploma Career Program of Studies or Career and Technical Education is a sequence of high school courses that are identified as leading to industry-recognized certification. To be considered successful, the student must complete all coursework with a C or better and graduate from high school with a regular diploma.

Concurrent Enrollment/Dual Credit Accredited NM post-secondary institutions College credits Post-secondary preparation Non-remedial Benchmarks for success C or better Concurrent Enrollment/Dual Credit in an accredited New Mexico post-secondary institution offering college credit is counted as evidence of post-secondary preparation. All courses that are nonremedial are counted.

International Baccalaureate (IB) Standardized and enhanced high school curriculum Six study areas Originated in Sweden Recognized internationally Benchmarks for success: 4 IB, or the International Baccalaureate program of studies, is a standardized and enhanced high school curriculum where students must demonstrate competency in six study areas or earn an IB diploma. The program originated in Sweden and grants credentials that are recognized outside the U.S.

PSAT Two age groups Benchmarks for success Sophomores and younger, juniors and older Benchmarks for success Before November 2015 Mathematics: 47 Critical Reading, Writing: 45 After November 2015 Mathematics: 480 Evidence Based Reading & Writing: 430 PSAT is the Preliminary SAT and is co-sponsored by the College Board and the National Merit Scholarship Corporation. This assessment yields scores in evidence-based Reading and Writing and in Mathematics and offers benchmark scores that indicate college readiness in two age groups, sophomores and younger, and juniors and older.

SAT College admissions examination Benchmarks for success Before March 2016 Mathematics, Critical Reading, Writing: 500 After March 2016 Mathematics: 530 Reading & Writing: 480 SAT is a widely used college admissions examination that measures the skills in Mathematics, Reading, and Writing.

SAT Subject Tests Standardized subtests that complement the SAT Taken to improve a student’s credentials One-hour long Available in 21 subjects Benchmarks for success Vary by subject See the A-F School Grading Technical Guide. SAT Subject Tests, standardized subtests that complement the SAT, are usually taken to improve a student's credentials for admission to colleges in the United States. Each test is timed at one-hour, and tests are available in multiple subjects related to a student’s interests or a college’s requirements. The 21 subject area tests are listed in the A-F School Grading Technical Guide on PED’s website.

Armed Services Vocational Aptitude Battery (ASVAB) Composite score: AFQT score Identify occupations Used to qualify for enlistment Allowable only for designated SAM schools Benchmarks for success: 31 ASVAB, the Armed Services Vocational Aptitude Battery, provides a composite score called the Armed Forces Qualification Test (AFQT) score. The ASVAB scores identify occupations that best suit a candidate’s abilities and can be used to qualify for enlistment. This test is allowable only for designated SAM schools.

Test of Adult Basic Education (TABE) Measures a person’s grade level Allowable only for designated SAM schools Benchmarks for success Mathematics: 506 Reading: 518 Writing: 524 TABE, Test of Adult Basic Education, is an assessment that measures a person’s grade level in reading, mathematics, and language. This test is allowable only for designated SAM schools.

WorkKeys Job skill assessment Allowable only for designated SAM schools Benchmarks for success Applied Mathematics/Reading for Information: 5 Listening for Understanding/Teamwork/Location Information: 4 Business Writing/Applied Technology: 3 WorkKeys is a job skills assessment created by ACT and is used by businesses to measure workplace skills of job applicants and by schools and colleges to help prepare students for the workplace. This test is allowable only for designated SAM schools and assesses Mathematics, Reading, Writing, Science, and Social Studies.

Questions? For additional information, please contact Yun Yao, Ph.D., Statistician yun.yao@state.nm.us That’s all for this training module. If you have any questions, please do not hesitate to contact Yun Yao at the email address shown. We also have several documents that provide additional information on CCR computation, such as the “School Grading Technical Guide.” All of these documents are available on PED’s website. Thank you for your attention.