Kaliko Oligo Senior Consultant ACT Mountain/Plains Region Kansas ACT Workshop Southwest Kansas Kaliko Oligo Senior Consultant ACT Mountain/Plains Region

Workshop Agenda About ACT The ACT College Readiness System The Foundation National Curriculum Survey® College Readiness Standards™ College Readiness Benchmarks The Data Student Score Report Profile Summary Report Item Response Summary Report Early Intervention Rosters Questions & Resources

Helping people achieve education and workplace success. About Us Not-for-profit Education and the workforce 50+ years worth of functional data Research agenda Customer service Almost 20 years of data on non-self selected student data Our Mission Helping people achieve education and workplace success.

College and Career Readiness System MEASURING STUDENT PROGRESS TOWARD READINESS IMPROVING COURSE RIGOR SUPPORTING SOLUTIONS PLANNING SCHOOL IMPROVEMENT EXPLORE 8th and 9th grade curriculum-based educational and career planning program PLAN 10th grade curriculum-based educational and career planning program The ACT 11th and 12 grade curriculum-based assessment for learning outcomes ENGAGE Middle and high school assessment that measures all factors of academic success QualityCore Research-driven solutions for strengthening curriculum CoreWork Diagnostics Online service to diagnose and improve content and practice areas Core Practice Audit Framework for evaluating current practices

The New ACT Assessment System Vertically-articulated, internationally benchmarked, standards- based system of assessments highlighting progress toward College Readiness as well as the Common Core Summative assessments on a vertical scale that spans Grades 3-10 and links to ACT’s College Readiness score scale (1-36) Computer delivered (with paper-and-pencil options) Fast, online, actionable reporting ACT’s next generation system is designed to address the expressed needs of states (mentioned on previous page), as well as: The system will have a vertically-articulated, standards-based system of assessments to ensure that there is alignment within a subject and skills build upon each other in a logical order from one grade to the next. The assessment system will be constructed using evidence-based design to provide reliable indicators of students’ college and career readiness. ACT’s evidence-driven design serves as the solid foundation for our claim that our assessments provide a strong indicator of college and career readiness. The new system will be aligned with the Common Core State Standards, which draw heavily from ACT’s College Readiness Standards, Benchmarks and research. The system will also utilize international benchmarks. Recent ACT research has established that, in mathematics and reading, the standard of college and career readiness represented by our College Readiness Benchmarks is competitive with the performance standard set by the highest-performing nations in the world ACT. (1) ACT will offer the next generation assessment system using a state of the art platform; however we understand the critical need to offer paper and pencil options. Technology will enable fast actionable reporting including the ability to analyze scores on constructs such as critical thinking, problems solving, and text complexity.

Longitudinal Assessments English, math, reading, science, optional writing test (ACT only) Career and educational components EXPLORE 8th–9th Grade Score Scale: 1—25 PLAN 10th Grade Score Scale: 1—32 The ACT 11th–12th Grade Score Scale: 1—36

Common Score Scale Relationship 5 10 15 20 25 30 35 40 EXPLORE PLAN ACT 36 32 25

Kansas ACT 2011-12 Data KS at a Glance: 23,907 graduates took the ACT 81% of graduates took the ACT 22,216 10th-grade students took PLAN 11,017 8th-grade students took EXPLORE www.act.org/readiness/2012

What is College Readiness? College Readiness is the level of preparation a student needs to be equipped to enroll and succeed-without remediation-in a credit-bearing first-year course at a two-year or four-year institution, trade school, or technical school. www.act.org/commoncore

College and Career Readiness System The Foundation National Curriculum Survey® College Readiness Standards™ College Readiness Benchmarks

ACT National Curriculum Survey® Determines what skills and knowledge postsecondary institutions expect Measures college-ready skills Surveys completed nationally Consultation with content area experts Every 3-5 years Middle school, secondary, & postsecondary teachers surveyed What’s currently being taught & considered important for college success Identifies gaps Disseminates results to inform P-16 alignment efforts Validates and updates assessment programs http://act.org/research/curricsurvey.html

College Readiness Standards™ Direct link between what students have learned and what they are ready to learn next. Help interpret the meaning of EXPLORE, PLAN and the ACT scores. Identify the knowledge and skills students are likely to demonstrate at various score ranges on each academic test. Script Follow slide. Make sure to use some of the slide information as a guide. Slide Information Spending time on CRS again for a couple of reasons. New educators and counselors who are new to CRS 2. This yr CRS reports are included in with E/P standard packages and we want to schools and districts make the best use of this information 3. We want to share examples of how users are incorporating this information in their process CRS provides educators with more than just a score. The score relates to empirically derived standards that identify a student’s proficiency in the four core subject areas tested on the ACT. They also provide information as to what the student needs in order to increase proficiency, and therefore their ACT score in a particular subject area. Background Review College Readiness Standards: http://www.act.org/standard/ Traps Worries about teaching to the test? You should teach to the standard, you won’t have to teach to the test. By teaching to the standard, students should then do better on the test. Again, the ACT tests for student proficiency in the four core subject areas. The more schools, teachers, etc. teach to the standard (and therefore provide rigorous courses at all levels) the more likely a student will achieve a satisfactory level of proficiency in a subject area and thus perform better on the test. Remember that ACT solutions are based on a national curriculum survey. http://act.org/standard/

College Readiness Standards Mathematics 80% of students who achieve a score in this range demonstrate these skills Score Range Basic Operations & Applications Probability, Statistics, & Data Analysis Numbers: Concepts & Properties 13-15 Standards Perform one-operation computation with whole numbers and decimals Solve problems in one or two steps using whole numbers Perform common conversions (e.g., inches to feet or hours to minutes) Calculate the average of a list of positive whole numbers Perform a single computation using information from a table or chart Recognize equivalent fractions and fractions in lowest terms Ideas for Progress Investigate and build understanding of the concept of percentage as a comparison of a part to a whole use multiple operations to solve multistep arithmetic problems solve real-world problems that involve measures of central tendency (e.g., mean, median, mode) interpret data from a variety of displays (e.g., box-and-whisker plot) and use it along with additional information to solve real-world problems conduct simple probability experiments and represent results using different formats recognize and apply place value, rounding, and elementary number theory concepts Suggestions to progress to a higher level of achievement 13 13

Standard from Math, 16-19, Measurement Strand: Compute the perimeter of polygons when all side lengths are given. EXPLORE Which of the following is a general expression for the perimeter of the right triangle below, in miles? z miles y miles x miles This is an example of how we might measure this skill with an EXPLORE question. Note: Drew a diagram. This in itself makes the question easier Labeled the sides No computation required We are measuring whether the student understands the concept. A. x + y + z B. 2(x + y) C. D. E. xy

Standard from Math, 16-19, Measurement Strand: Compute the perimeter of polygons when all side lengths are given. PLAN What is the perimeter, inches, of a square whose sides each measure 5 inches? This is an example of a test item measuring the same skill, but written for PLAN. Notice: No diagram. However, they can draw their own if they wish. No labeled sides Students have to know that a square has 4 sides – 4 equal sides They can add the fraction 4 times or they can multiply by 4, whichever is easier for them It is also appropriate to use estimation to get the answer.

Standard from Math, 16-19, Measurement Strand: Compute the perimeter of polygons when all side lengths are given. The ACT The out-of bounds lines around a basketball court in Central Park need to be repainted. The court is a rectangle 90 feet long and 50 feet wide. What is the perimeter, in feet? A. 140 B. 190 C. 230 D. 280 E. 4500 This is an example of a test item measuring the same skill, but written for the ACT. On the surface, it looks easier. The arithmetic is very easy. But this requires a higher level of reasoning than the other two. Remember, this is only at the 16-19 range. Common mistakes? Get only ½ the perimeter? Get the area instead of the perimeter? Overdependence on calculators?

How do you define and measure college and career readiness?

ACT’s College Readiness Benchmarks Trap: Biology as the standard. This decision was based on our National Curriculum Survey of over 200,000 middle school, high school and postsecondary educators and administrators that identified Biology as the ‘entry-level’ Science course. Trap: How stable are the benchmarks? The benchmarks are empirically derived and are based on course grades earned by more than 90,000 students in a nationally represented population of both two year and four year postsecondary institutions. They are nationally normed, so they cannot determine probability of success at a particular institution (ACT’s Course Placement Service can assist 2 and 4-year institutions in creating institutional specific benchmarks), but they provide a good overall view of a student’s college readiness. Trap: Why do the Benchmark scores vary? This was research based and ACT made choices on where to draw the lines- guidelines. Based off of the National Curriculum Survey, ACT also determined the levels of proficiency that initially indicate a higher probability of success in the four core subject areas, and therefore determined the specific benchmark scores for each. Trap: Placement vs. Benchmark The highlighted benchmarks on this slide provide an overall view of a student’s readiness for college. As mentioned before, these are nationally normed benchmarks, and in order to use such scores for placement at a particular college/university, an institution would have to work with ACT to complete a Course Placement Service that would provide schools with institutional specific benchmarks for use in placement. Test College Course PLAN ACT 8th Grade 9th Grade English English Composition 13 14 15 18 Math College Algebra 17 19 22 Reading Social Sciences 16 21 Science Biology 20 24 EXPLORE Empirically derived 50% chance of achieving a B or higher or about a 75% chance of achieving a C or higher in the corresponding credit-bearing college course

The Data Use your assessment data to: Provide timely interventions Inform instructional needs Monitor longitudinal growth Measure progress toward district goals Counsel students effectively (college & career)

The Student Score Report EXPLORE, PLAN, and the ACT

Student Score Reports Components Foundation of all aggregate reporting Academic strengths and weaknesses Career and postsecondary aspirations College readiness indicators Foundation of all aggregate reporting

The Interest Inventory

The ACT High School Score Report

How are you using your student score reports?

The Profile Summary Report EXPLORE, PLAN, and the ACT

What is in the Profile Summary Report? The information in this report is organized into tables with guiding questions at the top. These tables will help you: understand your students’ college & career readiness determine how to assist and enhance their success

Frequency Distribution Table Intervention Exercise EXPLORE: Page 2 in Profile Summary Report PLAN: Page 2 in Profile Summary Report ACT: Page 12 in the Profile Report

-19 within 2 points of benchmark 188 Total Students Frequency Distribution Table Where we want students to be 140 Students Above Benchmark Students on track to be college ready 50 Students on the Cusp -31 in danger of slipping -19 within 2 points of benchmark Students on the cusp 8th Grade English College Readiness Benchmark Benchmark Students in real need of remediation 48 Students Below Benchmark

Strategies to help the 31 students in danger of slipping www.act.org/standard

Frequency Distribution Table Application Exercise

College Readiness Standard Score Ranges EXPLORE: Page 4 in Profile Summary Report PLAN: Page 4 in Profile Summary Report ACT: Page 9 in Profile Summary Report

College Readiness Standard Score Ranges Questions to consider: In which ranges are the majority of your students? Given these score ranges, where are your students most proficient? Given these score ranges, where are your students least proficient? How will you incorporate this information into your curriculum?

` What are the future educational plans of your students? How can you encourage and support these educational plans?

Additional ACT Profile Report Tables

The Item Response Summary Report EXPLORE and PLAN

Item Response Summary Report Provides data to administrators, teachers, and counselors relative to student performance on every English, math, reading, and science question. When used along with the test booklet, the Item Response Summary Report becomes a very useful tool for curriculum review. Slide Information Customize to the presenter 38

The Item Response Summary Report shows you: the percentage of your students who selected the correct response to each item. the percentage of your students who selected each incorrect response. the percentage who did not answer the item. the average percentage of students who responded correctly to the items in each content area.

Given your curriculum, is the percentage of your report group answering each item correctly and consistent with your expectations? Is a large percentage of your report group choosing incorrect response options?

Notice how every other question alternates in lettering. Sample math ACT questions from: http://actstudent.org/sampletest/math/math_01.html

How did our students perform on each item? *Asterisked numbers are CORRECT answers.

EXPLORE/PLAN Test Booklet What specifically were the hard/easy questions for students? Bring the reports and test booklets back to your school/district and discuss with faculty. Color-Code question numbers in the Test Booklet to match what you have done on the Item Response Summary Report. Once that is done, teachers have a quick reference for which items were weak/strong for your students.

Curriculum Alignment Assists in organizing teaching practices Offers sequence for delivering content Provides clear scope of what must be taught to all students Curriculum Alignment ... a process of interpreting learning standards and developing learning objectives that are targeted directly to the standards. According to Norm Webb (1997) curriculum alignment can be defined as the degree to which expectations and assessments are in agreement and serve in conjunction with one another to guide the system in assuring that students learn what they are expected to know and do. In an aligned system all content standards must be accounted for in some manner (Mitchell 1996).

Curriculum Review Worksheets www.act.org/standard/instruct

Resources ACT Educator Site Educator Resources EXPLORE Student Site http://www.act.org/education/ Educator Resources http://www.act.org/education/resources.html EXPLORE Student Site http://www.actstudent.org/explore/ PLAN Student Site http://www.actstudent.org/plan/ ACT Student Site http://www.actstudent.org/ ACT Research www.act.org/research Online resources

Q&A

Your Kansas ACT Team Lisa Wolf Senior Consultant, Program Solutions ACT Mountain Plains Region 3131 South Vaughn Way Suite 218 Aurora, CO 80014 P) 303.337.3273 | F) 303.337.2613 joe.cruse@act.org Kaliko Oligo Senior Consultant, Client Outreach ACT Mountain Plains Region 3131 South Vaughn Way Suite 218 Aurora, CO 80014 P) 303.337.3273 | F) 303.337.2613 kaliko.oligo@act.org