NAU GEAR UP EXPLORE Interpretation Workshop Getting the Most Out of Your EXPLORE Data Cassi Barker-Carr January 2013

Stephanie Lewis Director, Program Solutions ACT, Inc., West Region Office /Cell

Workshop Objectives  Learn how to maximize interpretation of your data  Find out what your data is telling you  Create an communication/action plan

ACT’s College and Career Readiness System

ACT National Curriculum Survey ®  Determines what skills and knowledge postsecondary institutions expect  Measures college-ready skills  Surveys completed nationally  Consultation with content area experts

Longitudinal Assessments College Readiness System Scores

MEASURING STUDENT PROGRESS TOWARD READINESS IMPROVING COURSE RIGOR SUPPORTING SOLUTIONS PLANNING SCHOOL IMPROVEMENT EXPLORE 8th and 9th grade curriculum-based educational and career planning program PLAN 10th grade curriculum-based educational and career planning program The ACT 11th and 12th grade curriculum- based assessment for learning outcomes ENGAGE Middle and high school assessment that measures all factors of academic success QualityCore Research-driven solutions for strengthening curriculum CoreWork Diagnostics Online service to diagnose and improve content and practice areas Core Practice Audit Framework for evaluating current practices Designed to help 8th and 9th graders explore a broad range of options for their future, EXPLORE is a curriculum-based educational and career planning program that measures achievement in English, math, reading, and science. As an early indicator of college readiness, EXPLORE gives educators the means to structure high school planning and career exploration for students and parents. EXPLORE

 Grades 8 and 9  English, Mathematics, Reading, and Science  Total time for tests: 2 hours  Needs Assessment  Plans and Background Information  UNIACT Interest Inventory

Assessment Literacy ACT College and Career Readiness Standards Statements describing the knowledge and skills typically demonstrated by students who score in a particular score range CoreMinimum number of high school courses recommended to prepare for college NormsIndicate the percent of examinees in the norming study who received the same scale score or a lower scale score

Assessment Literacy ACT College Readiness BenchmarksThe minimum English, reading, mathematics, and science assessment scores required for student to have a 50 percent change of obtaining a B or higher or about a 75 percent chance of obtaining a C or higher in the corresponding credit-bearing college course Percentilesthe value of a variable below which a certain percent of variables fallpercent

Communication Defined  Exchange or sharing of information to all people involved in the stages of implementation

Why Important  Help prevent poor performance by planning ahead  Ensures action is taken  Avoid assumptions  Increase stakeholder buy-in

Students Need to Know  Purpose, significance, what is being measured, how they can use the results in planning their high school career and how the school will use the assessment results

Parent/Guardian Needs to Know  Purpose, significance, what is being measured and how the school will use the assessment results

Educators Need to Know  Purpose, significance, what is being measured  How the results can be used –Teaching and guiding individual students –Curricular improvement –Instructional improvement

Dissemination of Results Table

Student Score Report  An individualized score report detailing scores received on the assessment, career plans and needs indicated by the student.

Student Score Report  The Student Score Report will provide: suggestions for improving student academic skills careers that match student interests indicators of college readiness

Student Score Report

Scan Date Student ’ s Grade Level Student ’ s School School EXPLORE ® Code Test Date

Student Score Report

Questions to Consider:  What can norms tell you about this student?  How does this student compare with other students nationally?

Student Score Report

 Help interpret what the scores earned in EXPLORE, PLAN, and the ACT mean  Identify the knowledge and skills students are likely to demonstrate at various score levels on each academic test

Student Score Report

Questions to Consider:  How can your school meet the needs of what the student is asking for?  Are you already addressing these?

Student Score Report Question to Consider:  How does the student ’ s courses and scores relate to their educational plans and career choices?

Student Score Report Questions to Consider:  Is the student on track to be college and career ready?  What course of action can be taken with the student to increase their college readiness?  Where does the student need intervention to get on track?

ACT’s College Readiness Benchmarks  Empirically derived  50% chance of achieving a B or higher or about a 75% chance of achieving a C or higher in the corresponding credit-bearing college course TestCollege CoursePLANACT 8th Grade9th Grade EnglishEnglish Composition MathCollege Algebra ReadingSocial Sciences ScienceBiology EXPLORE

ACT’s College Readiness Standards Direct link between what students have learned and what they are ready to learn next.  Help interpret what the scores earned in EXPLORE ®, PLAN ® and the ACT ® mean  Identify the knowledge and skills students are likely to demonstrate at various score levels on each academic test

College Readiness Standards Score Range Basic Operations & Applications Probability, Statistics, & Data Analysis Numbers: Concepts & Properties Standards  Perform one-operation computation with whole numbers and decimals  Solve problems in one or two steps using whole numbers  Perform common conversions (e.g., inches to feet or hours to minutes)  Calculate the average of a list of positive whole numbers  Perform a single computation using information from a table or chart Recognize equivalent fractions and fractions in lowest terms Ideas for Progress  Investigate and build understanding of the concept of percentage as a comparison of a part to a whole  use multiple operations to solve multistep arithmetic problems  solve real-world problems that involve measures of central tendency (e.g., mean, median, mode)  interpret data from a variety of displays (e.g., box-and-whisker plot) and use it along with additional information to solve real-world problems  conduct simple probability experiments and represent results using different formats  recognize and apply place value, rounding, and elementary number theory concepts Mathematics And statements that provide suggestions to progress to a higher level of achievement Statements that describe what students are likely to know and be able to do...

Student Score Report

You and the World of Work Your Interests Exploring Career Options

Student Score Report

Dissemination of Results Table

Profile Summary Report What is the Profile Summary Report (PSR)?  summarizes the performance of your students  available at the school and district level

Profile Summary Report  compare your students’ score and subscores with those of students nationally  measure if your students are On Track to be college ready when they graduate from high school  differentiate scores by ethnic and gender groups  show how our students respond to the local items

Profile Summary Report  relate student composite scores and coursework plans: –to their educational plans –to their expressed needs for help –postsecondary plans to their career preferences from the Career Areas List. –postsecondary plans relate to their career clusters from the World-of-Work Map.

Let’s Look at Table 1c in the Profile Summary Report

Profile Summary Report Table 1c –Questions to Consider  How does your local ‘% At or Above Benchmark’ in each content area compare to the national %?  In which ranges are the majority of your students?  Given these score ranges, in which College Readiness Standards are your students most proficient?

Let’s Look at Tables 3 & 4 in the Profile Summary Report

Questions to Consider – Table 3  What are the future educational plans of your students?  How can you encourage and support these educational plans?

Questions to Consider – Table 4  What do the majority of your students list as areas of needed assistance?  What resources are available to address their expressed needs?

Dissemination of Results Table

Item Response Summary Report  Describes the item-by-item performance of your examinees.  Determine your students’ academic strengths and weaknesses relative to the skills and knowledge measured by the test items, and address apparent weaknesses at the content area level.

Item Response Summary Report  percentage who selected the correct response to each item  percentage who selected each incorrect response  percentage who did not answer the item  the average percentage who responded correctly to the items in each content area

Item Response Summary Report Student Response Information

Item Response Summary Report Given your curriculum, is the percentage of your report group answering each item correctly and consistent with your expectations? Is a large percentage of your report group choosing incorrect response options?

Item Response Summary Exercise Part 1 1.Locate the asterisked number in each Item Number. –If highest number, circle it green. –If not the highest number, circle it red. 2. If asterisked number is circled green, look at the Reference Group Percentage. If this number is higher than your green circled number, circle the Reference Group Percentage in red. 3. Continue the same for each Item number.

63

Item Response Summary Exercise Part 2 How much green do you see? How much red do you see? What does this tell you?

Item Response Summary Exercise Part 3 1.Take out your test booklet, turn to the ENGLISH section. 2.Look for the Item Number that corresponds to the first red circle on your table. 3.Read the question; look at the correct answer. 4.What does this tell you?

Item Response Summary Exercise Homework: Follow the previous steps for each table in the remaining reports 1.Locate the asterisked number in each Item Number. –If highest number, circle it green. –If not the highest number, circle it red. 2. If asterisked number is circled green, look at the Reference Group Percentage. If this number is higher than your green circled number, circle the Reference Group Percentage in red. 3. Continue the same for each Item number.

Curriculum Alignment  Assists in organizing teaching practices  Offers sequence for delivering content  Provides clear scope of what must be taught to all students

Connecting College Readiness Standards to the Classroom ACT has developed this guide to help principals, guidance counselors, and curriculum coordinators interpret the College Readiness Standards Report data for EXPLORE. The guide provides a description of ACT’s Educational Planning and Assessment System now referred to as the College and Career Readiness System. 68

Abbreviated Test Booklets: Test Question Analysis Activity Booklets: Sample Test Questions Matched to College Readiness Standards Booklets Next Steps  Curriculum Review Worksheets

Next Steps

Dissemination of Results Table

Early Intervention Roster  School-level reports that identify students who qualify under three possible categories: –Early Identification –Coursework Intervention –Need for Assistance  Prioritizes students that need or have requested attention.

EXPLORE Early Intervention Rosters A Question to Consider: What strategies could or should be implemented to assist these students with their postsecondary plans?

EXPLORE Early Intervention Rosters

Which of your students express a need for help in one or more areas? What resources do you have or might you access to provide support?

Dissemination of Results Table

Questions/Reflections?