Ace it! Summer Conference 2011 Assessment Brenda Neuman-Sheldon

Topics for today For what purpose(s) do we use assessment? What do you find challenging about assessment in Ace it! ? “Assessment Basics” Addressing common challenges related to Ace it! and assessment What steps are we taking to improve Ace it! assessment?

How Ace it! uses assessment Assess student skills and needs before beginning an Ace it! program Place students in instructional groupings Determine appropriate skills/lessons Measure student progress after completing an Ace it! program Meet the needs of individual contracts Report to State agencies Include in applications Determine the overall impact of Ace it!

Activity #1: Articulating Assessment Challenges Describe for your group a time when assessment/testing has been challenging for you in the past year How did you address this challenge? Select a group representative and share Break into small groups at tables Describe for your group a time when assessment/testing has been challenging for you in the past year How did you address this challenge? Have a record keeper share with the whole group

What is a standardized test? Developed and administered following specific rules for testing from the American Psychological Association. A standardized test must meet guidelines about: Test Construction, Evaluation, and Documentation Fairness in Testing Use of the Tests and Test Results

A high quality standardized test needs to have: Validity: Does the test accurately measure what we think it measures? Reliability: Is the test “accurate”? Common scoring and reporting metrics Uniform administration procedures

Types of standardized tests Criterion Referenced: measure the level of proficiency obtained on a certain skill set Norm Referenced: compare the scores of one student or group of students to another, or the same student or group at different points in time A test can be one or both Criterion Referenced: measures measure the level of proficiency obtained on a certain skill set Allows for comparing student scores with performance standards Does not provide grade equivalencies or other comparative scoring Norm Referenced: compare the scores of one student or group of students to another, or the same student or group at different points in time Allows for comparative scoring Scores do not tell what students know only how a student compares with that of others in the norm group

Scores and Metrics Two types of scores Those that reflect INDIVIDUAL student performance Those that reflect individual student performance IN COMPARISON to others Using different scores provides a more complete picture of student achievement

Basics of Scores/Metrics Individual Performance Raw score Scale score Skill Needs or mastery information Comparative Performance Percentile rank Stanine Normal curve equivalent Grade equivalencies Provide BRIEF definition

Metrics available on different kinds of tests Norm Referenced Criterion Referenced Raw Scores  Scale Score Skill proficiency data Grade Equivalencies Percentile Ranks Stanines Normal Curve Equivalent

Activity #2: Assessment Problem Solving Scenarios You each have an “Assessment Basics” reference guide which provides more detailed information about standardized testing and scoring metrics. The last pages of the reference guide has some real life scenarios that we have run into with assessment in Ace it! Using the information from this session so far, and well as the assessment reference guide, review the scenario assigned to your table and respond to the questions provided. Be prepared to share your responses.

Meeting the assessment needs of Ace it! program providers Assessment basics reference and training materials in development “Talking points” for assessment results in development Research underway to powerfully describe Ace it! impact and results Looking at possible new assessments Provide greater support to Ace it! providers around assessment

What does an Ace it! test need to be? Paper-based Criterion Referenced Norm Referenced Results available instantly Test administration in under an hour Available for use in all 50 states Cost-effective Do we have this right? Is there anything missing?

Exploring the possibility of new assessments Issued an RFP for a norm and criterion referenced paper-based assessment that can be administered in an hour or less 4 testing companies expressed initial interest 2 pursued Only one has submitted Developing a plan for revising the ASA Exploring options to have ASA better meet Ace it! provider needs Scoping tech system requirements

Questions?