Understanding Assessment Mark Leal Director of Assessment Alaska Department of Education and Early Development Office phone number

STANDARDS-BASED REFORM Content and Performance Standards State clear expectations Standards-based tests Test Standards Adjust instruction Adjust instruction to achieve standardsRemediation standards-based instruction Standards-based instruction Teach Standards

EXPECTATIONS FOR STANDARDS-BASED ASSESSMENT SYSTEMS 4 Rigorous standards 4 Tests based on and aligned with standards 4 Fair, valid, reliable tests and assessment system 4 Systems that include ALL students 4 Results reported to public/stakeholders

STANDARDS-BASED ASSESSMENT SYSTEMS Individual students reportresultsforcompareresults by subgroup for Statewide Districts Schools Disability status English proficiency Low-income status Migrant status Gender/Race/Ethnicity

ASSESSMENT SYSTEMS USING ASSESSMENT SYSTEMS TO MAKE DECISIONS

ASSESSMENT SYSTEMS (Possible Components) Norm-referenced tests compares student performance to norm group Standards-based tests compares student performance to standard Blended assessment systems combines norm-referenced and standards-based assessments

STANDARDS-BASED TESTS Also known as criterion referenced tests compares performance to standards or criteria compares performance to standards or criteria

Sixmile Creek is a comfortable distance from Anchorage. However it is not a comfortable river. Sixmile has similar character to many Washington and Oregon whitewater runs: lush forest, deep canyons, and tremendous rapids. The river is easily accessible from Seward Highway. Sixmile Creek has three distinct sections. The Upper Canyon rates a Class IV. The toughest stretch is the lower canyon, where the river rampages along at a constant Class V for two killer miles. The Wildest One! Difficulty: Class V Season: May thru September Trip Length: Half-day to one day

Whitewater Classifications Most experts use a I to VI rating system for whitewater difficulty. Heres a quick rundown of what to expect from each classification: I- Small rapids with low waves and no obstructions II- More frequent rapids with few or no obstructions III- Waves up to four feet, with some maneuvering around obstructions IV- Very Difficult, extended rapids. Scouting is necessary. V- Long and violent rapids. Large waves that are unavoidable. Complex course. Scouting is a must. VI- Maximum difficulty. Involves serious risk of life.

TRIAL USFSA JUDGING FORM Adult Gold Free Skating TRIAL Candidates Name _________________________________________USFSA # ____________________________ OFFICIAL JUDGES TRIAL COMMENTS Technical Merit Presentation TotalPassRetry Judge-on charge overwrite total in ink Judge-in-Charge Signature_____________________________________________________ USFSA # ______________________ Duration: Not to exceed 2 minutes, 40 seconds to the music of the skaters choice Passing Average 3.0 Candidate must give a strong performance and demonstrate a command of the following required elements showing very good form and carriage, strong true edges, effortless flow and preciseness to steps. JUMPS: ____Axel ____Lutz ____Two different additional single jumps ________________ SPINS: ____Two different solo spins (minimum of 4 revolutions) ____One of the following: Split, Stag, Host Club___________________________________________Date _______________________________ Falling Leaf, or Half Loop ____One jump combination consisting of two (2) different single jumps (no turn or change of foot between jumps) ________________ ____One combination spin with at least one change of foot (minimum 4 revolutions in position) STEPS: ____One step sequence – either straight line or circular) ____Connecting moves consisting of spirals, strong edges good use of music and full utilization of ice surface Extra elements may be added without penalty. Only two of the required elements may be retried, if necessary Trial Judges Signature _______________________________________USFSA # _________________________ Trial Judges Club ___________________________________Current Judging Status_______________________

Skater Technical Merit Mark Presentation Mark Total of Marks Ordinal Value Amber Jill Tori Haley Nikki

STANDARDS-BASED TESTS compares student performance to standards Results report levels of proficiency exemplary Advanced or exemplary mastery Proficient or mastery basic Below proficient or basic novice Not-proficient or novice Items look more like classroom work Multiple-choice Open-ended Extended Constructed-response and essay Short Constructed-response

Standards Standards-Based Tests USE Monitor mastery of standards Measure performance Sample depth of student knowledge & skills Accurate with validity & reliability MISUSE As sole indicator of learning As stand-alone evidence of a schools success As accurate without validity & reliability

NORM-REFERENCED TESTS compares performance (data) to norm group.

Reference Population: Standard Normal Curve 95 th 5 th 50 th Percentile

Body Mass Index- Norm Referenced Measure Boys: 2 to 20 years BMI

Indicators of Nutritional Status <5 th percentile Stunting/shortness length or height-for-age Head circumference-for- age <5 th percentile >95 th percentile Underweight weight-for-length BMI-for-age

NORM-REFERENCED TESTS NORM-REFERENCED TESTS compares student performance to norm group Results are reported as Results are reported as Standard score Standard score Percentile score Percentile score Grade equivalent Grade equivalent Half performed better Half performed better Half performed worse Half performed worse Referenced to 50 th %ile Referenced to 50 th %ile

Norm-Referenced Tests USE To measure group performance To compare to norm group To measure subgroups represented in norm group As reliable & valid MISUSE To measure individual student performance To compare to specific standards through general domains To measure excluded/unaccounted for subgroups As modified without new validation/reliability

Norm-Referenced Tests OUR students Does the NRT measure OUR students? standard Does the score tell if the standard is MET? OUR standards Do questions match OUR standards? standards Did OUR experts match items/tasks to standards? OUR standards Are alignment studies specific to OUR standards? OUR standards Does the NRT measure ALL students against OUR standards?

BLENDED ASSESSMENTS combines norm-referenced and standards-based tests Norm-referenced tests provide Comparisons to national norms Comparisons to national norms High levels of technical quality High levels of technical quality Familiarity and confidence Familiarity and confidence Standards-based tests provide Comparisons to standards Comparisons to standards Feedback on learning what is intended Feedback on learning what is intended Data useful to classroom teaching and learning Data useful to classroom teaching and learning

Alaska Large-Scale Assessment Matrix

LEGAL ISSUES Technical quality of results Validity and reliability of results Accuracy of results Appropriate use of results Stakes for schools Stakes for students and teachers Legal issues Rewards and sanctions Opportunity to learn All Accounting for/Accommodating All

Changes to the HSGQE Framework for Continuous Renewal SB 133

Committee Actions and Recommendations Affirmed the Appropriateness of the Performance Standards Reviewed bank of test questions Suggested that a number of questions be changed or eliminated. Suggested that the test be refocused Suggested that certain portions of the test be revised.

Refocused HSGQE Examination focusing on assessing minimum competencies in essential skills Data used to insure that all students who receive a diploma in the state of Alaska can demonstrate minimum competencies in essential skills

Revised HSGQE More specific direction in items and questions According to the article… Give a reason and list two examples Format of Test Reading or charts on facing pages Writing check list Varied Reading Passages Length of Test Real World Focus and Application

Process for Development of Refocused and Revised HSGQE New Test Item Maps Constructed New Test Forms Generated based on new item maps Spring 2002 (Refocused) New passing scores established August 2002

What does this mean for the classroom? HSGQE Refocused to assess minimum competencies of essential skills Teachers need to continue teaching all of the standards to all of the students Student performance needs to be evaluated using multiple assessments

Alaska State Assessment and TerraNova-CAT/6 Second Edition Copyright 2000 K-12 Basic Skills Achievement Battery Three assessment models Multiple Assessments Complete Battery Survey

TerraNova-CAT/6 Complete Battery Plus Selected Response Items Subtests Reading/Language Arts* Mathematics* Science Social Studies Vocabulary* Language Mechanics* Computation* Spelling

TerraNova-CAT/6 Content Objective Structure Developed across levels Thematic approach in reading/language Real life context Model for CSSA item

Participation Guidelines New booklets will be sent to schools by early December. Indicates 2002/2003 on cover Will include changes specified by SB 133 Optional assessment program

Test Security Both NRT, HSGQE and Benchmark Tests must be administered by school personnel at a school test center. Correspondence programs must arrange to have tests administered by school personnel at school test centers. Test materials must be stored in a secure manner

Test Security-continued All test questions are confidential-no unauthorized access- TerraNova/CAT,HSGQE,Benchmark HSGQE Associate test coordinator Alaska Certification required Proctor ratio 1:30 Alaska Certification required Cannot be teachers who, at time of testing have the student in class (in subject tested)

Duties of School Personnel Ensure confidentiality of test questions Responsible for test administration Inventory and track materials Control distribution, storage, administration and collection of tests Ensure that no test is copied in any manner

Duties of School Personnel- continued District Test Coordinator-HSGQE Assure that each HSGQE test booklet and test administration manual is accounted for from the time the materials arrive at the district office until the time they are returned to the test publisher. Ensure adequate supervision of students taking the HSGQE

Duties of School Personnel- continued School Test Center Code tests according to directions Inventory and track materials Securely store tests before and after use Control distribution of tests Ensure that: no test is copied examinees only use reference materials specified in directions no student receive a copy of test or learns questions in advance Examinees do not exchange information Examinees answer is not altered after testing is completed

Results returned to parents/students HSGQE within 15 days of receiving or before end of the school year. Benchmark Exams within 20 days of receiving or before end of the school year. Norm Referenced Test In a timely manner

Results distributed to teachers Class and individual results to each teacher for the teachers classes Intent is that teachers have assessment data for students in their classes

Confidentiality Individual student results are confidential Family Educational Rights and Privacy Act (FERPA) Scores can be shared with school and other education officials to complete their duties May not be publicly reported without parent consent Fewer than 5 students tested/3 in any cell

Issues in Reporting Data Confidentiality Family Educational Rights and Privacy Act Technical Issues Small Schools Release of Information Results not reported if fewer that 5 students tested or fewer than 3 students are reported in any cell.

Protocol for Reporting Results Two Categories of Achievement Complete results will be reported as long as at least 3 students are reported in any cell. If either proficiency level contains 0,1, or 2 students the number of students will be eliminated and the percentage of students at each achievement level will be reported as a range. In reporting a range of performance the following rules will apply. NOTE: No results will be reported if less than 5 students are tested.

Number of Achievement Level Reported Students Tested 560% or more proficient (or not proficient) 40% or fewer not proficient (or proficient) 875% or more proficient (or not proficient) 25% or fewer not proficient (or proficient) 1080% or more proficient (or not proficient) 20% or fewer not proficient (or proficient) 2090% or more proficient (or not proficient) 10% or fewer not proficient (or proficient) 4095% or more proficient (or not proficient) (or more)5% or fewer not proficient (or proficient) Reporting Protocol

Making effective use of the DATA Provide results to parents, students and teachers School wide Individual Devise ways to provide easy access to assessment data to teachers Include as much information as possible Designation API

Mining the DATA Look beyond the mean Disaggregate data to determine the effect of program on various groups Quartiles, item analysis API Scores Look at trends over time Work with staff to determine what you can learn from the data how you will react to the data

Using the Results to Make Decisions school improvement planning About school improvement planning About policy changes and program evaluation rewardssanctions About rewards and sanctions

Use the results to: focus instruction target student intervention identify needs develop programs evaluate programs and initiatives guide Staff Development activities set goals for improvement

Examining the Data to Set Goals Weakest results Domain scores = reading? Specific standards = use comprehension strategies? Specific skills within the standard = paraphrase key points? Demographics Population impact = low socio-economic majority? Opportunity to learn = poor attendance rates?

SCHOOL IMPROVEMENT PLAN Selecting successful instructional interventions For the area of concern For the age, grade, and skill level of the students For the demographics of the school For long term effectiveness Study and training Support through coaching, observation, and feedback Follow-up for ongoing training and reflection Monitoring through supervision and results Specific Reasonable Time lines and assignment Planning staff development

Data Collection Importance of Proper Coding Demographic Data Program data District or school data Importance of Accurate and timely Data Submissions Importance of Verification of Data

Data Collection Changes for 2002/2003 Increase in OASIS information Special Education Fields Summer Collection Changes to Common Core Data-October 2002 Reporting Teacher Assignments Certification Implementation of Unique Student Identification Number

Questions, Comments, Concerns