Extended Assessment Administration Training Oregon Department of Education Behavioral Research and Teaching January 2007

Welcome! Dr. Gerald Tindal University of Oregon Steve Jonas Lane ESD Dianna Carrizales Oregon Department of Education

Agenda DAY 1 Wednesday, January 24th –7:30 Check in/registration (arrangement will vary by site) –8:00Training begins: –8: :30 Introduction, background, and pre-survey –9:30 – 11:30 Mathematics –11:30 -12:30 Lunch (local host sites have different arrangements) –12:30 – 1:00Mathematics (continued) –1:00 – 3:30 Reading –3:30 – 4:00 Wrap up DAY 2 Thursday, January 25th –7:30 Check in (arrangement will vary by site) –8:00 Training begins: –8:00 – 8:30 Follow-up/review –8:30 –11:00 Writing –11:00—11:30Science –11:30 – 12:30 Lunch (local host sites have different arrangements) –12:30 – 2:30 Science (continued) –2:30 – 4:00 Data Entry, next steps, post-survey and wrap up

Objectives To become familiar with the new format of the Extended Assessments To gain understanding of the general scoring process associated with the new format To gain understanding of the administration process associated with the new format in general and for each of the subject areas To gain an understanding of the different administrations of the Extended Assessments To become familiar with the structure of the Extended Assessment system

Regarding Technical Difficulties Local issues Issues in the studio Act of Nature 1 ½ hour delay: access DVD for that area DVD availability: approximate mid- February

Facilitator Role Facilitate pre- and post-survey Consolidate and field questions for ODE Maintain a list of participant names, phone numbers and addresses for qualification Orient participants toward appropriate materials Facilitate break-out exercises

Facilitators Introduce Yourselves

Format of Training 1200 trainees statewide 50 sites Frequent breaks Opportunities for facilitation and organization –Write down questions about the materials and administration –Consider the differences in administration you will anticipate from student to student –Examine the student materials

On Becoming a Qualified Assessor 2007 (QA07) Qualified Assessors must: –Attend and complete an entire state-approved training (Cadre I, II or DVD) –Submit name to ODE to be maintained in the state assessor database –Conduct one level (elementary or middle/high) of the assessment with a peer or student –Submit materials* for all four subject areas –*Two scaffold administration tasks and two standard administration tasks –*One task each (both pre-requisite tasks, and content prompts) –Send administered assessments to ODE –Be prepared for ODE audit follow-up questions

Pre-Survey

Assessment Options Then and Now: What’s left? CIM Extended CLRAS Extended Assessment Standard Administration Extended Assessment Scaffold Administration Accommodated Assessment General Assessment Targeted Assessment Juried Assessment Accommodated Assessment Modified Assessment Extended K - 2

What is Scaffold Administration? Same items More prompting and support Possible different student materials

IEP Team Decision Making

The IEP team may choose to administer a scaffold administration of one subject area and a standard administration of another subject area

Consider General Assessment with or without accommodations if: Student: Performs at or around grade level Has academic difficulties that primarily surround reading but falls within the normal range in other subject areas Has academic difficulties in areas other than reading that are “mild to moderate” and can typically be addressed by using simplified language Is reading within two to three grades of his or her enrolled level Instruction: Is primarily general curriculum instruction (but may also use a specialized curriculum in some areas) Some Judgment variables: What assessment did he take last year? How is his attention? What types of behaviors should be considered?

Consider Extended Assessment Standard Administration if: Student: Performs well below grade level Is significantly below grade level in reading Has academic difficulties that are generalized (to all subject areas) and are significant Benefits from specialized individual supports Instruction: Is primarily from a specialized curriculum and/or From general curriculum must be significantly reduced in breadth, depth, and complexity Some Judgment variables: What assessment did he take last year? How is his attention? What types of behaviors should be considered? Previous relevant experiences

Consider Scaffolded Administration of Extended Assessment if: Student: Performance is significantly impacted due to the nature of/characteristics of the disability Does not read Has academic, mobility, receptive and expressive language difficulties that are generalized and significant Relies on individual and significant supports and adaptations to access reduced content materials Instruction: Is from a specialized curriculum and has functional components and/or Includes academic goals that are significantly reduced in depth, breadth, and complexity from grade level content Some Judgment variables: Is the student able to interact with instructional material in a way that provides meaningful feedback?

Accommodations and Modifications Accommodations: Do not change the intended content standards but provide access to the assessment without altering the construct Modifications: Change the intended content standards of the measure and impact the validity of the item/measure

Accommodations Accommodations of –Setting –Response –Presentation –Timing/Scheduling Examples –Extended time –Frequent breaks –Change in test directions –Change in font or size of text –Assistive technology –Test setting –Manipulatives When used? –In the classroom vs. Testing only?

Accommodations, Modifications, & the Extended Assessment Accommodations Table usage Use of accommodations –All students –As used in the classroom –Submission of recommendations to panel Modifications table and usage Use of modifications –IEP team judgment (consideration of meaningful access) –As needed based on team judgment –No recommendations are currently being submitted or considered

Testing Schedule Extended Assessment Window March through April Students follow same testing grades as TESA schd0607.pdf See table

Extended Assessment Grade Levels –Elementary level assessment grades 3 - 5(Scaffold or Standard administration) –Middle/High level assessment grades 6 – 8 & 10 (Scaffold or Standard administration) Content –Mathematics, Reading, Writing, Science Skill levels –Grade level content standards reduced in depth, breadth, and complexity –5 Prerequisite skills per task –5 Content prompts per task Scoring –Levels of Independence –Accuracy

Materials (Elementary) DocumentReadingWritingMathScience Elementary – Extended Standard Administration Scoring Protocol Elementary – Extended Standard Administration Student Materials Elementary – Scaffold Administration Scoring Protocol Elementary – Scaffold Administration Student Materials Power Point presentation materials

Materials (Middle High) DocumentReadingWritingMathScience Middle/High – Extended Standard Administration Scoring Protocol Middle/High – Extended Standard Administration Student Materials Middle/High– Scaffold Administration Scoring Protocol Middle/High – Scaffold Administration Student Materials Power Point presentation materials

Format of the New Assessment

New Scoring Protocol

Prerequisite Skills Questions that all students can participate in Administer to students who are primarily working on life skills Appear on each task for all extended assessments Five questions per task

Prerequisite Skills Each of the pre-requisites for each task have a similar sequence of difficulty All can be answered by pointing or communicating when the assessor is touching the correct item

Prerequisite Skills Each question measures the level of support the student needs in order to be successful in answering the question correctly What you are measuring is the support you give to the student Scoring is not for accuracy

Prerequisite Skills Scoring is the measure of the level of independence for success Always allow the student to have an opportunity to Independently answer the question

Prerequisite Skills Levels of Independence: –4 = Independent –3= With Verbal, Gesture, or Visual Prompt –2= With Partial Physical Assistance –1= With Full Physical Assistance

Prerequisite Skills I= Question is Inappropriate based on the nature of the student’s disability R= Does not Complete with Any Form of Assistance (Refusal)

Level of Independence selection process for proceeding to content prompts Prerequisite total is the sum of the five prerequisite item scores Level of Independence (level of support provided) for content area is based on the lowest value selected during administration of the prerequisite skills (i.e. the most support provided to the student during prerequisite administration)

Examples Example1: Lily receives prerequisite scores of: 4, 3, 2, 2, 1 on any given task, then the Level of Independence = 1 Example 2: Brad receives prerequisite scores of: 1, 1, 1, 2, 1; Level of Independence =1 Example 3: Roy receives 4, 1, 1,R*, R*; Level of Independence = 1

Partial Scoring for Participation Student must complete a minimum of 5 Prerequisite Skills tasks with the following contingencies If a student receives a score of 5 or less per prerequisite task the student is eligible to stop testing and can still count for participation If a student receives a score greater than 5 on the Prerequisite Skills associated with a task, the student must proceed to the Content Prompt for the associated task even if the teacher must score an "I" or "D" (as appropriate). Completion of 5 prerequisites is the minimum participation expectation, however, all accommodations and appropriate provisions should be considered thoroughly prior to discontinuing an Extended Assessment administration

Proceeding to Content Prompts If tasks are too difficult for student, administrator may score “D” for each item. If tasks are completed accurately via full physical support (i.e. Independence was scored as 1), then Accuracy Score can be either 1 or 2 If a student taking the Standard Administration is deemed proficient at all prerequisite skills then Level of Independence is scored as 4, and Content Prompts are administered with no supports.

Scoring Summary Prerequisites are administered with whatever level of support is required and appropriate for the student to be successful on the item. Prerequisite total: is the sum of the prerequisite items. Level of independence for moving into the content prompts is the lowest value from the prerequisites: this demonstrates the level of permission for the additional supports that can be provided (beyond standard administration) to allow the student to access the materials in the content prompts. Partial participation: A minimum of 5 prerequisite tasks are necessary to count toward participation. A score of 5 or less on prerequisites means that the student is not obligated to proceed into the content prompts.

Elementary Mathematics General Overview

Elementary Mathematics Task Names Task 1: Numbers Task 2: Computation Task 3: Measurement of Time/Temperature Task 4: Geometry/Shapes Task 5: Measurement Task 6: Interpret Data and Graphs Task 7: Applications Task 8: Fractions and Conversions Task 9: Probabilities and Predictions Task 10: Algebra

Administration – Big Picture Task Content Pre-requisite Skills Content Prompts Materials Preparation Scoring Anticipations and Possibilities Role play

Elementary Mathematics: Task Content Numbers: Recognition, ordering, digits placement, count-bys Computation: Simple addition, simple subtraction, multiplication, division Time/Temperature: Telling time, calculating time, temperature, season Geometry/Shapes: Shape recognition, angles, perimeter calculation Measurement: Measuring in units, measurement calculations, use of weights and measures in day to day life Interpret data and graphs: Interpreting a simple graph Application: Money recognition, counting money, adding money, cost and making change Fractions & Conversions: Recognizing fractions, decimals, and percentages Probabilities & Predictions: Coin tosses and probability, odds, likelihood Algebra: Missing values, missing symbols, word problem

Elementary Mathematics Pre-requisite Skills Measure of a student’s level of independence in response to : –Requests for attention –Requests to interact –Specific queries regarding: numbers, shapes, clocks, or coins –Specific queries regarding: symbols, shapes, time, graph, tool, bills, coins Numbers: put the cards into two groups Computation: put two number cards together Time/Temperature: show me a picture of a thermometer and snow Geometry/Shapes: show me two objects that are similar Measurement: tell me show me something you can measure with a ruler Interpret data and graphs: show me the tallest bar on the graph Application: put two coins together Fractions & Conversions: show me a fraction Probabilities & Predictions: show me which coin shows heads Algebra: show me a number sentence

Elementary Mathematics Materials Preparation TaskCuttingOther materials (Have scratch paper available for all tasks) 1Item 2Student materials only (with cover page) 2OptionalStudent materials only (with cover page) 3OptionalStudent materials only (with cover page) 4OptionalStudent materials only (with cover page) 5OptionalRuler and student materials (with cover page) 6NoStudent materials only (with cover page) 7OptionalCoins, dollar bills and student materials (with cover page) 8OptionalStudent materials only (with cover page) 9OptionalStudent materials. Optional: coins, 3-point spinner, jelly beans or manipulative (with cover page) 10OptionalStudent materials only (with cover page)

Elementary Mathematics Scoring Correct and specific answer = full credit Incorrect answer or refusal inaccurate attempt = no credit General correct answer = partial credit On the right track to a correct conclusion = partial credit Correct working incorrect final calculation = partial credit

Scoring (Continued) Additional querying/prompting may be used if the student’s response suggests that he or she misunderstood/misheard the item Preamble consistency

Elementary Mathematics Anticipations and Possibilities Difficulties: –Some answers have only a right or wrong answer –Some of the graphics are focal points only –Tasks are not in order of difficulty Opportunities: –Some of the questions lend themselves to the use of classroom materials that are familiar to the students –This format provides instructional ideas for presenting concepts that students have not yet encountered Next steps: –Streamlined graphics –More details regarding partial scoring

Middle/High Mathematics General Overview

Middle/High Mathematics Task Names Task 1: Computation & Calculation (C & C) — Numbers, Computation, Estimation Task 2: C & C — Numbers, Problem Solving, Rational Numbers, and Equivalence Task 3: C&C — Fractions and Numbers Task 4: Statistical Measures and Probabilities Task 5: Probability (Representation) and Algebra (Patterns) Task 6: Algebra — Variables, Equations, and Relationships Task 7: Algebra — Graphing and Change Task 8: Measurement — Determining Measurement Task 9: Geometry — Properties, Transformation, and Symmetry Task 10: Geometry — Coordinate Graphing & Modeling

Administration – Big Picture Task Content Pre-requisite Skills Content Prompts Materials Preparation Scoring Anticipations and Possibilities Role play

Middle/High Mathematics: Task Content Task 1: Recognition of equality, equality in fractions, reduction of fractions Task 2: prime numbers, creating fractions, recognizing relative size in fractions/decimals Task 3: Recognizing fractions in shapes, fractions in day to day life Task 4: calculating averages, medians, combinations and patterns, likelihood with coin tosses Task 5: Representing data, likelihood, patterns of numbers in sequence Task 6: Missing variables, number patterns Task 7: identifying growth on a graph, points on a graph, value of y given x Task 8: measuring, perimeter of a triangle, speed (miles per hour), selecting correct units of measurement Task 9: Recognizing shapes (equilateral triangles, parallel lines, radius of a circle, lines of symmetry), applying the Pythagoras theorum. Task 10: Properties of shapes (rectangles), measurement of a shape

Middle/High Mathematics Pre-requisite Skills Measure of a student’s level of independence in response to : Requests for attention Requests to interact Specific queries regarding: numbers, graphs, shapes Specific queries regarding: percents, decimals, math problems, general knowledge, various objects, graphs and axes, specific shapes, and properties of shapes Task 1: show me a fraction Task 2: show me a fraction Task 3: show me a sequence of numbers Task 4: show me which coin shows heads Task 5: show me a pattern of numbers Task 6: show me a table on the paper Task 7: show me a point on the graph Task 8: show me a length Task 9: show me an angle Task 10: show me the long side of a rectangle

Middle/High Mathematics Materials Preparation TaskCuttingOther materials (Have scratch paper available for all tasks) 1OptionalStudent materials only (with cover page) 2Optional (but recommended) Student materials only (with cover page 3OptionalStudent materials only (with cover page) 4OptionalCoin to flip (optional) Student materials only (with cover page) 5Item 1Photo-copy table for item 1 and provide student materials (with cover page) for remainder 6OptionalStudent materials only (with cover page) 7OptionalStudent materials only (with cover page) 8OptionalStudent materials (with cover page), ruler (optional) 9OptionalStudent materials only 10OptionalRuler, Student materials, cover page

Middle/High Mathematics Scoring Correct and specific answer = full credit Incorrect answer or refusal inaccurate attempt = no credit General correct answer = partial credit On the right track to a correct conclusion = partial credit Correct working incorrect final calculation = partial credit

Middle/High Mathematics Anticipations and Possibilities Difficulties: –Some answers have only a right or wrong answer –Some of the graphics are focal points only –Tasks are not in order of difficulty Opportunities: –Some of the questions lend themselves to the use of classroom materials that are familiar to the students –This format provides instructional ideas for presenting concepts that students have not yet encountered –If students are more comfortable with familiar manipulatives they may be substituted as appropriate Next steps: –Streamlined graphics –More details regarding partial scoring

Standard and Scaffold Administration Materials Exercises

Querying and Prompting in Content Prompts Once in content prompt, administration is standardized Support is provided to Access to material and not to ensure success on an item Queries and prompting is provided in the script, however, Administrators may query further (or probe) if a student’s response suggests that the student has misunderstood the question due to its presentation