The Effect of Public Policy on Alternate Assessments Sue Rigney Alternate Assessment Conference University of Maryland College Park, MD October 2007

Historical : Many SWD routinely excluded from state & national assessments Exemption of a special education student requires: a) “the student has been found eligible for special educations services through an IEP; and b) receives Special Educations services prior to the first day of testing; and c) receives 49% or less of his/her reading/English instruction per week through general education instruction.” Source: MEAP Assessment Administration Manual, 1991 Alternate Assessment

Key Federal Statutes IASA 1994  Standards and assessments by  All SWD to be included in assessments IDEA 1997  Access to general curriculum  Alternate assessment in place July 2000 NCLB 2001  SWD included in assessments & accountability for all public schools IDEA 2007  Follows NCLB Alternate Assessment

NCLB + Regulations 1% AA-AAS December 2003 Permits alternate achievement standard for students with most significant cognitive disability 2% AA-MAS April 2007 Permits modified academic achievement standard for students whose disability prevents them from meeting grade level standard in period covered by current IEP Alternate Assessment

Examining Policy Effects Intent Implementation Impact on State practice Alternate Assessment

Intent Is always good Realized through implementation  Diverse actions, actors  Slow, must be sustained Consequences may be unexpected  Perception vs reality  Perception is reality Alternate Assessment

Intent - IDEA 04 & IASA Paradigm Shifts IDEA 04 Access to general curriculum for SWD IASA 97 for Title I Schools All students included in State assessments Scores of SW must be publicly reported for school and district accountability State must explain how scores from alternate assessment are integrated into accountability system Alternate Assessment

Intent - NCLB “To ensure that all children have a fair, equal, and significant opportunity to obtain a high quality education…”  All schools publicly accountable for performance of SWD  Alternate achievement standard permitted only for students with most significant cognitive disability  1% cap as safeguard for students Alternate Assessment

Implementation Statute clarified by guidance Occurs in the field - monitoring must examine evidence of compliance Compliance alone may not ensure that policy goals are reached Successful implementation requires State as well as federal action Alternate Assessment

Federal Policy Implementation Statute, regulations & guidance drafted and disseminated Compliance monitoring carried out by multiple offices e.g.,OSEP, OESE, SASA Peer review of Title I State Plan required Technical assistance $$ Alternate Assessment

State Policy Implementation Inclusion policies and procedures Optional development & implementation of AA-AAS or AA-MAS consistent with statute Support for test administration and use Infrastructure for local implementation  Assessment training  Professional development to support effective instruction Alternate Assessment

Implementation - IASA Compliance monitoring Assessment system peer review  Focus on test administered in  Continued under NCLB for States not approved Alternate Assessment

IASA Peer Review – AA Must “When assessment procedures are altered, it is critical to ensure that scores, decisions, and judgments based on those assessments are fair, reliable, and valid. The criteria for technical quality outlined in…. “Professional Standards of Technical Quality,” apply to modified, accommodated, and alternate assessments. IASA Peer Review Guidance, p. 15

Implementation - NCLB Accountability workbooks Title I monitoring OSEP monitoring Peer review of State assessment systems Alternate Assessment

“…the NCLB standards and assessment peer review process increased the requirements for documenting the technical quality of all assessments, but the biggest shift was for AA-AAS. The type of technical documentation necessary to fulfill the peer review requirements has never been expected from AA-AAS developers previously.” Marion & Pellegrino

NCLB Peer Review: AA-AAS Must  Yield results separately in reading and math  Clear guidelines for student participation provided to all LEAs  Designed and implemented in a manner that supports use of results for AYP Aligned with state content standards Assessment design - appropriate for school accountability measure (e.g., results comparable across schools and districts)  State provides evidence of technical quality, Validity, reliability accessibility, objectivity, and consistency with nationally recognized professional and technical standards Description of the standard-setting process, the judges and their qualifications, and state adoption of alternate achievement standards  Reports results to teachers and parents in a manner consistent with the alternate achievement standards Alternate Assessment

Impact-IASA On January 19, 2001 Alternate Assessment Decision#States Full Approval11DE, IN, KS, LA, MD, MA, PA, RI, VT, VA, WY Conditional Approval (Complete by Spring 2001) 6KY, MO, NC, OR, TX, WA Timeline waiver14CO, CT, GA, HI, ME, MS, NE, NV, NH, NY, ND, OH, SC, SD Compliance Agreement 3CA, WV, WI Still under review18AL, AK, AZ, AR DC, FL, ID, IL, IA, MI, MN, MT, NJ, NM, OK, PR, TN, UT,

Impact-IASA Issues Facing States on January 19, 2001 Requirement # Inclusion of limited English proficient students 22 Inclusion of students with disabilities 14 Disaggregated Reporting 30 Finish Standards-based System 11

Impact NCLB States Revising/Developing Alternate Assessment in 2005 Alternate Assessment Area # States Approach 8 Content10 Standard-setting13 Scoring Criteria17 Source: 2005 State Special Education Outcomes, NCEO

Current Status As of 8/6/07 31 States = Approved + Approval Expected States working on AA-AAS Major concerns:  alignment with grade level content  documenting technical quality

Completing the AA-AAS DEADLINE EXTENDED Approval Pending (does not meet all of the requirements) If only significant issues with an alternate assessment based on alternate achievement standards or an assessment for limited English proficient students…  Condition on its fiscal year 2007 Title I, Part A grant award  Mandatory Oversight, pursuant to 34 C.F.R. §  Agreement with the Department  demonstrating a commitment and investment of resources to resolve all outstanding issues for the 2007–08 administration of its assessments.  a mutually acceptable timeline for how and when the remaining work toward having a fully approved standards and assessment system will be accomplished.

Review of AA-MAS Standards and Assessment Peer Review Guidance: Information and examples for meeting requirements of the No Child Left Behind Act of 2001 Revised to include AA-MAS requirements Distribution to States TBA Peer reviewer training Jan 2008

Impact on Assessment Practice  Virtually all State assessment participation policies changed since IASA  Participation of SWD in State assessments is substantially increased  22/50 states have changed participation policies/guidelines for AA-AAS since the Dec 9, 2003 regulation  Peer Review has prompted linkage to academic content for all states Alternate Assessment

Impact on Assessment Practice  State examples of rigorous practice emerging, e.g. Alabama standard setting report  New methodology emerging: e.g. Links for Learning, NAAC Learner Characteristics Inventory  Articles in professional journals focus on AA- AAS  Questions about validity of AA-AAS challenges some assumptions about general test

Impact on Instruction Anecdotal and case studies Most pre-date requirement for academic content Inclusion in accountability makes a difference: “I think our expectations are higher.”

Impact on Student Outcomes Evidence of student outcomes limited  Reports do not separate general test results and alternate results  OSEP collects detailed data in biennial report but it’s hard to find Alternate Assessment

MSA Snapshot (State) With trend data ALT-MSA Snapshot (State) With trend data ATA=State&K=99AAAA#ALTsnapshot

Impact on Student Outcomes Evidence of student outcomes limited  Reports do not separate general test results and alternate results  OSEP collects detailed data in biennial report Evidence of student outcomes difficult to interpret  Many state alternates redesigned in last 3 years, so trend data is not interpretable  Test results confounded with OTL Alternate Assessment

Lessons Learned? Collaboration needed to develop alternate assessments: assessment, special ed, content experts Resources needed to build local support systems Consequences must be documented Interpretation of outcomes difficult because student results confounded with opportunity to learn Alternate Assessment