1. Perspectives on Equating: Considerations for Alternate Assessments
Presenters
Vincent J. Dean (Michigan Department of Education)
Adam E. Wyse (Michigan Department of Education)
Timothy R. Vansickle (Questar Assessment)

2. Michigan Assessments
110,000 – 130,000 students per grade assessed each year
Approximately 140,000 SWD K-12
No computer administered assessments to date
Piloting ELPA next spring
SMARTER Balanced Consortium
7/17/2019

3. Michigan Assessments
Grade 3-8 testing begins 2nd week of October and lasts approx 3 weeks
Grade 11 testing late March/early April
General Assessments-Specific content on specific day
Alternate Assessments-6 week window to accommodate individual administrations
7/17/2019

4. Michigan Alternate Assessments
MI-Access (AA-AAS)
Functional Independence (Mild CI)
Supported Independence (Moderate CI)
Participation (Severe CI)
Extended Content Standards for each population
7/17/2019

5. Michigan Alternate Assessments
MEAP-Access (AA-MAS)
Operational fall 2009
Regular Content Standards
Standards-Based IEP timeline
MEAP-Access participation
1 year delay for additional development/piloting
7/17/2019

6. Common Core Standards & Assessments
Michigan adopted Common Core Standards for ELA and Mathematics on 6/15/10
Participating in the SMARTER Balanced Assessment Consortium as a Governing State
7/17/2019

7. Common Core Standards & Assessments
Next generation of assessments
New item types
Performance tasks
Performance events
Technology enhanced items
Computer administered and/or computer adaptive
7/17/2019

8. Common Core Standards & Assessments
New item types and delivery
Accommodations questions
Scoring questions
Technical questions
7/17/2019

9. AA-AAS GSEG June 8, 2010 Federal Register Response due July 23rd
4 year grant
Up to 15 individual awards (less if consortia receive awards)
7/17/2019

10. AA-AAS GSEG
Must take lessons learned from NCLB peer review technical evidence requirements
Huge implications for AA-AAS equating
New item types
Small sample sizes
SBAC Survey – Combination desirable
7/17/2019

11. What is Equating?
A method for statistically adjusting test scores covering the same content
Puts the scores of the two tests on a common scale
Purposes:
To be able to compare test scores
To be able use test scores interchangeably
7/17/2019

12. Requirements for Equating
Tests X and Y must:
Measure the same construct
A symmetric equating function must be used to equate tests
Examinees should be indifferent as to the test form that they take once scores are equated
The equating function should be population invariant
7/17/2019

13. Designs for Equating
Randomly Equivalent Groups- Two different groups randomly drawn from the same population take one test form each and are assumed to be identical to each other
Single Groups- One common group takes both test forms
Common Items (Anchor Test or Linking Items)- Two different groups take each test form, but each group takes a set of common items that are included on both test forms
7/17/2019

14. Equating in General Assessments
Typically uses a common item design
IRT based methods most popular
Tests usually contain mostly multiple-choice items with a limited number of constructed response items
Population includes at least some students without disabilities
7/17/2019

15. Equating in General Assessments
Common item set typically at least 20% of total test
Common item set designed to mirror full length test (but does not usually include constructed response items)
Sample sizes usually exceed common recommendations for equating
7/17/2019

16. Equating in Alternate Assessments
Smaller sample sizes
Unique item types and formats
Small number of items
Assessments vary across student populations and states
Context for administer items may not be standardized
Assumptions for equating may be violated
Population invariance assumption in particular
7/17/2019

17. Equating in Alternate Assessments
Need for technical rigor and quality
Limited amount of research
Clear guidelines for equating in alternate assessments do not exist
According to National Study of Alternate Assessment only 8% of states use formal equating and have conducted some investigation to support it
7/17/2019

18. Case Studies
Michigan 11th grade MI-Access Participation English Language Arts Assessments from 2008 and 2009
Students who have or function as if they have severe cognitive impairments
Teachers can adapt items as part of administration
Ten items each scored by two raters on a scale of 0 to 3
Total score ranges from 0 to 60
Four common items included to facilitate equating
7/17/2019

19. Case Studies
Michigan 11th grade MI-Access Participation English Language Arts Assessments from 2008 and 2009
High level of agreement between raters
Three performance levels with top two combined for purposes of AYP
296 students took assessment in 2008 and 349 took assessment in 2009
7/17/2019

20. Case Studies
Michigan 11th grade MI-Access Supported Independence Science Assessments from 2008 and 2009
Students who have or function as if they have moderate cognitive impairments
Teachers can adapt items as part of administration
Seventeen items each scored by two raters on a scale of 0 to 2
Total score ranges from 0 to 68
Five common items included to facilitate equating
7/17/2019

21. Case Studies
Michigan 11th grade MI-Access Supported Independence Science Assessments from 2008 and 2009
High level of agreement between raters
Three performance levels with top two combined for purposes of AYP
565 students took assessment in 2009 and 576 took assessment in 2008
7/17/2019

22. Equating Methods Identity (No Equating) Linear equating methods
Tucker, Levine Observed Score, Levine True Score, and Chained
IRT and equipercentile methods considered but not used in study
7/17/2019

23. Equating Criteria Standard Error of Equating
Measures the amount of random error present in equating transformation
Similarity of performance level classifications with different equating methods
7/17/2019

24. MI-Access ELA Participation Assessments
7/17/2019

25. MI-Access ELA Participation Assessments
7/17/2019

26. MI-Access Supported Independence Science Assessments
7/17/2019

27. MI-Access Supported Independence Science Assessments
7/17/2019

28. Discussion and Conclusions
Special Education
Special Education is a federally funded program designed to provide access to a free and appropriate education to children with disabilities up to age 21 in public school systems. Schools must provide services according to the regulations set forth in the Individuals with Disabilities Act, known as IDEA. All public schools in the U.S. are required by law to adhere to these regulations and provide direct and supportive services to assist children with disabilities.
7/17/2019

29. Discussion and Conclusions
IDEA
(4) A statement of the special education and related services and supplementary aids and services, based on peer-reviewed research to the extent practicable, to be provided to the child, or on behalf of the child, and a statement of the program modifications or supports for school personnel that will be provided to enable the child—
(i) To advance appropriately toward attaining the annual goals;
(ii) To be involved in and make progress in the general education curriculum in accordance with paragraph (a)(1) of this section, and to participate in extracurricular and other nonacademic activities; and
(iii) To be educated and participate with other children with disabilities and nondisabled children in the activities described in this section. [§ (a)(4)] 
7/17/2019

30. Discussion and Conclusions
By definition special students are unique enough to require individualized instruction plans
May receive instruction in different modes
May require differential help in producing responses
7/17/2019

31. Discussion and Conclusions
A standardized test is a test that is administered and scored in a consistent, or "standard", manner. Standardized tests are designed in such a way that the questions, conditions for administering, scoring procedures, and interpretations are consistent and are administered and scored in a predetermined, standard manner.
7/17/2019

32. Discussion and Conclusions
Are Alternate Assessments
Standardized Tests
Valid
Reliable
7/17/2019

33. Discussion and Conclusions
Most states allow teachers to choose some of the content a student will be tested on
Most states allow teachers to determine the mode of response and the level of assistance
7/17/2019

34. Discussion and Conclusions
Equating requires some overlap between a set of tests to estimate the difficulty level in order to make an adjustment to one or the other test
Most AA do not have this overlap
7/17/2019

35. Discussion and Conclusions
Technical challenges
Small n-count
No overlap of items or tasks
Individual response patterns, modes, and assistance
Varying content measured across students
7/17/2019

36. Discussion and Conclusions
Scaling and Equating may be desirable but not feasible in many situations
By definition, equating seems unlikely and maybe irrelevant
7/17/2019

37. Discussion and Conclusions
Although some negative results were found for MI-ACCESS,
Small differences at the most granular of levels may be non significant difference either statistically or practically
May not be generalizable to other types of AA
Need a number of additional studies
7/17/2019

38. Contact Info
Adam E. Wyse
Vincent J. Dean
Timothy R. Vansickle
7/17/2019