Who in the World Develops and Writes the MI-Access Items? OEAA Conference 2007 – Day 2 MI-Access Coordinators

2 Session Presenters Alison Place, Questar Assessment, Inc. Les Sewall, Questar Assessment, Inc. Desiree Spikings, Questar Assessment, Inc.

3 Goals for the Presentation Provide an overview of MI-Access assessment item development Explain the purpose and activities of the associated committees: APWT, CAC, SRC, RFC, & SSP Describe how interested participants can share expertise and experience in the development process

4 Assessment Development Process Identify Assessable Content Committee: APWT Design Assessment Plan Committee: APWT Develop Assessment Blueprint Committee: APWT Write Passage or Stimulus Materials Train Item Writers Develop Items

5 Assessment Development Process Edit Items State Internal Item Review Meeting External Item Review Meeting Committees: CAC and SRC Field-Testing Data Review Meeting Committees: CAC and SRC

6 Assessment Development Process Assessment Administration Rangefinding for Field- Tested Constructed Response Items Committee: RFC Scoring the Assessment New Form Construction

7 Assessment Development Process Standard Setting is Conducted Committee: SSP The TAC Reviews and Approves Recommendations of the SSP The Performance Standards are Presented to the SBE Assessment Results Are Released to the Public

8 Advisory Roles and Functions Assessment Plan Writing Team (APWT) Content Advisory Committee (CAC) Sensitivity Review Committee (SRC) Rangefinding Committee (RFC) Standard Setting Panel (SSP)

9 Advisory Committees Composition Geographically & demographically diverse representatives of

10 Advisory Committees Composition Special & general education teachers Content specialists Teacher consultants Administrators School psychologists, counselors, diagnosticians Community & business representatives

11 Assessment Plan Writing Teams Purpose: Recommendations for the design of the assessment plan

12 Assessment Plan Writing Teams Activities: Examine research base & resources. Extend MI Content Standards, Benchmarks, & Grade Level Content Expectations (GLCEs). Using the Extended GLCEs and Benchmarks (EBs), identify content assessable at the state level. Develop the assessment plan.

13 Assessment Plan Writing Teams The Assessment Plan includes Underlying assumptions Content area constructs to be assessed Format of assessment & prototype items Administration, scoring, & reporting recommendations Preliminary assessment blueprint (length of time, number of items per EGLCE/EB, administration, accommodations, and item specifications)

14 Sensitivity Review Committee Purpose: To review all assessment items & materials for fairness & potential bias issues To make sure assessments are sensitive to cultural & other differences among students

15 Sensitivity Review Committee Activities: Check each item to determine if it includes stereotypes or negative perceptions knowledge that is elite or private cultural insensitivity or misrepresentation a balance of gender and race

16 Content Advisory Committee Purpose: Review of assessment items for Alignment with Content Standards, Benchmarks, & EGLCEs/EBs Appropriate grade & difficulty level Quality, accuracy, & completeness Wording clarity

17 Content Advisory Committee Activities: Check each item to determine if it matches the content of the EGLCEs and EBs is coded correctly is appropriate for the grade & developmental level of the students

18 Rangefinding Committee Purpose: From a large sample of student responses for each assessed prompt, the RFC selects sets of papers that can be used for training and qualifying scorers.

19 Rangefinding Committee Activities: Independently score a variety of actual student responses to a particular Expressing Ideas prompt. With the group, discuss & attempt to reach consensus on scores for papers that represent the whole range (low, medium, high) of each of the 4 score points. Repeat process with each prompt.

20 Standard Setting Panel Purpose: To recommend the “cut” scores that delineate achievement at 3 performance levels Emerging Attained Surpassed

21 Standard Setting Panel Activities: Learn the standard setting technique. Define & refine the Performance Level Descriptors (PLDs). “Take” the assessment.

22 Serving on an Advisory Panel Active role in shaping the assessments Networking and collegiality Learning and professional development Contributions to the quality of education in Michigan

23 MI-Access Committees How do we apply? Go to Click on MI-Access Committee Application (pdf form) Who can answer further questions? Peggy Dutcher