New Hampshire Enhanced Assessment Initiative: Technical Documentation for Alternate Assessments Consequential Validity Inclusive Assessment Seminar Elizabeth.

Slides:



Advertisements
Similar presentations
Action Research Not traditional educational research often research tests theory not practical Teacher research in classrooms and/or schools/districts.
Advertisements

Robert L. Linn CRESST, University of Colorado at Boulder Presentation at the Ninth Annual Maryland Assessment Conference: The Concept of Validity : Revisions,
How to Evaluate Your Health Literacy Project Jill Lucht, MS Project Director, Center for Health Policy
Alternative Assesment There is no single definition of ‘alternative assessment’ in the relevant literature. For some educators, alternative assessment.
UTILIZING FORMATIVE EVALUATION IN A PROFESSIONAL DEVELOPMENT PROGRAM Tiah Alphonso Louisiana State University Department of Educational Theory, Policy,
Minnesota Manual of Accommodations for Students with Disabilities Training Guide
1 An Introduction to Validity Arguments for Alternate Assessments Scott Marion Center for Assessment Eighth Annual MARCES Conference University of Maryland.
CLOSING THOUGHTS The long and winding road of alternate assessments Where we started, where we are now, and the road ahead! Rachel F. Quenemoen, Senior.
New Hampshire Enhanced Assessment Initiative: Technical Documentation for Alternate Assessments Standard Setting Inclusive Assessment Seminar Marianne.
Introduction & Background Laurene Christensen National Center on Educational Outcomes National Center on Educational Outcomes (NCEO)
National Center on Educational Outcomes N C E O What the heck does proficiency mean for students with significant cognitive disabilities? Nancy Arnold,
1 Some Key Points for Test Evaluators and Developers Scott Marion Center for Assessment Eighth Annual MARCES Conference University of Maryland October.
Setting Alternate Achievement Standards Prepared by Sue Rigney U.S. Department of Education NCEO Teleconference March 21, 2005.
Large Scale Assessment Conference June 22, 2004 Sue Rigney U.S. Department of Education Assessments Shall Provide for… Participation of all students Reasonable.
Consistency/Reliability
2004 CCSSO Large-scale Conference Peasley, Deeter, Quenemoen Measurement Purgatory or Best Practice? Alternate Assessment for Students with Significant.
Assessment Population and the Validity Evaluation
New Hampshire Enhanced Assessment Initiative: Technical Documentation for Alternate Assessments 1 Introduction to Comparability Inclusive Assessment Seminar.
New Hampshire Enhanced Assessment Initiative: Technical Documentation for Alternate Assessments Alignment Inclusive Assessment Seminar Brian Gong Claudia.
Minnesota Manual of Accommodations for Students with Disabilities Training Guide
Classroom Assessment A Practical Guide for Educators by Craig A
Understanding Validity for Teachers
Chapter 4. Validity: Does the test cover what we are told (or believe)
What should be the basis of
Data Collection and Student Portfolios Shared by: Anne Arundel County Public Schools.
Session Materials  Wiki
Product Evaluation the outcome phase. Do the magic bullets work? How do you know when an innovative educational program has “worked”? How do you know.
NCCSAD Advisory Board1 Research Objective Two Alignment Methodologies Diane M. Browder, PhD Claudia Flowers, PhD University of North Carolina at Charlotte.
DEVELOPING ALGEBRA-READY STUDENTS FOR MIDDLE SCHOOL: EXPLORING THE IMPACT OF EARLY ALGEBRA PRINCIPAL INVESTIGATORS:Maria L. Blanton, University of Massachusetts.
Becoming a Teacher Ninth Edition
Measurement in Exercise and Sport Psychology Research EPHE 348.
Assessment Group for Provincial Assessments, June Kadriye Ercikan University of British Columbia.
Martha Thurlow and Laurene Christensen National Center on Educational Outcomes CEC Preconvention Workshop #4 April 21, 2010.
Leadership: Connecting Vision With Action Presented by: Jan Stanley Spring 2010 Title I Directors’ Meeting.
Curriculum and Learning Omaha Public Schools
Including Quality Assurance Within The Theory of Action Presented to: CCSSO 2012 National Conference on Student Assessment June 27, 2012.
McGraw-Hill/Irwin © 2012 The McGraw-Hill Companies, Inc. All rights reserved. CLASSROOM ASSESSMENT FOR TEACHERS The New Professional Expectation.
Student assessment AH Mehrparvar,MD Occupational Medicine department Yazd University of Medical Sciences.
Visions for the Future: Inclusive Assessments Jacqueline F. Kearns, Ed.D. University of Kentucky.
Assessing Learning for Students with Disabilities Tom Haladyna Arizona State University.
Jenefer Husman Arizona State University Jenefer Husman Arizona State University When learning seems (un)important: Future Time Perspective and post-secondary.
Enhancing the Technical Quality of the North Carolina Testing Program: An Overview of Current Research Studies Nadine McBride, NCDPI Melinda Taylor, NCDPI.
Dr. Nancy S. Grasmick July 26,2012.  Maryland is proud to be the top-ranked state in U.S. growth as reported in this study, and judged by Education Week.
Evaluating Consequential Validity of AA-AAS Presented at OSEP Conference January 15, 2008 by Marianne Perie Center for Assessment.
Validity Validity is an overall evaluation that supports the intended interpretations, use, in consequences of the obtained scores. (McMillan 17)
A presentation of methods and selected results 1.
Minnesota Manual of Accommodations for Students with Disabilities Training January 2010.
School Leadership for Students With Disabilities Project #H325A Course Enhancement Module Anchor Presentation #3.
Validity Evaluation NCSA Presentation NAAC/NCIEA GSEG CONSTORIA.
PBL Instructional Design. PBL Instructional Design Name: Name of PBL: Grade Level: Content Area:
So What is Going to be Happening with State Assessment for Students with Disabilities for 2007/2008? Peggy Dutcher Fall 2007 Assessment and Accountability.
ELED 6560 Summer Learning Exercises #10 The Un-Natural Part of Teaching  Five Ways that Teaching Behavior is Un-Natural 1. Helping Others 2.
Required Skills for Assessment Balance and Quality: 10 Competencies for Educational Leaders Assessment for Learning: An Action Guide for School Leaders.
Evaluation and Assessment Evaluation is a broad term which involves the systematic way of gathering reliable and relevant information for the purpose.
Teacher Professional Learning and Development Presentation for PPTA Curriculum Workshops 2009.
Research and Evaluation
Consequential Validity
Lecturette 2: Steps to Improve Data Use
Validity and Reliability
Classroom-based assessment to promote equity
Science, Technology, and Engineering
Federal Policy & Statewide Assessments for Students with Disabilities
Assessment Population and the Validity Evaluation
William D. Schafer, Joyce Wang, and Vivian Wang University of Maryland
Lecturette 2: Steps to Improve Data Use
Links for Academic Learning: Planning An Alignment Study
Assessment Literacy: Test Purpose and Use
Presentation transcript:

New Hampshire Enhanced Assessment Initiative: Technical Documentation for Alternate Assessments Consequential Validity Inclusive Assessment Seminar Elizabeth Towles-Reeves

2 Peer Review Validity Criterion NCLB requires that state assessment systems, including alternate assessments, “be valid for the purposes for which the assessment system is used; be consistent with relevant, nationally recognized professional and technical standards, and be supported by evidence…of adequate technical quality for each purpose” (NCLB, 2001, §200.2(b)(4)(I,ii)).

OBSERVATION INTERPRETATION COGNITION  Student Population  Academic Content  Theory of Learning  Assessment System  Test Development  Administration  Scoring  Reporting  Alignment  Item Analysis/DIF/Bias  Measurement Error  Scaling and Equating  Standard Setting VALIDITY EVALUATION  Empirical Evidence  Theory and Logic (argument)  Consequential Features The Assessment Triangle and Validity Evaluation (Marion, Quenemoen, & Kearns, 2006)

4 What is Consequential Validity? Messick (1989) originally introduced consequences to the validity argument. Later, Shepard (1993, 1997) broadened the definition by arguing one must investigate both positive/negative and intended/unintended consequences of score-based inferences to properly evaluate the validity of the assessment system.

5 So What? There is overwhelming support for answering the “So What” question (Haertal, 1999; Kane, 2002; Kleinert et al., 2001; Lane & Stone 2002; Shepard, 1997), but at the same time differing stakeholder views must be included to present a convincing validity argument (Lane & Stone, 2002; Linn 1998; Ryan, 2002).

6 Intended Consequences Lane and Stone (2002) suggest that state assessments are intended to impact: – Student, teacher, and administrator motivation and effort; – Curriculum and instructional content and strategies; – Content and format of classroom assessments; – Improved learning for all students; – Professional development support; – Use and nature of test preparation activities; and – Student, teacher, administrator, and public awareness and beliefs about the assessment, criteria for judging performance, and the use of assessment results.

7 Unintended Consequences At times, however, Lane and Stone (2002) propose unintended consequences are possible such as: – Narrowing of curriculum and instruction to focus only on the specific learning outcomes assessed; – Use of test preparation materials that are closely linked to the assessment without making changes to the curriculum and instruction; – Use of unethical test preparation materials; and – Inappropriate use of test scores by administrators.

8 Consequential Validity Evaluation Questions Before you consider investigating any consequential validity questions for your alternate assessment judged against alternate achievement standards (AA-AAS), you must determine: – What is the purpose of the AA-AAS? – How will the scores of the AA-AAS be used? – What stakeholders are important to helping you understand the consequences of the AA-AAS: students, parents, teachers, administrators, community members, experts?

9 Consequential Validity Evaluation Questions Once you determine purpose and use, you may then ask: – What are the intended and unintended consequences based on the purpose and use of the AA-AAS? – Are the intended and unintended consequences positive or negative?

10 Looking to our Past to Prepare for the Future Research on the consequential validity of alternate assessments from the perspective of: – Students/Parents Research Questions: – What benefits to students have accrued from the participation in AA-AAS? – What is the extent to which students have accessed the general education curriculum? – What is the impact of the AA-AAS on students’ IEP development? – What is the relationship between student performance in AA- AAS and post-school life outcomes? – What student, teacher, and instructional variables influence parents’ perceptions regarding the AA-AAS?

11 Looking to our Past to Prepare for the Future Research on the consequential validity of alternate assessments from the perspective of: – Teachers Research Questions: – What benefits to teachers have accrued from the participation of students in the AA-AAS? – What is the extent to which alternate assessments are a part of daily classroom routine? – What is the relationship between alternate assessment scores and the amount of time spent working on the assessment? – To what extent do teacher and instructional variables predict alternate assessment scores? – Which student, teacher, and instructional variables influence teachers’ perceptions regarding the AA-AAS? – What is the impact of the AA-AAS on teachers’ daily instruction?

12 Looking to our Past to Prepare for the Future Research on the consequential validity of alternate assessments from the perspective of: – School Research Questions: – To what extent are students included in the accountability process? – Is there any relationship between student performance in the AA-AAS and student performance in the general assessment? Obviously, there is no way to address all these research questions at once. This afternoon, we will examine some of these studies in-depth and also discuss ways in which to prioritize what studies are most important for your state to conduct in the short term-and long-term.