Including Quality Assurance Within The Theory of Action Presented to: CCSSO 2012 National Conference on Student Assessment June 27, 2012

Quality Assurance - A continuous cycle that improves and refines the processes necessary to produce valid assessments to the highest standards. Ensures that excellence is inherent in all components of the process. – Plan – Do – Study – Act Quality Assurance as a Continuous Cycle

Evaluated assessment systems in most states and the U.S. territories through the federal standards and assessment peer review process. Developed interpretative arguments for the validity of assessments and conducted studies to test the claims and assumptions in the interpretive argument for a number of states. Provided technical assistance for reviewing the technical quality of assessment and data quality of assessment systems. Edcount’s relevant work:

Students are appropriately tested for identified testing options Test scores accurately reflect student knowledge and skills Reports provide valid scores and are used appropriately Quality Assurance Within the Theory of Action - Claims

The Learner Characteristics Inventory (LCI) – A survey designed to collect information about the population of students who participate in alternate assessments based on alternate achievement standards (AA-AAS). – Helps states identify and evaluate “who” is participating in the alternate assessments. Provides information to help direct participation rules and guidance. Students Are Appropriately Tested for Identified Testing Options – Special Studies

Accommodation Studies – The extent to which accommodations are provided on the state assessments. – Literature review on best practices for accommodation use. – Provides information and evidence to improve best practices, guidance, and monitoring of accommodation use. Students Are Appropriately Tested for Identified Testing Options – Special Studies

Performance Level Descriptor (PLD) studies – Examine the extent to which assessment cut scores appropriately distinguish between student performance levels, and whether the performance level descriptors adequately capture meaningful distinctions in grade-level performance. – Engage teachers in rating and feedback processes to help administrators understand how teachers use and interpret PLD’s – Evaluate the PLD’s for coherence, consistency, and accuracy. Test Scores Accurately Reflect Student Knowledge and Skills – Special Studies

Performance Level Descriptor (PLD) studies cont. – Provides information to help states refine their standard setting processes, focus their professional development and support their interpretations on the state assessments. Test Scores Accurately Reflect Student Knowledge and Skills – Special Studies

Alignment Studies – Evaluations of the alignment between academic content standards and alternate assessments. – Evaluations of the alignment between academic content standards and general assessments. – Provides information to support and improve the design and development of assessments. Test Scores Accurately Reflect Student Knowledge and Skills – Specialized Studies

Cognitive Labs – Collect information about whether assessments are eliciting the intended cognitive processes. – The cognitive labs can help states understand why students may struggle with certain items or content areas on an assessment, and, as a result, provide feedback about ways to develop accessible assessments and ways to improve instruction. Test Scores Accurately Reflect Student Knowledge and Skills – Special Studies

Consequential Validity Studies – Evaluate and collect evidence about the intended and unintended consequences of a states large scale assessment system. – Provides evidence to inform policies and processes and focus professional development. Reports Provide Valid Scores and Are Used Appropriately - Special Studies

Special Studies – PLD study – Accommodation Study – Consequential Validity Evaluation – Cognitive labs Professional Development – Data Use Workshop – Curriculum Mapping Boot Camps Work Within Puerto Rico

Special Studies – PLD studies – Cognitive Process Validity Study Technical Assistance For: – Development of a balanced assessment system with formative and interim assessments – Development of an accountability System – Small school review – Development of a data management system Work Within the Laurent Clerc Center at Gallaudet University

Provided focus for professional development Provided information to improve the design and development activities for assessments and reports Provided context as well as empirical evidence to support policies and processes How Did the Work Inform Quality Assurance and Processes

Contact Information Jennifer Stegman, Senior Associate edCount LLC 5335 Wisconsin Avenue NW Washington DC edCount.com