Comparing State Reading and Math Performance Standards Using NAEP Don McLaughlin Victor Bandeira de Mello National Conference on Large-Scale Assessment.

Slides:

Advertisements

Similar presentations

Starting Early With English Language Learners: First Lessons from Illinois June 21, 2012 Maggie Severns National Association of Latino Elected and Appointed.

Advertisements

Title I & Title III Annual Parent Meeting

1 Understanding the New School Report Card. Education Is Changing 45 States and 3 territories have adopted the Common Core State Standards GOAL: ALL students.

Pennsylvania Value-Added Assessment System (PVAAS) High Growth, High Achieving Schools: Is It Possible? Fall, 2011 PVAAS Webinar.

. Information from “Countdown to Accountability” Summer Leadership Institute July 2002 Arizona School Boards Association from presentations by Chris Thomas.

ALIGNING STATE SYSTEMS TO IMPROVE TEACHER QUALITY FOR STUDENTS WITH DISABILITIES.

Achieve Data Profile: Pennsylvania April AMERICAN DIPLOMA PROJECT NETWORK The Big Picture n To be successful in today’s economy, all students.

Understanding Grade Level and School Growth Reports Office of Assessment and Accountability.

Pennsylvania’s Continuous Improvement Process. Understanding AYP How much do you know about AYP?

Highlights from TIMSS 2011 Jack Buckley National Center for Education Statistics Washington, DC December 11, 2012.

Beyond the Classroom: The Use of Essential Skills for Remediation and Extension Christine Koch November 2008.

1 NAEP th Grade Economics Assessment. 2 ► First NAEP assessment of economics ► Content areas: market economy, national economy, and international.

Teachers for a New Era Pupil Assessment Phase I: Lessons Learned From a Town- Gown Partnership Jason Irizzarry, Ed.D. Teachers for a New Era Project University.

Parent, Student, and Teacher Perceptions of School Climate: Investigations Across Organizational Levels Christine DiStefano Diane M. Monrad R. John May.

Using School Climate Surveys to Categorize Schools and Examine Relationships with School Achievement Christine DiStefano, Diane M. Monrad, R.J. May, Patricia.

Catherine Cross Maple, Ph.D. Deputy Secretary Learning and Accountability

1 The New York State Education Department New York State’s Student Reporting and Accountability System.

Mark DeCandia Kentucky NAEP State Coordinator

WELCOME ACS Beirut Elementary School Progress Reports.

Impact of fine arts on academic success A Comparison study EDRS 5305 Fall 2004 Dr. Teresa Cortez.

Creating Assessments with English Language Learners in Mind In this module we will examine: Who are English Language Learners (ELL) and how are they identified?

Including a detailed description of the Colorado Growth Model 1.

High Stakes Testing EDU 330: Educational Psychology Daniel Moos.

Identifying the gaps in state assessment systems CCSSO Large-Scale Assessment Conference Nashville June 19, 2007 Sue Bechard Office of Inclusive Educational.

High School Mathematics: Where Are We Headed? W. Gary Martin Auburn University.

Austin Elementary School August 20 and 27, 2015

Copyright ©2006. Battelle for Kids. Understanding & Using Value-Added Analysis.

Florida’s Implementation of NCLB John L. Winn Deputy Commissioner Florida Department of Education.

SY PVAAS Scatter Plots State to IU Region to School District Grades 4-8, 11 Math & Reading PVAAS Statewide Team for PDE Contact your IU PVAAS contact.

Slide 1 Estimating Performance Below the National Level Applying Simulation Methods to TIMSS Fourth Annual IES Research Conference Dan Sherman, Ph.D. American.

Mathematics and Science Education U.S. Department of Education.

Bob Mattila.  No Child Left Behind (NCLB) is an act that was passed in 2002 with the goal of - "An act to close the achievement gap with accountability,

Teaching Today: An Introduction to Education 8th edition

1 Watertown Public Schools Assessment Reports 2010 Ann Koufman-Frederick and Administrative Council School Committee Meetings Oct, Nov, Dec, 2010 Part.

CALIFORNIA DEPARTMENT OF EDUCATION Jack O’Connell, State Superintendent of Public Instruction Results of the 2005 National Assessment of Educational Progress.

0 Michele Sonnenfeld NAEP State Coordinator Florida Department of Education October 2006 Florida Association of Science Supervisors.

Mark DeCandia Kentucky NAEP State Coordinator

NAEP 2011 Mathematics and Reading Results Challis Breithaupt November 1, 2011.

NAEP 2011 Mathematics and Reading Results NAEP State Coordinator Mark DeCandia.

The Do’s and Don’ts of High-Stakes Student Achievement Testing Andrew Porter Vanderbilt University August 2006.

35th Annual National Conference on Large-Scale Assessment June 18, 2005 How to compare NAEP and State Assessment Results NAEP State Analysis Project Don.

MEAP / MME New Cut Scores Gill Elementary February 2012.

Mathematics and Science Partnerships: Summary of the FY2006 Annual Reports U.S. Department of Education.

The Nation’s Report Card: Geography National Assessment of Educational Progress (NAEP)

Future Ready Schools National Assessment of Educational Progress (NAEP) in North Carolina Wednesday, February 13, 2008 Auditorium III 8:30 – 9:30 a.m.

School-level Correlates of Achievement: Linking NAEP, State Assessments, and SASS NAEP State Analysis Project Sami Kitmitto CCSSO National Conference on.

CREATE – National Evaluation Institute Annual Conference – October 8-10, 2009 The Brown Hotel, Louisville, Kentucky Research and Evaluation that inform.

1 Children Left Behind in AYP and Non-AYP Schools: Using Student Progress and the Distribution of Student Gains to Validate AYP Kilchan Choi Michael Seltzer.

Understanding ITBS Scores A Parent Guide. Overview ITBS testing is done in October at BMS. The testing is a “norm-referenced” test that compares a student’s.

Preliminary Data: Not a Final Accountability Document1 SAISD TAKS Performance Board Work Session June 2005 Office of Research, Evaluation,

Special Education Plan Feedback Sessions. Agenda Welcome and Introductions Department of Student Services Purpose Why are we updating the Special Education.

ACT ASPIRE GROWTH REPORTS. DISTRICTS AND SCHOOLS THAT PARTICIPATED IN ACT ASPIRE ASSESSMENTS (READING, MATH, ENGLISH, SCIENCE AND WRITING) WITH AN N COUNT.

1 Maximizing Predictive Accuracy of District Benchmarks Illuminate Education, Inc. User’s Conference Aliso Viejo, California June 4&5, 2012.

Understanding AzMERIT Results and Score Reporting An Overview.

2015 State PARCC Results Presented to the Board of Elementary and Secondary Education Robert Lee MCAS Chief Analyst and Acting PARCC Coordinator October.

Vertical Articulation Reality Orientation (Achieving Coherence in a Less-Than-Coherent World) NCSA June 25, 2014 Deb Lindsey, Director of State Assessment.

C R E S S T / CU University of Colorado at Boulder National Center for Research on Evaluation, Standards, and Student Testing Measuring Adequate Yearly.

Gary W. Phillips Vice President & Institute Fellow American Institutes for Research Next Generation Achievement Standard Setting Symposium CCSSO NCSA New.

1. 2 National Assessment of Educational Progress (NAEP) Survey of academic achievement for the nation and the states Assesses various subjects at grades.

Evaluation Requirements for MSP and Characteristics of Designs to Estimate Impacts with Confidence Ellen Bobronnikov March 23, 2011.

Department of Research and Evaluation

Comparability of Assessment Results in the Era of Flexibility

2015 PARCC Results for R.I: Work to do, focus on teaching and learning

Dr. Robert H. Meyer Research Professor and Director

NECAP PRESENTATION.

Using outcomes data for program improvement

Assessment for Learning

Why We Should be Skeptical about the Common Core

Lake Bluff Gifted Education Benchmarking Progress Report

AACC Mini Conference June 8-9, 2011

Presentation transcript:

Comparing State Reading and Math Performance Standards Using NAEP Don McLaughlin Victor Bandeira de Mello National Conference on Large-Scale Assessment San Antonio June 2003

Another look at NAEP’s role in the context of NCLB Confirm state gains. Help state assessment programs to help each state’s schools and students. Answer questions for/about the state assessment. Provide a test bed for new solutions.

Complexities in confirming state gains Sampling Exclusions, Absences, Small samples Measurement Accommodations, Framework, Format, Time of year, Motivational context Differential sensitivity Analysis Gaps Design Grade-to-grade growth

NAEP’s strengths for helping state assessment programs NAEP is administered with uniform procedures to a representative sample of public school students in every state. This can illuminate cross-assessment comparisons. NAEP can carry out validity research to address questions relevant to all assessments (e.g., validity of accommodations).

Questions NAEP can answer for state assessments 1. How do your state’s performance standards for math and reading compare with other states’? 2. How do your state’s assessment instruments compare to others in focus and difficulty? 3. Can your state assessment’s identification of schools that need improvement be enhanced?

Questions NAEP can answer for state assessments 4. Is your state’s assessment more (or less) sensitive to race or poverty differences in achievement than other states’? 5. Are differences between types of schools in your state similar to analogous differences in other states? 6. How do your identification and testing procedures for SD/ELL students differ from those in other states?

Comparison of state performance standards, using NAEP Achievement levels are set separately in each state, using a variety of standard- setting methodologies. Does “meeting the standard” mean something different in each state? … How different? How do states’ targets for adequate yearly progress differ?

Comparison of state performance standards, using NAEP By comparing NAEP results with state assessment results in the same schools, we can approximate where on the NAEP scale the cutpoint for the state’s standard lies. This requires matching a database of state assessment results to the NAEP database.

National Longitudinal School-Level State Assessment Score Database Scores provided by State Education Agencies Processed by the American Institutes for Research Funded by the U.S. Department of Education Policy and Program Studies Service School-level scores in the public domain.

National Longitudinal School-Level State Assessment Score Database Scores for > 80,000 public schools in 49 states, Puerto Rico, and DC from through (but not all years in all states) merged with Common Core of Data

National Longitudinal School-Level State Assessment Score Database Available at Contact:

Method Match the “percent meeting the state’s standard” in each school to the distribution of NAEP plausible values in the school’s NAEP sample.

e. g., If 30 percent of the students in a school are reported by the state testing program to meet the state’s standard, Find the NAEP plausible value that has 30 percent of the distribution higher than it. (If the NAEP mean in the school is 220, that value might be 230.)

o o o o o N o o o o o N o o o o o N o N o o o o o o N o o o o No o o o o N o N o o o o N o o o o o o N o o o o o o o N o o o o o N o o o N o o o o o o o o o o N o o o o N o o o o o N o o o o o o N o o o o o o N o o o N o o o o in a school selected to participate in NAEP o o o o o o o o o o o o o o o o o o o o o o o o N o o o o o N o o o o o N o N o o o o o o N o o o o N o o o o o N o N o o o o N o o o o o o N o o o o o o o N o o o o o N o o o N o o o o o o o o o o N o o o o N o o o o o N o o o o o o N o o o o o o N o o o N o o o o Grade 4 Students Some are selected for NAEP Some meet state standard

o N o o o N o o o N o o o o o N o o N o o o N o o o o o o o N o o o o o o o o N o o N o o o o o o o o o o o N o o o o o N o o o o o N o o o o o o N o o o o o o N o o o o o o N o o N o o o o o o o o N o o o o o o o o N o o N o Achievement Scale o o o o o N o o o o o N o o o o o N o N o o o o o o N o o o o No o o o o N o N o o o o N o o o o o o N o o o o o o o N o o o o o N o o o N o o o o o o o o o o N o o o o N o o o o o N o o o o o o N o o o o o o N o o o N o o o o in a school selected to participate in NAEP o N o o o N o o o N o o o o o N o o N o o o N o o o o o o o N o o o o o o o o N o o N o o o o o o o o N o o o o o o o o N o o o o o N o o o o o o N o o o o o o N o o o o o o N o o N o o o o o o o o N o o o o o o o o N o o N o Achievement Scale Achievement frequency distribution in the school

n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n Achievement Scale (NAEP Reading) o N o o o N o o o N o o o o o N o o N o o o N o o o o o o o N o o o o o o o o N o o N o o o o o o o o o o o N o o o o o N o o o o o N o o o o o o N o o o o o o N o o o o o o N o o N o o o o o o o o N o o o o o o o o N o o N o Achievement Scale (State Reading Assessment) in a school selected to participate in NAEP 30 % of the distribution in one school might correspond to 230 on the NAEP scale, in that school.

Assumption State reading assessment score variation is strongly related to NAEP score variation.

Assumption State reading assessment score variation is strongly related to NAEP score variation. If so, then we should get approximately the same estimate of the position of the state standard on the NAEP scale from nearly all NAEP schools in the state.

Test of the Assumption

Conclusions NAEP can provide a link between state achievement standards, especially at the proficient and advanced levels. There is substantial variation in standards between states. Percentages of students meeting state standards are lower in states in which the standards are set high. Most of the links for “below proficient” standards are weak.

Summary NAEP can contribute most to Leaving No Child Behind by helping state assessment programs give useful feedback to parents, teachers, and school administrators.