Raising the Bar: Debunking Myths About Scoring and Data Charts Dan Wiener Administrator of Inclusive Assessment Massachusetts Department of.

Slides:



Advertisements
Similar presentations
Assessing Student Performance
Advertisements

Grade-level and Competency Portfolios for 2014 MCAS-Alt
Independence Module X NJ APA Teacher Training - Module X 1.
Steps to Developing an Entry Part Two Module V NJ APA Teacher Training – Module V 1.
The SCPS Professional Growth System
Portfolio Review Process Georgia Alternate Assessment.
Designing Scoring Rubrics. What is a Rubric? Guidelines by which a product is judged Guidelines by which a product is judged Explain the standards for.
2015 MCAS-Alt: How to Submit a “Complete” Portfolio
MCAS-Alt: Alternate Assessment in Massachusetts Technical Challenges and Approaches to Validity Daniel J. Wiener, Administrator of Inclusive Assessment.
What (the Heck) Does “Proficient” Mean? Standard Setting on the MCAS Alternate Assessment Daniel J. Wiener Massachusetts Dept. of Education CCSSO - San.
West Virginia State Assessments: What do families need to know? [INSERT DATE]
Presented by: Laura Hines MCAS-Alt Teacher Consultant October 2014 MCAS Alternate Assessment (MCAS-Alt) Creating Portfolios that Address Access Skills.
Welcome to TCAP-Alt PA K-2 Scoring Training Section 1: Required Components: Is this Portfolio Scorable?
SCORING. INTRODUCTION & PURPOSE Define what SCORING means for the purpose of these modules Explain how and why you should use well-designed tools, such.
Welcome to Third Grade’s Parent Night
Extended Assessments Elementary Mathematics Oregon Department of Education and Behavioral Research and Teaching January 2007.
WELCOME to the Fall 2015 MCAS-Alt EDUCATOR TRAINING MCAS-Alt Update: What’s New for 2016? Dan Wiener Administrator of Inclusive Assessment With Measured.
ACT EXPLORE Understanding Your Results. Why the EXPLORE? Research shows that monitoring student progress in middle school, raising parent and student.
Developing a Monitoring and Pre-Scoring Plan for the Virginia Grade Level Alternative (VGLA) Adapted from the Virginia Department of Education Division.
LHS Communication Arts Department Summer 2011 What kind of team are we going to be?
EQAO Assessments and Rangefinding
Assessment and Testing
What do they really know, and what can they really do? Standards-Based Formative Assessment ©This document is the property of the Quakertown Community.
Alternate Proficiency Assessment Erin Lichtenwalner.
APA NJ APA Teacher Training 2 What is the Purpose of the APA? To measure performance of students with the most significant cognitive disabilities.
Georgia Alternate Assessment The Importance of Portfolio Review Prior to Submission February 2011Dr. John D. Barge, State School Superintendent.
Reviewing, Revising and Writing Effective Constructed- Response Mathematics Items 1 Writing and scoring test questions based on Ohio’s Academic Content.
Welcome Parents! FCAT Information Session. O Next Generation Sunshine State Standards O Released Test Items O Sample Test.
Educational Measurement and School Accountability Directorate National Assessment Program – Literacy and Numeracy (NAPLAN) Understanding your child’s report.
ACCESS for ELLs Score Changes
Every Student Succeeds Act (ESSA)
Designing Scoring Rubrics
Grading in 8th Grade Social Studies
Secondary Assessment & Grade Reporting
MCAS-Alt Teacher Consultants
Understanding Standards Event Fashion & Textile Technology
Understanding the Alternate ACCESS for ELLs Individual Student Report
“Grade-level” and “Competency” Portfolios
ELA–Writing: Access Skills and Low Entry Points
Administrator of Inclusive Assessment
Wisconsin Department of Public Instruction
Update on Data Collection and Reporting
Concept of Test Validity
Standards-based Grading Module 4: Inappropriate Grade Calculation
Understanding the Next-Generation MCAS
PD Monday March 14, 2016.
In-Service Teacher Training
Understanding the Next-Generation MCAS
Writing Tasks and Prompts
Mathematics Subject Leaders’ Network Meeting
Release of PARCC Student Results
Standards- based grading for parents- grades k-2
ELA–Writing: Access Skills and Low Entry Points
Region 14 Mathematics Curriculum Update K-8
Analyzing data: First steps
MCAS-Alt “Grade-level” and “Competency” Portfolios
MCAS-Alt Teacher Consultants
LEAP TH GRADE.
Raising the Bar: Debunking Myths About Scoring
Understanding the Next-Generation MCAS
Understanding the Next-Generation MCAS
Genius Hour Presentation
Using outcomes data for program improvement
Orientation for District IEP Teams and Other Interested Staff
Using the 4.0 Gradebook.
Standards-Based Reporting
Somerset Primary Data Report/SBG Information Session
Assessing Students With Disabilities: IDEA and NCLB Working Together
MCAS-Alt “Grade-level” and “Competency” Portfolios
Standards-Based Reporting
Presentation transcript:

Raising the Bar: Debunking Myths About Scoring and Data Charts Dan Wiener Administrator of Inclusive Assessment Massachusetts Department of Elementary and Secondary Education

“Playing It Safe” to Get the Highest Score

What Does This Data Chart Tell You About the Student’s Achievement? Student mastered after 2-3 attempts: probably was too easy. Time better spent teaching more challenging skills.

Scoring Portfolios – Quick Review Level of Complexity (LOC) Unmatched Access Skills Entry Points Grade Level/Competency To Standard NOTE: Most portfolio strands are LOC=3 (Entry Points) Massachusetts Department of Elementary and Secondary Education

Scoring Accuracy and Independence Demonstration of Skills and Concepts (DSC) (Percent Accuracy) Independence (IND) (Percent unassisted responses)

How is the overall score determined from the LOC, DSC, and IND scores? The scores in LOC, DSC, and IND are combined into a “strand score,” based on the “final 1/3 time frame.” Strand scores are averaged together to give an overall score in a content area. Scoring is explained in Educator’s Manual (Pp. 63-73) This table is used to calculate strand scores for LOC=3 (Also see Educator’s Manual for MCAS-Alt) Massachusetts Department of Elementary and Secondary Education

A score of 100% accuracy and 100% independence isn’t necessary! Table shows that any scores above 51% accuracy and 51% independence earn a score of Progressing. The take-away: Students can attempt more complex entry points without fear of lowering their score. Therefore… Push your students to learn more challenging skills. Have students attempt new, different skills each year.

Data Charts that Confuse Scorers

“Stair-Step” Data Charts Accuracy and independence are virtually identical on every date. Massachusetts Department of Elementary and Secondary Education

“Purple Line” Data Charts Is this technically possible? Massachusetts Department of Elementary and Secondary Education

Data Charts: Three Likely Scenarios

“Zero-Prompt Instruction:” Always Independent, Working on Accuracy Massachusetts Department of Elementary and Secondary Education

“Errorless Teaching:” Always Accurate, Working on Independence Massachusetts Department of Elementary and Secondary Education

Working on both Accuracy and Independence Massachusetts Department of Elementary and Secondary Education

Some Thoughts About Accuracy and Independence

Accuracy and Independence: Corrections/Edits/Prompts If student gets the answer correct, whether prompted or not, that response is accurate. Write “P” next to the prompted response on a work sample. If response is inaccurate, then it’s incorrect. If a prompt of any kind is given, then the response is “non-independent” (i.e., prompted) If corrections/edits are made only by the student, then it is an accurate and independent response. Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Charting Accuracy and Independence for a Series of Responses Question Number Accuracy Independence Question 1 Correct response (accurate) Verbal prompt (not independent) Question 2 Incorrect response (inaccurate) Question 3 Gestural prompt Question 4 Question 5 No prompt (independent) Overall Percent 60% accuracy (3/5 correct) 20% independence (1/5 independent) All prompted responses, even those with partial prompts, are considered not independent.

…But Percentages Must Make Sense! This sample was not scorable because scorer couldn’t verify how four questions could be 43% accurate and 43% independent.

Portfolio data and evidence must be authentic! Please do not fabricate information, falsify dates, replicate or mass-produce evidence or data for more than one student. In cases where this occurs, investigations will be requested and scores will be invalidated. Start early enough in the year to complete your students’ portfolios by March 29, 2018. Attend training and review sessions, and ask for help if you need it, before it’s too late.

We sometimes hear this comment… “I submitted exactly the same work for all my students, but some were scored Incomplete while others received a score.” Maybe something wasn’t clear to one (or more) of the scorers. If vague or difficult to understand, multiple interpretations may result. This should signal the need for greater clarity. …Or maybe the work wasn’t all the same.

In Conclusion… Be clear and concise in describing how the work was created. Score all student work. Document accuracy and prompts clearly, item-by- item. Scorers should see which responses were accurate and independent. If not obvious, explain how it was scored. Review your portfolio with another teacher before submitting. Use all available supports and resources.

Questions? Thank you!