But What Does It All Mean? Key Concepts for Getting the Most Out of Your Assessments Emily Moiduddin.

Slides:



Advertisements
Similar presentations
Ed-D 420 Inclusion of Exceptional Learners. CAT time Learner-Centered - Learner-centered techniques focus on strategies and approaches to improve learning.
Advertisements

Teaching Strategies Gold
Standardized Tests: What Are They? Why Use Them?
Nettie’s Nickel 4 Comprehensive AT Report Writing 2 Support a District’s Offer of FAPE Nettie Fischer, ATP July 22, 2014.
Wortham: Chapter 2 Assessing young children Why are infants and Preschoolers measured differently than older children and adults? How does the demand for.
Standardized Tests What They Measure How They Measure.
Grading. Why do we grade? To communicate To tell students how they are doing To tell parents how students are doing To make students uneasy To wield power.
The Head Start Child Development and Early Learning Framework A Focus on School Readiness for Infant and Toddler Children August 19, 2014 RGV Pre-Service.
Kindergarten Readiness:
CHILD OUTCOMES BASELINE AND TARGETS FOR INDICATOR 7 ON THE STATE PERFORMANCE PLAN State Advisory Panel for Exceptional Children November 12, 2009 January.
STANDARDIZED TESTING MEASUREMENT AND EVALUATION  All teaching involves evaluation. We compare information to criteria and ten make judgments.  Measurement.
Assessment. What are assessments supposed to be used for Group children- reading groups, giving children challenging activities. Group children- reading.
Assessing Intelligence
Classroom Assessment A Practical Guide for Educators by Craig A
Supporting PreK Teachers During Act 3 Implementation.
Measurement Concepts & Interpretation. Scores on tests can be interpreted: By comparing a client to a peer in the norm group to determine how different.
VPK Assessment Data Analysis Topics of Discussion  Reflections  Overview of VPK Assessment Process and Reports  AP1 Data Analysis  Resources  Lesson.
Standardization the properties of objective tests.
Overall Teacher Judgements and
 ECE II and ECE III.  Watching children and recording (documenting) capabilities over time.  This information is then used as the basis for curriculum.
Getting Ready to Get the Most out of Your Child Assessments.
LECTURE 06B BEGINS HERE THIS IS WHERE MATERIAL FOR EXAM 3 BEGINS.
Analyzing Reliability and Validity in Outcomes Assessment (Part 1) Robert W. Lingard and Deborah K. van Alphen California State University, Northridge.
SPP Indicators B-7 and B-8: Overview and Results to Date for the Florida Prekindergarten Program for Children with Disabilities PreK Coordinators Meeting.
School Readiness: We’re Better Together
Chapter 3 Understanding Test Scores Robert J. Drummond and Karyn Dayle Jones Assessment Procedures for Counselors and Helping Professionals, 6 th edition.
Stat 1510: Statistical Thinking and Concepts 1 Density Curves and Normal Distribution.
Instrumentation (cont.) February 28 Note: Measurement Plan Due Next Week.
Miller Function & Participation Scales (M-FUN)
Minnesota’s School Readiness Study 1 Developmental Assessment at Kindergarten Entrance.
The Creative Curriculum for Infants, Toddlers, & Twos
Diagnostics Mathematics Assessments: Main Ideas  Now typically assess the knowledge and skill on the subsets of the 10 standards specified by the National.
+ Third Party Evaluation – Interim Report Presentation for Early Childhood Advisory Council December 19, 2013.
Welcome The challenges of the new National Curriculum & Life without Levels.
Issues in Selecting Assessments for Measuring Outcomes for Young Children Issues in Selecting Assessments for Measuring Outcomes for Young Children Dale.
MELS 601 Ch. 7. If curriculum can be defined most simply as what is taught in the school, then instruction is the how —the methods and techniques that.
Introduction Gathering Information Observation Interviewing Norm Referenced Tools Authentic Assessment Characteristics of Authentic Assessment – 7M’s Validity.
Understanding and Using the Results from the NCSEAM Family Survey Batya Elbaum, Ph.D. NCSEAM Measuring Child and Family Outcomes NECTAC National TA Meeting.
Chapter 2 ~~~~~ Standardized Assessment: Types, Scores, Reporting.
TYPES OF SCORES (PP IN HUTCHINSON; MERRELL & PLANTE; SECOND PART OF HAYNES AND PINDZOLA )
Assessing Learners with Special Needs: An Applied Approach, 6e © 2009 Pearson Education, Inc. All rights reserved. Chapter 5: Introduction to Norm- Referenced.
Assessment Information from multiple sources that describes a student’s level of achievement Used to make educational decisions about students Gives feedback.
Early Childhood Outcomes Center Orientation for New Outcomes Conference Participants Kathy Hebbeler Lynne Kahn The Early Childhood Outcomes (ECO) Center.
Assessment without Levels September Effective  Effective Assessment Systems should;  Give reliable information to parents about how their child,
Technical Adequacy of Tests Dr. Julie Esparza Brown SPED 512: Diagnostic Assessment.
SAT’s Information Parent’s Meeting 10 th February February 2016.
The Normal Distribution and Norm-Referenced Testing Norm-referenced tests compare students with their age or grade peers. Scores on these tests are compared.
Helmingham Community Primary School Assessment Information Evening 10 February 2016.
Chapter 3 Selection of Assessment Tools. Council of Exceptional Children’s Professional Standards All special educators should possess a common core of.
Lostock Gralam CE Primary School Parent Information Meeting January 2016.
Testing Intelligence. ARE YOU OVER-TESTED?  Your age group is the most tested group in the history of the United States.
KARORI NORMAL SCHOOL We encourage our children to be the best they can be “Tino pai ake” 2016 Reporting Cycles Information.
Assessment Assessment is the collection, recording and analysis of data about students as they work over a period of time. This should include, teacher,
Bennington County Head Start Child Outcomes Midpoint Data Report.
Understanding ERB Scores
Policy & practice Some thoughts There is no assessment panacea. There are strengths and weaknesses for both observation and question- based assessment.
Measurement Chapter 6. Measuring Variables Measurement Classifying units of analysis by categories to represent variable concepts.
Prepared for the Office of Head Start by ICF International School Readiness Goals: Using Data to Support Child Outcomes.
In 2014/15 a new national curriculum framework was introduced by the government for Years 1, 3, 4 and 5 However, Years 2 and 6 (due to statutory testing)
Good Morning and welcome. Thank you for attending this meeting to discuss assessment of learning, pupil progress and end of year school reports.
Developmentally Appropriate Practices. Five Guidelines For Developmentally Appropriate Practices.
Assessment in Language Teaching: part 1 Lecture # 23
CHAPTER 3: Practical Measurement Concepts
Bennington County Head Start and early head start Report child outcomes school readiness goals.
Integrating Outcomes Learning Community Call February 8, 2012
Developmentally Appropriate Practices
Chapter 10: Intelligence & Testing
Descriptive Statistics
EDUC 2130 Quiz #10 W. Huitt.
Presentation transcript:

But What Does It All Mean? Key Concepts for Getting the Most Out of Your Assessments Emily Moiduddin

Why Do We Assess? Improve children’s learning –Meet the needs of children who are struggling –Challenge children who are excelling Identify key challenges in our classrooms Figure out what resources we need Improve children’s learning –Meet the needs of children who are struggling –Challenge children who are excelling Identify key challenges in our classrooms Figure out what resources we need 2

Objectives Understand key assessment concepts 1. Data 2. Data quality 3. Mean scores 4. Distribution of scores 5. Normal curve 6. Growth over time Feel empowered to communicate about your program Understand key assessment concepts 1. Data 2. Data quality 3. Mean scores 4. Distribution of scores 5. Normal curve 6. Growth over time Feel empowered to communicate about your program 4

Additional Resources Getting Ready to Get the Most out of Your Child Assessments 5

Key Concept 1: Data Data = Information you collect about children, families, and classrooms –Information on children’s progress for reporting purposes –Ongoing observational assessments –Attendance records –Information about families –Etc. In other words, any information you can use to improve child learning! Data = Information you collect about children, families, and classrooms –Information on children’s progress for reporting purposes –Ongoing observational assessments –Attendance records –Information about families –Etc. In other words, any information you can use to improve child learning! 6

Key Concept 2: Data Quality The birth date problem: –The birth year on record would mean the child is 12 –Parent says birthday is different than what is in your records –Two different birth dates recorded for one child Quality data avoids the birth date problem. It is: –Correct –Complete –Consistent The birth date problem: –The birth year on record would mean the child is 12 –Parent says birthday is different than what is in your records –Two different birth dates recorded for one child Quality data avoids the birth date problem. It is: –Correct –Complete –Consistent 7

Data Quality: Validity Tool has validity when it actually measures the area it is supposed to measure. –Example: Language assessment for a child with high verbal skills  If child is shy, may not respond to unfamiliar assessor/rater  Low scores would reflect social development rather than language –Ask, “Am I measuring what I think I’m measuring?” Tool has validity when it actually measures the area it is supposed to measure. –Example: Language assessment for a child with high verbal skills  If child is shy, may not respond to unfamiliar assessor/rater  Low scores would reflect social development rather than language –Ask, “Am I measuring what I think I’m measuring?” 8

Data Quality: Reliability An assessment tool is reliable if it provides consistent, dependable information. –Inter-rater reliability: The assessment leads different assessors/raters to the same conclusion. –Internal consistency: Items in the same tool all measure the same area of learning and development. –Test-retest reliability: If the assessment were repeated in a very short period of time, you would get the same outcome. An assessment tool is reliable if it provides consistent, dependable information. –Inter-rater reliability: The assessment leads different assessors/raters to the same conclusion. –Internal consistency: Items in the same tool all measure the same area of learning and development. –Test-retest reliability: If the assessment were repeated in a very short period of time, you would get the same outcome. 9

Data Quality: Using the Right Tools Assessments should be valid AND reliable. For assessment systems purchased or that come with a curriculum, publishers usually report validity and reliability in a manual. For ongoing, observational assessments, ask yourself: –Am I measuring what I want to measure? –Am I measuring it well? Consult with experts. Assessments should be valid AND reliable. For assessment systems purchased or that come with a curriculum, publishers usually report validity and reliability in a manual. For ongoing, observational assessments, ask yourself: –Am I measuring what I want to measure? –Am I measuring it well? Consult with experts. 10

Tips for Assuring Data Quality Train (new) teachers in: –Your program’s methods for recording important information (e.g., birth dates) –Recording information in assessments –Interpreting assessment results When reviewing data, look for odd patterns. Ask questions such as: –Do I believe the results? –Is there enough information for different teachers to come to the same conclusion? Select assessments with a record of validity and reliability for children like those in your program (check the manual!). Train (new) teachers in: –Your program’s methods for recording important information (e.g., birth dates) –Recording information in assessments –Interpreting assessment results When reviewing data, look for odd patterns. Ask questions such as: –Do I believe the results? –Is there enough information for different teachers to come to the same conclusion? Select assessments with a record of validity and reliability for children like those in your program (check the manual!). 11

Key Concept 3: Averages Average = sum of scores divided by the number of scores you have. –Also referred to as the “mean” Example: Class of 10 children –How many number words from one to ten do they know, on average? Average = sum of scores divided by the number of scores you have. –Also referred to as the “mean” Example: Class of 10 children –How many number words from one to ten do they know, on average? ChildAbbyRyanJoseJuliaMikeSuiCaliKyleAnnaKim Score Sum of scores = = 82 Average = sum of scores divided by number of children = 82 / 10 =

Using Average Scores Compare classrooms, programs or centers For example, children in Class C may need more support for learning number words than children in A or B Compare classrooms, programs or centers For example, children in Class C may need more support for learning number words than children in A or B 13

Averages at Higher Levels Examine distributions of scores at the class, program, or center level. 14

Key Concept 4: Distribution of Scores Distribution of scores = how spread out around the average a group of scores are. Class A: Children perform at similar levels. Class B: Some children excel and others struggle. Class A (5 children)Class B(5 children) Alyssa4Maria1 Derek5John2 Roberto6Angela9 Juliana5Seth8 Kevin5Jacob5 Average =5 5 15

Picturing the Distribution Number of children receiving each score Possible scores on the number-naming task 16

Key Concept 5: The Normal Curve ← Lower ScoresHigher Scores → Average 17

The Normal Curve, cont. ← Lower ScoresHigher Scores → Average Standard Deviation:

Children’s Expected Performance Norm-referenced, standard scores – Special scoring systems that can be used to compare the performance of children in your classroom to similar groups of children across the United States –Norming sample – A (large) group of children with certain characteristics whose test performance is used to develop special scoring systems –Norms – A summary of the average score and distribution of scores of children in a norming sample  How we expect children with particular characteristics to perform Norm-referenced, standard scores – Special scoring systems that can be used to compare the performance of children in your classroom to similar groups of children across the United States –Norming sample – A (large) group of children with certain characteristics whose test performance is used to develop special scoring systems –Norms – A summary of the average score and distribution of scores of children in a norming sample  How we expect children with particular characteristics to perform 19

The Normal Curve, Cont. Scores: Standard Deviation % 16%

National Comparisons Standardized Scores: 85 to Head Start Average National Average Standard Deviation:-1 to -1/20 National Average = 100 points 1 Standard Deviation = 15 points Source: FACES 2003 data, ACF 2006 The purple Head Start curve crosses the national average, showing that 25% of Head Start children score above the national average. 21

Using Standard Scores Is my assessment standardized? –Check the manual that came with your materials –The LAP is a standardized tool commonly used in Head Start. Each standardized assessment sets norms on a different sample of children (i.e., each has its own norming sample with particular characteristics). –Check your manual to make sure you know what comparison you are making and if it is appropriate! Is my assessment standardized? –Check the manual that came with your materials –The LAP is a standardized tool commonly used in Head Start. Each standardized assessment sets norms on a different sample of children (i.e., each has its own norming sample with particular characteristics). –Check your manual to make sure you know what comparison you are making and if it is appropriate! 22

Translating Raw Scores into Standard Scores Raw: The score you calculate based on responses Standard: The score you pick from a table in the manual. Information you sometimes need includes: –The raw score (always needed) –Child age/birth date –Child gender (especially for social-emotional assessments) –Who reported the information (teacher, parent, direct assessment of child) Technical assistance providers can provide help in determining standard scores or can train education coordinators to do so. Raw: The score you calculate based on responses Standard: The score you pick from a table in the manual. Information you sometimes need includes: –The raw score (always needed) –Child age/birth date –Child gender (especially for social-emotional assessments) –Who reported the information (teacher, parent, direct assessment of child) Technical assistance providers can provide help in determining standard scores or can train education coordinators to do so.

Key Lessons from the Normal Curve Through the principles of the normal curve, scores on some assessments can be translated to “standard scores” for comparisons to norms. In large groups of children, most (68 percent) will score near the average (between 1 standard deviation below and 1 above the average/mean). On average, children in Head Start score ½ to 1 standard deviation below norms in language, literacy, and mathematics. –Some Head Start children score above the norms. Through the principles of the normal curve, scores on some assessments can be translated to “standard scores” for comparisons to norms. In large groups of children, most (68 percent) will score near the average (between 1 standard deviation below and 1 above the average/mean). On average, children in Head Start score ½ to 1 standard deviation below norms in language, literacy, and mathematics. –Some Head Start children score above the norms. 24

Key Concept 6: Growth Over Time Where are children’s scores relative to: –Their own scores earlier in the year? –Other children in Head Start or norms at different points in the year? Do different subgroups show different patterns of growth? –Dual language learners –Children with special education needs –Younger children Where are children’s scores relative to: –Their own scores earlier in the year? –Other children in Head Start or norms at different points in the year? Do different subgroups show different patterns of growth? –Dual language learners –Children with special education needs –Younger children 25

Identifying Progress Over Time Depends on the type of assessment used. –Norm-referenced – Compare child’s performance to other children  Due to norming, standard scores only go up if children’s progress exceeds what would be expected from just getting older. –Criterion referenced – Compare children’s performance to pre-defined standard or criteria.  Progress is made as standards are met. Depends on the type of assessment used. –Norm-referenced – Compare child’s performance to other children  Due to norming, standard scores only go up if children’s progress exceeds what would be expected from just getting older. –Criterion referenced – Compare children’s performance to pre-defined standard or criteria.  Progress is made as standards are met. 26

Making Other Appropriate Comparisons A developmental continuum is an outline of: –Expected developmental accomplishments –The age most children should reach them Example: Literacy - Most children can identify words printed in their environment by the time they are 3-4 years old –Christopher identifies printed words at age 3: In line with expectations for his age –Megan is not noticing printed words at age 4 ½: A potential area for concern A developmental continuum is an outline of: –Expected developmental accomplishments –The age most children should reach them Example: Literacy - Most children can identify words printed in their environment by the time they are 3-4 years old –Christopher identifies printed words at age 3: In line with expectations for his age –Megan is not noticing printed words at age 4 ½: A potential area for concern 27

Identifying a Developmental Continuum In your curriculum or with your assessment tool In a text on child development See Resources in “Learning From Assessment” toolkit In your curriculum or with your assessment tool In a text on child development See Resources in “Learning From Assessment” toolkit 28

Compare Fall Scores in Two Programs 29

Compare Growth from Fall to Winter, Program 1 Children make large gains from the fall to the winter but still score below the national average in the winter. 30

Compare Growth from Fall to Winter, Program 2 Children make small gains between the fall and winter but score at or above the national average in the winter in all areas. 31

Comparing Gains in Different Programs How would you help think about improving outcomes in each program? 32

Objectives, Revisited Understand key assessment concepts Data Data quality Mean scores Distribution of scores Normal curve Growth over time Feel empowered to communicate about your program Understand key assessment concepts Data Data quality Mean scores Distribution of scores Normal curve Growth over time Feel empowered to communicate about your program 33