Using NAPLAN summative data to identify instructional successes and challenges. Presented by Philip Holmes-Smith School Research Evaluation and Measurement.

Slides:



Advertisements
Similar presentations
Educational Measurement and School Accountability Directorate
Advertisements

Statistical Reasoning for everyday life
C. D. Toliver AP Statistics
Objective Box-and-Whisker Plots Draw a box-and-whisker plot to organize real-life data. Read and interpret a box-and-whisker plot of real-life data.
Unit 16: Statistics Sections 16AB Central Tendency/Measures of Spread.
Overview of Formative Assessment (Diagnostic & Feedback) vs. Summative Assessment (Mastery & Standardised) Presented by Philip Holmes-Smith School Research.
Educational Measurement and School Accountability Directorate Better informed, better positioned, better outcomes National Assessment Program – Literacy.
Section #1 October 5 th Research & Variables 2.Frequency Distributions 3.Graphs 4.Percentiles 5.Central Tendency 6.Variability.
Choosing appropriate summative tests.
Using Summative Data to Monitor Student Performance: Choosing appropriate summative tests. Presented by Philip Holmes-Smith School Research Evaluation.
School Research Evaluation and Measurement Services
Measures of Central Tendency& Variability.
MEASURES OF SPREAD – VARIABILITY- DIVERSITY- VARIATION-DISPERSION
Box and Whisker Plots and Quartiles Sixth Grade. Five Statistical Summary When describing a set of data we have seen that we can use measures such as.
Using Formative (Diagnostic) Data to Plan for Instruction: NAPLAN Reading, Numeracy, Spelling & Grammar and Punctuation Presented by Philip Holmes-Smith.
DATA  Responsible National Curriculum  National Assessment Program (NAPLAN)  National data collection and reporting program.
FFT Data Analysis Project – Supporting Self Evaluation  Fischer Family Trust / Fischer Education Project Extracts may be reproduced for non commercial.
ASSESSMENT FOR BETTER LEARNING USING NAPLAN DATA Presented by Philip Holmes-Smith School Research Evaluation and Measurement Services.
12.4 – Measures of Position In some cases, the analysis of certain individual items in the data set is of more interest rather than the entire set. It.
Overview of Summative (Mastery & Standardised) Testing vs. Formative (Diagnostic) Testing Presented by Philip Holmes-Smith School Research Evaluation and.
Port Phillip/Bayside Network: Using student achievement data to inform improved teaching programs Presented by Philip Holmes-Smith (School Research Evaluation.
Review Measures of central tendency
What is variability in data? Measuring how much the group as a whole deviates from the center. Gives you an indication of what is the spread of the data.
Interpreting Performance Data
By: Amani Albraikan 1. 2  Synonym for variability  Often called “spread” or “scatter”  Indicator of consistency among a data set  Indicates how close.
NAPLAN Workshop Assessment for Better Learning using NAPLAN Data Presented by Peter Congdon, Principal Consultant – Kmetrics On behalf of the VCAA.
3.3 – Percentiles & Box-and-Whisker Plots Mean and Standard Deviation deal with exact numbers to measure. Sometimes measuring position is important too.
Intentions To talk about English on line for 2012 To remind ourselves of the processes To look at the next steps after completing the assessment How one.
A tour of fundamental statistics introducing Basic Statistics.
Measures of Central Tendency: The Mean, Median, and Mode
Z-Scores Standardized Scores. Standardizing scores With non-equivalent assessments it is not possible to develop additive summary statistics. –e.g., averaging.
Measures of Variability Percentile Rank. Comparison of averages is not enough. Consider a class with the following marks 80%, 80%, 80%, 90%, 20%, 70%,
Harcourt NAPLAN Results Years3 School3 State5 School5 State7 School7 State Reading20%13%28%18%15%16% Writing21%10%30%17%28%32% Spelling11%15%12%13%7%17%
Principles of the New National Curriculum and Assessment process.
 Quartiles and Percentiles. Quartiles  A quartile divides a sorted (least to greatest) data set into 4 equal parts, so that each part represents ¼ of.
CHILD POVERTY and factors that affect it. What is child poverty? Child poverty: is a significant lack of the basic needs required for healthy physical,
New Curriculum, SATs and assessment arrangements for 2016.
Descriptive Statistics Chapter 2. § 2.5 Measures of Position.
Assessment to assure quality Barry McGaw Professorial Fellow, University of Melbourne Former Chair, Australian Curriculum, Assessment and Reporting Authority.
Unit 3: Averages and Variations Part 3 Statistics Mr. Evans.
Collecting  Describing  Summarizing.   Statistics is a set of data (information) that has been collected. The data can be categorical or numerical.
To graph and interpret Box-and-Whisker Plots 9/17/13 Do Now: Order the set of numbers from least to greatest, then find the median.
Chapter 5 Describing Distributions Numerically Describing a Quantitative Variable using Percentiles Percentile –A given percent of the observations are.
Probability & Statistics Box Plots. Describing Distributions Numerically Five Number Summary and Box Plots (Box & Whisker Plots )
Setting NAPLAN DATA Targets Presented by Philip Holmes-Smith School Research Evaluation and Measurement Services.
‘A Flying Start’ Achievement Update November 2014 Chris Snudden Head of Education Achievement Service Head of Virtual School for Children in Care John.
Educational Measurement and School Accountability Directorate National Assessment Program – Literacy and Numeracy (NAPLAN) Understanding your child’s report.
Descriptive Statistics Chapter 2. § 2.5 Measures of Position.
7-2 Box-and-Whisker Plots. 7-2 Box-and-Whisker Plots Lesson 7.2.
Measures of Position – Quartiles and Percentiles
Measures of Position Section 2-6
IT’S ALL ABOUT GROWTH!. Hemet Unified School District’s Use of Measures of Academic Progress (MAP)
U4D3 Warmup: Find the mean (rounded to the nearest tenth) and median for the following data: 73, 50, 72, 70, 70, 84, 85, 89, 89, 70, 73, 70, 72, 74 Mean:
RAISEonline Data Analysis for Governors and Staff
EVAAS Overview.
Chapter 6.4 Box and Whisker Plots
Numerical Measures: Skewness and Location
11.2 box and whisker plots.
Tammy Wallace Varina High
Evaluation and Assessment of the Individual: Week 2 Discussion
Analysing your pat data
Common Core State Standards:
pencil, red pen, highlighter, GP notebook, graphing calculator
Unit 12: Intro to Statistics
Understanding ASP and FFT Data
Tammy Wallace Varina High
pencil, red pen, highlighter, GP notebook, graphing calculator
Skills test Numeracy support
Review of 6th grade material to help with new Statistics unit
Presentation transcript:

Using NAPLAN summative data to identify instructional successes and challenges. Presented by Philip Holmes-Smith School Research Evaluation and Measurement Services

Overview of Session Using NAPLAN summative data to identify instructional successes and challenges Understanding the National Scale, School Summary analysis, Five-Year Trend analysis, Individual student-level data analysis Value added (Growth) analysis

Understanding the National Scale

National Curriculum The National Scale is an arbitrary scale – at this stage it is not related to points along a developmental curriculum. But, it is highly likely that it will be mapped onto the National Curriculum at some time in the future.

fixed in 2008 The National Scale was fixed in 2008 as follows: – – Range: – – Mean: 500 – – Standard Deviation: 100 (i.e. 68% students between ) +1 STD 2008 National Mean -1 STD Understanding the National Scale

National Averages (2008) Only 20.1% of Australian school communities are more advantaged My School Website (2010) School’s Average National Average Average for school’s with similar ICSEA

National Curriculum The National Scale is an arbitrary scale – at this stage it is not related to points along a developmental curriculum. But, it is highly likely that it will be mapped onto the National Curriculum at some time in the future. fixed in 2008 The National Scale was fixed in 2008 as follows: – – Range: – – Mean: 500 – – Standard Deviation: 100 (i.e. 68% students between ) Bands The Scale has been divided into ten Bands which are used for reporting to parents. – – Band 1 covers all scores equal to or less than 270. – – Bands 2 – 9 increment by 52 score points each Band – – Band 10 covers all scores above 686. Understanding the National Scale

National Curriculum The National Scale is an arbitrary scale – at this stage it is not related to points along a developmental curriculum. But, it is highly likely that it will be mapped onto the National Curriculum at some time in the future. fixed in 2008 The National Scale was fixed in 2008 as follows: – – Range: – – Mean: 500 – – Standard Deviation: 100 (i.e. 68% students between ) Bands The Scale has been divided into ten Bands which are used for reporting to parents. – – Band 1 covers all scores equal to or less than 270. – – Bands 2 – 9 increment by 52 score points each Band – – Band 10 covers all scores above 686. Victorian students in 2011: At this stage Bands have no explicit curriculum meaning but results show that for Victorian students in 2011: – – A typical Yr3 level of performance is at the bottom of Band 5 – – A typical Yr5 level of performance is almost halfway into Band 6 – – A typical Yr7 level of performance is a third into Band 7 – – A typical Yr9 level of performance is at the bottom of Band 8 Yr 3 Yr 5 Yr 7 Yr 9 Understanding the National Scale

Parent Reports (NB. Bands do not equate to Victorian Year Level means) Year 3 Year 5 Year 7 Year 9

Victorian State Averages (By Year Level and Dimension)

Cautionary Note #1 Equal scores amongst different dimensions (on the National Scale) do not equate to equal levels of performance in terms of expected VELS levels. e.g. – A National Yr9 Reading score of is equivalent to a VELS score of 5.25, but – A higher National Yr9 Numeracy score of is equivalent to a lower VELS score of 4.82.

Comparing National Scale Scores to Estimated VELS Equivalent scores Comparing National Scale Scores to Estimated VELS Equivalent scores The Victorian means for Year 3 and Year 9 Reading and Numeracy on the National scale are compared to the estimated VELS equivalent scores below: Compared to our expected curriculum outcomes for Year 3 students (2.175), the State Reading mean is about 1½ term ahead of where we expect a typical Year 3 student to be. However, the State Numeracy mean is about 2½ terms below where we expect a typical Year 3 student to be. Compared to our expected curriculum outcomes for Year 9 students (5.175), the State Reading mean is just above where we expect a typical Year 9 student to be. However, the State Numeracy mean is about 2½ terms below where we expect a typical Year 9 student to be. Year 3 Victoria MeansYear 9 Victoria Means Dimension National Scale VELS Equivalent National Scale VELS Equivalent Reading Numeracy

Comparing AIM scores (VELS May equivalent) to NAPLAN scores (Est. VELS equivalent)

Comparing AIM scores (VELS May equivalent) to NAPLAN scores (Est. VELS equivalent)

Comparing AIM scores (VELS May equivalent) to NAPLAN scores (Est. VELS equivalent)

Comparing AIM scores (VELS May equivalent) to NAPLAN scores (Est. VELS equivalent)

Comparing AIM scores (VELS May equivalent) to NAPLAN scores (Est. VELS equivalent)

Cautionary Note #2 “Estimated VELS Equivalent Scores” should be viewed with caution. Specifically: “Estimated VELS Equivalent Scores” should be viewed with caution. Specifically: – It is doubtful that the “Estimated VELS Equivalent Scores” for Writing, Spelling or Grammar & Punctuation are truly VELS scores. (The writing marking rubric does not use VELS criteria and there are no VELS criteria for Spelling or Grammar & Punctuation.) – “Estimated VELS Equivalent Scores” are based on an equating study that is now three years old.

School Summary Analysis

The School Summary Report Choose “School Summary Report” Click on “Preview Report” to view results Choose to look at All Students or Girls vs. Boys or LBOTE Students or ATSI Students Choose to look at National Scale Scores (Estimated VELS scores are problematic) 2010

The School Summary Report

Interpreting “box and whisker” graphs ? ? ?? There are 30 students in this Year Level. Therefore: – – 50% (or 15 students) are above the median. – – 50% (or 15 students are below the median. – – 50% (or 15 students) are inside the “box”. Half of these (7-8 students) are above the median and half (7-8 students) are below the median. – – 10% (or 3 students) are at or below the 10 th percentile “whisker”. – – 10% (or 3 students are at or above the 90 th percentile “whisker”. – – 15% (or 4-5 students) are spread between the 25 th down to the 10 th percentile. – – 15% (or 4-5 students) are spread between the 75 th up to the 90 th percentile. 15 students above median 15 students below median 7-8 spread above the median. 15 students inside the box. 7-8 bunched below the median. 3 students on or below the 10 th percentile. (Note, all we know is at least one student scored on the 10 th percentile.) 3 students on or above the 90 th percentile. (Note, all we know is at least one student scored on the 90 th percentile.) 4-5 students between the 75 th up to the 90 th percentile. 4-5 students between the 25 th down to the 10 th percentile.

Interpreting the School Summary Report 1. 1.Is the school’s median above, at or below the State median? School’s median is about half a band below the State median

Interpreting the School Summary Report 1. 1.Is the school’s median above, at or below the State median? 2. 2.For VELS only, is the Reading median above, at or below the Numeracy median? School’s median is about half a band below the State median Reading is higher than Numeracy

Interpreting the School Summary Report 1. 1.Is the school’s median above, at or below the State median? 2. 2.For VELS only, is the Reading median above, at or below the Numeracy median? 3. 3.How does the school’s spread compare to the State spread? School’s median is about half a band below the State median School has far fewer high performers than the State School has far more low performers than the State School’s median is at the State’s 25 th percentile Reading is higher than Numeracy

The School Summary Report Review your own School Summary Report Go to: (Use your school’s NAPLAN Data Service logon and password to access the report)

Five-Year Trend Analysis

Difference scores show how much better or worse you are compared to the State average This year’s score appears to be part of an improving trend compared to the State average

Five-Year Trend Analysis Look at your own SPA data Go to: (Use your SPA logon and password to access you SPA account or click on “Purchase SPA” to purchase and set up a new account for your school)

Individual Student-level Analysis: Grouping Students for Teaching

Individual Student-level Analysis Score that place a student one or more years below expected are coloured red Score that place a student at or about expected level are coloured yellow Score that place a student one or more years above expected are coloured green

Individual Student-level Analysis Group requiring more basic level material Group requiring grade level material Group requiring more advanced material Click on a dimension heading to sort the scores

Individual Student-level Analysis Look at your own SPA data Go to: (Use your SPA logon and password to access you SPA account or click on “Purchase SPA” to purchase and set up a new account for your school)

Value-Added (Growth) Analysis

Growth Analysis (NAPLAN scale) Letitia appears to have grown units from in Yr3 to in Yr5. Javier appears to have gone backwards 40.7 units from in Yr3 to in Yr5.

Growth Analysis (NAPLAN scale) On Average, the school has grown its students units compared to the State’s 83.2 units. This is units less than the state.

Growth Analysis (VELS scale) Letitia appears to have grown 1.64 VELS units from 2.52 in Yr3 to 4.16 in Yr5. Javier appears to have gone backwards 0.43 VELS units from 3.11 in Yr3 to 2.68 in Yr5.

Growth Analysis (VELS scale) On Average, the school has grown its students 0.68 VELS units compared to the State’s 0.84 VELS units. This is 0.16 VELS units less than the state.

Growth Analysis (VELS scale) Difference scores show how much more or less growth you show compared to the State growth This year’s growth appears to be part of an worsening trend compared to the State growth

Value-Added (Growth) Analysis Look at your own SPA data Go to: (Use your SPA logon and password to access you SPA account or click on “Purchase SPA” to purchase and set up a new account for your school)