1. Using accurate data to drive decision making Paula Brown Director of Instructional Accountability Hampton City Schools October 2010 1

2. Congratulations Your schools have worked very hard Most of you represent localities where all schools are fully accredited Most are making AYP (even with changing achievement goals)

3. Rising Goals 2009-2010 English: 81% passing Math: 79% passing With most schools reaching the 90’s in core subjects, the question then becomes how do we reach the last 10 percent?

4. The Last 10 Percent Today, we are going to look closely at a process for getting the last 10% to be successful on SOL assessments 100% passing is an attainable goal and many great instructional practices will work IF… they are being driven by accurate data

5. Data Source After SOL testing, we can retrieve our SOL assessment data in a CSV file from Pearson Access We can retrieve that file by division and by each individual school From that file, we make pivot tables and report outcomes Today, I want to show you another way to use that file to give your teachers and administrators very clear direction about where to work 5

6. 6

7. Data Source After the file is downloaded, we often cut the file down by deleting columns that make the file a little cumbersome to work with; especially at the school level We also usually change it to an Excel file for ease of use Most of the time we keep the following labels: School name, group (teacher)name, student information, test name, reporting categories, overall scaled score and proficiency rating 7

8. EIMS EIMS is another source for data retrieval that is often in a user friendly format Over the years, it has become easier to manipulate and gives many schools and school divisions the opportunity to view their data more specifically It also allows the viewer to compare data over several years in a variety of ways 8

9. 9 Very Helpful Data This report is very helpful for getting pass rates for a variety of students and individual teacher pass rates

10. EIMS One of those reports includes scaled category data for each SOL assessment This data was useful initially, but as pass rates have increased, it tends to send educators in the wrong direction The scaled score compares categories as if they are equal in number of questions and it uses a different pass bar for each reporting category These two factors diminish its usefulness 10

11. 11 VERY Unhelpful Data This reports that for this school in grade 3 science the category of weakness is Scientific Investigation with 76% passing the category scaled mark of 30. The strength for this school is identified as Earth/Space Systems.

12. Accurate Data By using the method that I will show you today, you will see that the actual weakness for this school was Life Processes and Living Systems and that Scientific Investigation was the strongest area of performance 12

13. Why Do You Need This Process ALL instructional decisions and practices must be based on accurate data

14. Misleading Data Everyone data This data includes all three performance groups Failing Passing proficient Passing advanced

15. Operative Definitions Failing students are those who received a scaled score from 0 to 399 Passing students are those who received a scaled score from 400 to 499 Advanced students are those who received a scaled score from 500 to 600 15

16. Data Facts Look at Failing data to change pass rates Passing data to change advanced rates Look at category data by raw numbers to identify accurate areas of weakness in both performance groups Look at skill data (SPBQ) to narrow down another step Failing students Passing students

17. Subtract the Average Raw Score in each category from the total number of questions. This identifies the size of the gap. Reducing the largest gap is the fastest path to student success on state tests This figure shows the largest gap (by a tiny margin) to be in the area of Computation and Estimation GAP 2.61 Everyone Data 2.91 2.93 1.70 1.68

18. This is the same grade level at the same school, but this time the passing students have been removed. This time the gap is much wider between the categories. This narrows the focus to Computation and Estimation. GAP 5.86 5.00 3.14 2.71 4.64 Failing Data

19. SPBQ Report: Everyone data It looks like there are four skills that are the weakest in this category…

20. SPBQ Report: Failing students BUT, when we take out the passing kids we see that “finding the product” ( or multiplication) has the lowest three scores(44%, 48%, 56%) followed by subtraction

21. How Do You Do It? Providing this information to teachers and administrators will change how they look at data and how they use it to make decisions In Hampton City Schools, we use SOLAR to provide this data to our schools Today, I want to show you how to do this with your Pearson data file and how to use it once you have created it 21

22. How Do You Do It? 1. Take your raw data file (CSV) and convert it to an Excel file for easier use 2. Eliminate all of the columns except test name, reporting categories 1-7 and the total scaled score 3. Using the sort feature, sort the edited file by test name 4. Select, cut and paste each test name (with its adjoining data) into its own data sheet 5. Label the tabs with the test names as you create each sheet (math, science and social studies) 6. You can delete the reading scores for this activity 22

23. How Do You Do It? 7. Make sure to retain the labels at the top for each new page that you create ( see step 2) 8. Using the sort feature, sort the edited file by scaled score (lowest to highest or highest to lowest) 9. Add rows in the data to visually separate the failing scores from the passing scores and the passing scores from the advanced scores 10. Using the test blueprint, rename the categories by their true name (i.e. number and number sense, computation and estimation, etc.) and delete extra columns 23

24. How Do You Do It? 11. Add the number of questions in the category next to the name (i.e. NNS 8q, C&E 12q, M&G 12q, etc.) 12. Starting with the failing scores, use the averaging function provided in Excel to average each column and type in that number at the end of each row 13. Then subtract the averaged number from the total number of questions (i.e 8-6.53=1.47) 14. That creates a question gap (1.47 questions is the gap) 15. Do this for each category…the larger the gap the bigger the problem 24

25. How Do You Do It? 12. Once all of the gaps have been identified for the failing data, then do the same process for the passing data. 13. Enter all of the gap data onto a collection chart 14. This will show the schools and teachers exactly where the areas of weakness are and how to structure their review plans 15. Doing this process over a few years, really shows entrenched practices and cultures that haven’t changed 25

26. 26 This is a copy of a collection chart developed in Hampton City Schools The category of weakness for both achievement groups is Patterns, Functions and Algebra

27. What is the Next Step? Using the data to plan Yearlong changes to the pacing guide (usually in the form of adding review time or pre-teaching some SOLs) Example: Science review booklet All remediation programs focus on category of weakness After and before school programs Tutoring and volunteers Summer School 27

28. Things to Watch Be careful of review plans that review everything This leads to a repeat of the scores earned last year It also causes the large categories to be short changed and therefore leads to the failure of many students who could have passed It doesn’t matter what you believe, if you ignore the size of the large categories, they will be the root of student failure 28

29. 29 Reviewing Evenly

30. The Next Step: Skill Data In order to divide skill data into passing and failing groups, you will need some very advanced Excel skills or you can use another manual process to do this: 1. Print out a copy of the SPBQ report for each student (School level or classroom level) 2. Divide the copies into three stacks: failing, passing and advanced(set the advanced aside, you will not need them for this activity) 3. Number a sheet of paper to match the number of questions on the test 30

31. The Next Step: Skill Data 4. Take the failing stack and make a mark every time a student misses a question 5. When you are done with the stack, look at the category you identified earlier as the weakness 6. Identify the questions in that category where there was the most failure 7. This process will narrow your focus even deeper 31

32. 32 Highly missed skills

33. The First Step: Skill Data English focus skills Seven skills per grade level Context clues Main idea, summarizing Drawing conclusions Author’s purpose Ask and Answer questions Characterization Cause and Effect Must be reviewed to mastery

34. English Skill Data 1. Using the individual SPBQ sheets, do a tally of missed skills (see slide 32) 2. Combined questions that are for the same skill 3. Enter the data onto the data collections sheet (see slide 35) 4. Identify the focus skills (they are shaded) 5. Order them from lowest to highest performing 34

35. 35

36. 100% Passing We can ALL reach this goal