1. Advancing Assessment Literacy Data Analysis II: Examining & Interpreting Data

2. Advancing Assessment Literacy Modules: Data Analysis II (February 2008) 2 Predicting All data are meaningless until we attach meaning through interpretation.

3. NATIONAL PERCENTILE (NP) NATIONAL STANINE (NS) GRADE EQUIVALENT (GE) WHAT IT IS Places students on a scale of 1 to 99, where 99 is highest. An NP score of 99 means the student performs better than 99 per cent of the national sample of the population at that grade level. A national percentile of 50 means that one is average for one's grade level. Places scores into nine divisions (1 to 9) called stanines. The higher the stanine, the better the performance Compares a student's performance with that of students in other grades WHY IT IS USEFUL Shows precisely how a student is performing in comparison with others at the same grade level Provides quick and simple information Shows a student's progress from year to year WHY IT CAN BE CONFUSING It is so precise that variations are bound to occur, partly from chance Simplicity means that a student may make tremendous efforts and actually improve, without moving up a stanine, or stop working without it showing for a while Grade equivalent does not indicate that the student should be moved up or down to another grade From http://www.canadiantestcentre.com/Teachers/InformationForTeacher.asp#result

4. Advancing Assessment Literacy Modules: Data Analysis II (February 2008) 4 Cut Scores On page 4 of the detailed reports you will find the cut scores detailing the percentage correct required for students to be classified at one of two levels: –Threshold of adequacy –Threshold of proficiency Reader response and Math challenge scores are presented on a five-level scale (1-low to 5-high).

5. Advancing Assessment Literacy Modules: Data Analysis II (February 2008) 5

6. 6

7. 7 Locating Cut Scores in the Report Turn to pg. 4 in the detailed report for your grade level. Opportunity-to-Learn ElementsPerformance Component Excellent Standard Sufficient Standard Proficient Standard Adequate Standard A score out of 5 (1-low to 5-high)% correct required to reach the standard You will need to refer to these scores during the following prediction activity.

8. Advancing Assessment Literacy Modules: Data Analysis II (February 2008) 8 Predicting AFL MathOTL – R & MAFL Reading On the chart for your grade level, predict how many students achieved the ADEQUATE Standard on the 2007 AFL in Math for each strand: Math Content Skills Integrated Applications Estimation Skills Calculator Skills Computation Skills On the reverse are the Math Content Skills. On the charts, predict what percentage of students had SUFFICIENT opportunity to Learn in the following areas: Reading: Student preparation and commitment to learn Student knowledge & use of reading strategies Home support for reading Math: Student preparation and commitment to learn Student persistence Home support for learning Home support for learning math On the charts, predict what percentage of students met the ADEQUATE Standard in the following strands: Reading Comprehension Explicit Comprehension Implicit Comprehension Critical Comprehension Reader Response On the reverse are the Reading Comprehension Skills.

9. Advancing Assessment Literacy Modules: Data Analysis II (February 2008) 9 AFL Math Percentage of Students who met the Adequate Standard set by Saskatchewan Educators

10. Shade in your prediction on the supplied prediction chart. Wellman, B. & Lipton, L. (2004). Data driven dialogue. Mira Via, LLC. AFL Math Percentage of Students who met the Adequate Standard set by Saskatchewan Educators

11. Advancing Assessment Literacy Modules: Data Analysis II (February 2008) 11 Hypotheses and Assumptions Based on your predictions, create a set of hypotheses for some or all of them. As you create each hypothesis, identify the underlying assumptions. Hypothesis – “X” will contain the highest scores. Assumption – we created a common assessment for “X” in 2005. Hypothesis – students will report higher on digging in and starting reading than looking at the front and back covers to get an idea of what the book is about. Assumption – students expect to discover what the book is about in the first few pages. Write each hypothesis and its accompanying assumption on the cards provided. Please write legibly.

12. Advancing Assessment Literacy Modules: Data Analysis II (February 2008) 12 Sharing Gather the cards together at your table and either pass them to the next table or shuffle them to discuss as a group. When you receive a set of cards, discuss the hypotheses and assumptions. Do you see any patterns? Are any similar or different? Why might this be?

13. Advancing Assessment Literacy Modules: Data Analysis II (February 2008) 13 Comparisons The completed bar graphs have been supplied to your table. What are you noticing about the data? What surprised you? What are the benefits of approaching data in this manner? What other data would you like to see to better inform the results you’ve seen so far? Wellman, B. & Lipton, L. (2004). Data-driven dialogue. Mira Via, LLC.

14. Advancing Assessment Literacy Modules: Data Analysis II (February 2008) 14 Designing Interventions Assumptions must be unpacked, because our interventions will be based on them. We must strive to correctly identify the causal factors. Don’t fall in love with the theory until you have other data. Use a strength-based approach to interventions.

15. Advancing Assessment Literacy Modules: Data Analysis II (February 2008) 15 Data Displays Create groups of three – appoint a recorder, materials manager, and facilitator. On the wall is a piece of chart paper and: –AFL – The test questions with results by question Package with objectives to supplement. –CAT3 – Criterion Referenced Scores Observe the data silently. Wellman, B. & Lipton, L. (2004). Data-driven dialogue. Mira Via, LLC.

16. Advancing Assessment Literacy Modules: Data Analysis II (February 2008) 16 Data Displays After 5-8 minutes share your observations, questions or comments about the data. What is unique? Unexpected? Don’t interpret, just look for what pops out. Please record your comments and questions on the chart paper provided. Wellman, B. & Lipton, L. (2004). Data-driven dialogue. Mira Via, LLC.

17. Advancing Assessment Literacy Modules: Data Analysis II (February 2008) 17 Gallery Tour Leave a representative behind to answer questions. Members of other groups may now circulate and look at the other groups’ findings. Return to your original display. Discuss what you saw on your tour.

18. Advancing Assessment Literacy Modules: Data Analysis II (February 2008) 18 Team Action Plan At your data display, complete the Team Action Plan sheet. Write one observed strength and one opportunity for improvement in student learning on the post-it notes provided and post them on the chart for your school at the front of the room.

19. Advancing Assessment Literacy Modules: Data Analysis II (February 2008) 19 Fishbone Tool At your table, analyze one strength and one area for improvement. Consider all possible causes. When you are done, compare and contrast the two diagrams.

20. Advancing Assessment Literacy Modules: Data Analysis II (February 2008) 20 Creating a Statement of Success From your analysis of what is working, develop a statement of success. Write it in the space provided on this worksheet. What elements from your statement of success might we include in the beginning stages of an action plan? This process will be continued and expanded upon in the next module.