Advancing Assessment Literacy Data Analysis III: Extending the Assessment

Advancing Assessment Literacy Modules: Data Analysis III (February 2008) 2 Revisiting Hypotheses You have been provided with the hypotheses generated following the initial examination of the data. This session is going to apply some of the principles from earlier modules to clarify the hypotheses created and expand the analysis to other data sources in order to confirm or deny what the data seems to be indicating.

3 Effect of the Scale of the Data Source From Understanding the numbers. Saskatchewan Learning DefinitiveIndicative Individual Classroom SchoolDivision Provincial National International Student EvaluationsSystem Evaluations The analysis to this point has used provincial level data both in terms of general achievement and was followed by an item analysis which demonstrated how students performed relative to specific objectives at a school wide level.

Advancing Assessment Literacy Modules: Data Analysis III (February 2008) 4 Different Lenses What other types of data might be required to gain a clearer picture of how specific groups of students are doing? You may need to discuss how students might be grouped. How might this data shed a different light on what has been discovered so far? To do this, Bernhardt’s triangulation process will be used.

Advancing Assessment Literacy Modules: Data Analysis III (February 2008) 5 Applying Data Collection and Triangulation Using the hypotheses created from the initial examination of the data, complete a more detailed analysis thinking about the four categories of data available – demographics, perceptions, student learning and school processes. You will be furnished with a data intersections template.

Advancing Assessment Literacy Modules: Data Analysis III (February 2008) 6 Refining Questions Take each hypothesis and write it at the top of a template. Write questions of the hypothesis in the questions column. In the left column, use the diagram to identify the intersections or triangulations implied within the question. –What other intersections would increase the specificity of this question? –If necessary, rewrite the question to reflect the new intersections or triangulations. –Including an “over time” element to a question expands the data that might be accessed to answer the question. For each question identify the existing data source available or what tool will be required to collect it. Create new questions using a variety of the intersections identified earlier.

7 What types of strategies are used in reading? What data is available or needed to answer the question? What data collection tools will be required? Questions Intersections (Colour in and name the intersection) D P SL SP Adapted from: Bernhardt, V. L. (2004). Data analysis for continuous school improvement, 2 nd Edition. Larchmont, NY: Eye on Education. Hypothesis: Similar to process lists in math, providing detailed strategy lists for student reading will increase the number of strategies used and therefore increase comprehension. Place the question in the questions column

8 Available Data: Processes from the curriculum guide. Processes outlined in the AFL documents. What types of strategies are used in reading? (School Processes (SP)) What data is available or needed to answer the question? What data collection tools will be required? Questions Intersections (Colour in and name the intersection) D P SL SP Adapted from: Bernhardt, V. L. (2004). Data analysis for continuous school improvement, 2 nd Edition. Larchmont, NY: Eye on Education. Identify the intersection within the question Hypothesis: Similar to process lists in math, providing detailed strategy lists for student reading will increase the number of strategies used and therefore increase comprehension.

9 Available Data: Student reading journal responses Data Needed: Student opinions gathered via questionnaire. What reading strategies do students seem to favour? (School Processes by Perceptions) What data is available or needed to answer the question? What data collection tools will be required? Questions Intersections (Colour in and name the intersection) D P SL SP Adapted from: Bernhardt, V. L. (2004). Data analysis for continuous school improvement, 2 nd Edition. Larchmont, NY: Eye on Education. What other intersections and data sources might enrich the analysis? Hypothesis: Similar to process lists in math, providing detailed strategy lists for student reading will increase the number of strategies used and therefore increase comprehension.

10 Available Data: Student reading journal responses Data Needed: Pre- and post-test data for all reading strategies Which reading strategies from the curriculum produce the greatest gains in comprehension for students? Which of those were favoured by students? (School Processes by Perceptions by Student Learning) What data is available or needed to answer the question? What data collection tools will be required? Questions Intersections (Colour in and name the intersection) D P SL SP Adapted from: Bernhardt, V. L. (2004). Data analysis for continuous school improvement, 2 nd Edition. Larchmont, NY: Eye on Education. What other intersections and data sources might enrich the analysis? Hypothesis: Similar to process lists in math, providing detailed strategy lists for student reading will increase the number of strategies used and therefore increase comprehension.

11 Available Data: Student reading journal responses by gender. Data Needed: Pre- and post test data for favoured reading strategies by gender. Which of the reading strategies from the curriculum produce the greatest gains in comprehension for students? Which of those were favoured? Are certain strategies more powerful at different grade levels? (School Processes by Perceptions by Student Learning by Demographics) What data is available or needed to answer the question? What data collection tools will be required? Questions Intersections (Colour in and name the intersection) D P SL SP Adapted from: Bernhardt, V. L. (2004). Data analysis for continuous school improvement, 2 nd Edition. Larchmont, NY: Eye on Education. What other intersections and data sources might enrich the analysis? Hypothesis: Similar to process lists in math, providing detailed strategy lists for student reading will increase the number of strategies used and therefore increase comprehension.

Advancing Assessment Literacy Modules: Data Analysis III (February 2008) 12 Refining and Finalizing Questions Complete the process for each question created. When finished, evaluate the quality of the questions then decide which should go forward and which should be abandoned. On the sheet provided, write the hypothesis and the refined questions that your group has decided to keep. These questions will be used to gather data as more in-depth goal statements are created.

Advancing Assessment Literacy Modules: Data Analysis III (February 2008) 13 Reflection Individually, complete the following stems: Key concepts I have learned... Implications for me... Questions I have...