1. Expanding Your Data Toolkit Irene Ricci Sullivant Supervisor of Data Office of Research & Planning Camden City Public Schools

2. Professional Development Standards for NJ Educators: Context Standards: 2. Leadership Process Standards: 4. Data Driven 5. Research-based 9. Collaboration Content Standards: 11. Quality Teaching

3. Objectives Participants will: Examine the current data practices of the school community. Compare current practice to what research has identified as effective data use.

4. Objectives Participants will: Identify multiple measures of data to examine. Develop an understanding of student growth data and its use in continuous school improvement.

5. Our district is committed to using data effectively.

6. Data...  Must drive Instruction  Helps us to identify root causes of student learning problems  Is an essential component of continuous school improve.  Addresses important indicators under QSAC and Priority School Achievement Indicators  Used to drive planning such as the Unified Plan, Professional Development Plan, and all the plans to come.

7. 7 THINK BEYOND TEST SCORES “Output data are not the only kind of data”. Source: “Making sense of data-driven decision making in education”, J. A. Marsh, J. F. Pane and L. S. Hamilton, Rand Corporation, 2006

8. 8

9. Data Improvement & Review Team (D.I.R.T) Purpose To provide a vehicle for the on- going review of school-based data for school improvement purposes.

10. DATA TEAM ROLE Review Achievement and Non- Achievement Data Prepare Recommendations Develop a School Improvement Plan Monitor Implementation of the Plan Generate Questions for Further Analyses

11. What the research tells us...

12. U. S. Department of Education: Recommendations for Data Use  Make data part of an ongoing cycle of instructional improvement.  Teach students to examine their own data and set learning goals.  Establish a clear vision for school-wide data use Hamilton, L., Halverson, R. et. al.(2009) Using student achievement data to support instructional decision making. (NCEE 2009-4067) Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.

13. U. S. Department of Education: Recommendations for Data Use  Provide supports that foster a data- driven culture.  Develop and maintain a district-wide data system. Hamilton, L., Halverson, R. et. al.(2009) Using student achievement data to support instructional decision making. (NCEE 2009-4067) Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.

14. A Data Coach’s Guide to Improving Learning for All Students: Unleashing the Power of Collaborative Inquiry © 2008 by Corwin Press. All rights reserved. Culture, Equity, Trust Instructional Improvement Collaboration Data Use Leadership & Capacity Shifts That Are Evident in Effective Data Driven Schools Less EmphasisMore Emphasis External accountability, cultural blindness, little trust Internal and collective responsibility, cultural proficiency, trust Data to sort, learning left to chance Data to serve, expanding opportunities for all Top-down, data-driven decision making Ongoing Data-Driven Dialogue and collaborative inquiry Punishment/reward, avoidance Feedback for continuous improvement, frequent and in- depth use by teachers and students Individual charismatic leaders as change agents Learning communities with many change agents

15. Keep the Data Continuum in Mind Data Information Exist in a raw state w/out meaning. Knowledge Data given meaning in context Collection of info. deemed useful to guide action Mandinach, E. & Jackson, S. Using student data to support instruction. Presentation, 9/23/2010. REL Mid-Atlantic.

16. It is essential to activate the knowledge and work collaboratively in order to impact student achievement.

17. Data Safety Regulations Never use data as a weapon. Data discussions are a “NO FAULT ZONE!” Avoid the “blame game.” Avoid coming to hasty conclusions. Investigate multiple data before drawing inferences. Never use data as an excuse for implementing a quick fix. Focus on improving instruction.

18. A Data Coach’s Guide to Improving Learning for All Students: Unleashing the Power of Collaborative Inquiry © 2008 by Corwin Press. All rights reserved. Effective Systems Taught Curriculum Written Curriculum Assessed Curriculum STANDARDS Adapted from Fenwick W. English, Deciding What to Teach and Test: Developing, Aligning, and Auditing the Curriculum, Millennium Edition, 2000, p. 13. Thousand Oaks, CA: Corwin. Used with permission.

19. When any part of our system is not aligned, we will not get the desired results. Taught Curriculum Written Curriculum Assessed Curriculum STANDARDS ? Results

20. Multiple Measures of Data Demographic Data Perception Data Student Learning Data School Process Data Adapted from: Bernhardt, V. L. (1998). Data analysis for comprehensive school-wide improvement. Larchmont, NY: Eye on Education.

21. What data are we using in our data world?

22. Time to Reflect How does our current practice compare to what the research tells us is effective?

23. Let’s Talk About Moving Forward! Where does our work with data go from here?

24. Our Shifting Focus From proficiency rates to growth

25. Understanding Student Growth The state is moving to a growth model. NJSMART has an excellent video resource help us understand how to use the new NJSMART district report NJASK Growth Profiles & student growth percentiles as a part of our examination of student learning data. Video link http://survey.pcgus.com/njgrowth/player.html

26. Key points to remember about growth NJASK is a criterion referenced test. Students are measured against the standards. Cutoffs are set identifying whether a student is at, above, or below. Adapted from: Using Student Growth Percentiles (2012). Downloaded from the www at http://survey.pcgus.com/njgrowth/player.html.

27. Key points to remember about growth Student Growth Percentile Compares student’s achievement to his/her peers over time. Redefines student performance as: Achievement + Growth A student’s peers are those students throughout NJ with similar test histories on the NJASK. Adapted from: Using Student Growth Percentiles (2012). Downloaded from the www at http://survey.pcgus.com/njgrowth/player.html.

28. Key points to remember about growth Student Growth Percentile: Percentile – a number from 1-99 indicating the percentage of the reference group with scores equal to or less than the individual student’s score. To determine this, the state looks at the scale scores of the student and his/her academic peers. Adapted from: Using Student Growth Percentiles (2012). Downloaded from the www at http://survey.pcgus.com/njgrowth/player.html.

29. Key points to remember about growth Student Growth Level Describes the student’s growth as : High - 66+ Typical-35-65 Low-1- 34 No absolute criteria for determining ; set by the state. Adapted from: Using Student Growth Percentiles (2012). Downloaded from the www at http://survey.pcgus.com/njgrowth/player.html.

30. Key points to remember about growth Median Student Growth Percentile Use this to determine growth of a class, a grade, a school, district. Median SGP is the middle of SGP if students are grouped from highest to lowest or, The average of the 2 middle scores in a distribution with an even number of scores. Adapted from: Using Student Growth Percentiles (2012). Downloaded from the www at http://survey.pcgus.com/njgrowth/player.html.

31. Key points to remember about growth Median Student Growth Percentile for the state is 50 Adapted from: Using Student Growth Percentiles (2012). Downloaded from the www at http://survey.pcgus.com/njgrowth/player.html.

32. Key questions for examining Median SGP  How does this year’s median SGP compare with comparable groups in other districts?  Is our median SGP consistent year to year?  How does the median SGP for this class compare with the grade? With other grades in the school? With the district? Adapted from: Using Student Growth Percentiles (2012). Downloaded from the www at http://survey.pcgus.com/njgrowth/player.html.

33. So you have the data, what’s next?

34. Suggestions for Examining Student Achievement Data Use multiple measures. Examine aggregate and disaggregated data (total population, sub-groups). Drill down –cluster, strand, item analysis. Engage stakeholders in the process. Teach students to examine their own data.

35. When analyzing data, use three reference points...  Proficiency (compared to the standard)  Progress (compared to ourselves; compare individual student performance over time)  Relative performance (compared to others – district, DFG, state)

36.  Look for trends over time.  Look at the aggregated and the disaggregated data Proficiency Rate Data

37. A Data Coach’s Guide to Improving Learning for All Students: Unleashing the Power of Collaborative Inquiry © 2008 by Corwin Press. All rights reserved. Observation Reminders  Made by the five senses.  Are quantitative and qualitative.  Contain no explanations.

38. When making observations look for:  What “pops out” or unexpected.  Patterns or trends which develop.  New questions.  Above all remember...

39. Observation Reminders  Vital to set aside assumptions.  Hallmark is exploration & discovery.  Refrain from making judgments or drawing conclusions.

40. Keep the Data Continuum in Mind Data Information Exist in a raw state w/out meaning. Knowledge Data given meaning in context Collection of info. deemed useful to guide action Mandinach, E. & Jackson, S. Using student data to support instruction. Presentation, 9/23/2010. REL Mid-Atlantic.

41. A Data Coach’s Guide to Improving Learning for All Students: Unleashing the Power of Collaborative Inquiry © 2008 by Corwin Press. All rights reserved. I notice that… I see that… I am struck by…. I am surprised that… Observe Starters

42. Tools to organize your data

43. Drawing Inferences, Posing Questions, & Exploring Possible Causes

44. A Data Coach’s Guide to Improving Learning for All Students: Unleashing the Power of Collaborative Inquiry © 2008 by Corwin Press. All rights reserved. Infer/Question  Generate interpretations.  Generate multiple possible explanations or implications from the data.  Consider possible causes and all aspects of the community which might impact what is occurring.

45. Talk it out: Trends, Strengths, Weaknesses

46. Consider...  What other data do we need?  Is it on hand?  If not, how will we collect the data?

47. Having data is only the beginning... What we choose to do with the knowledge developed through data analysis is critical.

48. Based on the data, what actions will we take? Action plan should include: Action steps, changes to instruction, practice, or process. Identify what success will look like. How will we assess/monitor?

49. Implement, take action, and monitor the results. Celebrate success. Balance long-and short-term improvement goals On the pathway to improved achievement

50. A data driven learning community embraces collaborative inquiry.

51. Principles of Data Use for Collaborative Inquiry  Go visual with the data: create large, colorful, and simple displays of data to aid understanding.  Use data to build understanding and ownership of problems. Engage in dialogue with data so that takes ownership of the problem and embraces the solutions together. Adapted from Love, et al (2008). A Data Coach’s Guide to Improving Learning for All Students - Corwin Press

52. Principles of Data Use for Collaborative Inquiry  “Hang out in uncertainty”: take time to learn as much as possible from the data. The first solution might not be the best one.  Separate observation from inference. Fully explore what is there to be learned before imposing interpretations on the data. Adapted from Love, et al (2008). A Data Coach’s Guide to Improving Learning for All Students - Corwin Press

53. Principles of Data Use for Collaborative Inquiry  Pay attention to the process: carefully structure meetings to maximize engagement, learning, attention to equity issues, and the integrity and safety of the group.  Assure that diverse voices are brought into the analysis. Multiple perspectives provide the richest information. Adapted from Love, et al (2008). A Data Coach’s Guide to Improving Learning for All Students - Corwin Press

54. Takes action based on the data!

55. Connecting the Dots Higher Student Achievement High Capacity Data Use Effective decision making Standards- based, rigorous, aligned curriculum Improved instruction

56. “Data provide power to... make good decisions, work intelligently, work effectively and efficiently, change things in better ways, know the impact of our hard work and how it benefits children, and help us prepare for the future” - Victoria Bernhardt As cited in V. Bernhardt. 2004. Data analysis for continuous school improvement 2 nd edition. Larchmont, NY

57. Questions & Concerns

58. Contact information Irene Ricci Sullivant Supervisor of Data Office of Research & Planning Phone: 966-2000 ext. 38410 e-mail: isullivant@camden.k12.nj.us