1. Bureau of Indian Education Special Education Academy Using State and Local Data to Improve Results Sandy Schmitz, Ph.,D DAC Tampa, FL September 12 - 15, 2011

2. DAC’s Goal Form partnerships in states that join state and local agencies in the use of data to drive improved results

3. Premises Data Use involves: Working through a Collaborative Team approach. Engaging Team in a Continuous Improvement Process. Relating the Data to specific Problem/Issue. Using Data is an Iterative Process!

4. DAC Concepts for Data Use Consist of three phases w/ several steps: Phase 1: Preparation – Identify relevant data Phase 2: Inquiry – Conduct data analysis – Determine Root Cause Phase 3: Action – Plan for improvement – Implement Plan – Evaluate progress 1. Identify relevant data 2. Conduct data analysis to generate hypothesis 3. Test Hypothesis to determine root cause 4. Plan for Improvement 6. Evaluate Progress Data Analytics 5. Implement Plan

5. Data Quality Standards Timely Accurate o Reliable  Consistent  Objective o Valid  Complete  Credible Secure Useful o Interpretable o Relevant o Transparent o Accessible Data collected, submitted, analyzed, and reported must be:

6. MUST INVOLVE A TEAM Administrator Data Person Special Education General Education Parents Others as needed

7. MODULE 1: IDENTIFY RELEVANT DATA How to identify relevant data?

8. The key to identifying relevant data is to ensure that you clearly define or select a specific problem or issue.

9. The problem should be a clear concise statement of the issue(s) that need to be addressed by a problem solving team. Problem Description

10. The described problem should answer: 1.Who has the problem? This should explain who needs the solution and who will decide the problem/issue has been resolved 2. What is the problem? This should explain why the team is needed 3. Where did the problem occur? 4. When did the problem occur? This should provide the context and timeframe of the problem/issue

11. EXAMPLE: ISBE Problem For more than three years, as documented in the APR and in public reports, the State of Illinois has reported percentages of students with disabilities included in regular classrooms 80% or more of the day at a lower rate than the national average.

12. Educational Environments: 2006 Percent Served Inside Regular Class >80% of the School Day ( Highest and Lowest ranked Entities) (Reporting States, DC & BIE Schools Average = 54% ) Source: Dec 1, 2006 count. IDEA Data provided by OSEP Table 5.8 See www.ideadata.org – State Ranks-Part Bwww.ideadata.org 21% (49) 49% (47) 65% (10) 74% (2) BIE 78% (1) 71% (3) 69% (5) 67% (6) 41% (48) 49% (46) X= suppressed data 10% (50) 70% (4) Highest Ranked for Percent Served in Setting Lowest Ranked for Percent Served in Setting 67% (7) 66% (8) 65% (9) 50% (44) 50% (45) 50% (43) D.C. States Not Ranked- Denominator <10 50% (41)

13. Educational Environments: 2007 Percent Served Inside Regular Class >80% of the School Day ( Highest and Lowest ranked Entities) (Reporting States, DC & BIE Schools Average= 57% ) Source: Dec 1, 2007 count. IDEA Data provided by OSEP Table 5.2. See www.ideadata.org – State Ranks-Part Bwww.ideadata.org 18% (49) 48% (47) 79% (1) 77% (2) 69% (5) 71% (3) 42% (48) 51% (45) X= suppressed data 52% (44) 70% (4) Highest Ranked for Percent Served in Setting Lowest Ranked for Percent Served in Setting 17% (50) D.C. B.I.E 69% (6) 69% (7) 67% (8) 67% (9) 51% (46) 52% (41) 52% (42) 52% (43) 64% (10) States Not Ranked- Denominator <10 64% (10)

14. Educational Environments: 2008 Percent Served Inside Regular Class >80% of the School Day ( Highest and Lowest ranked Entities) (Reporting States, DC & BIE Schools Average= 57% ) Source: Dec 1, 2007 count. IDEA Data provided by OSEP Table 5.2. See www.ideadata.org – State Ranks-Part Bwww.ideadata.org 15% (50) 49% (46) 81% (1) 77% (2) 70% (5) 71% (4) 44% (47) 52% (42) X= suppressed data 52% (41) 72% (3) Highest Ranked for Percent Served in Setting Lowest Ranked for Percent Served in Setting 18% (49) D.C. B.I.E 70% (7) 70% (6) 68% (9) 50% (45) 52% (44) 52% (43) States Not Ranked- Denominator <10 69% (8) 68% (10) 43% (48)

15. Educational Environments: 2009 Percent Served Inside Regular Class >80% of the School Day ( Highest and Lowest ranked Entities) (Reporting States, DC & BIE Schools Average= 57% ) Source: Dec 1, 2007 count. IDEA Data provided by OSEP Table 5.2. See www.ideadata.org – State Ranks-Part Bwww.ideadata.org 17% (50) 51% (45) 82% (1) 76% (2) 70% (9) 73% (3) 45% (48) 51% (44) X= suppressed data 54% (41) 72% (4) Highest Ranked for Percent Served in Setting Lowest Ranked for Percent Served in Setting 36% (49) D.C. B.I.E 71% (7) 70% (8) 50% (46) 51% (43) 53% (42) States Not Ranked- Denominator <10 71% (6) 70% (10) 49% (47) 72% (5)

16. Illinois Sw/D Indicators 5a, 5b, 5c

17. Problem Description 1.Begin with an initial State Agency problem statement. 2.Local agencies revise the statement as necessary when additional data are collected and analyzed throughout the data analytics process

18. Given your district problem statement…. Identify which data needed to gather the evidence necessary to answer “why” the problem or issue exist.

19. Identify Relevant Data (cont’d) Relevant information may include: – District, building, or school - level data – Disaggregated student population data – Data on the status of highly qualified personnel disaggregated by building/school

20. Module 2 Step 2: Conduct Data Analysis to Generate Hypothesis

21. DATA ANALYSIS What is data analysis and how is it done?

22. Analysis is based upon what the problem is… “WHY” The question “WHY” comes into play

23. Analysis involves organizing and understanding data based on criteria you develop; it is useful when you want to find some trend or pattern. Source: Purdue Online Writing Lab

24. Source: Adapted from Webopedia Drill down involves accessing information by starting with a general category and moving through the hierarchy of field to file to record; it is the act of focusing in to get to the root cause.

25. HYPOTHESIS What is a hypothesis and how do you generate one?

26. A hypothesis is defined as “……. a starting-point for further investigation from known facts”. (The Concise Oxford Dictionary, 1990)

27. Example of a Hypothesis Sw/Ds who have greater access to the general curriculum as measured by time spent with typical peers will perform better on tests of reading proficiency than those Sw/Ds who have less access to the general.

28. Another example of a Hypothesis IF-Then If reading proficiency is related to access to the general education curriculum, then increasing SW/Ds access to the general education curriculum will improve their performance on the statewide reading assessment.

29. DATA ANALYSIS PLAN What is an analysis plan and why develop one?

30. The analysis plan provides an outline of additional data that need to be analyzed to test the hypothesis and determine root cause; it helps with preparing a clear and concise presentation of the results of your analysis activities

31. Data Analysis Report - Components The analysis should include: Types of data to be examined (e.g., demographic data; data about programs, process and outcomes; etc.) – Student records – Interviews (e.g., position and roles of building and district personnel) – Observations

32. Let’s get you started on Module 3 Step 3: Test Hypothesis to Determine Root Cause

33. “ In order to shake a hypothesis, it is sometimes not necessary to do anything more than to push it as far as it will go. Denis Diderot

34. DATA TRIANGULATION What is it and how do you do it?

35. Data triangulation is a process of examining multiple sources of data to form a conclusion or generalization. Source: Wikipedia

36. ROOT CAUSE ANALYSIS Why determine the root cause?

37. Determining the Root Cause: Helps Dissolve the Problem Eliminates Patching Conserves Resources Facilitates Discussion Provides Rationale for Strategy Selection

38. Module 4 Plan for Improvement

39. When data indicate a problem or issue, districts and schools should develop or revise an improvement plan that outlines the course of action it will take to improve results. Adapted from ESEA, Title I, Sec. 1116 (b)(3)

40. IMPROVEMENT PLAN What is an improvement plan and why develop one?

41. Basic Components of an Improvement Plan  Goals  Activities  Timelines  Person(s) Responsible  Resources  Evidence of Change

42. Setting goals is critical to determine how much progress (as documented in an improvement plan) is acceptable and what amount of progress is not; it establishes internal accountability, high expectations and a trajectory by which to evaluate progress. Boudette, City, and Murnane - Datawise

43. First Consider Evidence of Change Indicate that the changes in the system have yielded “significantly” improved results for students with disabilities in the problem area Must be demonstrated through gains in student results data Not about “effort” but about “impact”

44. Setting SMART Goals Specific Measurable Attainable Realistic Timely Source: goal-setting-guide.com

45. Organize Data Sources to Assess Progress Short-term data – information that can be collected daily or weekly Medium-term data – information gathered systematically at the building-, grade-, or district- level at periodic intervals during the year Long-term data – information gathered annually (e.g., students’ performance on statewide tests)

46. Module 5 Step 5: Implement Plan

47. Remember! Implement with Fidelity Team work Timely Continuously assess progress Keep your eye on the ball (Evidence of Change)

48. Module 6 Step 6: Evaluate Progress

49. 1) District MET goals and showed evidence of change. Action: Create evaluation strategies to ensure sustainability OR 2) District DID NOT meet goals and show evidence of change. Action: Re-evaluate process

50. Pathway 1: Sustainability Pathway 1: Sustainability Does the team have: a plan to ensure sustainability over time? strategies to keep the work fresh/ongoing? routine checks to review data to ensure sustainability? strategies to ‘raise the bar’?

51. Pathway 2: RE-EVALUATE Looking back, did the team: 1.implement the Improvement Plan with fidelity? 2.set unreasonable goals ? 3.identify the correct root cause(s) ? 4.analyze the appropriate data to test the hypothesis ?

52. Pathway 2: RE-EVALUATE (Cont’d) Did the team: 5.develop an adequate Data Analysis Plan (considered all additional data needed to test hypothesis & determine root cause)? 6.have sufficient data to develop hypothesis ? 7.articulate a specific & concise problem statement ? 8.Involve the appropriate people ?

53. Questions?

54. Thank you! For more information contact Sandy Schmitz (sschmi@lsuhsc.edu)sschmi@lsuhsc.edu