Introduction to Data Statistics: Session 1

What is data?

What is a Variable? A Variable is

“Without data, all anyone has are opinions. Data elevates the probability that you’ll make the right decision.” -W. Edwards Deming

Data driven decision-making is a process that involves:

1. Mining (collecting and managing) the data

2. Analyzing data to create knowledge

3. Communicating data to support organizational learning

4. Using the data to inform school improvement planning

In which area could your school use strengthening ? Mining Data Analyzing Data Communicating Data Using data for school improvement

Survey: My staff is comfortable collecting data. My staff is comfortable talking about data with their teams. My staff members can manipulate their own students’ data. My staff uses data to make daily instructional decisions. I am satisfied with data-driven decision making at my school. A – Strongly AgreeB – Mostly Agree C – Mostly DisagreeD – Strongly Disagree

What data are available to us in Hanover?

What Principals Are Saying: Strengths in Data Driven Decision-Making Gathering data Analyzing benchmark data using ROS Works Having conversations about data with teams or faculties Grouping based on reading assessment data Sharing data with all teachers who instruct that student Setting goals/targets based on SOL test data

What Principals Are Saying: Weaknesses/Challenges in Data Driven Decision-Making Finding time to review data and reflect on what it means Understanding the significance of data analysis as a diagnostic tool to assist students rather than a personal reflection on teachers Having teachers see the big picture rather than simply focus on “their” students

What Principals Are Saying: Weaknesses/Challenges in Data Driven Decision-Making Knowing what to do after the analysis-- determining what intervention to use/how to remediate when weaknesses are apparent in the data Knowing what data to collect and analyze for grade levels without benchmark assessments Determining if benchmark assessment data is a reliable indicator of SOL test performance

Partner Talk How does your school currently collect, analyze, and use data to make instructional decisions?

What’s HOT in Hanover?

Student Response Systems

Math Pre- and Post- Tests TfHS

IEP Goal Data Academic: Behavioral:

Data Boards Electronic Data Board example

Involving Resource Teachers Grade levels can put strands of weaknesses on a Blackboard Discussion Board and anyone can add integration ideas to it.

Student Data Folders - Elementary Student Led Conferences - Secondary

Teacher Data Binders

PALS Quick Checks

Curriculum Based Measures (CBMs)

CBMs with handheld devices

ROS Data

How can YOU use ROS? Item analysis by student, class, or grade level Grade level classroom comparisons by standard/strand Student remediation grouping by weak strand performance Subgroup reports (by counts or percentages) Classroom assessments and keys added online

Business Objects Core Reports

Case Studies Elementary ReadingSecondary Math

Guiding Questions for Case Study Analysis Which strands have students mastered with at least 80% proficiency? Which strands require continued remediation? How does individual class performance compare? What next steps would you take as the school leader? teacher?

What do you see?

Ideas for Engaging Staff in Data Driven Decision-Making Define top 10 common data analysis terms individually, then as a group to reach consensus Give teachers access to ROS Works Offer professional development on data analysis tools Build common planning and remediation blocks into the master schedule Structure conversations--develop guiding questions for data discussions to be used by teams Develop common teacher data binders

Mini Break Out Sessions Practice with ROS Works Practice with electronic data boards Practice with student response systems

What’s next? What AHA moments did you have in this session? What new ideas would like to take back and implement with your staff? What training is needed to help with data-driven decision making for your staff?

Session Outcomes Define data-based decision making Identify key elements of data-based decision making process Describe DBDM for the purpose of program improvement

Continuous Improvement Planning Review & Interpret PPSS Data Make Decisions & Plan Actions Select & Implement EBP/PPSS

DataInformation Actionable Data Decisions

Data Information When numbers are interpreted and meaning is made of data Data are organized in a meaningful way to generate information View data graphically to identify patterns in youth’s in- school transition experiences and programs 41

When stakeholders synthesize information from various sources apply judgment to prioritize information consider the merits of different possible solutions 42 Information Actionable Data

Benefits of DBDM Supports school leaders to analyze, identify, and solve problems; Helps school leaders differentiate between knowledge and beliefs; Leads educators to invest in gathering important additional data that are missing; Focuses important discussions on outcomes rather than on individual’s styles or preferences; Leverages political support (Ancess, Barnett, & Allen, 2007)

Why is Data-Based Decision Making Important? DBDM helps educators (a) make informed decisions that lead to improved student achievement, (b) gain an objective picture of what needs to be improved, (c) focus on what is important, (d) discover what is working and what is not, and (e) monitor and celebrate movement toward desired outcomes. DBDM takes place within the broader context of what is happening in the school, district, and state, not in isolation of a one time event. Kowalski, Lasley, & Mahoney, 2008

Barriers to DBDM Lack of time to collect and interpret data (Kerr, Marsh, Ikemoto, Darilek, Barney, 2006; Fusarelli, 2008; Lachat & Smith, 2004;); Inaccessibility to data (Lachat & Smith, 2004; Wayman, 2005); Lack of capacity and technical skills for analysis and interpretation (Marsh, Pane, Hamilton, 2006; Ronka, Lachat, Slaughter, & Meltzer, 2009; Wayman, 2005); and Need for organizational structure to support multiple analyses, link multiple data sources, and adopt or develop a culture for using data (Lachat & Smith, 2004; Marsh, Pane, & Hamilton, 2006; Ancess, Barnett, & Allen, 2007).

School State District Actionable Data Types of Data  Input  Process  Outcomes  Satisfaction Types of Data  Input  Process  Outcomes  Satisfaction Information Types of Decisions to Drive Program Improvement  Address students’ needs (e.g., access to transition programs, and skill development) relative to youths’ post- school success  Prioritize student needs to improve post-school success  Set and assess progress toward goals in targeted action plan  Develop targeted action plan for implementation  Identify/reallocate resources in reaction to youths’ post- school success  Enhance processes to improve outcomes  Evaluate effectiveness of targeted action plan  Assess whether student needs are being met Types of Decisions to Drive Program Improvement  Address students’ needs (e.g., access to transition programs, and skill development) relative to youths’ post- school success  Prioritize student needs to improve post-school success  Set and assess progress toward goals in targeted action plan  Develop targeted action plan for implementation  Identify/reallocate resources in reaction to youths’ post- school success  Enhance processes to improve outcomes  Evaluate effectiveness of targeted action plan  Assess whether student needs are being met Framework for DBDM

What is Data-Based Decision Making (DBDM)? A systematic, yet dynamic, collection and analysis of various data to guide decisions

Why use data to make decisions? Recall the adage, “What gets measured and monitored gets improved”? Without monitoring and measuring activities, all you have are gut feelings, hunches, opinions, and unsupported professional judgment. DBDM helps improve the success of students and schools by grounding decisions in descriptive (qualitative) or numerical (quantitative) evidence. More access to better information enables educational professionals to test their assumptions, identify needs, and measure outcomes.

Types of Data Input data – student population demographics Disability category, Race/ethnicity, Gender, Age, and Method of Exit (graduate/dropout/ageout) Process data – how things happen instructional quality, in-school transition experiences, quality IEPs, LRE experiences, attendance/expulsion, Outcome data – results of the processes Achievement data, enrollment in higher education, attaining competitive employment, enrollment in postsecondary education/training, and attaining some other employment. Satisfaction data – how well something is liked Extent to which the high school program prepared youth for life after high school and assuming the roles of adult, such as going to college, participating in training program, or obtaining and maintaining a job.

Information Information is obtained through interpretation that leads to making meaning from the data. For example: Knowing the outcomes of specific subgroups of youth with disabilities Identifying patterns in students’ in-school transition experiences

Actionable Data: Prioritize Needs Synthesis information What are your data saying? Use professional judgment to prioritize the information In your data comparisons, are you OK or not OK with differences between groups? Weigh relative merits of possible solutions

Activity Use the blank DBDM framework on the next slide to: Identify the specific sources for data that you have available for each type of data in your state, district, and school that you might use as the basis for making decisions. List 3 – 5 types of decisions that you could make, specific to your school and/or district, to drive program improvement.

School State District Actionable Data Types of Data Information Types of Decisions to Drive Program Improvement Framework for DBDM

Getting Started with DBDM Start by asking the right questions. Prior to collecting data, be clear on what question you want answered. Decide what you want to know as well as what data are available to help you answer your questions.

What’s needed to link data use for decision making to outcomes for youth with disabilities? Quick and easy access to relevant data Capacity to analyze and interpret data Dedicated time to review data, identify appropriate strategies, plan, implement, and evaluate meaningful actions likely to improve transition programs. …..remember this is a Stakeholder Team effort

Data-based decision making: It’s a process! 1.Review data (e.g., post-school outcomes, current policies, procedures, predictors of post-school success, implementation of EBPs) 2.Interpret the data (e.g. identify patterns) 3.Prioritize needs based on data and context 4.Establish S.M.A.R.T. goals related to improving outcomes for youth with disabilities 5.Select specific strategies designed to improve post- school outcomes (e.g., evidence-based practices) 6.Make Decisions & Plan Actions 7.Monitor and evaluate results 8.Do it all again!!! Data Information Actionable Data

Continuous Improvement Planning Review & Interpret PPSS Data Make Decisions & Plan Actions Select & Implement EBP/PPSS

Contact Dawn A. Rowe, Ph.D. Project Coordinator National Post-School Outcomes Center University of Oregon