1. Using Data from Student Information Systems to Improve Student Success
A webinar for IO Education
Jordan E. Horowitz, Vice President March 2018
© all material copyright IEBC, 2018

2. ▪ Telling Your Story with Data ▪ Continuous Improvement Continuously
IO Education is partnering with the Institute for Evidence-Based Change (IEBC) to improve the way data from your student information system is used.
IEBC provides expert assistance in the form of workshops and coaching to school districts that does not require educators to be analysts; this is not about data literacy.
What you’re about to experience is taken from a number of IEBC workshops including:
Over the past 20 years.
▪ Telling Your Story with Data
▪ Continuous Improvement Continuously
▪ How to Integrate Data into Your Meetings
▪ Psychology of Data Use
▪ Leading and Lagging Indicators

3. IEBC’s Model of Data UseTM
Analytics
Organizational Habits
Improved Student Success
Human Judgment and
Decision Making

4. IEBC’s Model of Data UseTM
Analytics
Organizational Habits
Improved Student Success
Human Judgment and
Decision Making

5. IEBC’s Model of Data UseTM
Analytics
Organizational Habits
Improved Student Success
Human Judgment and
Decision Making

7. It is important to collect and use data because of what we know about human judgment and decision-making
Four Villains:
Confirmation bias
Narrow focus bias
Short-term emotional interference
Overconfidence
See pp in our book: from Heath and Heath “Decisive” (2013)

8. Data also helps us get to consensus…
Is this information accurate?
What jumps out at you and why?
What are the themes?
Does this information challenge current assumptions about this population?
What might be contributing to success?
What might be detracting from success?
Is this the data we need to make a decision about a change in policy or practice?
What is the most important information?
What is missing?

9. Turn numbers into students
Remember, data is most valuable when it can be easily accessed and analyzed with the right tools.
Dashboards
Reports & Displays
Turn numbers into students

10. Data vs. Information
This is what we are used to – too much data – where is the information?

11. Data vs. Information
Useful
Usable
Actionable

12. Issue: How do you identify subpopulations of students who are systematically underperforming as a group and get them the support they need?

13. Leading and lagging indicators

14. Leading & Lagging Indicators
 Leading indicators are in your control and lead to your hoped- for success
 Lagging indicators are affected by what you do to influence your leading indicators

15. Lagging Indicators
Tend to be our drivers because they are what legislators and funders hold us accountable for.
High School graduation rates, college entry, college persistence. The dreaded undefined “college and career ready” is, unfortunately, often the primary lagging indicator.

16. Lagging Indicators
Not in our direct control and the students already are gone.

17. Leading Indicators
1. GPA, 2. your formative assessments, and 3. don’t forget about non-academic indicators such as attendance, emotional connection to school, and sense that teachers believe in you.

18. Leading and Lagging Indicators
Leading Indicators
Backward mapping

19. Leading and Lagging Indicators
Performance on Standardized Math Assessments
Leading Indicators
Attendance
Reading Skills
Formative Assessments

20. Leading and Lagging Indicators
Performance on Standardized Math Assessments
Who?
Leading Indicators
Attendance
Reading Skills
Formative Assessments

21. Leading and lagging indicators
Presenting data for action

22. The Typical Data Meeting
Too long; unfocused; no clear purpose; dreaded in advance

23. Keep It Simple

24. Chemistry Enrollment by Ethnicity
Make it explicit
Chemistry Enrollment by Ethnicity
Year One
Year Two
Year Three
African American
Male
15%
17%
16%
Female
18%
20%
Hispanic/Latino
19%
23%
Caucasian
22%
25%
Asian
26%
24%
21%
What is the headline here???

25. Latina Students Demonstrate Increased Chemistry Enrollment
Make it explicit
Latina Students Demonstrate Increased Chemistry Enrollment
Year One
Year Two
Year Three
African American
Male
15%
17%
16%
Female
18%
20%
Hispanic/Latino
19%
23%
Caucasian
22%
25%
Asian
26%
24%
21%

26. The Importance of Framing
Different ways of presenting the same information often evoke different emotions.
The statement that “the odds of survival one month after surgery are 90%” is more reassuring than the equivalent statement “one in ten die within one month of surgery.”
Similarly, cold cuts described as “90% fat-free” are more appetizing than when they are described as “10% fat.”

27. Successful information systems help you to access data in ways that are useful and usable to you; and they support presenting information in ways that lead to action.
They help you to get the right data to the right people at the right time (while students are still available for interventions)

28. Leading and lagging indicators
Presenting data for action
Choosing the intervention

29. Choosing the Interventions
Avoid the “Ornaments on a Christmas Tree Approach” of implementing a lot of disconnected and uncoordinated interventions.
Better to implement a few interventions that are…

30. Choosing the Interventions
Research-based
Match the solution to the problem
Scalable
High impact

31. Leading and lagging indicators
Presenting data for action
Choosing the intervention
Checking on impact

32. The Continuous Improvement Cycle
Assessment
Evaluation
Planning
Planning: development, implementation (rollout), communications, monitoring
Implementation

33. Evaluation
Formative: Carried out while programs are still being implemented. They help to form the program.
Summative: Carried out after a program is over. They are a summation of what happened.

34. When the chef tastes the soup that’s formative.
Evaluation
When the chef tastes the soup that’s formative.
When the guest tastes the soup that’s summative.

35. Qualitative = How or Why
Evaluation
Quantitative = What
Qualitative = How or Why

36. The Importance of Voice
Evaluation
The Importance of Voice

37. Leading and lagging indicators
Presenting data for action
Choosing the intervention
Checking on impact

38. Questions?

39. Jordan E. Horowitz, Vice President jhorowitz@iebcnow.org 562-743-7920
Thank You!
Jordan E. Horowitz, Vice President