1. Intro to NRS Data Diving
Mary A. Gaston, Ed.D. &
Jennifer Cooper-Keels
February 4, 2011

2. Why Look at Data? Data help us to…
Replace hunches and anecdotes with facts concerning the changes that are needed;
Identify root causes of problems;
Identify whether student or program goals are being met; and
Tell our stakeholders, including students, about the value of our programs and the return on their investments.
Ask participants the question—let them suggest answers—then show what we have, tying into what they have said.
11/15/2018

3. Data: A Carrot or a Stick?
Data may be used…
To highlight, clarify, and explain what’s happening in your program OR
To show what’s not happening in your program.
“However beautiful the strategy, you should occasionally look at the results.”
–W. Churchill
11/15/2018

4. Data Tell You Where you’ve been Where you are Where you’re going
How to get there
Data can help you design a quality program to help meet learners’ goals.
11/15/2018

5. Applied to Adult Education…
What can data do?
Guide you to improve instruction
Measure program success & effectiveness
Tell you if what you are doing is making a difference
Tell you which classes are getting the results you want—and which are not
Get to the root of problems, such as poor retention, low educational gains, or low transition rates
Let participants answer the question, “What can data do?” and list on chart paper—then go over the suggestions on the slide.
11/15/2018

6. Starting the Dive

7. Starting the Dive Educational Gain
While there are a number of measures in our Data System, the one we will focus on today is:
We really have three main measures in our data system. These are (let them name): Attendance, Educational Gain, and Transition Outcomes (goals). For this workshop, we are going to focus on Attendance and Educational Gain.
Educational Gain
11/15/2018

8. Educational Gain Advancement through 12 educational functioning levels
Core NRS measure
Can tell us:
Whether the program/students are meeting goals
Which sites/classes/teachers are most effective
Extent of student progress
Impact of changes
11/15/2018

9. Examples: What Increases Ed Gain
Make classes learner-centered
Focus on relevant knowledge
Opportunity for practice and application
Coherence
Sufficient Intensity and Duration 
(NRC, 1999; Garet, Porter, Desimone, Birman, & Yoon, 2001)
Ask participants to brain-storm about what increases educational gain Have them report out and make a list. Then—compare with what we have.
11/15/2018

10. Data analysis is only as good as the original data allow.
Do You Trust Your Data?
Data analysis is only as good as the original data allow.
Keys to good data collection systems include:
Clear policies and procedures for data entry
Data is entered & reviewed daily, weekly, or monthly
Teachers, staff, administrator all have access to data and review regularly
Teachers share data with students
Let participants brainstorm about what their own programs do to improve data quality as well as data access. Let some report out.
What does your program do to ensure data is accurate, reporting is timely, and staff have access to the data?
11/15/2018

11. Take a Dip in the Data Pool

12. Dive into the Data Pool
For each of the next few slides write down your observations
for discussion
What do you see?
What is interesting or unusual?
Do any questions or hypotheses come to mind as a result?
11/15/2018

13. Write Observations/Questions?
Make sure participants understand what they are seeing here. Give participants time to study the chart and write down observations. Then, in small groups, have them share observations and come up with ideas for where to go next? What other data do they want to see?
100
11/15/2018

14. Write Observations/Questions?
Make sure participants understand what they are seeing here. Give participants time to study the chart and write down observations. Then, in small groups, have them share observations and come up with ideas for where to go next? What other data do they want to see?
11/15/2018

15. Write Observations/Questions?
For time’s sake, we are limiting the dive today to 4 GED classes, represented on this chart. Make observations and come up with hypotheses about what might be happening in these classes. What other info do you want?
11/15/2018

16. Write Observations/Questions?
Here are overall completion rates (educational gain) as a comparison—included is the statewide average for perspective. Have participants make observations—and again, come up with other data they might want to see.
11/15/2018

17. Write Observations/Questions?
For time’s sake, we are limiting the dive today to 4 GED classes, represented on this chart. Make observations and come up with hypotheses about what might be happening in these classes. What other info do you want?
11/15/2018

18. Write Observations/Questions?
Main AM
This chart compares the 4 classes’ retention rate (avg. attendance per student) with respect to the number of class hours per week.
Observations? Other questions/data you want to see?
Young Adult
Main PM
Satellite R
11/15/2018

19. Write Observations/Questions?
Three variables here—size of bubble represents number of students served in the class that year. Each bubble is plotted based on educational gain and retention rate.
Observations? Questions? Suggestions for change?
11/15/2018

20. Where to Go From Here?
Based on what was learned from this “data dive”:
What should I change or replicate?
What data supports this change? What additional data should be reviewed?
What is the timeframe for change? Is it realistic?
What obstacles/barriers will we encounter?
What is the follow-up plan to measure and evaluate change?
11/15/2018

21. You Just Did a Data Dive! 11/15/2018
Review what has just happened—how we have walked through an NRS Data Dive. Share with participants that the Excel spreadsheet online has several data dives already worked out—BUT—
DO NOT limit yourself to just these dives—you need to have a system but answer the questions you need answered with the data/info you have.
11/15/2018

22. Questions: Educational Gain
Sample question
Further questions
Data collection & quality
What is the range of pre/posttest scores in my program/site?
Are all the test scores within the correct range for the test and class level?
Students
Which students are most likely to complete a level (student characteristics)?
Do students with higher contact hours have greater completion rates?
Teachers
What teacher characteristics are most related to level completion?
How high is teacher turnover at each site? Which sites retain teachers longest/best?
Instruction
Which instructional approaches have the greatest impact on gain?
Do assessments match course content?
Program
How many hours of PD do our teachers participate in?
Which PD have the greatest impact on student learning?
Program policy
Do placement policies differ among sites?
Which placement policies have an impact on educational gains?

23. Tools for Data Diving 11/15/2018
Time left? Let programs use Handout Five in packet to begin planning their own data dive.
11/15/2018

24. What questions do you want to answer about your own local program?
What Do I Want to Know?
What questions do you want to answer about your own local program?
Time left? Let programs use Handout Five in packet to begin planning their own data dive.
11/15/2018