Dr. Cory J. Steiner Data Steward NDASA Round Table—Midwinter Conference State of North Dakota The Future of Education: Creating a Culture of Data-Based Decision Making

Objectives By the end of this presentation, participants will: 1. Understand the role leadership plays for utilizing data in educational organizations Understand the four frames of ‘reframing organizations’ and roles the frames play in leadership Understand importance of creating a culture of quality data Identify roles needed to build a culture of quality data 2. Understand the State Longitudinal Data System (SLDS) past, present, and future 3. Understand job responsibilities of Data Steward 4. Understand practical use of the School Profile, Assessment Inventory, ACT Student Detail, NDSA Growth Model Roster, and NDSA Assessment Trend, Student Directory reports, and Developmental Courses by Subject Area, and District Developmental Courses

Purpose The focus must be on moving from good to great Get a little better every day `If you keep doing what you have been doing, you will keep getting what you have been getting Stockdale Paradox ‘Retain faith that you will prevail in the end, regardless of the difficulties and at the same time confront the most brutal facts of your current reality whatever they might be’ Good to Great: Why Some Companies Make the Leap and Others Don't (Collins, 2001)

Talking Point #1: Job Responsibilities

A Data What? A Data Steward… Provide data administration for ND K-12 State Longitudinal Data System (SLDS) Facilitate ‘cleaning’ of data (i.e.—garbage in, garbage out) Provide professional development for teachers and administrators Present to future teachers and administrators Develop security course Develop ethics course Develop assessment workshop Develop, lead and support services for K-12 Registrars Major initiative and development of training program Participate in strategic planning for the K-12 SLDS Utilize a Strengths, Weaknesses, Opportunities, and Threats Analysis (SWOT) Maintain a high level of professional competency Communication is key

Talking Point #2: Building a Culture with Quality Leadership

Get Two Who are the leaders in your organization? What is one thing an educational leader can do that (if done well) will make a difference in an organization?

Roles for Leaders in the ‘Data’ Movement AdministratorsTeachers

How Do You Shift Your Culture? Create A Sense of Urgency Any successful change starts with urgency Complacency is common throughout organizations o‘We are much too complacent…and we don’t even know it’ oDon’t mistake busy for non-complacent Change is not coming… It is here We must pay attention to change, but do not put it on a pedestal Focus must shift from change to the idea of continuous improvement oBefore this can happen, correction must take place Bottom Line is… We are paying insufficient attention to key opportunities SLDS is the opportunity to reframe your organization

Reframing Organizations Four frames of reference in decision-making (for all positions in education): 1. Human resources (people within organization) 2. Symbolic (culture) 3. Structural (procedures & policies) 4. Political (public) Every decision you make in your organization has an element from each frame (i.e.—Changing your lunch schedule, school calendar, etc.). Every decision has an affect on your culture Reframing Organizations: Artistry, Choice, and Leadership (Bolman & Deal, 2003)

Data-Driven Culture—A Model All student achievement decisions are based on data and not on adult preferences All instructional staff are involved in decisions All instructional staff members are involved in collaborative teams (PLC’s) that analyze state, district, school and classroom assessment data to: Plan instruction Set curricular priorities (strong link with common core) Develop action plans (with smart goals) Work towards achieving adequate yearly progress Engage in program evaluation On-going support and professional development is provided to refine skills in using data to make decisions that affect students and programs Value quality data Value a careful and ethical approach to using and sharing data Must create a culture that values self-reflection

Data Quality ‘Data pays us and grades us’ (NCES) Organizations/schools that value quality data engage in the following behaviors: Challenge stereotypes that are linked to data Don’t rush to judgment Don’t abuse the ‘Y’ axis Don’t use data to punish or create anxiety Don’t share identifiable information (FERPA)

Remember: Rushing to Judgment Has Legal Ramifications Personally Identifiable Information: Other information that alone, or in combination is linked or linkable to a specific student that would allow a person in the school community to identify the student with reasonable certainty

Something to Keep in Mind North Dakota’s general rule is: A group’s data, whose size is less than 10, can not be publicly displayed.

Truth, Lies & Data Quality A school created a website mistakenly telling 76 students they had been accepted into the school when in fact they had not been. The US Government mistakenly sent Christmas ornaments to 1,150 deceased marines and sailors with a flyer about an athletic reconditioning program. Some students were able to obtain administrative password to the system and logged in and cleared absences /tardies for a fee. A stolen laptop that contained command codes used to control the international space station was not encrypted. A high school in Las Vegas was ranked 13th among U.S. News and World Report based upon their student teacher ratio of 4 to 1 and a 100% pass rate on AP exams.

Talking Point #3: The State Longitudinal Data System

State Longitudinal Data System (SLDS)

What is a Data Warehouse?  Logical and strategic ordering and storage of data in central area  System consists of a statewide data warehouse that allows program evaluation over single or multiple years  Integrates data from several state agencies

SLDS Grantee States

North Dakota Statewide Longitudinal Data System (SLDS) SLDS is a cooperative project between: Information Technology Department (ITD) Department of Public Instruction(DPI) North Dakota University System (NDUS) Department of Commerce Department of Career and Technical Education Job Service of North Dakota Education Technology Council (ETC) Department of Health Department of Human Services Elements for education (K-12 and higher education), training, and employment programs For K-12, provides data for: Program evaluation Student evaluation Student programming (next day availability)

 Evaluate teacher programs to improve instruction  Know if graduates have skills to succeed in postsecondary and/or workforce  Simplify local, state, and federal reporting  Support data-driven decision-making for all educators Goals of Statewide Longitudinal Data Systems

Tool for Analyzing Data How are we doing? 1.Compared to Self Grade level, Sub Groups, Trends 2. Compared to Others National, State, Similar Schools 3.Compared to Absolutes Standards, Cut Scores, Scale Scores, Readiness Michael Fullan

In-Depth Analysis Creating Information and Avoiding DRIP, ( Data Rich Information Poor ) 1.Drill Down 2. Go Visual 3. Export

Elements of Longitudinal Data Systems 1. Student Enrollment Information 2. Information on Graduates, Transfers, Dropouts 3. State Assessment Scores 4. Information on Students Not Tested 5. College-Readiness Test Scores 6. A Teacher Identifier System 7. Student Transcript Information 8. Data on Student Transition and Success in College 9. Data on Preparation for Success in Postsecondary Education 10. An Audit System to Ensure Data Quality 11. Ability to Share Data from Preschool Through College 12. Unique Student Identifiers

ND SLDS in Stages Completed (infant) Provided data to the Regional Education Associations (REAs) for Hess Grant Produced matches to Workforce data for Adult Ed and CTE to create federal reports Modified Vital Statistics system to include birth records in the SLDS In Progress (adolescent) Expanding master person index (Active Directory Project) Gathering requirements for postsecondary data in the SLDS Developing data pump for Vital Statistics data Assigning state student IDs at birth Future (adult) Review portal products, potential vendor offerings, and the best direction for ND Gather new report requirements and expand SLDS to gather relevant data Plan work necessary to fulfill postsecondary grant

Phases…Now and in the Future Training Phase 1 is ongoing Introduce SLDS and basic features to administrative assistants, administrators, coordinators, and directors throughout the state 288 participants oWill be doing another round of initial training Phase 2 is beginning How to use SLDS to impact student achievement (‘drill down’ to student level) ‘SEED’ project with individual schools from each REA (Grafton, Maddock, and Kulm)

Talking Point #4: What Really Matters

SLDS: An Avenue for Self-Reflection ‘Use data for good and not evil’ Data Set #1 School A Scored 3 points more per game than previous year School B Scored 20 points more per game than previous year Discussion Question—Get Two What can you say about this set of data? Data Set #2 School A Scored 70 points per game in previous year School B Scored 38 points per game in previous year Discussion Question—Get Two What can you say about this set of data? Discussion Question—Get Two What other data do you need to say?

Questions Districts Must Ask…And Be Able to Answer…SLDS Provides Answers AttendanceAcademics What is the percentage of overall attendance? What groups have the lowest attendance rate? What groups show growth? Where is their a gap in achievement? Who is meeting their growth targets and goals? What are the characteristics of groups above, at, or below grade level? What progress monitoring is taking place?

SLDS: A Practical Approach to Training Lead to root cause analysis Reduce the problem and not just symptom Reduce wasted efforts and resources Encourage critical conversations and self-reflection Reinforce rationale for decisions Root Cause Analysis: School Leader's Guide to Using Data to Dissolve Problems (Preuss, 2003)

The Five Whys Simple problem-solving and identification Ask the question ‘why’ five times Dig ‘deeper’ to find a cause Example: ‘We have too many students tardy to class’ Why do we have so many tardies ‘Students are saying they don’t have time to get to their next class’ Why don’t students have time to get from one class to another? ‘Students are saying that four minutes is not enough time to get around’ Why is the passing time only four minutes? ‘We wanted to reduce the amount of time students were in the halls’ Why did we want to reduce time students were in the halls? ‘We were having a lot of issues with student behavior and discipline’ Why did we want to reduce disciplinary problems? ‘It was taking us away from our classes and taking students out of our classes’

Talking Point #5: Key Reports

Five Key Reports

School Profile Report Purpose: Provides an overview of enrollment, attendance, and NDSA results.

Assessment Inventory Report Purpose: List all assessments that are currently available in SLDS.

School Assessment Summary Report Purpose: Provides summary information selectable by assessment and subject area (NDSA, ACT, MAP, AimsWeb). Drills to school assessment details by assessment subject to allow for filtering on program and student demographics.

NDSA Growth Model Roster Report Purpose: Provides an overview of individual students in comparison to meeting their growth index

NDSA Growth Model Roster Report: So What, Now What ‘Goal setting is the single most powerful motivation tool in a leader’s toolkit’ (Blanchard) Allows you to set SMART goals with students

NDSA Trend Report Purpose: Provides NDSA trend data for a school.

Student Directory Report Purpose: Displays student proficiency details selectable by school year, grade, school, proficiency level, and student demographics

Student Dashboard Will be able to ‘designate’ students as at-risk Will do so by: Attendance Assessment Scores Course Grades 10% or more drop in average Grades below a ‘C’ level Courses repeating Failing courses

The Student Dashboard

New and Improved Student Dashboard

Student Directory: So What, Now What Course Grades: In workshop, review incoming student’s previous data Design seating chart around this data Provide ‘TLC’ for at-risk students Assessment Results Review assessment results with students (individually) in your class; set goals for future standardized assessments

Developmental Courses By Subject Area Purpose: A districts postsecondary enrollment in Developmental Courses, broken out by course subject. Shows totals for high school graduating classes who are enrolled at an NDUS campus.

District Developmental Courses Purpose: A districts postsecondary enrollment in Developmental Courses. Shows totals for high school graduating classes who are enrolled at an NDUS campus.

Talking Point #6: Odds n’ Ends

Odds n’ Ends Permissions REA Administrator (district) Teacher (school) Data Share Agreements Needed for: NWEA DIBELS AimsWeb mClass

Things to Know Student Mismatched Information Vertical Reporting Transportation Pictures on SLDS Aligning attendance codes Support Questions Section on website me with questions

Final Thoughts Have a genuine appreciation for the effort and commitment that everyone makes because together, we shall succeed. Casey Bradley, Jacksonville Jaguars Head Coach

Questions?? Dr. Cory J Steiner Blog: Phone: