You have a ton of data about your 1:1 program, now what? Jeni Corn, Elizabeth Halstead, Ruchi Patel, Danny Stanhope, NCLTI Winter Leadership Institute Friday Institute for Educational Innovation NC State University

Description School and district leaders of 1:1 learning initiatives need data from teachers and students to guide effective decision-making around planning and implementing successful projects. This session will provide opportunities to examine survey and observation data from real 1:1 schools and to discuss with colleagues what decisions should be made based on those results.

Welcome and Introductions Session Agenda o Our name, role, FI Evaluation Team Description o Overview of Evaluation and Instruments o Small Group Review of Results and Report Out o Large Group Discussion

Evaluation Overview Evaluation: Finding what works and what does not work for your project. Purpose of Educational Evaluation: Helping schools/districts use evaluation to continuously monitor and make effective decisions to improve their education projects for the benefit of their students, staff, and community. 4

Evaluation Overview “When the cook tastes the soup, that's formative evaluation. When the guests taste the soup that is summative evaluation.” ~ Bob Stake 5

Evaluation Overview Every evaluation has these components: Planning Data collection Data analysis Results 6

Evaluation Overview Monitor and Adjust 7

Evaluation Overview 8 Evaluation is an iterative process.

Measures/Data Sources Where do you get the data you need? o Check whether data already available to you might help answer your questions. o Determine the data sources you might use to meet remaining data needs. o Note that data sources are not data.  Example – teachers’ lessons plans are a rich data source, but it is necessary to “mine” them for actual data. Meaning you need a Lesson Plan Checklist or Lesson Plan Rubric. 9

Measures/Data Sources Common Data Traps o “Biting off more than you can chew” o Not collecting data needed to answer important questions o Collecting data that is not really useful o Jumping to decisions about sources of data o Neglecting hard-to-quantify data o Not formalizing “informal data” (e.g., anecdote, unrecorded observations) 10

Measures/Data Sources Common Data Traps o Not using valuable data after it has been collected o Neglecting implementation data (about activities and strategies) o Neglecting impact data (about objectives) o Making inferences about implementation from impact data and vice-versa …time spent planning saves more time later! 11

Collecting Evaluative Information: Data Sources and Methods, Analysis, and Interpretation o Selection of sources and methods are dependent on the evaluation questions. o Should consider a wide array of methods to collect data. o Quantitative data are analyzed using descriptive or inferential statistics. o Qualitative data are analyzed for patterns or themes. o Data must be interpreted not just analyzed. 12 Measures/Data Sources

Types of Data Sources o Documents o Records o Observations o Site Visits o Surveys o Interviews o Focus Groups o Tests 13 Measures/Data Sources

Some Examples for Evaluating Technology Projects 1.Documents: Rubrics for lesson plans, student products 2.Records: Logs of teacher technology use 3.Structured Observations: LoFTI 4.Likert-Scale Surveys: STNA, 1:1 Student Survey collaborative/adminresources/ 14 Measures/Data Sources

Introductions Instruments - STNA, LoFTI, and 1:1 Student Survey Results Guiding Questions Share Out 15 Session Activities – Small Groups

Guiding Questions: 15 min. 1. What are the results? How would you summarize the data? 2. What do the results mean for your school or district? 3. What are you going to do now? What decisions would you make about things like PD, infrastructure, tools/resources, staffing, School Improvement Plans, submitting a grant, or partnerships? Share Out 16 Session Activities

Whole Group Guiding Question: What might be next steps or decisions based on all three data sources? 17 Session Activities