1. Center For Technology in Learning SRI International Investigating the Cumulative Impacts of Educational Technology Barbara Means, Mary Wagner, Geneva Haertel and Harold Javitz February 25, 2000 SRI International

2. Center For Technology in Learning SRI International 2 Study Rationale l Address over-arching concern of policymakers and the public l Investigate the accumulation of academic and motivational benefits associated with educational uses of technology l Identify the key dimensions of technology use that contribute to better outcomes for students l Describe variations in patterns of technology use in and out of school over time for different kinds of students, schools, and communities

3. Center For Technology in Learning SRI International 3 Example Data Collection Timeline l State the purpose of the discussion l Identify yourself

4. Center For Technology in Learning SRI International 4 Core Features l Nationally representative sample of seventh-graders l Secondary sample drawn from high-technology integration schools l Longitudinal (six years) l Student as primary unit of analysis l Incorporates diverse groups and contexts l Repeated measures of a broad set of student outcomes (e.g., basic skills, advanced inquiry and communication skills, engagement) and aspects of technology use (e.g., purpose, grouping, control)

5. Center For Technology in Learning SRI International 5 Sample Research Questions l To what extent are different approaches to using technology associated with different student outcomes? l Do the technology dimensions associated with significant gains differ for different types of schools and subgroups? l Are significantly better outcomes obtained in schools that make a concerted effort to integrate technology with instruction? l What patterns of technology use over time make a difference in producing and sustaining outcomes?

7. Center For Technology in Learning SRI International 7 Operationalizing Technology Use l Classification of practices into idealized “models” not feasible l Data can be collected on multiple dimensions of technology use, e.g.: l Typical grouping for technology use l Typical learning goals l Typical degree of student control l Average frequency and duration l Issues of characterizing technology experiences across classes, in and out of school, across years

8. Center For Technology in Learning SRI International 8 Balanced Assessment of Student Learning Outcomes Standardized tests of reading and mathematics Problem-based assessment of inquiry and communication skills - Complex context - On-line and off-line components - Embedded assessment of technology skills - Embedded assessment of metacognitive skills

9. Center For Technology in Learning SRI International 9 Factors Affecting the Nationally Representative Sample Focus on students Stratified sample Longitudinal Multiple data sources Multiple analytic purposes

10. Center For Technology in Learning SRI International 10 Generating the Nationally Representative Sample l Stage 1: Stratified sample of LEAs l Possible stratifiers l N > 200 to promote design efficiency l Stage 2: Students l Limit clustering within LEAs l N > 10,000

11. Center For Technology in Learning SRI International 11 Technology-Intensive Sample l Purposive sample l Start with districts and schools with strong technology infrastructures l Obtain indicators of strong, consistent technology use within these schools l N > 2,000 students

12. Center For Technology in Learning SRI International 12 Data Collection Components l School Background Survey l Student Survey l Parent Survey l Student Assessments l Teacher Survey l Student Transcripts

13. Center For Technology in Learning SRI International 13 Example Data Collection Timeline

14. Center For Technology in Learning SRI International 14 Analytic Issues l Weighting the Data l Analysis of Growth l Integrating Analyses of the Two Samples l Multiple Levels of Data

15. Center For Technology in Learning SRI International 15 Putting the Study in Perspective l Major investments in technology for schools l Emergence of the Digital Divide as a key policy issue l Precedent of longitudinal studies of special and compensatory education students l Need to keep pace with a changing world

16. Center For Technology in Learning SRI International 16

17. Center For Technology in Learning SRI International 17

18. Center For Technology in Learning SRI International 18

19. Center For Technology in Learning SRI International 19

20. Center For Technology in Learning SRI International 20

21. Center For Technology in Learning SRI International 21

22. Center For Technology in Learning SRI International 22

23. Center For Technology in Learning SRI International 23