1. Curriculum Customization Service Tamara Sumner, Holly Devaul University of Colorado at Boulder Digital Learning Sciences sumner@colorado.edu, devaul@ucar.edusumner@colorado.edudevaul@ucar.edu

2. About Digital Learning Sciences 1.Digital libraries 2.Software tools and services 3.User-centered design and evaluation 4.Leadership, technology transfer, training 5.Research DLESE National Science Digital Library Project 2061 at AAAS Denver Public Schools WGBH – Teachers’ Domain National Middle School Association California Counties Educational Technology Consortia

3. Our Sponsor Institutions Institute of Cognitive Science at CU  Joint PhD program with Computer Science; Education; Psychology; Linguistics; Philosophy; Speech, Language, and Hearing; Architecture and Planning  Education and Training, Language Processing, Cognitive Neuroscience University Corporation for Atmospheric Research  70 PhD-granting Universities; 17 Undergraduate Affiliates; 46 International Affiliates  National Center for Atmospheric Research and UCAR Community Programs

4. Program Goals Increase achievement for all students through customized instruction  Model for embedding customization using digital libraries into mainstream classroom practice  Measure impact on teaching and learning  Know-how to port to other districts and curricula

5. Motivation and Rationale  Accelerating demographic trends  Longitudinal student tracking  Teacher effectiveness and accountability  District portals  Digital learners

6. Approach: The Curriculum Customization Model District Curriculum Characterize Learner Needs Digital Library Resources Contribute and Share

7. Project History  Demonstrate feasibility for middle and high school Earth science  Technology available from NSDL; Resources from DLESE; Publisher content from AGI/IAT  Participatory design process with DPS  Pilot Study in Fall 2008 – v1.0, 10 teachers,10 weeks  District-Wide Field Trial in 2009/2010 – v2.0, 124 teachers, academic year

8. Curriculum Customization Service: Demo

9. Curriculum Customization Logic Model Nat’l Need and Prior Research CCS Intervention Impacts Effective instruction builds on learners’ current knowledge and background Large differences in teachers’ abilities to tailor instruction to learner needs High quality DL resources aligned to learning goals and curriculum School Districts: Curriculum Guides Teacher PD processes and incentives Technical infrastructure Teachers customize instruction to meet learner needs and to improve learner engagement CCS supports instructional planning, customization, and professional learning Learning Goals integrate: Core curriculum DL resources Assessments Common student conceptions User-contributed content Teachers share customizations and other contributions online Increases in student learning Customization is widespread and instruction is improved Teachers use the CCS to support their own professional learning in informal and formal settings Classrooms are becoming increasingly diverse Extensive and purposeful planning is a hallmark of effective teaching DL technologies enabling scalable access, use, and sharing Use of the CCS for ongoing professional learning is widespread Customizations support curricular coherence and use high quality DL resources Anticipated Outcomes Inputs District PD incentives encourage CCS use and customization, and recognize contributions / sharing Teachers develop improved skills and knowledge for making pedagogically sound customizations Teachers integrate DL resources into their instruction with greater confidence and frequency

10. 2009/2010 Denver Field Trial  All middle and high school Earth science teachers (n = 124)  Initial training session (101 teachers)  Incentives: projector, standard district hourly compensation for initial training session and evaluation activities  Research team contact: bi-weekly community updates and support email

11. Mixed Methods Research Design Teacher Usage, Attitudes, and Behaviors  Demographic data  Usage instrumentation**  Series of three surveys** Teacher Learning  Cognitive interviews Student Learning  District-wide, end-of-class student assessments administered by DPS  Adoption interviews**  Classroom Observations  Artifact Analysis

12. Adoption and Penetration Heavy Users Moderate Light Irregular 84 out of 124 logged in Tossed Out Web logs: 51% Adoption rate (63 out of 124) 10 – 25 hours 4 – 10 hours 1 – 4 hours Surveys: 57% Rely on CCS 69% Use it frequently

13. Customizing Instruction CCS has made it easier to customize instruction (61%)  Better engage their students (61%)  Offer alternative representations of science concepts or phenomena (47%)  Tailor instruction to student abilities (29%) CCS has made it easier to use formative assessments (84%)

14. From the Adoption Interviews  Below-grade level reading and ELL  Provide alternative visual representations of science phenomena  Leverage peer knowledge and experience Two planning uses: (1) supplement or customize curriculum and (2) as an archive/repository for digital resources I tried to use it [the CCS] with almost every lesson that I could… We have so many second-language learners that they needed the [visual] resources provided in the system… A lot of times when you look at the teacher's guide, the traditional paper teacher guide, the assessments that they provide, and the visuals, didn't exactly do the trick and give the second-language learners what they needed. But the visuals and the interactivity of the resources provided on your site does. We were talking about volcanoes… Where the volcanoes are on the face of the Earth, and a big misconception with the kids is that… Volcanoes can just form wherever and that they don’t follow a pattern, so I wanted them to get in their notes somehow that there are three very special places on the Earth where volcanoes can form…I used the CCS [and found] some animations of convergent, divergent, and hotspots… Rather than get up there and say, “[A] Convergent plate boundary is…” I could just say, “I’m going to show you an animation of a convergent plate boundary...” There are some kids who like, seeing that movement, it’s now embedded in their head.

15. Integrating DL Resources CCS has made it easier to find DL resources relevant to their teaching (89%) It [the CCS] is a space for me to save my materials on that won’t be erased…It’s a centralized location where I can find that extra material that I know is going to be, nine times out of ten, useful for me. It actually has cut down on [my] random searching on the Internet.

17. Interactive Resources

18. What is being “saved”? 58% of users have “saved”

19. Sharing Contributions Online SharedStuff – very highly used area  I look at SharedStuff for new ideas (96%)  Ability to upload and share is very useful (84%)  33% of users shared something, with top 5 “sharers” accounting for 35% of items Everybody has their own creative take on the curriculum, and how they enhance it is what I'm looking for when I go in there [the CCS]. When a teacher from across town might have found a great Web site or a video clip or something that really does bring the point home, makes it more relevant to the students, that's what I'm looking for, and so far, I have found some things like that on [the CCS].

20. Support for Professional Learning  Use daily or several times a week to improve science background knowledge (46%)  DPS, on their own, integrated the CCS into their ongoing professional development (“New Users Workshop”) “This is wonderful timing. Yves and I will be able to use the site for our new user's group on Saturday, Feb. 6. Thank-you, thank-you, THANK-YOU!"

21. Classroom Instructional Tool Two classroom uses: (1) portal of instructional artifacts to project and (2) source of supplementary materials for students From Survey:  Create web projects containing interactive resources for students to access  Save AGI/IAT embedded assessments and instructional materials to MyStuff I think that since students are visual learners, they're hands-on learners, they're growing up in this technological age… Seeing the animations… Really drives home the idea or the topic… I ask them if it does help them [to see graphic representations]… If whatever they just viewed made the material make more sense, and often times, they [say] "Wow… That totally made sense… Can we see it again?" I think they benefit from it a lot, and they're vocal about it. They love it. If I come across links [to digital resources] while I'm doing planning that I think that would help students study or recall information that we've covered in class, I'll take the link from the CCS and I put it onto my own Web page, so that the students can actually access it.

22. Overall Value as a Teaching Tool CCS helps them to use their time more efficiently (80%) The BSCS, our biology book, they've come up with the websites where they have the book online, but that's about it... They don't have [science] standards… They don't have the animations… They don't have the share tab where you can look at different people's lesson plans… They're still behind and I wish that they would catch up to you guys... [The CCS is] kind of the ideal, um, website that everyone's kind of looking at now because it helps teachers a lot. [Without the CCS] the students would have not nearly as rich an experience and probably wouldn't, um, probably wouldn’t learn as deeply as they do.

23. Barriers  Adoption – Lack of time and decent technology in classroom  Use – Better ways to manage MyStuff and SharedStuff

24. Curriculum Customization Logic Model Nat’l Need and Prior Research CCS Intervention Impacts Effective instruction builds on learners’ current knowledge and background Large differences in teachers’ abilities to tailor instruction to learner needs High quality DL resources aligned to learning goals and curriculum School Districts: Curriculum Guides Teacher PD processes and incentives Technical infrastructure Teachers customize instruction to meet learner needs and to improve learner engagement CCS supports instructional planning, customization, and professional learning Learning Goals integrate: Core curriculum DL resources Assessments Common student conceptions User-contributed content Teachers share customizations and other contributions online Increases in student learning Customization is widespread and instruction is improved Teachers use the CCS to support their own professional learning in informal and formal settings Classrooms are becoming increasingly diverse Extensive and purposeful planning is a hallmark of effective teaching DL technologies enabling scalable access, use, and sharing Use of the CCS for ongoing professional learning is widespread Customizations support curricular coherence and use high quality DL resources Anticipated Outcomes Inputs District PD incentives encourage CCS use and customization, and recognize contributions / sharing Teachers develop improved skills and knowledge for making pedagogically sound customizations Teachers integrate DL resources into their instruction with greater confidence and frequency

25. Additional Analyses Underway  Cognitive Interview  Artifact Analysis  District- administered exams  Data Mining Target Knowledge Idea Units (from Cognitive Interviews) Conceptual Ideas (Deep) “They should have a basic process knowledge for igneous, metamorphic, and sedimentary rocks, how do those things get formed-” (Teacher 1) “If I’m a student looking at this we, um, would've hit continental drift and seafloor spreading, … as a student, I would have to know the difference between those two” (Teacher 2) Materials/Rote Facts (Shallow) “Um, so the background, they need some basic vocabulary for this” (Teacher 1)

26. Reflections and Next Steps  Why is it working?  Embedding, user experience, classroom challenge, aligned interactive materials  Why do our partners want to scale up now?  Implementing inquiry curriculum with a platform for district-supported professional learning  New STEM curricula with DPS  New districts with EarthComm and IES programs (IAT/AGI)