1. Download an electronic copy of the slides and a handout here: projectgladstudy.educationnorthwest.org

2. Lessons Learned from An Experimental Study of Project GLAD ® Theresa Deussen & Claudia Rodriguez-Mojica EL Alliance Conference – March 14, 2014

3. Do you tweet? @TLDeussen #ProjectGLAD #educationnw

4. Today’s Agenda Describe Project GLAD and how it works Report Year 1 results from our experimental study Take a closer look at what our findings mean for closing the achievement gap Share some reflections about how professional development might have a bigger impact

5. Language and content SIOP Sheltered Instruction Observation Protocol QuEST Quality English and Science Teaching Project GLAD Guided Language Acquisition Design

6. Project GLAD ® (Guided Language Acquisition Design)

7. Key program elements 35 instructional strategies Usable with any curriculum Intended as a coherent package that builds Readiness and motivation to learn Content knowledge Ability to converse at a high level about the topic Ability to read and write at a high level about the topic

8. What does it look like?

9. Project GLAD Input Chart (One of 35 instructional strategies)

11. 11

12. How does this compare to what you do in your school?

13. I have never had training that has been this good!

14. Research Questions What is the impact of Project GLAD ® teacher training on fifth-grade students’ achievement in ELA and science? –For ELs –For non ELs?

15. Study population 30 Idaho schools 21 districts 50% located in rural communities 2250 grade 5 students 65% Free/Reduced-Price Lunch 33% Latino 62% White 13% ELs

16. Cluster Randomized Trial (CRT) 15 received Project GLAD ® 15 had “business as usual” 30 schools agreed to participate

17. Outcome measures English language arts Reading comprehension Vocabulary Essay writing Science Rocks & minerals unit test State science assessment

18. Year 1 Literacy Outcomes ELs only

20. 0.21 0.24

21. What’s an effect size again? Difference between the Tx and C Standard deviation of the group

22. MeasureELsNon ELs Vocabulary.21 ~.04 Comprehension.24 ~.04 Writing Ideas.32 *.21 ~ Organization.27 ~.13 Voice.05.08 Word Choice.22.14 Sent. Fluency.05.12 Conventions.02.07 Rocks & Minerals.19.23 State Science.12.13

23. MeasureELsNon ELs Vocabulary.21 ~.04 Comprehension.24 ~.04 Writing Ideas.32 *.21 ~ Organization.27 ~.13 Voice.05.08 Word Choice.22.14 Sent. Fluency.05.12 Conventions.02.07 Rocks & Minerals.19.23 State Science.12.13

24. MeasureELsNon ELs Vocabulary.21 ~.04 Comprehension.24 ~.04 Writing Ideas.32 *.21 ~ Organization.27 ~.13 Voice.05.08 Word Choice.22.14 Sent. Fluency.05.12 Conventions.02.07 Rocks & Minerals.19.23 State Science.12.13

25. MeasureELsNon ELs Vocabulary.21 ~.04 Comprehension.24 ~.04 Writing Ideas.32 *.21 ~ Organization.27 ~.13 Word Choice.22.14 Voice.05.08 Sent. Fluency.05.12 Conventions.02.07 Rocks & Minerals.19.23 State Science.12.13

26. What’s a good effect size? Use empirical comparisons.

27. Literacy Effect sizes for ELs ReadingWriting ReadingVocabularyIdeasOrganization Project GLAD 0.240.210.320.27 SIOP0.16*0.19*0.31** QuEST0.26 * Small sample with developers involved in training. ** Estimated based on data provided in Echevarria, Short & Powers 2006.

28. Literacy Effect sizes for ELs ReadingWriting ReadingVocabularyIdeasOrganization Project GLAD 0.240.210.320.27 SIOP0.16*0.19*0.31** QuEST0.26 * Small sample with developers involved in training. ** Estimated based on data provided in Echevarria, Short & Powers 2006.

29. Literacy Effect sizes for ELs ReadingWriting ReadingVocabularyIdeasOrganization Project GLAD 0.240.210.320.27 SIOP0.16*0.19*0.31** QuEST0.26 * Small sample with developers involved in training. ** Estimated based on data provided in Echevarria, Short & Powers 2006.

30. Literacy Effect sizes for ELs ReadingWriting ReadingVocabularyIdeasOrganization Project GLAD 0.240.210.320.27 SIOP0.16*0.19*0.31** QuEST0.26 * Small sample with developers involved in training. ** Estimated based on data provided in Echevarria, Short & Powers 2006.

31. Literacy Effect sizes for ELs ReadingWriting ReadingVocabularyIdeasOrganization Project GLAD 0.240.210.320.27 SIOP0.16 1 0.19 2 0.31 2 QuEST0.26 1 Small sample with developers involved in training. 2 Estimated based on data provided in Echevarria, Short & Powers 2006.

32. Science Effect sizes for ELs Unit TestState Test Project GLAD 0.190.12 QuEST 1 0.16(NA) 1 QuEST also had positive impacts for nonELLs.

33. Back to our research questions What is the impact on Project GLAD on students’ reading, vocabulary, writing, and science achievement? For ELLs? For nonELLs?

38. Starting lower means you need a bigger boost.

39. Vocabulary

40. Comprehension

41. Can Project GLAD close that gap? Multiple years Additive effect? Compounding effect?

42. Reflections Content of the PD Factors affecting implementation

43. Content of the PD Does it prepare teachers to make a difference? YesNo Multiple representations of concepts Structured interactions with academic focus Use of primary language Focus on academically useful words Powerful science instruction

44. Content of the PD Does it prepare teachers to make a difference? YesNo Multiple representations of concepts X Structured interactions with academic focus X Use of primary language Focus on academically useful words Powerful science instruction

45. Content of the PD Does it prepare teachers to make a difference? YesNo Multiple representations of concepts X Structured interactions with academic focus X Use of primary language X Focus on academically useful words X Powerful science instruction X

46. Inquiry-based Evidence Collect Interpret Communicate Scaffolding + FOSS kits (ES = +1.39) Powerful science instruction

47. What about implementation?

48. AverageRange Average number of strategies per week (surveys) 12.50-22 Average quality rating (observations) 69%19-100% 48 Implementation

49. AverageRange Average number of strategies per week (surveys) 12.50-22 Average quality rating (observations) 69%19-100% 49

50. AverageRange Average number of strategies per week (surveys) 12.50-22 Average fidelity rating (observations) 69%19-100% 50

51. AverageRange Average number of strategies per week (surveys) 12.50-22 Average fidelity rating (observations) 69%19-100% 51 The frequency and fidelity of implementation varied significantly across teachers.

52. Factors affecting implementation Does the PD make it easy for teachers to implement? YesNo On-going Collective participation Explicit protocols for team work Focus on solutions, not activities Coherence

53. Factors affecting implementation Does it make it easy for teachers to implement? YesNo On-going X Collective participation X Explicit protocols for team work X Focus on solutions, not activities X Coherence XX

54. Teachers also told us what made it hard to implement.

55. Lack of time… Story map—this takes too much time to develop and implement. This is something to do in the summer. Mostly I haven’t used some of the strategies as a matter of time. Social studies is only 30 minutes.

56. Lack of materials… I was worried about the home school connections because I sent things home and I thought about the paper I would have to use. We have a copy limit per month.

57. Lack of confidence in using the strategy… Don’t feel competent as yet to do certain strategies like Team Tasks that are linked to other strategies. Same is true with Group Frame, ELD Review and the Process Grid and Mind Map. I’m not experienced enough to do the reading strategies either.

58. Collaboration may support Project GLAD implementation …[collaboration] changed how I implemented the strategies and how often I did them. The other person gave me an incentive to do things more often and if she hadn't been there I would have done far less.

59. What else might help?

60. 60