1. An Experimental Study of Project GLAD ® : Results from Year 1 Theresa Deussen - Elizabeth Autio Angela Roccograndi - Makoto Hanita WERA * December 13, 2013

2. Who’s here and why? 2

3. 3

4. A case in point: Growth of EL population 4

5. Project GLAD ® (Guided Language Acquisition Design)

6. 6

7. Project GLAD Input Chart (One of 35 instructional strategies)

8. Professional development 8

9. Research Questions 9 What about the impact specifically for English learners? Does that look the same? Do teachers implement it as intended? Does it impact students’ achievement in: Reading comprehension? Vocabulary? Writing? Science?

10. Study Design: Cluster Randomized Trial

11. 30 schools 21 districts 101 classrooms Grade 5

12. Teachers CharacteristicsTreatmentControl Mean years experience (SD)11.6 (8.4)17.2 (11.7) Percent female85%82% Percent w MA degree95%96% Percent w ESL endorsement5% Prior SIOP training66%70% 12

13. Students GroupN (percent) Total students tested2778 Students w pre & post2223 Current ELLs80 (4%) Former ELLs187 (8%) Non ELLs1956 (87%) 13

14. Outcomes Measures SubjectAssessment Reading comprehensionGates-MacGinitie VocabularyGates-MacGinitie Science: rocks & mineralsScott Foresman end-of-unit General science achievement Grade 5 Idaho state test Writing6+1 Traits writing, science topic 14

15. Year 1 Outcomes: Non ELLs SubjectTreatmentControlp =Effect size Comprehension509.57506.850.1770.07 Vocabulary513.21511.180.1870.06 Project science6.556.050.1010.24 State science210.24208.970.1480.13 15

16. Year 1 Outcomes: Non ELLs SubjectTreatmentControlp =Effect size Comprehension509.57506.850.1770.07 Vocabulary513.21511.180.1870.06 Project science6.556.050.1010.24 State science210.24208.970.1480.13 16

17. Year 1 Outcomes: Non ELLs Subjectp =Effect size Comprehension0.1770.07 Vocabulary0.1870.06 Project science0.1010.24 State science0.1480.13 17

18. Year 1 Outcomes: Non ELLs SubjectTreatmentControlp =Effect size Ideas 4.113.970.0600.22 Organization 4.253.750.2350.11 Voice 4.114.080.5900.07 Word Choice 4.114.050.1810.16 Sent Fluency 3.903.840.2390.11 Conventions 4.064.020.3010.07 18

19. Year 1 Outcomes: Non ELLs Writing TraitTreatmentControlp =Effect size Ideas 4.113.970.0600.22 Organization 4.253.750.2350.11 Voice 4.114.080.5900.07 Word Choice 4.114.050.1810.16 Sent Fluency 3.903.840.2390.11 Conventions 4.064.020.3010.07 19

20. Only 13% of students were current or former ELLs. 20

21. Year 1 Outcomes: ELLs SubjectTreatmentControlp =Effect size Comprehension482.1474.30.0990.24 Vocabulary479.7474.00.0920.21 Project science5.75.30.3030.19 State science0.3090.12 21 ˜ ˜

22. Year 1 Outcomes: ELLs Writing TraitTreatmentControlp =Effect size Ideas 3.753.540.0530.32 Organization 3.563.410.0860.27 Voice 3.973.940.7230.05 Word Choice 3.943.840.1120.22 Sent Fluency 3.583.560.5450.05 Conventions 3.773.760.8820.02 22 ˜ ˜

23. Practical Significance? …a work in progress… 23

24. What about implementation? 24

25. 25

26. Project GLAD Input Chart (One of 35 instructional strategies)

27. 27

28. 28

29. Year 1 by the numbers Included 101 teachers Conducted 167 observations (1.6 observations/teacher) 29

31. Treatment classrooms were different from control classrooms. Project GLAD ® strategies were observed in: 99% of observations in treatment classrooms 5% of observations in control classrooms

32. While all treatment teachers used Project GLAD ® strategies, the frequency of implementation varied.

33. The quality of implementation also varied among treatment teachers.

34. 34

35. Is uneven implementation good enough? 35

36. Next Steps: More Questions 36 What does impact look like in teachers’ second year of implementation? What does implementation look like in the second year? Do we find higher impact in classrooms with higher levels of implementation?

37. Implications What’s the take home message for you? What would you tell a district thinking about purchasing Project GLAD ® professional development? 37

38. Implications 38 “Good enough” implementation Suggestive findings for ELLs Exploring factors that could promote strong implementation

39. http://projectgladstudy.educationnorthwest.org/ Theresa.Deussen@educationnorthwest.org The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R305A100583 to Education Northwest. The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education. 39