Download an electronic copy of the slides and a handout here: projectgladstudy.educationnorthwest.org

Lessons Learned from An Experimental Study of Project GLAD ® Theresa Deussen & Claudia Rodriguez-Mojica EL Alliance Conference – March 14, 2014

Do you #ProjectGLAD #educationnw

Today’s Agenda Describe Project GLAD and how it works Report Year 1 results from our experimental study Take a closer look at what our findings mean for closing the achievement gap Share some reflections about how professional development might have a bigger impact

Language and content SIOP Sheltered Instruction Observation Protocol QuEST Quality English and Science Teaching Project GLAD Guided Language Acquisition Design

Project GLAD ® (Guided Language Acquisition Design)

Key program elements 35 instructional strategies Usable with any curriculum Intended as a coherent package that builds Readiness and motivation to learn Content knowledge Ability to converse at a high level about the topic Ability to read and write at a high level about the topic

What does it look like?

Project GLAD Input Chart (One of 35 instructional strategies)

11

How does this compare to what you do in your school?

I have never had training that has been this good!

Research Questions What is the impact of Project GLAD ® teacher training on fifth-grade students’ achievement in ELA and science? –For ELs –For non ELs?

Study population 30 Idaho schools 21 districts 50% located in rural communities 2250 grade 5 students 65% Free/Reduced-Price Lunch 33% Latino 62% White 13% ELs

Cluster Randomized Trial (CRT) 15 received Project GLAD ® 15 had “business as usual” 30 schools agreed to participate

Outcome measures English language arts Reading comprehension Vocabulary Essay writing Science Rocks & minerals unit test State science assessment

Year 1 Literacy Outcomes ELs only

What’s an effect size again? Difference between the Tx and C Standard deviation of the group

MeasureELsNon ELs Vocabulary.21 ~.04 Comprehension.24 ~.04 Writing Ideas.32 *.21 ~ Organization.27 ~.13 Voice Word Choice Sent. Fluency Conventions Rocks & Minerals State Science.12.13

MeasureELsNon ELs Vocabulary.21 ~.04 Comprehension.24 ~.04 Writing Ideas.32 *.21 ~ Organization.27 ~.13 Voice Word Choice Sent. Fluency Conventions Rocks & Minerals State Science.12.13

MeasureELsNon ELs Vocabulary.21 ~.04 Comprehension.24 ~.04 Writing Ideas.32 *.21 ~ Organization.27 ~.13 Voice Word Choice Sent. Fluency Conventions Rocks & Minerals State Science.12.13

MeasureELsNon ELs Vocabulary.21 ~.04 Comprehension.24 ~.04 Writing Ideas.32 *.21 ~ Organization.27 ~.13 Word Choice Voice Sent. Fluency Conventions Rocks & Minerals State Science.12.13

What’s a good effect size? Use empirical comparisons.

Literacy Effect sizes for ELs ReadingWriting ReadingVocabularyIdeasOrganization Project GLAD SIOP0.16*0.19*0.31** QuEST0.26 * Small sample with developers involved in training. ** Estimated based on data provided in Echevarria, Short & Powers 2006.

Literacy Effect sizes for ELs ReadingWriting ReadingVocabularyIdeasOrganization Project GLAD SIOP0.16*0.19*0.31** QuEST0.26 * Small sample with developers involved in training. ** Estimated based on data provided in Echevarria, Short & Powers 2006.

Literacy Effect sizes for ELs ReadingWriting ReadingVocabularyIdeasOrganization Project GLAD SIOP0.16*0.19*0.31** QuEST0.26 * Small sample with developers involved in training. ** Estimated based on data provided in Echevarria, Short & Powers 2006.

Literacy Effect sizes for ELs ReadingWriting ReadingVocabularyIdeasOrganization Project GLAD SIOP0.16*0.19*0.31** QuEST0.26 * Small sample with developers involved in training. ** Estimated based on data provided in Echevarria, Short & Powers 2006.

Literacy Effect sizes for ELs ReadingWriting ReadingVocabularyIdeasOrganization Project GLAD SIOP QuEST Small sample with developers involved in training. 2 Estimated based on data provided in Echevarria, Short & Powers 2006.

Science Effect sizes for ELs Unit TestState Test Project GLAD QuEST (NA) 1 QuEST also had positive impacts for nonELLs.

Back to our research questions What is the impact on Project GLAD on students’ reading, vocabulary, writing, and science achievement? For ELLs? For nonELLs?

Starting lower means you need a bigger boost.

Vocabulary

Comprehension

Can Project GLAD close that gap? Multiple years Additive effect? Compounding effect?

Reflections Content of the PD Factors affecting implementation

Content of the PD Does it prepare teachers to make a difference? YesNo Multiple representations of concepts Structured interactions with academic focus Use of primary language Focus on academically useful words Powerful science instruction

Content of the PD Does it prepare teachers to make a difference? YesNo Multiple representations of concepts X Structured interactions with academic focus X Use of primary language Focus on academically useful words Powerful science instruction

Content of the PD Does it prepare teachers to make a difference? YesNo Multiple representations of concepts X Structured interactions with academic focus X Use of primary language X Focus on academically useful words X Powerful science instruction X

Inquiry-based Evidence Collect Interpret Communicate Scaffolding + FOSS kits (ES = +1.39) Powerful science instruction

What about implementation?

AverageRange Average number of strategies per week (surveys) Average quality rating (observations) 69%19-100% 48 Implementation

AverageRange Average number of strategies per week (surveys) Average quality rating (observations) 69%19-100% 49

AverageRange Average number of strategies per week (surveys) Average fidelity rating (observations) 69%19-100% 50

AverageRange Average number of strategies per week (surveys) Average fidelity rating (observations) 69%19-100% 51 The frequency and fidelity of implementation varied significantly across teachers.

Factors affecting implementation Does the PD make it easy for teachers to implement? YesNo On-going Collective participation Explicit protocols for team work Focus on solutions, not activities Coherence

Factors affecting implementation Does it make it easy for teachers to implement? YesNo On-going X Collective participation X Explicit protocols for team work X Focus on solutions, not activities X Coherence XX

Teachers also told us what made it hard to implement.

Lack of time… Story map—this takes too much time to develop and implement. This is something to do in the summer. Mostly I haven’t used some of the strategies as a matter of time. Social studies is only 30 minutes.

Lack of materials… I was worried about the home school connections because I sent things home and I thought about the paper I would have to use. We have a copy limit per month.

Lack of confidence in using the strategy… Don’t feel competent as yet to do certain strategies like Team Tasks that are linked to other strategies. Same is true with Group Frame, ELD Review and the Process Grid and Mind Map. I’m not experienced enough to do the reading strategies either.

Collaboration may support Project GLAD implementation …[collaboration] changed how I implemented the strategies and how often I did them. The other person gave me an incentive to do things more often and if she hadn't been there I would have done far less.

What else might help?

60