The York READing for MEaning Project A randomised controlled trial of three interventions designed to improve poor comprehenders reading comprehension skills Paula Clarke, Emma Truelove, Maggie Snowling, Charles Hulme Centre for Reading and Language, Department of Psychology

The Simple Model of Reading Poor comprehenders Dyslexia Typical Reader Poor Reader Poor Comprehender Average word readers but poor at reading comprehension 10% of primary school aged children (Nation & Snowling,1997) Unnoticed in the classroom Persistent difficulties (Ehlich, Remond & Tardieu, 1999; Cain & Oakhill, 2006) - Listening Comprehension + - Decoding + The Simple Model of Reading (after Gough & Tunmer, 1986) Is there a core cognitive deficit in poor comprehenders? Are there a number of different underlying cognitive deficits which might cause a poor comprehender profile?

Poor comprehenders Evidence of impairments on a range of different language tasks: Vocabulary Nation, Clarke & Snowling, 2002; Nation, Clarke, Marshall & Durand, 2004, Stothard & Hulme,1992 Oral expression Nation, Clarke, Marshall & Durand, 2004 Figurative language Narrative skills Cragg & Nation, 2006; Cain & Oakhill, 1996; 2006 Grammatical development Nation, Clarke, Marshall & Durand, 2004; Nation & Snowling, 2000 Verbal reasoning Inferencing Oakhill, 1984; Cain & Oakhill, 1999; Nation, Clarke, Marshall & Durand, 2004

Different selection criteria and selection measures have been used. Poor comprehenders Evidence of impairments in other areas: Comprehension monitoring Erlich, Remond & Tardieu 1996; Yuill, Oakhill & Parkin, 1989; Cain, Oakhill & Bryant, 2004; Oakhill, Hart & Samols, 2005 Verbal working memory Nation, Adams, Bowyer-Crane & Snowling, 1999; Cain & Oakhill, 2006; Cain, 2006 Suppression/Inhibition Cain, 2006 However a key issue is that poor comprehender samples being examined across studies are not equivalent. Different selection criteria and selection measures have been used.

Why is an intervention study needed? There is no consensus concerning the core cognitive deficit in poor comprehenders. The majority of studies have focused on highly selected samples of children and have been small in scale. Studies have used largely cross sectional, control group comparison or correlational designs so have not yet been able to demonstrate causal influences. A large scale intervention study will allow us to: Examine different poor comprehender profiles and subgroups. Investigate which factors drive response to intervention and improvements in reading comprehension. Create resources and training programmes for schools (if the interventions evaluated can be shown to be effective).

Previous research Yuill and Oakhill (1988) - Inference Training Oakhill & Patel (1991) - Mental Imagery Training Yuill and Joscelyne (1988) - Story Structure & Inference Training Johnson-Glenberg (2000) - Verbal vs. Visual RT Training Interventions teaching specific components of reading comprehension have generated some impressive results in poor comprehenders. Promising findings have not yet been replicated. Studies have focused largely on text level and metacognitive skills. No studies have pulled these components together to create a comprehensive reading comprehension training programme. The only study to include an untreated control group did not use a RCT design.

Project aims To investigate three approaches to improving reading comprehension skills in poor comprehenders. To compare these approaches to existing classroom practice by monitoring the performance of an untreated waiting control group. To address the objectives of the primary framework (NLS) and equip teaching assistants with a wide range of skills and materials, useful in supporting children with reading comprehension and oral language difficulties. Oral Language (OL) Text Comprehension (TC) Combined (COM)

Oral Language Programme Text Comprehension Programme The intervention programmes Oral Language Programme Vocabulary Listening Comprehension Figurative Language Spoken Narrative Text Comprehension Programme Metacognitive Strategies Reading Comprehension Inferencing from Text Written Narrative Combined Programme All eight components connecting oral language and text-based activities in an integrated and naturalistic approach. All sessions contained both reading and listening comprehension to support complementary components. Opportunities for children to encounter new vocabulary/idioms/inferences in both written and spoken language.

Session structure Introduction 3 mins Vocabulary 5 mins Activity Approx. time per session Introduction 3 mins Vocabulary 5 mins Listening comprehension 7 mins Figurative language Spoken narrative Plenary Activity Approx. time per session Introduction 3 mins Metacognitive strategies 5 mins Reading comprehension 7 mins Inferencing from text Written narrative Plenary

TC programme contents 1. Metacognitive Strategies (Cain, 1999) Re-read (Garner, et al., 1984) Look-back (Garner, 1982) Think aloud (Farr & Connor, 2004) Mental imagery (Oakhill & Patel, 1991) Explain & reflect (McNamara, 2004) 2. Reading Comprehension (Reciprocal Teaching - Palinscar & Brown,1984,1985, Palinscar,1986) Clarification Summarisation Prediction Question generation 3. Inferencing from Text (Yuill & Oakhill, 1988) Lexical inferencing Bridging inferencing Elaborative inferencing Guessing missing information (Yuill & Joselyne, 1988) Evaluative inferencing 4. Written Narrative Story structure (Beck & McKeown, 1981; Pearson, 1982; Idol & Croll, 1987) Sequencing Story production

OL programme contents 1. Vocabulary (Multiple Context Learning, Beck, McKeown & Kucan, 2002) Graphic Organisers (Nash & Snowling, 2006) Verbal Reasoning Mnemonic Strategies (Levin, 1993; Peters & Levin, 1986; Graves & Levin, 1989) Illustrations 2. Listening Comprehension (Reciprocal Teaching - Palinscar & Brown,1985; Palinscar,1986) Clarification Summarisation Prediction Question generation 3. Figurative Language Idioms (Legler, 1991) Riddles (Yuill, 1988) Jokes (Yuill, in press) Similes and metaphors 4. Spoken Narrative Story structure (Beck & McKeown, 1981; Pearson, 1982; Idol & Croll, 1987) Sequencing Story production

Phases of the project Oct 2006 8-9 years Oct 2007 9-10 years Oct 2008 Screening Intervention development T1 assessment T2 assessment T3 assessment T4 assessment Control Block 1 Control Block 2 Block 1 Block 2 Oct 2006 8-9 years Oct 2007 9-10 years Oct 2008 10-11 years Oct 2009

Participant flow Not meeting inclusion criteria (n = 634) Excluded (n = 45) 2 schools excluded due to low numbers of children showing poor comprehender profile Absent for key measures (n=13) Refused to be assessed (n = 2) Group Screening (Oct-Dec 2006) Y4 children in 23 schools in York & N.Yorks; eligible for assessment (n = 1120) Group assessments: Listening Comp (n = 1042); Ravens (n = 1054); Spelling (n = 1045); Numerical ops (n = 1050) Complete data on key measures (n= 977) Individual Screening (Jan-Feb 2007) 21 schools; eligible for assessment (n = 296) Individual assessments: NARA reading comp (n = 284); TOWRE (n = 282); WASI Verbal IQ (n = 277) Complete data on key measures (n= 282) Not meeting inclusion criteria (n = 114) Excluded on behavioural grounds (n =1) Declined to participate in intervention (n = 7) Selection & Pre-test (Feb-Apr 2007) 21 schools Selected for intervention (n =168)

Eligibility criteria Eligibility criteria Sample size Discrepancy in standard score points between NARA II reading comprehension and TOWRE real word reading efficiency. NARA II reading accuracy standard scores of 85 and above. NARA II reading comprehension scores of 105 and below. Age appropriate spelling ability. Non-Verbal IQ of 85 or above. Of these children, we selected eight children within each school with the greatest discrepancies. Sample size Power calculations for this study were carried out using estimates of effect sizes from previous studies of a similar nature.    A sample size of 40 children in each arm of the intervention provides 90% power to detect a difference of 0.6 standard deviations between the control group and the three intervention groups combined (with alpha = 0.05 two-tailed).

Selection & Pre-test (Feb-Apr 2007) Participant flow Selection & Pre-test (Feb-Apr 2007) 21 schools Selected for intervention (n =168) Random allocation to intervention groups (Apr-Jul 2007) 20 schools Block 1 of intervention programmes (n = 157) Mid-Intervention Assessments (n = 159) OL programme Took part in block 1 of intervention (n = 39) TC programme Took part in block 1 of intervention (n = 40) COM programme Took part in block 1 of intervention (n = 39) Waiting Control Took part in block 1 of intervention (n = 39) Intervention: Excluded (n = 8) 1 school withdrawn from project Declined to participate in intervention (n = 1) Moved school (n = 2) OL programme Took part in block 2 of intervention (n = 38) Post-intervention assessments (n = 38) TC programme Took part in block 2 of intervention (n = 40) Post-intervention assessments (n = 40) COM programme Took part in block 2 of intervention (n = 38) Post-intervention assessments (n = 38) Waiting Control Took part in block 2 of intervention (n = 39) Post-intervention assessments (n = 39) (Sept-Dec 2007) Block 2 of intervention programmes (n = 155) Post-Intervention Assessment (n = 159) Intervention: Moved school (n = 2) Post-intervention Assessments (wave 1) (n=155) In new schools (n = 4)

Baseline characteristics of each group Whole sample TOWRE Real words 110.48 108.88 108.30 108.70 109.09 TOWRE Nonwords 106.45 106.33 109.13 106.18 107.02 TOWRE Total 110.20 109.18 108.85 109.68 NARA II Reading accuracy 102.93 104.33 101.55 NARA II Reading comprehension 92.95 93.36 92.82 92.99 WIAT II Reading comprehension 94.31 95.43 96.38 97.77 95.96 CELF Listening comprehension 78.75* 80.42 87.36* 83.06 82.40 WASI Vocabulary (t score) 38.31 40.32 39.44 38.26 39.07 WASI Similarities (t score) 47.40 49.05 49.97 49.49 48.97 WASI VIQ 89.41 92.03 91.89 91.00 91.07 Ravens Matrices NVIQ 99.26 100.23 100.66 100.78 WMTBC Listening recall 94.50 94.19 93.08 92.19 93.49 WOND Numerical operations 97.53 98.98 97.59 95.30 97.35 *Statistically significant between groups difference (p<0.05)

Intervention delivery Two 10-week blocks of intensive teaching in individual and pair sessions. Each session is 30 mins. Children receive 2 pair sessions and 1 individual session per week (1½ hours per week). Teaching took place in designated areas within school (small classrooms/meeting rooms etc.). Teaching times varied depending upon existing timetabled commitments. Monday Tuesday Wednesday Thursday Friday Pair OL (ab) Pair TC (cd) Pair COM (ef) Individual OL (a) Individual OL (b) Individual TC (c) Individual TC (d) Individual COM (e) Individual COM (f) Tutorial (2 hrs every alternate week) Prep time (2 hrs per week) 1.5 hours 3 hours per week

Treatment fidelity Manuals Detailed, prescriptive manual and pre-prepared worksheets, readers and resources Training Training took place over 3.5 days. Delivered by the research team. Fortnightly tutorials Opportunity to monitor delivery of programmes by discussing experiences, ideas and observations. Some sessions took the form of top up training in which we focused on particular components of the programmes. Observations Each TA was observed by a member of the research team at least twice in each intervention block. Careful records were kept and onsite feedback and support was given. Filmed sessions Five TAs gave us permission to film teaching sessions.

Primary outcome measures WIAT II Reading Comprehension Children read (aloud or silently) a range of passages and sentences (narrative, adverts, non-fiction information etc.) Includes literal, inference and vocabulary dependent question types. NARA II Reading Comprehension (Form 2 at pretest, Form 1 at post test) Children read aloud short passages then answer a range of literal and inferential open ended questions. TORCH Reading Comprehension Silent reading. Comprehension assessed using a cloze procedure. Responses require a range of skills including inferencing and vocabulary knowledge.

Secondary measures Measure Corresponding Component Measure Measure TOLC Inferencing Inferencing from text CMS Story Recall Listening Comprehension WASI Vocabulary Vocabulary WASI Verbal Similarities CELF Listening to Paragraphs Measure WMTBC Listening Recall Complex Working Memory NARA II Passage Reading Accuracy TOWRE Reading Efficiency WOND Numerical Operations CONTROL TASK Myself as a Learner Self Esteem Strengths and Difficulties Questionnaire Teacher report of behaviour Measure Corresponding Component Story Writing Written Narrative Beaver & Hedgehog Stories Metacognitive Strategies Reading Attitudes Questionnaire Reading Comprehension Storytelling Spoken Narrative Vocabulary (target & non-target) Vocabulary Idioms (target & non-target) Figurative Language

Data collection & analysis T1 data was collected in school by the research team (blind) and the trained teaching assistants (blind at this point). T2 & T3 data was collected by the research team (blind). Testing conditions varied across schools. Some assessments were individually administered, others were group administered. All score sheets analysed blind. 10% double marking for reliability where necessary. Missing data estimated using data imputation. Regression based approach used, controlling for performance at T1. Report 95% robust confidence intervals. Describe the process

Reading comprehension WIAT II NARA II

Listening comprehension CELF Listening to paragraphs

Vocabulary p<0.001 p<0.01 p<0.001 Taught Non taught

Verbal IQ WASI Vocabulary WASI Verbal Similarities WASI Verbal IQ p<0.05 WASI Vocabulary WASI Verbal Similarities p<0.05 WASI Verbal IQ

Points to consider The groups do not significantly differ in terms of changes in performance on the control task (Numerical Operations). Additional bespoke and standardised measures are currently being analysed (e.g. narrative, comprehension monitoring). How do we interpret the improvements in verbal IQ? Might certain core skills be better suited to different teaching approaches? Further exploration of how the different standardised measures of comprehension correspond to the content of the programmes is required.

Conclusions What’s next? 20-week intervention programmes can produce significant gains in both expressive vocabulary and reading comprehension in poor comprehenders. Importantly these gains are relative to an untreated waiting control group. What’s next? Further data collection at T4 to examine maintenance effects. Statistical analyses to examine subgroups response to intervention. Investigate whether expressive vocabulary improvement is a mediating factor in reading comprehension gains. Development of waiting control group intervention.

Research Team Many thanks to: Liaison Group Teaching Assistants Schools Children Many thanks to: pjc118@york.ac.uk Contact: