Building a Knowledge Base for Teaching Adult Decoding Richard Venezky, Principal Investigator Deborah Knight, Co-Investigator University of Delaware Judith A. Alamprese, Co-Principal Investigator Fumiyo Tao, Field Study Director Abt Associates Inc. Bethesda, MD

1 Studys Research Questions How does the use of enriched decoding strategies affect adults development of decoding skills? What is the relationship between adults development of decoding skills and their development of fluency and comprehension? How are adults background characteristics and their amount of participation in reading instruction related to the improvement of their reading skills? How is the type of reading instruction that adults receive related to their improvement of reading skills?

2 Overview of Study Phases of study activities Intervention conceptualization (Year 1) Design Studies (Years 1-2) Field Study (Years 2-4) Analysis and report preparation (Year 5) Experimental design Naturalistic study in operating adult basic education programs Approach of use-inspired basic research (Stokes, 1997) Target population: adults with 4-7 th grade-level equivalent in comprehension

Program Management Program Recruitment and Intake Support Services Program Improvement AssessmentProgram Articulation Independent Variables: Programmatic Characteristics: Overall Program Operations and Instruction Overall Program Operations Instruction Mediating Variables: Learner Characteristics Learners Background Demographic characteristics Educational and training experience Work experience Health background Learner Persistence Total participation hours Duration of participation Learner Development Self empowerment enhancement Dependent Variables: Learner Outcomes Short-Term (3-12 months) Changes in learning Personal growth Program Management Program Recruitment and Intake Support Services Program Improvement AssessmentProgram Articulation Independent Variables: Programmatic Characteristics: Overall Program Operations and Instruction Overall Program Operations Instruction Program Management Program Recruitment and Intake Support Services Program Improvement AssessmentProgram Articulation Program Management Program Management Program Recruitment and Intake Program Recruitment and Intake Support Services Support Services Program Improvement Program Improvement Assessment Program Articulation Learner Articulation Independent Variables: Programmatic Characteristics: Overall Program Operations and Instruction Overall Program Operations Instruction Independent Variables: Programmatic Characteristics: Overall Program Operations and Instruction Overall Program Operations Instruction Mediating Variables: Learner Characteristics Learners Background Demographic characteristics Educational and training experience Work experience Health background Learner Persistence Total participation hours Duration of participation Learner Development Self empowerment enhancement Mediating Variables: Learner Characteristics Learners Background Demographic characteristics Educational and training experience Work experience Health background Learners Background Demographic characteristics Educational and training experience Work experience Health background Learner Persistence Total participation hours Duration of participation Learner Persistence Total participation hours Duration of participation Learner Development Self empowerment enhancement Learner Development Self empowerment enhancement Dependent Variables: Learner Outcomes Short-Term (3-12 months) Changes in learning Personal growth Dependent Variables: Learner Outcomes Short-Term (3-12 months) Changes in learning Personal growth Short-Term (6-12 months) Changes in reading skills Personal growth Framework for Study Design 3

Assess enriched decoding instruction compared to existing approaches and to comparison group of structured reading classes Sample of 21 ABE programs, 30 reading classes, 600 learners Randomly assign 7 programs to decoding intervention and 7 to control group Train treatment instructors in decoding intervention Data collection: 2 years for treatment group 2 years for control group 3 years for comparison group Conduct Field Study Study Design and Phases Review K-12 and adult reading literature Examine results from Abt Associates reading study Identify Decoding Interventions Identify types of decoding methods that might be tested in Design Studies Generate hypotheses about role of decoding in learning to read and test in situ; vary aspects of instruction Conduct 4 cycles of testing interventions in 4 classes Collect data over 2 program years Refine reading intervention based on results of each cycle Conduct Design Studies 4

5 Design Studies Purpose Generate hypotheses about role of decoding in adults learning to read and test them in situ, so that practices can be implemented in actual instructional contexts Systematically vary aspects of adult decoding instruction to determine the most effective practices for the field study

6 Design Studies Questions Guiding Design Studies How do rule-based (deductive/abstract) and exemplar- based (inductive/analogical) instructional methods affect adult learners development of decoding skills? How much and which metalinguistic knowledge enhances adult learning? Does reading fluency increase differentially if adult learners participate in repeated readings that emphasize speed versus accuracy? Do adult learners decoding skills improve when they are given spelling practice (investigation in progress)?

7 Design Studies Sample and Methodology Sample Four ABE classes Three ABE instructors, 2 university researchers 26 adult learners Gender: 17 females, 9 males Race/Ethnicity: 13 African Americans, 9 white non-Hispanic, 1 white Hispanic, 2 Asian, 1 mixed race Age: Mean=25 years Methodology 4 cycles of intervention studies focused on specific questions during each cycle Continued refinement of instruction through each cycle based on results of previous cycle and new question

8 Design Studies Features of Interventions Examined in Design Studies Learner outcomes (determined by embedded assessments, observations) Classroom practicality Learner engagement Instructor responsiveness and effectiveness

9 Design Studies Findings from Design Studies To Date Learners benefit from a combination of rule-based and exemplar approaches Learners are interested in and can apply metalinguistic information about orthographic patterns and etymology, but this information needs to be presented clearly with a minimum of complexity When asked to read for speed and accuracy, adult learners sacrifice accuracy for speed if they are timing each other using a stopwatch Adult learners are motivated to learn to spell, and often believe that they can read words better than they actually do Instructors will use scripted reading lessons

10 Intervention Curriculum Based on findings from Design Studies, the intervention curriculum will: Consist of 40 lessons, including 10 review; each lesson will be 35 minutes (integrated into a reading class) and include a mixture of reading and spelling patterns Emphasize metalinguistic knowledge that is of high utility or that explains why English behaves the way it does Include spelling tips Include a component designed to improve accuracy and speed of reading Embed assessment in the lessons (except review) for monitoring learner progress

11 Intervention Curriculum The intervention curriculum will: Include periodic optional practice lessons that do not introduce new concepts so that differentiated instruction can be offered Include an instructors guide with scripted lessons

12 Intervention Curriculum Format of Lessons Review of prior learning Use of an authentic document Introduction of new material in a mini-lesson Use of guided practice reading and spelling words Exercises in fluency/accuracy and/or spelling Independent practice/embedded assessment

13 Field Study Purpose: Conduct an experiment in situ of the effectiveness of the intervention curriculum in improving adult learners reading skills Sample 21 ABE programs, approximately 30 ABE reading classes and 27 instructors; approximately 600 adult learners Program meets a basic level of operational planning, organization, and structure 13

14 Field Study Sample 7 comparison ABE programs that use commercially produced, structured reading programs 7 ABE programs randomly assigned to treatment group (enriched decoding); instructors will be trained to use intervention curriculum 7 ABE programs randomly assigned to control group; instructors will conduct reading classes as they usually do

15 Field Study Measures: Adult Learner Decoding, spelling, vocabulary, and comprehension tests: WRAT-3 (Word Reading and Word Spelling), Woodcock- Johnson (Word Attack, Letter-Word Identification), Nelson Reading Comprehension and Word Meaning Tests Reading fluency measures: TOWRE (Sight Word Efficiency, Phonemic Reading Efficiency) NAAL Passage Reading Background Interview: demographic characteristics, education and training, work experience, health status Self-empowerment scales: Abts Adaptation of the General Self-Efficacy Scale and Rosenbergs Self-Esteem Scale

16 Field Study Measures: Adult Learner Attendance: hours and weeks per program year; participation in other ABE classes during same period Measures: Instructor Reading Instructional Content, Strategies, and Materials: Abt Associates Class Observation Instrument Organization of Class and Class Environment: Instructor interview Instructor Background: Instructor interview regarding demographic characteristics, training and experience in reading instruction

17 Field Study Measures: Program ABE Program Operations: ABE Program Rating Form, Abt Associates Inc., which measures all components of program functioning Program Characteristics: size, organizational structure, funding, number of staff, other characteristics

18 Field Study Data Collection Design Learner: Pre-post design--fall (pre) and spring (post) of each year of data collection Instructor: Treatment instructors: multiple observations and documentation of teaching; interview after each observation Control and comparison instructors: one observation per year; interview after each observation Program: pre-post site visit (pre during year 1 of data collection, post during year 2 of data collection)

19 Field Study Progress to Date and Schedule Comparison Group (7 programs participating): Cohort 1 (47 learners) pre-tested; post-test in May 2004 Year 1 class observations and interviews with program staff completed in May 2004 Treatment and Control Groups (14 programs): Majority of programs recruited Random assignment in June Treatment instructors will be trained in intervention curriculum during the summer Data collectors will be trained during late summer