Chapter 5 Informal Assessment: Progress Monitoring

Introduction The Assessment Continuum Reliability and Validity Criterion-Referenced Assessment Progress Monitoring Computer-based Assessment of Reading Special Considerations for Young, Adult, and English Language Learners

The Assessment Continuum

Figure 5.1

Reliability and Validity

Types of Reliability and Validity Tables 5.1 and 5.2

Reliability Reliability = Consistency Correlation: Positive, Negative, No Relationship Correlation Coefficient Ranges from to –1.00 Measures of Reliability: Pearson’s R, Kuder-Richardson, Split-Half, Coefficient Alpha, Test-Retest, Equivalent Forms, Interrater Standard Error of Measure Obtained Score = True Score+Error Estimated True Score

Types of Reliability Inter-Rater % age agreement Intra-Rater % age agreement Alternate Form Pearson r Internal Consistency Spearman Brown, Cronbach’s alpha Test-Retest Pearson r Consistency between 2 or more raters 1 rater’s consistency across assessments Consistency of scores over equivalent forms Consistency of scores on items within same test Consistency of scores from one testing to another

Validity Validity = Accuracy Types of Validity: Criterion-Related, Concurrent, Predictive, Content, Construct Reliability versus Validity Can a Test Be Valid without Being Reliable?

Types of Validity Content Expert review, test blueprint (Figure 5.2) Construct Correlations with similar measures Concurrent vs. Predictive Treatment Prediction of student learning gains How well do items assess desired content? How well do items assess a particular domain, skill, etc.? Same time vs. future Analysis of gain scores after instruction

Criterion-Referenced Assessment

Assess mastery of specific objectives Determine present levels of performance Establish future educational objective Determine IEP objectives Monitor progress

Brigance Inventories Brigance Diagnostic Inventory of Early Development-III Brigance Diagnostic Comprehensive Inventory of Basic Skills-III Criterion-referenced but yields some standard scores Gives grade placement information Assesses a wide range of reading and other skills Can be used to monitor progress, set IEP goals

Linkages Between Brigance and Common Core State Standards Table 5.3

Examples from Brigance CIBS-III Figures 5.3, 5.4, 5.5

Figure 5.3

Figure 5.4

Figure 5.5

Progress Monitoring

Trends in Progress Monitoring National Center on Student Progress Monitoring National Center on Response to Intervention National Center on Intensive Intervention Hybrids Computerized Assessment

“ Hybrid” Assessments Assessments that combine aspects of both formal and informal assessment Standardization sample (Formal) Curriculum-like tasks (Informal) Ability to monitor progress (Informal) Dynamic Indicators of Basic Early Literacy Skills (DIBELS) & AIMSWeb progress monitoring system

AIMSweb Reading Fluency Comprehension Early Literacy Measures Spanish Early Literacy Early Numeracy Math Computation Spelling Written Expression

AIMSweb Reading Reading Fluency Measures Oral Reading Fluency Reading Comprehension (MAZE) Measures Reading Comprehension

AIMSweb Early Literacy Measures Letter Naming Letter Sound Fluency Phoneme Segmentation Nonsense Word Fluency

AIMSweb Spanish Early Literacy Measures Letter Naming Fluency Letter Sound Fluency Syllable Sound Fluency Syllable Reading Fluency Syllable and Word Spelling Oral Reading Fluency

Example: DIBELS (Good & Kaminski, 2002) Brief, standardized, individually administered measures Initial/First Sound Fluency- Oral Reading Fluency Letter Naming Fluency- Oral Retelling Fluency Phoneme Segment. Fluency- Word Use Fluency Nonsense Word Fluency Daze (DIBELS Next only) Assess development of pre-reading and early reading skills Includes DIBELS Data System

What DIBELS Purports to Assess Area of Reading Phonics Phonemic Awareness Fluency Comprehension Vocabulary DIBELS Assessment Initial Sound/First Sound Phoneme Segmentation Nonsense Word Letter Naming Oral Reading Retell Daze (DIBELS Next) Word Use

Initial Sounds Fluency (First Sound Fluency in DIBELS Next) Grade administered: Kindergarten Measures: A student’s ability to recognize and produce the beginning sound(s) in an orally presented word Students performing below expectations will benefit from instruction in phonological awareness. Overview of DIBELS Measures

Le tter Naming Fluency Grades administered: Kindergarten through fall of first grade Measures: A student’s proficiency in naming upper and lower case letters Students identified at risk should be instructed in phonological awareness and alphabetic principle

Phoneme Segmentation Fluency Grades Administered: Mid-kindergarten to end of first grade Measures: A student’s ability to segment three- and four-phoneme words into their individual phonemes Students performing below expectations in this measure will benefit from instruction in phonological awareness activities

Overview of DIBELS Measures Nonsense Word Fluency Grades administered: Mid-kindergarten through first grade Measures: The student’s knowledge of letter-sound correspondence and ability to blend letters into words Students performing below expectations will benefit from activities focusing on decoding

Oral Reading Fluency Grades administered: Mid-year first grade through third grade Measures: fluency in oral reading of connected text Students at risk will often have difficulty in comprehension

Overview of DIBELS Measures Oral Retelling Fluency Grades administered: Mid-year first to third grade Measures: Comprehension of the passage read orally Children generally use about half the number of words in their retell that they read aloud in a 1-minute passage

Overview of DIBELS Measures Word Use Fluency Grades Administered: Kindergarten through third grade Measures: Vocabulary knowledge and expressive language

Overview of DIBELS Measures Daze Grades Administered: Third through sixth grade Measures: Comprehension

Strengths and Weaknesses Strengths Progress monitoring Assesses all NRP areas Relatively brief Weaknesses Pressley, Hilden, & Shankland (2005) Predictive Power Emphasis on speed vs. comprehension Cost The Truth About DIBELS: What It Is, What It Does (Goodman, 2006) “Barking at print?” (Samuels, 2007)

Computer-Based Assessment of Reading

Computerized Assessment Star Reading Linked to Accelerated Reader Read DORA and DOMA Total Reader

Lexiles Computer-generated Sophisticated but user-friendly system to track student progress over time and to link students to appropriate level texts Ranges from 200 (beginning reading) to (advanced readers) Students and books can be matched along the continuum of Lexile scores Common Core State Standard Lexile bands define expected levels (in Lexile scores) of reading for each grade level

Special Considerations for Young, Adult, and English Language Learners

Special Considerations in the Informal Assessment of Young Children May want to use portfolios to keep track of student progress Eight essential literacy skills for Kindergarteners Letter/sound associations Phonological awareness skills Print concepts Phonemic segmentation/representation Word reading Word writing Text reading Text writing

Special Considerations in the Informal Assessment of Adults Text Box 5.1 Adults learning to read are typically sensitive, and in some cases, ashamed. Take time to establish rapport Assessment materials should be based on adult themes (e.g., work, family life). Assess various areas of reading, rather than assuming possession of basic word analysis skills. Assess oral reading rate; if 125 wept or less, further assessment is needed. Use stronger vocabulary acquired through life experiences as a building block of reading.

Special Considerations in the Informal Assessment of English Language Learners Text Box 5.2 Four categories of ELLs Newly arrived students with limited formal schooling Newly arrived students with adequate formal schooling Students exposed to two languages simultaneously Long-term English-language learners Assessments should match the literacy background and knowledge base of the student. Ensure assessments conform to the general assessment standards set by the Teachers of English to Speakers of Other Languages (TESOL;

Assessment at a Glance: Criterion Referenced and Progress Monitoring Tables 5.5 and 5.6 Characteristics of Informal Assessment and Progress Monitoring Measures Psychometric Properties of Informal Assessment and Progress Monitoring Measures

Summary The Assessment Continuum Reliability and Validity Criterion-Referenced Assessment Progress Monitoring Computer-based Assessment of Reading Special Considerations for Young, Adult, and English Language Learners