Woodcock Johnson III Tests of Achievement Transition Academy @ Shadd October 5, 2010 Woodcock Johnson III Tests of Achievement (Adapted from Riverside Publishing Materials)

Purpose of Academic Testing Identify areas of academic strength and weakness Diagnose specific learning disabilities Develop recommendations to address academic deficiencies

Learning Disabilities Two operational definitions: Significant discrepancy between ability and achievement Response to intervention Different from DSM-IV-TR Learning Disorder, yet related.

Issues with Academic Testing Age norms vs. Grade norms Kids enter school at different ages Children can be accelerated or retained in grade Which normative sample to use? If you are computing ability-achievement discrepancies, you will need to use age norms. However, if the student has been retained or accelerated in grade, should also re-compute scores using grade norms.

Psychometrics of WJ-III ACH Major strength of the WJ-III ACH test battery – co-normed with the WJ-III COG test battery. Standardization sample: 8,818 subjects 100+ geographic areas (including Moorpark!) Reliability: For individual subtests across all ages: .70 to .96 For clusters across all ages: .93 to .99 Alternative form reliabilities for Passage Comprehension range between .85 and .96 over all ages. Lowest reliability: Story Recall (.70) Highest reliability: Applied Problems (.96); Letter-Word ID (.95); Math Fluency (.95); Spelling (.95) Writing clusters lowest at .93; Total Achievement highest at .99

Psychometrics of WJ-III ACH Validity Differential correlational patterns between different cluster scores suggest discriminant validity. Factor structure aligns with the narrow Cattell-Horn-Carroll abilities Validity studies with the KTEA and WIAT show modest agreement Total Achievement on WJ-III with KTEA: .79 Total Achievement on WJ-III with WIAT: .65 Convergent and Discriminant validity coefficients are a little higher e.g.: WJ-III Broad Reading with WIAT Reading Composite: .82 WJ-III Broad Reading with WIAT Math Composite: .50

Psychometrics of WJ-III ACH Discriminant validity with WJ-III COG, WAIS-III, and Kaufman Adolescent & Adult Intelligence testing produces lower coefficients (most below .50).

Organization Achievement (BLUE) Standard Battery 12 Tests 10 Clusters Extended Battery 10 Tests 9 Clusters (Form A & B)

Clusters Broad Reading Basic Reading Skills Broad Math Math Calculation Skills Math Reasoning Broad Written Language Written Expression Academic Knowledge Academic Skills Academic Fluency Oral Language Each cluster is made up of at least two subtests; some require supplemental subtests Broad Reading: Reading decoding, reading speed, and using syntactic and semantic cueing systems when reading for meaning. Basic Reading: Sign vocabulary, phonics, and structural analysis skills. Broad Math: Math achievement including problem solving, number facility, automaticity with facts, and reasoning. Math Calculation Skills: Computational Skills and auotmaticity with math facts. Math Reasoning: Problem solving, concepts, and math vocabulary. Broad Written Language: Spelling, writing rate, and written expression. Written Expression: Quality of written sentences and fluency of production. Academic Knowledge: Knowledge of science, social studies, and humanities (really, one subtest) Academic Skills: Basic academic skills. Academic Fluency: Ease and speed by which an individual performs simple to more complex academic tasks. Oral Language: Linguistic competency, listening ability, oral comprehension (and attention).

Cluster and Subtest Structure Broad Reading Letter-Word Identification Reading Fluency Passage Comprehension Basic Reading Skills Word Attack (extended battery, commonly administered) Broad Math Calculation Math Fluency Applied Problems

Cluster and Subtest Structure Math Calculation Skills Calculation Math Fluency Math Reasoning Applied Problems Quantitative Concepts (extended battery) Broad Written Language Spelling Writing Fluency Writing Samples

Cluster and Subtest Structure Written Expression Writing Fluency Writing Samples Academic Knowledge Academic knowledge (3 extended subtests: Sci, SS, Hum) Academic Skills Letter-Word Identification Spelling Calculation

Cluster and Subtest Structure Academic Fluency Reading Fluency Writing Fluency Math Fluency Oral Language Story Recall Understanding Directions

Cluster and Subtest Structure Total Achievement Letter-Word Identification Reading Fluency Passage Comprehension Spelling Writing Fluency Writing Samples Calculation Math Fluency Applied Problems

“IQ tests are a poor way to judge people's abilities, failing as they do to account for magic, which has its own importance, both by itself and as a compliment to logic.” - Robert James Waller

Test Administration: Tools & Techniques Strive for exact & brisk administration Have necessary materials Test books Test records Subject response booklet Tapes Tape player Headphones, Y-jack, Earpiece Pencils Stopwatch/Timer

Administration Tool: Test Session Observation Checklist Level of conversational proficiency Level of cooperation Level of activity Attention & concentration Self-confidence Care in responding Response to difficult tasks

Administration Time (ACH) Preliminary Testing at BC Intake Booklet & Ach Tests 2, 5, 6, 7, 8, 11 Approx. 120 minutes Standard Achievement Tests 1, 3, 4, 9, 10 60 minutes Extended Achievement Tests 13 - 18

Standard Battery Letter-Word Identification Reading Fluency (timed) Story Recall (taped) Understanding Directions (taped) Calculation Math Fluency (timed) Spelling Writing Fluency (timed) Passage Comprehension Applied Problems Writing Samples Story Recall – Delayed

Extended Battery Word Attack Picture Vocabulary Oral Comprehension (taped) Editing Reading Vocabulary Quantitative Concepts Academic Knowledge Spelling of Sounds Sound Awareness Punctuation & Capitalization

Administration Techniques Know your tools Follow standardized procedures Use suggested starting points Know basal and ceiling rules Know pronunciations Don’t penalize mispronunciations due to speech deficits or regional differences Student must smoothly pronounce words for credit (automaticity is critical)

“If you think you are over your head with a client, you probably are “If you think you are over your head with a client, you probably are.” - Richard Bolles

Administration Techniques Test by complete pages when subject sees items Score LAST response given Give benefit of doubt every other time Don’t repeat items unless directed Use pause button if needed If interrupted by a sound, repeat item later

Administration Techniques Adhere to time limits Record exact finishing times On timed written tests, direct student to cross out errors Check questionable responses later; proceed as if correct

“We never do anything well till we cease to think about the manner of doing it.” - William Hazlitt

“The IQ and the life expectancy of the average American recently passed each other going in the opposite direction.” - George Carlin

Standard Battery & Extended Battery Tests of Achievement Standard Battery & Extended Battery

Tests of Achievement Standard Battery Extended Battery Letter-Word Identification Reading Fluency Story Recall Understanding Directions Calculation Math Fluency Spelling Writing Fluency Passage Comprehension Applied Problems Writing Samples Story Recall – Delayed Extended Battery Word Attack Picture Vocabulary Oral Comprehension Editing Reading Vocabulary Quantitative Concepts Academic Knowledge Spelling of Sounds Sound Awareness Punctuation & Capitalization

Standard Battery Test 1: Letter-Word Identification Identifying printed letters and words Test 2: Reading Fluency Reading printed statements rapidly and responding true or false (Y or N) Test 3: Story Recall Listening to and recalling details of stories Test 4: Understanding Directions Listening to a sequence of instructions and then following the directions (by pointing) Test 5: Calculation Performing various mathematical calculations; retrieving math facts.

Standard Battery Test 6: Math Fluency Test 7: Spelling Adding, subtracting, and multiplying rapidly Test 7: Spelling Spelling orally presented words Test 8: Writing Fluency Formulating and writing simple sentences rapidly with a visual cue Test 9: Passage Comprehension Completing a sentence by giving the missing key word that makes sense in the context. Test 10: Applied Problems Performing math calculations in response to orally presented problems with a visual cue.

Standard Battery Test 11: Writing Samples Writing meaningful sentences for a given purpose with a visual and oral prompt. Test 12: Story Recall – Delayed Recalling previously presented story elements Test 13: Word Attack EXTENDED Battery test, often included in most standard battery administrations Reading phonically regular non-words

Extended Battery Test 13: Word Attack Reading phonically regular non-words Test 14: Picture Vocabulary Naming pictures Test 15: Oral Comprehension Completing an oral sentence by giving the missing key word that makes sense in the context Test 16: Editing Identifying and correcting errors in written passages

Extended Battery Test 17: Reading Vocabulary Reading words and supplying synonyms and antonyms; reading and completing verbal analogies Test 18: Quantitative Concepts Identifying math terms and formulae; identifying number patterns Test 19: Academic Knowledge Responding to questions about science, social studies, and humanities Test 20: Spelling of Sounds Spelling letter combinations that are regular patterns in written English

Extended Battery Test 21: Sound Awareness Providing rhyming words; deleting, substituting, and reversing parts of words to make new words. Test 22: Punctuation & Capitalization Applying punctuation and capitalization rules

“Sometimes when I'm in a bookstore or a library, I am overwhelmed by all the things that I do not know.” - Arthur C. Clarke

Basal and Ceiling Rules For almost all tests (exceptions are noted in the scoring record sheets), to obtain a valid score on any subtest, you need a basal level of 6 items correct and a ceiling level of 6 items incorrect. Add up the total number of items correct (including items BELOW basal) = Raw Score. Enter Raw Scores in the WJ-III Scoring Assistant (hand scoring not really an option).

Scoring Output W-scores Equal-interval scale where 500 is set to approximate the average performance of a 10-0 child. Used to measure growth. Age Equivalents / Grade Equivalents Raw score cross-referenced to the median score within the age/grade band. Highly variable. RPI – Relative Proficiency Index Predicts a student’s level of proficiency on tasks that typical age- or grade-level peers would perform with 90% proficiency. RPI of 55/90 indicates impairment – they did 55%, a peer would do 90% RPI of 98/90 indicates strength – they did 98%, a peer would do 90%

Scoring Output Percentile Ranks Standard Scores Z Scores Mean 100; SD 15 Used in determining ability-achievement discrepancies. Z Scores

“Your mind should be a place where you work things out, not store a bunch of stuff.” - Albert Einstein

1. Letter-Word Identification Must be fluidly pronounced Basal/Ceiling = 6 lowest correct / 6 highest correct Know exact pronunciation of words Used in Broad Reading, Basic Reading Skills, Academic Skills, Total Achievement

2. Reading Fluency  Rapidly read simple sentences and mark whether it is true or false Start with Item 1 3 minute time limit Skipped items are scored “0” If fewer than 3 correct on Practice Exercises C-F, discontinue and score “0” Enter Number Correct and Number Incorrect Used in Broad Reading, Academic Fluency, Total Achievement

3. Story Recall Use tape Student listens to story on tape, then retells it Must repeat Bold elements exactly; other parts may be paraphrased (in any order) Use suggested starting points Follow continuation rules Pause tape player after each story Do not repeat or replay any stories Used in Oral Language (Std & Ext), Oral Expression

4. Understanding Directions  Use Tape; pause tape if needed Student points to items on picture according to taped instructions Allow student 10 seconds to view each picture before starting tape Use suggested starting points; follow discontinuation instructions Lay Test Manual flat Tip: Only watch for correct answers – don’t try to figure it out Used in Oral Language (Std & Ext), Listening Comprehension

5. Calculation Untimed Don’t point out signs Basal/Ceiling = 6 lowest correct/6 highest incorrect Accept reversals but not transpositions Used in Broad Math, Math Calculation Skills, Academic Skills, Total Achievement

6. Math Fluency  Timed Math facts Begin with Item 1 3 minutes timed Do not point out signs Lay booklet open Discontinue if 3 or fewer correct in 1 minute Used in Broad Math, Math Calculation Skills, Academic Fluency, Total Achievement

7. Spelling () Spelling (not usage & punctuation) Basal/Ceiling = 6/6 Used in Broad Written Language, Basic Writing Skills, Academic Skills, Total Achievement

8. Writing Fluency  Student quickly writes simple sentences using prompt words Begin with Item 1 7 Minutes timed Discontinue if “0” correct on Samples or fewer than 3 correct in first 2 minutes Read stimulus words as needed Stimulus words cannot be altered Used in Broad Written Language, Written Expression, Academic Fluency, Total Achievement

9. Passage Comprehension No changes from WJ-R Basal/Ceiling = 6/6 Accept responses that differ in number or tense, but not from different parts of speech Used in Broad Reading, Reading Comprehension, Academic Applications, Total Achievement

10. Applied Problems Read all blue prompts to student Basal/Ceiling = 6/6 Used in Broad Math, Math Reasoning, Academic Applications, Total Achievement

11. Writing Samples () Student writes complete sentences to prompts which increase in difficulty Administer entire block of items Administer additional blocks of items if score falls in shaded area of scoring table Examiner may use 2, 1.5, 1, .5 or 0 for each item Used in Broad Written Language, Written Expression, Academic Applications, Total Achievement

Tests of Achievement: Extended Battery Word Attack Picture Vocabulary Oral Comprehension Editing Reading Vocabulary Quantitative Concepts Academic Knowledge Spelling of Sounds Sound Awareness Punctuation/Capitalization

Preliminary Testing (at BC) Special Instructions Item Administered By: Date & Initials Intake Booklet & Consent Form 3 Minutes Timed Test 2: Reading Fluency Test 5: Calculation Test 6: Math Fluency Use Tape Test 7: Spelling 7 Minutes Timed Test 8: Writing Fluency Test 11: Writing Samples Other:

“To avoid criticism, do nothing, say nothing, be nothing “To avoid criticism, do nothing, say nothing, be nothing.” - Elbert Hubbard

Scoring and Reporting Normative Update

Normative Updates (Nu) Check accuracy of all scores Launch Normative Update Program Set up program once: Click <Options> Click <Program Options> Under “Additional Score Column” choose CALP For “Confidence Band” choose 68% For “Discrepancy Cutoff” choose 1.30

Normative Update Continued For Verbal & Numeric Data: Enter scores for Cognitive & Achievement, Standard & Extended Batteries Enter observations Click <Report> <Summary & Score Report> Choose Age Select “Include GIA/BIA” For “Ability/Ach” choose “GIA-Extended”, then also run “Oral Language” or “GIA-Standard” For “Intra-Ability” choose “Intra-Cog and/or Intra-Ach” Send to “Print Preview” Click <OK>

Normative Update Report To see next page in report, press PAGE DOWN key Click icon to Print the report Click icon to Close Preview (or press ESC)

Normative Update Continued For Graphical Data: Click <Report>, <Age/Grade Profiles> Choose “Age” Choose “Include GIA/BIA” For “Data to Include” choose “Standard Score” Send report to “Print Preview” Click <OK>

“Machines increase the number of things we can do without thinking “Machines increase the number of things we can do without thinking. Things we do without thinking - there's the real danger.” - Frank Herbert