Please check, just in case…

Announcements 1.Standardized Test Description due in two weeks. 2.Questions about upcoming assignments? Make an appointment.

Quick questions or quandaries?

APA Tip of the Day: e.g. vs. i.e. e.g. means “for example” i.e. means “that is” cf. means “compare” …, etc. means “and so forth” vs. means “versus, against” viz. means “namely” (information from APA, 2010, p. 108)

Topic: Interpreting Standardized Test Results October 29, 2014

IDEA Requirements for Evaluation: “ Tests are selected and administered so as best to ensure that if a test is administered to a child with impaired sensory, manual, or speaking skills, the test results accurately reflect the child’s aptitude or achievement level or whatever other factors the test purports to measure, rather than reflecting the child’s impaired sensory, manual, or speaking skills (unless those skills are the factors that the test purports to measure).”

“For all test takers, any tests that employs language is, in part, a measure of their language skills.” (AERA, APA, NCME, 1999, p. 91)

Norm-referenced Tests Describe “performance in terms of the relative position held in some known group (e.g., typed better than 90 percent of the class members).” (Linn & Gronlund, 2000, p. 42) Norm-referenced assessments compare individual performance against others’ performance.

Questions to ask: Who was included in the normative sample? Was the normative sample adequately described? Was the normative sample similar to the students who are being tested with the instrument?

More questions to ask: Were students from minority backgrounds included in similar proportions to our local population? What about bilingual students? Were they included? If so, were they adequately described?

Even more questions to ask: What about dialect? Would use of a non-standard dialect influence test responses? If so, was that taken into consideration in the normative sample? What about socio-economic status? Were students from a range of SES backgrounds included? How was SES determined? Was that adequately explained?

HINT!

Quick small group activity Read at the quotes on the next 4 slides individually and make notes on: What you understand. What you don’t understand. With 1 or 2 other colleagues, share your understandings and questions. Decide on 1 major question to pose to the group as a whole.

“If normative data do not exist for the population for which a test is being administered, it is the responsibility of the examiner to validate the test for that specific population. If the examiner fails to do so, the results of the test will be considered invalid.” (Leary & Boscardian, 1992, p. 53)

“Tests should not be administered to individuals who are known to have specific handicapping conditions if that test has not been appropriately normed on their specific population.” (Leary & Boscardian, 1992, p. 54)

When testing students who have been exposed to more than one language “norms developed for monolingual English-speaking populations should either not be used or should be interpreted with the understanding that English language proficiency is a contaminating factor.” (President’s Advisory Commission on Educational Excellence for Hispanic Americans, 2000, p. 11)

"When a bilingual individual confronts a monolingual test, developed by monolingual individuals, and standardized and normed on a monolingual population, both the test taker and the test are asked to do something they cannot do. The bilingual test taker cannot perform like a monolingual. The monolingual test cannot ‘measure’ in the other language." (Valdés & Figueroa, 1994, p. 87)

Really? “Test developers should collect for each linguistic subgroup studied the same form of validity evidence collected for the examinee population as a whole.” (AERA, APA, NCME, 1999, p. 97)

So why don’t we translate each test? We might be able to later develop norms for the translated version.

“A translated test is an inappropriate test. The practice should be proscribed.” (President’s Advisory Commission on Educational Excellence for Hispanic Americans, 2000, p. 11)

“Translating a test assumes that the items in one language arrange themselves in the same order of difficulty as in another language.” (Valdés & Figueroa, 1994, p. 105)

“ The use of interpreters should be discouraged, if not proscribed. Interpreters are basically poor substitutes for what should be provided... culturally knowledgeable, linguistically competent testers... in special education, the use of interpreters may lead to invalid inferences and conclusions.... This is a practice that may really be malpractice.” (President’s Advisory Commission on Educational Excellence for Hispanic Americans, 2000, p. 47)

Quick Write: So what are some emerging ideas you now have on using standardized, norm-referenced tests with diverse populations?

“Tests should not be administered to individuals who are known to have specific handicapping conditions if that test has not been appropriately normed on their specific population.” (Leary & Boscardian, 1992, p. 54)

Questions to ask: Who was included in the normative sample? Was the normative sample adequately described? Was the normative sample similar to the students who are being tested with the instrument?

More questions to ask: Were students with disabilities included in the normative sample? If so:  What disabilities?  How were they identified?  What about students who require alternative response formats?

Even more questions to ask: Are any accommodations for student disability (i.e. sensory impairments) allowed on the assessment? If so, what changes in administration would be considered accommodations and what would be considered a modification?

Accommodations vs. Modifications

Accommodations do NOT change fundamental aspects of the test. Ex. Extra time, breaks, limiting the number of questions visible at one time, bubbling answers in for students.

Modifications DO change fundamental aspects of the test and are therefore, not allowed. Ex. Reading a passage for students on a reading comprehension test, or, using a calculator on a basic math skills test.

CAUTION! What is considered an “accommodation” versus a “modification” differs by EACH test. Unless specified as an allowable accommodation, ANY change in administration procedures from those specified in the manual will invalidate the use of normative comparisons.

“If normative data do not exist for the population for which a test is being administered, it is the responsibility of the examiner to validate the test for that specific population. If the examiner fails to do so, the results of the test will be considered invalid.” (Leary & Boscardian, 1992, p. 53)

Buyer Beware! Don’t buy a used car without looking under the hood!

Main Points 1.It is extremely difficult, if not impossible, to develop adequate norms for diverse populations, such as individuals with multiple disabilities or culturally and linguistically diverse populations.

Main Points 2.Given the purpose of norm- referenced tests (in contrast to criterion-referenced tests) their usefulness with individuals who differ significantly from the normative group is highly questionable.

Main Points 3.The test results for these individuals must be interpreted with great caution.

Small Group Activity: Formulate a list of SPECIFIC suggestions within your group as to how the information from tonight's class could be used in the test description assignment.

Please take a minute for the minute paper. And don’t forget to turn your phone back on.