Chapter 4 Measurement, Assessment, and Program Evaluation 4 Measurement, Assessment, and Program Evaluation Manny Felix and Garth Tymeson C H A P T E R

Measurement, Assessment, and Program Evaluation Common assessment strategies Standardized approaches Alternative approaches

Standardized Approaches Usually published tests Limited selection of test items Specific directions for administration Usually known validity and reliability Generally strong psychometrically but weak authentically Standards provided to make judgments about student test performance

Standards Norm referenced Criterion referenced

Norm-Referenced Standards Comparisons are made with others from a specifically defined group (e.g., age, sex, disability). Norms are usually developed by testing large numbers, and results are tabulated. Percentiles, T-scores, and z-scores are used. (continued)

Norm-Referenced Standards (continued) Above average and below average are the types of judgments made. IQ tests and older physical education tests are examples.

Criterion-Referenced Standards Comparisons are made with predetermined mastery scores (minimally acceptable scores for a particular purpose). Criteria are determined by expert opinion, research data, logic, experience, and so on. (continued)

Criterion-Referenced Standards (continued) Competent (meets standard) and noncompetent (does not meet standards) are the types of judgments made. Fitnessgram is an example.

Alternative Approaches Link assessment to instruction. Have day-to-day applicability. Often teacher constructed for specific situations or physical education content. Strong authentically but weak psychometrically (premium on subjective evaluation). Checklists, rubrics, task analyses, and portfolios are examples.

Checklist Identifies presence or absence of behavior or skill. Does not indicate the quality of behavior. Helpful when using task-analytic or ecological approaches. Use with a specific skill or series of skills.

Rubric In this type of rating scale, a student’s performance is matched to one of multiple levels of a skill via a set of criteria. Students know where they stand and what needs to be done for improvement. Works well with IEPs: If PLP = 2, then short-term objectives = 3 or 4 (or certain aspects of 3 or 4).

Task Analysis Many types exist, but all involve breaking skills down into smaller, perhaps sequentially ordered, steps or focal points. When used as an assessment instrument, missing components are identified, and a strategy for teaching is revealed. It can be expanded for use in ecological assessment (including functional and top- down approaches).

Portfolio A collection of representative student work Entries: videos, test results (standardized and alternative), peer evaluations, journals, logs, and so on Strength: multiple assessments on multiple occasions

Standardized Versus Alternative Generally use both. Give preference to standardized for unique need questions. –IDEA requires valid, reliable, objective, and nondiscriminatory testing. Give preference to alternative when devising instructional strategies. –Assessment is curriculum embedded.

Determination of Unique Need Low motor development Low motor skill performance Low health-related physical fitness (continued)

Determination of Unique Need (continued) Possible criteria: <15th percentile >1 standard deviation below the mean (T < 40) 2-year developmental delay or more Fails to meet criterion-referenced standards Fails to meet 70% of the competencies in the curriculum Trial placement recommended for corroboration

Other Factors to Consider in Determining Unique Need Behavior Communication Need for adapted physical education Need for safe participation Medical condition or disability Potential for intramural and interscholastic athletic experiences

Meeting State- or District-Wide Testing Requirements All students, including those with disabilities, should be incorporated in any state- or district-wide assessment programs. Many physical education testing programs (e.g., health-related physical fitness) are required by districts. (continued)

Meeting State- or District-Wide Testing Requirements (continued) Participation levels include the following: –Use same standardized test. –Provide appropriate accommodations. –Provide an alternative test.

What to Assess in Adapted Physical Education? Remember IDEA definition of physical education. –Physical and motor fitness –Fundamental motor skills and patterns –Skills in aquatics, dance, and individual and group games and sports (including intramural and lifetime sports) (continued)

What to Assess in Adapted Physical Education? (continued) Minimally test physical fitness (or physical activity) and motor skills (including reflexes and rudimentary, fundamental, and specialized skills, as appropriate). Affective skills may also be assessed in authentic settings.

Milani-Comparetti Purpose: to assess motor development, including reflexes and reactions, in young children (birth-2 years) and children with developmental disabilities Description: 27 total test items; 9 test head control, body control, and active movements; 5 test primitive reflexes; 13 test postural reactions Scoring: age norms for each test are provided Comment: often administered by therapists

Peabody Developmental Motor Scales Purpose: to assess the fine and gross motor development of children (birth-5 years) Description: 249 test items (mostly developmental milestones) arranged across six categories and age levels Scoring: gross motor quotient, fine motor quotient, and total motor quotient Comment: normative data available

Test of Gross Motor Development Purpose: to test fundamental movement patterns in preschool and early elementary children with emphasis on process rather than product of performance Description: 12 patterns tested within locomotor and object-control subtests Scoring: based on focal points listed for each pattern Comment: criterion-based scores compared with norm-referenced standards

Sport Skills Program Guides Purpose: to assist in the assessment and instruction of sport skills for people with disabilities (aged 8 and beyond) Description: task-analyzed assessments available for 29 sports Scoring: focal points checked off as athletes demonstrate correct techniques Comment: used by Special Olympics; strong authentically but no validity or reliability reported

Brockport Physical Fitness Test Purpose: to assess the health-related fitness of young people (aged 10-17) with certain disabilities Description: typically 4 to 6 test items selected from 27 possibilities based on a personalized approach Scoring: test scores compared with criterion- referenced standards based on gender, age, and in some cases disability Comment: closely related to Fitnessgram and supported by computer software (Fitness Challenge)

Activitygram Purpose: to record, analyze, and save student physical activity data and produce reports based on the data Description: computer program prompts students to recall previous day’s physical activity in 30- minute time blocks Scoring: total number of minutes of at least moderate level of activity for previous 3 days Comment: good utility for students with disabilities (but they may need help recalling or entering data)

Program Evaluation Increasingly important to demonstrate that an instructional program is good, not merely claim it is good. Requires that program goals be clearly articulated. Program goal should include a criterion (e.g., 90% of all students will engage in at least 30 minutes of moderate-level physical activity at least 4 days per week). Student data are aggregated to evaluate the program goal.