Review: Performance-Based Assessments Performanc-based assessment Real-life setting H.O.T.S. Techniques: Observation Individual or Group Projects Portfolios Performances Student Logs or Journals

Developing performance-based assessments Determining the purpose of assessment Deciding what constitutes student learning Selecting the appropriate assessment task Setting performance criteria

Review: Grading Grading process: Making grading fair, reliable, and valid Determine defensible objectives Ability group students Construct tests which reflect objectivity No test is perfectly reliable Grades should reflect status, not improvement Do not use grades to reward good effort Consider grades as measurements, not evaluations Objectives of instruction Test selection and administration Results compared to standards Final grades

Cognitive Assessments Physical Fitness Knowledge HPER 3150 Dr. Ayers Physical Fitness Knowledge

Test Planning Types Mastery (driver’s license) Meet minimum requirements Achievement (mid-term) Discriminate among levels of accomplishment

Table of Specifications (content-related validity) Content Objectives history, values, equipment, etiquette, safety, rules, strategy, techniques of play Educational Objectives (Blooms’ taxonomy, 1956) knowledge, comprehension, application, analysis, synthesis, evaluation

Table of Specifications for a 33 Item Exercise Physiology Concepts Test (Ask-PE, Ayers, 2003) T of SPECS-E.doc

Test Characteristics When to test Often enough for reliability but not too often to be useless How many questions (p guidelines) More items yield greater reliability Format to use (p. 147 guidelines) Oral (NO), group (NO), written (YES) Open book/note, take-home Advantages: ↓anxiety, ask more application Qs Disadvantages: ↓ incentive to prepare, uncertainty of who does work

Test Characteristics Question types Semi-objective short-answer completion mathematical Objective t/f Matching multiple-choice Classification Essay

Semi-objective Questions Short-answer, completion, mathematical When to use (factual & recall material) Weaknesses Construction Recommendations (p. 151) Scoring Recommendations (p. 152)

Objective Questions True/False, matching, multiple-choice When to use (M-C: MOST IDEAL) FORM7 (B,E).doc Pg : M-C guidelines Construction Recommendations (p ) Scoring Recommendations (p )

Figure 8.1 The difference between extrinsic and intrinsic ambiguity (A is correct) Too easy Extrinsic ambiguity (weak Ss miss) Intrinsic Ambiguity (all foils = appealing)

Cognitive Assessments I Explain one thing that you learned today to a classmate

Review: Cognitive Assessments I Test types MasteryAchievement Table of Specifications Identify content, assign cognitive demands, weight areas Provides support for what type of validity? Questions Types Semi-objective: short-answer, completion, mathematical Objective: t/f, match, multiple-choice Which is desirable: intrinsic/extrinsic ambiguity

Essay Questions When to use (definitions, interpretations, comparisons) Weaknesses Scoring Objectivity Construction & Scoring recommendations (p )

Administering the Written Test Before the Test During the Test After the Test

Characteristics of “Good” Tests Reliable Valid Average difficulty Discriminate Gotten correct by more knowledgeable students Missed by less knowledgeable students Time consuming to write

Quality of the Test Reliability Role of error in an observed score Error sources in written tests Inadequate sampling Examinee’s mental/physical condition Environmental conditions Guessing Changes in the field (dynamic variable being measured)

Quality of the Test Validity CONTENT key for written tests Is critical information assessed by a test? T of Specs helps support validity Overall Test Quality Based on individual item quality (steps 1-8, pg )

Item Analysis Used to determine quality of individual test items Item Difficulty Percent answering correctly Item Discrimination How well the item "functions“ Also how “valid” the item is based on the total test score criterion

Item Difficulty 0 (nobody got right) – 100 (everybody got right) Goal=50%

Item Discrimination <20% & negative (poor)20-40% (acceptable) Goal > 40%

Figure 8.4 The relationship between item discrimination and difficulty Moderate difficulty maximizes discrimination

Sources of Written Tests Professionally Constructed Tests (FitSmart, Ask-PE) Textbooks (McGee & Farrow, 1987) Periodicals, Theses, and Dissertations

Questionnaires Determine the objectives Delimit the sample Construct the questionnaire Conduct a pilot study Write a cover letter Send the questionnaire Follow-up with non-respondents Analyze the results and prepare the report

Constructing Open-Ended Questions Advantages Allow for creative answers Allow for respondent to detail answers Can be used when possible categories are large Probably better when complex questions are involved Disadvantages Analysis is difficult because of non-standard responses Require more respondent time to complete Can be ambiguous Can result in irrelevant data

Constructing Closed-Ended Questions Advantages Easy to code Result in standard responses Usually less ambiguous Ease of response relates to increased response rate Disadvantages Frustration if correct category is not present Respondent may chose inappropriate category May require many categories to get ALL responses Subject to possible recording errors

Factors Affecting the Questionnaire Response Cover Letter Be brief and informative Ease of Return You DO want it back! Neatness and Length Be professional and brief Inducements Money and flattery Timing and Deadlines Time of year and sufficient time to complete Follow-up At least once (2 about the best response rate you will get)

The BIG Issues in Questionnaire Development Reliability Consistency of measurement Stability reliability: 2-4 wks between administrations Validity Truthfulness of response Good items, expert reviewed, pilot testing, confidentiality/anonymity Representativeness of the sample To whom can you generalize?

Cognitive Assessments II Ask for clarity on something that challenged you today