Part II – Chapters 6 and beyond…. Reliability, Validity, & Grading

Reliability Definition – consistency or repeatability Objectivity A special kind of reliability Interrater reliability Can be measured by test-retest correlation

Validity Definition – truthfulness of a test score Dependent on reliability and relevance (the degree to which a test pertains to its objectives)

Development of criterion-referenced standards The judgmental approach The normative approach The empirical approach The combination method

Grading Issues with grading in physical education Mostly from chapter 8 & 9

All grades have some degree of subjectivity But some have more than others Should we try to reduce the subjectivity of our grading practices? How can we do that?

Formative and Summative Evaluations Which do we use in PE? Which should we use?

The Process of Grading (page 130) Grading across the 3 domains of performance

The process 1. Determine objectives 2. Select tests 3. Administer tests and compare to standards 4. Compute final grades

Objectives Is the objective an important educational outcome? Does every student have an equal opportunity to demonstrate his or her progress towards reaching the objective? Can the objective be measured with reliability and objectivity?

Test Selection Reliable and valid Formative or summative

Results compared to standard Normative or criterion based references?

Final Grades How will evaluations be combined to produce a final grade? What is acceptable and/or unacceptable for final grade distributions?

Identify Current practices For better or worse… Look at Figure 8.2 on page 131 How do these practices compare with your own experiences? How do these practices fare compared to the previous criteria?

What are the special problems facing a PE teacher when it comes to grading?

What not to grade on Attendance Dressing out Shower taking Sportsmanship? Participation? Team rank? Improvement?

What to grade on Team Sports classes Sports skill tests Rating scales Round-robin tournament performance Individual sports classes Performance ratings relative to objectives

What to grade on Fitness classes Fitness tests Knowledge tests should be used in all classes

What to grade on Grades should reflect the level of achievement in meeting the course objectives and nothing else Grades should be based on status and not improvement Grades should not be used for rewards or punishment Grades should be taken as measurements rather than evaluations

Grading Mechanics Determine and weight course objectives Measure the amount of achievement toward the course objectives Combine measurements into a composite score Convert the composite score to a final grade

Combining scores Standard scores Weighted percentages

Assigning grades Cutoffs Curves

Artifacts from Syllabus 1. normative versus criterion based assessment 2. reflection paper on grading in HPER 3. student developed grading philosophy 4. student developed assessment model 5. student developed assessment plan 6. student developed rubric 7. grading spreadsheet 8. sample stat analysis with SPSS (and/or Excel)

Assignments coming up (all artifacts) Write grading philosophy paper Develop assessment plan/model Develop spreadsheet with sample data After next chapter develop rubric Reflection paper

Grading Philosophy Typed and double spaced About one page in length Must describe the following issues: Will you grade in all 3 domains? If so then what sorts of things will you include? To what extent will you grade on improvement versus achievement (explain why)? What sort of grade distribution will you feel comfortable with?

Develop assessment plan/model Basically a course syllabus for your hypothetical course Includes Course description Course objectives  Explain how you will measure each of these Evaluation information  Explain how you will combine measurements of course objectives

Assessment Plan cont…. Must align with your philosophy statement Must include multiple objectives Rationale should be provided for any significant deviations from concepts discussed in class

Spreadsheet Must be compatible with course syllabus Must include sample data Should be as “user-friendly” as possible That means as automated as possible – should show variety of functions and helpful formatting Must automatically compute final grade Must use some form of standard scores

Spreadsheet cont…. You should turn in a “soft copy” of your spreadsheet as an attachment Send to

Chapter 9 Alternative assessment

Recent trends Use assessment rather than test Assessment The process of collecting information More politically correct and less threatening

Recent trends Alternative assessments Authentic assessments Often terms are used synonymously

Alternative assessments Portfolios Role-playing Interviews Event tasks Individual or group projects Open-ended questions Student logs or journals Exhibitions Observation

Alternative assessments Pros Popular May be fairer – multiple intelligence idea May be more “authentic” Cons Subjective?

Authentic assessment Designed to take place in a real-life setting rather than a contrived or artificial setting Frequently involves directly observable behavior

Recent Trends “At the heart of the most recent wave of educational reform is the identification of learner outcomes, framed in the context of standards that define what students should know and be able to do.” – page 155

Criteria for judging performance Criteria for judging student performance have been called many things: Scoring guidelines Rating scales Checklists Rubrics Rubrics is currently the most popular

Rubric A fixed scale and a list of characteristics describing performance for each of the points on the scale See Table 9-2 page 162 See table 9-4 page 163 Can be useful for assessments in the affective domain Hellison’s model of social responsibility See Table 9-6 – page 164

Ideas for rubrics PE Central Rubistar

Rubric Assignment Create a rubric that you could use for grading a motor skill – part of your evaluation plan You may use the sample rubrics in the text as a guide to develop your own but you should not copy them word for word

QPA What is QPA? How does QPA affect physical education teachers in Kansas? KSBE website

No Child Left Behind What is “NCLB”? How does “NCLB” affect physical education teachers? Web Sites Government site 1 Government site 2 A different view

Reflection Paper Reflect on knowledge of assessment and predict how you may use this knowledge in your future career. Has your philosophy on assessment changed? Some things you might discuss include: Grade inflation Improvement versus achievement Current trends in assessment