Download presentation
Presentation is loading. Please wait.
1
Making Assessment Meaningful
Marian University August 2012 Susan Hatfield Winona State University Visiting Professor, Higher Learning
2
1. Assessment is about student learning
3
2. It isn’t going away
4
3. You are probably already doing it
5
Evaluation Assessment Used to see if students understand
Quizzes Count toward final grade Used to see if students understand Tests Scored and returned Scored, tabulated. returned & discussed; adjustments to syllabus Rubrics Returned to students with grade
6
Does not meet Meets Exceeds Verbal Delivery x Nonverbal Delivery x Organization x Evidence x Transitions x
7
Evaluation Assessment Used to see if students understand
Quizzes Count toward final grade Used to see if students understand Tests Scored and returned Scored, tabulated. returned & discussed; adjustments to syllabus Returned after being aggregated & analyzed; adjustments to syllabus Rubrics Returned to students with grade
8
Does not meet Meets Exceeds Verbal Delivery 14% 81% 5% Nonverbal Delivery 22% 74% 14% Organization 14% 79% 7% Evidence 9% 72% 19% Transitions 7% 85% 8% Program Summary
9
4. Assessment requires exertion and intention
10
Exertion without Intention
11
Intention without Exertion
12
Intention and Exertion
13
5. There is a difference between assessing Program Effectiveness and assessing Student Learning Outcomes
14
Assessment of Program Effectiveness
What the program will do or achieve Curriculum Retention Graduation Placement Satisfaction (graduate and employer)
15
Assessment of Student Learning Outcomes
What students will do or achieve Direct Methods of Assessment
16
6. Assessment drives improvement & demonstrates integrity
17
How Assessment Works Phase 2 Phase 3 Phase 1 O U T C M E Compare
Against Benchmarks, Standards, Targets, Past Performance New / Revised learning event New / Revised learning event learning event component BASELINE component component
18
7. Assessment makes more sense if you focus on what is important to you
19
The relationship between values and measures
20
8. The success of any assessment efforts depends on the outcomes
21
Student Learning Outcomes
Students should be able to <<action verb>> <<something>>
22
Student Learning Outcomes
Learner Centered Specific Action oriented Public Distinctive Frame Perceptions Cognitively appropriate
23
Lower division course outcomes ANALYSIS COMPREHENSION EVALUATION
APPLICATION SYNTHESIS KNOWLEDGE Associate Classify Compare Compute Contrast Differentiate Discuss Distinguish Estimate Explain Express Extrapolate Interpolate Locate Predict Report Restate Review Tell Translate Analyze Appraise Calculate Categorize Classify Compare Debate Diagram Differentiate Distinguish Examine Experiment Inspect Inventory Question Separate Summarize Test Arrange Assemble Collect Compose Construct Create Design Formulate Integrate Manage Organize Plan Prepare Prescribe Produce Propose Specify Synthesize Write Appraise Assess Choose Compare Criticize Determine Estimate Evaluate Grade Judge Measure Rank Rate Recommend Revise Score Select Standardize Test Validate Cite Count Define Draw Identify List Name Point Quote Read Recite Record Repeat Select State Tabulate Tell Trace Underline Apply Calculate Classify Demonstrate Determine Dramatize Employ Examine Illustrate Interpret Locate Operate Order Practice Report Restructure Schedule Sketch Solve Translate Use Write Lower division course outcomes
24
Upper division Course / Program outcomes ANALYSIS COMPREHENSION
EVALUATION APPLICATION SYNTHESIS KNOWLEDGE Associate Classify Compare Compute Contrast Differentiate Discuss Distinguish Estimate Explain Express Extrapolate Interpolate Locate Predict Report Restate Review Tell Translate Analyze Appraise Calculate Categorize Classify Compare Debate Diagram Differentiate Distinguish Examine Experiment Inspect Inventory Question Separate Summarize Test Arrange Assemble Collect Compose Construct Create Design Formulate Integrate Manage Organize Plan Prepare Prescribe Produce Propose Specify Synthesize Write Appraise Assess Choose Compare Criticize Determine Estimate Evaluate Grade Judge Measure Rank Rate Recommend Revise Score Select Standardize Test Validate Cite Count Define Draw Identify List Name Point Quote Read Recite Record Repeat Select State Tabulate Tell Trace Underline Apply Calculate Classify Demonstrate Determine Dramatize Employ Examine Illustrate Interpret Locate Operate Order Practice Report Restructure Schedule Sketch Solve Translate Use Write Upper division Course / Program outcomes
25
9. The outcomes need to be defined
26
Components Define student learning outcomes
Provide a common language for describing student learning Must be outcome specific Must be shared across faculty Number of components will vary by outcome
27
Communication Write Relate Speak Listen Participate Component
28
Components Communication Write Relate Speak Listen Participate Process
delivery Process Summary content organization
29
Components Communication Write Relate Speak Listen Participate Letter
mechanics Letter style organization
30
The Reality of Assessing Student Learning Outcomes
Why you need common components
31
teacher4 teacher2 teacher1 teacher3 teacher5 Speaking eye contact
gestures volume sources transitions style rate poise examples verbal variety appearance evidence conclusion organization attention getter
32
1 K A A A A S 2 K A S 3 K K K K 4 5 K S 6 K K A A S 7 S A A S
Program Level Student Learning Outcomes 1xx 1xx 2xx 2xx 2xx 3xx 3xx 3xx 4xx Capstone 1 K A A A A S 2 K A S 3 K K K K 4 5 K S 6 K K A A S 7 S A A S K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation
33
1 K A A S A S 2 K A A A Build a curriculum 3 K K A S 4 K K S 5 K K A S
Program Level Student Learning Outcomes 1xx 1xx 2xx 2xx 2xx 3xx 3xx 3xx 4xx Capstone 1 K A A S A S 2 K A A A Build a curriculum 3 K K A S 4 K K S 5 K K A S S 6 K K K A S 7 S A A S S K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation
34
Analyze Curriculum 1 K A A S A S 2 K A A A 3 K K A S Diagnose Learning
Program Level Student Learning Outcomes 1xx 1xx 2xx 2xx 2xx 3xx 3xx 3xx 4xx Capstone Analyze Curriculum 1 K A A S A S 2 K A A A 3 K K A S Diagnose Learning 4 K K S 5 K K A S S 6 K K K A S 7 S A A S S K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation
35
1 K A A S A S 2 K A A A 3 K K A S Diagnose Learning 4 K K S 5 K K A S
Program Level Student Learning Outcomes 1xx 1xx 2xx 2xx 2xx 3xx 3xx 3xx 4xx Capstone 1 K A A S A S 2 K A A A 3 K K A S Diagnose Learning 4 K K S 5 K K A S S 6 K K K A S 7 S A A S S K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation
36
11. You don’t have to assess every outcome in every course
37
Assessment Points Assessment is NOT assessing every student on every outcome in every class by every faculty member every semester
38
1 K A A S A S 2 K A A A S 3 K A A A A S 4 K A S S 5 K A K A S S 6 K K
Program Level Student Learning Outcomes 1xx 1xx 2xx 2xx 2xx 3xx 3xx 3xx 4xx Capstone 1 K A A S A S 2 K A A A S 3 K A A A A S 4 K A S S 5 K A K A S S 6 K K A A S 7 S A A S S S K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation
39
12. There are multiple ways to assess student learning outcomes
40
Assessment Tools Quantitative Data Numbers Percentage Ratios Rates
Qualitative Data Descriptions Explanations Perceptions
41
Types of Assessment Data
Objective Data records Databases, incidents, reports Surveys, Questionnaires, Interviews, Focus Groups Subjective Self-Report feelings Logbook, Action Report Behavioral Self-Report actions Subjective Analysis thoughts Reflective Essay Objective Analysis observations Checklist, Rubric, Scale
42
Learning Objects Standardized Exam, abstract, advertisement, annotated bibliography, biography, briefing, brochure, budget, care plan, case analysis, chart, cognitive map, court brief, debate, definition, description, diagram, dialogue, diary, essay, executive summary, exam, flow chart, group discussion, instruction manual, inventory, lab notes, letter to the editor, matching test, mathematical problem, memo, micro theme, multiple choice test, narrative, news story, notes, oral report, outline, performance review, plan, presentation, process analysis, proposal, regulation, research proposal, review of literature, taxonomy, technical report, term paper, thesis, word problem, work of art. (Walvoord Anderson 1998).
43
1 2 3 4 Student Learning Outcome Test Questions 1 2 3 4 5 6 7 8 9 10
11 12 4 13 14 15 16 17 18
44
Student Learning Outcome Assignment 1 1 2 2 3 3 4 4
45
Student Learning Outcome Assignment 1 1 2 3 2 4
46
Student Learning Outcome Experience 1 2 3 4
47
Options for Assessing Learning
Test Project Case Study Care Plan Patient History Presentation Portfolio Brochure Internship Number or percentage correct Comparison to national norms
48
1 2 3 4 Student Learning Outcome Test Questions % correct 1 2 3 4 73%
5 6 7 8 9 82% 3 10 11 12 64% 4 13 14 15 16 17 18 87%
49
Options for Assessing Learning
Test Project Case Study Care Plan Patient History Presentation Portfolio Brochure Internship Number or percentage correct Comparison to national norms Scale Holistic Rubric
50
1 2 3 4 5 Presentation 1 2 3 4 5 1 2 3 4 5 Student Learning Outcome
Components Presentation Content Familiar New Delivery Distracts Enhances Organization Unclear Focused
51
Student Learning Outcome Components Presentation Content Delivery Organization
52
Rubrics for Students Clarify expectations Focus attention
Set standards Provide detailed feedback
53
Rubrics for Faculty Clarify goals of assignment Standardize feedback
More detailed feedback Enhance objectivity Prevent evaluation drift Focus discussions with students
54
Rubrics for Assessment
Consistent Feedback across students, course, program allowing for the monitoring of learning outcomes Provide Direction for Course, Class, Program Improvement
55
Rubrics Analytical Numerous components which you want to assess individually Summative
56
Exceeds Presentation Student Learning Outcome Components Content
Delivery Organization
57
Rubrics Developmental
Expectations for performance rise throughout the class or curriculum
58
Presentation sophistication Student Learning Outcome Components
Content sophistication Delivery Organization
59
Fourth Assignment Presentation Student Learning Outcome Components
Content Delivery Organization
60
Rubrics Analytical Holistic
- generally smaller assignments with fewer criteria, shorter turn around time.
61
Presentation Proficient Excellent Notable Student Learning Outcome
Components Presentation Content Delivery Needs Work Proficient Excellent Unacceptable Notable Organization
62
F D C B A Presentation Student Learning Outcome Components Content
Delivery Organization
63
Resource
64
13. Look for and analyze patterns of data
65
Interpreting Data Consistency - over time
Consensus - different populations Distinctiveness - different situations/ variables / items
66
Consistency Examines the same practice of and individual or group over time Key question: Has this person or group acted, felt, or performed this way in the past / over time?
67
Consistency How well are students performing on the learning outcome?
High performance Low performance 05 06 07 08 09 10 11 12
68
Consensus Comparison to or among groups of students Key questions:
Variation between disciplines, gender, other demographic variables Key questions: What is the general feeling, outcome, attitude, behavior? Do other groups of people act, perform or feel this way?
69
Consensus How well are students performing on the learning outcome?
High performance Low performance Females Males Transfers OTA
70
Distinctiveness Examines individual or cohort perspectives across different situations, categories Key Question: Does a person or group respond differently based upon the situation, item, issue?
71
Distinctiveness How well are our students achieving our
Core Learning Outcomes? High Performance Low Performance
72
One data point is not a pattern of evidence no matter how soon you have your reaccreditation visit.
73
14. Assessment results in Action
R A
74
How Assessment Works Year 2 Year 3 Year 1 O U T C M E Compare Against
Benchmarks, Standards, Targets, Past Performance New / Revised learning event New / Revised learning event learning event component BASELINE component component
75
Acting on Results Development Infrastructure Curriculum
Faculty, Staff, Student Infrastructure Policy, Process, Planning Curriculum Learning Opportunities
76
The Seven Principles for Good Practice in Undergraduate Education
1. Student-Faculty Contact 2. Cooperative Learning 3. Active Learning 4. (Prompt) Feedback 5. Time on Task 6. High Expectations 7. Respect for Diverse Talents and Ways of Learning
77
15. Reporting Out R A
78
Learning Outcome: students will be able to <<action verb>><<something>>
Components Grading criteria, Central traits of successful achievement Courses & Experiences list of courses and experiences supporting achievement of the outcome Assessment Point course, timeframe, experience 1 2 3 4. 5. 6 Learning Object Assignment, test, other student work Benchmark Acceptable performance level on assessment tool Assessment Tool Test score, national norm rubric, scale Assessment Point Benchmark Assessment tool Learning Object Assessment Point Benchmark Assessment tool Learning Object 3
79
<<object>>
What we looked at <<object>> How we assess it <<assessment tool>> What we found <<results>> What it means <<interpretation>> What we’re going to do about it <<action>> What happened Outcome <<feedback>> 6
Similar presentations
