Agenda Peer Assessment Roundtables – Student Learning 10: :10 – Welcome and Explain Process 10: :30 – Full Group: Coaching Assessment – SLOs 10:30 – 10:50 –Roundtable Discussion 1: Student Learning Outcomes 10:50 – 11:00 - Full Group: Coaching Assessment - Curriculum Mapping 10:50 – 11:10 – Roundtable Discussion 2: Curriculum Mapping 11:10 – 11:30 – Roundtable Discussion 3: Example Outcomes: Findings and Changes 11:30 – 11:50 – Roundtable Discussion 4: Assessment Process 11:50 – 12:00 – Wrap Up

Peer Assessment Roundtables Coaching Assessment Avoiding Garbage In/Garbage Out: Student Learning Outcomes Curriculum Mapping

Fundamental Assumption: Assessing student learning at the program level starts with well written student learning outcomes.

Fundamental TIP: There’s nowhere for a program to go if they are starting with poorly written student learning outcomes

What the student will √ Know √ Do ….at the end of the course or program Assessing Fundamental Structure of SLO

Fundamental TIP: The more complex the outcome, the harder it will be to assess

Format for learning outcomes: Students will be able to > >

Fundamental TIP: Use whatever language on which the program can agree

Fundamental TIP: ….but stay away from multiple and multiple

Students will be able to,, and,,, and.

KNOWLEDGE COMPREHENSION APPLICATION ANALYSIS SYNTHESIS EVALUATION Cite Count Define Draw Identify List Name Point Quote Read Recite Record Repeat Select State Tabulate Tell Trace Underline Associate Classify Compare Compute Contrast Differentiate Discuss Distinguish Estimate Explain Express Extrapolate Interpolate Locate Predict Report Restate Review Tell Translate Apply Calculate Classify Demonstrate Determine Dramatize Employ Examine Illustrate Interpret Locate Operate Order Practice Report Restructure Schedule Sketch Solve Translate Use Write Analyze Appraise Calculate Categorize Classify Compare Debate Diagram Differentiate Distinguish Examine Experiment Inspect Inventory Question Separate Su rize Test Arrange Assemble Collect Compose Construct Create Design Formulate Integrate Manage Organize Plan Prepare Prescribe Produce Propose Specify Synthesize Write Appraise Assess Choose Compare Criticize Determine Estimate Evaluate Grade Judge Measure Rank Rate Recommend Revise Score Select Standardize Test Validate Lower level course outcomes

KNOWLEDGE COMPREHENSION APPLICATION ANALYSIS SYNTHESIS EVALUATION Cite Count Define Draw Identify List Name Point Quote Read Recite Record Repeat Select State Tabulate Tell Trace Underline Associate Classify Compare Compute Contrast Differentiate Discuss Distinguish Estimate Explain Express Extrapolate Interpolate Locate Predict Report Restate Review Tell Translate Apply Calculate Classify Demonstrate Determine Dramatize Employ Examine Illustrate Interpret Locate Operate Order Practice Report Restructure Schedule Sketch Solve Translate Use Write Analyze Appraise Calculate Categorize Classify Compare Debate Diagram Differentiate Distinguish Examine Experiment Inspect Inventory Question Separate Summarize Test Arrange Assemble Collect Compose Construct Create Design Formulate Integrate Manage Organize Plan Prepare Prescribe Produce Propose Specify Synthesize Write Appraise Assess Choose Compare Criticize Determine Estimate Evaluate Grade Judge Measure Rank Rate Recommend Revise Score Select Standardize Test Validate Advanced Course / Program outcomes

Fundamental TIP: If the outcomes have multiple action verbs, use the highest order verb

Student Learning Outcomes Students will be able to select, describe, and interpret…..

Components – Grading Criteria What are you looking for in student work to be able to tell if they “get it”

Example #1 Gather factual information and apply it to a given problem in a manner that is relevant, clear, comprehensive, and conscious of possible bias in the information selected BETTER: Students will apply factual information to a problem COMPONENTS- Grading Criteria: Relevance Clarity Comprehensiveness Aware of Bias

Example #2 Imagine and seek out a variety of possible goals, assumptions, interpretations, or perspectives which can give alternative meanings or solutions to given situations or problems BETTER: Students will provide alternative solutions to situations or problems COMPONENTS - Grading Criteria: Variety of assumptions, perspectives, interpretations Analysis of comparative advantage

Fundmental TIP: The curriculum should support the learning outcomes

1xx X X X Student Learning Outcomes 1xx X X X 2xx X X X X X 3xx X X X X X X 4xx X X X X Capstone X X

1xx K K K Student Learning Outcomes K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation 1xx S K K 2xx A A S A A K 3xx A K A A K A 4xx S A K S Capstone S S

1xx K K K Student Learning Outcomes K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation 1xx S K K 2xx A A S A A K 3xx A K A A K A 4xx S A K S Capstone S S

1xx K K K Student Learning Outcomes K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation 1xx S K K 2xx A A S A A K 3xx A K A A K A 4xx S A K S Capstone S S

1xx K K K Student Learning Outcomes K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation 1xx S K K 2xx A A S A A K 3xx A K A A K A 4xx S A K S Capstone S S

1xx K K K Student Learning Outcomes K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation 1xx S K K 2xx A A S A A K 3xx A K A A K A 4xx S A K S Capstone S S

1xx K K K Student Learning Outcomes K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation 1xx S K K 2xx A A S A A K 3xx A K A A K A 4xx S A K S Capstone S S

1xx K K K Student Learning Outcomes K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation 1xx S K K 2xx A A S A A K 3xx A K A A K A 4xx S A K S Capstone S S

1xx K K K Student Learning Outcomes K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation 1xx S K K 2xx A A S A A K 3xx A K A A K A 4xx S A K S Capstone S S

1xx K K K Student Learning Outcomes K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation 1xx S K K K 2xx A A A A A K K 3xx A K K K A A S S A S S A 4xx S A S S Capstone S S Build a curriculum

1xx K K K Student Learning Outcomes K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation 1xx S K K K 2xx A A A A A K K 3xx A K K K A A S S A S S A 4xx S A S S Capstone S S Diagnose Learning Analyze Curriculum

1xx K K K Student Learning Outcomes K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation 1xx S K K K 2xx A A A A A K K 3xx A K K K A A S S A S S A 4xx S A S S Capstone S S Diagnose Learning

Fundamental TIP: It is important to share the map with adjunct faculty and students

Fundamental TIP: Use the curriculum map to identify assessment points

Peer Assessment Roundtables Coaching Assessment Avoiding Garbage In/Garbage Out : 1.Student Learning Outcomes 2.Curriculum Mapping 3.Collecting Assessment Information 4.Sharing and Acting on Results

Agenda Peer Assessment Roundtables – Student Learning 10: :10 - Welcome and Explain Process 10: :20 - Full Group Coaching - Collecting Assess. Information 10: :00 - Roundtable Discussion 1: Collecting Assess. Information 11: :05 - Full Group Coaching - Faculty Motivation/Collaboration 11: :15 - Roundtable Discussion 2: Faculty Motivation/Collaboration 11: :25 – Full Group Coaching - Sharing, Acting on Results 11: :50 - Roundtable Discussion 3: Sharing, Acting on Results 11:50 – 12:00 - Wrap Up

Peer Assessment Roundtables Coaching Assessment Avoiding Garbage In/Garbage Out : Student Learning Outcomes 1=Goals Curriculum Mapping Collecting Assessment 2=Information Sharing and 3=Acting on Assessment Results

Peer Assessment Roundtables Coaching Assessment Avoiding Garbage In/Garbage Out : Student Learning Outcomes 1=Goals Curriculum Mapping Collecting Assessment 2=Information Sharing and 3=Acting on Assessment Results

Collecting Assessment Information “The end of assessment is action.”

Direct and Indirect Measures Direct: Ratings by their field supervisors Licensure or certification exams Capstone experiences Other work scored using a rubric Portfolios Scores on tests Electronic class discussion threads Feedback from computer-simulated tasks

Indirect: Course grades Retention and graduation rates Admission rates into graduate programs Post grad activity of alum Placement rates of graduates Alumni perceptions Student feedback Course student rating forms Student, alumni, and employer satisfaction

Using Samples of Student Work for Assessment Advantages: Information is already available No student motivation problems No direct cost Reflects what faculty actually teach

Using Samples of Student Work for Assessment Disadvantages: Evidence not comparable across institutions Information is in multiple parts and formats Quite a bit of work Common standards/rubrics/training needed

Rubrics 1. Task description 2. Scale 3. Dimensions of the assignment 4. Descriptions of performance levels

Performance Element Exemplary (4) Proficient (3) Developing (2) Emerging (1) Not Present (0) Score I. Communication Define problem in your own words. Identifies the main idea or problem with numerous supporting details and examples which are organized logically and coherently. Identifies the main idea or problem with some supporting details and examples in an organized manner. Identifies the main idea or problem with few details or examples in a somewhat organized manner. Identifies the main idea or problem poorly with few or no details or states the main idea or problem verbatim from the text. Does not identify the main idea or problem      N/A  Comments: II. Analysis Compare & contrast the available solutions. Uses specific inductive or deductive reasoning to make inferences regarding premises; addresses implications and consequences; identifies facts and relevant information correctly. Uses logical reasoning to make inferences regarding solutions; addresses implications and consequences; Identifies facts and relevant information correctly. Uses superficial reasoning to make inferences regarding solutions; Shows some confusion regarding facts, opinions, and relevant, evidence, data, or information. Makes unexplained, unsupported, or unreasonable inferences regarding solutions; makes multiple errors in distinguishing fact from fiction or in selecting relevant evidence. Does not analyze multiple solutions      N/A  Comments: III. Problem Solving Select & defend your chosen solution. Thoroughly identifies and addresses key aspects of the problem and insightfully uses facts and relevant evidence from analysis to support and defend potentially valid solutions. Identifies and addresses key aspects of the problem and uses facts and relevant evidence from analysis to develop potentially valid conclusions or solutions. Identifies and addresses some aspects of the problem; develops possible conclusions or solutions using some inappropriate opinions and irrelevant information from analysis. Identifies and addresses only one aspect of the problem but develops untestable hypothesis; or develops invalid conclusions or solutions based on opinion or irrelevant information. Does not select and defend a solution      N/A  Comments:

Benefits of Rubrics Timely feedback Detailed feedback Critical thinking Communication Teaching skills Level playing field

Faith and Christ-Centered Learning How do you collect data on your student faith- growth learning outcomes?

Agenda Peer Assessment Roundtables – Student Learning 10: :10 - Welcome and Explain Process 10: :20 - Full Group Coaching - Collecting Assess. Information 10: :00 - Roundtable Discussion 1: Collecting Assess. Information 11: :05 - Full Group Coaching - Faculty Motivation/Collaboration 11: :15 - Roundtable Discussion 2: Faculty Motivation/Collaboration 11: :25 – Full Group Coaching - Sharing, Acting on Results 11: :50 - Roundtable Discussion 3: Sharing, Acting on Results 11:50 – 12:00 - Wrap Up

Peer Assessment Roundtables Coaching Assessment Avoiding Garbage In/Garbage Out : Student Learning Outcomes 1=Goals Curriculum Mapping Collecting Assessment 2=Information Faculty Motivation/Collaboration Sharing and 3=Acting on Assessment Results

Why Aren’t Grades Enough? Relativity Attendance-Participation Extra Credit Consistency Curriculum

Tangible Actions To Promote Faculty Participation In Assessment Faculty Meetings Syllabi Performance Expectations Scholarship

Agenda Peer Assessment Roundtables – Student Learning 10: :10 - Welcome and Explain Process 10: :20 - Full Group Coaching - Collecting Assess. Information 10: :00 - Roundtable Discussion 1: Collecting Assess. Information 11: :05 - Full Group Coaching - Faculty Motivation/Collaboration 11: :15 - Roundtable Discussion 2: Faculty Motivation/Collaboration 11: :25 – Full Group Coaching - Sharing, Acting on Results 11: :50 - Roundtable Discussion 3: Sharing, Acting on Results 11:50 – 12:00 - Wrap Up

Peer Assessment Roundtables Coaching Assessment Avoiding Garbage In/Garbage Out : Student Learning Outcomes 1=Goals Curriculum Mapping Collecting Assessment 2=Information Sharing and 3=Acting on Assessment Results

Brief Guidelines On Sharing Assessment Results Use good teaching practices Brevity Most important information: 1. How you and your colleagues define a successful student 2. Whether you are satisfied with your evidence of student success 3. What you are doing about unsatisfactory results Visuals

Useful Reporting of Assessment Results What will audiences for assessment results care most about? Matters they can do something about Interesting and unanticipated findings Meaningful differences

Venues for Sharing Assessment Results Web sites s Newsletters Alumni magazines Departmental memos Press releases Brochures Presentations Posters or banners

Impact Story telling Opening Major points to details Context Informed commentary Brevity

Using Assessment Results Effectively and Appropriately What’s most important? Think about process Stakeholders Communication/Information Watch interpretation

What if the results are bad? Do you have the right learning goals? Do you have too many learning goals? Take a hard look at your courses: Content and requirements, Sequencing and prerequisites, Admissions criteria, Placement criteria, Advising, Tutoring, Teaching methods, Co-curricular activities Keep going

What if the results are good? Celebrate! Reward! Share! Keep going

Agenda Peer Assessment Roundtables – Student Learning 10: :10 - Welcome and Explain Process 10: :20 - Full Group Coaching - Collecting Assess. Information 10: :00 - Roundtable Discussion 1: Collecting Assess. Information 11: :05 - Full Group Coaching - Faculty Motivation/Collaboration 11: :15 - Roundtable Discussion 2: Faculty Motivation/Collaboration 11: :25 – Full Group Coaching - Sharing, Acting on Results 11: :50 - Roundtable Discussion 3: Sharing, Acting on Results 11:50 – 12:00 - Wrap Up