How to Assess Students at the Course and Program Level Rubric How to Assess Students at the Course and Program Level Seema C. Shah-Fairbank, P.E., PhD Director of Assessment and Program Review, Division of Academic Affairs Associate Professor of Civil Engineering Sarah M. Hershman Research Technician

Learning Objective During Workshop Identify artifacts for assessment Rubric Learning Objective During Workshop Identify artifacts for assessment Identify assessment tools to evaluate artifacts Analyze assessment artifacts Introduction to strategies to close the loop Seema

Why Do We Assess? Improve our operations Ensure that students are learning the outcomes, not just the content Inform others of the contributions and impact of our program/unit Demonstrate what we are accomplishing for our students, faculty, staff, and community Support the university/college/department strategic plan and accountability activities

Learning Outcomes Program Level – Student Learning Outcomes Course Learning Outcomes Student learning outcomes clearly state the specific and measureable behaviors students will display to verify learning has occurred at the program level. Key characteristics of student learning outcomes include 1) clarity, 2) specificity, (this means they are worded with active verbs stating observable behaviors) and, 3) measurability. CLOs clearly relate to topics, assignments, and exams that are covered in the present course. CLOs should be measurable and map to SLOs for assessment. CLOs are more detailed and specific, they identify the unique knowledge and skills expected to be gained from a given course. Use action verbs aligned with Blooms Taxonomy at the cognitive levels (levels 4, 5, and 6) especially for graduate programs Observable, measurable and able to be demonstrated! Consider what you want your students to be able to do near or at graduation. By completed or participated in a program/activity/project. Develop specifically for students in your program/activity/project. Foundation from assessing the teaching and learning process. Curriculum Matrix

Artifacts for Assessment of Student Learning Rubric Artifacts for Assessment of Student Learning Artifacts/ Assignment Exam Embedded Questions within Courses Standardize Exams Program Level Exam Presentation: Oral/Poster Observations Written Assignment Project/Report/Thesis/Capstone Essay Reflection/Journals Portfolio Art Pieces/Performances Resumes Marisol

Type of Learning – Learning Outcomes What Is Required from Students Rubric Type of Learning – Learning Outcomes What Is Required from Students Assessment Artifacts/Assignments Think critically and making judgements Development of arguments, reflection, judgment, evaluation Essay, Report, Book review, Literature Review Solving problems/making plans Identify problems, define problems, analyze data, review, design experiments, plan, apply information Problem scenario; Work-based problem; Case Study Analysis; Conference paper Performing procedures/demonstrating techniques Take readings, use equipment, follow laboratory procedures, follow protocols, carry out instructions Demonstration; Role Play; Write script and produce a video); Produce a poster; Lab report Managing and developing oneself Work co-operatively and independently, be self-directed, manage time, manage tasks  Learning journal; Portfolio; Self-evaluation; Peer assessment  Designing, creating, performing Design, create, perform, produce, etc. Design project; Portfolio; Presentation; Performance  Assessing and managing information Information search and retrieval, investigate, interpret, review information Annotated bibliographies; Use of bibliographic software; Library research assignment; Data based project; Report; Essay Communication Written, oral, visual, and technical skills  Written presentation; Oral presentation; Discussions /Debates/Role plays It is vital to link the assessment methods to learning outcomes.  It is useful to think about the abilities or graduate attributes that you would like your students to demonstrate.  Assessment rubrics assist in making the assessment expectations transparent.  One over-arching UCD rubric is the UCD Level Descriptors.  For more on assessment rubrics, see Designing Grading and Assessment Rubrics.

Assessment Tools

Why Make A Rubric? Supports Learning Feedback Contains Scoring Criteria Communicates Expectation Assessing Student Work Achievement Level Supports Learning Feedback Marisol…This slide might be redundant. Consistent, Efficient & Objective Directions on What Is Good Demonstrates Ways to Improve Plan Activities Accordingly

Rubric Rubrics https://www.cultofpedagogy.com/holistic-analytic-single-point-rubrics/

Example of a Course Rubric – Assessment and Grading Seema

Rubric Example from a Course: Seema M = 86.39, SD = 6.54, Range = 20.5

Recommendation - Strategies Rubric Recommendation - Strategies Combine Assessment and Grading Levels of Achievement Assessment may only look at a few criteria Avoid Reinventing - Search for Existing Rubrics Available online Available from colleagues on campus Available from off-campus colleagues Modify existing rubrics to serve your needs Faculty and Student Affairs professionals working together to assess student learning

Exam Design Align questions to learning outcomes How many questions/items? As many as you need - Capture student learning Think about the time (student(s) may need 3x as long) Instructions Group items by test type How to record answers Whether or not to show work Point values for each item Neatness Rule of Thumb Don’t let one early incorrect answer repeatedly penalize a student Alternative: tell students to assume specific answer to prior item Student’s capacity is a limited resource

Assessment Align exams and problems that students performed on to CLO or SLO Grade/Assess if the students have met the learning Set a criterion for success – 75% of students will get each item correct Determine percent-correct for each item Exam Questions Alignment to SLO Alignment to CLO # Student Correct Incorrect 1 SLO 1 CLO 2 10 20 2 SLO 4 CLO 4 3 CLO 5 30 4 18 12 5 15 6 16 14 7 26 8 24 9 17 13 Curriculum Matrix

Evaluation of Results CLO 2 48% CLO 4 64% CLO 5 89% SLO 1 48% SLO 4 Course Level - CLOs Program Level - SLOs CLO 2 48% CLO 4 64% CLO 5 89% SLO 1 48% SLO 4 77%

Evaluate Results – Close the Loop Was the item/assignment written clearly Are there changes that need to be made within this course or a prior course Are there curricular changes needed at the program level Are there different pedogeological methods to teach the material

Assessment at Program Level Curriculum Matrix

Workload Too much assessment may lead to superficial approaches to learning (surface learning). Consider your assessment tasks as part of the overall assessment workload for the student (students are doing several modules, not just your own module). Use both formative and summative assessment, combined in continuous assessment to assess student learning. Formative assessment involves giving feedback during the module so that students can continuously develop and improve. Summative assessment sums up achievement. Too much assessment may impede providing constructive and timely feedback to students.

Questions – Comments - Practice Rubric Questions – Comments - Practice All of us Seema C. Shah-Fairbank, P.E., PhD shahfairbank@cpp.edu Marisol Cardenas marisolc1@cpp.edu

Rubric Resources https://www.cpp.edu/~academic-programs/program- review/assessment-student-learning/rubrics.shtml http://woodard.latech.edu/~kklopez/EDCI489CReadWriteThinkWeb/po dcastrubric.html http://manoa.hawaii.edu/assessment/howto/rubrics.htm#p4 https://www.aacu.org/value-rubrics