Student Learning Outcomes Best Practices in Writing Student Learning Outcomes Jill Allison Kern, PhD Director of Assessment & Accreditation Eastern Washington University June 2018

Learning Outcomes for Today’s Training At the end of today’s training, participants should be able to do the following: Distinguish among course, program, and institution learning outcomes State the two necessary and sufficient criteria that make a student learning outcome assessable Write an assessable student learning outcome for their course or program Provide an example of an assignment they could use to assess the student learning outcome they generate Use the recommendations put forth in this presentation to evaluate the soundness of a student learning outcome

Learn Scales Play Bach

What Are Student Learning Outcomes? The knowledge and skills students are required to demonstrate at the end of a learning unit

are NOT: NOT What Student Learning Outcomes Inputs (e.g., readings, lectures, experiential exercises, internships) Student achievement outcomes (e.g., post-grad employment, graduate school admission, student- authored publications) Program goals (e.g., increase course offerings, earn programmatic accreditation, improve retention and graduation rates of majors) Self-reports (i.e., students’ reports of how much they learned about X) Inputs: The educational experiences designed to generate learning (e.g., what students will read) Attendant-educational outcomes: The desired consequences that may result from completing an educational program (e.g., getting a job, gaining entry into a graduate program, being an informed citizen) Self-reports: Students’ judgements about their own learning

3 levels ClO’s PlO’s IlO’s

Bedrock of Assessment Edifice

Poorly designed SLOs

Weak Foundation

Fortify the Bricks

Equation for Building Sound SLOs

“Hook” and “Ladder”

The “Hook” Students who successfully complete [name the learning unit] should be able to do the following:

Hook Examples CLO hook PLO hook ILO hook Upon the successful completion of Psychology 101, students should be able to do the following: Graduates from the Computer Engineering BS program Eastern Washington University should be able to do the following: Those who earn an undergraduate degree at EWU should be able to: CLO hook PLO hook ILO hook

List of skills and knowledge The “Ladder” List of skills and knowledge

Visual reminder; Not a hierarchy Ladder I call it a ladder as a visual reminder of a bullet-pointed list. The term “ladder” is not meant to imply a hierarchy or progression of skills. Visual reminder; Not a hierarchy

Hook Upon the successful completion of the BS degree in Computational and Applied Mathematics at EWU, students should be able to: COMP MATH-PLO – 1: Solve problems using calculus. COMP MATH-PLO – 2: Solve problems using real analysis. COMP MATH-PLO – 3: Describe the logical structure of proofs. COMP MATH-PLO – 4: Construct proofs of core mathematical results. COMP MATH-PLO – 5: Develop viable mathematical models of physical or economic phenomena. COMP MATH-PLO – 6: Utilize a programming environment to conduct a successful simulation. Ladder

Hook Graduates of EWU’s BA program in Studio Art should be able to: Create a body of work that demonstrates conceptual intent. Create a body of work that demonstrates technical skill. Critically evaluate art work using appropriate terminology. Produce a professional quality portfolio that includes an artist’s statement, résumé, and documentation of their work. Ladder

necessary & sufficient For each rung to be assessable, it needs to meet two necessary & sufficient criteria:

Assessable SLOs Observable Single barreled

Criterion 1: Observable * Starts with a verb that describes an action the student will engage in that a fac member can see or hear at a particular moment in time If a faculty member can’t see or hear it, he or she can’t assess it. *by the faculty member

Skills Perform a monologue Mix chemicals Create a lesson plan Conduct an audiology exam Sing an aria Build a robot Write a press release Program a computer Compose an original jazz score Prepare a financial statement Take a patient history

What is inside the student’s head is not directly observable to the faculty member

Examples of Non-observable Verbs * Become familiar with Be knowledgeable about Think Value Realize Become aware of See Know Learn Understand Comprehend Appreciate Study Experience *Don’t use these!

Examples of Observable Verbs To Assess Student’s Knowledge Describe Explain List Present Write Formulate Create Analyze Arrange Critique Synthesize Predict

Convert Not Observable Observable Know the arguments Reflect on the issues Think critically Understand the principles Comprehend the methods Appreciate art Summarize the arguments Discuss one’s reflections on the issues Question the assumptions implicit in theories Apply the principles Explain the methods Describe the aesthetic properties of art works that appeal to you

Criterion 2: Single-Barreled * *One action

* Double-Barreled Students who earn a degree in French should be able to write and speak fluent French. Just like a survey item. You’ve had the experience of a survey that asks you something like, to what degree was the teacher helpful and the class interesting? Well, if the teacher was helpful but the class was boring, you’re stuck on how to answer the question. It’s exactly the same principle with double barreled SLOs. Your measurement will conflate two things creating ambiguous results. * not assessable

* Single-Barreled Students who successfully earn a degree in French should be able to write fluently in French. Students who successfully earn a degree in French should be able to speak fluently in French. *assessable

Multi-Barreled Students who successfully earn a degree in French should be able to write fluently in French about literature, philosophy, and science. Students who successfully earn a degree in French should be able to write fluently, persuasively, and expressively in French.

Avoid “and”

However, SLOs that connect two or more verbs with “ ” are . assessable

Those who earn a degree in music should be expected to compose complex scores or play an instrument skillfully.

Requirements Describes an action that’s observable to a faculty member at a moment in time Describes one action only

Recommendation 1: Be succinct

Explain how inequality between identity groups is perpetuated Rather than . . . Examine the ways in which economic, political and social systems perpetuate and reinforce inequality based on differences in class, race, culture and gender, and access to power. Oy! Go with . . . Explain how inequality between identity groups is perpetuated

Students who earn a BA in Economics from EWU should be able to: Apply the cost/benefit principle to a current socio- economic issue. Use supply/demand modeling to explain an economic event. Identify the appropriate economic concept to resolve a policy issue.  

Avoid comparative verbs or adjectives Recommendation #2 Avoid comparative verbs or adjectives

Avoid Improve Improved Enhance Enhanced Deepen Deepened Strengthen Strengthened Develop Developed

Examples of “Comparative” SLOs Those who earn a degree in computer science should improve their capacity to solve software engineering problems. Graduates of the BA program in Sociology are expected to demonstrate an enhanced ability to pose researchable questions about societal problems.

DOUBLES YOUR WORK! Pre Post You need to collect pre- and post-measures, which makes assessment more laborious and time intensive. It’s harder to sustain. If you want to collect pre- and post-data because it’ll give you insight into the skills and knowledge students have at the start of the quarter or program, I certainly wouldn’t stop you. I’m just saying that it’s not necessary provided that you follow the next best practice I’m going to discuss in a bit, which is to craft learning that you can reasonably assume most students wouldn’t be able to demonstrate at the start of the course or program. Pre Post

Doesn’t set threshold

Convert Non-Comparative Comparative Those who earn a degree in computer science should improve their capacity to solve software engineering problems. Those who earn a degree in computer science should be able to solve software engineering problems.

Collect “post” data only

Unlikely acquired prior to learning unit Recommendation #3 Specify Skill Body of knowledge, or Level of mastery Unlikely acquired prior to learning unit

Example Upon successful completion of EWU’s BA program in Art History, students should be able to produce original art history research.

Recommendation #4 For every learning outcome statement you write, you should be able to create a graded activity that flows logically from the observable verb that begins each rung of the SLO ladder.

When writing your SLOs, think about what you will ask students to do When writing your SLOs, think about what you will ask students to do. Then choose the SLO’s verb.

Well designed assignments are key to good assessment

Well designed SLOs are key to good assignments

Keep list of SLOs short About four to six CLOs for a three-unit course Recommendation #5 Keep list of SLOs short About four to six CLOs for a three-unit course About five to seven PLOs for a major About five to seven ILOs for a degree (GE outcomes)

SLOs Are Precious Real Estate

Your PLOs: Meaningful Necessary & sufficient

The End!

SLO Checklist Do the SLOs have a “hook” and “ladder”? Does each rung in the ladder start with a verb that can be observed by the instructor at a specific moment in time? Is each rung in the ladder single-barreled? Do all rungs in the ladder avoid using verbs or adjectives that imply a comparison between two points in time (e.g., develop, improve, transformed, strengthened)? Does each SLO specify a skill, body of knowledge, or degree of mastery that one would anticipate that students would not have acquired prior to enrolling in the course or starting the program? Is each SLO succinct? Could each SLO lead logically to an assignment? Is the number of rungs on the ladder a manageable number to assess—that is, about four to six CLOs for a four-unit course or about five to seven PLOs for a program?