Assessment of Student Learning An Integrated View

Why Assess? “When we care about something , we measure it.” Diane Halpern

…to improve student learning. Why Assess? …to improve student learning.

…to improve student learning. Why Assess? …to improve student learning. “Assessment is an ongoing process aimed at understanding and improving student learning…systematically gathering, analyzing, and interpreting evidence to determine how well performance matches those expectations and standards … using the resulting information to … improve performance … assessment can help us focus our collective attention, examine our assumptions, and create a shared academic culture dedicated to assuring and improving the quality of higher education” (Angelo, AAHE Bulletin, November 1995, p. 7).

…to improve student learning. Why Assess? …to improve student learning. “Assessment is an ongoing process aimed at understanding and improving student learning…systematically gathering, analyzing, and interpreting evidence to determine how well performance matches those expectations and standards…using the resulting information to … improve performance … assessment can help us focus our collective attention, examine our assumptions, and create a shared academic culture dedicated to assuring and improving the quality of higher education” (Angelo, AAHE Bulletin, November 1995, p. 7).

Multiple Assessment Stakeholders General Education SLO Assessment Program-specific SLO Assessment Course-specific SLO Assessment

What is an SLO? Student Learning Outcome (SLO) – An outcome that describes what a student is expected to learn as a result of participating in academic activities or experiences. * Operational phrasing of learning outcomes is a crucial part of the process * SLOs focus on knowledge gained, skills and abilities acquired or demonstrated, and attitudes or values changed.

Where do SLOs come from? University Vision/Mission/Strategic Priorities The University is committed to assist students in their search for knowledge and understanding and to prepare them with the attitudes, skills, and habits of lifelong learning in order to assume responsibility in a democratic community and to be useful members of a global society. College Mission & Goals The College mission is to…deepen understanding of behavioral, social, cultural, and environmental issues.  Program-specific Goals Students will exercise critical thinking, problem solving strategies, and scientific methodology in their academic studies and everyday lives. Course-specific Goals “Students will be able to effectively critique ‘lay persons’ and ‘technical’ article on the same topic.” (Method: Embedded; Evaluation Criteria: Rubric; Benchmark: 70% of students rated as at least “adequate”) Note. Accredited programs usually have their own competency-based/SLO requirements. GE-related SLOs may also have to be taken into consideration.

Selecting which SLOs to Assess Consider the stakeholders General Education / Pathway SLOs Program-specific SLOs Course-specific SLOs Key questions to consider What knowledge, skills, and values is a graduate from the program expected to acquire? How to determine whether the student has achieved some minimum standard of performance?

The Big Picture Creating a Curriculum Matrix Cross index courses and program SLOs Specifies the courses where competency for each SLO is “introduced”, “practiced”, and “demonstrated” Guides which courses are appropriate SLO assessment targets Helps to ‘close the loop’ Should specify where SLO competencies are Introduced Practiced Demonstrated

Criminal Justice (BA) Curriculum Matrix Criminal Justice Course Outcome 1 Critical Thinking Outcome 2 Oral Communication Outcome 3 Effective Writing Outcome 4 Content Knowledge Outcome 5 Understand Diversity Outcome 6 Understand Ethics POLS 250 Introduce (I) I POLS 257 Practice (P) P POLS 259A POLS 352 Demonstrate (D) POLS 353 POLS 459D P, D I, P D SOCI 384 Electives POLS 331 PHIL 332 CJ & the Individual CJ & Society Theoretical Foundations Courts & CJ System BA in Criminal Justice curriculum matrix. Shown with permission.

Economics (BA) Curriculum Matrix Student Learning Outcomes Courses Goal: I = Introduce P = Practice C = Competency Be able to apply the microeconomic theory to explain and predict price changes and economic behavior in individual markets. 101, 102, 103 301, 330, 335, 340, 253/333/ 355, 360, 365, 370, 375, 410 435, 440, 450, 462, 465, 466, 470, 495 I P C 2. Be able to apply mainstream macroeconomic theory to explain and predict events in the aggregate economy, including roles played by fiscal and monetary policies. 302, 330, 70, 375, 376, 410 431, 470, 483, 495 3. Be able to identify economic issues and problems, gather data needed to evaluate them, and analyze the data to gain insights into economic behavior and formulate possible solutions. 301, 302 335, 340, 352/333, 360, 365, 376, 389 431, 435, 440, 450, 462, 465, 495 466, 470, 481, 482, 483, 495 4. Be able to communicate with written and spoken word in the discipline of economics. 301, 302, 305, 307, 320, 340, 352/333, 355, 360, 365, 370, 376, 399, 410 431, 435, 440, 450, 462, 465, 466, 481, 483, 495 BA in Economics curriculum matrix. Shown with permission.

Operationalizing SLOs “write clear, correct, and appropriate sentences and paragraphs, avoiding major grammatical and semantic problems.” What are your thoughts about this SLO?

Operationalizing SLOs “write clear, correct, and appropriate sentences and paragraphs, avoiding major grammatical and semantic problems.” Is vague (what constitutes ‘appropriate’?) Is an example of ‘double barreling’ Rubric evaluation of an embedded assignment is the most likely method and evaluation

What do you want to know? Three broad SLO categories Knowledge SLOs (most commonly assessed) Bloom’s Taxonomy for example of Knowledge domains to assess Skill SLOs (quite commonly assessed) Writing and Oral communication are typical Skill SLOs Values / Attitude SLOs (least commonly assessed) Attitude surveys have been successfully used

It’s all about specificity Data targeted for assessment should align with competencies specified in the SLO Rubrics, rubrics; everywhere rubrics Information should be ‘actionable’ Does evaluation of the data provide information as to where improvements can be made? Bottom line… can you close the loop?

Setting Achievement Standards Benchmark acceptable standard of achievement (e.g., based on rubric) Percentage meeting some a priori level of achievement (e.g., 70% of students achieving at least ‘adequate’ on rubric; 70% of students correctly answering embedded question, etc.)

Thinking about Student Learning Outcomes: Closing the Loop

Assessment Methods and Data Embedded Assessment Target exam items, assignments, key course components Common final (not just overall grade; target exam concepts) Pre- and Post-test comparisons (“big idea” items) Third Party Assessments Professional exams Portfolios Attitude Surveys

Assessment Schedule Develop an overall SLO plan Align with program’s 5 year review or other review cycle Consider evaluating more than one SLO per AY cycle A single artifact (e.g., final paper) can be used to assess more than one SLO

Final Thoughts Clearly operationalize your SLOs What you measure should allow you to “Close the Loop” Assess all SLOs within a review cycle Make it a point to share assessment results during department meetings and/or with other stakeholders

“If we care about something, then we should measure it.” Diane Halpern