Developing Appropriate Assessment Measures and Methods for Student Learning Outcomes November 17, 2016 Cathy Sanders Director of Assessment Office of Assessment and Accreditation

Participant Introductions Name Department Motivation for registering for workshop Involvement with assessment of student learning

Workshop Learning Outcomes Participants will understand: the difference between direct and indirect measures, authentic and traditional assessments; and the importance of aligning the measure with the expected level of learning.

Direct Measures Direct measures require students to demonstrate the application of knowledge, skills, and abilities. Examples of direct measures are written papers, oral presentations, recitals, and capstone projects. Although one direct measure is required for each SLO; multiple measures are even better.

Indirect Measures Indirect measures do not require students to demonstrate the application of knowledge, skills, and abilities. Indirect measures assess opinions or thoughts (e.g., surveys and focus groups) or provide indicators of program effectiveness (e.g., retention and graduation rates). Indirect measures are optional for program student learning outcomes.

Traditional Assessments Traditional assessments are forced-choice assessments such as multiple-choice, true-false, fill-in-the-blank, and matching tests. Students typically select an answer or recall information to complete the assessment. These tests can be standardized or teacher-created.

Authentic Assessments A form of assessment in which students are asked to perform real- world tasks that demonstrate meaningful application of essential knowledge and skills. Authentic assessments ask students to analyze, synthesize, and apply what they have learned in a substantial manner, and to create new meaning for students in the process.

Traditional Assessment Model In the traditional assessment model, the curriculum drives assessment. The body of knowledge is determined first. That knowledge becomes the curriculum that is delivered. Then the assessments are developed and administered to determine if acquisition of the curriculum occurred.

Authentic Assessment Model In the Authentic Assessment Model, assessment drives the curriculum. The tasks that students will perform to demonstrate their mastery of the student learning outcomes is determined first. Then a curriculum is developed that will enable students to perform those tasks well which includes the acquisition of essential knowledge and skills.

Comparison of Traditional and Authentic Assessments Performing a Task Real-life Construction/Application Student-structured Selecting a Response Contrived Recall/Recognition Teacher-structured

Bloom’s Taxonomy Useful in developing assessment measures that align with expected level of learning. Created in 1956 by Benjamin Bloom, an educational psychologist and several of his colleagues that were interested in improving student learning. Revised later by a former student of Bloom, Lorin Anderson to better fit educational practices in the 21st century. Consists of a hierarchical framework where each level is subsumed by the higher, more complex level.

Bloom’s Revised Taxonomy: 6 Levels of Learning

Bloom’s Levels of Learning Defined Remembering Bloom’s Levels of Learning Defined Remembering Can student recall, restate, and remember learned information? Understanding Can student explain ideas and concepts? Applying Can student use the information in another situation? Analyzing Can students break information into parts to explore understandings and relationships? Evaluating Can students justify a decision or a course of action? Creating Can students generate new products, ideas, or ways of viewing things?

Creating Student creates new ideas and information using what previously has been learned. Verbs associated with Creating: Construct Design Devise Invent Compose Plan Produce Propose Use the . . . strategy to invent a new type of sports shoe. Invent a machine to do a specific task. Design a robot to do your homework. Create a new product. Give it a name and plan a marketing campaign. Write about your feelings in relation to... Write a TV show play, puppet show, role play, song or pantomime about.. Design a new monetary system Develop a menu for a new restaurant using a variety of healthy foods Design a CD, book or magazine cover for... Sell an idea Devise a way to... Make up a new language and use it in an example Write a jingle to advertise a new product. (Adapted from Dalton, 1986)  Can students generate new products, ideas, or ways of viewing things? Northern Illinois University, Faculty Development and Instructional Design Center

Sample Question Stems for Higher-level Learning: Creating Question stems for Bloom’s “Creating” level of learning: Design a…to… Devise a possible solution to… Develop a proposal which would… Construct a new model that would change… Formulate a theory for… (Pohl, 2006)

Sample Curriculum Map for Academic Program Required Course # SLO1 SLO2 SLO3 Course 1 (I) Assessment: Short Answer Test Questions   Assessment: MC Test Course 2 Assessment: Short Answer Questions Course 3 (R) Assessment: Programming Project Assessment: Paper Course 4  (R) Course 5 (M) Assessment: Capstone Research Project Assessment: Team Project Assessment: Portfolio  Legend  I=Introduced R=Reinforced  M=Mastered

Alignment of Bloom’s Taxonomy with Expected Level of Learning Introductory-level of Learning Remembering Understanding Reinforced-level of Learning Applying Analyzing Mastery-level of Learning Evaluating Creating

Recap on Measures Use direct measures that require students to demonstrate the acquisition of knowledge, skills, and abilities; Use authentic measures that, by the end of the academic program require students to analyze, synthesize, and apply what they have learned and to create new meaning; Consider the level of learning at which the student learning outcome will be taught; and Develop measures (and scoring rubrics) that align with the expected level of learning.

Documenting Methodology in Student Learning Outcome Assessment Plans and Reports

In the past, only academic programs have submitted SLO Assessment Plans and Reports. However, academic support offices within AA have been discussing their contributions to student learning outside of the classroom. Compliance Assist provides a place for academic support units as well as academic programs to develop SLO assessment plans. Template will look the same for both

Methodology Section of SLO Assessment Plan and Report Document: When, where, and how the assessment of the SLO will be administered and evaluated; The process the dept. will use to collect, analyze and disseminate assessment data to program faculty; and The process to decide changes/improvements to make on the basis of assessment data.

Questions?