1. Selecting Appropriate Assessment Measures for Student Learning Outcomes October 27, 2015 Cathy Sanders Director of Assessment Office of Assessment and Accreditation

2. Workshop Learning Outcomes Upon conclusion of the workshop, participants will understand the difference between: formative and summative assessments; direct and indirect measures; and traditional and authentic assessments. Participants will also understand how the expected level of learning affects the selection of appropriate assessment measures for student learning outcomes.

3. Formative Assessment Formative assessment is assessment that is conducted by instructors during the learning process for the purpose of monitoring student learning and modifying teaching and learning activities to improve student learning.

4. Formative Assessment is Useful for: getting a clearer idea of where students are, and thus, where to begin instruction; revealing how well the students are following the lesson in progress; and uncovering gaps in understanding before they become serious impediments to further learning.

5. Summative Assessment Summative assessment is assessment that occurs at the end of a program to evaluate student learning by comparing it against some standard or benchmark.

6. Summative Assessment is Useful for: guiding curriculum revision; evaluating the effectiveness of educational programs; and responding to external reporting requirements (e.g., professional and regional accreditors). The focus of this workshop is on summative, program-level assessment (e.g., BS in Economics). OAA will be offering some formative, classroom assessment workshops in the spring.

7. Direct Measures Direct measures are assessments that require students to demonstrate the application of knowledge, skills and abilities. Examples of direct measures are written papers, presentations, recitals, performances and projects. At least one direct measure is required for each program student learning outcome. Multiple measures are encouraged.

8. Indirect Measures Indirect measures do not require students to demonstrate the application of knowledge, skills, and abilities. Indirect measures either assess opinions or thoughts (e.g., surveys and focus groups) or provide indicators of program effectiveness (e.g., retention and graduation rates). Indirect measures are optional for program student learning outcomes.

9. Traditional Assessments Traditional assessments are forced-choice assessments such as multiple-choice, true-false, fill-in-the-blank, and matching tests. Students typically select an answer or recall information to complete the assessment. These tests can be standardized or teacher-created.

10. Authentic Assessments A form of assessment in which students are asked to perform real- world tasks that demonstrate meaningful application of essential knowledge and skills. Authentic assessments ask students to analyze, synthesize and apply what they have learned in a substantial manner, and to create new meaning for students in the process. Authentic Assessment Toolbox Website (Jon Mueller) http://jfmueller.faculty.noctrl.edu/toolbox/whatisit.htm#definitions

11. Traditional Assessment Model In the traditional assessment model, the curriculum drives assessment. The body of knowledge is determined first. That knowledge becomes the curriculum that is delivered. Then the assessments are developed and administered to determine if acquisition of the curriculum occurred.

12. Authentic Assessment Model In the Authentic Assessment Model, assessment drives the curriculum. The instructor first determines the tasks that students will perform to demonstrate their mastery of the student learning outcomes. Then a curriculum is developed that will enable students to perform those tasks well which includes the acquisition of essential knowledge and skills.

13. Defining Attributes of Traditional and Authentic Assessments Traditional Selecting a Response Contrived Recall/Recognition Teacher-structured Indirect Evidence Authentic Performing a Task Real-life Construction/Application Student-structured Direct Evidence

14. Aligning the Assessment Measure with the Learning Expected level of learning also affects the design of an assessment measure. Since program-level student learning outcomes are assessed at the mastery level, more complex assessment measures are appropriate. Bloom’s Revised Taxonomy is useful in developing assessment measures that align with expected level of learning.

15. Background on Bloom’s Taxonomy Created in 1956 by Benjamin Bloom, an educational psychologist and several of his colleagues that were interested in improving student learning. Revised later by a former student of Bloom, Lorin Anderson to better fit educational practices in the 21 st century. Consists of hierarchical frameworks where each level is subsumed by the higher, more complex level.

16. Bloom’s Revised Taxonomy

17. Remembering The learner is able to recall, restate and remember learned information. Verbs associated with Remembering: Describe Find Identify List Recall Retrieve Name Locate Recognize Reproduce Can students recall information? Northern Illinois University, Faculty Development and Instructional Design Center

18. Understanding Student grasps meaning of information by interpreting and translating what has been learned. Verbs associated with Understanding: Classify Compare Exemplify Explain Translate Infer Interpret Paraphrase Summarize Expand Can students explain ideas or concepts? Northern Illinois University, Faculty Development and Instructional Design Center

19. Applying Student makes use of information in a context different from the one in which it was learned. Verbs associated with Applying: Implement Carry out Illustrate Solve Use Execute Examine Classify Can students use the information in another familiar situation? Northern Illinois University, Faculty Development and Instructional Design Center

20. Analyzing Student breaks learned information into its parts to best understand that information. Verbs associated with Analyzing: Attribute Compare Deconstruct Find Integrate Organize Outline Structure Can students break information into parts to explore understandings and relationships? Northern Illinois University, Faculty Development and Instructional Design Center

21. Evaluating Student makes decisions based on in-depth reflection, criticism and assessment. Verbs associated with Evaluating: Check Critique Detect Experiment Prioritize Hypothesize Judge Monitor Test Recommend Can students justify a decision or a course of action? Northern Illinois University, Faculty Development and Instructional Design Center

22. Creating Student creates new ideas and information using what previously has been learned. Verbs associated with Creating: Construct Design Devise Invent Compose Plan Produce Propose Can students generate new products, ideas, or ways of viewing things? Northern Illinois University, Faculty Development and Instructional Design Center

23. Sample Questions Stems for Lower-level Learning: Understanding Question stems for Bloom’s “Understanding” level of learning: How would you classify…? Compare and contrast… Which statements support…? How would you summarize…? What facts or ideas show…? (Pohl, 2000)

24. Sample Question Stems for Higher-level Learning: Creating Question stems for Bloom’s “Creating” level of learning: Design a…to… Devise a possible solution to… Develop a proposal which would… Construct a new model that would change… Formulate a theory for… (Pohl, 2006)

25. Summary: for Program-Level Student Learning Outcomes… Use direct measures that required students to demonstrate the acquisition of knowledge, skills and abilities. Use authentic measures that ask students to analyze, synthesize and apply what they have learned and to create new meaning. Develop the curriculum around the student learning outcomes that department faculty have identified as most important.

26. For Program-Level Student Learning Outcomes… Determine the level of learning expected in the student learning outcome (e.g., application, analysis, evaluation or creation). Design assignments, assessments and analytic scoring rubrics with the expected level of learning in mind. Reference verbs and question stems for each of Bloom’s levels of learning provided in your workshop folder.

27. Questions and Comments