1. 1 Module 1: Developing and Validating Learning Outcome Statements Presented at CSU San Marcos Peggy Maki PeggyMaki@aol.com April 25 and 25, 2007

2. 2 Integrated Learning…. Cognitive Affective Expressive Psychomotor

3. 3 Mission/Purposes Learning Outcomes How well do students achieve our outcomes? Gather Evidence Interpret Evidence Enhance teaching/ learning; inform institutional decision- making, planning, budgeting The Process

4. 4 Task 1: Developing and Validating Student Learning Outcome Statements: What Is a Learning Outcome Statement?  Describes learning desired within a context  Relies on active verbs (create, compose, calculate, construct, apply) (See handout)  Emerges from our collective intentions over time

5. 5  Can be mapped to curricular and co- curricular practices (ample, multiple and varied opportunities to learn over time)  Can be assessed quantitatively or qualitatively during students’ undergraduate and graduate careers

6. 6  Is written for a course, program, or institution

7. 7 Levels of Learning Outcome Statements Institution-level Outcome Statements Department-,Program-, Certificate-level Outcome Statements Course/Service/Educational Experience Outcome Statements

8. 8 Distinguishing between Objectives and Outcomes  Objectives state overarching expectations such as— Students will develop effective oral communication skills. OR Students will understand different economic principles.

9. 9 Learning Outcome Statement from ACRL Literate student evaluates information and its sources critically and incorporates selected information into his or her knowledge and value system. ONE OUTCOME: Student examines and compares information from various sources in order to evaluate reliability, validity,accuracy, timeliness, and point of view or bias.

10. 10 Quantitative Literate Graduates according to MAA Should be Able to: 1. Interpret mathematical models such as formulas, graphs, tables, and schematics, and draw inferences from them. 2. Represent mathematical information symbolically, visually, numerically, and verbally. 3. Use arithmetical, algebraic, geometric, and statistical methods to solve problems.

11. 11 4. Estimate and check answers to mathematical problems in order to determine reasonableness, identify alternatives, and select optimal results. 5. Recognize that mathematical and statistical methods have limits. (http://www.ma.org/pubs/books/grs.html) The Mathematics Association of America (Quantitative Reasoning for College Graduates: A Complement to the Standards, 1996).

12. 12 Ethics—Students should be able to…  Identify and analyze real world ethical problems or dilemmas, and identify those affected by the dilemma.  Describe and analyze the complexity and importance of choices that are available to the decision-makers concerned with this dilemma

13. 13  Articulate and acknowledge their own deeply held beliefs and assumptions as part of a conscious value system  Describe and analyze their own and others’ perceptions and ethical frameworks for decision-making  Consider and use multiple choices, beliefs, and diverse ethical frameworks when making decisions to respond to ethical dilemmas or problems. California State University Monterey Bay: University Learning Requirements, 2002

14. 14 Sample APA Learning Outcome Statements  1.3: Use the concepts, language, and major theories of the discipline to account for psychological phenomena.  1.4: Explain major perspectives of psychology (e.g., behavioral, biological, cognitive, evolutionary, humanistic, psychodynamic, and sociocultural). http://www.apa.org/ed/critique_study.html

15. 15 Sample Biology Learning Outcome Statements  Diagram and explain the major cellular processes in eukaryotes and prokaryotes.  Apply the scientific process, including designing and conducting experiments and testing hypotheses. (See additional examples in handouts)

16. 16 Ways to Articulate Outcomes  Adapt from professional organizations  Derive from mission of institution/program/department/service  Derive from students’ work

17. 17  Derive from ethnographic process  Derive from exercise focused on listing one or two outcomes “you attend to”  Draw from taxonomies, such as Bloom’s

18. 18 Draft an outcome statement that captures what your graduates should be able to demonstrate, represent or produce at the end of their major program of study or GE. For example, at the end of X-program, students should be able to__________

19. 19 How well do your outcome statements meet characteristics of a good statement? (See handout) Ask the person next to you to apply the characteristics of a good outcome statement to your outcome statements; then discuss that person’s assessment of your statements. How might you improve those statements?

20. 20 Developing Maps and Inventories—internal validation of learning outcome statements Reveal how we translate outcomes into educational practices offering students multiple and diverse opportunities to learn Help us to identify appropriate times to assess those outcomes Identify gaps in learning or opportunities to practice

21. 21  Help students understand our expectations of them  Place ownership of learning on students  Enable them to develop their own maps or learning chronologies (See handout)

22. 22 How will you use maps and inventories?  Discuss how you are or will go about the process of developing a curricular or curricular-co-curricular map and how you will label peoples’ entries.  Discuss how you might also use inventories of practice in your mapping process.

23. 23 External Validation  Advisory boards  Recent alums  Survey of individuals in a field  Professional organizations

24. 24 Before you Begin to Assess:  Identify your outcome(s) for the level you wish to assess: Institution-level Department-level or program-level Course-level or service-level

25. 25 What Do You Want to Learn about Teaching and Learning?  Pedagogy (problem-based, experiential, didactic)  Curricular or relevant course(s) design or co- curricular design  Instructional design (computer-based, for example)

26. 26 What Do You Want to Learn about Teaching and Learning?  Pedagogy (problem-based, experiential, didactic)  Curricular or relevant course(s) design or co- curricular design  Instructional design (computer-based, for example)

27. 27  Use of educational experiences  Use of out-of-course assistance, such as tutorials

28. 28 Questions about Pedagogy in promoting….  Recall and recognition  Transfer  Integration  Synthesis  Application and re-application  Use and re-use  Change in perspective or understanding?  Learning about alternatives to face-to- face instruction

29. 29 What Other Available Program- or Institution-level Data Might You Need?  National surveys  Maps or inventories of practice  Scores on standardized tests  Surveys or interviews  Standardized test results  Other data your School stores? ----------------------------

30. 30 What Other Data (besides Student Work) Do You Believe will be Helpful When You Interpret Results  From Institutional sources (NSSE, Noel- Levitz…)  From Students themselves  From educators at your institution  Others?