1. International Outcomes Assessment Dr. Barbara Wheeling Montana State University Billings Coordinator for Institutional Assessment College of Business Director of Accreditation and Assessment October 1, 2010

2. Outline Montana State University Billings2  International Learning Objectives: Examples  Primary Concepts of Assessment  Developing Learning Objectives  The Process  Key Questions for Writing Learning Objectives  Assessment Tasks  Closing the Loop

3. International Learning Objectives Example from MSU Billings: Montana State University Billings3  We want our students to be able to  1. Identify global issues and explain their implications.  2. Specify appropriate factors for consideration when assessing international markets and operations.  3. Recognize significance of cultural differences on business practice.

4. International Learning Objectives Example from Merrimack College: Montana State University Billings4  Demonstrate the ability to function effectively in an international environment.  Demonstrate the ability to apply international perspectives to local business decisions.

5. International Learning Objectives Additional Examples Montana State University Billings5  Identify differences in domestic and global business practices.  Explain the effects of exchange rates on transactions, weights and measures.  Identify a foreign market for a product or service.  Create a foreign market entry plan.  Demonstrate knowledge of global legal framework and trade regulations.  Distinguish between appropriate and inappropriate behavior in different cultural contexts.

6. International Learning Objectives Examples from Adelphi University Montana State University Billings6  Our students should be able to identify global and multicultural issues and explain their implications.  Our students should be able to make recommendations based on global and multicultural issues.

7. Primary Concepts of Assessment Montana State University Billings7  The learning objectives should clearly state what the learner should be able to do.  The assessment should measure if they can, in fact, do that.

8. Developing International Learning Objectives Montana State University Billings8  Look for indicators of a global perspective:  Knowledge about importing/exporting, trading partners and policies.  Analysis of foreign country’s business environment and proposing strategies that fit with local laws.  Identification of ethical issues.  Identification of problems in a cross-cultural business situation and recommendation of solutions.

9. The Process Montana State University Billings9  Step 1: What should students be able to do?  These are the objectives.  Step 2: What indicates students have met the objective?  These are assessment tasks.  Step 3: What does good performance on the task look like?  These are the criteria to assess.

10. Montana State University Billings10  Step 4: How well did the students perform?  Use a rubric with the criteria or  Compile a score for each objective  Step 5: How well should most students perform?  The minimum level at which you would want students to perform is a benchmark.  Step 6: What do students need to improve upon?  Information from the rubric will provide feedback and ideas for improving instruction.

11. Key Questions for Writing Learning Objectives Montana State University Billings11  1. Is it specific?  An objective is written too broadly if  It cannot be reasonably assessed with just one or two assessments  It covers several different elements of the subject matter from a course or semester

12. Montana State University Billings12  2. Is it observable and measurable?  Examples, not measurable:  “Students will understand how to divide two-digit numbers.”  “Students will develop an appreciation of cultural diversity in the workplace.”  Example, measurable:  “Students will correctly divide two-digit numbers.”  “Students will summarize in writing their feelings about cultural diversity in the workplace.”

13. Montana State University Billings13  Avoid phrases such as,  “have an understanding”, “have an appreciation for”, “be knowledgeable about”  Be careful of modifiers such as,  “will effectively”, “can accurately”, “should completely”  These can make measurement impossible

14. Montana State University Billings14  3. Is it actually a teaching outcome?  Avoid phrases such as:  “will be taught”, “will learn how to”, “will be evaluated on”  4. Does the objective include action verbs?  Overt behavior that can be observed and measured  Examples: compile, create, plan, revise, analyze, design, select, utilize, apply, prepare, use, compute, discuss, explain, predict, assess, compare, rate, critique.

15. Assessment Tasks Montana State University Billings15  Case study  Video assignment  Research paper International experience learning journal Knowledge test

16. Closing the Loop Montana State University Billings16  International course  Course in each discipline  Globalization across the curriculum  Virtual syllabus  Experiential exercises  Example: role playing in a cultural context  Study abroad

17. Q & A Montana State University Billings17