1. Direct and Indirect Measures INPUTS OUTCOMES

2. Assessment Basics: Overview Characteristics of learning outcomes Introduction to assessment tools Validity of methods Choosing assessment methods Lessons learned

3. Importance of measurable learning objectives  Facilitates meaningful evaluation  Provides a common language  Use tools to facilitate the process  Involve your key constituents  Communicate with students

4. Educational Objectives Responsible Citizenship Educational Outcomes Performance Criteria Ethics 1)Demonstrate knowledge of professional code of ethics 2)Evaluate the ethical dimensions of a problem in their field

5. Program Learning Outcome: “Students will demonstrate effective oral communication skills”

6. Performance Criteria: Acceptable Standard of Performance Effective oral communication  Personal appearance is appropriate  Speaks clearly and with sufficient volume  Achieves rapport with audience  Uses engaging vocalization  Responds effectively to questions and comments  Uses audience-appropriate vocabulary, content, and style

7. Public Speaking Evaluation Sheet Student: __________________________________ Date: ________ Title of Presentation: _____________________________________ Evaluation Scale: Yes, a lot (  ) 4 3 2 1 0 (  ) No, not at all CriteriaScoreTotals Presentation Style: 1. Personal appearance is appropriate _____ 2. Speaks clearly and with effective volume _____ 3. Achieves rapport with audience _____ 4. Uses engaging vocalization _____ 5. Responds effectively to questions and comments _____ 6. Uses audience-appropriate vocabulary, content, and style _____ Presentation style total:______ Content: 7. Uses the grammar of standard English _____ 8. Presentation includes introduction, body, and conclusion _____ 9. Organizes content logically and sequentially _____ 10. Presents ideas and arguments clearly and logically _____ 11. Uses appropriate audiovisual materials _____ 12. Cites sources appropriately _____

8. Direct Measures Direct measures provide for the direct examination or observation of student knowledge or skills against measurable learning objectives

9. Indirect Measures Indirect measures of student learning ascertain the perceived extent or value of learning experiences

10. Assessment Methods  Standardized exams  Locally-developed exams  Oral exams  Performance appraisals  Simulations  Written surveys and questionnaires  Exit and other surveys  Focus groups  External examiners  Behavioral observations  Archival records  Portfolios

11. Assignment -- Jigsaw  Study your assessment method(s)  Prepare a three (3) minute tutorial for your method(s)  It is not your responsibility to convince others that your method is the right one … only teach them about the method.  Join your original group  You will have three (3) minutes to instruct your team members for each method

12. Jigsaw continued  After all members have shared their methods, read the scenarios provided  Choose at least two, preferably three methods (at least one direct method) that would be appropriate to assess the outcome of the scenario  Use flip chart to record your decisions, with rationale

13. Validity  relevance – the assessment option measures the educational outcome as directly as possible (direct line of sight?)  accuracy – the option measures the educational outcome as precisely as possible  utility – the option provides formative and summative results with clear implications for educational program evaluation and improvement

14. Summary of “Bottom Lines”  All assessment options have advantages and disadvantages  Crucial to use multi-method / multi-source approach to maximize validity and reduce bias of any one approach  “Ideal” method means those that are best fit between program needs, satisfactory validity, and affordability (time, effort, and money)

15. Assessment Method Truisms  There will always be more than one way to measure any learning objective  No single method is good for measuring a wide variety of different student abilities  There is generally an inverse relationship between the quality of measurement methods and their ease of use  It is important to pilot test to see if a method is appropriate for your program