1 Assessment: the Good, the Bad, and the Ugly Kenneth E. Fernandez Assistant Professor Department of Political Science University of Nevada, Las Vegas

2 Learning Outcomes: the Good 1. Developed through input from all faculty 2. COLA Learning Objectives used as a model 3. Articles written in Journals 4. UNLV Assessment Workshop 5. Reexamined objectives after initial assessment reports

3 Learning Outcomes: the Bad 1. We chose not to require faculty to list standardized outcomes on syllabi 2. In the beginning we had 23 learning outcomes (not manageable). 3. We have 2 MA programs and a new Ph.D. program but currently are assessing graduate student learning outcomes together (not necessarily a problem).

4 Measuring Outcomes: The Good Undergraduate program: 1. Pre- and Post-test for all core courses - 10 item, multiple-choice test - Developed by faculty in those fields 2. Exit Survey for graduating seniors - Self-Administered questionnaire - Both closed and open-ended questions

5 Measuring Outcomes: The Good Graduate program uses multiple measures: 1. Grades 2. Comments on Teaching Evaluations 3. Focus group and/or interviews with graduate students 4. Graduate Faculty comments 5. With new PhD program we now have comprehensive exams after 18 credits

6 Measuring Outcomes: The Bad Undergraduate program: 1. Pre- and Post-tests - a 10 item, multiple-choice test is limited - limited resources (test projected on screen) - tests are anonymous and aggregated 2. Exit Survey for graduating seniors - Some questions not appropriate or useful - Data entry is time consuming

7 Measuring Outcomes: The Bad Graduate program: 1. Grades are not an effective external assessment mechanism 2. Students are at different stages 3. Not easy to do a careful aggregate study of outcomes every semester. Not enough students completing qualifying exams, prospectus, or theses, presenting at conferences. 4. Not easy to contact part-time students

8 Other Issues 1. What is the appropriate unit of analysis? - student (track individual student progress) - teacher - class (or even a lecture) - fields, GE courses, etc. - programs or entire department - semester, year, student career - how many reports are needed (BA, MA, EPS, PHD, GE, semester, yearly)?

9 Assessment Reports: The Good 1. Forces self-examination (teacher, course, field, department) 2. Several curricular changes due to assessment data collection and analysis 3. Placing reports online is helpful to share assessment approaches (at least to assessment officers)

10 Assessment Reports: The Bad 1. No one wants to do them. 2. Potential to develop an acceptable report template and then put it on autopilot and then ignore it. 3. Lack of resources creates obstacle to applying a rigorous method to data collection and analysis 4. This can create strange anomalies.

11 The Ugly: Standardization We should avoid standardization A request to place the GE international learning outcome in all PSC 211&231 syllabi: “Students will [demonstrate proficiency in a foreign language or] explain how international cultures, societies, or political economics relate to complex, modern world systems.”

12 The Ugly: Standardization The faculty who teach these courses felt such a learning outcome did not match the true learning objectives and would lead to confusion. Should all courses with the same prefix or fulfill a GE requirement have the same learning outcomes on their syllabi?

13 The Good News 1. UNLV’s Assessment Office has been supportive and flexible. 2. The office provides guidance rather than regulations or mandates. 3. It promotes a dialogue about teaching and its connection to learning that ironically is sometimes missing in a department setting.

14 Recommendations 1. Effective assessment needs funds (pre/post-test, focus groups, etc.) 2. Yearly reports rather than semester reports (helps reduce anomalies) 3. Maintain current flexibility. We do not want to impose standardization. 4. Use technology to make assessment easier (clickers, web-based surveys, etc.).

15 Appendix