1. Institutional Planning, Assessment & Research 2010 Institutional Planning, Assessment & Research Assessment Review Committee Report HCAS – Graduate Programs Heather Ries May 4, 2014 1 2013 Institutional Planning, Assessment, and Research

2. Mentoring/Review Process Committee Selection & Composition –Chair –1 Assoc. Dean (Cindy Putnam Evans) –Faculty UAC’s representing each of the broad areas natural science & math social sciences, humanities (PhD program) –Susan Beck Frazier –Belinda Patterson 22013 Institutional Planning, Assessment & Research

3. Mentoring/Review Process Cont’d 32010 Institutional Planning, Assessment & Research Meetings - November 1 (Organizational meeting) - November 18 th (After 1 st review) - December 9 th (Last minute questions) Primary and Secondary Reviews - Faculty Members’ Suggestion - 21 programs and 13 certificates - Reviewers confer before entering final review

4. 2012-13 Component Data DevelopingAcceptableProficient Outcome152760 Means of Assessment 456533 Criteria for Success 413666 Results604534 Actions Taken1022812 Follow-Up to Actions Taken 942912 42013 Institutional Planning, Assessment & Research

5. Data Visualization 52013 Institutional Planning, Assessment & Research

6. 2012-13 Best Practices – “Closing the Loop” Molecular Biology and BioTechnology Outcome: “Communication. Each student is expected to demonstrate effective oral and written communication.” Means of Assessment: “Evaluation of components of the student’s seminar on their thesis research, and the thesis itself according to a rubric. Components of the seminar evaluated are Speaking and Graphics; components of the thesis evaluated are Writing, Figures, and Tables.” Criteria for Success: “Each student’s average score for each component is at least 80% out of a possible 100.” 62013 Institutional Planning, Assessment & Research

7. Best Practices cont’d 2012-2013 results: “Thesis research seminars of 9 students were evaluated. The cohort average for oral communication (Speaking; Seminar Rubric) was 90.7 out of 100. However, two individuals did not meet the minimum standard for oral communication. Scores of the remaining 7 ranged from 89.9 – 100. All students met the minimum for visual communication (Graphics). “Theses of 9 students were evaluated. One student did not meet the standard for writing quality. Scores of the remaining students ranged from 83.3 – 95.0. The cohort average for writing quality was 88.8 out of 100. One student also did not meet the minimum for part of the visual communication (figures). All students met the minimum standard for communication via tables. 72010 Institutional Planning, Assessment & Research

8. Best Practices Cont’d Actions Taken (based on analysis of results): “An acceptable performance of the great majority of students led the faculty to maintain the current pedagogy and curriculum, and the process of mentoring by the student’s faculty advisor and other members of the thesis committee, to provide constructive feedback on students’ learning and research accomplishments.” (YIKES – Not a best practice!!!) 82010 Institutional Planning, Assessment & Research

9. Best Practices Cont’d Initiative/Change/Strategy Implemented for Program/Unit improvement: –New actions taken for all outcomes, including the one above. –New actions taken for communication outcome: Past two years – 10-11% have substandard speaking skills with an additional 20% of the students slightly above the minimum standard. Early and more comprehensive assessment of speaking skills (during first semester as part of a required research course). Similar evaluation during proposal defense. Additional training in oral communication to those needing it. Developing course in communication and graphics. For writing, more rigorous review of proposals. Coursework assigned to address writing deficiencies as needed. 92010 Institutional Planning, Assessment & Research

10. Substantive Changes Program/Unit: Geology MS Description of Changes: –Means of Assessment based on thesis –For 2011-2012 used rubrics to evaluate thesis, thesis defense –In 2012-2013 moved away from rubrics –faculty reluctance –Used more general means such as “student passed” thesis defense. –After review moving back to rubrics for 2013-2014 Justification for Changes: –Realized that needed to evaluate for specific criteria so that student learning could be improved in specific areas. 102013 Institutional Planning, Assessment & Research

11. Rubric and Review Process Feedback Feedback on the rubric and review process. –What worked for your ARC Multiple meetings Primary and secondary reviews Providing specific feedback (including suggested verbage) Faculty members increased “assessment awareness” – What difficulties were encountered? Rubric criteria open to interpretation Maintaining reviewer consistency 112013 Institutional Planning, Assessment & Research

12. Rubic and Review Process Feedback cont’d ARC’s process and progress towards addressing comments for “developing” and “acceptable” components received in the 2013 assessment review. –Follow up mostly from Associate Deans and not ARC –Bringing chairs back in Assessment orientation sessions for Chairs Chair review and sign-off –Follow up with recalcitrant departments E-mail and Phone –Meetings with individual UAC’s to address questions 122010 Institutional Planning, Assessment & Research

13. Preparations for 2013-2014 Notice to all UAC’s –Importance of correct “actions taken” –Specific example Dean’s strategy meeting Call for ‘interim reports’ on Actions Taken - About 2/3 responded - Provided feedback via e-mail and meetings - Encouraged remaining 1/3 to attend workshops Series of workshops held throughout April –Handouts showing new format –Tracdat screenshots –Table with positive and negative examples 132010 Institutional Planning, Assessment & Research