1. Implementing Peer Review in Student Affairs Annual Planning at an MSI Doctoral Research University
J. Judd Harbin, Ph.D. University of Nevada, Las Vegas
Monday, March 5, Convention Center, 116

2. Introduction
Time zones photo source:

3. 5:00 a.m. for me
Coffee image source:

4. Parameters
Focusing on review of annual planning at the department-level
Peer review refers to internal process within the division

5. Peer Review Audience Participation What comes to mind? History
Strengths
Limitations

6. Overview of Creating Peer Review at UNLV
Prior to 2014
Planning Cycle
Collaboration with Assessment Committee and Leadership Group
Implementation
Refinement
Refinement

7. Prior to 2014 Planning Definitions Planning Template (PDF)
Campus Life Pillars (long-term aspirational pursuits)
Aligned departmental goals with pillars
Aligned annual “strategies” with departmental goals
Supervisor Review

8. Pillar Dept Goal Unit Goal Strategy Immediate Term (0 – 3 years)
Aspirational
Dept Goal
Long Term (5 to 10 years)
Unit Goal
Shorter Term (3 to 5 years)
Strategy
Immediate Term (0 – 3 years)

9. Components of a Strategy
Strategy Statement
Leadership (Assignment)
Timeline
Budget
Assessment Plan
Metrics
Learning Outcomes

10. Coordinator
Supervisor
Department Director
Associate VP

11. Director of Assessment
Associate VP
Director of Assessment

12. 2014-15 Review Enter new Director of Assessment in Campus Life (me)
AVP advised me to review all strategies to gain familiarity
Bonus:
Provided baseline estimate of planning skill

13. 2014-15 Review: Solo Review Using existing template and definitions,
For each strategy submitted,
Did the staff member provide adequate information for each section?

14. Ratings Each Component Each Strategy Accept as Written
Accept with Minor Edits
Revision Needed
Incomplete
Each Strategy
Overall rating matched the lowest Section Rating

15. Audience “Incomplete” Why do these pose a special problem?
Often had good information in many if not all completed sections
BUT missed one or more sections

16. Results (2014-15) Strategies reviewed: 150 Portion Incomplete: 8.7%
Count: 13
Portion Accepted: 26.7%
Count: 40
Red X source:
Green check source:

18. Director of Assessment
Coordinator
Supervisor
Department Director
Associate VP
Director of Assessment

19. Collaboration with Leadership & Staff
Summarizing results of and my notes from solo review
What kinds of feedback have staff had in the past?
What kinds of feedback would be useful going forward?
How to improve the quality of plans developed and submitted?

20. Enter Assessment Committee

21. Composition of Committee
Departmental Representatives
Residential Life
Student Union
Campus Recreational Services
Facilities (including technology)
Student Engagement & Diversity
Most reps had served since 2012
Now entering 3rd year

22. Role of Assessment Committee
Consider the results of my review of the annual plans
Discuss with their directors and departmental colleagues
What hindered quality of plan in the past?
What will help improve quality of plans in the future?
Identify and implement staff development plan to improve planning.

23. Audience
What do you think the committee representatives heard from their directors and fellow departmental staff?
Lack of feedback about prior plans
Wondering if anyone ever read their plans
Unsure how much detail to include

24. Committee’s Staff Development Plan
Create guidelines for self-evaluation
Use director’s notes
Identify the most common mistakes or omissions
Develop questions to help staff gauge level of detail needed
Annual reminder training for all staff during July
Provide feedback to staff every year

25. Coordinator
Supervisor
Department Director
Assessment Review
Associate VP

26. Enter Peer Review

27. Director Initiated
Participated in peer review of academic program assessment plans
Collegial / Collaborative peer review process
Eager even enthusiastic committee response at the prospect of peer review!
AND… they had questions

28. Committee Collaboration
What would the criteria be for peer review?
How many strategies do we review each year?
Does everyone on committee review every strategy?
Do we review every Campus Life pillar every year?
Do we review every department every year?
When do we do this?
How do we do this?

29. Decisions
Use the guidelines for self-evaluation as frame peer review criteria
Review all departments each year
Review all pillars each year
Two day retreat
Divide time proportionately among departments
Depending on number of strategies submitted each year
Not enough time to peer review every strategy
Use feedback on the reviewed strategies to revise/update the other ones

30. Timeline: July
Staff members begin drafting their plans for the year.

31. Timeline: August
Staff members review draft plans with their supervisors and directors.
Committee Orientation
Welcome new members to committee
Review self-evaluation guide
Review peer review form
Discuss process
Schedule departmental feedback sessions during late September
By 31st – Draft plans are due

32. Timeline: September Peer Review Retreat
ASAP after Labor Day
Two full days
Prepare one-page summary for each department
Complete ten (10) departmental feedback sessions
Negotiate timeline for final plans from director

33. Timeline: October
Most final plans are due by September 30 but some will be due in early October
Review final plans to verify needed changes were made
Follow up as necessary

34. Process Collective review
Track changes and comments directly in planning documents (Word)
Track section and strategy ratings in Google Form
Departmental feedback meetings
Director of Assessment plus the Department Representative to Committee
Director of Department plus any staff they invite
Negotiated timeline for final plans

35. Reminder of Results (2014-15)
Strategies reviewed: 150
Portion Incomplete: 8.7%
Count: 13
Portion Accepted: 26.7%
Count: 40
Red X source:
Green check source:
35.4%

36. Subsequent Results (2015-16)
Strategies reviewed: 82
Portion Incomplete: 7.3%
Count: 6
Portion Accepted: 80.5%
Count: 66
Red X source:
Green check source:
87.8%

37. Image: https://netcommunity.unlv.edu/SrAVPDevelopment
Feeling AWESOME!
Image:

38. Subsequent Results (2016-17)
Strategies reviewed: 56
Portion Incomplete: 32.1%
Count: 18
Portion Accepted: 41.1%
Count: 23
Red X source:
Green check source:
73.2%

39. WHAT HAPPENED?! New hires
Large department restructured its plans starting in July but didn’t finish

40. New Hires Revise onboarding Workshop strategy writing
Emphasize resources available for strategy writing
Consultations
Guidelines for self-evaluating strategies

41. Subsequent Results (2017-18)
Strategies reviewed: 58
Portion Incomplete: 32.8%
Count: 19
Portion Accepted: 39.6%
Count: 23
Red X source:
Green check source:
73.2%

42. WHAT HAPPENED?!
Large department (same as year before) restructured into 3 departments
New Hires (added positions)
Two new directors
Four new assistant directors
Additional new hires

43. RESPONSE for AY2019 Cycle
Reiterate with the executives overseeing each department the need to forward complete strategies for review
Provide additional training for new supervisors and directors so they know what to look for during their reviews
Focus new employee training even more to workshop strategies

44. Costs Peer Review Retreat Follow-Up Meetings Final Review = ~2 hours
2 full days (19 clock hours)
9.5 staff members (180.5 staff hours)
Large conference room (SmartRoom) [No charge]
Beverages, Snacks, Lunches (~$600)
Follow-Up Meetings
10 summary 0.5 hour each = 5 staff hours
10 1 hour varying staff in each = ~72 staff hours
Final Review = ~2 hours

45. Questions & Discussion
J. Judd Harbin, Ph.D.
(702)
LinkedIn: juddharbin

46. Thank you for joining us today!
Please remember to complete your online evaluation following the conference.
See you in Los Angeles in 2019!