1. Assessment Driven Orientation: 5 Key Components for Success NASPA Region II Conference June 9, 2015
Dr. Mitchell A. Levy
Vice President of Student Affairs & Branch Campus Management
Atlantic Cape Community College
Higher Education Consultant
Adapted from Levy, M., & Thomas, M. (2011). Orientation Webinar presented for Paper-Clip Communications, Inc., 11/30/10.
Reframing faculty resistance: McGowan, Levy, & Polnariev, 2010

2. 5 Key Components for Success:
Determine Purpose
Identify Clients
Employ Client-Centered Approach
Use Data for Program Revision
Use Data to Assess Outcomes- utilize feedback-loop to inform program revisions
Reframing faculty resistance: McGowan, Levy, & Polnariev, 2010

3. #1:Determining purpose of orientation
Why do we have an orientation program?
What are we trying to accomplish? Why?
What are we trying to change?
What are our unspoken assumptions?
Reframing faculty resistance: McGowan, Levy, & Polnariev, 2010

4. #1:Determining purpose of orientation
Most dangerous sentence in Higher Education:
“Because that’s how we have always done it”
Reframing faculty resistance: McGowan, Levy, & Polnariev, 2010

5. #2: Identify Clients
Student v. Parent/Family v. Institution- Centered?
One-shot (everything now) v. Critical Junctures (developmental/longitudinal)?
Multiple clients?
Who are institutional/internal clients?

6. #2: Identify Clients
Levy, Polnariev & McGowan. The Integration of Reality-based Programming to Address “Schizophrenic” Support Services. Published on Student Affairs eNews; September 18, 2011.
Institutional Delusion: Belief what a small group of students say, on an occasional or limited basis, is representative of the larger student community:
“Clearly, if students do not have a voice on a continuous and significant basis, then we are essentially developing programs which may be unfounded, unnecessary, and/or ineffective. In fact, without more student ownership of the process, our assumptions of what students want and need may be merely a mirage (hallucination)”.

7. Paradigm shift… #3: Client Centered Approach
Needs assessment- what does client want? Need?
Current students, parents, alumni involved in planning?
Are client goals/needs more important than institutional agenda?
Feedback loop- client assessment drives program revisions?***

8. Where do we start? #4: Use Data for Program Revision
Case Study: LaGuardia CC
Challenge: Increase engagement, workshops addressing greatest concerns.
Response: Utilize continuing student data (n = 235) to design NSO workshops.

9. #4: Use Data for Program Revision Values Auction Activity

10. #4: Use Data for Program Revision Values Auction Activity
Please answer the following questions?
Why do you think your students should attend college?
How would you like them to be different when they graduate?

11. Must be built into program, not developed after program ends
#4: Use Data for Program Revision
Must be built into program, not developed after program ends
What changes do we notice?
How are clients different?
What did we learn? (Must be Specific)
“One-shot” vs. Developmental/Longitudinal assessment?
Who are internal clients? How does data inform institutional planning?

12. #4: Use Data for Program Revision
Challenge: Utilize longitudinal, developmental model informed by student responses, increase linkages between NSO and other initiatives.
Response: Revised workshop schedule to a) address topics relevant for 1st semester based on continuing student data (n=235) and NSO student data (n=1100+) and b) offer workshops earlier in semester promoting a greater linkage w/ the NSO program.
Challenge: Based on data, increase sense of “belonging” for new students via increased engagement in Academy/Major structure.
Response: Students provided colored T-shirts representing one of the three academies; sit in “Academy Section” of theater. Students meet with program faculty or advisor in designated “major group.”

13. #4: Use Data for Program Revision
Challenge: On evaluations, frequent comment workshops “too crowded.”
Response: Next the NSO was replicated on two days, ratio of students to staff and faculty.
Response: Workshops offered in larger, multiple sites providing greater access and comfort.
Challenge: Provide increased opportunity to identify and discuss individual goals for attending college.
Response: Replaced the Common Reading with goal clarification activities. Each facilitator used one activity (Values Auction, Values Bingo, Values Dialogue) inviting students to identify & prioritize academic and career goals while meeting other students and faculty in their majors/programs.

14. #4: Use Data for Program Revision
Challenge: Increase students access to / utilization of resource materials.
Response: Provide each student w/ pre-loaded flash drive, in the color of their academy containing student handbook and links to resources.

15. #5: Use Data to Assess Outcomes
What are you trying to measure?
What do you want to know/learn?
Who do you want to learn about?
What will you do with the data?
What outcomes are we expecting?

16. #5: Use Data to Assess Outcomes
Developmental Advising Assessment Instrument (Levy, et.al., 2009):
Did you learn something new? If so, what?
Did you benefit the way you wanted? If so, how?
Are you motivated to take action? If so, what?
Any suggestions for improving?

17. #5: Use Data to Assess Outcomes
What happened? What did we learn?
Values Auction:
Spring 2010:
Item #1: “How valuable was opportunity to think about & discuss your goals and reasons for attending college?,” 98% (n=345) indicated “Very” or “Somewhat” (72% responded “Very”, 25% responded “Somewhat”).
Fall 2010:
Item #1: “How valuable was the opportunity to think about and discuss your goals and reasons for attending college?,” 98% (n=274) indicated “Very” or “Somewhat” (79% responded “Very”, 19% responded “Somewhat”).
93% of respondents stated “Yes, this activity strengthened and/or reinforced my (your) commitment to be successful in college?”

18. #5: Use Data to Assess Outcomes
Values Bingo
Spring 2010: 98% (n=130) responded “Very” or “Somewhat” re: “How valuable was the opportunity to think about & discuss your goals and reasons for attending college?” (76% responded “very” and 22% responded “Somewhat”).
Fall 2010: 97% (n=576) responded “Very” or “Somewhat” re: “How valuable was the opportunity to think about and discuss your goals & reasons for attending college?” (65% responded “Very” , 32% responded “Somewhat”).
86% of respondents stated “Yes, this activity strengthened and/or reinforced my (your) commitment to be successful in college?”

19. #5: Use Data to Assess Outcomes
Values Dialogue
Spring 2010: 100% (n=39) responded “Very” or “Somewhat” re: “How valuable was the opportunity to think about and discuss your goals & reasons for attending college?” (87.5% responded “Very”, 12.5% responded “Somewhat”).
Fall 2010: 98% (n=238) responded “Very” or “Somewhat” re: “How valuable was the opportunity to think about and discuss your goals & reasons for attending college?” (73% responded “Very”, 25% responded “Somewhat”).
88% of respondents state “Yes, this activity strengthened and/or reinforced my (your) commitment to be successful in college?”

20. #5: Use Data to Assess Outcomes
Survey Data: Assessment of Activity Facilitator Experiences- Fall 2010
n=54 of the 64 facilitators completed surveys (84.4%)
37 (68%) were faculty, 17 (31%) were staff.
46% conducted Values Bingo, 20% Values Dialogue, 22% Values Auction.
7% combination of Values Bingo w/ Values Dialogue
4% combination of Values Bingo w/ Values Auction
96% “felt adequately prepared to conduct the activity.”
100% “believed activities and discussion were valuable for new students.”
Mean = 4.41 (40% indicated 5.0): “in your opinion, how did students respond to the activity & discussion you facilitated today?”
{1=very negatively to 5= very positively}
100% “would be willing to participate again in New Student Orientation.”
*Speaks to importance of organization, training, and support for staff

21. #5: Use Data to Assess Outcomes
Analysis of Pedagogy:
The fall 2010 activity w/ highest % of students selecting “Very Valuable” and highest % of students stating “activity strengthened and/or reinforced their (your) commitment to be successful in college” was the Values Auction.
Values Auction provides more freedom to identify individualized goals and provides more structured focused reflection and team-building than the Dialogue activity.

22. #5: Use Data to Assess Outcomes
New students participating in Orientation (spring 2009) which did not utilize self-reflection programming (pre-revisions) were 10% more likely to enroll next semester (fall 2009) vs. new students who did not participate.
New students participating in Orientation (spring 2010) which did utilize self-reflection programming (post-revisions) 16% (+6% w/in one year) more likely to enroll next semester (fall 2010) vs. new students who did not participate.
Continuing student registration +47% in vs. same date in
Continuing student enrollment 100%+ of target throughout fall 2010 registration process.

23. #5: Use Data to Assess Outcomes
Current initiatives at Atlantic Cape Community College:
Increased focus on career development (guided pathways- CCRC)
Developmental Model- critical junctures, Just-In-Time
Pre-Enrollment
Non-cognitive factors
Create developmental success checklist
Curricular infusion- Career Maturity Rubric (Levy, et.al. 2009)
Use data to change marketing and recruitment
Use data to program success centers
K-12 partnerships

24. Measures of Goal Achievement
Getting started at your institution…
Goals
Goal Activities
Measures of Goal Achievement
Target Date
What are we trying to achieve?
How will we achieve?
Who is necessary [key personnel] ?
How will we know that we have succeeded? 24

25. Feel free to contact me for further consultation and/or training
Thank you for your participation!
Feel free to contact me for further consultation and/or training
Mitchell Levy 25