What is the Answer?? Larry Dietz, PhD Doug Fraser, Student Voice Cheryl Presley, PhD Rosemary Simmons, PhD Gertrude Stein Story

Collaboration and Partnership in Successful Assessment: Everybody Wins

Goal of Assessment To improve learning and retention of our students To improve the quality of services delivered to our students To move assessment from a model of “reactive response” to “student centered response”

Cross Campus Strategies- Why? The success and ‘health’ of any given environment really requires a buy-in by all members of that environment. There are physical, financial, and personnel challenges that make thinking about learning outcomes and student success in a more holistic manner, almost impossible There are real physical barriers, time barriers and commitment barriers to starting the dialogue between and among agencies and departments on campuses as well as the apparent competing goals from agency to department. Funding issues are also first and foremost in peoples minds. State appropriations and revenue streams are relatively small compared to the needs of the campus as a whole. Accountability is seen sometimes as a means to point fingers rather than to improve the culture in addition to being an add on to already overloaded faculty and staff efforts.

Ideally How Would it Be? Everyone would have, first and foremost in their mind, quality improvement of their programs so that students benefited Assessment would be looked upon as an opportunity to ‘make things even better’ We would be fearless and honest with each other and reduce over lap…thereby, free up funds, free up competition, free up personnel

Cross Campus Design-Objectives Provide ‘real time’ information, enhancing ability to respond and act Develop outcomes for services and programs Increase student engagement and learning Provide stronger programs that have relentlessly studied themselves Improve communication and collaboration among departments Enhance accountability and increase the ability to respond

How Did We Think About It? Student Affairs is a very large unit within the SIUC system Each department is headed by a director in charge of many staff and budget $$ Essential, since there is a mix of funding sources (state, federal, fees) that those with fewer funding sources have equal opportunity to come to the assessment table.

What Steps Did We Take? Demonstration helps understand the process, but really not the details of the technology

Step 1 Demonstration of the program Demystify the process Meet with each of the Directors or their designees Prepare surveys that were part of the tool kit (it helps to have them printed in hard copy so that folks can read them) Have them bring surveys that they currently use.

Step 2 Error is entered into the results from poorly constructed items without focus on the purpose of the assessment Review items and purpose of the assessment with the directors Determine the type of survey: Needs Assessment Satisfaction Outcome

Step 3 Match their needs with the correct instrument Determine how to draw their sample to enhance the reliability and validity of the results Determine the time line when they want to do the assessment Determine whether the methodology is PDA or On-line Web based “Hang” the instrument with Student Voice

Step 4 Let it happen Having one person who coordinates all of this relieves directors of the tasks that are time consuming and they can move forward with business as usual

What is StudentVoice? Created by students, we only work in Higher Education… Technology Resources Consultation Gertrude Stein Story StudentVoice makes data easy to collect, interpret, share, integrate, and organize.

Data Collection: PDA Randomly intercept students at busy campus locations Position PDAs at points of service Use multiple PDAs after a campus event or program Access data immediately after PDA is hot synced Gertrude Stein Story

Data Collection: Online Send an email invitation with a link Post a survey on an existing website Data is available in real-time once a survey is submitted Gertrude Stein Story

Campus Assessment Portal Central, secure access point for projects Simple analysis tools Export Capabilities Assessment Toolkits Gertrude Stein Story

Making the Process Easy StudentVoice can provide assistance at any stage during the project lifecycle. Establishing measurable goals and objectives Questionnaire design Exposure to best practice(s) and “toolkits” Training on mobile and web data collection Determining sample size Gauging response rates Technical support Reacting to data Using analysis tools Incorporating data into reports and presentations Preparation Data Collection Gertrude Stein Story Results

SIUC Counseling Center Learning Outcomes Survey

Difficulties with past assessment experiences Finding time to design and create instrument Administration of instrument Counselors forget to give them to clients Waiting room chaotic if pre-appointment Cuts into session time Post-appointment, counselors forget Small response rate if mailed, plus costly Finding time to Enter and Analyze data Creating Reports

Theoretical Frame for Learning Outcomes Desired outcomes of Youth Skill Development Programs – knowledge, attitude, skills, and behaviors Personal/Social Competence Health/Physical Competence Cognitive/Creative Competence Career/Vocational Competence Citizenship Competence Aspects of Self – connections, confidence, and character (K. J. Pittman, 1997)

Theoretical Frame for Learning Outcomes Learning Reconsidered(2004); 2 (2006) Cognitive Complexity Knowledge Acquisition, Integration, Application Humanitarianism Civic Engagement Interpersonal and Intrapersonal Competence Practical Competence Persistence and Academic Achievement

Launching of Survey Creating items based on theoretical frame 24 Items created Demographic items reflective of current agency data, plus type of service Submitting information to StudentVoice Items, Demographics, & Method Reviewing and Approving survey Gathering client Emails Sending Email with link to web-based survey

Response Rate 217 emails sent 80 responses in first 10 days of survey 2nd email two weeks later 111 responses total (51.2% response rate for completed surveys) (56.7% began survey or 123 clients)

Result Highlights Presentation of Data Frequency, Graph, Cross Tab Strength of Services (Strongly Agree/Agree) <85%, 5 items <80%, 2 items <75%, 7 items <70%, 3 items <65%, 3 items Areas for Improvement (Disagree/Strongly Disagree) 41 – 52 %, 4 items

Use of Data Review with my staff Problem solve on how to increase our effectiveness for the four items needing improvement Share success with administrators Annual Report Special Reports regarding student learning, retention, satisfaction, etc.