Southeastern Association for Community College Research 2008 Conference Using Technology to Efficiently and Effectively Gather Information from Recent Alumni and Employers August 2008

SACCR 2008 Conference August 2008 August 6, 2008 Southeastern Association for Community College Research2 Alumni and Employer Surveys Presenters  Dr. James Coraggio, Director of Institutional Research and Effectiveness  Maggie Tymms, Assessment Coordinator for Academic Programs

SACCR 2008 Conference August 2008 August 6, 2008 Southeastern Association for Community College Research3 SPC Background  SPC, established in 1927, is the oldest 2-year college in Florida  First Community College in Florida to offer 4 year degrees (2002)  9 Campuses throughout the county  FTE: 16,086 (LD), 1,338 (UD)  Opening Spring 2008 headcount: 25,754  Annual enrollment: 62,200

SACCR 2008 Conference August 2008 August 6, 2008 Southeastern Association for Community College Research4 Institutional Effectiveness “Institutional Effectiveness is the integrated, systematic, explicit, and documented processes of measuring performance against the SPC mission for purposes of continuous improvement of academic programs, administrative services, and educational services offered by the college.” “Closing the Loop”

SACCR 2008 Conference August 2008 August 6, 2008 Southeastern Association for Community College Research5 Performance Improvement  From Compliance to Performance Improvement

SACCR 2008 Conference August 2008 August 6, 2008 Southeastern Association for Community College Research6 Agenda  Student Assessment Points  Purpose of Follow-up Surveys  Alumni/Employer Survey Process  Dissemination of Survey Results and Reports

SACCR 2008 Conference August 2008 August 6, 2008 Southeastern Association for Community College Research7 Student Assessment Points Students surveyed multiple times

SACCR 2008 Conference August 2008 August 6, 2008 Southeastern Association for Community College Research8 Purpose of Follow-up Surveys Recent Alumni Survey was designed to:  Gain insight into alumni’s perceptions of the quality of general education and career preparation  Determine the current activities of former students such as employment information and educational status

SACCR 2008 Conference August 2008 August 6, 2008 Southeastern Association for Community College Research9 Purpose of Follow-up Surveys Employer Survey was designed to:  Gain insight into an employer’s perceptions of hired SPC graduates in the areas of key general education competencies and foundational skills  Provide the employee’s major job responsibilities and wage information, as well as employer’s willingness to hire another SPC graduate  Identify employer’s willingness to support various college activities

SACCR 2008 Conference August 2008 August 6, 2008 Southeastern Association for Community College Research10 Purpose of Follow-up Surveys State Requirements  Survey data are also collected for state required annual Placement & Follow-up Report (FETPIP: Florida Education Training Placement Information Program) and supplemental information to meet workforce criteria

SACCR 2008 Conference August 2008 August 6, 2008 Southeastern Association for Community College Research11 Alumni/Employer Survey Process Previous model  Labor intensive  Manually tabulated paper surveys  Telephone interviews conducted using overtime hours

SACCR 2008 Conference August 2008 August 6, 2008 Southeastern Association for Community College Research12 Alumni/Employer Survey Process Issues with previous model  Costly survey process  Declining college budget  Increasing demand for accountability  Outdated contact information  Students over-surveyed  Declining response rates  Conducted in isolation

SACCR 2008 Conference August 2008 August 6, 2008 Southeastern Association for Community College Research13 Alumni/Employer Survey Process Goals of new survey process A.Improve data collection process B.Enhance quality of survey forms C.Include program specific information D.Increase response rates

SACCR 2008 Conference August 2008 August 6, 2008 Southeastern Association for Community College Research14 A. Improve Data Collection Process Data Collection Process  Eliminated telephone calls  Automated the process ( merge)  Employ online survey submission (Zoomerang)  Obtain accurate addresses  Use multiple points of contact

SACCR 2008 Conference August 2008 August 6, 2008 Southeastern Association for Community College Research15 A. Improve Data Collection Process Survey Schedule  Recent Alumni Survey  Graduates surveyed six months after the end of session (Three times per year)  Employer Survey  Employer of graduates surveyed at the end of academic year (Once per year)

SACCR 2008 Conference August 2008 August 6, 2008 Southeastern Association for Community College Research16 A. Improve Data Collection Process First Second Third First mailing Second mailing 2 weeks Multiple Points of Contact 2 weeks

SACCR 2008 Conference August 2008 August 6, 2008 Southeastern Association for Community College Research17 B. Enhance Quality Of Survey Forms Improved Survey Forms  Better “look and feel” of survey format  Developed scannable paper forms  Use online branching models to reduce survey length  Design items to gather only pertinent data

SACCR 2008 Conference August 2008 August 6, 2008 Southeastern Association for Community College Research18 Online Survey: and Form

SACCR 2008 Conference August 2008 August 6, 2008 Southeastern Association for Community College Research19 Paper Survey: Letter and Form

SACCR 2008 Conference August 2008 August 6, 2008 Southeastern Association for Community College Research20 C. Include Program Specific Information Program specific information  Meet with PDs  Discuss requirements for program accreditation and timelines  Include their necessary program specific items

SACCR 2008 Conference August 2008 August 6, 2008 Southeastern Association for Community College Research21 D. Increase Response Rates Create awareness and convey importance  Involve program faculty in the process  Request faculty assistance to inform students  Include a flyer with graduation materials  Coordinate with programs that hand-carry surveys to local employers

SACCR 2008 Conference August 2008 August 6, 2008 Southeastern Association for Community College Research22 Response Rates  Summer 07

SACCR 2008 Conference August 2008 August 6, 2008 Southeastern Association for Community College Research23 Response Rates  Fall 07

SACCR 2008 Conference August 2008 August 6, 2008 Southeastern Association for Community College Research24 Response Rates

SACCR 2008 Conference August 2008 August 6, 2008 Southeastern Association for Community College Research25 Response Rates

SACCR 2008 Conference August 2008 August 6, 2008 Southeastern Association for Community College Research26 Response Rates - Employer Other Responses Recent Alumni Surveys Sent Employer Surveys Sent Is a 44% Employer Response Rate Good? COE Responses With Permission Responses Without Permission

SACCR 2008 Conference August 2008 August 6, 2008 Southeastern Association for Community College Research27 Disseminating Information  College wide access to reports   Program Specific Reports  Published on the Educational Outcomes website 

SACCR 2008 Conference August 2008 August 6, 2008 Southeastern Association for Community College Research28 Disseminating Information

SACCR 2008 Conference August 2008 August 6, 2008 Southeastern Association for Community College Research29 Disseminating Information

SACCR 2008 Conference August 2008 August 6, 2008 Southeastern Association for Community College Research30 Disseminating Information

SACCR 2008 Conference August 2008 August 6, 2008 Southeastern Association for Community College Research31 Program Specific Reports

SACCR 2008 Conference August 2008 August 6, 2008 Southeastern Association for Community College Research32 SPC IE Website

SACCR 2008 Conference August 2008 August 6, 2008 Southeastern Association for Community College Research33 Lessons Learned  Do not expect that students remember their ID#  Administer the survey soon after the end of the session (six months)  Use multiple points of contact (two weeks)  Find the most recent contact information  Make students aware of the importance of the survey process (survey announcements)

SACCR 2008 Conference August 2008 August 6, 2008 Southeastern Association for Community College Research34 Lessons Learned  Partner with programs to acquire up-to- date information  Provide updates to college data systems (PeopleSoft, Alumni Foundation)  Monitor results and make changes as necessary  More accurate data (quicker turn around of results) with automated processes

SACCR 2008 Conference August 2008 August 6, 2008 Southeastern Association for Community College Research35 Future Direction  Ultimate goal is to provide Provosts (Deans) and Program Directors ‘timely’, ‘relevant’ ‘accurate’, and ‘interpretable’ information regarding their programs and their performance outcomes  Multi-measure ‘Dashboard’ style reports (Business Intelligence/Performance Measurement)

SACCR 2008 Conference August 2008 August 6, 2008 Southeastern Association for Community College Research36 Questions/Next Steps

Southeastern Association for Community College Research 2008 Conference Using Technology to Efficiently and Effectively Gather Information from Recent Alumni and Employers August 2008