1. Indiana University Center for Postsecondary Research: Assessing Student Engagement & Educational Effectiveness FYE Annual Conference February 17-20, 2007 The College Student Experiences Assessment Program

2. Agenda  The IU Center for Postsecondary Research (CPR)  The College Student Experiences Questionnaire Assessment Program (CSEQ CSXQ)  National Survey of Student Engagement (NSSE, BCSSE)  How Can CPR Help Your Assessment Efforts

3. What is the IU Center for Postsecondary Research?  Designated Research Center of Indiana University & the IU School of Education  CSEQ Assessment Program (CSEQ & CSXQ)  National Survey of Student Engagement (NSSE)  Beginning College Survey of Student Engagement (BCSSE)  Faculty Survey of Student Engagement (FSSE)  Law School Survey of Student Engagement (LSSSE)  Also home to the Indiana Project for Academic Success (IPAS) and various other faculty-sponsored and grant-funded projects

4. What Does the Center for Postsecondary Research do?  Promote student success and institutional effectiveness by helping postsecondary institutions and related agencies gather and use data for decision making and institutional improvement  Variety of survey tools  Institutional data reports and custom analyses  Campus consultations (merging the quantitative and qualitative)  Foster national dialogue and stimulate further research on the undergraduate experience, assessment, educational effectiveness, and institutional improvement

5. The College Student Experiences Questionnaire Assessment Program

6. The CSEQ Assessment Program  CSEQ: The College Student Experiences Questionnaire   Measures student progress and the quality of students’ experiences inside and outside the classroom.  Administered to undergraduate students at any point following first semester of enrollment.  CSXQ: College Student Expectations Questionnaire  Gauges how students expect to participate in educational activities, experience the campus environment, and realize specific learning outcomes.  Administered to first-year students before or immediately following start of classes

7. CSEQ Survey Instrument  Developed in 1979 by Dr. C. Robert Pace at UCLA  Now in 4 th edition (5 th edition in development)  Over 180,000 4th edition cases  More than 500 institutions  Divided into four sections  Background Characteristics  College Activities Scales  Measures of the College Environment  Estimates of Gains

8. Activities Scales  Reflect quality of effort students expend in activities related to:  Library Experiences  Computer and Information Technology  Course Learning  Writing Experiences  Experiences with Faculty  Art, Music, Theater  Campus Facilities  Clubs and Organizations  Personal Experiences  Student Acquaintances  Scientific and Quantitative Experiences  Topics of Conversation  Information in Conversations

9. The College Environment  Ten scales measuring student perceptions of the campus environment:  The extent to which the campus emphasizes diverse aspects of student learning and personal development (7 items)  Relationships with faculty, administrators, and other students (3 items)

10. Estimate of Gains  Consist of student ratings of progress toward important educational goals  Goals are presented in five major clusters:  General Education, Literature, Arts, and Social Sciences  Personal Development and Social Competence  Science and Technology  Intellectual Skills  Vocational Competence

11. CSXQ Survey Instrument  Developed from the CSEQ by Dr. C Robert Pace and Dr. George Kuh in 1996 with FIPSE support  Revised in 1998 (2nd edition)  Over 61,000 students at more than 60 institutions  Divided into three sections  Background Characteristics  College Activities Scales  Measures of the College Environment

12. Activities Scales  Quality of effort students expend on activities:  Library and Information Technology  Experiences with Faculty  Course Learning  Writing  Campus Facilities  Clubs, Organizations, Service  Student Acquaintances  Scientific and Quantitative Experiences  Topics of Conversation  Information in Conversations

13. The College Environment  Ten scales measuring perceptions of the campus environment with regard to:  The extent to which the campus emphasizes diverse aspects of student learning and personal development (7 items)  Relationships with faculty, administrators, and other students (3 items)

14. Why Do Expectations Matter?  Expectations affect the actual experiences of college students by: 1)Serving as a filter through which students evaluate and select various types of educational opportunities. 2)Predisposing students to seek specific learning activities.  Student expectations and subsequent experience are not always congruent.  Ex: First-generation or low-income students may have less tacit knowledge of what college is like.  May influence students’ overall performance in college and satisfaction with their experience.

15. Why measure expectations? National focus on setting high expectations Differences between student and institutional expectations Social-psychological theories

16. CSEQ/CSXQ Administration  Paper or online version  CSEQ takes 20-30 minutes  CSXQ takes 10-15 minutes  Institution receives descriptive statistics, codebook, and SPSS files  Normative comparison data available for each  Additional services: Student Advising Reports, custom statistical analyses

17. Student Advising Report  Displays student CSEQ or CSXQ responses alongside average responses of peers  Provides direct and personalized feedback  Deepens the student/advisor relationship  Allows early intervention with at-risk students  Increases the value of assessments to students  SAR can easily be adapted to the needs of your school or program

18. The National Survey of Student Engagement

19. What is NSSE?  Annual snapshot of student participation in programs and activities that institutions provide for their learning and personal development.  The results will provide an estimate of how undergraduates spend their time and what they gain from attending college.  National Survey of Student Engagement items represent empirically confirmed ‘good practices’; they reflect behaviors associated with desired outcomes of college.  The time and energy students devote to educationally purposeful activities is the single best predictor of their learning and personal development.  Survey based on the CSEQ, developed by C. Robert Pace

20. Two Components of Student Engagement What students do -- time and energy devoted to educationally purposeful activities What institutions do -- using effective educational practices to induce students to do the right things

21. History of NSSE  Began in 1998 with aid from Pew Charitable Trust  Alternative way to look at educational quality and foster institutional improvement  Survey is conceptually based on the CSEQ, developed by Robert Pace  NSSE 2006 Administration  More than one million first-year and senior students were invited to participate in the 2006 NSSE administration.  557 institutions in the United States and Canada  331,601 students responded.

22. NSSE Instrument  Research based on effective educational practices  Designed and tested for validity and reliability  Relatively stable over time  High credibility of self- reported data  Over 200,000 students at 500 institutions annually

23. NSSE Administration  Administration Mode  Paper: We need accurate mailing addresses, letterhead, signatures  Web+: 4x the paper sample, we need e-mail and mailing addresses  Web: 5x the paper sample, we need e-mail addresses

24. NSSE Administration  Sample Size  Numbers are based on mode and school size  Oversampling can increase sample size or ensure adequate representation of populations of interest

25. NSSE Administration  Special Groups  Consortia  Schools like you  Additional Questions  Selected Peers  Schools like you  Special Populations  Identified in Pop file  Special Analysis  Post Hoc Analysis  Experimental Items  Web schools only 2006 Consortia  American Democracy Project  Associated New American Colleges  AAUDE  Association of Independent Colleges of Art and Design  Association of Independent Technical Universities  Bringing Theory to Practice  Catholic Colleges and Universities  Colleges that Change Lives  Council for Christian Colleges & Universities  Council of Public Liberal Arts Colleges  Jesuit Colleges and Universities  Private Liberal Arts Colleges and Universities  Teagle Grant Consortium  Urban Universities  Women's Colleges  State and University Systems  Ontario Universities  South Dakota Public Universities  Tennessee Publics  University of Texas  University of Wisconsin

26. NSSE Deliverables  Institutional Report (August)  Institutional data  Respondent characteristics (Demographic Information)  Means and Frequencies (item averages and response percentages)  Benchmarks of Effective Educational Practice  Additional Data (If Applicable)  Experimental items  FSSE Report  BCSSE Combined Report  NSSE Institute Information  Using NSSE Data  Accreditation Toolkit  Data Facilitator’s Guide

27. Benchmark Report (August) Level of Academic Challenge Active & Collaborative Learning Enriching Educational Experiences SupportiveCampusEnvironment Student Faculty Interaction

28. NSSE Institute  NSSE Institute is the Service Arm of NSSE  The Institute draws upon NSSE data and related research to help faculty, administrators, and governing board members discover and implement effective mechanisms for linking information about student experiences to efforts to improve academic programs and support services.  Campus Audits  Consultations  Workshops  Ongoing Research and Evaluation  Accreditation Toolkits  Regional Workshops  Pocket Guides  Facilitator’s Guide to Using NSSE Data  Additional Research Projects  Building Engagement and Attainment in Minority Students (BEAMS)  Documenting Effective Educational Practices (DEEP)

29. What is BCSSE?  The Beginning College Survey of Student Engagement (BCSSE, pronounced “bessie”)  Measures entering first-year students’ pre-college academic and co-curricular experiences, as well as their interest in and expectations for participating in educationally purposeful activities during college.  HS experiences; Importance of college activities  Baseline self-assessment of knowledge and skills  Demographics

30. BCSSE Instrument  Two pilots: 2005 and 2006  2004-2005 pilot with 28 schools, 15,890 students  2005-2006 pilot with 70 institutions, 39,986 students  Use to examine gap between expectations and engagement  Study effect of students’ background on NSSE scores  Designed as a companion to the National Survey of Student Engagement (NSSE).

31. BCSSE Registration  BCSSE 2007-2008 Registration Open!  BCSSE measures entering first- year students’ pre-college academic and co-curricular experiences, as well as their interest in and expectations for participating in educationally purposeful activities during college.  To learn more about BCSSE, go to www.bcsse.iub.eduwww.bcsse.iub.edu

32. Choosing the CPR Tool That’s Right for Your Campus

33. Only one step in assessment process Step #1: Survey Data Survey students Review results Develop preliminary list of strengths and opportunities for improvement Step #2: Feedback Share results with faculty, administrators & students Identify themes & priorities Design action plan Step #2: Feedback Share results with faculty, administrators & students Identify themes & priorities Design action plan Step #3: Action Plan Finalize plan Share plan with appropriate groups Link to strategic plan Implement action Step #4: Follow-up Use results as benchmarks to monitor progress Faculty & student focus groups Step #4: Follow-up Use results as benchmarks to monitor progress Faculty & student focus groups

34. CPR and Your Assessment Plan  Which CPR survey should I use?  CSEQ offers maximum flexibility in terms of administration timeline; student population surveyed; question customization; cost-effective  NSSE provides additional national, sector, and other comparative information; standard survey population sampling protocol and structured timeline; comprehensive institutional results report  CSXQ focuses on students expectations for their college experience  BCSSE focuses on high school academic and co-curricular engagement, with select student expectation questions

35. CPR and Your Assessment Plan  How often should I administer a CPR survey?  Every Year: Gives you a snapshot of each class  Every Three Years: Gives you a picture of a cohort at the beginning and the end of their college experiences  Every Five Years: Works well with most accreditation cycles (Accreditation and Interim Reports)  Other factors to consider  Establishing a baseline  Costs  Additional Surveys/Sources of Data  Time to take absorb results, make changes

36. CPR and Your Assessment Plan  Program Assessment  Institutional Research  Accountability  Accreditation Studies  Retention Efforts  Curricular reform  Performance Indicators  Student Satisfaction  Recruitment  How can I use the survey results?

37. CPR and Your Assessment Plan  How else can CPR help our efforts?  Consulting  Help with making sense of data  Presentation assistance  Technical questions  Result reports and research resources  NSSE- Regional users workshops

38. Regional NSSE Users Wichita Drive-in Workshop Co-sponsored by Wichita State University, Friends University, and Newman University When: April 19-20, 2007 Where: Metropolitan Complex Wichita State Campus  Please join us for the spring Drive-in Workshop to learn how colleges and universities are using NSSE as a foundation for institutional improvement.  We invite session proposals from NSSE users and encourage you to bring a team to work on an action plan for your campus.  Full workshop details will be available in January 2007 on the NSSE Institute Web site: www.nsse.iub.edu/institute/.www.nsse.iub.edu/institute/

39. Discussion and Questions Julie Williams Research Analyst and CSEQ Project Manager Michelle Salinas Holmes CSEQ Research Associate Indiana University Center for Postsecondary Research 1900 East 10th Street Eigenmann Hall, Suite 419 Bloomington, IN 47406-7512 williaj4@indiana.edu msholmes@indiana.edu  Web sites:  www.cseq.iub.edu www.cseq.iub.edu  www.nsse.iub.edu www.nsse.iub.edu  E-mail:  cseq@indiana.edu cseq@indiana.edu  nsse@indiana.edu nsse@indiana.edu Ph: 812-856-5825 Fax: 812-856-5150