1. NSSE:RetrospectiveandProspective George Kuh UCO Regional Users Conference October 6, 2005

2. Overview 1.Why student engagement matters 2.What we’ve learned 3.Current activities & next steps 4.Discussion

3. We value what we measure Wise decisions are needed about what to measure in the context of campus mission, values, and desired outcomes.

4. What Really Matters in College: Student Engagement Because individual effort and involvement are the critical determinants of impact, institutions should focus on the ways they can shape their academic, interpersonal, and extracurricular offerings to encourage student engagement. Pascarella & Terenzini, How College Affects Students, 2005, p. 602

5. Foundations of Student Engagement Time on task (Tyler, 1930s) Quality of effort (Pace, 1960-70s) Student involvement (Astin, 1984) Social, academic integration (Tinto,1987, 1993) Good practices in undergraduate education (Chickering & Gamson, 1987) College impact (Pascarella, 1985) Student engagement (Kuh, 1991, 2005)

6. Good Practices in Undergraduate Education (Chickering & Gamson, 1987; Pascarella & Terenzini, 2005 ) Student-faculty contact Student-faculty contact Active learning Active learning Prompt feedback Prompt feedback Time on task Time on task High expectations High expectations Experiences with diversity Experiences with diversity Cooperation among students Cooperation among students

7. The Student Engagement Trinity  What students do -- time and energy devoted to educationally purposeful activities  What institutions do -- using effective educational practices to induce students to do the right things  Educationally effective institutions channel student energy toward the right activities

8. National Survey of Student Engagement Community College Survey of Student Engagement National Survey of Student Engagement (pronounced “nessie”) Community College Survey of Student Engagement (pronounced “cessie”) College student surveys that assess the extent to which students engage in educational practices associated with high levels of learning and development

9. The College Student Report NSSE’s Survey Instrument The College Student Report Student Behaviors Institutional Actions & Requirements Reactions to College Student Background Information Student Learning & Development

10. Documenting Good Practice InstitutionalImprovement Public Advocacy NSSE Core Activities NSSE Core Activities

11. NSSE Evolution Year Colleges & Universities 2000276 2001321 2002366 2003437 2004473 2005530

12. NSSE 2000-2005 Participating Colleges and Universities

13. NSSE Project Scope  900,000 students from 970+ different schools  77% of 4-yr U.S. undergraduate FTE  50 states, Puerto Rico, Canada  70+ consortia

14. State & University Consortia California State U U of Missouri CUNY U of New Hampshire ConnecticutNew Jersey U of HawaiiU of North Carolina Indiana USouth Dakota KentuckyTexas A&M MarylandU of Texas U of MassachusettsU of Wisconsin West Virginia

15. Effective Educational Practices Level of Academic Challenge Active & Collaborative Learning Enriching Educational Experiences SupportiveCampusEnvironment Student Faculty Interaction

16. Customized Institutional Report Overview Overview Institutional data Institutional data Means and frequencies Means and frequencies 1st year students and seniors 1st year students and seniors Comparisons by peers, Carnegie, national Comparisons by peers, Carnegie, national National benchmarks National benchmarks Data use tips Data use tips CD with raw data, etc. CD with raw data, etc. And more! And more!

17. What have we learned so far?

18. Grades, persistence, student satisfaction, and engagement go hand in hand

19. First-year students* Seniors* Academic Challenge.60.46 Active & Collaborative Learning.23.09 Student Faculty Interaction.28.37 Enriching Educational Experiences.53.48 Supportive Campus Environment.38.26 NSSE & Graduation Rates *All correlations are significant at p<.01

20. Does institutional size matter to engagement? Yes, size matters. Smaller is generally better.

21. Benchmark Scores for All Students by Undergraduate Enrollment

22. Academic Challenge, Active Learning, & Student-Faculty Interaction by Enrollment

23. Student engagement varies more within than between institutions.

25. Academic Challenge by Institutional Type Seniors 25 30 35 40 45 50 55 60 65 70 75 Doc Ext Doc Int MA Bac LA Bac Gen Nation First-Year Students 25 30 35 40 45 50 55 60 65 70 75 Doc Ext Doc Int MA Bac LA Bac Gen Nation Benchmark Scores

26. Academic Challenge at Two Public Universities

27. Worth Pondering How do we reach our least engaged students?

28. Who’s more engaged?  Women  Fraternity & sorority members  Full-time students  Students who live on campus  Learning community students  Students with diversity experiences

29. Effects of Learning Communities on Engagement

30. Who Is Most Likely to Experience Diversity? More Students of color Traditional-age students Women First-year students Less White students Older students Men Upper-division students

32. College Outcomes Quiz What percent of 1999-2000 college graduates attended two or more institutions? (a) 14% (b) 26% (c) 33% (d) 42% (e) 59% e. 59%

33. Reasons for Concurrent Enrollment 4% 9% 12% 17% 21% 23% 47% 0%5%10%15%20%25%30%35%40%45%50% Prepare to transfer Take extra courses Course not available l, Other reasons Easier courses Better course schedule l Finances Finish degree sooner

34. Documenting Good Practice InstitutionalImprovement Public Advocacy NSSE Core Activities NSSE Core Activities

35. The Language of Effective Educational Practices NSSE Pocket Guide NSSE Pocket Guide

37. The Language of Effective Educational Practices NSSE Pocket Guide NSSE Pocket Guide High School Survey of Student Engagement (HSSSE) High School Survey of Student Engagement (HSSSE) Beginning College Student Engagement Survey (BCSSE) Beginning College Student Engagement Survey (BCSSE) Law School Survey of Student Engagement(LSSSE) Law School Survey of Student Engagement(LSSSE)

38. First-Year Student Use of Campus Services

39. NSSE Institute Aims  Document and share success stories to help schools improve  Work with institutions: workshops workshops campus audits campus audits consultations consultations research & evaluation projects research & evaluation projects Sponsored by The Carnegie Foundation for the Advancement of Teaching with support from Lumina Foundation for Education and the Wabash College Center of Inquiry in the Liberal Arts.

40. Building Engagement and Attainment of Minority Students (BEAMS) 5-year project funded by Lumina Foundation for Education5-year project funded by Lumina Foundation for Education Alliance for Equity in Higher Education institutionsAlliance for Equity in Higher Education institutions Using student engagement data to guide change initiativesUsing student engagement data to guide change initiatives Provides resources for improvement initiativesProvides resources for improvement initiatives

41. Project DEEP To discover, document and describe what high performing institutions do and how they achieved this level of effectiveness.

42. Other Current Activities Accreditation tool kit Accreditation tool kit AASCU “American Democracy” AASCU “American Democracy” “Foundations of Excellence” “Foundations of Excellence” Collegiate Learning Assessment (RAND-CAE) Collegiate Learning Assessment (RAND-CAE) “Connecting the Dots” “Connecting the Dots” Collaborative research Wabash, with CCSSE, other scholars Collaborative research Wabash, with CCSSE, other scholars NCES “student success” synthesis NCES “student success” synthesis Linking BCSSE with NSSE Linking BCSSE with NSSE

43. NSSE Goes International Canada (G-10, Ontario consortium)Canada (G-10, Ontario consortium) MacedoniaMacedonia AustraliaAustralia KazakhstanKazakhstan EnglandEngland SpainSpain Future collaborations planned for: IrelandIreland LebanonLebanon RussiaRussia

44. Institutional Reflection Areas of EffectiveEducationalPractice Question or Improvement

45. What Can We Do? Administrators Students Faculty Members

46. “NSSE is a great way to stimulate reflection and debate about what we do more and less well, and why. For us it’s proving an exciting and enlivening tool for self-reflection and self- improvement.” Michael McPherson, President, Spencer Foundation (former President of Macalaster College) Stimulating Campus Conversations

47. 1. Get the ideas right Focus on a real problem  Persistence  Under-engaged students  Fragmented gen ed program  Tired pedagogical practices  Poor first-year experience  Low academic challenge  Connections to real world  Capstone experiences

48. Fayetteville State University Did NSSE in 2002 & 2003 Did NSSE in 2002 & 2003 Used NSSE data as a way to bring together people concerned about data on campus Used NSSE data as a way to bring together people concerned about data on campus Data confirmed effectiveness of FSU’s Freshman Year Initiative Data confirmed effectiveness of FSU’s Freshman Year Initiative

49. Iowa State University IndicatorIndicator DescriptionFY01FY0205 GOAL (Target) ISU 6Level of academic challenge (NSSE) *based on 100 pt. scale 46.9 (1st year) 48.6 (Seniors) (2000) 49.7 (1st year) 50.5 (Seniors ) (2001) 54.0 (1st year) 56.0 (Seniors) ISU 8Enriching educational experiences (NSSE) *based on 100 pt. scale 47.0 (1st year) (2000) 53.8 (1st year (2001) 59.0 (1st year) Iowa State linked NSSE results to Board of Regents Performance Indicators

50. 2. Align initiatives:  Institutional mission, values, and culture  Student preparation, ability, interests  Resources and reward systems  Existing complementary efforts NSSE & BEAMS NSSE & BEAMS AASCU American Democracy Project AASCU American Democracy Project AAC&U “Greater Expectations” AAC&U “Greater Expectations” Gen ed reform Gen ed reform Carnegie Campus Clusters/SOTL/CASTL Carnegie Campus Clusters/SOTL/CASTL Service learning/Campus Compact Service learning/Campus Compact Internationalization and diversity Internationalization and diversity

51. 3. Cultivate grass roots buy-in  Leaders endorse, but don’t dictate  Structures not (nearly) as important as relationships

52. Getting grass roots buy-in Confirm/corroborate results Confirm/corroborate results Drive data down to dept level Drive data down to dept level Gain consensus on student engagement priorities Gain consensus on student engagement priorities Faculty Survey of Student Engagement Faculty Survey of Student Engagement

53. Faculty Survey of Student Engagement (pronounced “fessie”) FSSE measures faculty expectations and activities related to student engagement in effective educational practices

54. FSSE and NSSE Point to “Disorienting Dilemmas” Situations in which usual perspectives or ways of responding do not work or don’t fit are more likely to motivate us to learn and change (Mezirow, 1990)

55. What Are Faculty and Students Telling Us?

56. FSSE-NSSE Gap Analysis  About two-thirds (65%) of faculty expect students to spend more than 25 hours preparing for class think  Only about one-fifth (20%) think that students spend this amount of time actually  Only about one in ten (12%) students actually spends this much time

57. Prompt Feedback FACULTY gave prompt feedback often or very often STUDENTS received prompt feedback often or very often 93% | 93% Lower Division Upper Division 64% | 76% 1 st yr. Students Seniors

58. Course Emphasis FACULTY report very much or quite a bit of emphasis on memorizing STUDENTS report very much or quite a bit of emphasis on memorizing 14% | 29% Lower Division Upper Division 65% | 63% 1 st yr. Students Seniors

59. Faculty Priorities and Student Engagement

60. Faculty Priorities and Selected Student Outcomes

61. How Men and Women Faculty Use Class Time Men Women How Men and Women Faculty Use Class Time Men Women

62. What to make of this? 1.When faculty members emphasize certain educational practices, students engage in them to a greater extent than their peers elsewhere. 2.Good things go together

63. University of Missouri- St. Louis  NSSE introduced to New Faculty Teaching Scholars  Workshops held with academic leaders to link results with University’s strategic indicators  NSSE items in course evaluations  Open forum to get student input about ways to improve learning environment

64. 4. Fashion data-informed monitoring systems  Use multiple sources of data ACT/SAT score reports BCSSE NSSE FSSE CIRP/CSS Noel Levitz CLA ACT CAAP

65. Other Data Sources Locally-developed measures National instruments –Cooperative Institutional Research Program (CIRP) –Your First College Year (YFCY) –College Student Experiences Questionnaire (CSEQ) –Noel Levitz Student Satisfaction Inventory –ETS Major Field Tests –ACT Collegiate Assessment of Academic Proficiency –Collegiate Learning Assessment (CLA) Institutional data -- GPA, financial aid, transcripts, retention, certification tests, alumni surveys, satisfaction surveys… Faculty Survey of Student Engagement (FSSE) and Beginning College Survey of Student Engagement (BCSSE)

66. 4. Fashion data-informed monitoring systems  Use multiple sources of data ACT/SAT score reports ACT/SAT score reports BCSSE BCSSE NSSE NSSE FSSE FSSE CIRP/CSS CIRP/CSS Noel Levitz Noel Levitz CLA CLA ACT CAAP ACT CAAP  Explain every number  Consider a systematic review of policies and practices (ISES)

67. Inventory to Enhance Educational Effectiveness

68. Oregon State University

69. 5. Stay the course The good-to-great- transformations never happened in one fell swoop. There was no single defining action, no grand program, no one killer innovation, no solitary lucky break, no miracle moment. Sustainable transformations follow a predictable pattern of buildup and breakthrough … (Collins, 2001, p. 186)

70. 5. Stay the course  Emphasize quality  Front-load resources  If it works, consider requiring it  Scale up effective practices  Sunset ineffective programs  Beware the implementation dip

71. Discussion www.iub.edu/~nsse