NSSE:RetrospectiveandProspective George Kuh UCO Regional Users Conference October 6, 2005

Overview 1.Why student engagement matters 2.What we’ve learned 3.Current activities & next steps 4.Discussion

We value what we measure Wise decisions are needed about what to measure in the context of campus mission, values, and desired outcomes.

What Really Matters in College: Student Engagement Because individual effort and involvement are the critical determinants of impact, institutions should focus on the ways they can shape their academic, interpersonal, and extracurricular offerings to encourage student engagement. Pascarella & Terenzini, How College Affects Students, 2005, p. 602

Foundations of Student Engagement Time on task (Tyler, 1930s) Quality of effort (Pace, s) Student involvement (Astin, 1984) Social, academic integration (Tinto,1987, 1993) Good practices in undergraduate education (Chickering & Gamson, 1987) College impact (Pascarella, 1985) Student engagement (Kuh, 1991, 2005)

Good Practices in Undergraduate Education (Chickering & Gamson, 1987; Pascarella & Terenzini, 2005 ) Student-faculty contact Student-faculty contact Active learning Active learning Prompt feedback Prompt feedback Time on task Time on task High expectations High expectations Experiences with diversity Experiences with diversity Cooperation among students Cooperation among students

The Student Engagement Trinity  What students do -- time and energy devoted to educationally purposeful activities  What institutions do -- using effective educational practices to induce students to do the right things  Educationally effective institutions channel student energy toward the right activities

National Survey of Student Engagement Community College Survey of Student Engagement National Survey of Student Engagement (pronounced “nessie”) Community College Survey of Student Engagement (pronounced “cessie”) College student surveys that assess the extent to which students engage in educational practices associated with high levels of learning and development

The College Student Report NSSE’s Survey Instrument The College Student Report Student Behaviors Institutional Actions & Requirements Reactions to College Student Background Information Student Learning & Development

Documenting Good Practice InstitutionalImprovement Public Advocacy NSSE Core Activities NSSE Core Activities

NSSE Evolution Year Colleges & Universities

NSSE Participating Colleges and Universities

NSSE Project Scope  900,000 students from 970+ different schools  77% of 4-yr U.S. undergraduate FTE  50 states, Puerto Rico, Canada  70+ consortia

State & University Consortia California State U U of Missouri CUNY U of New Hampshire ConnecticutNew Jersey U of HawaiiU of North Carolina Indiana USouth Dakota KentuckyTexas A&M MarylandU of Texas U of MassachusettsU of Wisconsin West Virginia

Effective Educational Practices Level of Academic Challenge Active & Collaborative Learning Enriching Educational Experiences SupportiveCampusEnvironment Student Faculty Interaction

Customized Institutional Report Overview Overview Institutional data Institutional data Means and frequencies Means and frequencies 1st year students and seniors 1st year students and seniors Comparisons by peers, Carnegie, national Comparisons by peers, Carnegie, national National benchmarks National benchmarks Data use tips Data use tips CD with raw data, etc. CD with raw data, etc. And more! And more!

What have we learned so far?

Grades, persistence, student satisfaction, and engagement go hand in hand

First-year students* Seniors* Academic Challenge Active & Collaborative Learning Student Faculty Interaction Enriching Educational Experiences Supportive Campus Environment NSSE & Graduation Rates *All correlations are significant at p<.01

Does institutional size matter to engagement? Yes, size matters. Smaller is generally better.

Benchmark Scores for All Students by Undergraduate Enrollment

Academic Challenge, Active Learning, & Student-Faculty Interaction by Enrollment

Student engagement varies more within than between institutions.

Academic Challenge by Institutional Type Seniors Doc Ext Doc Int MA Bac LA Bac Gen Nation First-Year Students Doc Ext Doc Int MA Bac LA Bac Gen Nation Benchmark Scores

Academic Challenge at Two Public Universities

Worth Pondering How do we reach our least engaged students?

Who’s more engaged?  Women  Fraternity & sorority members  Full-time students  Students who live on campus  Learning community students  Students with diversity experiences

Effects of Learning Communities on Engagement

Who Is Most Likely to Experience Diversity? More Students of color Traditional-age students Women First-year students Less White students Older students Men Upper-division students

College Outcomes Quiz What percent of college graduates attended two or more institutions? (a) 14% (b) 26% (c) 33% (d) 42% (e) 59% e. 59%

Reasons for Concurrent Enrollment 4% 9% 12% 17% 21% 23% 47% 0%5%10%15%20%25%30%35%40%45%50% Prepare to transfer Take extra courses Course not available l, Other reasons Easier courses Better course schedule l Finances Finish degree sooner

Documenting Good Practice InstitutionalImprovement Public Advocacy NSSE Core Activities NSSE Core Activities

The Language of Effective Educational Practices NSSE Pocket Guide NSSE Pocket Guide

The Language of Effective Educational Practices NSSE Pocket Guide NSSE Pocket Guide High School Survey of Student Engagement (HSSSE) High School Survey of Student Engagement (HSSSE) Beginning College Student Engagement Survey (BCSSE) Beginning College Student Engagement Survey (BCSSE) Law School Survey of Student Engagement(LSSSE) Law School Survey of Student Engagement(LSSSE)

First-Year Student Use of Campus Services

NSSE Institute Aims  Document and share success stories to help schools improve  Work with institutions: workshops workshops campus audits campus audits consultations consultations research & evaluation projects research & evaluation projects Sponsored by The Carnegie Foundation for the Advancement of Teaching with support from Lumina Foundation for Education and the Wabash College Center of Inquiry in the Liberal Arts.

Building Engagement and Attainment of Minority Students (BEAMS) 5-year project funded by Lumina Foundation for Education5-year project funded by Lumina Foundation for Education Alliance for Equity in Higher Education institutionsAlliance for Equity in Higher Education institutions Using student engagement data to guide change initiativesUsing student engagement data to guide change initiatives Provides resources for improvement initiativesProvides resources for improvement initiatives

Project DEEP To discover, document and describe what high performing institutions do and how they achieved this level of effectiveness.

Other Current Activities Accreditation tool kit Accreditation tool kit AASCU “American Democracy” AASCU “American Democracy” “Foundations of Excellence” “Foundations of Excellence” Collegiate Learning Assessment (RAND-CAE) Collegiate Learning Assessment (RAND-CAE) “Connecting the Dots” “Connecting the Dots” Collaborative research Wabash, with CCSSE, other scholars Collaborative research Wabash, with CCSSE, other scholars NCES “student success” synthesis NCES “student success” synthesis Linking BCSSE with NSSE Linking BCSSE with NSSE

NSSE Goes International Canada (G-10, Ontario consortium)Canada (G-10, Ontario consortium) MacedoniaMacedonia AustraliaAustralia KazakhstanKazakhstan EnglandEngland SpainSpain Future collaborations planned for: IrelandIreland LebanonLebanon RussiaRussia

Institutional Reflection Areas of EffectiveEducationalPractice Question or Improvement

What Can We Do? Administrators Students Faculty Members

“NSSE is a great way to stimulate reflection and debate about what we do more and less well, and why. For us it’s proving an exciting and enlivening tool for self-reflection and self- improvement.” Michael McPherson, President, Spencer Foundation (former President of Macalaster College) Stimulating Campus Conversations

1. Get the ideas right Focus on a real problem  Persistence  Under-engaged students  Fragmented gen ed program  Tired pedagogical practices  Poor first-year experience  Low academic challenge  Connections to real world  Capstone experiences

Fayetteville State University Did NSSE in 2002 & 2003 Did NSSE in 2002 & 2003 Used NSSE data as a way to bring together people concerned about data on campus Used NSSE data as a way to bring together people concerned about data on campus Data confirmed effectiveness of FSU’s Freshman Year Initiative Data confirmed effectiveness of FSU’s Freshman Year Initiative

Iowa State University IndicatorIndicator DescriptionFY01FY0205 GOAL (Target) ISU 6Level of academic challenge (NSSE) *based on 100 pt. scale 46.9 (1st year) 48.6 (Seniors) (2000) 49.7 (1st year) 50.5 (Seniors ) (2001) 54.0 (1st year) 56.0 (Seniors) ISU 8Enriching educational experiences (NSSE) *based on 100 pt. scale 47.0 (1st year) (2000) 53.8 (1st year (2001) 59.0 (1st year) Iowa State linked NSSE results to Board of Regents Performance Indicators

2. Align initiatives:  Institutional mission, values, and culture  Student preparation, ability, interests  Resources and reward systems  Existing complementary efforts NSSE & BEAMS NSSE & BEAMS AASCU American Democracy Project AASCU American Democracy Project AAC&U “Greater Expectations” AAC&U “Greater Expectations” Gen ed reform Gen ed reform Carnegie Campus Clusters/SOTL/CASTL Carnegie Campus Clusters/SOTL/CASTL Service learning/Campus Compact Service learning/Campus Compact Internationalization and diversity Internationalization and diversity

3. Cultivate grass roots buy-in  Leaders endorse, but don’t dictate  Structures not (nearly) as important as relationships

Getting grass roots buy-in Confirm/corroborate results Confirm/corroborate results Drive data down to dept level Drive data down to dept level Gain consensus on student engagement priorities Gain consensus on student engagement priorities Faculty Survey of Student Engagement Faculty Survey of Student Engagement

Faculty Survey of Student Engagement (pronounced “fessie”) FSSE measures faculty expectations and activities related to student engagement in effective educational practices

FSSE and NSSE Point to “Disorienting Dilemmas” Situations in which usual perspectives or ways of responding do not work or don’t fit are more likely to motivate us to learn and change (Mezirow, 1990)

What Are Faculty and Students Telling Us?

FSSE-NSSE Gap Analysis  About two-thirds (65%) of faculty expect students to spend more than 25 hours preparing for class think  Only about one-fifth (20%) think that students spend this amount of time actually  Only about one in ten (12%) students actually spends this much time

Prompt Feedback FACULTY gave prompt feedback often or very often STUDENTS received prompt feedback often or very often 93% | 93% Lower Division Upper Division 64% | 76% 1 st yr. Students Seniors

Course Emphasis FACULTY report very much or quite a bit of emphasis on memorizing STUDENTS report very much or quite a bit of emphasis on memorizing 14% | 29% Lower Division Upper Division 65% | 63% 1 st yr. Students Seniors

Faculty Priorities and Student Engagement

Faculty Priorities and Selected Student Outcomes

How Men and Women Faculty Use Class Time Men Women How Men and Women Faculty Use Class Time Men Women

What to make of this? 1.When faculty members emphasize certain educational practices, students engage in them to a greater extent than their peers elsewhere. 2.Good things go together

University of Missouri- St. Louis  NSSE introduced to New Faculty Teaching Scholars  Workshops held with academic leaders to link results with University’s strategic indicators  NSSE items in course evaluations  Open forum to get student input about ways to improve learning environment

4. Fashion data-informed monitoring systems  Use multiple sources of data ACT/SAT score reports BCSSE NSSE FSSE CIRP/CSS Noel Levitz CLA ACT CAAP

Other Data Sources Locally-developed measures National instruments –Cooperative Institutional Research Program (CIRP) –Your First College Year (YFCY) –College Student Experiences Questionnaire (CSEQ) –Noel Levitz Student Satisfaction Inventory –ETS Major Field Tests –ACT Collegiate Assessment of Academic Proficiency –Collegiate Learning Assessment (CLA) Institutional data -- GPA, financial aid, transcripts, retention, certification tests, alumni surveys, satisfaction surveys… Faculty Survey of Student Engagement (FSSE) and Beginning College Survey of Student Engagement (BCSSE)

4. Fashion data-informed monitoring systems  Use multiple sources of data ACT/SAT score reports ACT/SAT score reports BCSSE BCSSE NSSE NSSE FSSE FSSE CIRP/CSS CIRP/CSS Noel Levitz Noel Levitz CLA CLA ACT CAAP ACT CAAP  Explain every number  Consider a systematic review of policies and practices (ISES)

Inventory to Enhance Educational Effectiveness

Oregon State University

5. Stay the course The good-to-great- transformations never happened in one fell swoop. There was no single defining action, no grand program, no one killer innovation, no solitary lucky break, no miracle moment. Sustainable transformations follow a predictable pattern of buildup and breakthrough … (Collins, 2001, p. 186)

5. Stay the course  Emphasize quality  Front-load resources  If it works, consider requiring it  Scale up effective practices  Sunset ineffective programs  Beware the implementation dip

Discussion