NSSE:RetrospectiveandProspective George Kuh SCSU NSSE Users Conference October 19, 2006

Overview 1.Why student engagement matters 2.What we’ve learned 3.Using NSSE data 4.Current activities

We value what we measure Wise decisions are needed about what to measure in the context of campus mission, values, and desired outcomes.

What Really Matters in College: Student Engagement Because individual effort and involvement are the critical determinants of impact, institutions should focus on the ways they can shape their academic, interpersonal, and extracurricular offerings to encourage student engagement. Pascarella & Terenzini, How College Affects Students, 2005, p. 602

Foundations of Student Engagement Time on task (Tyler, 1930s) Quality of effort (Pace, s) Student involvement (Astin, 1984) Social, academic integration (Tinto,1987, 1993) Good practices in undergraduate education (Chickering & Gamson, 1987) College impact (Pascarella, 1985) Student engagement (Kuh, 1991, 2005)

Good Practices in Undergraduate Education (Chickering & Gamson, 1987; Pascarella & Terenzini, 2005 ) Student-faculty contact Student-faculty contact Active learning Active learning Prompt feedback Prompt feedback Time on task Time on task High expectations High expectations Experiences with diversity Experiences with diversity Cooperation among students Cooperation among students

The Student Engagement Trinity  What students do -- time and energy devoted to educationally purposeful activities  What institutions do -- using effective educational practices to induce students to do the right things  Educationally effective institutions channel student energy toward the right activities

National Survey of Student Engagement Community College Survey of Student Engagement National Survey of Student Engagement (pronounced “nessie”) Community College Survey of Student Engagement (pronounced “cessie”) College student surveys that assess the extent to which students engage in educational practices associated with high levels of learning and development

The College Student Report NSSE’s Survey Instrument The College Student Report Student Behaviors Institutional Actions & Requirements Reactions to College Student Background Information Student Learning & Development

Effective Educational Practices Level of Academic Challenge Active & Collaborative Learning Enriching Educational Experiences SupportiveCampusEnvironment Student Faculty Interaction

Documenting Good Practice InstitutionalImprovement Public Advocacy NSSE Core Purposes NSSE Core Purposes

NSSE Evolution Year Colleges & Universities

NSSE 2006 Participating Colleges and Universities

NSSE Project Scope  1,000,000+ students from 1,000+ different schools  80% of 4-yr U.S. undergraduate FTE  50 states, Puerto Rico, Canada  70+ consortia

State & University Consortia California State U U of New Hampshire CUNY New Jersey ConnecticutU of North Carolina GeorgiaNorth Dakota U of HawaiiOntario (CA) Indiana USouth Dakota KentuckyTennessee MarylandTexas A&M U of MassachusettsU of Texas U of MissouriU of Wisconsin West Virginia West Virginia

Customized Institutional Report Overview Overview Institutional data Institutional data Means and frequencies Means and frequencies 1st year students and seniors 1st year students and seniors Comparisons by peers, Carnegie, national Comparisons by peers, Carnegie, national National benchmarks National benchmarks Data use tips Data use tips CD with raw data, etc. CD with raw data, etc. And more! And more!

What have we learned so far?

Student Success Quiz What percent of first-year high school students complete college six years after high school graduation? (a) 18% (b) 27% (c) 40% (d) 68% (e) none of the above 18% a. 18% (“participation rate”) none of the above or e. – none of the above

Student Success Quiz What percent of high school seniors have college-level reading skills? (a) 51% (b) 59% (c) 68% (d) 77% (e) none of the above 51% ACT a. 51% (ACT, 2006)

Student Success Quiz True or false: About $300 million is spent annually on postsecondary remediation coursework. False. $1-2 billion

Student Success Quiz True or false: 25% of first-year first-time frosh at two-year colleges are required to take one or more remedial courses in college. False. 60%

Student Success Quiz What percent of students who take at least one remedial course in reading do not earn a certificate or degree within 8 years of first enrollment? (a) 18% (b) 33% (c) 43% (d) 61% (e) 70% e. 70%

Grades, persistence, student satisfaction, and engagement go hand in hand

First-year students* Seniors* Academic Challenge Active & Collaborative Learning Student Faculty Interaction Enriching Educational Experiences Supportive Campus Environment NSSE & Graduation Rates *All correlations are significant at p<.01

Behold the compensatory effects of engagement

Does institutional size matter to engagement? Yes, size matters. Smaller is generally better.

Benchmark Scores for All Students by Undergraduate Enrollment

Academic Challenge, Active Learning, & Student-Faculty Interaction by Enrollment

Student engagement varies more within than between institutions.

Academic Challenge by Institutional Type Seniors Doc Ext Doc Int MA Bac LA Bac Gen Nation First-Year Students Doc Ext Doc Int MA Bac LA Bac Gen Nation Benchmark Scores

Academic Challenge at Two Public Universities

T-test: p<.000; Effect Size: -.29 Senior Scores by Transfer Status

T-test: p<.000; Effect Size: -.28 Senior Scores by Transfer Status

Worth Pondering How do we reach our least engaged students?

Who’s more engaged?  Women  Fraternity & sorority members  Full-time students  Students who live on campus  Learning community students  Students with diversity experiences  Distance learners

Effects of Learning Communities on Engagement

Who Is Most Likely to Experience Diversity? More Students of color Traditional-age students Women First-year students Less White students Older students Men Upper-division students

College Outcomes Quiz What percent of college graduates attended two or more institutions? (a) 14% (b) 26% (c) 33% (d) 42% (e) 59% e. 59%

Reasons for Concurrent Enrollment 4% 9% 12% 17% 21% 23% 47% 0%5%10%15%20%25%30%35%40%45%50% Prepare to transfer Take extra courses Course not available l, Other reasons Easier courses Better course schedule l Finances Finish degree sooner

Institutional Reflection Areas of EffectiveEducationalPractice Question or Improvement

Using NSSE Data: What Can We Do? Administrators Students Faculty Members

1. Get the ideas right Focus on a real problem  Persistence  Under-engaged students  Fragmented gen ed program  Tired pedagogical practices  Poor first-year experience  Low academic challenge  Connections to real world  Capstone experiences

2. Examine the results from multiple perspectives  Link results to other information about the student experience and complementary initiatives

Possibilities Merge NSSE data with school records Merge NSSE data with school records Identify engagement patterns by student characteristics Identify engagement patterns by student characteristics Predict retention, degree attainment, grades, other outcomes Predict retention, degree attainment, grades, other outcomes Track student engagement year to year Track student engagement year to year Compare vs. peer, aspirational, same- region schools Compare vs. peer, aspirational, same- region schools Program evaluation Program evaluation Accreditation Accreditation Consortium and system data sharing Consortium and system data sharing Student outcomes research Student outcomes research Public reporting Public reporting A national reporting template?!? A national reporting template?!?

Other Data Sources Locally-developed measures National instruments –Cooperative Institutional Research Program (CIRP) –Your First College Year (YFCY) –College Student Experiences Questionnaire (CSEQ) –Noel Levitz Student Satisfaction Inventory –ETS MAPP and Major Field Tests –ACT Collegiate Assessment of Academic Proficiency –Collegiate Learning Assessment (CLA) Institutional data -- GPA, financial aid, transcripts, retention, certification tests, alumni surveys, satisfaction surveys… Faculty Survey of Student Engagement (FSSE) and Beginning College Survey of Student Engagement (BCSSE)

NSSE Accreditation Toolkit Links Between NSSE & Accreditation Mapping NSSE to Accreditation Standards Vignettes of Institutional Usage NSSE Questions 1.l.l. Used an electronic medium (list-serv, chat group, Internet, etc.) to discuss or complete an assignment 1.m.Used to communicate with an instructor 10.g.Using computers in academic work 11.g.Using computing and information technology SACS Standard The institution’s use of technology enhances student learning, is appropriate for meeting the objectives of its programs, and ensures that students have access to and training in the use of technology.

Benchmarking Approaches Benchmarking Approaches Normative Normative - compares your students to students at other colleges and universities. Criterion Criterion - compares your school’s performance against a predetermined value or level appropriate for your students, institutional mission, size, curricular offerings, funding, etc.

Student Engagement Tip: Eliminate “Nevers”

Inventory to Enhance Educational Effectiveness

DEEP Practice Briefs Available:

2. Examine the results from multiple perspectives  Link results to other information about the student experience and complementary initiatives  Don’t allow the numbers to speak for themselves  Manage the message and the media

3. Align initiatives:  Institutional mission, values, and culture  Student preparation, ability, interests  Resources and reward systems  Existing complementary efforts NSSE & BEAMS NSSE & BEAMS AASCU American Democracy Project AASCU American Democracy Project AAC&U “Greater Expectations” AAC&U “Greater Expectations” Gen ed reform Gen ed reform Carnegie Campus Clusters/SOTL/CASTL Carnegie Campus Clusters/SOTL/CASTL Service learning/Campus Compact Service learning/Campus Compact Internationalization and diversity Internationalization and diversity

University of Missouri- St. Louis  NSSE introduced to New Faculty Teaching Scholars  Workshops held with academic leaders to link results with University’s strategic indicators  NSSE items in course evaluations  Open forum to get student input about ways to improve learning environment

4. Cultivate grass roots buy-in  Leaders endorse, but don’t dictate  Structures not (nearly) as important as relationships

Cultivating grass roots buy-in Make sure faculty and staff understand the concept of student engagement Make sure faculty and staff understand the concept of student engagement Confirm/corroborate results Confirm/corroborate results Drive data down to dept level Drive data down to dept level Gain consensus on student engagement priorities Gain consensus on student engagement priorities Faculty Survey of Student Engagement Faculty Survey of Student Engagement

Faculty Survey of Student Engagement (pronounced “fessie”) FSSE measures faculty expectations and activities related to student engagement in effective educational practices

FSSE and NSSE Point to “Disorienting Dilemmas” Situations in which usual perspectives or ways of responding do not work or don’t fit are more likely to motivate us to learn and change (Mezirow, 1990)

FSSE-NSSE Gap Analysis  About two-thirds (65%) of faculty expect students to spend more than 25 hours preparing for class think  Only about one-fifth (20%) think that students spend this amount of time actually  Only about one in ten (12%) students actually spends this much time

Course Emphasis FACULTY report very much or quite a bit of emphasis on memorizing STUDENTS report very much or quite a bit of emphasis on memorizing 29% | 14% Lower Division Upper Division 65% | 63% 1 st yr. Students Seniors

Prompt Feedback FACULTY gave prompt feedback often or very often STUDENTS received prompt feedback often or very often 93% | 93% Lower Division Upper Division 64% | 76% 1 st yr. Students Seniors

Faculty Priorities and Student Engagement

Faculty Priorities and Selected Student Outcomes

Faculty Who Value Effective Educational Practices

What to Make of This? 1.When faculty members emphasize certain educational practices, students engage in them to a greater extent than their peers elsewhere. 2.Good things go together

5. Stay the course The good-to-great- transformations never happened in one fell swoop. There was no single defining action, no grand program, no one killer innovation, no solitary lucky break, no miracle moment. Sustainable transformations follow a predictable pattern of buildup and breakthrough … (Collins, 2001, p. 186)

5. Stay the course  Emphasize quality  Front-load resources  If it works, consider requiring it

Lessons from National Center for Academic Transformation If doing something is important, require it (first-year students don’t do ‘optional’) If doing something is important, require it (first-year students don’t do ‘optional’) Assign course points to the activity Assign course points to the activity Monitor and intervene when necessary Monitor and intervene when necessaryhttp://

5. Stay the course  Emphasize quality  Front-load resources  If it works, consider requiring it  Scale up effective practices  Sunset ineffective programs  Beware the implementation dip

The Language of Effective Educational Practices NSSE Pocket Guide NSSE Pocket Guide

The Language of Effective Educational Practices NSSE Pocket Guide NSSE Pocket Guide High School Survey of Student Engagement (HSSSE) High School Survey of Student Engagement (HSSSE) Beginning College Student Engagement Survey (BCSSE) Beginning College Student Engagement Survey (BCSSE) Faculty Survey of Student Engagement (FSSE) Faculty Survey of Student Engagement (FSSE) Law School Survey of Student Engagement(LSSSE) Law School Survey of Student Engagement(LSSSE)

NSSE Institute Aims NSSE Institute Aims  Document and share success stories to help schools improve  Work with institutions: workshops workshops campus audits campus audits consultations consultations research & evaluation projects research & evaluation projects

Building Engagement and Attainment of Minority Students (BEAMS) 5-year project funded by Lumina Foundation for Education5-year project funded by Lumina Foundation for Education Alliance for Equity in Higher Education institutionsAlliance for Equity in Higher Education institutions Using student engagement data to guide change initiativesUsing student engagement data to guide change initiatives Provides resources for improvement initiativesProvides resources for improvement initiatives

Other Current Activities NCES What Matters to Student Success” synthesis NCES What Matters to Student Success” synthesis Wabash National Study of Liberal Arts Education Wabash National Study of Liberal Arts Education Penn State’s “Parsing the First College Year” Penn State’s “Parsing the First College Year” AAC&U’s “Bringing Theory to Practice” AAC&U’s “Bringing Theory to Practice” Linking BCSSE, NSSE & FSSE Linking BCSSE, NSSE & FSSE

First-Year Student Use of Campus Services

NSSE Virtues  Research based  Focus on educationally purposeful activities  Results point to areas where improvement is desirable  Compelling face validity  Established psychometrics  Transparent operations  Third party administration

NSSE Virtues NSSE Virtues  Random sampling  Targeted sampling  Flexibility: consortium question, modules  Contributes to value-added estimates when linked to BCSSE, CLA, other outcomes measures  Benchmarks for peer, national and other comparisons

NSSE Cautions NSSE Cautions  Only one source of information  Corroborate results  Engagement skeptics  Not an outcomes measure  Denial in face of less-than- desirable results  Eschew rankings

Discussion