NC State Office of Institutional Research & Planning September, 2014

Presentation Overview  NSSE and the Concept of Student Engagement  Research Design, NC State Comparison Groups, and Response Rates  NSSE 2014 & Selected NC State Results (NC State first-year students vs seniors, and vs their respective peers at RU/VH institutions)  Overall Satisfaction  Peer Comparisons: Summary  Engagement Indicators: Detailed Results  High Impact Practices  Using Our NSSE Data  For More Information…

NSSE and the Concept of Student Engagement

What is Student Engagement?  What students do -- time and energy devoted to studies and other educationally purposeful activities  What institutions do -- using resources and effective educational practices to induce students to do the right things  Educationally effective institutions channel student energy toward the right activities

Seven Principles of Good Practice in Undergraduate Education Student-faculty contact Active learning Prompt feedback Time on task High expectations Experiences with diversity Cooperation among students Chickering, A. W. & Gamson, Z. F. (1987). Seven principles for good practice in undergraduate education. AAHE: Bulletin, 39 (7), 3-7.

NSSE Survey Content Engagement in meaningful academic experiences Engagement in meaningful academic experiences Engagement in High Impact Practices Engagement in High Impact Practices Student Reactions to College Student Reactions to College Student Background Information Student Background Information Student Learning & Development

NSSE Engagement Indicators Student – Faculty Interaction Learning Strategies Quantitative Reasoning Reflective & Integrative Learning Higher-Order Learning Theme: Academic Challenge Academic Challenge Experiences with Faculty Learning with Peers Campus Environment Meaningful Academic Engagement Themes Engagement Indicators

Research Design, NC State Comparison Groups, & Response Rates

NC State NSSE Survey Administration  Spring 2014  Census of first-year students & graduating seniors  Online Survey  invitation and reminders “from” the Chancellor  Additional survey modules  Academic Advising  Experiences with Writing

NC State NSSE Comparison Groups  Carnegie Classification: RU/VH (Research Univ [very high research activity])  43 institutions  Participating Official Peers  Georgia Tech  Iowa State Univ.  Michigan State Univ.  Univ. of Illinois  Univ. of Maryland  Univ. of Wisconsin  Virginia Tech  Large, Public, Research Intensive Universities  Michigan State Univ.  Univ. at Buffalo, SUNY  Univ. of Alabama  Univ. of Cincinnati  Univ. of Colorado- Boulder  Univ. of Colorado-Denver  Univ. of South Florida  Univ. of Utah  Virginia Tech  Washington State Univ. 10

Response Rates: NC State & Comparison Groups NC State’s response rate for both first-year students and seniors was similar to that of our official peers, but slightly lower than those in other RU/VH and large, public, research institutions.

NC State Respondents by Gender & Race/Ethnicity/Nationality First-YearSenior % of Resps% of Pop% of Resps% of Pop Gender Female 56%45%51%44% Male 46%55%49%56% Race/Ethnicity/Nationality African American/Black 6%5% 6% Amer. Indian/Alaskan Native <1% Asian 5% White 73%76%77%75% Hispanic/Latino 4% 5%4% Multiracial/Ethnic 4% 3% Foreign/nonresident alien 5%4%1% Unknown 1%2%3%5% The race/ethnicity/nationality distribution of NC State’s NSSE respondents generally matches that of the student population overall. While females are over-represented among NC State respondents, NSSE results are weighted to accurately reflect the true proportion of males and females in the population.

Selected Results:  Overall satisfaction  Summary comparisons  Engagement indicators: detailed results  High impact practices

Overall Satisfaction NC State seniors were much more likely than first-year students to rate their educational experience as “excellent.” NC State first-year students, and especially seniors, were more likely than their RU/VH peers to rate their experience as “excellent.”

Overall Satisfaction Nine-in-ten NC State first-year students and seniors said they would probably or definitely choose NC State again if they were starting over. Both groups were much more likely than their RU/VH peers to say they would definitely choose to do so.

Highest and Lowest Performing Items Relative to RU/VH Peers NC State first-year students were much more likely than their RU/VH peers to give positive ratings to their interactions with student services staff, and to the institutions’ emphasis on their well- being, and much less likely than their RU/VH peers to have had a course that included a service-learning component.

Highest and Lowest Performing Items Relative to RU/VH Peers NC State seniors were much more likely than their RU/VH peers to have had a capstone experience, to give positive ratings to their interactions with student services staff, and to have worked with other students on course projects. The only NSSE item on which they had notably less positive ratings than their RU/VH peers was on having had courses that included a service-learning component.

Engagement Indicators: NC State vs RU/VH Peers NC State seniors gave more positive ratings than their peers at other RU/VH institutions on almost all the engagement indicators, while NC State first- year students gave more positive ratings than their peers on half of the indicators. Note: Results for each individual item included in an engagement indicator, with comparisons to each of our NSSE comparisons groups (official peers, Carnegie class, and large/public/research institutions), are provided in “NSSE14 Engagement Indicators (NC State).pdf”

Academic Challenge: Higher-Order Learning Majorities of NC State first-year students and seniors indicated that their coursework in the current semester “very much” or “quite a bit” emphasized higher-order learning. NC State seniors, however, were notably more likely than first-year students to say their course work “very much” emphasized such experiences. Applying facts, theories, or methods to practical problems or new situations FIRST YEAR SENIORS Analyzing an idea, experience, or line of reasoning in depth by examining its parts FIRST YEAR SENIORS Evaluating a point of view, decision, or information source FIRST YEAR SENIORS Forming a new idea or understanding from various pieces of information FIRST YEAR SENIORS

Academic Challenge: Higher-Order Learning NC State first-year students were generally less likely than their peers at RU/VH institutions to say their course work emphasized higher-order learning. Applying facts, theories, or methods to practical problems or new situations Analyzing an idea, experience, or line of reasoning in depth by examining its parts Evaluating a point of view, decision, or information source Forming a new idea or understanding from various pieces of information

Academic Challenge: Higher-Order Learning NC State seniors were slightly more likely than their peers at RU/VH institutions to say their course work emphasized “applying facts, theories, or methods to practical problems or new situations,” and analyzing an idea, experience, or line of reasoning in depth by examining its parts.” Applying facts, theories, or methods to practical problems or new situations Carnegie NCSU Analyzing an idea, experience, or line of reasoning in depth by examining its parts Carnegie NCSU Evaluating a point of view, decision, or information source Carnegie NCSU Forming a new idea or understanding from various pieces of information Carnegie NCSU

Academic Challenge: Reflective & Integrative Learning The extent of engagement in reflective and integrative learning varied widely depending on the specific strategy asked about. NC State seniors were much more likely than first-year students to say they “often” or “very often” engaged in or had experiences with reflective and integrative learning. Differences were especially large for “combined ideas from different courses when completing assignments.” Combined ideas from different courses when completing assignments FIRST YEAR SENIORS Connected your learning to societal problems or issues FIRST YEAR SENIORS Included diverse perspectives (political, religious, racial/ ethnic, gender, etc.) in course discussions or assignments FIRST YEAR SENIORS Examined the strengths and weaknesses of your own views on a topic or issue FIRST YEAR SENIORS Tried to better understand someone else's views by imagining how an issue looks from his or her perspective FIRST YEAR SENIORS Learned something that changed the way you understand an issue or concept FIRST YEAR SENIORS Connected ideas from your courses to your prior experiences and knowledge FIRST YEAR SENIORS

Academic Challenge: Reflective & Integrative Learning NC State first-year students were slightly less likely than their RU/VH peers to engage in or have experiences with reflective and integrative learning. First-year students were least likely to have “included diverse perspectives…in course discussions or assignments.” Combined ideas from different courses when completing assignments Carnegie NCSU Connected your learning to societal problems or issues Carnegie NCSU Included diverse perspectives (political, religious, racial/ ethnic, gender, etc.) in course discussions or assignments Carnegie NCSU Examined the strengths and weaknesses of your own views on a topic or issue Carnegie NCSU Tried to better understand someone else's views by imagining how an issue looks from his or her perspective Carnegie NCSU Learned something that changed the way you understand an issue or concept Carnegie NCSU Connected ideas from your courses to your prior experiences and knowledge Carnegie NCSU

Academic Challenge: Reflective & Integrative Learning NC State seniors were generally similar to their RU/VH peers in the extent to which they engaged in or had experiences with reflective and integrative learning. Combined ideas from different courses when completing assignments Carnegie NCSU Connected your learning to societal problems or issues Carnegie NCSU Included diverse perspectives (political, religious, racial/ ethnic, gender, etc.) in course discussions or assignments Carnegie NCSU Examined the strengths and weaknesses of your own views on a topic or issue Carnegie NCSU Tried to better understand someone else's views by imagining how an issue looks from his or her perspective Carnegie NCSU Learned something that changed the way you understand an issue or concept Carnegie NCSU Connected ideas from your courses to your prior experiences and knowledge Carnegie NCSU

Academic Challenge: Learning Strategies NC State seniors and first-year students were notably more likely to say they “identified key information from reading assignments” than to have used other learning strategies. Seniors were more likely than first-year students to say they “very often” “identified key information from reading assignments,” but were slightly less likely than first-year students to say the “often” or “very often” “reviewed [their] notes after class.” Identified key information from reading assignments FIRST YEAR SENIORS Reviewed your notes after class FIRST YEAR SENIORS Summarized what you learned in class or from course materials FIRST YEAR SENIORS

Academic Challenge: Learning Strategies NC State first-year students were slightly more likely than their RU/VH peers to use various learning strategies, most notably “reviewed [your] notes after class.” Identified key information from reading assignments Carnegie NCSU Reviewed your notes after class Carnegie NCSU Summarized what you learned in class or from course materials Carnegie NCSU

Academic Challenge: Learning Strategies There were no notable differences between NC State seniors and their RU/VH peers in the extent to which they use various learning strategies. Identified key information from reading assignments Carnegie NCSU Reviewed your notes after class Carnegie NCSU Summarized what you learned in class or from course materials Carnegie NCSU

Academic Challenge: Quantitative Reasoning About 10% or more of NC State seniors and first-year students said they “never” engage in various types of quantitative reasoning asked about. Seniors were twice as likely as first-year students to say they “very often” “used numerical information to examine a real-world problem or issue” and to have “evaluated what others have concluded from numerical information.” Reached conclusions based on your own analysis of numerical information (numbers, graphs, statistics, etc.) FIRST YEAR SENIORS Used numerical information to examine a real-world problem or issue (unemployment, climate change, public health, etc.) FIRST YEAR SENIORS Evaluated what others have concluded from numerical information FIRST YEAR SENIORS

Academic Challenge: Quantitative Reasoning There were no differences between NC State first-year students and their RU/VH peers in the extent to which they engaged in quantitative reasoning. More than half of the first-year students said they “never” or only “sometimes” “used numerical information to examine a real-world problem or issue” or “evaluated what others have concluded from numerical information.” Reached conclusions based on your own analysis of numerical information (numbers, graphs, statistics, etc.) Carnegie NCSU Used numerical information to examine a real-world problem or issue (unemployment, climate change, public health, etc.) Carnegie NCSU Evaluated what others have concluded from numerical information Carnegie NCSU

Academic Challenge: Quantitative Reasoning NC State seniors were slightly more likely than their RU/VH peers to “often” or “very often” use various types of quantitative reasoning. Reached conclusions based on your own analysis of numerical information (numbers, graphs, statistics, etc.) Carnegie NCSU Used numerical information to examine a real-world problem or issue (unemployment, climate change, public health, etc.) Carnegie NCSU Evaluated what others have concluded from numerical information Carnegie NCSU

Learning with Peers: Collaborative Learning Overall, about one-in-five NC State seniors and first-year students said they “very often” engage in various collaborative learning strategies, with a notable exception being that over 40% of seniors said they have “worked with other students on course projects or assignments.” First-year students were more likely than seniors to have “asked another student to help [them] understand course material.” Asked another student to help you understand course material FIRST YEAR SENIORS Explained course material to one or more students FIRST YEAR SENIORS Prepared for exams by discussing or working through course material with other students FIRST YEAR SENIORS Worked with other students on course projects or assignments FIRST YEAR SENIORS

Learning with Peers: Collaborative Learning NC State first-year students are similar to their RU/VH peers in the extent to which they engage in collaborative learning, with the exception of being that NC State students were slightly more likely than their RU/V peers to have “asked another student to help you understand course material.” Asked another student to help you understand course material Carnegie NCSU Explained course material to one or more students Carnegie NCSU Prepared for exams by discussing or working through course material with other students Carnegie NCSU Worked with other students on course projects or assignments Carnegie NCSU

Learning with Peers: Collaborative Learning NC State seniors were more likely than their RU/VH peers to “often” or “very often” engage in collaborative learning. Differences were especially large for “worked with other students on course projects or assignments”. Asked another student to help you understand course material Carnegie NCSU Explained course material to one or more students Carnegie NCSU Prepared for exams by discussing or working through course material with other students Carnegie NCSU Worked with other students on course projects or assignments Carnegie NCSU

Learning with Peers: Discussions with Diverse Others Majorities of NC State seniors and first-year students said they have discussions with others who are different from themselves, although seniors were slightly more likely than first-year students to say they do so “very often.” People of a race or ethnicity other than your own FIRST YEAR SENIORS People from an economic background other than your own FIRST YEAR SENIORS People with religious beliefs other than your own FIRST YEAR SENIORS People with political views other than your own FIRST YEAR SENIORS

Learning with Peers: Discussions with Diverse Others Other than NC State first-year students being slightly less likely to “very often” have discussions with “people with religious beliefs other than [their] own,” there were no notable differences between them and their RU/VH peer in the frequency with which they have discussions with those who are different from themselves. People of a race or ethnicity other than your own Carnegie NCSU People from an economic background other than your own Carnegie NCSU People with religious beliefs other than your own Carnegie NCSU People with political views other than your own Carnegie NCSU

Learning with Peers: Discussions with Diverse Others NC State seniors were more likely than their RU/VH peers to have discussions with diverse others. More than three-quarters said they had discussions “often” or “very often” with people of “a race or ethnicity other than [their] own”, “from an economic background other than [their] own, “with religious beliefs other than [their] own”, and “with political views other than [their] own”.. People of a race or ethnicity other than your own Carnegie NCSU People from an economic background other than your own Carnegie NCSU People with religious beliefs other than your own Carnegie NCSU People with political views other than your own Carnegie NCSU

Experiences with Faculty: Student-Faculty Interaction Talked about career plans with a faculty member FIRST YEAR SENIORS Worked w/faculty on activities other than coursework (committees, student groups, etc.) FIRST YEAR SENIORS Discussed course topics, ideas, or concepts with a faculty member outside of class FIRST YEAR SENIORS Discussed your academic performance with a faculty member FIRST YEAR SENIORS Sizeable numbers of NC State seniors and first-year students indicated they had “never” interacted with faculty in various ways outside of the classroom. Seniors, however, were much more likely than first-year students to say they had interacted with faculty “often” or “very often.” Differences were especially large for “worked w/faculty on activities other than coursework (committees, student groups, etc.).”

Experiences with Faculty: Student-Faculty Interaction NC State first-year students were slightly more likely than their RU/VH peers have “talked about career plans with a faculty member,” and to have “discussed your academic performance with a faculty member.” Talked about career plans with a faculty member Carnegie NCSU Worked w/faculty on activities other than coursework (committees, student groups, etc.) Carnegie NCSU Discussed course topics, ideas, or concepts with a faculty member outside of class Carnegie NCSU Discussed your academic performance with a faculty member Carnegie NCSU

Experiences with Faculty: Student-Faculty Interaction Similar to first-year students, NC State seniors were slightly more likely than their RU/VH peers to have “often” or “very often” “talked about career plans with a faculty member,” and to have “discussed your academic performance with a faculty member.” They were also more likely to have “worked with faculty on activities other than coursework.” Talked about career plans with a faculty member Carnegie NCSU Worked w/faculty on activities other than coursework (committees, student groups, etc.) Carnegie NCSU Discussed course topics, ideas, or concepts with a faculty member outside of class Carnegie NCSU Discussed your academic performance with a faculty member Carnegie NCSU

Campus Environment: Quality of Interactions Students FIRST YEAR SENIORS Academic advisors FIRST YEAR SENIORS Faculty FIRST YEAR SENIORS Student services staff (career services, student activities, housing, etc.) FIRST YEAR SENIORS Other administrative staff and offices (registrar, financial aid, etc.) FIRST YEAR SENIORS In general, about one-fourth or more of NC State seniors and first-year students rate the quality of their interactions with various other people on campus as “excellent.” Seniors were much more likely than first-year students to say they had high quality interactions with “faculty,” and slightly less likely than first-year students to say they had high quality interactions with other “students.”

Students Carnegie NCSU Academic advisors Carnegie NCSU Faculty Carnegie NCSU Student services staff (career services, student activities, housing, etc.) Carnegie NCSU Other administrative staff and offices (registrar, financial aid, etc.) Carnegie NCSU Campus Environment: Quality of Interactions NC State first-year students were notably more likely than their RU/VH peers to report high quality interactions with all groups asked about.

Campus Environment: Quality of Interactions Students Carnegie NCSU Academic advisors Carnegie NCSU Faculty Carnegie NCSU Student services staff (career services, student activities, housing, etc.) Carnegie NCSU Other administrative staff and offices (registrar, financial aid, etc.) Carnegie NCSU NC State seniors were notably more likely than their RU/VH peers to report high quality interactions with “faculty,” “student services staff,” and “other administrative staff.”

Campus Environment: Supportive Environment Providing support to help students succeed academically FIRST YEAR SENIORS Using learning support services (tutoring services, writing center, etc.) FIRST YEAR SENIORS Encouraging contact among students from diff. backgrounds (soc., racial/eth., relig., etc.) FIRST YEAR SENIORS Providing opportunities to be involved socially FIRST YEAR SENIORS Providing support for your overall well-being (recreation, health care, counseling, etc.) FIRST YEAR SENIORS Helping you manage your non- academic responsibilities (work, family, etc.) FIRST YEAR SENIORS Attending campus activities and events (performing arts, athletic events, etc.) FIRST YEAR SENIORS Attending events that address important social, economic, or political issues FIRST YEAR SENIORS With few exceptions, majorities of NC State seniors and first-year students see the campus as providing a supportive environment. First-year students, however, were consistently more likely than seniors to say NC State “very much” emphasized various aspects of a supportive environment.

Campus Environment: Supportive Environment NC State first year students were more likely than their RU/VH peers to say their institution “very much” emphasized multiple aspects of a supportive environment. Providing support to help students succeed academically Carnegie NCSU Using learning support services (tutoring services, writing center, etc.) Carnegie NCSU Encouraging contact among students from diff. backgrounds (soc., racial/eth., relig., etc.) Carnegie NCSU Providing opportunities to be involved socially Carnegie NCSU Providing support for your overall well-being (recreation, health care, counseling, etc.) Carnegie NCSU Helping you manage your non- academic responsibilities (work, family, etc.) Carnegie NCSU Attending campus activities and events (performing arts, athletic events, etc.) Carnegie NCSU Attending events that address important social, economic, or political issues Carnegie NCSU

Campus Environment: Supportive Environment Providing support to help students succeed academically Carnegie NCSU Using learning support services (tutoring services, writing center, etc.) Carnegie NCSU Encouraging contact among students from diff. backgrounds (soc., racial/eth., relig., etc.) Carnegie NCSU Providing opportunities to be involved socially Carnegie NCSU Providing support for your overall well-being (recreation, health care, counseling, etc.) Carnegie NCSU Helping you manage your non- academic responsibilities (work, family, etc.) Carnegie NCSU Attending campus activities and events (performing arts, athletic events, etc.) Carnegie NCSU Attending events that address important social, economic, or political issues Carnegie NCSU NC State seniors were more likely than their RU/VH peers to say their institution “very much” emphasized multiple aspects of a supportive environment.

High Impact Practices NC State first-year students were more likely than their RU/VH peers to say they have participated in a learning community, but less likely to have taken a course with a service learning component.

High Impact Practices NC State first-year students participated in about the same number of different types of high impact learning experiences as did their RU/VH peers.

High Impact Practices NC State seniors were more likely than their RU/VH peers to have participated in a range of high impact learning experiences, most notably in internships or having a capstone course.

High Impact Practices Three-fourths of seniors had participated in two or more different types of high impact experiences during their time at NC State; less than one-in-ten indicated having had no such experience.

Using Our NSSE Data

Using NSSE Data  Discovering and sharing ways student engagement results are being used is one of NSSE’s most important activities.  NSSE results are being used across all types of institutions.  The following slides illustrate how NSSE data can inform educational policy and practice and provide examples of how specific institutions have used their NSSE results in productive ways. Areas of Effective Educational Practice Areas for Institutional Improvement

Internal Campus Uses  Gauge status of campus priorities  Examine changes in student engagement between first and senior years  Assess campus progress over time  Encourage dialogue about good practice  Link with other data to test hypotheses, evaluate programs  Improve curricula, instruction, services Institutional Improvement Learning Communities 1 ST Year and Senior Experience Academic Affairs Learning Assessment Faculty Development Academic Advising Peer Comparison Student Affairs Institutional Research Enrollment Management

External Campus Uses  Assess status vis-à-vis peers, competitors  Identify, develop, market distinctive competencies  Encourage collaboration in consortia (e.g., statewide NSSE conference)  Provide evidence of accountability for good processes (while awaiting improvement in outcomes) Public Accountability Fund Raising Governing Boards Prospective Students Alumni State Policy Makers Performance Indicators Focus on Right Things Accrediting Bodies Media Parents

Supporting NSSE Use in Accreditation  NSSE Accreditation Toolkits – resource tailored to regional and program accreditors  Maps NSSE items to accreditation standards /criteria to support data use in accreditation

Example of Data Use: Increasing Academic Challenge FAYETTEVILLE STATE UNIVERSITY  Finding: Student reported levels of writing and time spent preparing for class were lower than institution desired.  Action: Began providing NSSE data disaggregated by major to department chairs so that areas of potential improvement could be identified in various fields of study. The institution also increased its investment in learning communities and development of capstone courses to strengthen writing across the curriculum and levels of class preparation.

Example of Data Use: Enriching the First Year Experience WASHINGTON STATE UNIVERSITY  Finding: NSSE results indicated the campus was not meeting student expectations for collaborative learning, student-faculty interaction, and learning in community.  Action: Freshman Focus learning communities were created to provide all incoming freshmen the opportunity to engage in an extensive living-learning community system.

Example of Data Use: Student – Faculty Interaction CALIFORNIA STATE UNIVERSITY - FRESNO  Finding: NSSE results showed that student-faculty interaction was lower than expected.  Action: Student success task force identified ways to improve student success. Participated in Building Engagement and Attainment for Minority Students (BEAMS) program to develop Mentoring Institute. Now 200+ faculty members, staff and student mentors have been trained.

Example of Data Use: Enriching and High Impact Practices JACKSONVILLE STATE UNIVERSITY  Finding: Student reported engagement in service-learning and other high-impact practices were not as high as institution desired.  Action: A new Office of Leadership and Service was created to coordinate service learning opportunities, promote service learning, and provide support to faculty interested in developing service learning courses.

Example of Data Use: Supportive Environment and Retention SOUTHERN CONNECTICUT STATE UNIVERSITY  Finding: BCSSE and NSSE data from multiple years showed that non- returning students had different levels of relationships with faculty, peers, and administrative personnel than returning students.  Action: NSSE results on supportive environment and quality of interactions related to persistence helped focused institutional action on increasing support for learning and promoting quality interactions to aid retention efforts.

Example of Data Use: Faculty and Staff Development ILLINOIS STATE UNIVERSITY  Finding: Needed to increase campus dialogue relevant to student learning among students, faculty, and student affairs personnel alike.  Action: A four-part series, focusing on manageable methods to improve the quality of student writing, was developed for faculty based on the data from the FSSE and NSSE surveys.

Example of Data Use: Foster Collaboration and Focus TEXAS CHRISTIAN UNIVERSITY  Finding: Initially saw lower NSSE and FSSE scores than desired on various engagement activities.  Action: Increased institutional attention and energy surrounding student engagement. Worked to increase collaborative initiatives between Academic and Student Affairs to enhance student engagement in and out of the classroom.

For more information…

Reports Online Detailed data and documentation are available online, including: Supporting Documents  NSSE project overview  NC State comparison groups  Survey instruments  Demographic profile of respondents Data Reports  NC State results compared to each of our three comparison groups  Engagement Indicators  Frequencies and means  Graphical displays  NC State results compared with top 50% and top 10% of all participating institutions  NC State results to “high impact” questions, by student characteristics 63

Indiana University Center for Postsecondary Research 1900 East Tenth Street, Suite 419 Bloomington, IN Phone: Fax: Web: nsse.iub.edu NC State Contact: Nancy Whelchel, Ph.D. Associate Director for Survey Research Office of Institutional Planning and Analysis Phone: Web: Contact Information