National Survey of Student Engagement, 2008 Results for UBC-Vancouver

NSSE Overview   Supported as an assessment tool by UBC many other Canadian Universities including all of G13   Over 1200 colleges and universities across Canada and the U.S. have participated in NSSE since the first pilot in 1999; ~450,000 first and senior year respondents in 2008   Supported by strong research; NSSE provides an estimate of how undergraduates spend their time and what they gain from attending college   Survey items represent empirically confirmed "good practices" in undergraduate education. That is, they reflect behaviors by students and institutions that are associated with desired outcomes of college.   125+ questions (core, contextual, experimental, consortium) and 5 composite engagement benchmarks   Results provided for participating university and selected comparator groups   Additional info at

Engagement Benchmarks NSSE provides five benchmarks of effective educational practices:   Level of Academic Challenge (LAC): amount of time studying, reading, writing; academic effort; coursework emphasis on analysis, synthesis, applying theories   Active and Collaborative Learning (ACL): asking questions, class presentations, teamwork in class, discussions and learning activities outside of class and involvement in community based projects   Student Faculty Interaction (SFI): discussions with faculty on grades, coursework, careers, research involvement with faculty, worked on other projects or committee with faculty   Enriching the Educational Experience (EEE): participation in co- curricular activities, volunteer work, field experience, co-op, community service, serious conversations with students from different ethnic backgrounds, political/religious beliefs, etc.   Supportive Campus Environment (SCE): student, faculty, staff relationships; campus services to help students with both their academic and non-academic responsibilities

Peer Groups   G13 Peer Group: Ontario and Quebec universities had their own consortiums for participation in NSSE in 2008 so the G13 peer group includes UBC, The University of Alberta, University of Calgary and Dalhousie (eventually we will have comparisons for all G13 institutions)   “Carnegie” Cdn Peers consists of McGill, Concordia, McMaster, Toronto, Alberta, Calgary, York (large undergraduate populations)   Selected Peers consists of participating large research intensive universities in the U.S.: The University of Texas at Austin, University of Illinois at Urbana-Champaign, University of Minnesota-Twin Cities, University of North Carolina at Chapel Hill, University of Virginia, University of Wisconsin-Madison

Interpreting the Data   Responses are reported in the form of means and frequencies. Items that make up the five benchmarks of effective educational practices are identified in the means report and are also aggregated in the benchmarks report.   This year we report differences among peer groups AND differences for UBC since Important differences are identified by: a) statistical significance (is the change unlikely to be simple chance variation?) and b) effect sizes   Effect size indicates the “practical significance” of the mean difference. In practice, an effect size of.2 is often considered small,.5 moderate, and.8 large.

Summary of Findings   UBC 2006 compared to UBC 2008: Areas of improvement and decline   UBC compared to Canadian Peers   UBC compared to U.S. Peers   Overall Satisfaction   Student Priorities on improving the learning environment   Next Steps

All changes are statistically significant. All effect sizes are small.

Areas of improvement, First Year, Very small, statistically significant increases, especially in areas where we were weak.   ACL, Worked with classmates outside of class   ACL, Tutored or taught other students   ACL, Participated in a community-based project as part of course   EEE, Had serious conversations with students of a different race or ethnicity   EEE, Community service or volunteer work   EEE, Encouraging contact among students from different economic, social, ethnic backgrounds   LAC, Worked harder to meet expectations   LAC, Making judgments about the value of information, arguments, or methods   SCE, Providing support to succeed academically   SCE, Helping you cope with your non-academic responsibilities   SFI, Talked about career plans with a faculty member or advisor   SFI, Discussed ideas from your readings or classes with faculty members outside of class   SFI, Received prompt written or oral feedback from faculty on your academic performance   SFI, Worked with faculty members on activities other than coursework (committees, orientation, student life activities, etc.)   SFI, Work on a research project with a faculty member outside of course or program requirements

Areas of decline, First Year, None of the decreases in benchmark items were statistically significant. (and very few items actually declined)

Areas of improvement, Fourth Year, Very small, statistically significant increases, especially in areas where we were weak.   ACL, Worked with other students on projects during class   ACL, Worked with classmates outside of class to prepare class assignments   ACL, Tutored or taught other students (paid or voluntary)   ACL, Participated in a community-based project as part of a regular course   EEE, Used an electronic medium to discuss or complete an assignment   EEE, Had serious conversations with students of a different race or ethnicity than your own   EEE, Practicum, internship, field experience, co-op experience, or clinical assignment   EEE, Participate in a learning community where groups of students take two or more classes together   EEE, Culminating senior experience (capstone course, senior project or thesis, comprehensive exam, etc.)   EEE, Encouraging contact among students from different economic, social, and racial or ethnic backgrounds   LAC, Worked harder to meet expectations   LAC, Making judgments about the value of information, arguments, or methods   LAC, Applying theories or concepts to practical problems or in new situations   LAC, Hours per 7-day week spent preparing for class   LAC, Spending significant amounts of time studying and on academic work   SCE, Relationships with faculty members   SCE, Providing the support you need to help you succeed academically   SCE, Helping you cope with your non-academic responsibilities (work, family, etc.)   SCE, Providing the support you need to thrive socially   SFI, Talked about career plans with a faculty member or advisor   SFI, Discussed ideas from your readings or classes with faculty members outside of class   SFI, Worked with faculty members on activities other than coursework (committees, orientation, etc.)   SFI, Work on a research project with a faculty member outside of course or program requirements

Areas of decline, Fourth Year, None of the decreases in benchmark items were statistically significant. (and very few items actually declined)

UBC compared to Canadian and U.S. Peers Canadian Peers:   UBC’s benchmark scores are similar to those of other Canadian Peer Universities.   UBC does relatively better in the area of Enriching Educational Experiences (first and fourth year) and Active and Collaborative Learning (first year)   UBC scores slightly lower in the area of Supportive Campus Environment (first year) and Level of Academic Challenge (fourth year). U.S. Peers:   UBC’s scores are significantly lower on all benchmarks compared to U.S. Peer Institutions (first and fourth year, statistically significant, moderate effect sizes).   We can hypothesize several reasons for this: - - Different demographics - - Different Socio-Economic status of students - - Lower funding, larger classes - - Commuter campus

Overall Satisfaction   In addition to the questions which comprise the five educational benchmarks, NSSE also asks questions which explore overall student satisfaction as well as other areas related to both the academic and non-academic environment.

Student Priorities NSSE allows for the inclusion of institutional specific questions for consortium participants. UBC, together with Dalhousie, Alberta and Calgary, provided customized questions which were asked of all first and fourth year students. Students were asked to identify two areas that UBC needs to address in order to improve student learning in the classroom and outside the classroom

Next Steps   Analyze student comments and compare to select peer institutions   Provide NSSE data to deans at faculty level   Drill down to specific majors for Arts, majors/honors for science, and ILP programs (eg. CAP)   NSSE Deep learning scales and analysis by aboriginal, visible minority, international   Use NSSE as the stimulus for discipline specific learning outcomes (e.g.: what more do you need to know to guide learning in your discipline?)   Focus NSSE discussion to align with institutional strategic goals (Trek 2010)