The University of Texas-Pan American Faculty Survey of Student Engagement 2004, 2005 & 2006 Results & Observations Presented by: Welcome Leroy E. Philbrook Office of Institutional Research & Effectiveness March 2007 Office of Institutional Research and Effectiveness

What is FSSE? (pronounced “fessie”) An instrument designed to measure faculty expectations of student engagement in educational practices that are linked with high levels of learning and development. Office of Institutional Research and Effectiveness

UTPA Participation in FSSE The response rates of 35%, 39% and 35% were below the averages of all participating schools of 46%, 54% & 54% Office of Institutional Research and Effectiveness

NSSE Survey Benchmarks 5 Benchmarks of Effective Educational Practice Level of Academic Challenge (LAC) Challenging intellectual and creative work is central to student learning and collegiate quality. Colleges and universities promote high levels of student achievement by emphasizing the importance of academic effort and setting high expectations for student performance. Active and Collaborative Learning (ACL) Students learn more when they are intensely involved in their education and asked to think about what they are learning in different settings. Collaborating with others in solving problems or mastering difficult material prepares students for the messy, unscripted problems they will encounter daily during and after college. Student-Faculty Interaction (SFI) Students learn firsthand how experts think about and solve practical problems by interacting with faculty members inside and outside the classroom. As a result, their teachers become role models, mentors, and guides for continuous, life-long learning. Enriching Educational Experiences (EEE) Complementary learning opportunities enhance academic programs. Diversity experiences teach students valuable things about themselves and others. Technology facilitates collaboration between peers and instructors. Internships, community service, and senior capstone courses provide opportunities to integrate and apply knowledge. Supportive Campus Environment (SCE) Students perform better and are more satisfied at colleges that are committed to their success and cultivate positive working and social relations among different groups on campus. These are the 5 benchmarks of Effective Educational Practice from the NSSE survey. (Read Names) We will look at each Benchmark and some of the items related to these aspect of student engagement. Office of Institutional Research and Effectiveness

NSSE Benchmark 2006 Scores for UTPA First Year Senior NSSE Benchmark UTPA Score Compared with … UT System Carnegie Peers NSSE 2006 Level of Academic Challenge (LAC) 49   _ 54 Active and Collaborative Learning (ACL) 43 a 55 Student-Faculty Interaction (SFI) 33 42 Enriching Educational Experiences (EEE) 25 34 Supportive Campus Environment (SCE) 65 60 These are the scores for each benchmark from the most recent NSSE + can be considered as good and the – considered as not so good We are doing well on SCE and ACL compared to the other groups SFI is on the plus side of neutral On LAC, compared to UT System we are at par. Compared to the others, we are below. On EEE for the first year students we are on par with UT and Carnegie Peers. For Seniors we need to work on improvements which should begin when they are freshmen. The Scale is 100 indicates the score of UTPA is less than this comparison group Indicates the score of UTPA is greater than this comparison group Blank indicates no statistically significant difference Office of Institutional Research and Effectiveness

Items related to NSSE Benchmarks Level of Academic Challenge (LAC) These are a few of the elements related to Level of Academic challenge. The colors correspond to the year of the survey. From left to right on each table you have Faculty Lower division, upper division, 2004 2005 2006 Percentage of faculty who reported that more than half of the students … Percentage of students who reported that they “Often” or “Very often” … Office of Institutional Research and Effectiveness

Office of Institutional Research and Effectiveness Items related to NSSE Benchmarks Active and Collaborative Learning (ACL) 2004 2005 2006 Percentage of faculty who reported that more than half of the students … Percentage of students who reported that they “Often” or “Very often” … Office of Institutional Research and Effectiveness

Items related to NSSE Benchmarks Student – Faculty Interaction (SFI) 2004 2005 2006 Percentage of faculty who reported that more than half of the students … Percentage of students who reported that they “Often” or “Very often” … Office of Institutional Research and Effectiveness

Office of Institutional Research and Effectiveness Items related to NSSE Benchmarks Enriching Educational Experiences (EEE) Percentage of faculty who reported that more than half of the students … Percentage of faculty who reported that it is important or very important that students at their institution do the following 2004 2005 2006 Percentage of students who reported that they “Often” or “Very often” … Percentage of students responding ‘have done’ or ‘plan to do’ before graduating … Office of Institutional Research and Effectiveness

Items related to NSSE Benchmarks Supportive Campus Environment (SCE) 2004 2005 2006 Percentage of faculty who reported that students had a positive relationship ... Percentage of students who reported having a positive relationship … Office of Institutional Research and Effectiveness

Students ‘Often’ & ‘Very Often’ High Discrepancy Frequently work harder than usual to meet instructor’s standards Class Faculty Responses Discrepancy Student Responses 2004 2005 2006 50% or Higher of Students Difference Students ‘Often’ & ‘Very Often’ LD 24% 19% 23% -31% -40% -34% 54% 59% 57% FY UD 31% 39% 32% -32% -33% 63% 72% SR Now I looked at elements where the responses differed by 33% or more. Percentage of faculty who reported that more than half of the students from their courses… Percentage of students reporting they ‘Often’ or ‘Very Often’ … Difference below - 33% Office of Institutional Research and Effectiveness

‘Very Important’ or ‘Important’ Students Often & Very Often High Discrepancy Students spending significant amounts of time ‘studying’ and ‘on academic work’ Class Faculty Responses Discrepancy Student Responses 2004 2005 2006 ‘Very Important’ or ‘Important’ Difference Students Often & Very Often LD 56% 39% 49% -20% -39% -31% 76% 78% 80% FY UD 51% 52% 62% -21% -24% -9% 72% 71% SR Percentage of faculty who reported placing quite a bit or very much emphasis on … Percentage of students who responded that there was ‘Very Much’ or ‘Quite a Bit’ of emphasis on… Difference below - 33% Office of Institutional Research and Effectiveness

Students ‘Often’ & ‘Very Often’ High Discrepancy Asked questions in class or contributed to class discussions Class Faculty Responses Discrepancy Student Responses 2004 2005 2006 50% or Higher of Students Difference Students ‘Often’ & ‘Very Often’ LD 13% 12% 18% -31% -39% -37% 45% 51% 55% FY UD 21% 26% 24% -40% -44% 61% 65% 68% SR Percentage of faculty who reported that more than half of the students from their courses… Percentage of students reporting they ‘Often’ or ‘Very Often’ … Difference below - 33% Office of Institutional Research and Effectiveness

‘Very Important’ or ‘Important’ Students ‘Often’ & ‘Very Often’ High Discrepancy Tutor or teach other students (paid or voluntary) Class Faculty Responses Discrepancy Student Responses 2004 2005 2006 ‘Very Important’ or ‘Important’ Difference Students ‘Often’ & ‘Very Often’ LD 31% 55% 52% 18% 33% 32% 14% 22% 20% FY UD 36% 38% 7% 17% 11% 29% 21% 27% SR Percentage of faculty who reported that it is ‘Important’ or ‘Very important’ that students … Percentage of students reporting they ‘Often’ or ‘Very Often’ … Difference above 33% Office of Institutional Research and Effectiveness

Students ‘Often’ & ‘Very Often’ High Discrepancy Discussed grades or assignments with an instructor Class Faculty Responses Discrepancy Student Responses 2004 2005 2006 50% or Higher of Students Difference Students ‘Often’ & ‘Very Often’ LD 13% 9% 19% -35% -47% -28% 49% 56% 47% FY UD 28% 27% 25% -26% -36% -37% 55% 63% 62% SR Percentage of faculty who reported that more than half of the students from their courses… Percentage of students reporting they ‘Often’ or ‘Very Often’ … Difference below - 33% Office of Institutional Research and Effectiveness

High Discrepancy At least once, student talked about career plans with you Class Faculty Responses Discrepancy Student Responses 2004 2005 2006 50% or Higher of Students Difference Students ‘Often’ & ‘Very Often’ LD 2% 7% 15% -31% -37% -27% 33% 44% 42% FY UD 22% 24% 28% -23% -17% -15% 45% 41% 43% SR Percentage of faculty who reported that more than half of the students from their courses… Percentage of students reporting they ‘Often’ or ‘Very Often’ … Difference below - 33% Office of Institutional Research and Effectiveness

High Discrepancy Received prompt written or oral feedback from faculty on academic performance Class Faculty Responses Discrepancy Student Responses 2004 2005 2006 Students ‘Often’ & ‘Very Often’ Difference LD 83% 75% 86% 39% 27% 43% 44% 48% FY UD 91% 90% 85% 34% 28% 21% 58% 62% 64% SR Percentage of faculty who reported that students ‘Often’ or ‘Very Often’ … Percentage of students reporting they ‘Often’ or ‘Very Often’ … Difference above 33% Office of Institutional Research and Effectiveness

‘Important’ or ‘Very important’ ‘Have Done’ or ‘Will Do’ a High Discrepancy Culminating senior experience (capstone course, senior project or thesis, comprehensive exam, etc.) Class Faculty Responses Discrepancy Student Responses 2004 2005 2006 ‘Important’ or ‘Very important’ Difference ‘Have Done’ or ‘Will Do’ a LD 59% 72% 83% 18% 20% 39% 41% 52% 44% FY UD 66% 79% 80% 12% 21% 25% 54% 58% 55% SR Percentage of faculty who reported that it is ‘Important’ or ‘Very important’ that students participate in a … Percentage of students who responded that they ‘Have Done’ or ‘Will Do’ a… Difference above 33% Office of Institutional Research and Effectiveness

High Discrepancy Administrative personnel and offices Class Faculty Responses Discrepancy Student Responses 2004 2005 2006 Students have a ‘Positive Relationship’ with Difference Students ‘Often’ & ‘Very Often’ LD 56% 39% 49% -20% -39% -31% 76% 78% 80% FY UD 51% 52% 62% -21% -24% -9% 72% 71% SR Percentage of faculty who reported that students have a ‘positive relationship’ with … Percentage of students ratings of the quality of their relationships with… Difference below - 33% Office of Institutional Research and Effectiveness

High Discrepancy Used e-mail to communicate with an instructor Class Faculty Responses Discrepancy Student Responses 2004 2005 2006 50% or Higher of Students Difference Students ‘Often’ & ‘Very Often’ LD 8% 12% 18% -49% -51% -46% 56% 63% 64% FY UD 21% 25% 27% -48% -54% 70% 73% 81% SR Percentage of faculty who reported that more than half of the students from their courses… Percentage of students reporting they ‘Often’ or ‘Very Often’ … Difference below - 33% Office of Institutional Research and Effectiveness

High Discrepancy Examined the strengths and weaknesses of own views on a topic or issue Class Faculty Responses Discrepancy Student Responses 2004 2005 2006 ‘Very Important’ or ‘Important’ Difference Students ‘Often’ & ‘Very Often’ LD   73% 76% 26% 34% 47% 42% FY UD 77% 68% 27% 22% 50% 46% SR Percentage of faculty who reported that it is ‘Important’ or ‘Very important’ that students … Percentage of students reporting they ‘Often’ or ‘Very Often’ … Difference above 33% Note: This question not on the 2004 survey. Office of Institutional Research and Effectiveness

‘Very Much’ or ‘Quite a Bit’ High Discrepancy Memorizing facts, ideas, or methods from course and readings Class Faculty Responses Discrepancy Student Responses 2004 2005 2006 ‘Very Much’ or ‘Quite a Bit’ Difference LD 29% 27% 31% -47% -48% -38% 76% 75% 69% FY UD 34% -44% -31% -33% 71% 65% 60% SR Percentage of faculty who reported placing ‘quite a bit’ or ‘very much’ emphasis on … Percentage of students who responded that there was ‘very much’ or ‘quite a bit’ of emphasis on… Difference below - 33% Office of Institutional Research and Effectiveness

‘Very Much’ or ‘Quite a Bit’ High Discrepancy Using computing and information technology Class Faculty Responses Discrepancy Student Responses 2004 2005 2006 ‘Very Much’ or ‘Quite a Bit’ Difference LD 44% 34% 39% -37% -46% -43% 82% 80% FY UD 41% 48% 51% -42% 83% 85% 88% SR Percentage of faculty who structured their courses ‘quite a bit’ or ‘very much’ so that students learn and develop … Percentage of students who responded that their college experience contributed to their knowledge, skills, and personal development ‘very much’ or ‘quite a bit’ in… Difference below - 33% Office of Institutional Research and Effectiveness

Students ‘Often’ & ‘Very Often’ High Discrepancy Developing a deepened sense of spirituality Class Faculty Responses Discrepancy Student Responses 2004 2005 2006 Students ‘Often’ & ‘Very Often’ Difference LD 14% 11% 12% -20% -25% -38% 33% 36% 50% FY UD 29% 25% 23% 1% -8% -7% 28% 30% SR Percentage of faculty who structured their courses ‘quite a bit’ or ‘very much’ so that students learn and develop … Percentage of students who responded that their college experience contributed to their knowledge, skills, and personal development ‘very much’ or ‘quite a bit’ in… Difference below - 33% Office of Institutional Research and Effectiveness

Related to NSSE Benchmark: Level of Academic Challenge (LAC) Reflections Related to NSSE Benchmark: Level of Academic Challenge (LAC) Faculty over-estimate student perception of the emphasis on critical thinking skills, i. e., Analysis, Synthesis, Evaluation, and Application. Faculty underestimate student perception of how hard they work and the curriculum emphasis of memorization. Is this a symptom of the “Teaching to the Test” conflicting with Critical Thinking? Note: The element related to memorizing facts and ideas is not included in this Benchmark. Office of Institutional Research and Effectiveness

Related to NSSE Benchmark: Active and Collaborative Learning (ACL) Reflections Related to NSSE Benchmark: Active and Collaborative Learning (ACL) Faculty estimate of lower division student participation in “out of class” discussion of course content with classmates and others was above the student estimate. Faculty expressed higher levels of the importance of tutoring or teaching other students while students reported lower levels of tutoring or teaching other students. Faculty estimate of student participation in class (frequency of asking question and of in-class projects) and out of class discussion content for upper division students with classmates and others was below the student estimate. UTPA’s NSSE Scores for this benchmark were higher than most of the comparison groups. Office of Institutional Research and Effectiveness

Related to NSSE Benchmarks Student – Faculty Interaction (SFI) Reflections Related to NSSE Benchmarks Student – Faculty Interaction (SFI) Faculty estimate of student faculty “outside of class” conversations was below the student estimate. Faculty estimate of frequency of prompt written or oral feedback on their academic performance and of student work on a research project with a faculty member outside of course or program requirements was higher than student perception. Faculty initiated Student-Faculty relationships tend to be overestimated by Faculty while student initiated Student-Faculty relationships tend to be underestimated. Office of Institutional Research and Effectiveness

Related to NSSE Benchmarks Enriching Educational Experiences (EEE) Reflections Related to NSSE Benchmarks Enriching Educational Experiences (EEE) Faculty expressed higher levels of the importance for formal educational enriching experiences than is reflected in student reported levels of have done or plan to do, i.e., practicum, Foreign language coursework, study abroad, or capstone course. Faculty expressed lower levels of the importance for informal educational enriching experiences than is reflected in student reported levels of have done or plan to do, i.e., conversations with people of other ethnicity or social status, use of electronic media to discuss or complete course work, or conversation with people with other religious, social, political or personal values. This may reflect the difference in the questions between faculty and students. Faculty are asked a theoretical question. Students ask a practical question. UTPA’s NSSE Scores for this benchmark were lower than most of the comparison groups. Office of Institutional Research and Effectiveness

Related to NSSE Benchmarks Supportive Campus Environment (SCE) Reflections Related to NSSE Benchmarks Supportive Campus Environment (SCE) Faculty estimates of the emphasis on aspects of supportive campus environment were generally consistent with student estimates. Faculty underestimate student perception of a positive relationship to other students, faculty and administrative personnel. UTPA’s NSSE Scores for this benchmark were higher than all comparison groups. Office of Institutional Research and Effectiveness

Office of Institutional Research and Effectiveness Reflections Miscellaneous Items Faculty underestimate student perception of the frequency of e-mail use to communicate with faculty. This item may reflect the faculty evaluating a course and students evaluating general experience. Faculty under-estimate student perception of the emphasis on Memorizing facts, ideas, or methods from your course and readings This item was mentioned earlier Faculty under-estimate student perception of the emphasis on using computing and information technology This item may reflect the faculty evaluating a course and students evaluating general experience. Faculty under-estimate the extent to which college is contributing to student developing a deepened sense of spirituality. This item may not be related to the educational context for most faculty. Faculty level of importance of student examining the strengths and weaknesses of their views on a topic or issue is higher than the reported level of student participation. This may reflect the difference between a theoretical question for the faculty and a practical question for students. Office of Institutional Research and Effectiveness

Questions & Discussion Leroy E. Philbrook lphilbrook@utpa.edu Contact Information: Leroy E. Philbrook lphilbrook@utpa.edu Phone: (956) 316-7146 This presentation is online at: http://www.oire.utpa.edu/publications/fsse2004-6.ppt Office of Institutional Research and Effectiveness