1. The University of Texas-Pan American
Faculty Survey of Student Engagement 2004, 2005 & 2006
Results & Observations
Presented by:
Welcome
Leroy E. Philbrook
Office of Institutional Research & Effectiveness
March 2007
Office of Institutional Research and Effectiveness

2. What is FSSE? (pronounced “fessie”)
An instrument designed to measure faculty expectations of student engagement in educational practices that are linked with high levels of learning and development.
Office of Institutional Research and Effectiveness

3. UTPA Participation in FSSE
The response rates of 35%, 39% and 35% were below the averages of all participating schools of 46%, 54% & 54%
Office of Institutional Research and Effectiveness

4. NSSE Survey Benchmarks 5 Benchmarks of Effective Educational Practice
Level of Academic Challenge (LAC)
Challenging intellectual and creative work is central to student learning and collegiate quality. Colleges and universities promote high levels of student achievement by emphasizing the importance of academic effort and setting high expectations for student performance.
Active and Collaborative Learning (ACL)
Students learn more when they are intensely involved in their education and asked to think about what they are learning in different settings. Collaborating with others in solving problems or mastering difficult material prepares students for the messy, unscripted problems they will encounter daily during and after college.
Student-Faculty Interaction (SFI)
Students learn firsthand how experts think about and solve practical problems by interacting with faculty members inside and outside the classroom. As a result, their teachers become role models, mentors, and guides for continuous, life-long learning.
Enriching Educational Experiences (EEE)
Complementary learning opportunities enhance academic programs. Diversity experiences teach students valuable things about themselves and others. Technology facilitates collaboration between peers and instructors. Internships, community service, and senior capstone courses provide opportunities to integrate and apply knowledge.
Supportive Campus Environment (SCE)
Students perform better and are more satisfied at colleges that are committed to their success and cultivate positive working and social relations among different groups on campus.
These are the 5 benchmarks of Effective Educational Practice from the NSSE survey.
(Read Names)
We will look at each Benchmark and some of the items related to these aspect of student engagement.
Office of Institutional Research and Effectiveness

5. NSSE Benchmark 2006 Scores for UTPA
First Year
Senior
NSSE Benchmark
UTPA Score
Compared with …
UT System
Carnegie Peers
NSSE 2006
Level of Academic Challenge (LAC) 49 _ 54
Active and Collaborative Learning (ACL) 43 a 55
Student-Faculty Interaction (SFI) 33 42
Enriching Educational Experiences (EEE) 25 34
Supportive Campus Environment (SCE) 65 60
These are the scores for each benchmark from the most recent NSSE
+ can be considered as good and the – considered as not so good
We are doing well on SCE and ACL compared to the other groups
SFI is on the plus side of neutral
On LAC, compared to UT System we are at par. Compared to the others, we are below.
On EEE for the first year students we are on par with UT and Carnegie Peers.
For Seniors we need to work on improvements which should begin when they are freshmen.
The Scale is 100
indicates the score of UTPA is less than this comparison group
Indicates the score of UTPA is greater than this comparison group
Blank indicates no statistically significant difference
Office of Institutional Research and Effectiveness

6. Items related to NSSE Benchmarks Level of Academic Challenge (LAC)
These are a few of the elements related to Level of Academic challenge. The colors correspond to the year of the survey. From left to right on each table you have Faculty Lower division, upper division,
2004
2005
2006
Percentage of faculty who reported that more than half of the students …
Percentage of students who reported that they “Often” or “Very often” …
Office of Institutional Research and Effectiveness

7. Office of Institutional Research and Effectiveness
Items related to NSSE Benchmarks Active and Collaborative Learning (ACL)
2004
2005
2006
Percentage of faculty who reported that more than half of the students …
Percentage of students who reported that they “Often” or “Very often” …
Office of Institutional Research and Effectiveness

8. Items related to NSSE Benchmarks Student – Faculty Interaction (SFI)
2004
2005
2006
Percentage of faculty who reported that more than half of the students …
Percentage of students who reported that they “Often” or “Very often” …
Office of Institutional Research and Effectiveness

9. Office of Institutional Research and Effectiveness
Items related to NSSE Benchmarks Enriching Educational Experiences (EEE)
Percentage of faculty who reported that more than half of the students …
Percentage of faculty who reported that it is important or very important that students at their institution do the following
2004
2005
2006
Percentage of students who reported that they “Often” or “Very often” …
Percentage of students responding ‘have done’ or ‘plan to do’ before graduating …
Office of Institutional Research and Effectiveness

10. Items related to NSSE Benchmarks Supportive Campus Environment (SCE)
2004
2005
2006
Percentage of faculty who reported that students had a positive relationship ...
Percentage of students who reported having a positive relationship …
Office of Institutional Research and Effectiveness

11. Students ‘Often’ & ‘Very Often’
High Discrepancy Frequently work harder than usual to meet instructor’s standards
Class
Faculty Responses
Discrepancy
Student Responses
2004
2005
2006
50% or Higher of Students
Difference
Students ‘Often’ & ‘Very Often’ LD
24%
19%
23%
-31%
-40%
-34%
54%
59%
57% FY UD
31%
39%
32%
-32%
-33%
63%
72% SR
Now I looked at elements where the responses differed by 33% or more.
Percentage of faculty who reported that more than half of the students from their courses…
Percentage of students reporting they ‘Often’ or ‘Very Often’ …
Difference below - 33%
Office of Institutional Research and Effectiveness

12. ‘Very Important’ or ‘Important’ Students Often & Very Often
High Discrepancy Students spending significant amounts of time ‘studying’ and ‘on academic work’
Class
Faculty Responses
Discrepancy
Student Responses
2004
2005
2006
‘Very Important’ or ‘Important’
Difference
Students Often & Very Often LD
56%
39%
49%
-20%
-39%
-31%
76%
78%
80% FY UD
51%
52%
62%
-21%
-24%
-9%
72%
71% SR
Percentage of faculty who reported placing quite a bit or very much emphasis on …
Percentage of students who responded that there was ‘Very Much’ or ‘Quite a Bit’ of emphasis on…
Difference below - 33%
Office of Institutional Research and Effectiveness

13. Students ‘Often’ & ‘Very Often’
High Discrepancy Asked questions in class or contributed to class discussions
Class
Faculty Responses
Discrepancy
Student Responses
2004
2005
2006
50% or Higher of Students
Difference
Students ‘Often’ & ‘Very Often’ LD
13%
12%
18%
-31%
-39%
-37%
45%
51%
55% FY UD
21%
26%
24%
-40%
-44%
61%
65%
68% SR
Percentage of faculty who reported that more than half of the students from their courses…
Percentage of students reporting they ‘Often’ or ‘Very Often’ …
Difference below - 33%
Office of Institutional Research and Effectiveness

14. ‘Very Important’ or ‘Important’ Students ‘Often’ & ‘Very Often’
High Discrepancy Tutor or teach other students (paid or voluntary)
Class
Faculty Responses
Discrepancy
Student Responses
2004
2005
2006
‘Very Important’ or ‘Important’
Difference
Students ‘Often’ & ‘Very Often’ LD
31%
55%
52%
18%
33%
32%
14%
22%
20% FY UD
36%
38% 7%
17%
11%
29%
21%
27% SR
Percentage of faculty who reported that it is ‘Important’ or ‘Very important’ that students …
Percentage of students reporting they ‘Often’ or ‘Very Often’ …
Difference above 33%
Office of Institutional Research and Effectiveness

15. Students ‘Often’ & ‘Very Often’
High Discrepancy Discussed grades or assignments with an instructor
Class
Faculty Responses
Discrepancy
Student Responses
2004
2005
2006
50% or Higher of Students
Difference
Students ‘Often’ & ‘Very Often’ LD
13% 9%
19%
-35%
-47%
-28%
49%
56%
47% FY UD
28%
27%
25%
-26%
-36%
-37%
55%
63%
62% SR
Percentage of faculty who reported that more than half of the students from their courses…
Percentage of students reporting they ‘Often’ or ‘Very Often’ …
Difference below - 33%
Office of Institutional Research and Effectiveness

16. High Discrepancy At least once, student talked about career plans with you
Class
Faculty Responses
Discrepancy
Student Responses
2004
2005
2006
50% or Higher of Students
Difference
Students ‘Often’ & ‘Very Often’ LD 2% 7%
15%
-31%
-37%
-27%
33%
44%
42% FY UD
22%
24%
28%
-23%
-17%
-15%
45%
41%
43% SR
Percentage of faculty who reported that more than half of the students from their courses…
Percentage of students reporting they ‘Often’ or ‘Very Often’ …
Difference below - 33%
Office of Institutional Research and Effectiveness

17. High Discrepancy Received prompt written or oral feedback from faculty on academic performance
Class
Faculty Responses
Discrepancy
Student Responses
2004
2005
2006
Students ‘Often’ & ‘Very Often’
Difference LD
83%
75%
86%
39%
27%
43%
44%
48% FY UD
91%
90%
85%
34%
28%
21%
58%
62%
64% SR
Percentage of faculty who reported that students ‘Often’ or ‘Very Often’ …
Percentage of students reporting they ‘Often’ or ‘Very Often’ …
Difference above 33%
Office of Institutional Research and Effectiveness

18. ‘Important’ or ‘Very important’ ‘Have Done’ or ‘Will Do’ a
High Discrepancy Culminating senior experience (capstone course, senior project or thesis, comprehensive exam, etc.)
Class
Faculty Responses
Discrepancy
Student Responses
2004
2005
2006
‘Important’ or ‘Very important’
Difference
‘Have Done’ or ‘Will Do’ a LD
59%
72%
83%
18%
20%
39%
41%
52%
44% FY UD
66%
79%
80%
12%
21%
25%
54%
58%
55% SR
Percentage of faculty who reported that it is ‘Important’ or ‘Very important’ that students participate in a …
Percentage of students who responded that they ‘Have Done’ or ‘Will Do’ a…
Difference above 33%
Office of Institutional Research and Effectiveness

19. High Discrepancy Administrative personnel and offices
Class
Faculty Responses
Discrepancy
Student Responses
2004
2005
2006
Students have a ‘Positive Relationship’ with
Difference
Students ‘Often’ & ‘Very Often’ LD
56%
39%
49%
-20%
-39%
-31%
76%
78%
80% FY UD
51%
52%
62%
-21%
-24%
-9%
72%
71% SR
Percentage of faculty who reported that students have a ‘positive relationship’ with …
Percentage of students ratings of the quality of their relationships with…
Difference below - 33%
Office of Institutional Research and Effectiveness

20. High Discrepancy Used e-mail to communicate with an instructor
Class
Faculty Responses
Discrepancy
Student Responses
2004
2005
2006
50% or Higher of Students
Difference
Students ‘Often’ & ‘Very Often’ LD 8%
12%
18%
-49%
-51%
-46%
56%
63%
64% FY UD
21%
25%
27%
-48%
-54%
70%
73%
81% SR
Percentage of faculty who reported that more than half of the students from their courses…
Percentage of students reporting they ‘Often’ or ‘Very Often’ …
Difference below - 33%
Office of Institutional Research and Effectiveness

21. High Discrepancy Examined the strengths and weaknesses of own views on a topic or issue
Class
Faculty Responses
Discrepancy
Student Responses
2004
2005
2006
‘Very Important’ or ‘Important’
Difference
Students ‘Often’ & ‘Very Often’ LD
73%
76%
26%
34%
47%
42% FY UD
77%
68%
27%
22%
50%
46% SR
Percentage of faculty who reported that it is ‘Important’ or ‘Very important’ that students …
Percentage of students reporting they ‘Often’ or ‘Very Often’ …
Difference above 33%
Note: This question not on the 2004 survey.
Office of Institutional Research and Effectiveness

22. ‘Very Much’ or ‘Quite a Bit’
High Discrepancy Memorizing facts, ideas, or methods from course and readings
Class
Faculty Responses
Discrepancy
Student Responses
2004
2005
2006
‘Very Much’ or ‘Quite a Bit’
Difference LD
29%
27%
31%
-47%
-48%
-38%
76%
75%
69% FY UD
34%
-44%
-31%
-33%
71%
65%
60% SR
Percentage of faculty who reported placing ‘quite a bit’ or ‘very much’ emphasis on …
Percentage of students who responded that there was ‘very much’ or ‘quite a bit’ of emphasis on…
Difference below - 33%
Office of Institutional Research and Effectiveness

23. ‘Very Much’ or ‘Quite a Bit’
High Discrepancy Using computing and information technology
Class
Faculty Responses
Discrepancy
Student Responses
2004
2005
2006
‘Very Much’ or ‘Quite a Bit’
Difference LD
44%
34%
39%
-37%
-46%
-43%
82%
80% FY UD
41%
48%
51%
-42%
83%
85%
88% SR
Percentage of faculty who structured their courses ‘quite a bit’ or ‘very much’ so that students learn and develop …
Percentage of students who responded that their college experience contributed to their knowledge, skills, and personal development ‘very much’ or ‘quite a bit’ in…
Difference below - 33%
Office of Institutional Research and Effectiveness

24. Students ‘Often’ & ‘Very Often’
High Discrepancy Developing a deepened sense of spirituality
Class
Faculty Responses
Discrepancy
Student Responses
2004
2005
2006
Students ‘Often’ & ‘Very Often’
Difference LD
14%
11%
12%
-20%
-25%
-38%
33%
36%
50% FY UD
29%
25%
23% 1%
-8%
-7%
28%
30% SR
Percentage of faculty who structured their courses ‘quite a bit’ or ‘very much’ so that students learn and develop …
Percentage of students who responded that their college experience contributed to their knowledge, skills, and personal development ‘very much’ or ‘quite a bit’ in…
Difference below - 33%
Office of Institutional Research and Effectiveness

25. Related to NSSE Benchmark: Level of Academic Challenge (LAC)
Reflections
Related to NSSE Benchmark: Level of Academic Challenge (LAC)
Faculty over-estimate student perception of the emphasis on critical thinking skills, i. e., Analysis, Synthesis, Evaluation, and Application.
Faculty underestimate student perception of how hard they work and the curriculum emphasis of memorization.
Is this a symptom of the “Teaching to the Test” conflicting with Critical Thinking?
Note: The element related to memorizing facts and ideas is not included in this Benchmark.
Office of Institutional Research and Effectiveness

26. Related to NSSE Benchmark: Active and Collaborative Learning (ACL)
Reflections
Related to NSSE Benchmark: Active and Collaborative Learning (ACL)
Faculty estimate of lower division student participation in “out of class” discussion of course content with classmates and others was above the student estimate.
Faculty expressed higher levels of the importance of tutoring or teaching other students while students reported lower levels of tutoring or teaching other students.
Faculty estimate of student participation in class (frequency of asking question and of in-class projects) and out of class discussion content for upper division students with classmates and others was below the student estimate.
UTPA’s NSSE Scores for this benchmark were higher than most of the comparison groups.
Office of Institutional Research and Effectiveness

27. Related to NSSE Benchmarks Student – Faculty Interaction (SFI)
Reflections
Related to NSSE Benchmarks Student – Faculty Interaction (SFI)
Faculty estimate of student faculty “outside of class” conversations was below the student estimate.
Faculty estimate of frequency of prompt written or oral feedback on their academic performance and of student work on a research project with a faculty member outside of course or program requirements was higher than student perception.
Faculty initiated Student-Faculty relationships tend to be overestimated by Faculty while student initiated Student-Faculty relationships tend to be underestimated.
Office of Institutional Research and Effectiveness

28. Related to NSSE Benchmarks Enriching Educational Experiences (EEE)
Reflections
Related to NSSE Benchmarks Enriching Educational Experiences (EEE)
Faculty expressed higher levels of the importance for formal educational enriching experiences than is reflected in student reported levels of have done or plan to do, i.e., practicum, Foreign language coursework, study abroad, or capstone course.
Faculty expressed lower levels of the importance for informal educational enriching experiences than is reflected in student reported levels of have done or plan to do, i.e., conversations with people of other ethnicity or social status, use of electronic media to discuss or complete course work, or conversation with people with other religious, social, political or personal values.
This may reflect the difference in the questions between faculty and students. Faculty are asked a theoretical question. Students ask a practical question.
UTPA’s NSSE Scores for this benchmark were lower than most of the comparison groups.
Office of Institutional Research and Effectiveness

29. Related to NSSE Benchmarks Supportive Campus Environment (SCE)
Reflections
Related to NSSE Benchmarks Supportive Campus Environment (SCE)
Faculty estimates of the emphasis on aspects of supportive campus environment were generally consistent with student estimates.
Faculty underestimate student perception of a positive relationship to other students, faculty and administrative personnel.
UTPA’s NSSE Scores for this benchmark were higher than all comparison groups.
Office of Institutional Research and Effectiveness

30. Office of Institutional Research and Effectiveness
Reflections
Miscellaneous Items
Faculty underestimate student perception of the frequency of use to communicate with faculty. This item may reflect the faculty evaluating a course and students evaluating general experience.
Faculty under-estimate student perception of the emphasis on Memorizing facts, ideas, or methods from your course and readings This item was mentioned earlier
Faculty under-estimate student perception of the emphasis on using computing and information technology This item may reflect the faculty evaluating a course and students evaluating general experience.
Faculty under-estimate the extent to which college is contributing to student developing a deepened sense of spirituality. This item may not be related to the educational context for most faculty.
Faculty level of importance of student examining the strengths and weaknesses of their views on a topic or issue is higher than the reported level of student participation. This may reflect the difference between a theoretical question for the faculty and a practical question for students.
Office of Institutional Research and Effectiveness

31. Questions & Discussion Leroy E. Philbrook lphilbrook@utpa.edu
Contact Information:
Leroy E. Philbrook
Phone: (956)
This presentation is online at:
Office of Institutional Research and Effectiveness