CARL-N Mini-conference Students and Faculty Perspectives: Are We Engaged Yet? Presented by Yuhfen Diana Wu Dr. Martin Luther King, Jr. Library San Jose State University December 3, 2007

CARL-N Mini-conference Teacher-Scholar Cohort of Emily Allen, Engineering Joanne Rossi Becker, Science Elizabeth Cara, Applied Sciences and Arts Peg Hughes, Education Arvinder Loomba, Business Linda Mitchell, Humanities and the Arts Ronald Rogers, Social Sciences Diana Wu, University Library

CARL-N Mini-conference Outline Genesis / Purpose of Research Student Engagement Defined Research Questions Survey Methods / CLASSE Pilot Instrument Analysis of Data Lessons Learned Future Research Directions

CARL-N Mini-conference Genesis / Purpose of Research Purpose −To understand university faculty members' and students’ perspectives regarding multiple facets of student engagement. Importance −To monitor student engagement, a students’ “connectedness,” and sense of belonging is very important because these factors individually and collectively contribute to student success.

CARL-N Mini-conference Definition of Student Engagement Student engagement depicts students’ willingness to participate in routine school activities, such as attending classes, submitting required work, and following teachers’ directions in class (Nystrand & Gamoran, 1992). Motivated behavior is indexed by the kinds of cognitive strategies students choose to use and by their willingness to persist with difficult tasks by regulating their own learning behavior (Pintrich & De Groot, 1990; Pintrich & Schrauben, 1992).

CARL-N Mini-conference NSSE Instrument The National Survey of Student Engagement is a "best practices" survey. The survey asks students to report on how they choose to spend their time while in college, recognizing that some uses of time produce more benefits to students than others. The five NSSE benchmarks are aggregations of clusters of survey items and, as such, are defined by those clusters. −Level of Academic Challenge −Active and Collaborative Learning −Student Interactions with Faculty −Enriching Educational Experiences −Supportive Campus Environment

CARL-N Mini-conference Background Literature Much of the research on student engagement has been conducted at the high school and elementary school levels; little exists on the university level (CLASSE still in pilot stage). Much of the existing research is from the perspectives of the students; faculty perspectives are relatively unexplored. Little of the research investigates faculty perceptions of the linkage between social characteristics (such as race and ethnicity, class, and gender) and the degree of student engagement.

CARL-N Mini-conference Research Questions 1.What do faculty rate as important in a class? 2.Are there differences between faculty ratings in on- line and face-to-face classes? 3.Are there differences in student-reported performance between on-line and face-to-face classes? 4.Can such differences be accounted for by demographic variables such as native language?

CARL-N Mini-conference Survey Methods – the CLASSE Survey A two-component tool that compares faculty expectations with what students report experiencing in a class. −CLASSE faculty −CLASSE student Based on National Survey of Student Engagement (NSSE) supported by CIBER at Indiana University.

CARL-N Mini-conference Survey Question Format Survey has 38 questions distributed among domains −Engagement Activities −Cognitive Skills −Other Educational Practices −Class Atmosphere 4-point Likert scale on all items Additional demographic items for both faculty and students

CARL-N Mini-conference Survey Question Format Faculty Survey −Faculty rate the importance of each item −Score is high if the item is important for student to be successful Student Survey −Students rate frequency or amount of performance −Score is high if they have higher performance on this item

CARL-N Mini-conference Engagement Study Data Analysis

CARL-N Mini-conference Interpreting our Results Comparison between faculty and student scores −Difference Scores −Quadrant Analysis Differences between the item rankings of Online and Face-to-face Faculty

CARL-N Mini-conference Interpreting our Results: Difference Scores Calculation: −Teacher Rating minus Student Rating What do the differences mean? −Positive Differences Teacher rates item higher than students −Negative Differences Students rate item higher than teachers

CARL-N Mini-conference Difference Scores: All Classes Engagement Activities Cognitive Skills Other Ed Practices Class Atmosphere

CARL-N Mini-conference Largest Positive Difference (Engagement): How many times have you Question #4: “Worked on a paper or a project in your [Course XYZ] class that required integrating ideas or information from various sources?” Implies that students don’t recognize when they are “integrating ideas or information from various sources,” or Faculty are not actually requiring this of their students despite considering it important Question #6: “Came to your [Course XYZ] class without having completed readings or assignments?” Reversed item: Actually implies that students try to come to class prepared

CARL-N Mini-conference Largest Positive Difference (Engagement): How many times have you Questions #15: “Made a class presentation in your [Course XYZ] class?” Implies… Students perhaps don’t recognize when they are making a “class presentation,” or Faculty don’t assign as many as they think are important, or The presentations are concentrated at end of semester, i.e., after our data collection

CARL-N Mini-conference Largest Positive Difference (Cognitive Skills): So far this semester, how much of your coursework in your [Course XYZ] class emphasized the following mental activities? Question #22: “ Synthesizing and organizing ideas, information, or experiences into new, more complex interpretations and relationships” Difference implies that.. Students don’t recognize when they are “Synthesizing and organizing ideas, information, or experiences,” or Students actually aren’t performing this in class

CARL-N Mini-conference Interpreting our Results: Quadrant Analysis

CARL-N Mini-conference How faculty rank the importance of different aspects of the course Next slides show questions divided by domains −Engagement Activities Q1-Q19 −Cognitive Skills Q20-Q24 −Other Educational Practices Q25-Q34 −Class Atmosphere Q35-Q38

CARL-N Mini-conference

Questions with Significant Mode Differences (F-to-F ranks higher than OL) How important is it that students in your class… Question #7: “Work with other students on projects during your [Course XYZ] class” Question #8: “Work with classmates outside of your [Course XYZ] class to prepare class assignments” Question #15: “Make a class presentation in your [Course XYZ] class?”

CARL-N Mini-conference Questions with Significant Mode Differences (OL ranks higher than F-to-F) How important is it that students in your class… Question 11: “ Use an electronic medium (list-serv, chat group, Internet, instant messaging, etc.) to discuss or complete an assignment in your [Course XYZ] class” Question #12: “Use to communicate with you as the instructor of your [Course XYZ] class”

CARL-N Mini-conference

Summary of Lessons Learned 1.Very useful tool for faculty as formative assessment 2.SJSU strength is approachability of faculty 3.Surprising differences in terms of faculty expectations in on-line vs. face-to-face 4.Low response rate from on-line students 5.More on-line faculty declined to participate (citing #4 above)

CARL-N Mini-conference Future Research Directions Engage proactively by academic librarian Administer survey in more classes next year Look for statistical significance Use for formative assessment Examine demographic data Revise survey for on-line classes

CARL-N Mini-conference THANK YOU … Questions!!!