NLII Focus Session Transformative Inklings and the Mystery of “Faculty Buy-In.” Gary Brown, Director The Center for Teaching, Learning, & Technology Washington State University

The Center for Teaching, Learning, & Technology

CTLT — Who Are We? Staff of 15 technology, assessment, and learning specialists. Work with about 100 WSU faculty each semester

Design Develop Deliver Assess

Student Enrollments in WSU Online Learning Spaces

The Goals, Activities, and Practices Invite faculty to join in the process of formulating the assessment. A series of three short, online surveys designed to provide formative assessment: one instructor survey and two student surveys.

The 3 Survey Goals 1. To help faculty gather useful data about their teaching goals, values and strategies… 2. To help faculty learn how those strategies compare and influence their students’ goals, values, and learning behaviors… 3. And to examine the interaction between goals and practices that faculty might share with each other —and thereby establish a transformative culture of evidence.

GAPs: Encouraging Participation Brown Bags - informal luncheons to help initiate the conversation on assessing teaching and learning. We ed each faculty approximately 10 times to encourage participation… Student surveys were linked to course sites of instructors who answered the instructor survey. We analyzed and returned results promptly to each participating faculty member... Regular updates were sent to instructors with their class results and overall results. We offered and have co-authored papers with faculty to help enhance their professional development. We have talked with chairs, deans, and the assessment coordinator’s office to encourage departments to begin the dialogue with the faculty and to include results as part of their teaching portfolios. We wore out our shoes crossing campus to meet with faculty and invite their contributions to the GAPs instruments…

Participation in GAPs

Sample Findings Significant mismatch between faculty and student goals Online courses are significantly more effective than video based courses. Designed courses scored significantly higher on all principles of good practice Faculty motivation predicts perceptions of efficacy of online learning  Faculty who want to keep abreast of the scholarship of T & L report that online learning is at least as effective as traditional classroom learning  Faculty who teach online primarily for the money report that online learning is not as good as the traditional classroom Buy-in?

The Assessment Gold Standard —Participants Who Used Data to Transform….

However…. One of the two programs endured a budget related, faculty initiated inquisition…..  Demonstrated the program improved freshman retention  Demonstrated the program was cost effective  Demonstrated the program improved student grades, including “special admits” expected to struggle.  Demonstrated the “creative use of technology” and gains in critical information literacy  Demonstrated evidence of good practice.  Demonstrated, by virtue of GAPS, systematic, formative data gathering and responsiveness to that data… GAPS, unlike traditional student evaluations, shows HOW this came about….

Which has resulted in…. The faculty committee that initiated the inquiry was commended by administration for its attentiveness to issues of quality…. They have subsequently been invited to examine other programs… Which has shown signs of transforming the CTLT assessment group into a collegial partner… We are seeing the inklings of a culture of evidence sprouting in a rough, research dominated terrain…

The Boyer Commission Construct an inquiry-based freshman year Build on the freshman foundation Remove barriers to interdisciplinary education Link communication skills and course work Use information technology creatively Make research-based learning the standard Educate graduate-students as teachers Culminate in a capstone experience

The Learning Context: Student WSU Cum GPA by Admissions Quartiles

Dimensions of Critical Thinking 1. Identifies and summarizes the problem/question at issue (and/or the source's position). 2. Identifies and presents the STUDENT’S OWN perspective and position as it is important to the analysis of the issue. 3. Identifies and considers OTHER salient perspectives and positions that are important to the analysis of the issue. 4. Identifies and assesses the key assumptions. 5. Identifies and assesses the quality of supporting data/evidence and provides additional data/evidence related to the issue. 6. Identifies and considers the influence of the context * on the issue. 7. Identifies and assesses conclusions, implications and consequences.

Critical Thinking and Measures of Growth Scant _____________________________________ Substantial Identifies and summarizes the problem/question at issue (and/or the source's position). Does not identify and summarize the problem, is confused or identifies a different and inappropriate problem. Does not identify or is confused by the issue, or represents the issue inaccurately. Identifies the main problem and subsidiary, embedded, or implicit aspects of the problem, and identifies them clearly, addressing their relationships to each other. Identifies not only the basics of the issue, but recognizes nuances of the issue.

Critical Thinking Study—Results Gains in courses when rubric is used—when the faculty in this project integrated the WSU Critical Thinking Rubric into their instruction and assessment, evidence of student gains in critical thinking increased dramatically. Gains from freshmen to junior years—Critical thinking was significantly higher among juniors than among freshmen. But even the writing of juniors had only a mean of 3.1 on a 6 point scale. Comparisons to WSU’s writing assessment—As critical thinking scores went up, writing placement scores and portfolio exam scores went down.

Critical Thinking Gains With CT W/O CT

Additional Findings, Observations, & Implications The dimension of least gain was in students’ abilities to articulate their own viewpoints. The greatest gains by juniors reflect improved abilities to analyze issues from multiple perspectives. Financial incentives have increased faculty adoption of the rubric. The faculty questionnaire revealed a focus on grading over fostering critical thinking for broader life-long learning.