Assessing Integrated Learning Maureen Pettitt, Ph.D. Skagit Valley College Beth Hartsoch, M.A. Western Washington University PNAIRP – Oct ‘07

Session Overview Learning Communities at SVC Established Approaches Student Writing, Course Satisfaction Survey & CCSSE SVC/WWU Project WELS Survey & Focus Groups Themes from the Data New Studies in Progress

Interdisciplinary Learning at SVC Interdisciplinary courses are a means of delivering instruction and fostering student learning A response to curricular issues: Faculty felt that students did not see connections between and among disciplines, and needed to engage subjects more fully, to see education as a dynamic and interconnected process of exploration and discovery

CCSSE Student Writing Student Satisfaction Survey Past Approaches: Multiple Assessment Methods

1. Student Writing Study Faculty/IR team developed two-year research project to assess student attainment of overarching General Education learning outcomes

Student Writing Study: Goals Assess students' ability to: Apply a variety of concepts/texts/contexts and perspectives to solving problems, thinking about issues Connect one’s own life experience, ideas and abilities with those that others bring Understand and value the learning process for oneself and for others Connect to external, lifelong social issues

Student Writing Study: Method Identified courses being taught in Learning Communities, stand-alone, and DE that could be “matched” over a two-year period Faculty agreed to participate and give students course credit Developed a set of questions based on Gen Ed goals and faculty input Students surveyed beginning-, mid-, and end- course

End of Course Questions 1. Have your learning expectations been met? 2. Have you learned things that you hadn't anticipated? If so, please describe. 3. What do you think are the most important aspects of your experiences in this course that account for your learning? 4. What have you learned in this course that will matter to you five years from now?

Student Writing Study: Findings Students’ responses regarding valuing learning, connecting classroom learning with the world, and connecting with other were fairly similar in both stand-alone and LC courses. Students in LC courses were far more likely to cite applying or appreciating a variety of perspectives to problem-solving (55% versus 15%)

Student Writing Study: Student Comments “After taking this course I feel that I can make connections to various things, such as history, influences, people, and culture. This course taught me the value of making connections and things from my own perspective.”

Student Writing Study: Student Comments “By combining course topics you get the ‘bigger picture’ and are able to sort of apply what we are learning better. By applying a subject or topic to another subject or topic you have to comprehend what you are learning and apply it to other things.”

Student Writing Study: Comparisons Comparing student responses in the stand-alone courses with students in collaborative courses—same courses with the same instructor(s): Students in stand-alone courses were less likely to write about these connections or about learning, and tended to focus on personal growth, liking faculty, etc.

2. Quarterly Surveys Faculty-developed pen-and-paper survey Scale from “strongly disagree” to “strongly agree” Faculty are provided their course results and comments, plus the cumulative for the quarter for all courses Several years ago, we created a version that could be scanned

Learning Community Survey Questions 1. "This learning community has been a valuable educational experience." 2. "I believe it was probably more valuable to have taken these classes together than it would have been to take them separately."

Mean Responses to Value Questions (N = 5156)

Linked & Federated Composition LCs (N = 2780)

Student Comments LC: “This class has been fun and not sucky at all. I think I have learned a lot.” Link: “I did not learn too much other than how to explore more in depth, how to write more.” LC: “If they had been separate, I would have known the what and where, but not the why, and the why is always the most important question.”

3. Community College Survey of Student Engagement Administered the CCSSE in 2003, 2005, and 2007 Went to this survey because it most closely matched the institution’s focus on student learning and engagement The LC question was a bonus extra!

CCSSE LC Question Stem: “Which of the following have you done, are you doing, or do you plan to do while attending this college?” Category: “Organized learning communities (linked courses/study groups led by faculty or counselors)” Response Categories: I have done I plan to do I have not done nor plan to do

Method Used T-test for independent samples to examine differences in effort and engagement between students who had taken Learning Communities and those who had not done nor planned to do. Results are only for the latest survey administration (Spring 2007)

Learning Community Sig. VariableHave Taken (n = 173 ) Not Taken (n = 232) Asked questions in class or contributed to class discussions Made a class presentation Responses range from 1 (“Never”) to 4 (“Very Often”) Prompt: “In your experiences at this college during the current school year, about how often have you done each of the following?” Responses range from 1 (“Never”) to 4 (“Very Often”) Findings

Learning Community Sig. VariableHave Taken (n = 173 ) Not Taken (n = 232) Prepared two or more drafts of a paper or assignment before turning it in Worked on a paper or project that required integrating ideas or information from various sources Worked with other students on projects during class Worked with classmates outside of class to prepare class assignments Discussed ideas from your readings or classes with others outside of class (students, family members, co-workers, etc.)

Learning Community Sig. VariableHave Taken (n = 173 ) Not Taken (n = 232) Worked harder than you thought you could to meet an instructor’s standards or expectations Used to communicate with an instructor Discussed grades or assignments with an instructor Discussed ideas from your readings or classes with instructors outside of class Worked with instructors on activities other than coursework

The advantage of using multiple methods…. …you get to do a “Reality Check” on a periodic basis …you get to do a “Reality Check” on a periodic basis

What we didn’t have…. Perceptions of students after they transferred from Skagit  Collaboration with Western Washington University’s Office of Survey Research WELS (Western Educational Longitudinal Study) Data Focus Groups

WELS Transfer Study Process began in 2001; baseline data Fall 2003 Survey items related to students’ expectations, pre-WWU experiences, college preparedness, etc. Three separate surveys of incoming transfer students administered on-line, with phone follow-up Prior to starting classes After the first quarter After finishing three quarters

Results for SVC/Non-SVC Transfers Percent responding “5 or more times” Not SVC (n = 507 ) SVC (n = 45) Talked to a professor outside of class54.6%66.7% Personally contributed to a classroom discussion 76.3%80.0% Volunteered on or off campus outside of coursework 20.1%33.3% Participated in a club or activity16.8%24.4% Wrote a substantial scientific or other research paper of 5 pages or more 17.9%17.8% Wrote any other type of substantial paper of 5 pages or more 25.6%4.4%

Focus Groups WWU generated list; SVC did transcript analysis to identify student course taking, esp. Learning Communities and English Links Identified two groups for interviews: 1) a mix of none, one, or two or more LCs taken at SVC 2) two or more LCs taken at SVC

Questions for all SVC transfers What aspects of your learning experiences at Skagit Valley College have helped you in your courses at Western Washington? If you could change anything about your learning experiences at Skagit Valley College so that you would be better prepared to succeed at Western, what would that be? Can you talk a little bit about how Learning Community courses have helped or hindered your academic progress at Western? If you took an English course that was linked with another course, can you describe how that learning experience has been useful or not in your studies at Western?

Additional Group Questions For the first group: Of you who have taken more than one Learning Community or English Link, how did taking a second or third learning experience of this kind impact the development of your skills or knowledge? For the second group: How did taking the second or third learning experience of this kind impact the development of your skills or knowledge?

Helpful in Transition to WWU Students say the following aspects have helped: interdisciplinary learning, analytical/critical thinking, research and writing, group work, one on one interaction with instructors, and course content in specific courses Additional helpful aspects noted: completion of GURs, knowing people who have attended Western, and the financial benefit of attending SVC

Interdisciplinary Learning  Students say that the interdisciplinary aspect of learning communities helps to make classes more interesting and fun at SVC. Exposure to interdisciplinary learning at SVC helps students to feel more confident in their classes at Western. Students say they have learned to make connections and to apply what they’ve learned to another topic or to the real world.

English Links Most students say English links were useful in their studies at SVC and in transferring to Western—because English links challenged students to think and write analytically, outside of the box. Students say English was more interesting because it was linked to a content area which provided something to write about. They felt better prepared for researching and writing analytical papers at Western.

English Links A few students say their English link was not useful in their studies at SVC. They say the focus was more on grammar and punctuation, as it was in high school.

Research and Writing  While some students say that Skagit prepared them to think critically and write analytical papers (discussed earlier), other students say they were not prepared for writing analytical papers at Western. Research and writing experiences in English links and learning communities at SVC helped to prepare students for research and writing assignments at Western.

Other Student Outcomes Analytical/critical thinking The emphasis on critical thinking at SVC helps students in their classes at Western because they learned how to analyze information, make connections, and synthesize major ideas. Group work Group work in links and learning communities helped to prepare students for group work assignments at Western.

Challenge & Workload An interesting side note is students’ perceptions about the challenge and workload involved in learning communities and links: Some students say that learning communities and links are more challenging because of the level of analyzing involved. For some students, there is also a greater workload than expected. Other students say there is not a greater workload.

Faculty Coordination When instructors do not coordinate well together, the classes seem separate—not linked. Students find that the lack of coordination between some instructors means lack of integration between courses that are supposed to be linked. Learning communities work best for students when the instructors coordinate well together, and particularly when instructors are in the classroom teaching together.

Changes Students Would Make Students say the biggest issue for them is academic advising—because advising directly impacts students’ learning experiences at SVC as well as their preparation for Western. Students also say they would finish a DTA to avoid difficulties in transferring, make sure they had learning experiences in a large lecture class, and do something about disruptive students.

Future WELS Analyses Comparisons SVC degree vs. no degree (or ranked by number of LCs taken) SVC vs. other CCs LC comparison between CCs SVC vs. other CC vs. non-transfer WWU third-year students

Future WELS Analyses Surveys Baseline – administered prior to start of Fall qtr Focus = experiences at previous college & expectations at WWU Transitions – administered at the end of Fall qtr Focus = first quarter WWU experiences Spring Follow-Up – administered at the end of third quarter Focus = first year experiences at WWU Analyses Descriptives and regression

GPA Analysis – LCs Taken There were no significant differences between groups on either variable.

GPA Analysis - Degree There were no significant difference between groups on entering GPA but there was for GPA after two quarters (p=.000)

N = 60Entering GPA GPA After 2 Qtrs Sig. Degree Completed No Yes LCs Completed None One Two or more

Convergence of the Data Findings from surveys, focus groups, and student writing strongly suggest that taking Learning Communities results in higher levels of effort and engagement collaborate with peers interaction with faculty

Convergence of the Data The WWU data about transfer students also suggest that students who complete Learning Communities at Skagit or the degree perform better than those who do not However, we may need to look at the kinds and level of writing students are required to do in their courses

It’s all about perceptions…

Latest SVC Efforts…. Learning Communities Assessment Project (Washington Center for the Improvement of Undergraduate Education) Protocol developed at Harvard (Project Zero) Rubric being tested Developmental Learning Communities Project SVC project to research student success based on developmental course delivery methods, including counseling-enhanced

Thanks for coming!