1. 1 Institutional Research: Focus on Outcomes and Student Success IR&P Board Presentation Robert B. Barr, Ph.D. Executive Director October 6, 2003

2. 2 Introduction Bob Barr

3. 3 Handouts Recent History of IR&P Institutional Student Outcomes Packet of Recent Studies

4. 4 Mission Support a culture of evidence Identify and disseminate best practices and program factors Help re-invent and re-design our colleges to improve learning

5. 5 Plans Conduct 2 nd & 3 rd generation of studies on student outcomes Increase sophistication of program review with more outcomes data Increase amount and detail of outcomes data on web site

6. 6 Plans Track cohorts by segment and other attributes; establish more baselines Create an interactive web site to enable others to research Involve more faculty and staff in researching what works

7. 7 Student Segmentation, Success and English Readiness, & Success in Sequenced Basic Skills Courses Rob Johnstone, Ph.D.

8. 8 What are the course taking patterns of our students? How successful are they in them?

9. 9 Segmentation Summary Fall 2002 Unduplicated Headcount Note: Foothill Unduplicated Headcount = 18,896; De Anza Unduplicated Headcount = 25,261. Note that students can be in only one segment.

10. 10 Do students who complete their Basic Skills courses have a higher rate of success in general education courses than those who don’t?

11. 11 Data Excerpted from Executive Summary P13 English Course Completion and BSS Success

12. 12 Does prior course grade relate to future course success in Basic Skills sequence courses?

13. 13 Prior Course Grade and Success: Math Data Excerpted from Executive Summary P85

14. 14 Prior Course Grade and Success: Eng 110 Data Excerpted from Executive Summary P85

15. 15 Impact of Research Karen Yoshihara, Dean, Language Arts How effective are our placement and advising procedures? –How many take the tests? –How many place into basic skills? –How many enroll after placement? –What are the success rates in basic skills classes? How well does our ESL program articulate with the English program? –Success relationships - Eng 1A vs. ESL 26 in Eng 1B –Success relationships - ESL 26 vs. Eng 110 in Eng 1A What is the impact of our student support services such as the Writing Center and Tutoring Lab?

16. 16 Impact of Research Tess Hansen, Faculty, English Department Research has helped justify the expense of the Puente program, as the statistics demonstrate the program’s effectiveness and thus our confidence in maintaining it. The studies have also illustrated the wide impact learning communities such as Puente can have beyond the classes within the program – success not only in English, but in higher transfer readiness. As a faculty member, the segmentation and basic skills research has helped me identify cohorts of students who are consistently not succeeding. This helps compel the English department to make programmatic and curricular changes to empower these students to succeed.

17. 17 An Example of What Research Can Tell Us: Selected Data From the Math Performance Success (MPS) Program at De Anza College Andrew LaManque Anne Leskinen Diane Mathios

18. 18 Overview of the Program Cohort - three quarter math sequence –Elementary Algebra (Math 101) –Intermediate Algebra (Math 105) –Elementary Stats / Probability (Math 10) –Students have shown previous math difficulties, including repeated attempts and failure Double the classroom time Instruction / Counseling collaboration Peer tutoring

19. 19 What Did We Want Research To Tell Us? 1.How do the grades of students in the MPS program compare with the grades for students not in the program for the same course and term? 2.Do MPS students starting in Math 101 persist through to higher level math courses at a higher rate than students not in the program? Given that the same content and material must be mastered:

20. 20 Summary of Research Findings

21. 21 Summary of Research Findings

22. 22 Summary of Research Findings

23. 23 So What? How Did Research Contribute? This research demonstrates that the program is achieving results in terms of student success Provides justification for program continuance or expansion The research provides a baseline of data to monitor changes over time Contributes to the ‘Body of Evidence’ examining student success

24. 24 Best Practices Validated by the Research Cohort development and course sequencing –Could benefit students in Math 51, 49A, 49B sequence Additional student time on task –Pilot the addition of 1-2 hr lab component –Use research to find optimum arrangement Instructional / student support relationship –Develop learning skills in first course –Holistic approach to building confidence

25. 25 Conclusion Bob Barr

26. 26 Challenges Assess learning more directly Support and engage faculty and staff in learning outcomes assessment Identify best practices and factors

27. 27 Challenges Lay groundwork for accreditation and self-study Develop more baseline data and assist in defining goals Make effective use of the data warehouse for decision-making

28. 28 Challenges Develop cost analyses on what works While good research takes time, provide findings that make a difference soon as possible Identify and encourage implementation of what works for student learning

29. 29 Best Practices/Principles Validated by Our Research Special programs work –Puente, MPS, LinC, Pass the Torch Community and personal relationships –Puente, Pass the Torch, MPS, LinC Additional student time on task –MPS, Puente Programs of small size –Puente, MPS, LinC, PTT, Nursing, Dental Hygiene

30. 30 Best Practices/Principles Validated by Our Research Staff belief in students’ ability to succeed –Puente, Pass the Torch, MPS, LinC Creating cohorts of students together in a program –MPS, Puente, Pass the Torch, LinC Combining basic skills with other subjects –LinC (e.g., math and business, ESL and history)

31. 31 Best Practices/Principles Validated by Our Research Students succeeding first in basic skills succeed at higher rates in other courses –E.g., Psych, Soc, Econ, Hist A “C” in basic skills may not be enough –E.g., a C in math 101 enables only 29% success in Math 105 vs. 85% if an A in 101 New students taking orientation to college persist at considerably higher rates than those who don’t

32. 32 Best Practices/Principles Validated by Our Research Learning styles and readiness matter –E.g., higher withdrawal rate from distance learning courses The right “structure” alone may produce the desired success factors and behaviors –E.g., structure of Puente, MPS, Pass the Torch, Dental Hygiene, Athletics