Utilizing Data to Foster Program-level Change Maureen Grasso Dean of the Graduate School The University of Georgia

Data is only as useful as the action it produces. Today I will address how we utilized the CGS data to foster program-level changes. I will explain the valuable lessons we learned through CGS grants. 1

CGS provided the impetus for our work. Phase I (2004-2007) Learning best methods for utilizing data Common theme of data-driven intervention to improve doctoral completion at the three universities (UGA, UF, & NCSU) Phase II (2007-2010) Graduate School’s University-wide Initiative Phase I- majority of presentation, Consisted of many interlocking activities Phase II- Based on what we learned in Phase I 1

Everything we did was derived from our theoretical framework. Four Conditions for Optimal Doctoral Completion Condition #1: The right people apply for doctoral study. Condition #2: The right applicants are admitted as doctoral students. Condition #3: Students and faculty form productive working relationships. Condition #4: Students experience social sup port from fellow students. theoretical framework based on literature in the area 1

Data promotes change among faculty better than administrative mandates. Faculty members: Resist administrative initiatives for program- level change. Want their programs to be as good as or better than comparable programs. Are willing to consider the need for program improvement. 1

Data is an important driver in the process for fostering change. A Five Step Process was used

Step 1: Appropriate data needs to be collected to promote more awareness. The CGS templates provided the basis for all data-based changes. Faculty members were not as receptive to this format. 1st bullet point- CGS templates are great for those who have the time and motivation to analyze the data thoroughly; However, our faculty did not do this given their other demands. On another note, we found it was important to keep a codebook for each nuance that arose with the data. 2nd bullet point- We wanted to provide data that highlighted the doctoral completion rates of the programs that alerted the faculty of their status. Prior perceptions of faculty awareness of their rates varied. 1

Step 2: Distilling data is important for meaningful comparisons. Program Practices Two key measures: Percent Completion Time to Completion We created comparative charts that allow faculty to determine where they stand within their field, university, and nationally (next slide show benchmarks) 1

Step 3: It is important for faculty to benchmark their data with similar programs. Phase I- We communicated these benchmarks on our website. This data was available to participants of the three- university collaboration via a password- protected mode.

UGA Programs (n) % Complete Rank A (34) 72.7 1 B (115) 67.6 2 C (12) 66.7 3 D (58) 65.3 4 E (27) 62.5 5 F (70) 59.4 6 G (27) 55.6 7 H (18) 47.1 8 I (16) 43.8 9 J (18) 37.5 10 K (36) 33.3 11

UGA Programs (n) TTD Rank A (16) 3.33 1 B (70) 4.33 2 C (12) 4.67 3 D (115) E (18) F (18) 5.33 6 G (34) H (27) I (58) 9 J (36) K (27)

Step 4: Review data with program leaders and ask them to determine their future direction. Is this where you want to be with respect to percent completion and time to degree? How so? What are you currently doing to improve doctoral completion? What are your future plans to improve doctoral completion? It is important for the Dean to meet with leaders in each program. It is the Dean’s job to put the data in front of faculty’s noses so they can collaborate to determine WHAT needs to be fixed and HOW to do it. I thanked them for their collaboration and explained the importance of best practices and standards in doctoral education. A uniform interview was then conducted by me with the following questions: •1st question- helpful for faculty buy-in •2nd question- program practices, which are on website •3rd question- each program had written goals, including who was ultimately responsible for these goals 1

Step 5: Program leaders are encouraged to develop their own plans for program-level change. Program leaders were asked to submit in writing their plans for improvement. One year later, programs submitted a written update regarding their progress in the areas.

Phase II- We will post statistics for all doctoral programs in a public domain in collaboration with Institutional Research.

The Graduate School is implementing a university-wide initiative based on findings from Phase I. Public Awareness Campaign University-wide Meetings in January 2008 Developing benchmarks for acceptable levels of doctoral completion Follow-up system implemented each semester Increase accountability by having each program explain each noncompleter Institute university-wide follow-up surveys for all completers and noncompleters •1st bullet Public Awareness Campaign involving articles in local newspapers and magazines, our improved website that will be marketed toward a larger population, feature on CGS’s webpage •2nd bullet University-wide Meetings in January 2008 involving introduction of university-wide initiative •3rd bullet Developing benchmarks for acceptable levels of doctoral completion all doctoral programs at UGA •4th bullet Follow-up system implemented each semester we want to hear about each graduate student’s doctoral study experience and use that information to improve •Increase accountability by having each program explain each noncompleter completed by faculty •Institute university-wide follow-up surveys for all completers and noncompleters completed by students 1

Project Website http://www.gradsch.uga.edu/cgs