1. National Research Council Assessment of Doctorate Programs: The Methodology Report E. L. Fink & S. Chai University of Maryland

2. Assessment Demographics Data Collection: Fall 06 – Spring 08 5000+ programs in 61 program fields 7 broad fields distinguished (e.g., physical sciences, humanities… – Communication included as a social science Communication: 83 programs at 65 institutions Criteria for inclusion of a program: Produced min. 5 Ph.D.s in 5 years prior to 2005-2006

3. Data Collection Questionnaires: – Institutional – Program – Faculty – Doctoral Student: collected in five fields; not in communication – Rating questionnaires sent to volunteer sample of faculty in each field Communication: 50 programs rated by 28-57 raters each (average 40.7) Missing data: – if 1-2 missing items: mean of other Communication programs used for that item – if 3 or more missing items: program dropped

4. Sub-Field Taxonomy for Communication Broadcast/Video Studies Communication Technology and New Media Critical and Cultural Studies Gender, Race, Sexuality, and Ethnicity in Communication Health Communication International and Intercultural Communication Interpersonal/Small Group Communication Journalism Studies Mass Communication Organizational Communication Public Relations/Advertising Social Influence and Political Communication Speech and Rhetorical Studies

5. Data Set: Three Main Dimensions Research activity of program faculty Doctoral student support and outcomes Diversity of the academic environment

6. Research Activity of Program Faculty Publications: Source for Social Sciences: ISI; faculty CVs used to check completeness of ISI data Citations: ISI citations 2000-2006 for publications 1981-2006 Grants: percent of faculty holding grants Honors & Awards: lists collected from scholarly societies – “prestigious”: 5 points, others: 1 point

7. Doctoral Student Support and Outcomes Percent of students fully funded in first year Percent of students completing degree in a given time period Time to degree Placement in academic positions Program collects data about employment outcomes for its students

8. Diversity of the Academic Environment Percent of faculty and students from underrepresented minority groups Percent of faculty and students who are female Percent of students who are international

9. Calculation of Dimensional Rankings of Programs Used all of a dimension’s measured characteristics Each weighted by importance of characteristic and its category derived from faculty survey (“direct” weights) Calculation of ranges of rankings for each program on each dimension

10. Weights for Dimensional Measures: Research Activity Average Weights Average publication per faculty Average citations/ publications % faculty with grants Awards per faculty Social Sciences.36.26.22.16

11. Weights for Dimensional Measures: Student Support and Outcomes Average Weights % w/ full support Average cohort completing in 6 years Time to degree full and part time Placement of students Program collects outcome data Social Sciences.27.24.12.18

12. Weights for Dimensional Measures: Diversity of the Academic Environment Average Weights % core or new faculty under- represented minority % core or new faculty female % students under- represented minority % students female % students international Social Sciences.22.20.26.17.16

13. Overall Rating of Program Quality “Direct” Weights: as for three dimensions “Regression-based” Weights: – Program characteristics used to predict ratings of program quality provided by faculty – Infer weights for characteristics from regression Combined direct and regression-based weight used to calculate range of rankings for programs Reported: Inter-quartile range (i.e., 25 th to 75 th percentile of rankings sampled for each program)

14. Example ResearchActivityStudent and Support Outcome Diversity Academic of the Environment OverallMeasure Program3 rd Quartile 1 st Quartile 3 rd Quartile 1 st Quartile 3 rd Quartile 1 st Quartile 3 rd Quartile 1 st Quartile Program X 1519182661761014 Program Y 103108918119198 Program Z 9410053722442104109