1. Evan Siemann for Faculty Senate 11/16/16
Academic Analytics
Evan Siemann
for Faculty Senate
11/16/16

2. What is Academic Analytics?
The NRC provided doctoral program rankings
1982 & 1995
most recently in 2010 using data collected in 2006/2007
for instance publications were
criticized for time delay, ranges of rankings, and data quality
Academic Analytics is a company established in 2005 that provides academic data following the NRC guidance on metrics for different disciplines
has become a commonly used resource for benchmarking faculty scholarship

3. How have AA data been used at Rice?
People who have access if they choose:
Dept. chairs (to their department)
Deans (to their school)
Central units
Provost (incl. VPR, VPAA, GPS), Office Inst. Research
Program / department reviews
Helpful in benchmarking to other universities
Currently is used in most department reviews
Impact of hires (especially senior ones)
Not used for assessment of the performance of individual faculty
Because we have much better data about our own faculty when it is not critical to benchmark them to peers
Especially now that FARIS is implemented

4. Weight Schemes –primarily from NRC 2010 survey of universities (relative importance of different scholarly activities)

5. not part of Biological Sciences weight scheme
AWARDS: Includes 5,725 honorific awards from 845 governing societies with 4,469 awards from 736 societies represented in the comparative database. They are weighted differently depending on award.
Includes grants from 13 federal agencies and 2 non-federal funding sources (American Heart Association; American Cancer Society) [ ]
Articles with DOI from ~33,000 titles published
Citations from of articles published from [Crossref]
Every university has the opportunity to provide information on the list of faculty expected to produce research
in 2014 based on their type of appointment
GRANTS
American Cancer Society (ACS)
American Heart Association (AHA)
Department of Agriculture (USDA)
Department of Commerce (DOC)
Department of Defense (DOD)
Department of Education (ED)
Department of Energy (DOE)
Department of Health and Human Services (HHS)
Department of Transportation (DOT)
Environmental Protection Agency (EPA)
Institute of Museum and Library Services (IMLS)
National Aeronautics and Space Administration (NASA)
National Endowment for the Arts (NEA)
National Endowment for the Humanities (NEH)
National Science Foundation (NSF)
Conference Proceedings
Aerospace Engineering | Engineering
Applied Mathematics | Physical and Mathematical Sciences
Applied Physics | Physical and Mathematical Sciences
Astronomy and Astrophysics | Physical and Mathematical Sciences
Biomedical Engineering | Engineering
Civil Engineering | Engineering
Computational Sciences | Physical and Mathematical Sciences
Computer and Information Sciences, various | Physical and Mathematical Sciences
Computer Engineering | Engineering
Computer Science | Physical and
Mathematical Sciences
Electrical Engineering | Engineering
Engineering Mechanics | Engineering
Engineering, General | Engineering
Engineering, various | Engineering
Environmental Engineering | Engineering
Geological and Mining Engineering | Engineering
Industrial Engineering | Engineering
Information Science/Studies | Physical and Mathematical Sciences
Information Technology/Information Systems | Physical and Mathematical Sciences
Management Information Systems | Business
Materials Engineering | Engineering
Materials Science and Engineering | Engineering
Mechanical Engineering | Engineering
Nuclear Engineering | Engineering
Operations Research | Engineering
Physics, General | Physical and Mathematical Sciences
Systems Engineering | Engineering
BOOKS: published or republished [ ] – list from Baker & Taylor, The British Library, and AA assembly
CONFERENCE PROCEEDINGS: data collected directly from publishers for selected disciplines [ ]
not part of Biological Sciences weight scheme

6. Data characteristics This is not an attempt to reproduce my cv
limited time
e.g. only recent publications and only citations of recent publications
limited scope
e.g. no industry contracts
Only captures research activity
But data are assembled in exactly the same way for faculty at all US PhD granting institutions
objective, consistent data that allow relative ranking of scholarship
only one source of information for making strategic decisions at Rice (never the only one) but a helpful one for objective comparisons to peers

7. Program Productivity

8. Program Productivity (and by weighting these following NRC scheme, can also produce an overall numerical ranking [in this case 9/72], percentile ranking [89th], or index score [+0.5])
AAU

9. Quintiles