1. Institute for Research on Innovation & Science (IRIS)
Jason Owen-Smith
IRIS/University of Michigan
Iris.isr.umich.edu
@IRIS_UMETRICS

2. Roadmap Background on IRIS What we currently do
USE Cases for MPC/FHE/etc.

3. The Challenge(s)

4. In 2016, our society invested $220 on academic research for every man, woman, and child in the country
For every $1: $0.55 from federal government, $0.24 from universities, $0.06 each from states, industry, non-profits, $0.03 from all other sources
We make those investments to develop human knowledge and to improve quality of life and well being.
How do we understand and improve those effects?

7. Competing Budgetary Priorities

8. A Thought Experiment

9. The Wisconsin Idea Proposed revision
The mission of the [University of Wisconsin] system is to develop human resources to discover and disseminate knowledge, to extend knowledge and its application beyond the boundaries of its campuses and to serve and stimulate society by developing in students heightened intellectual, cultural, and human sensitivities, scientific, professional, and technological expertise, and a sense of purpose. Inherent in this broad mission are methods of instruction, research, extended training and public service designed to educate people and improve the human condition. Basic to every purpose of the system is the search for truth.
The mission of the [University of Wisconsin] system is to develop human resources to meet the state’s workforce needs, to discover and disseminate knowledge, and to develop in students heightened intellectual, cultural, and human sensitivities, scientific, professional, and technological expertise, and a sense of purpose.

10. Our Response: IRIS Data for research and reporting to understand, explain, and improve the public value of academic research

11. (More) Rational Policy
Framework
Discovery
Learning
Dissemination
Innovation
Entrepreneurship
Economic Growth
Public Health
Food Safety
Security
(More) Rational Policy …
Propose
Knowledge, People, Skills
Fund
Science
Investments
Universities
Grants enable work which creates and sustains collaboration networks that encode the social capacity for discovery. The ‘conservative’ stable features of universities and their insittutional commitments make them anchors that increase the resilience and responsiveness of markets and regions, The Movement of people and ideas back and forth between campus and the rest of society makes them hubs and enables the application of knowledge.
Hiring, Spending
Jobs
Stimulus

12. Background Founded in 2015 Recession  STARMETRICS  UMETRICS  IRIS
Emerged from CIC/Big Ten
Transaction level sponsored projects expenditures on employees, vendors and sub-awards
33 current member institutions (11 Big 10) = ~30% of federal R&D spend
Members contribute to support infrastructure & receive reports and other data products
Goal is 150 members (~93% of federal R&D spend)
IRB approved data repository – Virtual Data Enclave
~60 current users w/ approved projects, signed DUAs
Disclosure proofing procedures
But basically a trust model

13. Institute for Research on Innovation and Science (IRIS)

14. Research and reporting to understand, explain and improve the public value of academic research
Key goal: long term, near comprehensive, longitudinal data about academic researchers
Key problems: no single data source, most extant data is about documents (grants, publications, patents) not people, no (public) persistent identifiers

15. Research and reporting to understand, explain and improve the public value of academic research
Key goal: long term, near comprehensive, longitudinal data about academic researchers
Key problems: no single data source, most extant data is about documents (grants, publications, patents) not people, no (public) persistent identifiers 1
University transaction data – Restricted

16. Research and reporting to understand, explain and improve the public value of academic research
Key goal: long term, near comprehensive, longitudinal data about academic researchers
Key problems: no single data source, most extant data is about documents (grants, publications, patents) not people, no (public) persistent identifiers 1 2
University transaction data – Restricted
US Census outcome data – Restricted

17. Research and reporting to understand, explain and improve the public value of academic research
Key goal: long term, near comprehensive, longitudinal data about academic researchers
Key problems: no single data source, most extant data is about documents (grants, publications, patents) not people, no (public) persistent identifiers 3 1 2
University transaction data – Restricted
US Census outcome data – Restricted
Federal grant data – Public

18. Research and reporting to understand, explain and improve the public value of academic research
Key goal: long term, near comprehensive, longitudinal data about academic researchers
Key problems: no single data source, most extant data is about documents (grants, publications, patents) not people, no (public) persistent identifiers 3 1 2
University transaction data – Restricted
US Census outcome data – Restricted
Federal grant data – Public
US Patent Office data – Public 4

19. Research and reporting to understand, explain and improve the public value of academic research
Key goal: long term, near comprehensive, longitudinal data about academic researchers
Key problems: no single data source, most extant data is about documents (grants, publications, patents) not people, no (public) persistent identifiers 3 1 2 5
University transaction data – Restricted
US Census outcome data – Restricted
Federal grant data – Public
US Patent Office data – Public
Publication data – Public & Restricted 4

20. Research and reporting to understand, explain and improve the public value of academic research
Key goal: long term, near comprehensive, longitudinal data about academic researchers
Key problems: no single data source, most extant data is about documents (grants, publications, patents) not people, no (public) persistent identifiers 3 1 2 5
University transaction data – Restricted
US Census outcome data – Restricted
Federal grant data – Public
US Patent Office data – Public
Publication data – Public & Restricted
Dissertation data – Public & Restricted 4 6

21. Submission Process Common data structure
Upload through a secure portal
Coded quality assurance checks
Immediate: e.g. value ranges, duplication, missing fields, record counts etc.
24 hours: e.g. normalization
Data depositors generally don’t know what’s really in the data they submit

22. Reporting

23. Making Data Available: IRIS

24. Process Challenges
Universities vary dramatically in quality of data produced
Normalization, actually unique identifiers, garbage strings, duplicates, missing data, negative values, wonky date ranges . . .
Labor intensive community and relationship building
Substantial work required to integrate multi-university data
No persistent individual or organizational identifiers, wildly different naming conventions
Disambiguation and unstructured linkage across univs and b/t integrated data and public sources (e.g. pubmed, ISI, patents, proquest)
Computationally intensive feature-based SVM approach in development
Formatting issues – e.g. Census works only in SAS and requires access to identified micro-data
Social science researchers are trained (and expect to) look closely at micro- data, construct unique variables, integrate new data sources
Data access is time consuming, may be a barrier, limitations in computing capacity, many “help desk” requests, disclosure review

25. Use Cases for FHE
Social science research, standard statistical methods
Key challenge, data are supposed to be flexible and usable for research we cannot envision
Universities who want to benchmark but don’t want to be identified
Universities love to compare themselves to others but hate to be compared
Agencies, policy-makers, the public (?) who want to explore aggregate data
Issues of trust/oversight, generally need to see information across a portfolio of institutions
????
“Living” data. Multiple updates per year, two transfers to Census, annual documented data release through a virtual and a physical enclave.

26. Thank You