1. Theme 3: Science of Science Policy to Address National Needs Summary December 4, 2008

2. High Level Questions Total number of people who answered this section : 74 Percent who rated high importance Percent who Ranked #1 Question 8: What impact does science have on innovation and competitiveness?74.32%51.35% Question 9: How competitive is the US scientific workforce?43.24%21.62% Question 10: What is the relative importance of policy instruments in science policy?39.19%25.68%

3. Takeaway 1: Broaden the Scope Theme implies that science serves only innovation- and competitiveness-oriented goals. However, science also serves broader societal goals (e.g., science for its own sake, environment, improvements in health, security, quality of life). The Roadmap should also reflect these goals Roadmap is too US-centric, there is much happening internationally, we should be better networked and learn from others (e.g., data collection efforts in Europe and Asia)

4. Items on Which to Vote 1.The roadmap should clarify that economic competitiveness is not the sole goal of science 2.Science of science policy should have a stronger international component in its development phase

5. High Level Observation on Findings 80% agreed with statement: The ITG finds that there is a real opportunity to develop new tools and data sets that could be used to quantify the impact of the scientific enterprise thus far on innovation and competitiveness (43% rated this number 1) Equal split among other findings

6. High level observation on Recommendations

7. Top Models/Tools Keeping in mind that we asked you to only comment on the tools you are familiar with, here are your ratings: Peer/Expert Review #1 Econometric; Cost/benefit Data Collection: Survey Visualization tools rated very low

8. Top Metrics Bibliometrics (citations; patents; scientific papers) highest rated Nothing else highly rated

9. Takeaway 2: Metrics/Methods/Tools are Meaningful Depending on Context Tools, methods and metrics need to be tied to levels (program, agency) and questions (what is the problem to be addressed); it is difficult to address them in abstract terms Relative weight of each depends on the question

10. Item on Which to Vote 3. Usefulness of tools/metrics/measures should only be considered in the context of the specific question being posed 4. New data collection cannot be done willy- nilly: integrate into existing efforts, including better linkages and data mining

11. Takeaway 3: Need Better Articulation of Concepts National Needs need better articulation –Tough to talk about using SOSP to address national needs without explaining what they are Some terms must be cleaned up or better defined –Say science and technology not just science –Don’t conflate science with innovation –Term competitiveness meant different things to different people (science, economy)