1. New Research Evaluation Model: Indentifying Obstacles and Remedies Jerald Hage, Director Center for Innovation, University of Maryland College Park, Maryland

3. New Ways of Thinking about Research Evaluations Five Ideas: 1.Emphasize the evolution of science and technology or knowledge generally; 2. Connect the outputs to larger societal objectives, ideally comparatively; 3. Recognize multiple levels of analysis -—at least three if not five; 4. Focus on obstacles and potential remedies at each level; 5. Remedies come from social science theory.

4. Larger Objective: Decline in US Economic Growth Five Signs of Decline: 1.Negative trade balances in most high tech sectors, niches; 2.Lower rate of radical innovations of many high tech companies; 3.Decline in patents of some high tech companies; 4.Weakening in wage levels of educated workers; 5.Loss of employment in many high tech sectors.

5. Comparative Data: Research NationRDT% Patents/ GDPMillion People Israel 4.15% 49 Sweden 3.75 100 Switzer. 3.00 115 South Korea 3.21 40 Japan 3.44 107 Germany 2.53 72 U.S.A. 2.77 46 Economist 1 October, 2010: 97

6. Evolution of Scientific and Technological Research: U.S. Growth in Knowledge leads to: differentiation of research organizations in arenas evolution of new networks to connect arenas Public research laboratories Pharmaceutical companies Universities Bio-tech companies Manufacturing research Quality research Commercialization research Basic research Applied research Development research

7. The Role of Theory in Policy Evaluation of Research Advantages:  The evolutionary theory of knowledge growth allows for predictions to be made;  Patterns of failed evolution are specified and different in U.S. and Europe  The causes lie at multiple levels;  Search for obstacles and remedies in social science theory and beyond in other countries

8. Obstacles Preventing Economic Growth: Research Obstacles:Managerial and Policy Remedies: o Static strategiesSeize evolutionary opportunities o Low risk researchBroaden research team vision o Lack of learningStimulate cross-fertilization of ideas o Stove pipesIntegrate the organization o Reactive leadersAppoint transformational teams o Valley of deathConstruct network coordination o No industrial policyPublic/private cooperation o Non-visible BlockagesPerform timely feedback on innovation at team level and blockages Behind each obstacle are multiple blockages and the remedies have alternatives, each with problems

9. Obstacle: Static Strategies Evolutionary changes: Growth in the number of radical innovations; Globalization in the development of radical innovations; Differentiation in consumer tastes; Changes in the way in which radical innovations are produced; Expansion in the number of non-economic concerns; Emergence of a new stage of manufacturing.

10. Remedy: Seize Evolutionary Opportunities Appropriate Response:  customize products/services;  recognize the evolution in the patterns of doing research;  solve national concerns and global problems;  increase flexibility in manufacturing by being able to change the composition of parts.

11. Obstacle: Low Risk Research Evolutionary changes in research problems The cost and risk (meaning choice of wrong solution) have increased; The variety of arenas that have to develop radical solutions for radical product innovation are now greater; More need to eliminate multiple kinds of risks to the environment and to the individual; Global solutions have to be customized for specific regional and national differences.

12. Remedy: Broaden the Vision Mechanisms for Increased Diversity;  Increase the emphasis on radicaliness of innovation in each of the arenas;  Add new kinds of specialists to team;  Change problem so that it considers processes and multiple levels;  Explore more arenas of research;  Create redundancy between differentiated research arenas to facilitate the transfer of tacit knowledge

14. Remedy: Cross-fertilization of Ideas Mechanisms for More Communication:  information that reduces the distance somewhat between individuals;  emotional bonding that slows the rate of decline in effective communication as diversity increase;  continuously learning that raises the equilibrium curve between effective communication and radical innovation

15. Obstacle: Stove Pipes Evidence for Study of universities IBM history GM history SEMATECH history

16. Remedy: Integrate the Organization Mechanisms of integration:  Complex organizational strategy that is visionary  Transformational leadership team  Multiple sources for funding

17. Obstacle: Reactive Leaders Evidence for: Failure to adopt flexible manufacturing Decline in innovativeness in large firms Loss of manufacturing jobs Loss of technologies developed in the US

18. Remedy: Transformational Leadership Teams Characteristics of:  Overcome obstacles;  Expand the frontiers by innovations in occupational specialties and research technologies;  Change the “know-how” of relevant researchers and professionals;  Work on a global problem.

19. Obstacle: Valley of Death Evidence for: Study of Brookhaven Trade deficit in health care products Lack of public-private cooperation in manufacturing

20. Remedy: Construct Networks Modal Types:  Japan’s supply chains  SEMATECH as a research consortium  Silicon Valley as a knowledge pool

21. Obstacle: Blockages in Organizations Kinds of Blockages: 1. Absence of team processes associated with innovation; 2. Lack of certain kinds of communication; 3. Inappropriate managerial style; 4. Inadequate reward structure for researchers; 5. Not enough research resources; 6. Absence of long term strategy. Jordan’s Environmental Research Survey

22. Remedy: Feedback Levels of Feedback 1.Connecting measures of technical progress to organizational blockages; 2.Situating the organizational research outputs relative to the sector nationally and internationally; 3.Counting the differentiation of research organizations; 4.Measuring the degree of integration between them.

23. Implications for Data Collection by NSF, STPI, IDA, USDA, etc. 1.Funding and capabilities in each of the six arenas in each sector/niche; 2.More measures of radical innovation performance in each arena; 3.Specialized studies of evolution and failed evolution in a specific sector/niche; 4.Specialized studies of the causes of failure to have radical innovation in each arena of a sector. Measurement is necessary to pinpoint both failure and its causes

24. Summary The Proposed Evaluation Model: 1.Uses a model of knowledge growth; 2.Connects specific research projects to larger issues; 3.Operates at multiple levels; 4.Identifies obstacles and suggests remedies; 5.Enriches theory and vice-versa