1. 1 Indicators of Knowledge Value Conference on Estimating the Benefits of Government Sponsored Energy R&D Department of Energy At Hilton Crystal City - March 4-5, 2002 Diana Hicks CHI Research, Inc. 10 White Horse Pike Haddon Heights, NJ 08035 Tel: 856 546 0600 Fax: 856 546 9633 E-mail: dhicks@chiresearch.com

2. 2 Outline  Who is CHI?  Knowledge value and bibliometrics  Examples: paper-patent citations  Examples: paper-paper citations  Patent citations to papers  Examples: patent-paper citations  People and knowledge value

3. 3 Who is CHI Research?  We are a 30 year old consulting firm specializing in development and analysis of science indicators and patent-based technology metrics.  Francis Narin, founder, developed techniques for analyzing national scientific performance in the 1970’s; then developed techniques to assess corporate technological intellectual property using patent- based metrics in the 1980’s; then extended techniques to relate technology portfolios to stock market valuation in the 1990’s.  We have a staff of 21 (10 analysts and 11 support people) with diverse technical backgrounds.  Among the senior staff we have 6 PhD’s and 9 Masters degrees in areas such as Physics, Chemistry, Mathematics, Computer Science, Science Policy, Bibliometrics, and Public Policy.

4. 4 CHI’s Databases  US Patent Database –Since 1975 –3 million US Patents, 1.1 million EPO patents –24 million citations  Analytically ready –Filters CHI has developed the expertise needed to cleanly pull sets of patents on any technology –Unification Company names unified to link parent/subsidiary/joint ventures –Restated for mergers, divestitures, and reassignments –Indicators More than 20 technology indicators –2.4 million references to non-patent literature classified 1.4 million of these are references to journal articles and have been put in standard form

5. 5 Technology Patents citing Patents Knowledge Value and Bibliometric Techniques Science Papers citing Papers Outcomes Patents & Stock Market citing Papers Performance

6. 6 Extract from the Front Page of a U.S. Patent

7. 7 DOE government interest patents CHI indicators Patents that indicate government interest and contain the word “energy” in the government interest field. Patents identified by David Eike Indicators defined at: http://www.chiresearch.com/about/data/tech/indicator.php3

8. 8 DOE government interest patents, leading technology areas

9. 9 IBM’s patents are valuable

10. 10 Does an agency resource support those creating high value technology? Incidence of Subscriber and Non- Subscriber Top-Decile CFC Replacement Patents

11. 11 Conceptual Diagram 9 U.S. Patents 5 Foreign Patents IBM Patent No. 5, 278,955 Issued 1994 IBM Patent No. 5, 278,955 Issued 1994 6 U.S. Patents 6 Other References, Including 3 Science References Backward Citations (References) Forward Citations Time 1985-921994 1995-98 A Starting Patent references prior art, and is cited by later patents

12. 12 Science Linkage is Increasing in Many Countries

13. 13 Excluding Biotechnology - Science Linkage is Increasing in Many Countries

14. 14 Public Sector Science is Valuable for U.S.-Invented Technology % of references on U.S. patents to U.S. scientific literature Source: NSF Science & Engineering Indicators and CHI Research, Inc.

15. 15 Technological and scientific value are often aligned Share of 1993-95 US papers cited in US invented, USPTO patents issued in 1997

16. 16 U.S. Companies Preferentially Cite In-state Public Sector Science in Their Patents Expected # of citations = "patent state's" # of citations to all states multiplied by "paper state's" share of cites received from all states Citations from Industry Patents to Public Sector Papers

17. 17 Does an agency’s research support technological innovation? Includes only papers with explicit support acknowledgements Funding agencies acknowledged on biomedical papers cited by patents 1993-94 patents citing US authored biomedical papers published 1981-1991

18. 18 DOE science supports US innovators US patents 1985-2000 citing DOE papers. Inventor addresses used to identify nationalities. A 10 year, 2 year lagged citation window is used. Citations fractionally counted

19. 19 DOE science has value for a range of technologies Number of citations from patents to technology area. Patents 1985-2000 citing papers in 10 year, 2 year lagged window Technology-science combinations with >200 citations are shown

20. 20 L. Bito and the Technology of Prostaglandins in the Treatment of Glaucoma

21. 21 Quadrant Model of Research Value Research has value for: Technology (patents) NoYes Nuclear & Particle Physics Can be found in: Science, Nature, Cell, PNAS NoUltrasonic Imaging AT&T Technology Review Non-science linked patents Science (Scientific Literature) After Stokes

22. 22 Knowledge value is very unevenly distributed People are key, especially for prospective studies

23. 23 Summary  Bibliometrics, properly constructed, are useful in assessing knowledge value.  The value of research for future scientific research and technological innovation can be examined.  People are crucial.