National Institute of Standards and Technology U.S. Department of Commerce Technology Program Evaluation: Methodologies from the Advanced Technology Program Gems: Success stories on a page Prasad Gupte Technology Innovation Program National Institute of Standards and Technology Collaborative Expedition Workshop National Science Foundation March 18, 2008

National Institute of Standards and Technology U.S. Department of Commerce What is a “gem”?  A one-page brief that describes the realized economic impact of funded ATP project.  A hybrid document  Elements of economics, technology (lay-level), and public relations  Qualitative and quantitative  High-level yet rigorous  Measures the project to ATP’s mission!

National Institute of Standards and Technology U.S. Department of Commerce Example of a gem

National Institute of Standards and Technology U.S. Department of Commerce How gems are used  To provide ATP with a means of communicating the impact of funded projects to the public  For ATP headquarters To answer congressional requests  For internal staff To use as examples at conferences, trade shows Feedback to project selection and management –What are R&D best practices? –Trend analysis

National Institute of Standards and Technology U.S. Department of Commerce Elements of a “gem”  It answers these three key questions:  What was the technical risk?  What did the project accomplish?  What were the broad economic benefits? Note the linkage to funding criteria!

National Institute of Standards and Technology U.S. Department of Commerce What was the technical risk?  What was the problem?  Why was it hard?  Why did it need government support?

National Institute of Standards and Technology U.S. Department of Commerce Example of Technical Risk  Throughout the early 1990’s GE’s central R&D lab, GE Global Research, began the basic research for the use of amorphous silicon detectors in digital medical imaging with funding from a number of government agencies including DARPA, U.S. Army, U.S. Navy, and Department of Health and Human Services. If the cost of manufacturing amorphous silicon could be reduced enough, it would enable detectors with a wide range of applications, not just from medical imaging but also in industrial uses such as non-destructive testing. While GE scientists conceived of ways to bring the cost of manufacturing down, according to GE Vice President Lewis Edelheit: “Getting internal support for the low cost manufacturing proposal was hard. At the time it was difficult for some executives to see the advantages of taking digital imaging beyond cardiac applications. Also the combining of process steps in the fabrication of amorphous silicon integrated circuits was considered to be a high risk proposition.”  Thus, in 1995, GE, with its manufacturing partner Perkin Elmer, applied for and received an ATP award due to the high level of risk, innovation, and potential for national benefits that the project proposed. According to Edelheit:  "Given the technical and other risks, had the ATP turned down the proposal, the promising, low-cost manufacturing process initiative would have been shelved."

National Institute of Standards and Technology U.S. Department of Commerce What did the project accomplish?  Technical achievements  Knowledge creation  Bibliometric output Patents, publications, presentations  Capital attraction or further investment/continuation of project  Linking back to ATP’s mission

National Institute of Standards and Technology U.S. Department of Commerce Example of Project Accomplishments  In the ATP-funded project, GE successfully completed most of the ambitious technical goals. Major accomplishments include:  Reducing the total number of process steps from 300 to 200  Reducing the total number of mask steps, the critical manufacturing challenge, from 11 steps to 7  Reducing the total cost of the process by 25%  Filing for and receiving three patents  However, when the ATP project was finished in 2000, there was still a significant amount of research to do before the technology could be ready for commercial application. Two years after project completion, the partners continued investing millions of dollars into the technology with no additional government funds.  In addition, prior to the ATP project, the low cost manufacturing technology was held by GE’s corporate lab. Through the ATP project, GE was able to transfer the technology to Perkin Elmer. Instead of being the manufacturer, GE made the strategic decision to be an end user of the technology, and allowed Perkin Elmer to focus on the manufacturing, where it could then sell the detectors to a wide variety of other users.

National Institute of Standards and Technology U.S. Department of Commerce What were the broad economic benefits?  Describe private benefits  Increased sales, decreased costs, better product/process performance  Describe public benefits (spillovers)  How other entities (firms, industries, customers, consumers, public) benefited  Quantify as much as possible

National Institute of Standards and Technology U.S. Department of Commerce Example of broad economic benefits In January of 2004, nearly ten years after the companies began working together on the ATP project, Perkin Elmer signed a $250 million deal to supply GE with amorphous silicon flat panel detectors, where:  GE Medical Systems will use these in a variety of medical cost-sensitive medical applications  GE Aircraft Engines will use the detectors for non-destructive testing The medical applications are focused on digital mammography and digital radiography. The major benefits of these uses include:  Superior breast cancer detection through lower false positives and fewer unnecessary biopsies  Reduced radiation exposure  Reduced medical costs through enhanced clinical productivity The potential economic impact of the digital mammography applications alone is large—a rigorous cost-benefit study 1 estimated economic impacts between $219 million and $339 million, and a benefit-to-cost range of 125:1 to 193:1.1 The enabling technology that ATP has funded has created even broader benefits, as Perkin Elmer is pursuing sales of the amorphous silicon detectors to users who will apply this toward:  Industrial non-destructive testing (NDT)  PCB inspection  Pipeline inspection  Bone densitometry and veterinary imaging  Airport and customs cargo inspection for homeland security

National Institute of Standards and Technology U.S. Department of Commerce Pre-process for creating gem 1. Identify candidates  Review survey data  Periodic meetings with project managers  Condensed versions of other ATP reports (cost-benefit studies, status reports) 2. Initial research  Web research, project files 3. Prioritize based on  “Readiness” – are benefits actual or potential?  Technology/geographic area (internal customer need) 4. Now start writing the prioritized gems!  Keep track of potential gems

National Institute of Standards and Technology U.S. Department of Commerce Process for creating gem 1. Write draft based on: Review of project files, web research 2. Internal review Project manager, Group leaders, Division chiefs 3. Company review 4. ATP Deputy director review 5. Put on web:

National Institute of Standards and Technology U.S. Department of Commerce Questions?  How does this apply to your organization?