Benefit-Cost Analysis: ATP Experience Jeanne Powell Economic Consultant Technology Program Evaluation: Methodologies from the Advanced Technology Program March 18, 2008 National Science Foundation

Jeanne Powell, Economic Consultant2 Benefit-Cost Analysis: Outline Benefit-Cost Analysis of Federally-funded Technology: What is it? What is it good for? What metrics are computed? Whose return on investment? How does it work? ATP Studies Advantages Disadvantages Summary

Jeanne Powell, Economic Consultant3 Benefit-Cost Analysis What is it? Evaluation tool for: – Estimating net impacts of a federal technology development project or group of related projects according to program: Mission Logic Model Timeline – Comparing total estimated costs with total estimated benefits over a project’s useful life Can be Prospective (reflect projected future effects), or Retrospective (reflect past effects), or a mix of both

Jeanne Powell, Economic Consultant4 What is it good for? Quantitative case study Economic estimation Generally most suited to long-term evaluation – Risks and uncertainties of R&D are large in early phases; therefore, the probability distributions of the economic estimates are broad until technical and business uncertainties are reduced over the longer term Generally most suited to applied R&D, close to market

Jeanne Powell, Economic Consultant5 What does it measure? Return on Investment: – Net benefits – Benefit-to-cost ratio – Internal rate of return What investment? – Depends on whose return is being measured

Jeanne Powell, Economic Consultant6 Whose Return? Social Return Analysis Private return – return to project participants on their own investment/cost share Private profits to technology innovator Spillover return – return to others beyond the project participants, including broad societal benefits, on combined sources of investment Benefits to technology users (intermediate and final) Broader societal benefits (e.g., public goods) Social return – return to project participants and nation broadly on combined public program and private investments needed to generate return

Jeanne Powell, Economic Consultant7 Whose Return? Public Return Analysis Public return – Increased social return enabled by public investment relative to that public investment – Additionality effect – If multiple public sources contributed to the benefit, need to allocate benefit

Jeanne Powell, Economic Consultant8 How Does It Work? (1) Usually is based on monetary valuation of multiple types of benefits expressed as cash-flow analysis

Jeanne Powell, Economic Consultant9 Cash Flows for Project Time Investments Cash Flow Benefits Offsets

Jeanne Powell, Economic Consultant10 How Does It Work? (2) Cash flows are time adjusted – To constant dollars – For opportunity cost of money OMB Circular A-94 mandates use of a 7% real rate. As a result, ATP has used a 7% real rate in nearly all benefit-cost studies Discount rate consistency will be important for comparing results of different investments Cash flows are most commonly discounted back to the start of the project/investment being analyzed Cash flows are adjusted for risks and uncertainties using expected value analysis Metrics are computed—typically with Excel

ATP’s Studies To Date Short Title (# of Projects) Pub Number Contractor Pub Date Green Technologies Cluster (2)GCR Delta Research2007 DNA Diagnostics Cluster (2)GCR RTI International2007 Photonics Cluster (2)GCR Delta Research2005 Composites Cluster (2)GCR Delta Research2004 2mm Project—Retrospective (1)GCR MIT2004 HDTV Joint Venture (1)GCR RTI International2004 A-Si Detectors for Digital Mammography (1)GCR Delta Research2003 Component-Based Software (8)GCR RTI International2002 Closed-Cycle Refrigeration (1)GCR Delta Research2001 Digital Data Storage (2)GCR RFF2000 Flow-Control Machining in Auto Industry (1)NISTIR 6373NIST-BFRL1999 Tissue Engineering (7)GCR RTI International1998 2mm—Early Assessment (1)GCR CONSAD1997 Printed Wiring Board (1)GCR Albert N. Link1997

Jeanne Powell, Economic Consultant12 Advantages (Cash-flow analysis approach) Provides quantitative estimates of impact that complement qualitative analysis Based on standard financial analysis used by private businesses. (The public finance literature has extended them to the analysis of public and societal benefits.) Models are intuitive and relatively easy for policy makers and others to understand Can be applied systematically and relatively uniformly Can be extended to analyze multiple projects

Jeanne Powell, Economic Consultant13 Disadvantages (cash-flow analysis approach) Modeling of spillover benefits is challenging and expensive as with other tools Many important spillover benefits may be difficult to express in monetary terms Private benefits are often confidential Results are sensitive to uncertainties about technical success, market adoption, and inter- industry diffusion

Jeanne Powell, Economic Consultant14 Advantages and Disadvantages Summary Benefit-cost analysis is a practical choice for long- term evaluation of program impacts Utility increases as empirical basis increases: Technical accomplishment Product development Entrepreneurial activity (including financing) Market adoption Economic diffusion

Jeanne Powell, Economic Consultant15 Send to: Visit ATP/TIP website: View ATP/TIP publications: For further information: