1. Collaborative Expedition Workshop #71 National Science Foundation
Technology Program Evaluation: Methodologies from the Advanced Technology Program Research Applications of Data
Andrew Wang
Technology Innovation Program National Institute of Standards and Technology
Collaborative Expedition Workshop #71
National Science Foundation
March 18, 2008

2. Research Applications Using Data
Use of patents as a measure of R&D output
Use of survey data to assess R&D alliances

3. Use of Patents Patents as a measure of R&D output
What do patents measure?
Number of patents
Quality of patents

4. Patent Data and Measures
Patent data sources
Administrative data
Survey data
Archival data
Constructing patent measures
Project-level
Firm-level
Industry and Technology field

5. Research Application: ATP Project Structure and Patent Outcomes
Joint Ventures
University participation
Patent data
Firm-level,
ATP projects,
Firm participation vs. Firm-project participation
Trends in propensity to patent

6. Research Application: ATP Project Structure and Patent Outcomes
Regression analysis
Panel data,
Before and after comparison, treatment effect
Patents, by date of application, one year lag
Control variables: firm size, technology field
Patent outcomes
ATP project increases firm patenting 4%-25%
Joint Venture and University participation have positive effect on patenting

7. Use of Survey Data Survey measures What do surveys measure?
Quantitative data, objective, “hard” data
Qualitative data, subjective, perceptual

8. Research Application: Determinants of Success in R&D Alliances
What factors influence the success of R&D alliances in generating innovation?
Alliance Formation factors
Alliance Structure (e.g., number, type of partners)
Partner attributes (e.g., prior experience, capabilities)
Alliance Execution factors
Commitment and Communication
Governance and Trust

9. Research Application: Determinants of Success in R&D Alliances
What are measures of success of R&D alliances in generating innovation?
Qualitative assessment of project success and value
Achieving technical objectives
Intangible value
Patent applications
Commercial impact (e.g., cost savings, revenues)

10. Research Application: Determinants of Success in R&D Alliances
Survey of ATP Joint Ventures
Data for 397 firms in 142 R&D alliances
Findings from regression analysis
Alliance execution factors are better predictors of success than alliance formation factors
Commitment of human capital and frequency of communication are key factors in success
Contractual trust matters, but goodwill trust does not
More ambitious projects are more likely to be successful