1. Are All Patent Examiners Equal?: The Impact of Examiner Characteristics on Patent Statistics & Litigation Outcomes Iain Cockburn, Boston University & NBER Sam Kortum, U Minnesota & NBER Scott Stern, Northwestern, Brookings & NBER

2. Why look at examiners? PTO examination process is fundamental to the IP system, but its relationship to policy proposals and economic assessment are poorly understood Research on the “PTO production function” can help to – Evaluate policy initiatives aimed at PTO operations – Inform academic research on economic impact of IP Differences among examiners may be an important driver of patent quality and breadth

3. Rules versus Discretion The patent examination process is highly structured, but some discretion is necessary Despite careful monitoring and detailed rules (MPEP) examiners use prior knowledge and individual judgment to search prior art, determine scope of allowed claims, etc. – Exercise of discretion may have consequences for the scope of intellectual property rights Impact of discretion may depend on examiners’ experience, specialization, and training

4. Methodology Build qualitative understanding of examination process Identify examiners responsible for patents whose validity is reviewed by CAFC (1997-2000) Evaluate relationship between CAFC-tested patent statistics and examiner characteristics Test whether examiner characteristics are associated with CAFC validity findings

5. Key Qualitative Findings “There are as many patent offices as there are patent examiners” Extensive “apprenticeship” system w/ mentoring of junior examiners Prior art search technology and method varies across… – art areas – time Examiner job design varies in the degree of… – technological specialization – collaborative interaction “All applications are created equal” … potential for idiosyncratic exercise of discretion

6. Examiner heterogeneity: Qualitative Insights May result in persistent variation in – breadth of claims allowed (“generosity”) – approval times – degree of “self-citation” But…link to patent statistics or litigation outcomes claimed to be “noisy” – “Finding a needle in a haystack”

7. Data 182 patents where validity tested by CAFC (1997-2000) – 50% found invalid, more likely for computer/communication, less likely for drugs/medical 196 examiners on these patents, responsible for ~290,000 patents 1976-2000

8. Distribution of Examiner Experience

9. Distribution of Technological Specialization

10. Distribution of Examiner Cites / Patent

11. ANOVA tests Examiner “id” accounts for 8-10% of variation in – Approval times – Citations Made – Citations Received Strongly statistically significant even after controlling for technology class and application year

12. Do Observable Examiner Characteristics Impact Patent Statistics? Citations on CAFC-tested patents are strongly associated with Examiner Cites per Patent Even after inclusion of detailed technology and cohort fixed effects Approval time on CAFC-tested patents uncorrelated with examiner characteristics Nuanced relationship with generality and originality of “forward” citation

13. Do Examiner Characteristics Impact Litigation Outcomes?

14. How do CAFC Validity Patents Compare to “Average” Patents? CAFC SAMPLE TYPICAL PATENT Claims20.59-14 Citations Received 14.06-8 Citations Made 16.76-8 Approval Time2.211.76-2.05 CAFC Patents have more claims, cite more prior art, are more highly cited, and are associated with slightly longer approval times.

15. Do Examiner Characteristics Impact CAFC Validity Decisions? Probability of patent being found invalid – NOT related to examiner’s experience, recent workload, propensity to self-cite, approval time – Significantly higher when examiner has higher “lifetime” average level of citations received per patent (controlling for technology class and application year)

16. Do Examiner Characteristics Impact CAFC Validity Decisions? INVALIDVALID EXAMINER CITES / PAT6.895.72

17. Are CAFC rulings sensitive to “over-generous” examiners? YES! – Econometric procedure links probability of invalidity to that part of “forward” citations received which is attributable to the examiner’s propensity to allow patents which attract many cites So, if…. – citations received is an indicator of breadth of the patent AND – exercise of discretion by examiners results in variation in the average # of citations that “their” patents receive One  increase in Examiner Cites / Patent associated with a 25% decrease in the probability of validity

18. Origins of “Generosity” Differences in information (access to prior art) The Apprenticeship system Consistency across patents of a given examiner, not necessarily consistency across examiners – Level of review not dependent on social or economic value of invention Effort: easiest thing to do is “allow” patents Incentives

19. Consequences of “Generosity” Process by which application are allocated to examiners is particularly important Specialization induces variation in the “size” of the rights awarded – particularly of concern for immature technologies Potential burdens on – the judicial system – private intellectual property contracting

20. Implications Training, mentoring and monitoring are at the core of preserving “not too much, not too little” Should level of review depend on the potential economic and social value of application? CAFC rulings are responsive to exercise of examiner discretion

21. Conclusions Examiners are heterogenous Preliminary evidence does not support naïve hypotheses about negative impact of inexperience Courts tend to invalidate patents associated with “generous” examiners – Important role for judicial review in “trimming” discretion