Misclassification between Patent Offices: Evidence from a Matched ...

MISCLASSIFICATION BETWEEN PATENT OFFICES: EVIDENCE FROM A MATCHED SAMPLE OF PATENT APPLICATIONS Alfons Palangkaraya, Elizabeth Webster, and Paul H. Jensen* Abstract—In this paper, we estimate the extent of misclassification in patent examination decisions between the European Patent Office (EPO) and the Japanese Patent Office (JPO), that is, applications that are incorrectly refused a patent or incorrectly granted a patent. Using a proxy for inventive step as the predictor of the correct decision, we find that the probability that a ‘‘true grant’’ application is refused is 6.1%, while the probability that a ‘‘true refusal’’ application is granted is 9.8%. However, we find no evidence of an increasing trend of granting ‘‘bad’’ patents at the EPO and JPO.

I.

Introduction

T

HE size of the inventive step threshold is the principal legal mechanism that separates innovations that should be granted a patent from those that should not. In economic terms, the inventive step is the fulcrum that balances patents’ investment-inducing effects and their anticompetitive consequences.1 In recent years, there has been a crisis of confidence about the efficacy of the patent system, the essence of which is that it has become too easy to obtain a patent. The concern is that the ensuing preponderance of ‘‘low-quality’’ patents2 may be causing friction in the innovation system, thereby retarding rather than stimulating innovation.3 Received for publication March 17, 2009. Revision accepted for publication March 24, 2010. * Melbourne Institute of Applied Economic and Social Research, and Intellectual Property Research Institute of Australia, University of Melbourne, Australia. The author order is randomized. We are grateful to Sean Applegate, Jongsay Yong, Bob Hunt, Dietmar Harhoff, David Encaoua, Sadao Nagaoka, John Creedy, Stefan Wagner and participants at the AEA conference in Tokyo, 2008, and the EPIP conference in Berne, 2008, for comments and suggestions. We are also very grateful to the editor, Philippe Aghion, and four anonymous referees for their valuable insights and comments on the paper. This work builds on a program of work on patent examinations and has benefited from earlier discussions and comments from Kay Collins, Gillian Jenkins and Victor Portelli, Des Ryan, Michael Caine, Dominique Guellec, Francis Gurry, Alan Marco, Damian Slyzis, John Slattery, Brian Wright, Suzanne Scotchmer, Tetsuo Wada, David Mowery, Kimberlee Weatherall, and Andrew Christie; participants at the Intellectual Property Research Institute of Australia seminar series in Brisbane, Sydney, and Melbourne; and patent examiners from the EPO and JPO. This project has been funded by the Intellectual Property Research Institute of Australia. Thanks are due to Helene Dernis, Akemi Tokai, and the Industrial Property Digital Library Help Desk Staff for assistance with compilation of the triadic and JPO data sets and Bronwyn Hall, Adam Jaffe, and Manuel Trajtenberg for provision of their NBER data set. 1 See Hunt (1999) for an analysis of the optimal size of the inventive step. Other studies have considered the optimal scope or length of a patent right (Gilbert & Shapiro, 1990; Gallini, 1992; Merges & Nelson, 1990) and the efficiency of the examination process (Lemley, 2001; Farrell & Merges, 2004). 2 In this paper, we use the term low-quality patent in a similar way to Sampat (2005) and Merges (1999): it refers to a patent that would not have been granted if the legal threshold of novelty, nonobviousness, and usefulness had been properly evaluated. In the terminology developed in this paper, the application has been incorrectly granted, or ‘‘misclassified.’’ Conversely, patent applications can also be incorrectly refused, and these are also referred to here as being ‘‘misclassified.’’ 3 Although it is hard to verify whether innovation is actually slowing down, there is enough anecdotal evidence to cause serious concern that the patent system is being abused.

Numerous possible remedies such as raising the height of the inventive step and increasing examination rigor have been proposed (Batabyal & Nijkamp, 2008; Shapiro, 2007; National Academies of Science, 2004; Jaffe & Lerner, 2004). In this paper, we take an empirical look at the issue of patent quality. Our focus is on the accuracy of patent office examination decisions at the European and Japanese Patent Offices (EPO and JPO, respectively)—that is, we examine whether patent applications that are revealed ex post to have a high inventive step are refused by either patent office and whether patent applications with a low inventive step are granted. The former can be thought of as an incorrect refusal (a type 1 error) and the latter as an incorrect grant (a type 2 error). Our underlying premise is that given that all patent offices party to the agreement on Trade-Related Aspects of Intellectual Property Rights (TRIPS) have the same fundamental intention,4 the decision to refuse or grant an application should be the same across offices for the same invention. Differences in country-specific legal text, office protocols, examiner capabilities and staffing levels may explain differences in decisions but should not be used as an excuse for them. Our empirical model is based on the misclassification model developed by Hausman, Abrevaya, and Scott-Morton (1998), which has subsequently been extended and applied to various empirical issues, including insurance claims (Artı´s, Ayuso, & Guilleń, 2002), language indicators (Dustman & van Soest, 2001), education (Caudill & Mixon, 2005), and smoking data (Kenkel, Lillard, & Mathios, 2004). Our subject matter is patent examination decisions made at the EPO and JPO between 1990 and 2004 on a matched sample (that is, twin) patent applications. These twin patent applications are patent applications for the same invention as indicated by the same unique priority number in the applications.5 We use external and ex post information about the patent applications—in particular, citation data from the U.S. Patent and Trademark Office (USPTO)—to estimate the probability of granting a patent, taking into account the possibility of an error having been made.6 Different patent examination decisions for otherwise identical twin applications indicate misclassification. Citations are used in this manner because other evidence suggests that 4 All signatories to TRIPS agree on some fundamental minimum standards for the patent system, including that patents are granted in all fields of technology, are enforceable for a minimum of twenty years, and are granted only to inventions that are novel, nonobvious, and useful. 5 Graham et al. (2003) and Graham and Harhoff (2006) use a similar approach in the analysis of patent quality issues. 6 Our empirical approach relies on the fact that all patent applications in our sample are granted by the USPTO. Thus, we have data on forward citations for every patent in our sample.

The Review of Economics and Statistics, August 2011, 93(3): 1063–1075 Ó 2011 by the President and Fellows of Harvard College and the Massachusetts Institute of Technology

1064

THE REVIEW OF ECONOMICS AND STATISTICS

forward citations are a good measure of the technological value (or inventive step) of an invention, the idea being that more highly cited patents represent a larger technological advance over the prior art (Carpenter, Narin, & Woolf, 1981; Narin, Noma, & Perry, 1987; Narin & Olivastro, 1988; Karki, 1997; Albert et al., 1991; Breitzman & Mogee, 2002). Using the estimated coefficients from this model, we compute a predicted patentability score and a minimum inventive step threshold. We rank-order patent applications according to their estimated patentability score and then use the minimum inventive step threshold to determine whether an application should have been granted (a ‘‘counterfactual true grant,’’ henceforth referred to as a ‘‘true grant’’) or should have been refused (a ‘‘counterfactual true refusal,’’ henceforth referred to as a ‘‘true refusal’’). Misclassification occurs when a true grant is refused or a true refusal is granted. We then examine the determinants of misclassification using additional exogenous information, including data on patent renewal at the USPTO, examination duration, the number of claims, the speed of technological change, and a set of control variables. The rest of the paper is structured as follows. In section II, we briefly discuss common legal patentability criteria that patent offices use to examine patent applications and describe the role that citations play in the patenting process. In section III, we specify the empirical model. In section IV, we explain the data used to estimate the model. In section V, we present and discuss the results. In section VI, we conclude. II.

Background

Patents are temporary legal rights granted to inventors in order to allow them to prohibit others from using their invention. At least since the mid-nineteenth century, they have played an important role in fostering innovation in the developed world. From an economic perspective, the fundamental purpose of a patent is to solve the ex ante investment problem: that is, in the absence of a legal right to recoup the returns generated by an invention, firms may not invest in the invention in the first place. By attenuating the underinvestment problem, however, patents create a deadweight loss since patent owners charge monopoly prices for their inventions. In a world of cumulative innovation, they may also impose a tax on subsequent innovators. Since issuing patents incurs a social cost, not all inventions should be patentable. Each national patent office in the developed world uses essentially the same criteria to evaluate patent applications: novelty, nonobviousness, and utility. An invention is eligible for a patent only if it is new to the world (that is, it is novel) and if it represents a nonobvious increment in the state of the art (often referred to as the inventive step). The former is objective (since an invention is either new or not), while the latter is subjective (the size of the inventive step is essentially an issue of judg-

ment). The examination process is designed to filter out applications that should not be granted. In practice, the examination cut-off point is difficult to articulate, and it is generally considered that the pivotal examination rule that patent offices use to separate the wheat from the chaff is ‘‘inventive step’’ or ‘‘nonobviousness.’’ Examiners make their examination decision by comparing the application with the state of existing knowledge, or the prior art. As part of preparing a patent application, inventors must disclose other published ‘‘art’’ (which may be contained in previous patents or other publications, often scientific), which is related to the technology contained in their application. These disclosures of prior art are typically referred to as a backward citation. Citations play an important legal role in terms of delineating the boundary of the property right associated with the invention. In addition, prior art citations by subsequent applications to an existing patent, otherwise known as forward citations, provide useful information on how inventive the technology is since we expect that standout inventions will be more likely to be cited (Carpenter et al., 1981; Narin et al., 1987; Narin & Olivastro, 1988; Karki, 1997; Albert et al., 1991). According to the 1995 U.S. Department of Commerce’s, Manual of Patent Examining Procedures, section 904.02, the patent examiner is ‘‘not called upon to cite all references that are available, but only the best’’ (cited by Breitzman & Mogee, 2002). Independent information on a patent’s technological significance has been used by Carpenter et al. (1981), a group of Worcester students and the USPTO, and Albert et al. (1991)7 to show that USPTO forward citations are significantly and positively related to the size of the technological step.8 At the EPO, an initial search report is undertaken (prior to the request to perform a complete examination) in order to evaluate the patent application’s novelty. This search report provides applicants with some feedback as to the application’s novelty compared to the state of the art. At this stage, applications may be given an X or Y citation. An X citation refers to documents that, when taken alone, imply that the claimed invention cannot be considered 7 Carpenter et al. (1981) compare citations to 100 patents given an IR100 award, which is a independent award granted to 100 of the most innovative new products each year, with a randomly selected set of 102 control patents. Their findings showed that IR-100 award patents were cited more than twice as frequently as the control group. The study by Worcester students and the USPTO used independent assessments by engineers and patent attorneys to show that technological step was positively correlated with citation rates (cited in Breitzman & Mogee, 2002). The study by Albert et al. (1991) took a set of 57 silver halide technology patents and found a strong and positive correlation between each patent’s citations and its corresponding technological importance as assessed three times by lab scientists and middle- and upper-research managers. Finally, the 1996 study by Breitzman and Narin (cited in Breitzman & Mogee, 2002) found that patents listed in the National Inventors’ Hall of Fame, patents in the U.S. Department of Commerce list of Historical Significance, and the federal district court’s list of pioneering patents were cited 7, 6, and 2.5 times, respectively, more than average. 8 This relationship has not yet been established for EPO or JPO citations.

MISCLASSIFICATION BETWEEN PATENT OFFICES

novel or cannot be considered to involve an inventive step. A Y citation is a lower standard of blocking patent and refers to relevant prior art if combined with another document of the same category. If a patent has been cited in a subsequent patent’s search report as an X or Y cited patent, then it could be considered to be more technologically important. III.

ð1Þ

where yi* is the unobserved inventive step of a patent application i, xi is an observed characteristic of the application correlated with the size of the inventive step, and ei is a random error associated with measuring the inventive step. Since yi* is a latent variable (unobserved by the econometrician but observed by patent examiners), equation 1 can be estimated based on the observed decision of the examiner. A. Estimating the Probability of Misclassification

Suppose we observe the actual patent examination decision yi (¼ 0 if refused; ¼ 1 if granted). If there is no misclassification of patent applications, the observed decision (yi) is identical to the correct decision and equation 1 can be estimated in the usual way by combining equation 1 and yi ¼ 1ðyi 0Þ;

ð2Þ

such that Eðyi jxi Þ ¼ Prðyi ¼ 1jxi Þ ¼ Fðx0i bÞ;

ð3Þ

where F() is a common c.d.f of ei. Now suppose that patentability is not perfectly observable, even to patent examiners. In other words, let us allow the possibility of misclassification. Here, the observed decision yi may no longer be related to yi* as given in equation 2. In this case, if we denote the true outcome of the patent application as y~i , then y~i ¼

Eðyi jxi Þ ¼ Prðyi ¼ 1jxi Þ ¼ Prð~ yi ¼ 1jxi ÞPrðyi ¼ 1j~ yi ¼ 1Þ þ Prð~ yi ¼ 0jxi Þ Prðyi ¼ 1j~ yi ¼ 0Þ 0 ¼ Fðxi bÞð1 aI Þ þ ð1 Fðx0i bÞÞaII

0Þ:

ð4Þ

Following Hausman et al. (1998), we can specify the probability of misclassification as follows. The probability of incorrectly refusing a patent application, aI, is defined as:

Notice that equation (7) collapses to equation (3) if the probabilities of misclassification (aI and aII) equal 0. Hausman et al. (1998) proposed that equation (7) can be estimated by maximizing the following likelihood function, ^ ¼ n1 ÎI ; bÞ Lð^ aI ; a

ð5Þ

while the probability of incorrectly granting a patent application, aII, is defined as yi ¼ 0Þ: aII ¼ Prðyi ¼ 1j~

ð6Þ

n n h i X ^ ÎI þ ð1 a ÎI a Î ÞFðx0i bÞ yi ln a i¼1

h io ^ ÎI ð1 a ÎI a Î ÞFðx0i bÞ þð1 yi Þ ln 1 a

ð8Þ

^ where a ^ are estimates of aI, aII, ÎI ; bÞ, Î , a ÎI , and b over ð^ aI ; a 10 and b, respectively. The condition for identification of (aI, aII, b) is shown by Hausman et al. (1998) to be the monotonicity condition: aI þ aII < 1:11

ð9Þ

We estimate equations (1) and (8) using normalized forward citations at the USPTO (Normalized Forward Citations) as our explanatory variable of patentability (described below). We apply the Hausman et al. (1998) approach by maximizing the likelihood function, equation (8), using pooled matched data on patent examination decisions from the EPO and the JPO and assuming a binary probit specification. B. Estimating the Determinants of Misclassification

Using the estimated coefficients of the model with misclassification, we compute a predicted patentability score for each application given its citation rate. This provides an estimate of the true status of the patent application (~yî ), ^ y Þ; ~^yi ¼ 1ðx0i b

ð10Þ

where y is an unobserved minimum patentability threshold. We estimated y such that the resulting proportions of misclassification are as close as possible to the estimates of aI 9

aI ¼ Prðyi ¼ 0j~ yi ¼ 1Þ;

ð7Þ

) Eðyi jjxi Þ ¼ aII þ ð1 aI aII ÞFðx0i bÞ:9

Our empirical model of patent decisions includes what Hausman et al. (1998) call ‘‘misclassification’’ in the dependent variable of a binary response model. Formally, it is

1ðyi

Thus, the expected value of the observed grant is given as

Empirical Model

yi ¼ x0i b þ ei ;

1065

From equations (1), (4), (5), and (6): Hausman et al. (1998) also proposed that the model can be estimated using nonlinear least squares. Ramalho (2002, 2007) proposed the generalized method of moments (GMM) estimators for a more general setup. Dustmann and van Soest (2001) extended Hausman, Abrevaya, and ScottMorton’s method into the panel and ordinal discrete dependent-variable case. 11 This restriction is imposed for estimation reasons only. 10

1066


and aII. Using the predicted values of ~^yi (the true status) and the observed examination decision yi (the actual status), we estimate the following models: 1. A binomial probit model of the probability of refusing an application given the true decision is ‘‘grant.’’ 2. A binomial probit model of the probability of granting an application given the true decision is ‘‘refuse.’’

The rationale for estimating models 1 and 2 is that they enable us to examine whether there is symmetry in the factors affecting the probability of incorrect grants and refusals. Although it is possible that some factors shape different types of misclassification uniformly, there is no a priori rationale for believing so. Thus, we model each type of misclassification separately and compare the results. C. Defining the Explanatory Variables

To estimate these two models, we relied on empirical evidence in the literature to develop a set of explanatory variables. As already argued, citations play an important legal role in terms of delineating the boundary of the property right associated with the invention. Moreover, forward citations at the USPTO provide useful information on how inventive the technology is, since we expect that standout inventions will be more likely to be cited. However, the relationship between forward citations and inventive step is noisy for a number of reasons. First, citation propensity varies by technology area. Older technology areas have a larger set of prior art and are therefore more likely to both make and receive citations than newer technology areas. Second, patents are more likely to be cited in their local patent office since, for example, it is harder for the USPTO to conduct prior art searches in Japanese nonscientific literature than it is to search in American patent databases. Third, more recent patents have had fewer years to accumulate citations and are thus truncated. Moreover, there may be an increasing trend in the likelihood of citation over time. All of these factors make raw citations data an imperfect proxy for inventive step. To minimize the noise in our proxy, we follow the fixedeffects method of Hall, Jaffe, and Trajtenberg (2002) and normalize forward citations by dividing the number of citations for each patent by the mean number of citations in their cohort. In our case, the cohort includes all patents within the same OST technology area,12 U.S. grant year, and U.S. inventor status group.13 To construct the variable Normalized Forward Citations, we take advantage of a unique characteristic of our data set: all of the patent appli12 OST technology area refers to the thirty U.K. Office of Science and Technology classifications derived from the International Patent Classification codes. 13 Normalized forward citations for patent i in tech area t, year y, U.S.inventor status u ¼ Number of cites to patent i=Number of cites to all patents in tech area t; year; y U:S:inventor status u.

cations we observe at the EPO and JPO have been granted by the USPTO. Therefore, they are all equally available for citation at the USPTO regardless of whether they were granted or refused at the EPO and JPO.14 We should also note that a recent stream of the literature has shown that the technological importance of the patent is influenced by whether the citations are inserted by the applicant or the examiner (see Sampat, 2005; Alcacer & Gittelman, 2006). Such data have been available at the USPTO only since 2001, so they cannot be utilized in this study. One of the key explanatory variables used in this part of the analysis is examination duration. An emerging theoretical and empirical literature is exploring the relationship between the duration of patent examination and the quality of the examination decision (see Regibeau & Rockett, 2007; Popp, Juhl, & Johnson, 2004; Merrill, Levin, & Myers, 2004). The rationale underlying this is that the more time that is taken to search the universe of prior art, the more likely the patent examiner is to accurately measure the invention’s novelty and inventiveness. It is possible, however, that the converse situation exists: that is, examiners spend more time with ‘‘bad’’ applications they intend to refuse in order to protect themselves against complaint. However, in our data set, we find support for the first hypothesis: the variable Exam Duration (the number of days between when the applicant requests an examination and when the patent office makes a final decision to grant or refuse the application) is positively correlated with normalized forward citations (in both a bivariate setting and with other regression controls such as technology, year, number of past applications, and claims).15 This suggests that examiners spend more time examining true grants. Ideally, the way to measure examination duration would be hours spent by the examiner in searching the universe of prior art of each specific application. However, this is unobserved. Another important potential determinant of misclassification is the examiners’ expertise at the patent office. Empirical evidence, such as the work by Cockburn, Kortum, and Stern (2003), has demonstrated systematic differences across individual patent examiners at the USPTO. Although we do not observe the individual characteristics of the patent examiners, such as their qualifications or professional experience, we argue that the relative specialization 14 Note that while it is possible to construct a similar citation indicator using the EPO patent citation database, it is likely that EPO patent citation data are endogenous to EPO patent examination decisions because granted EPO patent applications are more likely to be cited than refused ones, as argued above. This is not the case with the USPTO patent citation data. 15 We note that the length of the average examination duration has increased over time. This may be because despite the trend increase in fees and income at patent offices, there has been a disproportionately greater increase in the workload per examiner. One plausible explanation for this is that patent offices have found it difficult to hire and retain patent examiners (see, for example, King, 2003, as cited in Popp et al., 2004, and U.S. General Accounting Office, 2005, 2007, for the case of USPTO). Our trend variable should control for this effect.


of patent applications by technology area at each office might be a good proxy for the collective expertise of the office. In other words, given that Japan receives many more patent applications in the automotive industry than other offices do, Japanese patent examiners should find it easier to identify whether an application for a patent in an automotive-related technology area is in fact novel and nonobvious. To include this in our model, we construct a variable called Revealed Office Expertise, which captures the percentage of patent applications in each technology area at each office relative to the percentage of patent applications in that technology area worldwide.16 This variable is calculated using all patent applications for the filing years 1990 to 2004.17 Three other explanatory variables are of particular interest in understanding the determinants of misclassification: Years-In-Force, Speed of Technology Cycle, and Past Applications. The first of these is measured by using data from the USPTO on patent renewals. To be precise, it is calculated as the number of years-in-force as a U.S. patent divided by the mean of years-in-force in the same cohort. In our case, the cohort includes all patents within the same OST technology area, U.S. grant year, and U.S. inventor status group. Renewals are well known to proxy the commercial value of the invention since patent owners will not continue to pay maintenance (that is, renewal) fees on inventions that do not (or will not in the foreseeable future) generate returns.18 We use it here to proxy for the fact that applicants may have private information about the potential ex post value of the invention.19 To the extent that this is correct, applicants with potentially valuable inventions may be more persistent in their interactions with the patent office, and this may increase the likelihood that an examiner incorrectly grants an application. Anecdotal evidence suggests that there is less work pressure for examiners if they grant an application rather than refuse it, thereby providing an incentive for applicants to be persistent in their interaction with patent offices. Speed of Technology Cycle is included in the model since evidence suggests that in faster-moving technology areas, it is harder to identify the inventive step than in slower and older technology areas. If this conjecture is accurate, it may be that applications in faster technology 16

This is different form the Revealed Technological Advantage index, which is defined according to the country of origin of the patent (Soete & Wyatt, 1983). 17 Source: EPO Worldwide Patent Statistical Database (PATSTAT), April 2007. 18 At the USPTO, these maintenance fees must be paid in the fourth, eighth, and twelfth years after patent issuance and the fees associated increase at each renewal stage. In year 4, the maintenance fee is US$900 which increases to US$3,800 in year 12. Hegde and Sampat (2009) show that less than half (43%) of all patents issued by the USPTO in 1992 were renewed to full term. 19 Note that since all applications have been granted at the USPTO, this variable is not endogenous to the EPO or JPO decision, as argued earlier with respect to the use of USPTO patent citation data in preference to the EPO citation data.

1067

areas are more likely to be misclassified. Technology area is grouped at the OST level, and data on technological change have been supplied by CHI Research.20 Speed of technology cycle is calculated as the inverse of the mean age of backward citations in the relevant OST technology group over 1990 to 2001. Past Applications measures the number of previous applications the applicant made to each office over the period 1990 to 1995. It is included to control for the fact that applicants with many previous applications may have accumulated experience in the operation of a specific patent office and may therefore submit an application in such a way that it makes it easy for the examiner to make the correct decision. To capture any trends in the likelihood of misclassification that may be due to, for example, changes over time in patent office budgets, we have included the variable Decision Year. To control for effects associated with the fact that local applicants are able to use their own language to complete the application, which may make examination errors less likely we have included Local Inventor and U.S. Inventor dummy variables. The number of claims in the application, normalized by the average number of claims in that year (Number of Claims) has been included as a proxy for the complexity of the application. In addition, we have in the model a set of control variables, including patent office and technology area dummy variables. We recognize that these variables are not definitive measures of the underlying concepts we are trying to capture. This, however, is an unavoidable limitation of using a large data set based on summary variables. While econometric analysis based on large data sets has numerous advantages, we recognize that it should be complemented with more qualitative analysis based on case studies or surveys. A complete description of the explanatory variables and descriptive statistics is presented in the appendix. IV.

Data and Descriptive Statistics

A. Data Construction

The data for this study were derived from six main sources: 1. The OECD Triadic Patent Family (TPF) Database21 2. The EPO’s public access online database (esp@cenet)22 3. The JPO’s public access online Industrial Property Digital Library (IPDL) database 20 The variable for each OST technology group is the average median age of the patents cited on the front page of a patent document over the period 1990 to 1995. The measure assumes that the more recent the age of the cited patents, the more quickly one generation of inventions replaces another. 21 http://www.oecd.org/LongAbstract/0,2546,en_2649_33703_30921914_ 1_1_1_1,00.html. 22 http://ep.espacenet.com/search97cgi/s97_cgi.exe?Action=FormGen& Template=ep/EN/home.hts.

1068


4. The NBER Patent-Citations Data File (Hall et al. 2002) 5. The USPTO patent renewal data23 6. The EPO Worldwide Patent Statistical Database, April 2007

Considerable effort was taken to try to find multioffice (EPO-JPO-USPTO) applications that related to the same invention—that is, to find identical triplets. To do this, we started with the population of triadic patent families. These are defined as a set of patent applications for which the ‘‘priority application must have at least one equivalent patent at the EPO, at the USPTO, and at the JPO’’ (Dernis & Khan, 2004, p. 11). To control for the individual invention, we include only patent families where all applications had the same single priority application (the same parent) and the ‘‘parent’’ had no other ‘‘children.’’24 Essentially we removed all parent-child families that were one-to-many, many-to-one, and many-to-many. Since the USPTO did not publish patent applications prior to 2001, we can only consider application decisions at the EPO and JPO conditional on their being granted by the USPTO. Moreover, we only include non-PCT applications in our matched sample since data on PCT applications at the JPO were not readily available (however, at this time, PCT applications represented only 10% of the population). Short of reading every application to confirm that the claims are exactly the same, we believe that we have come as close as we can to obtaining a matched sample of applications.25 We also constrained the data set to include patent applications with priority years 1990 to 1995, for two reasons. First, it enables us to minimize the amount of data truncation with regard to the examination decision, since this provides at least eight years to examine the priority application (the data were downloaded in late 2004). Second, it enables us to avoid problems associated with the fact that the JPO had a policy of one claim per application prior to the introduction of the 1988 Japanese patent law reforms.26 While this cohort may appear dated, a few points should be borne in mind. First, many examinations in our cohort have been completed only recently and, in fact, 11.3% were still pending by the end of 2004. Hence, while the application is old, the final decision may not be. Second, we are trying to find systematic differences between patent office examination decisions that are indicative of inconsistencies with the intent of international policy. These differences may be due to a variety of reasons, such as the wording of the legal text, protocols, procedures, and staffing levels. 23

The renewal data were provided to us directly by the USPTO. For similar reasons, we also dropped any families involving continuation, continuation-in-parts, or divisional patent applications at the USPTO. About 20% of the Triadic Patent Family involves a U.S. patent with continuation and divisional applications. We expect that these will represent on average a narrower set of claims than its European and Japanese equivalent. 25 On this point, we also note that the mean difference in the number of claims between the USPTO and the EPO is 0. 26 See, for example, Sakakibara and Branstetter (2001). 24

TABLE 1.—OBSERVED EXAMINATION DECISIONS, BY PATENT OFFICE, FULL AND CLEANED SAMPLE EPO

Full sample JPO

Cleaned sample JPO

Refused (%)

Granted (%)

Total (%)

Refused (%) Granted (%) Total (%)

396 1.2 688 2.0 1,084 3.2

6,356 18.8 26,440 78.0 32,796 96.8

6,752 19.9 27,128 80.1 33,880 100.0

Refused (%) Granted (%) Total (%)

284 1.2 470 1.9 754 3.1

4,761 19.3 19,175 77.7 23,936 96.9

5,045 20.4 19,645 79.6 24,690 100.0

Although our results can guide prospective policy only to the extent that the systematic behavior we observe is still current, we believe this to be the case given that patent office rules and protocols are slow to change. Third, we note that triadic applications tend to be more valuable (and often more inventive) than the norm. The second and third data sources provide information on the status of applications at the EPO and the JPO, while the fourth and fifth data sets provided us with information on patent citations and renewals at the USPTO for our triadic families. The sixth data set provided us with X and Y citations from the EPO database. These data sets were match-merged on relevant patent or application number. Originally there were 70,473 non-PCT applications filed in the triadic patent offices that met our criteria. Of these, 33,880 received a final patent examination decision (grant or refuse) at both the EPO and JPO by the end of 2004. After dropping observations with missing values for any of our variables of interest, we have a cleaned sample of 24,690 twin applications that have been granted by the USPTO and have received final examination decisions from the EPO and the JPO.27 B. Descriptive Statistics

Examination decisions. Table 1 provides cross-tabulations of the EPO’s and JPO’s patent examination decisions for the full sample (33,880 observations) and the cleaned sample (24,690 observations). As can be seen from the table, there is virtually no difference between the cleaned and the full sample in terms of actual examination decisions. Overall, 78.0% of the applications received a grant decision in both offices, and 1.2% received a refusal in both offices.28 We caution readers against drawing strong infer27 See the appendix for more detailed discussion of the sample construction and possible selection bias. 28 Table A2 provides more descriptive statistics of the pooled sample.

MISCLASSIFICATION BETWEEN PATENT OFFICES TABLE 2.—MEAN NUMBER OF FORWARD CITATIONS AT USPTO (NORMALIZED FOR TECHNOLOGY, U.S. GRANT YEAR, U.S. INVENTOR STATUS), PRIORITY YEARS, 1990–1995 EPO JPO

Withdrawn

Refused

Pending

Granted

Total

Withdrawn Refused Pending Granted

0.804 0.937 1.051 1.028

0.845 0.899 1.227 1.043

0.857 0.902 1.173 1.263

0.836 0.955 1.079 1.114

0.827 0.946 1.090 1.110

Total

0.891

0.925

1.084

1.026

1.000

ences about patent standards from table 1 since it provides only a snapshot of those applications granted and refused at each office. For instance, it would be wrong to infer that the JPO has higher standards than the EPO simply because the rate of refusal is higher at the JPO (20.4%) than that at the EPO (3.1%) since it is well known that many applicants withdraw their EPO application after receiving an X or Y citation in the initial search report (for more on induced withdrawals at the EPO, see Lazaridis & van Pottelsberghe de la Potterie, 2007). Patent citations and examination decisions. To understand the relationship between examination decisions and forward citations, table 2 presents the mean number of normalized forward citations recorded by the USPTO for each patent application in our data set cross-tabulated by examination decision at the EPO and JPO. What is immediately apparent from the table is that granted patents in both offices have a higher inventive step (the normalized number of forward citations is 1.026 in the EPO and 1.110 in the JPO) than refused patents (0.925 in the EPO and 0.946 in the JPO). Applications that are refused at the EPO but granted at the JPO had a considerably higher number of normalized citations (1.043) than those that were granted at the EPO but refused at the JPO (0.955). This suggests a lower threshold at the EPO. The second point to note is that applications that are withdrawn at both offices have, as a group, the lowest level of inventive step. Applications that are still pending—between eight and fourteen years after filing—have a relatively high level of inventive step, especially at the EPO. Our data set is truncated as it does not include applications that were withdrawn, pending, or refused at the USPTO. As a consequence, we cannot use this as evidence to compare the quality of USPTO-granted with EPOgranted and JPO-granted patents. Moreover, it does not inform us about whether the JPO or EPO is making incorrect grants or refusals overall since we do not know the optimal examination threshold. We turn to an empirical determination of the inventive step threshold in the next section of the paper.29 29

We remind readers that we are not investigating the determinants of the optimal inventive step threshold. Rather, we are exploring a datadriven method for analyzing the actual inventive step threshold that different patent offices use.

1069

TABLE 3.—DETERMINANTS OF OBSERVED EXAMINATION DECISION Dependent Variable: Observed Decision Independent Variables

Equation (1)

Equation (8) (With Misclassification) Exclude Biotech and Software

Normalized forward citations Constant

0.045*** (0.006) 1.141*** (0.010)

LR

17,835.6

0.103*** (0.009) 1.398*** (0.003) 0.061*** (0.002) 0.098*** (0.008) 17,833.0

N

49,380

49,380

Î ¼ Prðyi ¼ 0j~ yi ¼ 1Þ a ÎI ¼ Prðyi ¼ 1j~ yi ¼ 0Þ a

0.106*** (0.017) 1.407*** (0.003) 0.063*** (0.002) 0.104*** (0.010) 17,425.0 48,222

Notes: Standard errors in parentheses. yi ¼ 1 is observed grant from patent office decision data; yi ¼ 0 is observed refusal from patent office decision data; y~i ¼ 1 is unobserved correct grant; and y~i ¼ 0 is unobserved correct refusal. *** Statistically significant at the 1% level.

V.

Results and Analysis

A. The Probability of Misclassification

The first set of results—on the probability of misclassification—is summarized in table 3. Since the estimation may be sensitive to the chosen starting values, we used the coefficient estimates obtained from the regular probit model without misclassification and zero misclassification probabilities as the starting values. The identification condition in equation (9) was imposed. In the first column, we provide the estimation of equation (1), which assumes that there is no misclassification.30 As expected a priori, the results indicate that the normalized forward citation variable is positive and statistically significant. In other words, more highly cited applications are associated with a higher probability of being granted. Although we cannot rule out the fact that this result may be driven by some unobservable characteristic of the examination process that is correlated with citations, we do not believe this is the case. In column 2 of table 2, we provide the estimates of equation (8), which allows the possibility of misclassification. Accordingly, we present not only the coefficient on forward citations (which remains positive and statistically significant) but also our estimates of the likelihood of both types of misclassification (for both offices jointly, since we are using pooled JPO and EPO data). In summary, the probability that a true grant application is refused is 6.1%, while the probability that a true refusal application is granted is 9.8%.31 Bear in mind that these estimates depend on the assumption that normalized forward citations are the correct indicator of whether an application should be granted or 30 Withdrawn and pending applications are excluded from these estimations. 31 We do not account for the fact that there may be other office errors made. For example, the EPO formally issues applicants the results of the preexamination international search report. Any errors made in these reports (such as an incorrect X or Y citation) are not addressed in this paper.

1070

THE REVIEW OF ECONOMICS AND STATISTICS FIGURE 1.—PROBABILITY OF TRUE GRANT

TABLE 4.—PERCENTAGE OF APPLICATIONS RECEIVING AT LEAST ONE FORWARD X OR Y CITATION, BY OBSERVED DECISION AND ‘‘TRUE’’ STATUS BY PATENT OFFICE True Status

EPO observed decision JPO observed decision

Refused Granted Refused Granted Total

Refused

Granted

Total

5.7 8.0a 5.3 8.6a 7.9

21.3a 23.3 22.5a 23.4 23.2

19.1 21.1 20.1 23.3 21.1

a

Incorrect decision.

refused. Finally, the last column of table 3 shows that the coefficient estimates are robust to the exclusion of the biotechnology and computer software technology areas.32 Figure 1 displays a smoothed kernel density plot of the predicted probability of grant based on the misclassification model estimates in the second column of table 3. It shows the distribution of true grants Prð~yî ¼ 1jxi Þ assuming that ‘‘truth’’ is a function of normalized forward citations. The computed minimum inventive step threshold value for a correct grant decision that minimizes the sum of squares of the difference between the resulting decision errors and the ÎI ) is 0.92 and is indiestimated decision errors (^ aI and a cated on the chart as a vertical dashed line.33 We do not convert the 6.1 and 9.8 percentage error rates into actual numbers of applications since these numbers depend on the actual sample selected. Our sample consists of more valuable and inventive applications (being triadic applications), so we would expect the absolute numbers of true refusals to be small (and therefore the number of incorrect grants to be small). In a more representative sample, we would expect the number of true refusals to be higher. To evaluate our method of misclassification, we call on an independent data source on whether the application has ever been cited as a blocking patent at the EPO.34 Table 4 presents the percentage of applications in each of our (mis)classification groups that, as of April 2007, had been cited as an X or Y citation on another application. A higher forward citation incidence implies higher importance.35 It 32 It has been argued that these two technology areas are where the triadic patent offices differ the most in terms of patenting requirements. 33 The resulting error probabilities are 0.123 and 0.101 as compared to ÎI ¼ 0:098, respectively. Note also that this figure is not Î ¼ 0:061 and a a a representation of the optimal inventive step threshold. Rather, it is an attempt to estimate the examination threshold that best fits the observed data and the estimated model of outcomes. 34 We thank an anonymous referee for suggesting this approach. 35 Unlike the situation for USPTO forward citations, there has not been a body of empirical literature using independent information to verify what an X and Y citation at the EPO means. The relationship posited here is a priori. However, we have no reason to believe that the relationship should be any different from that observed in the United States.

shows that while on average, 21.1% of applications overall have received this type of citation at least once, the percentage for applications that we deem a true refusal was between 5.3% and 8.6% while the percentage for those that we deem a true grant was between 21.3% and 23.4%. Thus, to the extent the forward X and Y citations are indicative of a higher inventive step, our classifications are more logically consistent than the observed examination decision.36 B. Determinants of Misclassification

To understand the determinants of misclassification, table 5 presents the estimated average percentage change due to a change in the explanatory variables on the probability of misclassification. These estimates are derived from two separate binomial logit models of incorrect refusal (given the true status is grant) and incorrect grant (given the true decision is refusal) using pooled EPO and JPO examination outcomes, where the ‘‘true’’ decisions are obtained as explained above.37 The marginal effects for our continuous variables measure the average percentage change in the probability of an incorrect decision if the independent variable changes from 1 standard deviation below the mean to 1 standard deviation above the mean. Incorrect refusals. Let us consider column 1 in table 5 regarding the determinants of an incorrect refusal. The factors determining the probability of an incorrect refusal can be grouped by whether they have a positive marginal effect (which, as a result of the model specification, means that an increase in the variable increases the likelihood of error) or a negative marginal effect (which means that an increase in the variable reduces the likelihood of error). One factor that causes a large reduction (9.2 percentage points) in the likelihood of an incorrect refusal is the number of years in 36 The same pattern is found if we limit our analysis to only the stronger X citations. We also examined the relative frequency of backward X and Y citations, as it could be indicative of the weakness of the patent application. We do find that refused applications at the EPO are more likely to have had at least one backward X or Y citation compared with granted applications. However, incorrect grants have a lower backward X and Y citation rate than average and incorrect refusals have a higher rate, which means that the EPO is making decisions that are internally consistent with their search reports. This does not, however, constitute independent verification of that decision. 37 The coefficient estimates are presented in table A3.


1071

TABLE 5.—AVERAGE PERCENTAGE CHANGE IN THE PROBABILITY OF MISCLASSIFICATION DUE TO A CHANGE IN APPLICATION CHARACTERISTICS Average Percentage Change in the Probability of: Type I Error: Refusal Given the True Status Is Granta

Type II Error Grant Given the True Status Is Refusalb

Examination duration (lþr cf. lr) Years-in-force as a U.S. patent (lþr cf. lr) Number of claims (lþr cf. lr) Speed of technology cycle (lþr cf. lr) Past applications (lþr cf. lr) Revealed office expertise (lþr cf. lr) Decision year (lþr cf. lr) Presence of a local inventor (cf. other) Presence of a U.S. inventor (cf. other) Technology area dummies Office dummy variable

0.8** 9.2*** 0.1 1.4*** 0.7** 0.9*** 9.7*** 3.2*** 1.6*** Yes Yes

2.2*** 10.4*** 1.7** 1.0 1.3** 1.8** 9.5*** 3.4*** 1.6** Yes Yes

Sample size Proportion y ¼ 1 Log likelihood Pseudo-R2

42,848 0.067 12,514 0.186

6,896 0.935 1,996 0.197

Explanatory Variables

Note: ***, **, * indicate that the coefficient estimates used to derive the average percentage changes are statistically significant at the 1%, 5%, and 10% significance level. Table A1 provides the coefficient estimates and their standard errors. The notation l r refers to 1 standard deviation above and below the mean, respectively. a Incorrect refusal ¼ 1 if the actual decision is refusal and the predicted true decision is grant; ¼ 0 if the observed decision is grant and the predicted true decision is grant. b Incorrect grant ¼ 1 if the observed decision is grant and the predicted true decision is refusal; ¼ 0 if the observed decision is refusal and the predicted true decision is refusal.

force at the USPTO. This implies that more valuable patents are less likely to be incorrectly refused. Although the precise welfare implications of this are difficult to examine, this suggests the social cost associated with an incorrect refusal is lower than one may expect because such refusal is associated with less valuable patents. Other factors that reduce the likelihood of making an incorrect refusal include examination duration (0.8 percentage points), presence of a local inventor (3.2 percentage points), and the number of past applications (0.7 percentage points). These results accord with our a priori expectations since our hypotheses were that increasing the resources dedicated to examination, familiarity with the culture and language of the domestic patent office, and the experience of the applicant should all improve the likelihood of making the correct decision.38 In contrast, the result on the speed of technology cycle is difficult to interpret: it suggests that incorrect refusals are less likely in fast-moving technologies. One plausible interpretation is that patent examiners err on the side of caution and grant too many applications in these technology areas, but this interpretation is perhaps better examined in the results on incorrect grants. In terms of factors increasing the likelihood of an incorrect refusal, decision year has a large effect (9.7 percentage points). This indicates that there has been a positive trend

Incorrect grants. The second column in table 5 presents the determinants of an incorrect grant. The first point to note about these results is that they are not a perfect mirror image of the results presented above on incorrect refusals. Although there are some variables—such as examination duration39 and number of past applications—whose influence is significant and the same sign as above, there are others whose effects are in opposing directions. One exam-

38 Examination duration is also positively correlated with normalized forward citations (in both a bivariate setting and with other regression controls such as technology, year, number of past applications, and claims). This suggests that examiners spend more time examining true grants and, perhaps because of this care, they do not inappropriately refuse them. We also find that more complex applications—those with greater claims—take longer to examine than otherwise. The number of claims has been controlled for in this regression.

39 We do not find evidence that more experienced applicants who know ‘‘the game’’ at each office receive a more favorable result—in this case an incorrect grant. In fact the opposite is true: more experienced applicants are less likely to receive an incorrect grant. Neither is there evidence that examination duration is proxying for ‘‘bad’’ patents (which then receive an incorrect grant) since examination duration is related to higher normalized forward citations. In any event, the length of the examination period is positively related to a lower incidence of incorrect grants.

over time in the probability of an incorrect refusal. Several other factors were also found to have a statistically significant effect: Somewhat curiously, the presence of a U.S. inventor also increased the likelihood of an incorrect refusal (1.6 percentage points), a result we find difficult to explain. Revealed office expertise also has an effect on the probability of an incorrect refusal (0.9 percentage points), which suggests that offices are more likely to be tougher in technology areas in which they specialize. Several technology areas were found to have higher incorrect refusal rates ceteris paribus: specifically, the information and communication technology, communications, electronics, and automobile technology areas. The normalized number of claims was not statistically significant. Similar to table 3, we estimated these equations for the sample without biotechnology and software patents as a robustness check and found essentially the same results.

1072


ple of this type of variable is years-in-force at the USPTO: here, an increase in the number of years actually increases the probability of the office making an incorrect grant by 10.4 percentage points. If our conjecture is correct that applicants have private (but imperfect) information about the potential ex post commercial value of the invention, our result may simply reflect the fact that applicants with valuable inventions are more persistent in their negotiations with the patent office and it is well known that examiners often face perverse incentives to grant patents to persistent applicants. Once again, the welfare implications are complex, but this would seem to be a welfare-reducing outcome since such patents may have genuine commercial value, but they may also have strategic value in the sense that they are used in cross-licensing negotiations. The fact that the patent has limited inventiveness is difficult for third parties to establish, and thus the incorrectly granted patent may create unjustified bargaining power for its owner in such crosslicensing arrangements. Two additional variables whose influence differs across the two types of misclassification are the office expertise and decision year. It appears that offices are less likely to incorrectly grant a patent when it is in their area of technological specialization. The other interesting comparison with the incorrect refusal results is that decision year is actually negative for incorrect grants—that is, there is a decreasing trend in the number of incorrect grants made, ceteris paribus. This implies that the recent (and ongoing) furor over the number of ‘‘bad’’ patents granted may be confined to the United States. More worrying, however, is that an increase in the number of claims appears to lead to a greater probability of an incorrect grant (1.7 percentage points). Claims inflation may be a strategy firms use to tilt the examiner’s decision toward erring on the permissive side. Some puzzles remain unanswered. For instance, the presence of a local inventor raised the probability of an incorrect grant, but the presence of a U.S. inventor lowered the probability of an incorrect grant. Perhaps the former result can be explained by the fact that locals push their domestic office to convince the patent office to grant a patent, but the latter result is a mystery. Over and above these effects, only the communications and electronics technology areas were found to have a significantly lower probability of an incorrect grant decision. Finally, similar to the incorrect refusal case, the estimations using the sample without biotechnology and software produced similar results to the full sample. VI. Conclusion

This paper models misclassification in patent office examination decisions. Our findings on misclassification depend on two assumptions. First, we have implicitly assumed that different patent offices should make the same decisions regarding the patentability of a specific invention. While there are obvious differences in examination protocols across the offices, we argue that this is a plausible

assumption since the fundamental principles of the examination process—that an invention is patentable only if it is novel, nonobvious, and useful—are at the core of all patent office examinations. While there may be some arguments in favor of allowing patent offices to make different decisions regarding the same patent application, it comes at a cost since it introduces complexity into firms’ investment decisions and complicates their freedom to operate. Second, our notion of a correct decision rests on the legal meaning of validity (that is, novelty, nonobviousness, and usefulness). From an economic perspective, however, whether an invention should be patentable depends on the relative net change to the incentive to invent and innovate and the size of the deadweight monopoly losses. The latter includes strategic ways to construct undesirable patent thickets, build patent portfolios to extract additional bargaining power in cross-licensing arrangements, or other rent-seeking activities. Our estimated size of misclassification effectively overlooks these issues. However, it is plausible that the legal and economic interpretations of patent validity are correlated. Third, similar to all identical twin studies, what we gain by controlling for the unobservable characteristics of the invention, we lose in not knowing how generalizable are the results. This is a function of the fact that twins are not random samples of the population. In particular, our sample includes only single-priority, non–Patent Communication Treaty (PCT) applications that are submitted for examination at each of the triadic offices. The main known bias this creates is toward more valuable and more inventive applications since filing at each of the three offices is costly, and applications with a small inventive step are most likely to be withdrawn prior to examination. Nonetheless, there is no reason that this bias in the quality of the application will cause higher or lower error rates in the examination decision and therefore no reason to believe our results are unrepresentative. As in all scientific endeavors, however, the robustness of the estimates depends on replicating the experiment using different data sets. Notwithstanding these caveats, our analysis reveals that the probability that a true grant application is refused is 6.1%, while the probability that a true refusal application is granted is 9.8%. Patent offices are less likely to misclassify an application the longer the duration of examination and the more experienced the applicant. It is possible that a trade-off between examination duration and accuracy exists, as has been discovered in other literature. Furthermore, in areas where the office has a relative specialization, incorrect grants are less likely to occur, but incorrect refusal errors are more likely to occur.

REFERENCES Albert, M. B., D. Avery, F. Narin, and P. McAllister, ‘‘Direct Validation of Citation Counts as Indicators of Industrially Important Patents,’’ Research Policy 20 (1991), 251–259.

MISCLASSIFICATION BETWEEN PATENT OFFICES Alcacer, J., and M. Gittelman, ‘‘How Do I Know What You Know? Patent Examiners and the Generation of Patent Citations,’’ this REVIEW 88:4 (2006), 774–779. Artı´s, M., M. Ayuso, and M. Guilleń, ‘‘Detection of Automobile Insurance Fraud with Discrete Choice Models and Misclassified Claims,’’ Journal of Risk and Insurance 69 (2002), 325–340. Batabyal, A., and P. Nijkamp, ‘‘Is There a Trade-off between Average Patent Pendency and Examination Errors?’’ International Review of Economics and Finance 17 (2008), 150–158. Breitzman, A., and M. Mogee, ‘‘The Many Applications of Patent Analysis,’’ Journal of Information Science 28 (2002), 187–205. Carpenter, M., F. Narin, and P. Woolf, ‘‘Citation Rates to Technologically Important Patents,’’ World Patent Information 3 (1981), 160–163. Caudill, S. B., and F. G. Mixon Jr., ‘‘Analysing Misleading Discrete Responses: A Logit Model Based on Misclassified Data,’’ Oxford Bulletin of Economics and Statistics 67:1 (2005), 105–113. Cockburn, I. M., S. Kortum, and S. Stern, ‘‘Are all Patent Examiners Equal? The Impact of Characteristics on Patent Statistics and Litigation Outcomes,’’ NBER working paper no. 8980 (2003). Dernis, H, and M. Khan, ‘‘Triadic Patent Families Methodology,’’ OECD STI working paper (2004). Dustmann, C., and A. van Soest, ‘‘Language Fluency and Earnings: Estimation with Misclassified Language Indicators,’’ this REVIEW 83:4 (2001), 663–674. Farrell, J., and R. P. Merges, ‘‘Incentives to Challenge and Defend Patents: Why Litigation Won’t Reliably Fix Patent Office Errors and Why Administrative Patent Review Might Help,’’ Berkeley Technology Law Journal 19 (2004), 943–970. Gallini, N., ‘‘Patent Policy and Costly Imitation,’’ RAND Journal of Economics 23 (1992), 52–63. Gilbert, R., and C. Shapiro, ‘‘Optimal Patent Length and Breadth,’’ RAND Journal of Economics 21 (1990), 106–112. Graham, S. J. H., B. H. Hall, D. Harhoff, and D. C. Mowery, ‘‘Post-Issue Patent ‘Quality Control’: A Comparative Study of U.S. Patent Reexaminations and European Patent Oppositions,’’ in W. M. Cohen, and S. A. Merrill (Eds.), Patents in the Knowledge Based Economy (Washington, DC: National Academies Press, 2003). Graham, S., and D. Harhoff, ‘‘Can Post-Grant Reviews Improve Patent System Design? A Twin Study of U.S. and European Patents,’’ CEPR discussion paper no. 5680 (2006). Graham, S., and D. Mowery, ‘‘Submarines in Software? Continuations in U.S. Software Patenting in the 1980s and 1990s,’’ Economics of Innovation and New Technology 13:5 (2004), 443–456. Hall, B. H., A. B. Jaffe, and M. Trajtenberg, ‘‘The NBER Patent Citations Data File: Lessons, Insights, and Methodological Tools,’’ in A. B. Jaffe, and M. Trajtenberg (Eds.), Patents, Citations, and Innovations: A Window on the Knowledge Economy (Cambridge, MA: MIT Press, 2002). Hausman, J. A., J. Abrevaya, and F. M. Scott-Morton, ‘‘Misclassification of the Dependent Variable in a Discrete-Response Setting,’’ Journal of Econometrics 87 (1998), 239–269. Hegde, D., and B. N. Sampat, ‘‘Examiner Citations, Applicant Citations and the Private Value of Patents,’’ Economics Letters 105 (2009), 287–289. Hunt, R. M., ‘‘Non-Obviousness and the Incentive to Innovate: An Economic Analysis of Intellectual Property Reform,’’ Federal Reserve Bank of Philadelphia working paper no. 99-3 (1999). Jaffe, A. B., and J. Lerner, Innovation and Its Discontents: How Our Broken Patent System Is Endangering Innovation and Progress and What to Do about It (Princeton, NJ: Princeton University Press, 2004). Karki, M. M. S., ‘‘Patent Citation Analysis: A Policy Analysis Tool,’’ World Patent Information 19:4 (1997), 269–272. Kenkel, D. S., D. R. Lillard, and A. D. Mathios, ‘‘Accounting for Misclassification Error in Retrospective Smoking Data,’’ Health Economics 13 (2004), 1031–1044. Lazaridis, G., and B. van Pottelsberghe de la Potterie, ‘‘The Rigour of EPO’s Patentability Criteria: An Insight into the ‘Induced Withdrawals,’’’ World Patent Information 29:4 (2007), 317–326. Lemley, M. A., ‘‘Rational Ignorance at the Patent Office,’’ Northwestern University Law Review 95 (2001), 1497–1532. Merges, R. P., ‘‘As Many as Six Impossible Patents before Breakfast: Property Rights for Business Concepts and Patent System Reform,’’ Berkeley Technology Law Journal 14 (1999), 577–615.

1073

Merges, R. P., and R. R. Nelson, ‘‘On the Complex Economics of Patent Scope,’’ Columbia Law Review 90:4 (1990), 839–916. Merrill, S., R. Levin, and M. Myers, A Patent System for the 21st Century (Washington, DC: National Academies Press, 2004). Narin, F., E. Noma, and R. Perry, ‘‘Patents as Indicators of Corporate Technological Strength,’’ Research Policy 16 (1987), 143–155. Narin, F., and D. Olivastro, ‘‘Patent Citation Analysis: New Validations Studies and Linkage Statistics,’’ in A. F. J. van Raan, A. J. Nederhoff, and H. F. Moed (Eds.), Science Indicators: Their Use in Science Policy and Their Role in Science Studies (Leiden: DSWO Press, 1988). National Academies of Science, A Patent System for the 21st Century (Washington, DC: National Academies Press, 2004). Popp, D., T. Juhl, and D. K. N. Johnson, ‘‘Time in Purgatory: Examining the Grant Lag for U.S. Patent Applications,’’ Topics in Economic Analysis and Policy 4:1 (2004), 1–43. Quillen, C. D. J. and O. H. Webster, ‘‘Continuing Patent Applications and Performance of the U.S. Patent Office—Updated,’’ Federal Circuit Bar Journal 15 (2006), 635–677. Ramalho, E. A., ‘‘Regression Models for Choice-Based Samples with Misclassification in the Response Variable,’’ Journal of Econometrics 106 (2002), 171–201. ——— ‘‘Binary Models with Misclassification in the Variable of Interest and Nonignorable Nonresponse,’’ Economics Letters 96 (2007), 70–76. Regibeau, P., and K. Rockett, ‘‘Are More Important Patents Approved More Slowly and Should They Be?’’ CEPR discussion paper no. 6178 (March 2007). Sakakibara, M., and L. Branstetter, ‘‘Do Stronger Patents Induce More Innovation? Evidence from the 1988 Japanese Patent Law Reforms,’’ RAND Journal of Economics, 32 (2001), 77–100. Sampat, B. N., ‘‘Determinants of Patent Quality,’’ Columbia University, mimeograph (September 2005). Shapiro, C., ‘‘Patent Reform: Aligning Reward and Contribution,’’ paper presented at the NBER Conference on Innovation Policy and the Economy (April 12, 2007). Soete, L., and S. Wyatt, ‘‘The Use of Foreign Patenting as an Internationally Comparable Science and Technology Output Indicator,’’ Scientometrics 5:1 (1983), 31–54. U.S. General Accounting Office, ‘‘Intellectual Property: USPTO Has Made Progress in Hiring Examiners, but Challenges to Retention Remain,’’ Report to Congressional Committees (2005). ——— ‘‘U.S. Patent and Trademark Office: Hiring Efforts Are Not Sufficient to Reduce the Patent Application Backlog,’’ Report to the Ranking Member, Committee on Oversight and Government Reform, House of Representatives (2007).

APPENDIX Table A1 shows that the total number of patent applications filed in the triadic patent offices was 190,583. Eliminating PCT and multiple-priority applications leaves 70,473 applications, of which 33,880 received a final patent examination decision (grant or refusal) at both the EPO and JPO by the end of 2004. The remaining 36,593 applications were either still pending or had been withdrawn in at least one office. We then match-merged the data for these 33,880 patent applications with the NBER patent database using the USPTO patent numbers (Hall TABLE A1.—SUMMARY OF COMPLETE PATENT APPLICATIONS IN THE TRIADIC OFFICES, 1990–1995 Office of Application All USPTO applications All EPO applications All JPO applications All triadic patent families PCT families Non-PCT families Single priority (examination decision in all 3 offices) Multiple priorities

Complete Patent Families 843,435 433,186 2,191,084 190,583 18,488 172,095 70,473 (33,880) 101,622

1074

THE REVIEW OF ECONOMICS AND STATISTICS TABLE A2.—VARIABLE DESCRIPTION AND SUMMARY STATISTICS Variable Grant ‘‘True grant’’ Normalized forward citations

Examination duration Years-in-force as a U.S. patent

Number of claims Speed of technology cycle Revealed office expertise

Decision year Presence of a local inventor Presence of a U.S. inventor Biotechnology

Drug Chemical Computer software (Graham & Mowery, 2004) ICT Communications Electronics Automobile Mechanical JPO as the examining office Pooled sample size

Description

Mean

Standard Deviation

Actual decision (1 if grant, 0 if refusal) Predicted ‘‘true grant’’ (1 if Prð~ yi ¼ 1Þ > 0:92) Number of forward citations for each patent divided by the mean number of citations in the OST technology area, U.S. grant year, and U.S. inventor status cohort for that patent Number of months lapsed between decision date and date of examination request Number of years the patent has been maintained (renewal fees paid) at the USPTO divided by the mean number of years in the OST technology area, U.S. grant year, and U.S. inventor status cohort for that patent Number of claims made in the U.S. patent, normalized by the average number of claims made in that year The inverse of the mean age of backward citations on USPTO patent applications in each OST technology group from 1990 to 2001. The percentage of all patent applications in each OST technology at each office as a proportion of percentage in each OST technology across the whole world during the period 1990–2004. Time variable indicating the year the patent application is refused or granted (1 ¼ 1990) Indicator with a value of 1 if at least one of the listed inventors on the U.S. patent resides in the jurisdiction of the examining office Indicator with a value of 1 if at least one of the listed inventors on the U.S. patent resides in the United States 1 if the IPC codes in U.S. patent include any of A01H (1/00, 4/00), A61K (38/00, 39/00, 48/00), C02F (3/ 34), C07G (11/00, 13/00, 15/00), C12M/N/P,Q,S, G01N (27/327, 33/53,54,55,57,68,74,76,78,88,92) 1 if the technological category in the NBER Patent Database Drugs and Medicals (excludes those listed in Biotechnology) (Hall et al., 2002) 1 if the technological category in the NBER Patent Database is Chemical 1 if the IPC codes in U.S. patent include any of G06F (3/&153/, 5/&165/, 7/&17/, 9/&19/&159/, 11/, 12/, 13/, 15/, 9/&19/, 15/, 9/&29/) 1 if the IPC codes in U.S. patent include any of G06, G11, H04 (except those listed in Computer Software) 1 if the technological category in the NBER Patent Database is Computers & Communications (exclude those in Computer Software and ICT above) 1 if the if the technological category in the NBER Patent Database is Electricals & Electronics 1 if the technological subcategory in the NBER Patent Database is 53 or 55 1 if the technological category in the NBER Patent Database is Mechanical (exclude those identified as in Automobile above) Office indicator: JPO ¼ 1, EPO ¼ 0 49,380

0.883 0.860 1.089

0.322 0.347 1.360

38.002

15.819

1.001

0.314

12.425

8.539

9.617

1.960

1.026

0.256

11.028

2.611

0.351

0.477

0.292

0.455

0.009

0.096

0.050

0.217

0.210

0.407

0.038

0.190

0.110

0.313

0.067

0.250

0.202

0.401

0.060

0.238

0.150

0.357

0.500

0.500

et al., 2002). This enabled us to collect more data on each patent application: data that are not available in the triadic patent family database such as application year, number and country of inventors, priority countries, number of claims, technology category, and the number of forward and backward citations. The last database provided us with information on whether the patent was renewed at the USPTO. The construction of our sample may introduce three selection biases: a U.S.-grant bias, a single-priority bias, and a non-PCT bias. The U.S.-grant bias occurs because we are not able to get information on applications that are refused by the USPTO. This bias is, however, probably small since the proportion of all original patent applications granted by the USPTO is as high as 80% to 90% (Quillen & Webster, 2006).

The single-priority bias is potentially more substantial since 53% of triadic family applications have multiple priorities. However, focusing on single-priority applications enables us to increase the certainty of the equivalence of the invention within a family. If multiple-priority applications were included, we would be less sure that differences in decisions are due to discrepancies in the standard of examination rather than variation in the quality of the invention. Similar to almost all matched-observation studies, sample selection biases are a concern only if they affect the interaction between the variables under consideration (in our case, the examination decision and citations). We have no clear reason to believe that this interaction will differ between our sample and the whole triadic population, and therefore the potential bias is minimal.


1075

TABLE A3.—LOGIT MODEL OF THE PROBABILITY OF MISCLASSIFICATION Independent Variables Examination duration Years-in-force as a U.S. patent Number of claims Speed of technology cycle Past applications Revealed office expertise Decision year Presence of a local inventor Presence of U.S. inventor Constant Technology dummies Patent office dummy Sample size Proportion y ¼ 1 Log likelihood Pseudo-R2

Incorrect Refuse (Type 1 Error) 0.002** 1.695*** 0.001 3.131*** 0.106** 0.204*** 0.212*** 0.545*** 0.240*** 4.072*** Yes Yes 42,484 0.067 12,514 0.186

(0.001) (0.050) (0.002) (0.822) (0.042) (0.074) (0.007) (0.042) (0.039) (0.171)

Incorrect Grant (Type 2 Error) 0.008*** 1.726*** 0.014** 2.137 0.197** 0.371** 0.215*** 0.576*** 0.254*** 4.724** Yes Yes

(0.003) (0.116) (0.006) (1.840) (0.097) (0.170) (0.017) (0.094) (0.115) (0.413)

6,896 0.936 1,996 0.197

Standard error in brackets. ***, **, * Significant at 1%, 5%, and 10% levels.

The exclusion of PCT applications may lead to a sample selection bias problem since it is probable that PCT applications are more valuable than non-PCT applications (applicants select the PCT route only if they intend to apply for patents in four or more countries. Given the substantial application costs, this suggests the inventions also have considerable commer-

cial potential). However, only 10% of patent applications in the time period studied here used the PCT route. Table A2 provides some descriptive statistics of the pooled sample, and table A3 presents the coefficient estimates for the determinants of misclassification.