Diagnostic performance of commercial serological

Diagnostic Microbiology and Infectious Disease xxx (2017) xxx–xxx

Contents lists available at ScienceDirect

Diagnostic Microbiology and Infectious Disease journal homepage: www.elsevier.com/locate/diagmicrobio

Diagnostic performance of commercial serological assays measuring Bordetella pertussis IgG antibodies Giorgio Fedele a,⁎, Pasqualina Leone a, Stefania Bellino a, Ilaria Schiavoni a, Claudia Pavia b, Tiziana Lazzarotto b,c, Paola Stefanelli a,⁎ a b c

Department of Infectious Diseases, Istituto Superiore di Sanità, Rome, Italy Operative Unit of Clinical Microbiology, St. Orsola-Malpighi Polyclinic, Bologna, Italy Department of Specialized, Experimental, and Diagnostic Medicine, St. Orsola-Malpighi Polyclinic, Bologna, Italy

a r t i c l e

i n f o

Article history: Received 19 July 2017 Received in revised form 26 October 2017 Accepted 8 November 2017 Available online xxxx Keywords: B. pertussis Serology Diagnosis IgG

a b s t r a c t Due to their specificity to B. pertussis antigens, immunoglobulin G (IgG) antibodies should be measured primarily for diagnosing pertussis. We compared the diagnostic performance of commercially available enzyme-linked immunosorbent assays (ELISAs) and chemiluminescent immunoassays (CLIAs) measuring IgG to B. pertussis antigens. An in-house ELISA with purified pertussis toxin (PT) was used as reference system. Commercial assays using PT only as coating antigen showed better performance as compared to those using a mixture of different antigens. The best diagnostic performances were achieved by CLIAs. Results were analyzed using a dual cutoff of either ≥125 IU/mL anti-PT IgG or ≥62 IU/mL anti-PT IgG for the in-house ELISA and accordingly to package inserts for commercial assays. Using the in-house ELISA at a 62 IU/mL cutoff, as the gold standard for interpretation of results from the commercial kits, resulted in lower sensitivity and higher specificity as compared to 125 IU/mL, thus, it may be especially useful in outbreak situations when high specificity is required. © 2017 Elsevier Inc. All rights reserved.

1. Introduction Pertussis is a vaccine-preventable disease; however, epidemic episodes have been recorded in countries with high vaccination coverage (Cherry, 2012; Tan et al., 2015; Winter et al., 2014). Multiple reasons for pertussis resurgence have been postulated, among them waning immunity, increased focus on noninfant pertussis, better diagnostic tests, and epidemiological changes that have made adolescents and adults a reservoir for Bordetella pertussis and the source of infection to the unvaccinated newborns (Althouse and Scarpino, 2015; Senzilet et al., 2001; Tan et al., 2015). The diagnosis of pertussis poses some issues. In early infancy, clinical manifestations, such as paroxysmal cough, apnea and cyanosis, can overlap with those of several other respiratory infections. School-age children are more likely to display the typical symptoms of pertussis, including the inspiratory “whoop”. Adolescents and adults are often paucisymptomatic and present cough and cold-like symptoms (Kilgore et al., 2016; Tozzi et al., 2003). Due to different clinical Abbreviations: gG, immunoglobulin G; ELISA, enzyme-linked immunosorbent assays; CLIA, chemiluminescent immunoassay; PT, pertussis toxin; IgG-PT, anti-pertussis toxin IgG; EQA, external quality assurance; ROC, receiver operator characteristics; AUC, area under the curve; PPV, positive predictive value; NPV, negative predictive value; PRI, proportion of recent infections. ⁎ Corresponding authors. Tel.: +39-06-4990-3354; fax: +39-06-4990-3168. E-mail addresses: [email protected] (G. Fedele), [email protected] (P. Stefanelli).

presentations among infants, adolescents, and adults, the laboratory confirmation of pertussis is required. Specific pertussis diagnosis includes direct bacterial detection in nasopharyngeal samples by culturing or polymerase chain reaction (PCR), and indirect detection by serological assay (Leber, 2014; van der Zee et al., 2015; Wirsing von König, 2014). In particular, serology is widely used for the diagnosis of pertussis in older children, adolescents, and adults and for seroepidemiological studies. Anti-B. pertussis antibodies can be detected by enzyme-linked immunosorbent assays (ELISAs). Recently, detection methods based on chemiluminescence (chemiluminescent immunoassays, CLIA) have received attention due to their low background, linearity, and wide dynamic range (de Ory et al., 2015; Haywood et al., 2014). Standardization of serological testing remains challenging despite international efforts to improve it. Commercial ELISAs are of different antigen composition and quality, while according to reference laboratories in the EU, serology testing should use purified nondetoxified pertussis toxin (PT) as coating antigen since only anti-PT antibodies are specific for B. pertussis (Guiso et al., 2011). Another key issue is that serological tests detecting immunoglobulin G (IgG) to purified PT are recommended for B. pertussis, while IgA and IgM antibodies have been proved to be less sensitive (Crowcroft and Pebody, 2006; de Greeff et al., 2012; Guiso et al., 2011; Watanabe et al., 2006). The interpretation of pertussis serology poses several obstacles. Despite vaccination, population studies found that B. pertussis circulates in the population and antibodies to PT are detectable in the majority of adolescent and adults

https://doi.org/10.1016/j.diagmicrobio.2017.11.006 0732-8893/© 2017 Elsevier Inc. All rights reserved.

Please cite this article as: Fedele G, et al, Diagnostic performance of commercial serological assays measuring Bordetella pertussis IgG antibodies, Diagn Microbiol Infect Dis (2017), https://doi.org/10.1016/j.diagmicrobio.2017.11.006

2

G. Fedele et al. / Diagnostic Microbiology and Infectious Disease xxx (2017) xxx–xxx

(Palazzo et al., 2016; Pebody et al., 2005; Watanabe et al., 2006). Moreover, the immune response against infection or vaccination cannot be distinguished and pertussis vaccination may interfere, thus knowledge of the patients' vaccination history is important for the correct interpretation of serologic results. Approaches to serological diagnosis of pertussis vary widely due to differences in assay characteristics, antigens used, and antibody isotypes detected. Reference laboratories in the EU recommend that serology testing should use purified PT as the only coating antigen (Guiso et al., 2011) and that concentrations of antibodies to PT should be expressed in international units per mL (IU/mL) according to the First WHO International Standard for Pertussis Antiserum (Human) (Xing et al., 2009). Nevertheless, assays using mixed B. pertussis antigens are available on the market and used for diagnosis. Here, we evaluate the diagnostic performance of commercial kits for the determination of anti-B. pertussis IgG in serum compared to an inhouse-developed ELISA detecting anti-pertussis toxin IgG (IgG-PT) which successfully passed 2 consecutive external quality assurance (EQA) schemes commissioned by the European Centre for Disease Prevention and Control (ECDC, 2014a; unpublished results). 2. Materials and Methods 2.1. Serum Samples A total of 25 serum samples were chosen among the leftover of a previous study aimed at the determination of IgG-PT seroprevalence in parents of infants hospitalized for respiratory tract infection (Fedele et al., 2017). Untouched serum aliquots were stored at −20 °C. Single aliquots were thawed, and IgG-PT determinations were performed in a single run for every different assay. 2.2. Serology Commercial assays were selected according to their availability on the market in Italy and based on the replies to a questionnaire on pertussis diagnostic capacity that we sent to microbiology labs throughout Italy (unpublished results). 2.2.1. ELISA Assay Kits were either purchased or kindly donated by the manufacturers. Table 1 displays information about the ELISA kits tested. Commercial ELISAs were: Bordetella pertussis IgG ELISA, from IBL International (Hamburg, Germany), referred to as IBL; NovaLisa™ Bordetella pertussis IgG-ELISA and NovaLisa™ Bordetella pertussis toxin (PT) IgG-ELISA from Novatec (Dietzenbach, Germany), referred to as NovaLisa and NovaLisa PT, respectively; Ridascreen® Bordetella PT IgG from R-Biopharm (Darmstadt, Germany), referred to as Ridascreen; Serion Elisa classic Bordetella pertussis toxin IgG from VirionSerion (Wurzburg, Germany),

Table 1 Information about in-house ELISA assay and commercial ELISA or CLIA kits for the diagnosis of pertussis. Kits

Antigens

Unit

Positive

Uncertain

Negative

In-house IBL NovaLisa NovaLisa PT Ridascreen Serion Euroimmun Liaison VirClia

PT PT and FHA PT and FHA PT PT PT PT PT PT

IU/mL Qualitative Qualitative IU/mL IU/mL IU/mL IU/mL IU/mL IU/mL

≥125 Positive Positive ≥100 ≥100 ≥100 ≥100 ≥100 N120

62–124 Intermediate Intermediate ≥40 and b100 ≥40 and b100 ≥40 and b100 ≥40 and b100 ≥40 and b100 62–120

b62 Negative Negative b40 b40 b40 b40 b40 b62

b40, no indication of B. pertussis; ≥40 and b100, testing of a second sample after 1–2 weeks; ≥100, indicative of recent infection. Dual cutoff for the in-house ELISA assay (62 and 125 IU/mL) was used based on ECDC Technical Document 2012, Guidance and Protocol for the Serological Diagnosis of Human Infection with Bordetella pertussis. As part of the EUpert-Labnet surveillance network.

referred to as Serion; Anti-Bordetella pertussis Toxin ELISA (IgG) from Euroimmun (Luebeck, Germany), referred to as Euroimmun. The assays were performed manually according to the instructions given in the package inserts. The package inserts were evaluated in respect to basic ELISA procedures, such as washing steps, incubation time and temperature, reading conditions, calculation, and interpretation of results. Washing steps for microtiter plates were done with a HydroFlex™ microplate washer (DiaSorin, Saluggia, VC, Italy) according to the instructions given in the package inserts. Microtiter plates were read by a BioTek™- ELX 800 reader (DiaSorin) with a dual-wavelength program according to the instructions given in the package inserts. If the tests used a calibration curve, this was calculated as specified by the manufacturer. Results were regarded as valid when all criteria set by the manufacturer were met and were interpreted according to the ELISA's package inserts as being negative, positive, or indeterminate. 2.2.2. CLIA Assay Kits were kindly donated by the manufacturers. Details are displayed in Table 1. Commercial CLIAs were LIAISON® Bordetella pertussis Toxin IgG from DiaSorin, referred to as Liaison, and Bordetella pertussis toxin VIRCLIA® IgG monotest from Vircell (Granada, Spain), referred to as VirCLIA. The assays rely on a fast and fully automated procedure, performed according to instructions provided by the manufacturers with dedicated instruments. Results were interpreted according to package inserts. 2.2.3. Reference In-House ELISA In-house ELISA measuring IgG-PT was used as a reference system. Purified PT was used as antigen (200 ng/mL). The ELISA assay was standardized within a European Sero-Epidemiology Network (Giammanco et al., 2003; Reizenstein et al., 1995). The assay was standardized to the First WHO International Standard for pertussis antiserum (06/140; NIBSC, Potter Bar, UK), and results were expressed in IU/mL IgG-PT. Purified PT was prepared in-house as described previously (Nasso et al., 2009). The minimal level of detection was 2 IU/mL. Detailed information about this method can be found in a previous study by Palazzo and colleagues (Palazzo et al., 2016). This method has successfully passed 2 consecutive European EQA commissioned by ECDC and organized by the EUpert-Labnet consortium (ECDC, 2014a; unpublished results) and is carefully monitored for imprecision and drift over time as previously described (Hallander et al., 2009). 2.3. Definition of Cutoff Values In clinical practice, diagnosis is mostly based on single-sample serology, and various cutoff values for IgG-PT have been proposed (Guiso et al., 2011). A serosurveillance of US sera modeled 3 separate populations according to their levels of IgG-PT: a higher cutoff was estimated at ≥94 IU/mL, and a lower cutoff was estimated at ≥49 IU/mL (Baughman et al., 2004). These data suggested that values between 49 IU/mL and 94 IU/mL may be regarded as indeterminate and would need further confirmation. A comparison of commercial ELISA assays available in Germany suggested the definition of negative samples with IgG-PT concentrations below 40 IU/mL, intermediate samples with IgG-PT concentrations ranging between 40 and 100 IU/mL, and positive samples with IgG-PT concentrations equal to or above 100 IU/mL (Riffelmann et al., 2010). A 2-component cluster analysis to serum samples of Dutch patients suspected for pertussis allowed to determine an optimal cutoff of 67 IU/mL (de Greeff et al., 2012). More recently, an ECDC technical document has suggested the use of a dual cutoff between 62 and 125 IU/mL to define a recent infection for patients who were not vaccinated during the last twelve months (ECDC, 2012). Most of the commercial assays refer to the ≥100 IU/mL and ≥40 IU/mL cutoff definition, with the exception of the VirCLIA assay which refers to



the 62 IU/mL and 120 IU/mL dual cutoff. For the in-house ELISA assay, we used a dual positivity cutoff: IgG-PT ≥125 or ≥62 IU/mL. 2.4. Statistical Analysis Cohen's kappa coefficient was used to measure the agreement between the in-house ELISA assay and each commercial ELISA or CLIA kit for the diagnosis of pertussis. Kappa was calculated considering the proportion of observations in accordance and due to chance. Values near zero indicate poor agreement; a value of 1 implies perfect agreement. To quantify the accuracy of each commercial kit in discriminating between serologically positive and negative individuals for IgG antibodies against Bordetella pertussis, receiver operator characteristics (ROC) curves and area under the curve (AUC) were calculated. The ROC curves were generated using the nonparametric ROC analysis, with the reference variable indicating the state of the serum sample (positive or negative) and each possible outcome of the diagnostic test as a classification cut point (thresholds included in the package inserts of the commercial kits). Moreover, the corresponding sensitivity and 1 − specificity of the points on the nonparametric ROC curves were calculated. The area under the resulting ROC curve was computed using the trapezoidal rule. Sensitivity, specificity, percentage of cases correctly classified, positive predictive value (PPV), and negative predictive value (NPV) were defined on the basis of 2 different cutoff points of the commercial kits, indicating a positive or uncertain result, and the dual positivity cutoff of the in-house ELISA considered as gold standard for the diagnosis of pertussis: 125 IU/mL (36% of infections) or 62 IU/mL (56% of infections), respectively. Finally, taking as reference the sensitivity and specificity levels observed in the study, PPV and NPV that would be observed in theoretical percentages of low (10%) and high (80%) disease prevalence were calculated. Statistical analyses were performed using Stata software, version 13 (Stata Cooperation, College Station, TX). 2.5. Ethics Statement This study was conducted in accordance with the Declaration of Helsinki. Informed written consent was obtained from individuals providing specimens at the time of sampling. 3. Results 3.1. Agreement Between the In-House Reference ELISA Assay and Commercial Kits for Pertussis Serology A linearity test was performed for kits that express concentrations of anti-PT IgG antibodies in IU/mL. The results, shown in Supplementary Fig. S1, indicate a linear correlation to the expected values from inhouse standard preparations. The higher regression coefficients were obtained by the in-house ELISA and the CLIA assays. The IgG-PT values of 25 serum samples selected for the present study have been previously measured by the in-house ELISA assay (Fedele et al., 2017). Among the individual sera, 9 samples had IgG-PT value ≥125 IU/mL (ranged between 128 and 538), 11 had IgG-PT value b62 IU/mL (ranged between b 2 and 47), and 5 had IgG-PT b125 IU/mL and ≥62 (ranged between 84 and 118), respectively, scoring positive, negative, and uncertain according to ECDC guidelines (ECDC, 2012) (Supplementary Table S1). Untouched frozen aliquots of the individual sera were reevaluated by the in-house ELISA assay, and the results obtained confirmed previous determinations (data not shown). Testing of individual sera with commercial assays was performed according to the manufacturer's instructions. Detailed results on 25 serum samples of the in-house ELISA assay and 8 commercial kits for the diagnosis of pertussis are shown in Supplementary Table S1. Positive, negative, or uncertain samples were defined as specified by the manufacturer (Table 1).

3

Table 2 Measurement of the agreement between the in-house ELISA assay and each commercial kit for the diagnosis of pertussis. Commercial kits

Cohen's kappa coefficient

95% C.I.

IBL NovaLisa NovaLisa PT Ridascreen Serion Euroimmun Liaison VirClia

0.07 0.36 0.74 0.49 0.74 0.69 0.81 0.86

−0.01 0.23 0.69 0.44 0.67 0.17 0.79 0.65

Agreement 0.14 0.55 0.87 0.65 0.81 0.8 0.94 0.93

Poor Fair Good Moderate Good Good Very good Very good

C.I., confidence interval.

The agreement between the in-house ELISA assay and each commercial kit for the diagnosis of pertussis was evaluated by the calculation of the Cohen's kappa coefficient. As shown in Table 2, the commercial ELISA assays which use mixed B. pertussis antigens displayed a poor or fair agreement with the reference in-house assay (κ values b0.40). ELISA assays which use only purified PT as antigen displayed a moderate or good agreement (κ values between 0.41 and 0.80), while commercial CLIA assays displayed a very good agreement (κ values ≥0.81). 3.2. Diagnostic Performance of Commercial Kits Sensitivity and specificity of the commercial kits were calculated. Two cutoff values were used to determine positive and negative samples by the in-house ELISA assay, i.e., IgG-PT ≥125 IU/mL and IgG-PT ≥62 IU/mL (Hallander et al., 2009). Regarding the commercial kits, we considered the 2 cutoff values specified in package inserts and shown in Table 1 (≥40 IU/mL and ≥100 IU/mL, or ≥62 IU/mL and ≥120 IU/mL for VirCLIA, or positive and intermediate for IBL and NovaLisa). ROC curves were generated plotting the true-positive rates (sensitivity) in function of the false-positive rates (1 − specificity). AUC values were considered as a measure of how well the commercial kits can discriminate positive samples. Fig. 1A–B shows the ROC curves generated using IgG-PT ≥125 IU/mL as positivity cutoff, while Fig. 1C–D shows those generated using IgG-PT ≥62 IU/mL. We found that kits using mixed antigens had lower AUC values than kits using purified PT only; in particular, the lowest AUC value was registered for the IBL assay, significantly lower as compared to the NovaLisa. AUC values of the CLIA assays were higher than those of ELISA kits using purified PT; however, the statistical significance was not reached. Diagnostic accuracy of ELISA assays was lower using IgG-PT ≥125 IU/mL as positivity cutoff compared to IgG-PT ≥62 IU/mL, while for CLIA assays, AUC values were comparable (Tables 3a–3b). Sensitivity (proportion of positives that are correctly identified) and specificity (proportion of negatives that are correctly identified) were inversely correlated to each other depending on the choice of the cutoff value: with a higher threshold of the commercial kits, specificity increased and sensitivity decreased; conversely, with a lower threshold, sensitivity increased and specificity decreased (Tables 3a–3b). In order to have a complete picture of the diagnostic performance of each commercial kit, the PPV and NPV were also calculated, which depend on sensitivity and specificity of the diagnostic test as well as on the prevalence of the disease. Based on dual positivity cutoff of the inhouse ELISA, different proportions of recent infections (PRIs) were obtained (36% and 56%; Tables 3a–3b). Higher NPV values were observed by increasing the sensitivity; conversely, higher PPV values were detected by increasing the specificity and the disease prevalence (Tables 3a– 3b). Specifically, when individual sera with IgG-PT ≥125 IU/mL in the in-house ELISA were considered positive, the higher cutoff for each commercial kit (≥100 or N 120) showed more balanced values of sensitivity,



1.00

B

0.75 0.00

0.00

0.25

0.50

Sensitivity

0.50 0.25

Sensitivity

0.75

A

1.00

4

0.00

0.25

0.50

0.75

1.00

0.00

1-Specificity IBL: 0.56 Reference

0.25

0.50

0.75

1.00

1-Specificity

NovaLi sa: 0.72

NovaLisa PT: 0.88

ROC Area comparison: P=0.0090, n = 25

Ridascreen: 0.81

Ser i on: 0.88

Euroi mmun: 0.92

Li ai son: 0.96

Vi r Cl i a: 0.99

Reference

0.50

Sensitivity

0.25

0.50 0.00

0.00

0.25

Sensitivity

0.75

0.75

1.00

D

1.00


C

0.00

0.25

0.50

0.75

0.00

1.00

0.25

0.50

1-Specificity IBL: 0.59 Reference

0.75

1.00

1-Specificity

NovaLi sa: 0.82

NovaLisa PT: 0.99

Ridascreen: 0.94

Ser i on: 0.93 Li ai son: 0.96 Reference


Eur oi mmun: 0.98 Vi rCl i a: 0.96

ROC Area comparison: P=0.6338, n = 22 Fig. 1. Receiver operator characteristics curves of 8 commercial ELISA or CLIA kits for the diagnosis of pertussis. The in-house ELISA was considered as reference using dual positivity cutoff: IgG-PT ≥125 IU/mL (A–B) and IgG-PT ≥62 IU/mL (C–D). The indicated values are the estimated area under the curve for each commercial kit. Values close to 0.5 and 1 indicate low and high accuracy on discriminating serologically B. pertussis-positive individuals, respectively.

specificity, PPV, and NPV (Table 3a), while when individual sera with IgG-PT ≥62 IU/mL were considered positive, the lower cutoff (≥40 or ≥ 62) conferred to the commercial kits more balanced values (Table 3b). In each case, Liaison, VirCLIA, and NovaLisa PT gave a better performance.

3.3. Diagnostic Performance of Commercial Kits in High or Low Settings of Pertussis Prevalence Our analysis evidenced that the diagnostic performance of commercial kits for pertussis serology is linked to the PRIs, i.e., to the prevalence

Table 3a Performance of commercial kits for the diagnosis of pertussis based on the in-house ELISA assay positivity cutoff: IgG-PT ≥125 IU/mL. Cut point: positive, IgG ≥100 or N120

Cut point: positive-intermediate, IgG ≥40 or ≥62

Commercial kits

ROC area

95% CI

Sensitivity (%)

Specificity (%)

Correctly classified (%)

PPV (%)

NPV (%)

Sensitivity (%)

Specificity (%)

Correctly classified (%)

PPV (%)

NPV (%)


0.56 0.72 0.91 0.81 0.89 0.90 0.97 0.98

0.35 0.51 0.74 0.59 0.68 0.69 0.80 0.85

100.0 100.0 100.0 100.0 66.7 77.8 100.0 88.9

12.5 43.7 81.2 62.5 100.0 87.5 93.7 100.0

44.0 64.0 88.0 76.0 87.5 84.0 96.0 96.6

39.1 50.0 75.0 60.0 100.0 77.8 89.9 100.0

100.0 100.0 100.0 100.0 84.2 87.5 100.0 94.1

100.0 100.0 100.0 100.0 88.9 100.0 100.0 100.0

6.2 37.5 62.5 50.0 66.7 62.5 68.7 71.4

40.0 60.0 76.0 68.0 75.0 76.0 80.0 82.6

37.5 47.4 60.0 52.9 60.0 60.0 64.2 66.3

100.0 100.0 100.0 100.0 91.4 100.0 100.0 100.0

0.76 0.88 0.99 0.93 0.97 0.97 1.00 1.00

PPV and NPV were calculated for PRI = 36%.



5

Table 3b Performance of commercial kits for the diagnosis of pertussis based on the in-house ELISA assay positivity cutoff: IgG-PT ≥62 IU/mL. Cut point: positive, IgG ≥100 or N120 Commercial kits

ROC area 95% CI


0.59 0.82 0.99 0.95 0.94 0.98 0.95 0.96

0.39 0.59 0.86 0.80 0.73 0.86 0.80 0.78


Sensitivity (%) Specificity (%) Correctly PPV (%) NPV (%) Sensitivity (%) Specificity (%) Correctly PPV (%) NPV (%) classified (%) classified (%)

0.79 100.0 0.93 100.0 1.00 85.7 1.00 100.0 0.99 46.1 1.00 64.3 1.00 71.4 1.00 57.1

18.2 63.6 100.0 90.9 100.0 100.0 100.0 100.0

64.0 84.0 92.0 96.0 70.8 80.0 84.0 73.9

60.9 77.8 100.0 93.3 100.0 100.0 100.0 100.0

100.0 100.0 84.6 100.0 59.3 68.8 73.3 64.7

100.0 100.0 100.0 100.0 92.3 100.0 92.9 92.9

9.1 54.5 90.9 72.7 90.9 90.9 90.9 100.0

60.0 80.0 96.0 88.0 91.7 96.0 92.0 95.6

58.3 73.7 93.3 82.3 92.8 93.3 92.9 100.0

100.0 100.0 100.0 100.0 90.3 100.0 91.0 91.7

PPV and NPV were calculated for PRI = 56%.

of the disease. The PPV and NPV of the commercial assays were then calculated for 2 hypothetical scenarios of pertussis prevalence. We assumed a PRI equal to 10% or 80%, as an index of low or high prevalence, respectively, taking as reference the sensitivity and specificity levels found in our study. As shown in Tables 4a–4b, PPV increased with the higher threshold of the commercial kits and the higher proportion of recent infections (80%); higher values of PPV were observed with the in-house ELISA at 62 IU/mL cutoff as gold standard. NPV increased with the lower threshold of the commercial kits and the lower proportion of recent infections (10%); higher values of NPV were observed with the in-house ELISA at 125 IU/mL cutoff as gold standard. Both in low- and high-prevalence settings, more balanced PPV and NPV were observed when individual sera with IgG-PT ≥62 IU/mL were considered positive (Tables 4a–4b). 4. Discussion Pertussis laboratory confirmation includes direct diagnosis by culture or detection of bacterial B. pertussis DNA on respiratory samples and indirect detection of specific B. pertussis antibodies in the serum of suspected patients (ECDC, 2012). B. pertussis can best be cultured during the first 2–3 weeks of cough. Bacterial culture is relatively cheap and simple to perform, but it is time consuming and suffers from low sensitivity. On the other hand, PCR and real-time PCR are highly sensitive and rapid, but they are expensive and technically more difficult to perform (ECDC, 2014b). After 2 weeks from the onset of the disease, serum B. pertussis-specific antibodies can be also detected for diagnostic purpose. Thus, serologic testing is useful for laboratory confirmation of patients with a late diagnosis for whom both culture and PCR are likely to be negative (Leber, 2014; Wirsing von König, 2014). The basis of serological tests for pertussis should be the identification of pertussis anti-PT IgG in serum. Various positivity cutoff values have been proposed for IgG-PT concentrations. In the Netherlands, a cutoff N125 IU/mL IgG-PT with higher specificity or 62 IU/mL with slightly lower specificity has been used (Melker et al., 2000). In this regard, a

large serologic survey of the US population found that 3 populations could be separated according to their levels of IgG-PT: one cutoff was estimated at 94 IU/mL, and a lower cutoff was estimated at 48 IU/mL (Baughman et al., 2004). Recent recommendations by an ECDC technical document suggested the use of a dual cutoff between 62 and 125 IU/mL to define negative, intermediate, or positive samples (ECDC, 2012). In this study, commercial serologic assay kits were evaluated taking into account 2 different cutoffs for the reference test (IgG-PT ≥125 IU/mL, IgG-PT ≥62 IU/mL) and for the commercial assays. Dual cutoff for the in-house ELISA was adopted according to international recommendations (ECDC, 2014a; Guiso et al., 2011), while for the commercial kits, the cutoffs were set as indicated in the package inserts (Table 1). The main result of the present study is that commercial assays using PT as the only antigen have a better performance as compared to those using a mixture of different antigens, displaying a good to an excellent diagnostic accuracy. This finding confirms previous data (Riffelmann et al., 2010) and the notion that other B. pertussis antigens, such as filamentous hemagglutinin, pertactin, and fimbriae, are less specific due to cross-reactivity with other microbial antigens (e.g., other Bordetella species, Haemophilus species, Mycoplasma pneumoniae, Escherichia coli) (Crowcroft and Pebody, 2006; Guiso et al., 2011). Some of the kits produced misclassifications, which could be in theory explained by random error. However, in the most worrying cases, samples were retested and overlapping results were obtained (data not shown). Noteworthy, the best diagnostic performance was achieved by the CLIA assay. An advantage of this method, likely contributing to the excellent results observed, is that it relies on a fast and fully automated procedure. However, it ought to be considered that the implementation of such methodology at regional level might be difficult. The ROC curves and the calculation of PPV and NPV revealed that the choice of the in-house ELISA at 62 IU/mL cutoff as gold standard results in a lower sensitivity and a higher specificity of the commercial kits, whereas the in-house ELISA at 125 IU/mL cutoff as gold standard corresponds to a higher sensitivity and a lower specificity. Overall, when individual sera with IgG-PT ≥62 IU/mL were considered positive, the

Table 4a Positive predictive value and negative predictive value in the identification of recent B. pertussis infection. Results based on the in-house positive cutoff IgG-PT ≥125 IU/mL. Cut point: positive, IgG ≥100 or N120

10%

Commercial kits

Sensitivity (%)

Specificity (%)

PPV (%)

NPV (%)

PPV (%)


100.0 100.0 100.0 100.0 66.7 77.8 100.0 88.9

12.5 43.7 81.2 62.5 100.0 87.5 93.7 100.0

11.3 16.5 37.1 22.9 100.0 40.9 63.8 100.0

100.0 100.0 100.0 100.0 96.4 97.3 100.0 98.8

82.1 87.7 95.5 91.4 100.0 96.1 98.4 100.0


10%

NPV (%)

Sensitivity (%)

Specificity (%)

PPV (%)

NPV (%)

PPV (%)

NPV (%)

100.0 100.0 100.0 100.0 42.9 49.6 100.0 69.3

100.0 100.0 100.0 100.0 88.9 100.0 100.0 100.0

6.2 37.5 62.5 50.0 66.7 62.5 68.7 71.4

10.6 15.1 22.9 18.2 22.9 22.9 26.2 28.0

100.0 100.0 100.0 100.0 98.2 100.0 100.0 100.0

81.0 86.5 91.4 88.9 91.4 91.4 92.7 93.3

100.0 100.0 100.0 100.0 60.0 100.0 100.0 100.0

80%

80%


6


Table 4b Positive predictive value and negative predictive value in the identification of recent B. pertussis infection. Results based on the in-house positive cut-off IgG-PT ≥62 IU/mL. Results for indicated PRI Cut-point: positive, IgG ≥100 or N120

10%

Commercial kits

Sensitivity (%)

Specificity (%)

PPV (%)

NPV (%)

PPV (%)


100.0 100.0 85.7 100.0 46.1 64.3 71.4 57.1

18.2 63.6 100.0 90.9 100.0 100.0 100.0 100.0

12.0 23.4 100.0 55.0 100.0 100.0 100.0 100.0

100.0 100.0 98.4 100.0 94.3 96.2 96.9 95.5

83.0 91.7 100.0 97.8 100.0 100.0 100.0 100.0

Results for indicated PRI

80%

lower threshold (≥40 or ≥ 62) conferred to the commercial kits more balanced values of sensitivity and specificity. In practice, the level of sensitivity and specificity of the assay depends on the specific objectives; a screening test would benefit most from a higher sensitivity, whereas a confirmation diagnostic test requires high specificity. Remarkably, when 2 hypothetical scenarios of low or high prevalence have been envisaged, the choice of an appropriate positivity cutoff may impact on serological diagnosis. In particular, when high prevalence was simulated, the higher thresholds of the commercial kits and the in-house ELISA at 62 IU/mL cutoff as gold standard seem to be more appropriate for diagnosis as also demonstrated in outbreak situations (Horby et al., 2005). However, in the evaluated sample, a value equal to 84 IU/mL was observed as discriminant positivity level. In conclusion, the present study suggests that, as pointed out by the EU reference laboratories (Guiso et al., 2011), commercial serology assays using a mixture of B. pertussis antigens do not seem to have a good performance for the diagnosis of pertussis. The use of purified PT as the only antigen and quantification of results in IU/mL would define a beneficial step for the serological diagnosis of pertussis. Supplementary data to this article can be found online at https://doi. org/10.1016/j.diagmicrobio.2017.11.006. Acknowledgments This study was partly supported by the Ministry of Health-CCM Project “Activities of common interest aimed at the continuation and arrangement of health surveillance”. The authors thank Simonetta Pietrangeli for excellent technical help. References Althouse BM, Scarpino SV. Asymptomatic transmission and the resurgence of Bordetella pertussis. BMC Med 2015;13:146. Baughman AL, Bisgard KM, Edwards KM, Guris D, Decker MD, Holland K, et al. Establishment of diagnostic cutoff points for levels of serum antibodies to pertussis toxin, filamentous hemagglutinin, and fimbriae in adolescents and adults in the United States. Clin Diagn Lab Immunol 2004;11:1045–53. Cherry JD. Epidemic pertussis in 2012—the resurgence of a vaccine-preventable disease. N Engl J Med 2012;367:785. Crowcroft NS, Pebody RG. Recent developments in pertussis. Lancet 2006;367(9526): 1926–36. de Greeff SC, Teunis P, de Melker HE, Mooi FR, Notermans DW, Elvers B, et al. Twocomponent cluster analysis of a large serodiagnostic database for specificity of increases of IgG antibodies against pertussis toxin in paired serum samples and of absolute values in single serum samples. Clin Vaccine Immunol 2012;19(9). de Ory F, Minguito T, Balfagón P, Sanz JC. Comparison of chemiluminescent immunoassay and ELISA for measles IgG and IgM. APMIS 2015;123:648–51. European Centre for Disease Prevention and Control. Guidance and protocol for the serological diagnosis of human infection with Bordetella pertussis. Stockholm: ECDC; 2012. https://ecdc.europa.eu/sites/portal/files/media/en/publications/Publications/ bordetella-pertussis-guidance-protocol-serological-diagnosis.pdf. European Centre for Disease Prevention and Control. External quality assurance scheme for Bordetella pertussis serology 2013. Stockholm: ECDC; 2014a. https://ecdc.europa.eu/ sites/portal/files/media/en/publications/Publications/EQA-pertussis-bordetella-serology.pdf.

Cut-point: positive-intermediate, IgG ≥40 or ≥62

10%

80%

NPV (%)

Sensitivity (%)

Specificity (%)

PPV (%)

NPV (%)

PPV (%)

NPV (%)

100.0 100.0 63.6 100.0 31.7 41.2 46.6 36.8

100.0 100.0 100.0 100.0 92.3 100.0 92.9 92.9

9.1 54.5 90.9 72.7 90.9 90.9 90.9 100.0

10.9 19.6 55.0 28.9 53.0 55.0 53.1 100.0

100.0 100.0 100.0 100.0 99.1 100.0 99.1 99.2

81.5 89.8 97.8 93.6 97.6 97.8 97.6 100.0

100.0 100.0 100.0 100.0 74.7 100.0 76.2 77.9

European Centre for Disease Prevention and Control. Expert consultation on pertussis—Barcelona, 20 November 2012. Stockholm: ECDC; 2014b. https://ecdc.europa.eu/sites/portal/files/media/en/publications/Publications/pertussis-meeting-2012.pdf. Fedele G, Carollo M, Palazzo R, Stefanelli P, Pandolfi E, Gesualdo F, et al. Parents as source of pertussis transmission in hospitalized young infants. Infection 2017;45:171–8. Giammanco A, Chiarini A, Maple PA, Andrews N, Pebody R, Gay N, et al. European SeroEpidemiology Network: standardisation of the assay results for pertussis. Vaccine 2003;22:112–20. Guiso N, Berbers G, Fry NK, He Q, Riffelmann M, Wirsing von König CH, et al. What to do and what not to do in serological diagnosis of pertussis: recommendations from EU reference laboratories. Eur J Clin Microbiol Infect Dis 2011;30(3):307–12. Hallander HO, Andersson M, Gustafsson L, Ljungman M, Netterlid E. Seroprevalence of pertussis antitoxin (anti-PT) in Sweden before and 10 years after the introduction of a universal childhood pertussis vaccination program. APMIS 2009;117:912–22. Haywood B, Patel M, Hurday S, Copping R, Webster D, Irish D, et al. Comparison of automated chemiluminescence immunoassays with capture enzyme immunoassays for the detection of measles and mumps IgM antibodies in serum. J Virol Methods 2014;196:15–7. Horby P, Macintyre CR, McIntyre PB, Gilbert GL, Staff M, Hanlon M, et al. A boarding school outbreak of pertussis in adolescents: value of laboratory diagnostic methods. Epidemiol Infect 2005;133:229–36. Kilgore PE, Salim AM, Zervos MJ, Schmitt HJ. Pertussis: microbiology, disease, treatment, and prevention. Clin Microbiol Rev 2016;29(3):449–86. Leber AL. Pertussis: relevant species and diagnostic update. Clin Lab Med 2014;34(2): 237–55. Melker HE, Versteegh FGA, Conyn-van Spaendonck MAE, Elvers LH, Berbers GAM, Zee A, et al. Specificity and sensitivity of high levels of immunoglobulin G antibodies against pertussis toxin in a single serum sample for diagnosis of infection with Bordetella pertussis. J Clin Microbiol 2000;38(2):800–6. Nasso M, Fedele G, Spensieri F, Palazzo R, Costantino P, Rappuoli R, et al. Genetically detoxified pertussis toxin induces Th1/Th17 immune response through MAPKs and IL-10-dependent mechanisms. J Immunol 2009;183:18929. Palazzo R, Carollo M, Fedele G, Rizzo C, Rota MC, Giammanco A, et al. Evidence of increased circulation of Bordetella pertussis in the Italian adult population from seroprevalence data (2012–2013). J Med Microbiol 2016;65:649–57. Pebody RG, Gay NJ, Giammanco A, Baron S, Schellekens J, Tischer A, et al. The seroepidemiology of Bordetella pertussis infection in Western Europe. Epidemiol Infect 2005;133(1):159–71. Reizenstein E, Hallander HO, Blackwelder WC, Kühn I, Ljungman M, Möllby R. Comparison of five calculation modes for antibody ELISA procedures using pertussis serology as a model. J Immunol Methods 1995;183(2):279–90. Riffelmann M, Thiel K, Schmetz J, Wirsing von König CH. Performance of commercial enzyme-linked immunosorbent assays for detection of antibodies to Bordetella pertussis. J Clin Microbiol 2010;48(12):4459–63. Senzilet LD, Halperin SA, Spika JS, Alagaratnam M, Morris A, Smith B, et al. Pertussis is a frequent cause of prolonged cough illness in adolescents and adults. Clin Infect Dis 2001;32:1691–7. Tan T, Dalby T, Forsyth K, Halperin SA, Heininger U, Hozbor D, et al. Pertussis across the globe: recent epidemiologic trends from 2000-2013. Pediatr Infect Dis J 2015;34(9):e222-2. Tozzi AE, Ravà L, Ciofi degli Atti ML, Salmaso S. Clinical presentation of pertussis in unvaccinated and vaccinated children in the first six years of life. Pediatrics 2003;112: 1069–75. van der Zee A, Schellekens JF, Mooi FR. Laboratory diagnosis of pertussis. Clin Microbiol Rev 2015;28:1005–26. Watanabe M, Connelly B, Weiss AA. Characterization of serological responses to pertussis. Clin Vaccine Immunol 2006;13:341–8. Winter K, Glaser C, Watt J, Harriman K, Centers for Disease Control and Prevention (CDC). Pertussis epidemic—California, 2014. MMWR Morb Mortal Wkly Rep 2014;63(48): 1129–32. Wirsing von König CH. Pertussis diagnostics: overview and impact of immunization. Expert Rev Vaccines 2014;13(10):1167–74. Xing D, Wirsing von König CH, Newland P, Riffelmann M, Meade BD, Corbel M, et al. Characterization of reference materials for human antiserum to pertussis antigens proposed by an international collaborative study. Clin Vaccine Immunol 2009;16(3): 303–11.
