certified by correctly testing a coded certification panel consisting of eight well-defined ... Spector, University of California-San Diego; S. Rasheed, Univer-.
JOURNAL OF CLINICAL MICROBIOLOGY, Dec. 1993,
p.
Vol. 31, No. 12
3123-3128
0095-1137/93/123123-06$02.00/0 Copyright © 1993, American Society for Microbiology
Establishment of a Quality Assurance Program for Human Immunodeficiency Virus Type 1 DNA Polymerase Chain Reaction Assays by the AIDS Clinical Trials Group J. BROOKS JACKSON,'* JAMES DREW,2 HSLANG JU LIN,3 PATRICIA OTTO,2 JAMES W. BREMER,3 F. BLAINE HOLLINGER,3'4 STEVE M. WOLINSKY,2 THE ACTG PCR WORKING GROUP,t AND THE ACTG PCR VIROLOGY LABORATORIESt Institute of Pathology, Case Western Reserve University and University Hospitals of Cleveland, Cleveland, Ohio 441061; Division of Infectious Diseases, Northwestern University, Chicago, Illinois 606112; and Division of Molecular Virology, Baylor College of Medicine,3 and Research Center on AIDS and HIVInfection, Veterans Affairs Medical Center,4 Houston, Te-xas 77030 Received 3 June 1993/Returned for modification 27 July 1993/Accepted 31 August 1993
An independent quality assurance program has been established by the Virology Committee of the AIDS Clinical Trials Group in the Division of AIDS, National Institute of Allergy and Infectious Diseases, for monitoring polymerase chain reaction (PCR) assays for human immunodeficiency virus type 1 (HlV-1) DNA that are performed by 11 laboratories participating in multicenter clinical trials in the United States. To perform HIV-1 DNA PCR for patients in AIDS Clinical Trials Group protocols, each laboratory was initiall certified by correctly testing a coded certification panel consisting of eight well-defined clinical whole-blood specimens and 30 cell pellets containing 0, 2, 5, 10, 20, or 50 8E5/LAV cells per 125,000 uninfected peripheral blood mononuclear cells. PCR was performed by one of two standardized commercial assays for amplification and nonisotopic detection of HIV-1 proviral DNA. For continuing certification, each laboratory must correctly test eight coded whole-blood samples per quarter and run three or four coded cell pellets and HIV-1 DNA copy standards with every PCR assay in real time. The PCR results for the coded pellets on each run are entered into an encrypted computer file, which immediately assesses the validity of the run. To date, 10 of 11 laboratories have correctly tested all HIV-i-positive and -negative samples in the initial certification panel on their first or second attempt. Subsequently, 9 of these 11 laboratories have continued to maintain their certified status. The use of commercial HIV-1 DNA PCR assays and an external quality assurance program have ensured that results from different laboratories are comparable and that problems with sensitivity and specificity are quickly identified.
The polymerase chain reaction (PCR) assay is an extremely sensitive technique for detecting nucleic acid sequences of infectious agents, including human immunodeficiency virus type 1 (HIV-1). In view of the well-documented *
accuracy of HIV-1 antibody tests, PCR probably has a limited role in the diagnosis of HIV-1 infection in adults. PCR may be especially useful, however, for detecting HIV-1 DNA in peripheral blood mononuclear cells (PBMC) of HIV-1-seropositive neonates, whose infection status is indeterminate for up to 18 months of age because of passively acquired maternal HIV-1 antibody (5). Preliminary studies suggest that HIV-1 DNA PCR is as sensitive or more sensitive than HIV-1 culturing for the early detection of HIV-1 in infected infants (4, 9). In addition, PCR assays can be performed in 1 day, as opposed to weeks for culturing. The ability of PCR to detect HIV-1 infection in approximately 50% of infected neonates early after birth (2, 9) has several advantages for conducting HIV-1 vaccine and treatment trials. These advantages include early end-point analysis in prophylactic vaccine trials, early diagnosis of HIV-1 infection, and early enrollment into clinical trials of infected infants who may receive potentially toxic drugs. Therefore, PCR could be diagnostically useful in multicenter HIV-1 experimental trials, such as those of the AIDS Clinical Trials Group (ACTG). Despite the many advantages of PCR-based detection assays, the extreme sensitivity of the assays may lead to sporadic false-positive results because of the carryover of
Corresponding author.
t Denis Henrard, Abbott Laboratories; F. B. Hollinger and H. J. Lin, Baylor College of Medicine; J. B. Jackson and B. Yen-Lieber-
man, Case Western Reserve University-Cleveland Clinic; J. Sever and T. Schutzbank, Children's Hospital of Washington; C. Michels, Dataworks; P. Reichelderfer and D. Livnat, Division of AIDS, National Institute of Allergy and Infectious Diseases; S. McDonough, Gen-Probe Inc.; S. Jones and C. Thornton, Maryland Medical Laboratories; P. Palumbo, New Jersey School of Medicine and Dentistry; S. Wolinsky and J. Drew, Northwestern University; B. Dragon, J. Spadoro, and A. Butcher, Roche Molecular Systems; J. Bremer, Rush-Presbyterian-St. Lukes Medical Center; M. Holodniy and Mark Winters, Stanford University; B. Poeisz, State University of New York-Syracuse; R. Pomerantz, Thomas Jefferson Medical College; R. Mitsuyasu, University of California-Los Angeles; S. Spector, University of California-San Diego; S. Rasheed, University of Southern California; and R. Coombs, University of Washington.
: F. B. Hollinger and H. J. Lin, Baylor College of Medicine; J. B. Jackson and B. Yen-Lieberman, Case Western Reserve UniversityCleveland Clinic; J. Sever and T. Schutzbank, Children's Hospital of Washington; S. Jones and C. Thornton, Maryland Medical Laboratories; P. Palumbo, New Jersey School of Medicine and Dentistry; J. Bremer, Rush-Presbyterian-St. Lukes Medical Center; M. Holodniy and Mark Winters, Stanford University; B. Poeisz, State University
of New York-Syracuse; S. Spector and K. Hsia, University of California-San Diego; S. Rasheed, University of Southern California; and R. Coombs and B. Paxton, University of Washington. 3123
3124
JACKSON ET AL.
minute amounts of previously amplified sequences (amplicons) to an uninfected sample. Additionally, false-negative results may occur because of suboptimal reaction conditions, incomplete denaturation of DNA, or errors in sample collection and/or processing. Initial attempts to ensure comparability and uniformity of data with nonstandardized HIV-1 DNA PCR assays across multiple sites resulted in variable sensitivity and specificity (1, 3, 12, 14). In view of the serious implications of falsepositive results, the ACTG temporarily abandoned its efforts to use HIV-1 PCR until the recent availability of commercial HIV-1 DNA PCR kits with standardized reagents and procedures. This report describes the quality assurance program that was then developed to certify and monitor in real time the HIV-1 DNA PCR performance of ACTG laboratories.
MATERIALS AND METHODS Participating laboratories. The participating laboratories consisted of 11 ACTG virology laboratories (Baylor College of Medicine, Houston, Tex.; Case Western Reserve University/Cleveland Clinic, Cleveland, Ohio; Children's Hospital of Washington, Washington, D.C.; Maryland Medical Laboratories, Baltimore, Md.; New Jersey School of Medicine and Dentistry, Newark; Rush-Presbyterian-St. Lukes Medical Center, Chicago, Ill.; Stanford University, Palo Alto, Calif.; State University of New York, Syracuse; University of California-San Diego, San Diego; University of Southern California, Los Angeles; and University of Washington, Seattle) performing one of two procedures with commercial kits for HIV-1 DNA PCR and detection. Two of these 11 laboratories used both commercial assays and, for statistical analysis, were considered four separate laboratories. All of the participating laboratories had prior PCR experience, but not necessarily with the use of these two commercial assays. Coded panels consisting of cell pellets were produced by Northwestern University ACTG Virology Laboratory (Chicago, Ill.) and distributed to participating laboratories by the Virology Reference Laboratory (VRL) at Baylor College of Medicine. Quality assurance procedures. (i) Certification panel. To be certified to perform qualitative HIV-1 DNA PCR testing for ACTG study protocols, a laboratory had to have demonstrated acceptable performance using one of two procedures with commercial kits. A coded panel of 30 cell pellets containing different HIV-1 proviral copy numbers and eight whole-blood samples was tested. (ii) Cell pellets. Each cell pellet consisted of 1.5 x 106 PBMC obtained from HIV-1-seronegative donors and spiked with a variable number of 8E5/LAV cells, such that there were 0, 2, 5, 10, 20, or 50 8E5/LAV cells per 125,000 PBMC. The 8E5/LAV cell line (provided by the AIDS and Human Retrovirus Repository, National Institutes of Health) contains one integrated HIV-1 proviral copy per cell (6). After extraction, an aliquot of cell lysate equivalent to 125,000 cells was amplified in duplicate. Detection was performed in singleton. If the results were discrepant (i.e., positive and negative), then a third amplification and detection were performed to resolve the discrepancy. Known cell pellet standards containing 0, 2, or 5 HIV-1 proviral copies were run in duplicate, the 10-, 20-, or 50-copy standards were run in singleton, and the reagent negative control (extraction reagent) was run in singleton. (iii) Whole-blood specimens. Each laboratory also tested eight coded samples containing 3 ml of whole blood in
J. CLIN. MICROBIOL.
citrate-phosphate-dextrose anticoagulant shipped at room temperature by the VRL. These whole-blood specimens were obtained from well-characterized HIV-1 antibody-positive and antibody-negative donors who have been repeatedly HIV-1 DNA PCR positive or negative in the past. Typically, the HIV-1 antibody-positive donor blood had a CD4 count of 67% of laboratories), the laboratory was "provisionally certified." That laboratory was required to perform the subsequent month's wholeblood series correctly to restore certification status. In the meantime, the laboratory was able to perform clinical testing for ACTG protocols. Failure to perform two whole-blood series correctly resulted in probation status, and the laboratory was not allowed to perform patient testing for ACTG protocols. The laboratory must then have gone through the initial certification procedure to be recertified. In addition, testing of coded cell pellets was performed in real time simultaneously with patient samples. These coded
VOL. 31, 1993
QUALITY ASSURANCE PROGRAM FOR HIV-1 DNA PCR ASSAYS
"QA" pellets consisted of 0 or 10 HIV-1 DNA proviral copies per 125,000 PBMC. One QA pellet was amplified in duplicate for every 10 patient pellets, a minimum of 3 coded pellets being tested per run. If the results were discrepant (i.e., positive and negative) for a QA pellet, then a third amplification and detection were performed to resolve the discrepancy. Results for the QA pellets were entered into an encrypted computer file, which immediately assessed the validity of the run in the laboratory. If a coded QA pellet was tested incorrectly (i.e., false-positive or false-negative), the run was invalidated. Failure to test the coded pellets correctly on two consecutive runs or in 3 of the last 10 consecutive runs resulted in probation status, and testing was not allowed to be performed on patient samples in ACTG protocols. The laboratory was then required to undergo the initial certification procedure successfully to be recertified. HIV-1 DNA PCR and detection procedures. The participating laboratories used one or both of two assays for the amplification and detection of HIV-1 DNA gag sequences. (i) Roche Amplicor assay. Nine laboratories used the Roche Amplicor kit, which contains all the reagents necessary to perform the sample preparation, amplification, and detection (8, 15). The kit also includes AmpErase (uracil-Nglycosylase) for the control of amplicon carryover contamination. In brief, coded cell pellets containing 1.5 x 106 PBMC and variable numbers of 8E5/LAV cells were incubated with 600 ,ul of extraction reagent at 60°C for 30 min and then at 98°C for 30 min. Cell pellets prepared from whole blood were incubated with 200 to 250 ,ul of extraction reagent at 60°C for 30 min and then at 98°C for 30 min. Fifty microliters of cell lysate was added to 50 ,ul of master mix, containing dATP, dCTP, dGTP, dUTP (instead of dTTP), AmpErase, Taq polymerase, and biotinylated SK 462/431 gag primers. Amplification was performed on a GeneAmp PCR System 9600 thermal cycler (Perkin-Elmer) by use of the following cycling parameters: 50°C for 2 min; 5 cycles of 95°C for 10 s, 55°C for 10 s, and 72°C for 10 s; and 30 cycles of 90°C for 10 s, 60°C for 10 s, and 72°C 10 s. The amplified biotinylated 142-bp product (100 ,ul) was denatured with 100 ,ul of denaturation solution for 10 min at room temperature. A 25-,ul sample of the denatured product was added to 100 ,ul of hybridization solution in a microwell coated with the capture probe (SK 102) for 1 h at 37°C. The microwells were washed, and 100 p,l of avidin-horseradish peroxidase conjugate was added to each well and incubated for 15 min at 37°C. After a final wash, a tetramethylbenzidine-H202 solution was added to each well and incubated for 10 min at 23 to 24°C. One hundred microliters of stop reagent was added to each well to stop the reaction. The A450 of each well was determined within 1 h with a microwell plate reader. A result was judged to be positive if the optical density of the sample tested was greater than 0.350. (ii) Perkin-Elmer and Gen-Probe assays. Four laboratories used the Perkin-Elmer HIV-1 core amplification kit (Norwalk, Conn.) and the Gen-Probe detection kit (San Diego, Calif.) to amplify a conserved 115-bp sequence of the HIV-1 gag region by use of primer pair SK 38/39. The Perkin-Elmer kit contains all of the reagents necessary to perform the amplification steps, and the Gen-Probe kit contains all of the reagents necessary to perform the detection procedures. Neither kit contains the reagents needed to prepare the cell pellet and extract the DNA; nor does either kit incorporate contamination control measures. In brief, coded cell pellets containing 1.5 x 106 PBMC and variable numbers of 8E5/ LAV cells were incubated with 600 ,ul of PCR lysis reagent
3125
(50 mM KCl, 10 mM Tris-HCl [pH 8.3], 2.5 mM MgCI2, 0.45% Nonidet P-40, 0.45% Tween 20, 100 to 200 ,ug of proteinase K per ml) at 60°C for 1 h with occasional mixing and then at 98 to 100°C for 10 min to inactivate proteinase K. Cell pellets containing 106 PBMC separated from whole blood were incubated with 250 ,ul of PCR lysis reagent at 60°C for 1 h with occasional mixing and then at 98 to 100°C for 10 min to inactivate proteinase K. Fifty microliters of cell lysate was added to 50 ,ul of master mix, containing 50 pmol of HIV-1 SK 38 and 50 pmol of SK 39 primer, 2.5 U of Taq polymerase, and 25 nmol each of dGTP, dATP, dCTP, and dTJ7P in a total volume of 100 ,ul of reaction buffer (50 mM KCl, 10 mM Tris-HCl [pH 8.3], 2.5 mM MgCl2). Amplification was performed either on a GeneAmp PCR system 9600 thermal cycler by use of the Roche Amplicor cycling parameters (5 cycles of 95°C for 10 s, 55°C for 10 s, and 72°C for 10 s and 30 cycles of 90°C for 10 s, 60°C for 10 s, and 72°C for 10 s) or on a Perkin-Elmer 480 thermal cycler by use of the following cycling parameters: 2 min at 95°C and then 35 cycles of 95°C for 60 s, 55°C for 30 s, and 72°C for 60 s. HIV-1 gag sequences were then detected with the Gen-Probe Accusearch HIV-1 gagl-gag2 chemiluminescent DNA probe and detection system. This assay uses a chemiluminescent acridinium ester (AE)-labeled probe to hybridize with HIV-1 gag sequences generated by the SK 38/39 primer pair (13, 15). After the addition of a base, hydrolysis of the AE group of the unhybridized probe occurs, and it is no longer chemiluminescent. However, the AE group of the hybridized probe is resistant to alkaline hydrolysis and generates measurable light units when exposed to hydrogen peroxide and sodium hydroxide in a luminometer. In brief, 20 ,ul of amplified product was added to a polypropylene assay tube (12 by 75 mm) containing 30 ,ul of PCR lysis reagent. The tubes were incubated at 95°C for 5 min to denature target sequences. The tubes were cooled for 1 min in an ice-water bath, and then 50 ,ul of the AE-labeled gag probe reagent was added, mixed, and incubated at 60°C for 10 to 14 min for hybridization to occur. Gen-Probe hydrolysis reagent (300 p1I) was then added to each tube, mixed, and incubated at 60°C for 6 to 7 min. The tubes were placed on ice for 1 to 20 min and then equilibrated to room temperature for 2 to 3 min. The tubes were read in a Leader luminometer after the addition of hydrogen peroxide and sodium hydroxide. The cutoff value for a positive result for the Gen-Probe assay was >10,000 relative light units. Analysis. The data were analyzed to determine the sensitivity and specificity of each laboratory and each commercial assay across all laboratories. Sensitivity was defined as the percentage of HIV-1-infected whole-blood specimens and 10-, 20-, and 50-copy pellets reported as positive. Specificity was defined as the percentage of uninfected whole-blood specimens and 0-copy pellets reported as negative. RESULTS Initial certification. In the first attempt at certification, 13 laboratories participated. Four laboratories performed the Perkin-Elmer and Gen-Probe assays, and nine laboratories performed the Roche Amplicor assay. Initially, four of nine laboratories using the Amplicor kit and four of four laboratories using the Perkin-Elmer and Gen-Probe assays met the certification criteria in testing the coded panel of 30 cell pellets and eight whole-blood samples (Table 1). Of the laboratories that failed to meet the certification criteria, two laboratories correctly tested the 30 cell pellets but failed to test the whole-blood samples correctly because of false-
3126
J. CLIN. MICROBIOL.
JACKSON ET AL.
TABLE 1. HIV-1 DNA PCR performance of ACITG laboratories
Laboratory 1 1 2 2 3 4
5 6 7 8 9 10 11
Kit* PG R PG R PG PG R R R R R R R
Present certification
statusb
C C
C C C C C C C C C P NC
No. of initial certification attempts
1 1 1 1 1 1 2 1 2 2 2 1
Totals
No. of mo with the following certification status:
c PC'C
No. of correctly tested whole-blood sample p
panels' since ~~~~~~~~~~certification
8
0
0
7/7
6
0
0
5/5
15 15 14 9
0 0 1 0 2 0 0 0 2
0 0 0 0 0 0 0 0 0
4/4 4/4 5/6 5/5 2/4 2/2 1/1 1/1 3/5
2
1
3/6
7
1
42/50
5 6 6 6 9 6
105
a PG, Perkin-Elmer and Gen-Probe; R, Roche Amplicor. b As of 1 May 1993. C, certified; P, on probation; NC, not certified. c PC, provisionally certified. d Each panel consists of eight patient whole-blood samples.
positive results. These two laboratories met the certification criteria in testing a different set of coded whole-blood samples correctly on the second attempt. Two additional laboratories tested the whole-blood samples correctly but failed to test all 30 cell pellets correctly because of 1 false-positive result among 20 negative pellets. These two laboratories met the certification criteria in correctly testing an additional 30 cell pellets on the second attempt. One laboratory failed to meet the certification criteria in testing both the 30 cell pellets and the whole-blood samples. This laboratory is not currently certified and was excluded from the analysis because of the reporting of false-negative results for all coded positive samples and the known positive standards. For at least two of the laboratories that failed the certification testing, a technical error and failure to follow instructions in the package insert were identified as the sources of problems. It should be noted that all of the laboratories performing the Perkin-Elmer and Gen-Probe assays had routinely been performing the assays for approximately 6 months, whereas several laboratories performing the Roche Amplicor assay had had only 1 to 2 months of experience with this assay prior to participation in this program.
The overall specificity in testing coded cell pellets and whole-blood samples for the 12 laboratories on their first attempt was 98.1% (257 of 262). The specificity varied between 83.3 and 100%, with 4 of 12 laboratories reporting false-positive results. The overall sensitivity was 100%. Ongoing quality assurance. Eight initially certified laboratories have been consistently correct in testing the coded whole-blood sample panels (Table 1). Four initially certified laboratories failed to correctly test at least one of the subsequent quarterly coded whole-blood panels. These four laboratories were placed on provisional certification. Three of four of these laboratories correctly tested the subsequent whole-blood panel the next month and were restored to certified status. The remaining laboratory failed to correctly test two consecutive whole-blood panels by repeatedly obtaining false-negative results. This laboratory was placed on probation and cannot perform HIV-1 DNA PCR assays for
ACTG protocols until it repeats and passes the initial certification testing. Overall, initially certified laboratories have maintained a status of certified 93% of the time, provisionally certified 6% of the time, and on probation 1% of the time (Table 1). The overall specificity and sensitivity for both assays in the 12 initially certified laboratories in testing 392 subsequent whole-blood samples were 94.8% (91 of 96) and 97.4% (289 of 296), respectively. The specificity varied between 75.0 and 100%, with 2 of 12 laboratories reporting false-positive results. There was no overall difference in specificity between laboratories which used the Perkin-Elmer and GenProbe assays and those which used AmpErase in the Roche assay (95.5 versus 94.2%, respectively). The sensitivity varied between 86.4 and 100%, with 3 of 12 laboratories reporting false-negative results. Eight of 12 laboratories reported no false-positive or false-negative results from the time of initial certification. Three of the certified ACTG laboratories, which have been testing infant samples for ACTG protocols, have also correctly tested all 20 coded real-time QA pellets in six separate PCR assays by using the encrypted file for immediate validation of the run. Assay sensitivity. On the basis of the algorithm, the overall sensitivities for the eight certified laboratories using the Roche assay in testing the most recent coded 30 cell pellet samples were 100% for the 10-, 20-, and 50-copy pellets, 92% for the 5-copy pellet, and 74% for the 2-copy pellet (Table 2). The overall sensitivities in detecting the various HIV-1 copy numbers per pellet for the four certified laboratories using the Perkin-Elmer and Gen-Probe assays were 100% for the 10-, 20-, and 50-copy pellets, 93% for the 5-copy pellet, and 56% for the 2-copy pellet (Table 2). Nine of 12 (75%) certified laboratories had at least one proficiency sample in the 30-cell-pellet panel which showed discrepant results and therefore needed a third amplification or amplification in duplicate of a second pellet for resolution. Twenty-six (7%) of 360 proficiency samples tested showed discrepant results on initial testing. Twenty (77%) of these 26 samples involved the 2- and 5-copy pellets, a result which
VOL. 31, 1993
QUALITY ASSURANCE PROGRAM FOR HIV-1 DNA PCR ASSAYS
3127
TABLE 2. Results for the 30-cell-pellet certification panel performed in laboratories (four using the Perkin-Elmer and Gen-Probe assays and eight using the Roche assay) which passed the certification testing Results (no. of samples) obtained with the following amplification: Assay(s)
Copy no.
No. tested
First two initial Positive
Discrepant
(-I-)
Perkin-Elmer and Gen-Probe
a
(-1+)
Negative eave
Positive Pstv
Negative Ngtie
Positive oiie
2 0 0
2 5 2
0
1
82 (100) 4 1 0 0 0
0 5 (56) 13 (93) 7 (100) 4 (100) 4 (100)
82 9 14 7 4 4 120
82 2 1 0 0 0
0 3 8 5 4 3
0 4 5 2 0 1 12
0 2 5 10 20 50
157 23 25 16 8 8 237
155 6 0 0 0 0
0 13 18 15 8 8
2 4 7 1 0 0 14
2 0 2 0
0 4 5 1
157 (100) 6 2 0 0 0
0 17 (74) 23 (92) 16(100) 8 (100) 8 (100)
0 2 5 10 20 50
239 32 39 23 12 12 357
237 8 1 0 0 0
0 16 26 20 12 11
2 8 12 3 0 1 26
2 2 2 0
0 6 10 3
0
1
239 (100) 10 3 0 0 0
0 22(69) 36 (92) 23 (100) 12(100) 12 (100)
Total
Totals (all assays)
(+1+)
0 2 5 10 20 50
Total
Roche
Third
Negative
No. (%) with the following algorithm result:
a
_, no third amplifications performed.
was to be expected. Three discrepant results involved the 10-copy pellets, one involved the 50-copy pellets, and two involved the 0-copy pellets (Table 2).
DISCUSSION This study is the first to report the development and implementation of a multilaboratory quality assurance program for HIV-1 DNA PCR that successfully incorporates sensitive and specific standardized commercial nonisotopic HIV-1 DNA PCR and detection assay kits. This quality assurance program also provides continuous proficiency monitoring in real time. Multicenter proficiency trials that have used nonstandardized HIV-1 DNA PCR assays have reported large variations in sensitivity and specificity among laboratories tested (1, 3, 12, 14), and none have reported the continuous use in real time of blinded proficiency samples. For example, Muul and Milman reported that 33% of ACTG laboratories using a uniform PCR protocol with common primer pairs and radioisotopic probes had severe problems with the carryover of HIV-1-positive DNA to coded negative samples and controls (12). Moreover, only 50% of the laboratories were able to detect 60 copies of HIV-1 DNA. Likewise, Defer et al. reported that falsepositive and false-negative results were observed in all seven laboratories testing two coded panels using their own PCR protocols (3). Even when the same primer pair was used, the concordance ranged from 40 to 100%. Sheppard et al. reported that only one of five laboratories, all with extensive PCR experience, correctly tested 204 coded patient specimens using their own PCR protocols (14). The overall false-positive rate was 1.8%, the false-negative rate was 0.8%, and the indeterminate rate was 1.9%. The sensitivity
and specificity for the five laboratories varied between 98.0 and 100% and between 90.5 and 100%, respectively. These latter results are very similar to those reported in this study, except that two-thirds of the laboratories in this study have shown 100% sensitivity and specificity. Much of the variability in performance among different laboratories has been attributed to the use of isotopic detection assays, different primer pairs, nonstandardized amplification reagents, different cycling parameters, inconsistent sample preparation, amplicon carryover, and different amounts of target DNA for testing. The use of commercial assays that provide standardized reagents and procedures for sample preparation, amplification, and detection of HIV-1 DNA sequences may have significantly reduced the variability in performance among the laboratories that participated in this study. Several points with regard to HIV-1 DNA testing by PCR have been underscored by this study. First, we believe that both whole-blood samples and cell pellets with relatively low known HIV-1 DNA copy numbers should be included in proficiency testing. Whole-blood samples provide a check for correct sample preparation. The use of uracil-N-glycosylase (AmpErase) (11) does not completely eliminate the possibility of cross-contamination, because the carryover of native HIV-1 DNA can also occur with improper sample processing techniques, resulting in false-positives (10). It will be interesting to see whether laboratories performing amplification without AmpErase will maintain specificity, as an increasing number of amplicons containing TTP will be generated in those laboratories over time. False-negatives due to faulty sample preparation are often attributable to the presence of erythrocytes. In contrast to whole-blood sam-
3128
JACKSON ET AL.
ples, cell pellets enable amplification procedures to be assessed for specificity and sensitivity without the possible confounding effects of whole-blood sample preparation. Second, the results from low-copy-number samples (Table 2) show the necessity of carrying out amplification in duplicate, with a third amplification to resolve any discrepancies. The fact that 75% of certified laboratories reported discrepant results for at least one coded pellet or whole-blood sample indicates that patient samples should be amplified in duplicate. Third, real-time proficiency testing for the validation of each run appears to be warranted in many clinical studies, because HIV-1 DNA test results can serve as a basis for study enrollment and primary end points of the study. Fourth, the testing of quantitative HIV-1 DNA standards reveals excellent and equivalent sensitivity between the Perkin-Elmer and Gen-Probe assays and the Roche assay. Moreover, the sensitivity of these assays approaches the theoretical limit of detection. Whether these assays will be able to detect HIV-1 DNA sequences in PBMC of infected individuals with high CD4 counts or in infected infants shortly after birth, in whom the viral load may be significantly smaller than in the proficiency samples tested, remains to be evaluated. In practice, more laboratories using the Roche Amplicor kit had sensitivity and specificity problems with initial certification and subsequent testing of whole-blood samples than had those using the Perkin-Elmer and Gen-Probe assays. These differences, however, appeared to be laboratory specific rather than kit specific, because the laboratories that had difficulties generally had less than 1 to 2 months of experience with the Roche assay, whereas the laboratories using the Perkin-Elmer and Gen-Probe assays had approximately 6 months of experience prior to performing the initial certification tests. In addition, the two laboratories which used both the Roche Amplicor assay and the Perkin-Elmer and Gen-Probe assays had 100% sensitivity and specificity with both methods. In any case, 11 of 13 laboratories are currently certified or provisionally certified, including 7 of 9 laboratories using the Roche Amplicor kit. In summary, the use of standardized commercial HIV-1 DNA PCR assays and an external quality assurance program has ensured that results from different laboratories are comparable. In addition, real-time proficiency monitoring has ensured that sensitivity and specificity problems are recognized immediately, before patient data are compromised unknowingly. It is important to emphasize, however, that false-positive results may still occur, despite the implementation of a rigorous quality assurance program. Therefore, a positive HIV-1 DNA PCR result should be confirmed by subsequent testing of a second patient specimen. This is especially true if PCR will be used to diagnose HIV-1 infections in the clinical setting or in determining a primary endpoint or inclusion in clinical protocols involving vaccines and/or potentially toxic drugs. ACKNOWLEDGMENTS This study was supported by Roche Molecular Systems, Branchburg, N.J.; Gen-Probe Inc., San Diego, Calif.; and grant AI-25879
J. CLIN. MICROBIOL.
and contract N01-AI-82517 from the Division of AIDS, NIAID, Bethesda, Md. REFERENCES 1. Busch, M. P., D. R. Henrard, I. K. Hewlett, W. F. Mehaffey, J. S. Epstein, J.-P. Allain, T.-H. Lee, J. W. Mosley, and the Transfusion Safety Study Group. 1992. Poor sensitivity, specificity, and reproducibility of detection of HIV-1 DNA in serum by polymerase chain reaction. J. Acquired Immune Defic. Syndr. 5:872-877. 2. Comeau, A. M., H.-W. Hsu, M. Schwerzler, G. Mushinsky, and G. F. Grady. 1992. Detection of HIV in specimens from newborn screening programs. N. Engl. J. Med. 326:1703. 3. Defer, C., H. Agut, A. Garbarg-Chenon, M. Moncany, F. Morinet, D. Vignon, M. Mariotti, and J. J. Lefrere. 1992. Multicentre quality control of polymerase chain reaction for detection of HIV DNA. AIDS 6:659-663. 4. De Rossi, A. D., L. Ometto, F. Mammano, C. Zanotto, C. Giaquinto, and L. Chieco-Bianchi. 1992. Vertical transmission of HIV-1: lack of detectable virus in peripheral blood cells of infected children at birth. AIDS 6:1117-1120. 5. European Collaborative Study Group. 1988. Mother-to-child transmission of HIV infection. Lancet ii:1039-1043. 6. Folks, T., D. Powell, M. Lightfoote, S. Koenig, A. S. Fauci, S. Benn, A. Rabson, D. Daugherty, H. E. Gendelman, M. D. Hoggan, S. Venkatesan, and M. A. Martin. 1986. Biological and biochemical characterization of a cloned Leu-3- cell surviving infection with the acquired immunodeficiency syndrome retrovirus. J. Exp. Med. 164:280-290. 7. Hollinger, F. B., J. W. Bremer, L. E. Meyers, J. W. M. Gold, L. McQuay, and the NIH/NIAID/DAIDS/ACTG Virology Laboratories. 1992. Standardization of sensitive human immunodeficiency virus coculture procedures and establishment of a multicenter quality assurance program for the AIDS Clinical Trials Group. J. Clin. Microbiol. 30:1787-1794. 8. Jackson, J. B., C. Ndugwa, F. Mmiro, P. Kataaha, L. Guay, E. A. Dragon, J. Goldfarb, and K. Olness. 1991. Non-isotopic polymerase chain reaction methods for the detection of HIV-1 in Ugandan mothers and infants. AIDS 5:1463-1467. 9. Krivine, A., G. Firtion, L. Cao, C. Francoual, R. Henrion, and P. Lebon. 1992. HIV replication during the first weeks of life. Lancet 339:1187-1189. 10. Kwok, S., and R. Higuchi. 1989. Avoiding false positives with PCR. Nature (London) 339:237-238. 11. Longo, M. C., M. S. Berninger, and J. L. Hartley. 1990. Use of uracil DNA glycosylase to control carry-over contamination in polymerase chain reaction. Gene 93:125-128. 12. Muul, L. M., and G. Milman. 1990. Polymerase chain reaction (PCR) standardization program. J. Cell. Biochem. 108:S14D. 13. Ou, C.-Y., S. H. McDonough, D. Cabanas, T. B. Ryder, M. Harper, J. Moore, and G. Schochetman. 1990. Rapid and quantitative detection of enzymatically amplified HIV-1 DNA using chemiluminescent oligonucleotide probes. AIDS Res. Hum. Retroviruses 6:1323-1329. 14. Sheppard, H. W., M. S. Ascher, M. P. Busch, P. R. Sohmer, M. Stanley, M. C. Luce, J. A. Chimera, R. Madej, G. C. Rodgers, C. Lynch, H. Khayam-Bashi, E. L. Murphy, B. Eble, W. Z. Bradford, R. A. Royce, and W. Winklestein, Jr. 1991. A multicenter proficiency trial of gene amplification (PCR) for the detection of HIV-1. J. Acquired Immune Defic. Syndr. 4:277283. 15. Whetsell, A. J., J. B. Drew, G. Milman, R. Hoff, E. A. Dragon, K. Adler, J. Hui, P. Otto, P. Gupta, H. Farzadegan, and S. M. Wolinsky. 1992. Comparison of three nonradioisotopic polymerase chain reaction-based methods for detection of human immunodeficiency virus type 1. J. Clin. Microbiol. 30:845-853.
