Usefulness of an Artificial Neural Network for ...

2 downloads 0 Views 2MB Size Report
Kunio Doi3. Received January 25, 2001; accepted after revision ..... Watanabe H, Egashira K, Nakamura K, et al. De- tailed helical CT of pulmonary nodule; ...
Downloaded from www.ajronline.org by 54.210.20.124 on 09/13/15 from IP address 54.210.20.124. Copyright ARRS. For personal use only; all rights reserved

Usefulness of an Artificial Neural Network for Differentiating Benign from Malignant Pulmonary Nodules on High-Resolution CT: Evaluation with Receiver Operating Characteristic Analysis Yuichi Matsuki 1 Katsumi Nakamura 1 Hideyuki Watanabe 1 Takatoshi Aoki 1 Hajime Nakata 1 Shigehiko Katsuragawa 2 Kunio Doi 3

OBJECTIVE. The purpose of our study was to use an artificial neural network to differentiate benign from malignant pulmonary nodules on high-resolution CT findings and to evaluate the effect of artificial neural network output on the performance of radiologists using receiver operating characteristic analysis. MATERIALS AND METHODS. We selected 155 cases with pulmonary nodules less than 3 cm (99 malignant nodules and 56 benign nodules). An artificial neural network was used to distinguish benign from malignant nodules on the basis of seven clinical parameters and 16 radiologic findings that were extracted by attending radiologists using subjective rating scales. In the observer test, 12 radiologists (four attending radiologists, four radiology fellows, and four radiology residents) were presented with high-resolution CT images, first without and then with the artificial neural network output. Observer performance was evaluated by means of receiver operating characteristic analysis using a continuous rating scale. RESULTS. The artificial neural network showed a high performance in differentiating benign from malignant pulmonary nodules (Az = 0.951). The average Az value for all radiologists increased by a statistically significant level, from 0.831 to 0.959, with the use of the artificial neural network output. CONCLUSION. Our computerized scheme using the artificial neural network can improve the diagnostic accuracy of radiologists who are differentiating benign from malignant pulmonary nodules on high-resolution CT.

A

Received January 25, 2001; accepted after revision August 30, 2001. Presented at the annual meeting of the American Roentgen Ray Society, Washington, DC, May 2000. 1 Department of Radiology, University of Occupational and Environmental Health School of Medicine, Iseigaoka 1-1, Yahatanishi-ku, Kitakyushu-shi, Japan 807-8555. Address correspondence to Y. Matsuki. 2

Nippon Bunri University General Research Center, Nippon Bunri University, Ichiki 1727, Oita-shi, Japan 870-0397. 3

Kurt Rossmann Laboratories for Radiologic Image Research, Department of Radiology, The University of Chicago, 5841 S. Maryland Ave., Chicago, IL 60637. AJR 2002;178:657–663 0361–803X/02/1783–657 © American Roentgen Ray Society

AJR:178, March 2002

n evaluation of a patient with a solitary pulmonary nodule is a common diagnostic problem in daily radiology practice because the treatment plan differs according to the possibility of malignancy [1]. A variety of radiologic procedures have been used for this purpose and currently high-resolution CT is considered one of the most important noninvasive diagnostic techniques. Although its clinical usefulness has been established [2–6], differentiating benign from malignant pulmonary nodules, even with the use of high-resolution CT, remains a difficult task for radiologists because there is an overlapping spectrum of radiographic appearances and clinical parameters among benign and malignant nodules. Artificial neural networks have been studied intensively in the field of computer science in recent years and have been shown to be a powerful tool for a variety of data-classification and pattern-recognition tasks. However, the usefulness of artificial neural networks in diagnostic

radiology was reported only in chest radiography and mammography in the diagnosis of pulmonary nodules, interstitial lung disease, pediatric lung lesions, and breast nodules [7– 13]. The purpose of this study was to apply an artificial neural network in the differentiation of benign from malignant pulmonary nodules on high-resolution CT images and to evaluate the effect of artificial neural network output on the performance of radiologists using receiver operating characteristic (ROC) analysis. Materials and Methods Case Selection and Classification The scans of 155 sequential patients with solitary pulmonary nodules less than 3 cm who underwent high-resolution CT from June 1992 to January 1998 were used. The patients were 94 men and 61 women (age range, 19–85 years; mean, 62 years). There were 99 malignant nodules, all of which were diagnosed pathologically (73 adenocarcinoma, 14 squamous cell carcinoma, four large cell carcinoma, three adenosquamous carcinoma, two small cell car-

657

Downloaded from www.ajronline.org by 54.210.20.124 on 09/13/15 from IP address 54.210.20.124. Copyright ARRS. For personal use only; all rights reserved

Matsuki et al. cinoma, one unclassified lung cancer, one metastatic lung tumor, and one carcinoid tumor). There were 56 benign nodules, 27 of which were diagnosed pathologically (eight pulmonary hamartoma, eight pulmonary tuberculosis, two atypical mycobacteriosis, two organizing pneumonia, two anthracotic nodules, one sclerosing hemangioma, one capillary hemangioma, one inflammatory pseudotumor, one amyloidosis, and one necrotizing tissue). Twentynine nodules were diagnosed as benign by confirming their interval resolution, decrease in size, or no growth over a 2-year period of observation. Nodules showing benign calcification patterns (central, laminated, diffuse, popcorn) or obvious fat were excluded from this study. CT was performed with a TCT-900S Helix (Toshiba, Tokyo, Japan) or Somatom Plus 4 (Siemens Medical Solutions, Erlangen, Germany). Routine scanning of the whole lung (120 kVp, 150 mA or 140 kVp, 170 mA) was first performed using the helical mode with a table speed of 10 mm/sec and a 10- or 5-mm collimation. Images were printed as fixed settings (lung window center, –650 to –700 H; lung window width, 1500–1600 H; mediastinum window center, 35–50 H; mediastinum window width, 300–360 H). Additional high-resolution CT with 2.0-mm collimation (120 kVp, 250 mA or 140 kVp, 145 mA and 1.0- or 0.75-sec scanning time) covering the tumor was performed in all patients. High-resolution CT images were reconstructed with a high-spatial-frequency algorithm and printed at fixed settings (lung window center, –650 or –700 H; lung window width, 1500 H; mediastinum window

center, 35 or 50 H; mediastinum window width, 300 H). All scans were obtained with the patients in the supine position and at end inspiration. Artificial Neural Network Scheme We used a three-layer, feed-forward, artificial neural network with a back-propagation algorithm that was developed at The University of Chicago. We designed the artificial neural network with 23 input units for seven clinical parameters and 16 radiologic findings and one output unit corresponding to the likelihood of malignancy. The clinical parameters included the patient’s age, sex, history of smoking, underlying malignancy, history of familial malignancy, weight loss, and severity of symptoms. Sixteen radiologic findings were classified into three categories: features related to nodules (size, shape, concavity, border definition, irregular undulation, spiculation, air space, air bronchogram, ground-glass opacity, and calcification); secondary abnormalities (pleural indentation, satellite lesions, arterial involvement, and venous involvement); and additional abnormalities (emphysematous changes and lymphadenopathy). Highresolution CT images were used for all radiologic findings, except for lymphadenopathy for which we used conventional CT images. Subjective ratings for the 16 radiologic findings were provided independently by three attending radiologists with more than 10 years’ experience in chest radiology who were unaware of the final diagnosis. Three attending radiologists used a score sheet with a scale from 0 to l0. The observers used a ruler to measure the maximum size of the nodule. The

TABLE 1

One Attending Radiologist’s Subjective Ratings of Lung Cancer (Fig. 1) and Organizing Pneumonia (Fig. 2)

High-Resolution CT

Subjective Ratings (0–10)

Feature

Fig. 1

Fig. 2

Size (mm) Shape Border definition Irregular undulation Spiculation Concavity Airspace Air bronchogram Ground-glass opacity Calcification Pleural indentation Satellite lesion Arterial involvement Venous involvement Emphysematous change Lymphadenopathya

22 8 0 4 10 9 3 5 2 0 10 0 8 4 0 0

18 9 0 0 0 10 0 0 0 0 0 0 2 2 6 0

aUsing conventional CT images.

shape ranged from strand (score, 0) to round (score, 10). Concavity was defined as the concave or the straight line of the border of the nodule measuring Fig. 1.—54-year-old woman with lung cancer. High-resolution CT scan shows nodule with spiculation in whole margin and obvious pleural indentation.

Fig. 2.—58-year-old man with organizing pneumonia. High-resolution CT scan shows nodule without spiculation or obvious pleural indentation.

1 658

2 AJR:178, March 2002

Downloaded from www.ajronline.org by 54.210.20.124 on 09/13/15 from IP address 54.210.20.124. Copyright ARRS. For personal use only; all rights reserved

High-Resolution CT of Pulmonary Nodules coursing along the bronchi were considered arteries and others, veins. Arterial (or venous) involvement was defined as the vessel running into the nodule; it ranged from no involvement (score, 0) to the involvements of more than two vessels (score, 10). Emphysematous change was defined as enlargement of air space without obvious fibrosis surrounding the nodule, ranging from no (score, 0) to marked emphysematous change with complete loss of normal lung parenchyma (score, 10). Conventional CT images were used for the evaluation of lymphadenopathy, ranging from no lymphadenopathy (score, 0) to lymphadenopathies consisting of more than two lymph nodes enlarged more than l cm in diameter (score, 10). Reference samples with scores of 0, 5, and 10 were shown to each attending radiologist for all these radiologic findings except size. Table 1 shows one example of the subjective ratings of one attending radiologist for lung cancer (Fig. 1) and organizing pneumonia (Fig. 2). Input data obtained from clinical parameters and subjective ratings for radiologic findings, which were rated by attending radiologists having experience of more than 10 years, were normalized to the range from 0 to 1. A round-robin method (leave-one-out method) was applied for training and testing to evaluate the performance of the artificial neural network [12]. In this method, all but one case in a database is used to train the artificial neural network. The single case that is left out is then used to test the artificial neural network. This procedure is repeated so that each case in the database is used once as a test case. Out-

Observer Test In the observer test, 50 cases (25 malignant, 25 benign) were selected in which the performance (Az = 0.951) was practically equal to that obtained with all cases in the database. Twelve radiologists, including four attending radiologists with 10 years’ or more experience, four radiology fellows with 4–6 years’ experience, and four radiology residents with less than 3 years’ experience participated in the observer test. We used a sequential test for ROC studies to evaluate the performance of radiologists [14]. First, observers were shown high-resolution CT and conventional CT images with seven clinical parameters for the initial rating of the confidence level of malignancy (without artificial neural network output). Subsequently, artificial neural network output was presented to the same observer, who rated the confidence level a second time (with artificial neural network output). The observer could either change the initial ratings or leave them unchanged. The observer’s confidence level about the likelihood of malignancy was represented using an analog continuous-rating scale with a line-checking method. For the initial ratings, the observers used a black ballpoint pen to mark their confidence levels along an 8-cm line. Ratings of “definitely benign” and “definitely malignant” were marked above the left and the right ends of the line, respectively. If the second ratings were different from the initial ones, the observers used a red ballpoint pen to mark their

Fig. 3.—Bar chart shows distribution of artificial neural network output that indicates likelihood of malignancy for malignant and benign nodules. Note that number of cases indicated equal to or more than 80% of likelihood of malignancy is 70. Sixty-seven cases (96%) were malignant, and only three cases (4%) were benign. Black bar = malignant, striped bar = benign.

Fig. 4.—Graph shows receiver operating characteristic curve of artificial neural network for differentiating benign from malignant nodules. Note that Az value of artificial neural network was 0.951, indicating high performance.

more than 4 mm in length. Concavity ranged from no such line (score, 10) to more than three lines (score, 0) with one line (score, 5) in the middle. Border definition ranged from well defined (score, 0) to poorly defined (score, 10). Irregular undulation was defined as the unevenness of the margin, which ranged from smooth (score, 0) to irregular in the whole margin (score, 10). Spiculation ranged from nonspiculated (score, 0) to spiculated in the whole margin (score, 10). We defined the focal air attenuation of the nodule as air space. Air space was assessed as for its ratio within the nodule, ranging from 0% (score, 0) to greater than 75% (score, 10). Cavitation was included in air space, and no attempt was made to separate it from air space. Air bronchogram was defined as a tubelike or branched air structure within the nodule, ranging from obviously having no structure (score, 0) to having more than two such structures (score, 10). Ground-glass opacity was evaluated for its ratio within the nodule, ranging from 0% (score, 0) to 100% (score, 10). Calcification was defined as an area of high attenuation observed on the mediastinal window setting and was assessed for its ratio within the nodule, ranging from 0% (score, 0) to greater than 50% (score, 10). The “edge-enhancement” artifact was ignored. Pleural indentation ranged from having no (score, 0) to obviously having more than two pleural indentations (score, 10). Satellite lesion was defined as a separate small discrete nodule observed within 5 mm of the dominant nodule. It ranged from having no lesion (score, 0) to obviously having more than two satellite lesions (score, 10). Some pulmonary vessels

AJR:178, March 2002

put values ranging from 0 to 1 indicated the likelihood of malignancy in each case (Fig. 3).

659

Matsuki et al.

Downloaded from www.ajronline.org by 54.210.20.124 on 09/13/15 from IP address 54.210.20.124. Copyright ARRS. For personal use only; all rights reserved

confidence levels along the same line. For data analysis, the confidence level was scored with a maximum of 100 units by measuring the distance from the left end of the line to the marked point. A total of 10 cases that were not used for the test were selected to train all observers. Data Analysis The diagnostic performance of radiologists with and without the artificial neural network output was evaluated using ROC analysis. Binormal ROC curves for diagnosing benign from malignant nodules were estimated using the LABROC 5 algorithm (available through the Internet from Metz, CE, The

University of Chicago, Chicago, IL). LABROC 5 was used to obtain maximum-likelihood estimates of binormal ROC curves from the continuous ordinal-scale rating data. Az values representing the area under the ROC curve were calculated. The statistical significance of differences between ROC curves for each reading condition was determined by applying a two-tailed t test for paired data to the reader-specific Az values. We also compared the diagnostic performance of attending radiologists, radiology fellows, and residents using a two-tailed test. To represent the overall performance for each group of observers, average ROC curves were generated for the four attending radiologists, the four radiology

fellows, the four radiology residents, and all radiologists by averaging the two binomial parameters of their individual ROC curves [15–17]. Another indication of performance was the number of correctly diagnosed cases for which the observer’s confidence level changed because of the artificial neural network output. We assumed that a clinically relevant change in the confidence rating occurred only when the difference calculated in this way was greater than 30 units on the confidence rating scale [14]. The difference between the average number of cases affected beneficially and those affected detrimentally using the artificial neural network output was analyzed using a two-tailed test.

Fig. 5.—Graph shows average receiver operating characteristic curves for differentiating benign from malignant nodules without and with artificial neural network (ANN) output by attending radiologists. Note that observer performance with ANN output was significantly improved. Solid line = with ANN (Az = 0.985), dashed line = without ANN (Az = 0.933).

Fig. 6.—Graph shows average receiver operating characteristic curves for differentiating benign from malignant nodules without and with artificial neural network (ANN) output by radiology fellows. Note that observer performance with ANN output was significantly improved. Solid line = with ANN, (Az = 0.932), dashed line = without ANN (Az = 0.821).

Fig. 7.—Graph shows average receiver operating characteristic curves for differentiating benign from malignant nodules without and with artificial neural network (ANN) output by radiology residents. Note that observer performance with ANN output was significantly improved. Solid line = with ANN (Az = 0.961), dashed line = without ANN (Az = 0.759).

Fig. 8.—Graph shows comparison of average receiver operating characteristic (ROC) curves for all observers without and with artificial neural network (ANN) output and ROC curves for ANN output alone. Note that observer performance with ANN output was significantly higher than that without ANN or than that with ANN alone. Solid line = with ANN (Az = 0.959), dotted line = ANN alone (Az = 0.951), dashed line = without ANN (Az = 0.831).

660

AJR:178, March 2002

High-Resolution CT of Pulmonary Nodules

Downloaded from www.ajronline.org by 54.210.20.124 on 09/13/15 from IP address 54.210.20.124. Copyright ARRS. For personal use only; all rights reserved

Results

The Az value of the artificial neural network alone was 0.951, showing a high performance (Fig. 4). We used the artificial neural network with seven input units for seven clinical parameters and 16 input units for 16 radiologic findings, separately. The Az value of the artificial neural network with seven clinical parameters alone and with 16 radiologic findings alone was 0.79 and 0.91, respectively. The average of the coefficients of correlation for 16 radiologic ratings was 0.76, which indicates a strong correlation among three observers ( p < 0.001). In particular, the coefficient of correlation for the size (0.92) and emphysematous change (0.91) was high. The coefficient of correlation for venous involvement (0.53) was the lowest. The performance by each group of observers and all combined observers is illustrated by the average ROC curves in Figures 5–8. The diagnostic performance for all observers improved when high-resolution CT images and clinical parameters were shown in conjunction with the artificial neural network output. The diagnostic performance of all radiologists improved when we used the artificial neural network output irrespective of the radiologist’s experience. The average Az value for radiology residents was substantially improved using the artificial neural network output. The average Az value for radiology residents using the artificial neural network (Az = 0.961) was higher than that for the artificial neural network alone (Az = 0.951) or than that for attending radiologists without the artificial neural network (Az = 0.933). The average Az value for all radiologists increased to a statistically significant level from 0.831 without the artificial neural network output to 0.959 with the artificial neural network output ( p < 0.001). Table 2 shows the Az values without and with the artificial neural network output for each radiologist. The performance improved for all radiologists when the artificial neural network output was used ( p < 0.001). The number of cases affected either beneficially or detrimentally by the artificial neural network output on benign and malignant nodules are shown in Figures 9 and 10. The artificial neural network output affected the observers’ confidence in 59 cases. The confidence level was affected beneficially for 25 malignant and 29 benign nodules. Only four malignant and one benign nodule were detrimentally affected. The number of cases affected beneficially were significantly higher than the number of cases affected detrimentally for both benign and malignant nodules (p < 0.001).

AJR:178, March 2002

Fig. 9.—Histogram shows number of cases affected (>30) due to artificial neural network output on benign nodules. Note that number of cases affected beneficially was significantly higher than number of cases affected detrimentally for benign nodules. Observers A–D are attending radiologists, observers E–H are radiology fellows, and observers I–K and L are radiology residents.

Fig. 10.—Histogram shows number of cases affected (>30) due to artificial neural network output on malignant nodules. Note that number of cases affected beneficially was significantly higher than number of cases affected detrimentally for malignant nodules. Observers A–D are attending radiologists, observers E–H are radiology fellows, and observers I–L are radiology residents.

Discussion

Our currently constructed computer-aided diagnostic scheme using an artificial neural network showed a high performance (Az = 0.951). This Az value was higher than the average Az value of attending radiologists without the artificial neural network output. Similar results were reported in studies using artificial neural networks in the differential diagnosis of breast can-

cer [10], interstitial lung disease [12], and solitary pulmonary nodules on chest radiography [18]. The results obtained in our study maybe explained by the fact that radiologists do not effectively use all the image features on high-resolution CT in their differential diagnosis of solitary pulmonary nodules. Radiologists may overestimate their knowledge and experience and tend to depend on a limited number of con-

661

Matsuki et al.

TABLE 2

Downloaded from www.ajronline.org by 54.210.20.124 on 09/13/15 from IP address 54.210.20.124. Copyright ARRS. For personal use only; all rights reserved

Observer Attending radiologist A B C D Mean Radiology fellows E F G H Mean Radiology residents I J K L Mean Overall mean

Comparison of Az Values Without and With ANN Output Without ANN

With ANN

0.880 0.965 0.946 0.941 0.933

0.973 0.996 0.991 0.978 0.984

< 0.001

0.905 0.865 0.806 0.707 0.821

0.970 0.957 0.918 0.882 0.932

< 0.001

0.805 0.763 0.776 0.693 0.759

0.975 0.967 0.940 0.966 0.961

< 0.001

0.831

0.959

< 0.001

pa

Note.—ANN = artificial neural network. aTwo-tailed t test for paired data.

spicuous features in their decision making. In addition, they may not consider all the features systematically. On the other hand, the artificial neural network is consistently and comprehensively affected by all the data. Nakamura et al. [18] used the artificial neural network with 10 input units for two clinical parameters and eight radiologic findings using chest radiography. The Az value (Az = 0.951) of our current computer-aided diagnostic scheme using high-resolution CT is higher than the Az value (Az = 0.854) of their scheme using chest radiography. A statistical multiple object detection and location system (S-MODELS) neural network technique to differentiate benign from malignant pulmonary nodules on CT findings was also reported [7]. This SMODELS showed the potential to reduce the number of biopsies without missing malignant nodules. However, unlike our current method with the artificial neural network, the weights of the S-MODELS are fixed, and there is no iterative learning process. Gurney and Swensen [6] used five radiographic (chest radiographs and CT) and two clinical characteristics and compared the per-

662

formances between the artificial neural network and the Bayesian method. They reported that the Az value of the Bayesian system (Az = 0.894) was significantly higher than that of the neural network (Az = 0.871) and that the neural networks offered no advantage over the Bayesian system in the prediction of probability of malignancy in solitary pulmonary nodules. Our scheme using the neural network achieved a higher performance (Az = 0.951). At present, we do not know the reasons for the difference between our results and the previous report because different cases were used in each study. It is difficult to examine the cause of this difference. The artificial neural network has a unique ability to learn specific patterns between input and output data if it is repeatedly trained with examples. However, this ability strongly depends on the quality of the input data. The quality of input data using the subjective ratings for the artificial neural network depends on the ability of radiologists. If the input data are randomly selected and have no correlation with the output data, the artificial neural network cannot learn any specific patterns. It should be noted that important input data in the present study were given by the subjective ratings for 16 radiologic findings on all nodules by three attending radiologists who had over 10 years’ experience. To further develop this diagnostic system for practical use, we need to construct a computer-aided diagnostic scheme with ratings by radiology residents who are inexperienced and then to evaluate its diagnostic performance. We used 16 radiologic and seven clinical findings, which are, at present, considered useful to differentiate benign from malignant solitary pulmonary nodules [2, 5]. If other useful clinical data such as tumor markers were available as input data, a better performance with the artificial neural network could be expected. However, in this study, we used a smaller number of essential features in our attempt to develop an artificial neural network scheme for use in practical clinical settings. Although the present 155 cases were limited in number and the combination of all input data used in the present study would not be necessarily generally applicable, we attempted to construct a computer-aided diagnostic scheme consisting of 23 input data that were proved useful. In the future, it may be necessary to find the combination of input data that provide the best performance, using a much larger number of cases. In the observer tests, our computer-aided diagnostic scheme using an artificial neural

network improved the diagnostic accuracy of radiologists in terms of differentiating benign from malignant pulmonary nodules on highresolution CT. The diagnostic performance by all the radiologists was significantly ( p < 0.001) improved with the artificial neural network output. The artificial neural network output affected the observer confidence in 59 cases in terms of both beneficial and detrimental effects. The number of cases affected beneficially was significantly larger than those affected detrimentally for both benign and malignant nodules (p < 0.001). In our study, the artificial neural network output caused little detrimental effect (0.17%) on observers’ confidence levels, especially in the diagnosis of benign nodules. To investigate why the artificial neural network had little detrimental effect (0.17%), we analyzed cases in which the artificial neural network produced a “wrong output.” Most radiologists, even the inexperienced radiology residents, were not confused by the artificial neural network output in such cases. The results may arise from the difference between the radiologists and the artificial neural network in the process of differentiating benign from malignant nodules, although input data for the artificial neural network were subjectively extracted by radiologists. This finding also supports the concept of using artificial neural networks as a second opinion to complement the performance of radiologists. In conclusion, our computer-aided diagnostic scheme using an artificial neural network showed a high performance and improved the diagnostic accuracy of radiologists in differentiating benign from malignant pulmonary nodules on high resolution CT.

References 1. Webb RW. Radiologic evaluation of the solitary pulmonary nodule. AJR 1990;154:701–708 2. Zwirewich CV, Vedal S, Miffler RR. Solitary pulmonary nodule: high-resolution CT and radiologic correlation. Radiology 1991;179:469–476 3. Kuriyama K, Tateishi F, Doi O, et al. CT-pathologic correlation in small peripheral lung cancers. AJR 1987;149:1139–1143 4. Siegelman SS, Khouri NF, Leo FP, Fishman EK, Braverman RM, Zerhouni EA. Solitary pulmonary nodules: CT assessment. Radiology 1986; 160:319–327 5. Watanabe H, Egashira K, Nakamura K, et al. Detailed helical CT of pulmonary nodule; differentiation between benignancy and malignancy, and CT-bronchoscopic correlation [in Japanese]. Rinsho Hoshasen 1997;42:11–19

AJR:178, March 2002

Downloaded from www.ajronline.org by 54.210.20.124 on 09/13/15 from IP address 54.210.20.124. Copyright ARRS. For personal use only; all rights reserved

High-Resolution CT of Pulmonary Nodules 6. Gurney M, Swensen SJ. Solitary pulmonary nodules: determining the likelihood of malignancy with neural network analysis. Radiology 1995; 196:823–829 7. Henschke CI, Yankelevitz DF, Mateescu I, et al. Neural networks for the analysis of small pulmonary nodules. Clin Imaging 1997;21:390–399 8. Asada N, Doi K, MacMahon H, et al. Potential usefulness of an artificial neural network for differential diagnosis of interstitial lung disease: pilot study. Radiology 1990;177:857–860 9. Ashizawa K, Ishida T, MacMahon H, et al. Artificial neural networks in chest radiography: application to the differential diagnosis of interstitial lung disease. Acad Radiol 1999;6:2–9 I0. Ashizawa K, MacMahon H, Ishida T, et al. Effect of an artificial neural network on radiologist’ per-

11.

12.

13.

14.

formance in the differential diagnosis of interstitial lung disease using chest radiographs. AJR 1999;172:1311–1315 Gross GW, Boone JM, Greco-Hunt V, et al. Neural networks in radiographic diagnosis. II. Interpretation of neonatal chest radiographs. Invest Radiol 1990;25:1017–1023 Wu Y, Giger ML, Doi K, et al. Artificial neural networks in mammograms: application to decision making in the diagnosis of breast cancer. Radiology 1993;187:81–87 Piraino DW, Arnartur SC, Richmond BJ, et al. Application of an artificial neural network in radiographic diagnosis. J Digit Imaging 1991;4:226–232 Kobayashi T, Xu XW, MacMahon H, et al. Effect of a computer-aided diagnosis scheme on radiolo-

15. 16.

17.

18.

gists’ performance in detection of lung nodules on radiographs. Radiology 1996;199:843–848 Metz CE. ROC methodology in radiographic imaging. Invest Radiol 1986;21:720–733 Metz CE, Herman BA, Shen JH. Maximum-likelihood estimation of receiver operating characteristic (ROC) curves from continuously-distributed data. Stat Med l998;17:1033–1053 Metz CE. Some practical issues of experimental design and data analysis in radiological ROC studies. Invest Radiol 1989;24:234–245 Nakamura K, Yoshida H, Engelmann P, et al. Computerized analysis of the likelihood of malignancy in solitary pulmonary nodules with use of artificial neural networks. Radiology 2000; 214:823–830

ARRS 2002 Annual Meeting Categorical Course focuses on body CT. Nationally recognized experts will provide information on radiation safety issues and recent advances in multidetector CT applicable to new CT techniques. The Annual Meeting will be held April 28–May 3 in Atlanta. Log on to www.arrs.org for further information.

AJR:178, March 2002

663