In this paper, we propose a new recognition method of lung nodules from X-ray CT images using 3D Markov ran- dom field(MRF) models. Pathological shadow ...
Recognition of Lung Nodules from X-ray CT Images Using 3D Markov Random Field Models Hotaka Takizawa, Shinji Yamamoto Toyohashi University of Technology 1-1 Hibarigaoka Tenpaku-cho Toyohashi-shi 441-8580 Aichi Japan {takizawa,yamamoto}@parl.tutkie.tut.ac.jp Tohru Matsumoto, Yukio Tateno, Takeshi Iinuma National Institute of Radiological Science 4-9-1 Inage-ku Chiba-shi 263-0024 Chiba Japan
Abstract In this paper, we propose a new recognition method of lung nodules from X-ray CT images using 3D Markov random field(MRF) models. Pathological shadow candidates are detected by a mathematical morphology filter, and volume of interest(VOI) areas which include the shadow candidates are extracted. The probabilities of the hypotheses that the VOI areas come from nodules(which are candidates of cancers) and blood vessels are calculated using nodule and blood vessel models evaluating the relations between these object models by 3D MRF models. If the probabilities for the nodule models are higher, the shadow candidates are determined to be abnormal. By applying this new recognition method to actual 38 CT images, good results has been acquired.
Mitsuomi Matsumoto Tokyo Metropolitan Univ. of Health Sciences 7-2-10 Arakawa-ku 116-8551 Tokyo Japan
however, normal shadows whose characteristics are similar to those of the abnormal ones are also detected as false positives. We have also developed an image filter called Quoit filter[6][3] which is a kind of mathematical morphology filter. This filter can detect pathological shadows with the sensitivity over 95[%], but it also detects a lot of false positives yet. For example, if a shadow candidate is too small as shown in Fig.1(a), it cannot be determined to be a nodule or not. In this case, we pay attentions not only to the shadow candidate but also the shadows around it. If ridge shadows related to the shadow candidate are newly found as shown in Fig.1(b), it can be determined to be a blood vessel(normal).
Is the candidate region
Related regions are informative.
1. Introduction We proposed a lung cancer screening system by CT for the mass screening[7]. By using this system, small and opaque nodules which are difficult to be found out in simple chest X-ray photographs become to be detected, but the system has one problem that the number of the images to be diagnosed by a doctor increases to about 30 slices per patient from 1 X-ray film. To overcome such a problem, we have to develop an algorithm which can reduce the number of the images by recognizing pathological shadow candidates in CT images automatically. There are several works on recognition of pathological shadow candidates[2][1][4] . They utilize image filters which respond selectively only to isolated shadows to detect the pathological shadow candidates. In these methods,
a nodule or a blood vessel ? (a)
It is a blood vessel ! (b)
Figure 1. Is the candidate shadow a nodule or not ?
In this paper, we propose a new recognition method of abnormalities of shadow candidates evaluating the relations between the shadow candidates and the shadows around them by using 3D Markov random field (MRF) models with geometrical object models such as nodules and blood vessels. Fig.2 shows the recognition process described in this pa-
1051-4651/02 $17.00 (c) 2002 IEEE
A pathological shadow candidate
...
A nodule Blood vessel branch models model A divided VOI
... Curved blood vessel models
...
... Cells
... Object model combinations
... Artificial CT images By comparing with the VOI area, the most possible object model combination is searched for.
Figure 2. Recognition process.
model using computer graphics techniques in the same way as [5], and a Sum of the Squared Differences(SSD) of the values of the corresponding pixels between the artificial CT image and the rectangle region of the VOI area is calculated. The optimal parameters (the position and the radius) minimizing the SSD value are searched for. The sphere model which has such optimal parameters is selected as the promising nodule model. [2] Curved blood vessels: We represent curved parts of blood vessels using two connected cylinder models. In order to generate a curved blood vessel model, we prepare two cylinder models, and ones of the end points of these two cylinder models are fixed at the center of a cell. By changing the directions of these cylinder models by 45[deg], 26 × 25 curved blood vessel models are generated. From each model, an artificial CT image is generated, a SSD value is calculated, and the optimal parameters which minimize the SSD value are searched for. The curved blood vessel models whose SSD values are less than a threshold are selected as the promising curved blood vessel models. [3] Blood vessel branches: 26 × 25 C2 blood vessel branch models are generated and the promising object models are selected in the same way as curved blood vessel models.
2.1. Generation of object model combinations per. First, a VOI area including a pathological shadow candidate detected by our quoit filter is extracted from an observed CT image, and is divided into a certain number of rectangle regions(in this experiment, 3×3×3). A 3D partial space corresponding to each rectangle region is called a cell. For each cell, geometrical object models which represent nodules and blood vessels are generated, and some promising object models are selected by evaluating the degrees of the correspondences with the rectangle regions. By combining these promising object models, a set of object model combinations is generated. From these object model combinations, artificial CT images are generated using computer graphics techniques. By comparing the VOI area and the artificial CT image, the probability of the hypothesis that the VOI area comes from each object model combination is calculated evaluating the relations between the object models in the combination by MRF models. If the most possible combination has any nodule models, the shadow candidate is determined to be abnormal. Otherwise, it is determined to be normal.
2. Object model generation [1] Nodules: We represent a nodule as a sphere model. For each cell, a sphere model is generated at the center of the cell, an artificial CT image is generated from the sphere
A set of object model combinations is generated by combining the promising object models. For each object model combination, a sum of the SSD values over the cells is calculated. The combinations are sorted by the sum of the SSD values, and we use Nmc (20) smallest (in the sum of the SSD value) combinations which have the nodule models and Nmc smallest combinations which have no nodule models for recognition process described in the next section.
3. Recognition of object model combinations using 3D MRF models Let Os be a random variable indicating an object model in a cell Ls located at a 3D position s, and the value of the random variable O s be os = {N D, BV, AR}, where N D represents a nodule model, BV a blood vessel(a curved blood vessel or a blood vessel branch) model, and AR an air model(which represents that the cell is empty), respectively. Let Ln(s) be a neighbor(26) of a cell L s . The state of Os is supposed to depend only on O n(s) . An object model combination is represented as o = {o1 , o2 , ...} ∈ O. Specially, let a combination including nodule models be o A ∈ O A , a combination without any nodule models be o N ∈ O N , respectively.
1051-4651/02 $17.00 (c) 2002 IEEE
Given a VOI area whose pixel values are v = {v1 , v2 , ...}, the most possible object model combination o∗ can be obtained by minimizing the following posteriori energy function(which is equal to the logarithm value of the probability of the hypothesis that the VOI area v comes from the combination o): U (o ∈ O|v) =
Vc (o) − T · log p(v|o),
Blood vessel models(BVM) An end point is isolated.
BVM1 (a) Two BVMs are CONNECTED normally.
r2 r1
(2) (PARALLEL) (TERMINATED)
PARALLEL or CONNECTED (TERMINATED)
(3)
(c) A BVM is TERMINATED abnormally.
Figure 3. Connections of blood vessels.
where C is a set of cliques of neighboring variables in the MRF, Vc (o) is a potential energy of a clique c of the object model combination o, p(v|o) is the likelihood and T is a constant value. The likelihood p(v|o) is calculated from the sum of the squared differences of the corresponding pixel values between the VOI area v and the artificial CT image generated from the object model combination o(see [5]). Next, the potential energies of the cliques are needed to be specified. For efficiency concerns, we only consider 1and 2-cliques when computing the energy function(in other words, Vc (o) = 0, c > 2). For 1-clique, clique c consists of a single MRF variable. The 1-clique energy of c = {os } corresponds to the prior probability p(o s ), V1 (os ∈ O s ) = − log p(os ). For example, the priori probability of the hypothesis that a cell has a nodule is represented as pN D · g(rN D ; µN D , σN D ), where pN D is the probability of the appearance of a nodule in a cell, rN D the radius of the nodule, and g(x; µ, σ) a normal distribution whose mean value is µ and standard deviation is σ. Thus, V1 (N D) = − log(pN D · g(rN D ; µN D , σN D )). For 2-clique, clique c consists of two neighboring MRF variables: c = {os , on(s) }, and the potential energy V 2 (os ∈ O s , on(s) ∈ O n(s) ) is defined considering the relation between os and on(s) . There are two kinds of relations. One is consistent as the 3D structure of the human lung, the other is inconsistent. For example, the state that two neighboring blood vessels are connected to each other as shown in Fig.3(a) and the state that one blood vessel model parallels the other one as shown in Fig.3(b) are consistent, while the state that the end point of a blood vessel is isolated like Fig.3(c) is inconsistent. V2 (os ∈ O s , on(s) ∈ O n(s) ) is defined from such consistency of the relations as follows:
V2 (AR, N D) = OK OK V2 (BV, BV ) = NG
(b) BVM1 PARALLELS BVM2 normally.
(1)
c∈C
V2 (AR, AR) = OK OK V2 (AR, BV ) = NG
BVM2
(4) (5)
r1
r2
(a) r1 /r 2 is enough large. (b) Not a SPICULE. The vessel is regared as a SPICULE of the nodule.
Figure 4. Relations between cancers and blood vessels. PARALLEL or OK SPICULE(Fig.4) V2 (BV, N D) = N G (NOT SPICULE).
(6)
In these equations, OK and N G indicate the consistent relation and the inconsistent relation, respectively. They are defined as OK = −1, N G = 10 in this experiment.
3.1. Searching for the most possible combination We search for the most possible combinations o ∗A and minimizing the posteriori energy function Eq.(1). U (o∗A |v) is less than a certain value If the ratio γ = U (o∗N |v) Tγ (Tγ = 1), then the VOI areas are determined to be abnormal. Otherwise they are determined to be normal. o∗N
4. Experimental Result In this paper, we use actual lung CT images of 38 samples(patients) for experiment. All the 38 samples have abnormalities. Each sample has more than 30 slice images which have 512 × 512 pixels. The resolutions of the images are 0.625[mm/pixel] in x-y plane and 10[mm/slice] in z axis. The parameters used in the 1-clique energies such as pN D , µN D and σN D are determined from the measurement results of 10 sample CT images(which are not same as the 38 samples used for testing). The other parameters such as OK, N G and Tγ are defined experimentally. By applying [6] to these images, 540 pathological shadow candidates are detected. 39 abnormal shadows are detected successfully, but there are 2 false negatives. By applying our new method described in this paper to these shadow candidates, we can reduce the number of the shadow candidates only to 181(the average number of the
1051-4651/02 $17.00 (c) 2002 IEEE
Figure 5. A sequence of a VOI area.
Y
X
Y
Figure 8. A sequence of the artificial CT images generated from Fig.6(b). X
Figure 9. A sequence of another example. Z
(a) including a nodule.
Z
(b) not including any nodules.
The future work is to improve the success ratio of recognition using the models of other parts of the lungs such as bones.
References Figure 6. The most possible object model combinations. shadow candidates per patient is about 4.76) without new false negatives. Fig.5 shows an example of a VOI area of a shadow candidate(marked by a white circle). Fig.6(a) and (b) show the most possible object model combinations. Fig.7 and Fig.8 show the artificial CT images generated from these object model combinations. Since γ = 1.071, the shadow candidate is successfully determined to be normal. The vessel model connecting to the nodule model is so thick in Fig.6(a) that it cannot be regarded as a spicule of the nodule. Fig.9 shows a VOI area of another shadow candidate. In this case, since there are no ridge shadows connecting to the detected shadow candidate, γ = 0.927, and the shadow candidate is successfully determined to be abnormal.
5. Conclusion In this paper, we proposed a new recognition method of lung nodules from X-ray CT images using 3D MRF models.
[1] S. Hideo and I. Noriko. “Development of a computer-aided detection system for lung cancer diagnosis”. In SPIE 1652, pages 567–571, 1992. [2] H.Kobatake, Y.Yoshinaga, and M.Murakami. “Automatic Detection of Malignant Tumor on Mammogram”. In Proc. Int. Conf. on Image Processing I, pages 407–410, 1994. [3] T. Okumura, T. Miwa, J. Kako, S. Yamamoto, M. Matsumoto, Y. Tateno, T. Iinuma, and T. Matshmoto. “Variable N-Quoit filter applied for automatic detection of lung cancer by X-ray CT”. In Computer Assisted Radiology and Surgery(CAR’98), pages 242–247, 1998. [4] A. Shimizu, M. Hagai, J. Hasegawa, and J. Toriwaki. “Performance Evaluation of 3D Enhancement Filters for Detection of Lung Cancer from 3D Chest X-ray CT Images (in japanese)”. Medical Imaging Technology, 13(6):853–863, 1995. [5] H. Takizawa, S. Yamamoto, G. Fukano, T. Matsumoto, Y. Tateno, T. Iinuma, and M. Matsumoto. “Recognition of Lung Cancers from X-ray CT Images Considering 3-D Structure of Objects and Uncertainty of Recognition”. In Proc. of Image Processing, part of SPIE, volume 1, pages 998–1007, 2000. [6] S. Yamamoto, M. Matsumoto, Y. Tateno, T. Iinuma, and T. Matshmoto. “Quoit Filter: A New Filter Based on Mathematical Morphology to Extract the Isolated Shadow, and Its Application to Automatic Detection of Lung Cancer in X-Ray CT”. In Proc. 13th Int. Conf. Pattern Recognition II, pages 3– 7, 1996. [7] S. Yamamoto, I. Tanaka, M. Senda, Y. Tateno, T. Iinuma, T. Matsumoto, and M. Matsumoto. “Image Processing for Computer-Aided Diagnosis of Lung Cancer by CT(LSCT)”. Systems and Computers in Japan, 25(2):67–80, 1994.
Figure 7. A sequence of the artificial CT images generated from Fig.6(a).
1051-4651/02 $17.00 (c) 2002 IEEE
