Inferior thoracic aperture (ITA) is a 3D closed contour near the periphery of the diaphragm and the accurate detection of ITA from. CT images has many important ...
Detecting the Inferior Thoracic Aperture using Statistical Shape Models Pahal Dalal1 , Brent C. Munsell1 , Hiroaki Ninomiya2 , Xiangrong Zhou2 , Hiroshi Fujita2 , Jijun Tang1 , and Song Wang1 1
Department of Computer Science and Engineering, University of South Carolina, Columbia, SC 29208, USA 2 Department of Intelligent Image Information, Graduate School of Medicine, Gifu University, Japan
Abstract. Inferior thoracic aperture (ITA) is a 3D closed contour near the periphery of the diaphragm and the accurate detection of ITA from CT images has many important applications. However, it is very difficult to identify certain parts of the ITA from CT images due to the properties of adjacent anatomic structures and image noise. In this paper, we develop a new method to detect the full ITA contour by combining the following information: (a) parts of ITA that can be easily and accurately detected from CT images, (b) several anatomically-meaningful landmark points near or on the ITA, and (c) a statistical shape model of the ITA. Experiments are conducted on 14 real cases using the leave-one-out technique. We show that the results produced by the proposed method are better than the ones produced by a simple spline-based interpolation method.
1
Introduction
Diaphragm is an elastic membrane that separates chest from abdomen and inferior thoracic aperture (ITA) is a 3D closed contour attached to the periphery of the diaphragm in human anatomy [1]. The shape and position of the diaphragm have been related with clinical applications such as the normal pulmonary function [2]. Accurately detecting the ITA is usually the first step in locating and analyzing the shape and position of the diaphragm from a 3D CT image. However, parts of diaphragm and ITA, mainly near the locations of Xiphoid process and Vertebra TXII, are not clearly visible in CT images: their intensities are very similar to the surrounding tissues. Therefore, it is usually difficult to apply traditional image processing techniques, such as edge detection or intensity thresholding, to detect them. On the other hand, we have several additional information sources that can help the accurate detection of ITA. First, parts of ITA, especially the parts near the bone structure can be easily and accurately detected by simple intensitythresholding algorithm given the clear intensity contrast between the surrounding tissues. Second, some anatomic landmarks that are on or close to the difficultto-detect parts of the ITA can be obtained by an automatic detection or manual
labeling. Third, most ITA’s bear a specific shape that can be conveniently described by a statistical shape model [3, 4]. Such a statistical shape model in fact provides important prior knowledge of the shape of ITA. In this paper we develop a new method to integrate the above three important information sources to accurately detect the full ITA. Particularly, we model the shape of the ITA using the widely used point distribution model (PDM) [5], in which the major problem is to identify a set of corresponded landmarks across a population of ITA’s. We collect 14 ITA shape contours by manual extraction from 14 torso CT images for evaluating the proposed method. We conduct an experiment study to compare the proposed method and a simple spline-based interpolation method.
2
Problem Description
As discussed above, inferior thoracic aperture (ITA) can be divided into the parts that are easy to detect and the parts that are difficult to detect, by an intensitythresholding algorithm. The easy-to-detect parts are two curve segments along the costal margins of Rib XII, the top point of Rib XI and the distal cartilaginous ends of Ribs VII to X. We denote these two curve segments as A, which are illustrated by solid blue curves in Fig. 1. The difficult-to-detect parts of ITA are near the Xiphoid process and Vertebra TXII. As shown in Fig. 1 by red dashed curves, they also consist of two curve segments that fill the gap between the two curve segments in A. For convenience, we denote them as C.
1
C
2 11 12
Torso
13 3 Spine
ITA
4
10 9
14
15
A
8
5 16 6
7
A: Easy to detect parts B: Anatomical landmarks C: Difficult to detect parts
Fig. 1. An illustration of the easy-to-detect parts of the ITA, the difficult-to-detect parts of the ITA, and 16 anatomic landmarks on or near the ITA. {1,2}: Ends of Xiphoid Process; {3,4,5}: Vertebra TX, TXI, TXII (right); {6,7,8}: Vertebra TX, TXI, TXII (left); {9}: front of first lumbar vertebra; {10,11,12}: front and transverse processes of Vertebra TX; {13,14,15,16}: endpoints of curve segments in A.
In addition, some anatomically meaningful landmarks on or close to the ITA can be accurately and easily identified from CT images. For each ITA, we identify 16 such landmarks: two ends of the Xiphoid process, Vertebra TX (right), Vertebra TXI (right), Vertebra TXII (right), Vertebra TX (left), Vertebra TXI (left), Vertebra TXII (left), front of the first lumbar vertebra, front of Vertebra TX and its two transverse processes and four endpoints of curve segments in A. For convenience, we denote these 16 landmarks as a point set B, which are shown in Fig. 1 as black dots. In this paper, our goal is to accurately estimate the two difficult-to-detect curve segments in C by using available information of A and B, together with a point distribution model (PDM) of the ITA that can be constructed from a set of training ITA contours.
3 3.1
Proposed Method Constructing Statistical Shape Models for ITA
We apply the widely used point distribution model (PDM) to describe the shape of ITA. Given a set of training ITA contours, the key step in PDM construction is to identify a set of corresponded landmarks on them. For a training ITA contour, we have all three parts A, B, and C. While B provides 16 corresponded landmarks, they are still not sufficiently dense to represent the whole ITA shape. Therefore, we need to further identify some landmarks along A and C to construct an ITA PDM. Automatic identification of corresponded landmarks for 2D and 3D PDM construction has been investigated by many researchers in recent years [6–8]. In this paper, we adapt the landmark-sliding approach developed by Wang et al [9] for this purpose because (a) it is effective and efficient to handle the contourbased shape correspondence, and (b) available corresponded landmarks, e.g., 16 landmarks shown in Fig. 1, can be easily incorporated as linear constraints. In [9], this landmark-sliding approach is developed to correspond 2D planar shape contours. In this paper, we extend it to correspond 3D ITA shape contours. When applying this landmark-sliding approach to correspond a set of training ITA contours, we first select one ITA contour as the template and sample denser landmarks, other than the 16 available ones in B, along the curve segments A and C on this template, as shown in Fig. 2. For clarity, we call the 16 given landmarks to be basic landmarks and the additional sampled ones to be auxiliary landmarks, as shown in Fig. 2. Then we correspond each of the remaining ITA contours to the template by identifying the same number of corresponded landmarks. The basic operation is then to correspond a target ITA contour to the template, which operates as follows: 1. Denote all the landmarks (including both basic and auxiliary ones) in the template as U = (u1 , u2 , . . . , un ). Initialize a set of corresponded landmarks V = (v1 , v2 , . . . , vn ) in the considered target ITA contour by including the 16 available basic landmarks in the target and sampling the auxiliary ones
along the curve segments A and C in the target. Specifically, initially selected auxiliary landmarks in V are chosen in a way that they have the same arclength distribution as the auxiliary ones in U, as shown in Fig. 2 [9]. 2. Slide all the auxiliary landmarks in V along the tangent direction of the target ITA contour. The goal of sliding is to minimize a correspondence error that is defined by the thin-plate bending energy [10, 11] between U and V0 , where V0 is the location of the landmarks after sliding V. Since ITA is a 3D contour, we use the 3D thin-plate bending energy in this paper. Two sets of additional constraints are imposed in solving this optimization problem: (a) all the basic landmarks are counted in calculating the bending energy but not allowed to slide, and (b) no landmark is allowed to move across its neighbors in the ITA contour so that the shape topology is preserved. Since the thinplate bending energy is of a quadratic form and both sets of constraints are linear, the optimal sliding distance (or the optimal solution of V0 ) can be efficiently calculated by solving a quadratic-programming problem [9]. 3. Set V0 to be V, go back to Step 2, and repeat until convergence, e.g., the difference between V0 and V is very small.
Template
Landmark in A Landmark in B Landmark in C
Target
Fig. 2. (left) Basic and auxiliary landmarks selected in the template ITA contour. (right) Basic and initially sampled auxiliary landmarks in the target ITA contour.
Note that, the new landmarks V0 might not be located exactly on the target ITA contour, since the landmark sliding is operated along the tangent direction. In [9], this is solved by an additional step of projecting all landmarks in V0 back to the shape contour. In this paper, we simply limit the sliding step length of each auxiliary landmark in each iteration to be smaller than a given small threshold ² > 0 and find that the resulting V0 is always very close to the target ITA contour, even after many iterations. Such a step-length constraint is also linear and therefore does not change the nature of the problem: in each iteration,
we only need to solve a quadratic-programming problem. In this paper, we set ² = 0.5 in all our experiments. Based on the corresponded landmarks, including both basic and auxiliary landmarks, a PDM can be constructed by (a) normalizing all the training ITA contours [12], including the identified landmarks by removing their relative size, orientation, and location differences, (b) columnizing both basic and auxiliary landmarks in each ITA contour into a shape vector v = (v1T , v2T , . . . , vnT )T , with vi = (xi , yi , zi )T being the i-th landmark coordinate (after normalization), and (c) calculating the mean shape vector v ¯ and covariance S of the shape vectors constructed from the training ITA contours. By assuming a multivariate Gaussian distribution, this mean shape vector and covariance probabilistically describes the shape model of the ITA. 3.2
Estimating the Difficult-to-Detect Parts in ITA
Recall that our task is to detect two curve segments C to connect the two given curve segments A, by using available basic landmarks B and the ITA PDM (¯ v, S). We reduce the problem to identifying the auxiliary landmarks that are more densely distributed along the ITA contour, including the difficult-to-detect parts C. We solve this problem by two steps. First, we apply the same landmark-sliding approach as discussed in Section 3.1 to locate the auxiliary landmarks in the two curve segments in A. Specifically, we only use the 16 basic landmarks and the auxiliary landmarks in A in the mean shape v ¯ as the template and treat the considered ITA contour (without part C) as the target. The algorithm discussed in Section 3.1 can then be directly applied to find the auxiliary landmarks in A for this target ITA contour. Based on the identified landmarks (without the auxiliary ones in A), we can apply a 3D thin-plate transform to normalize the considered ITA contour (and the landmarks) to be of the same size, orientation, and location as the mean shape v ¯. Second, we incorporate the ITA PDM to estimate the auxiliary landmarks in C. Without loss of generality, let the final shape vector of the considered ITA T , contour (after the thin-plate transform in the first step) be v = (v1T , v2T , . . . , vm T T T vm+1 , . . . , vn ) , where the first m landmarks are known basic landmarks and landmarks in A. Additionally, the last n − m landmarks are the to-be-estimated T T ones in C. We can simply write it as v = (vPT , vQ ) , where vPT is a 3mdimensional column vector that represents coordinates of known landmarks and T vQ is a 3(n − m)-dimensional column vector which represents coordinates of the to-be-estimated landmarks. According to the Gaussian assumption underlying the PDM (¯ v, S), the shape vector v should minimize the Mahalanobis distance D(v, v ¯) = (v − v ¯)T · S−1 · (v − v ¯), subject to the constraint that vPT , or the first 3m elements in v, are known. To solve this optimization problem, we divide S−1 , i.e., the inverse covariance
matrix as · S−1 =
¸ S1 S2 , S3 S4
where S1 is the 3m × 3m top-left submatrix of S−1 . We can then rewrite the Mahalanobis distance as · ¸ · ¸ S1 S2 vP − v ¯P T T D(v, v ¯) = [(vP − v ¯P ) (vQ − v ¯Q ) ] · · , S3 S4 vQ − v ¯Q which can be simplified to D(v, v ¯) = (vQ − v ¯Q )T · S4 · (vQ − v ¯Q ) + 2(vP − v ¯P )T · S2 · (vQ − v ¯Q ) + k, where k is a constant independent of vQ . Since this is a quadratic function, the optimal estimate of the landmarks vQ that minimizes D(v, v ¯) can be obtained ∂D by taking ∂v = 0, which leads to Q T vQ = v ¯Q − S−1 ¯P ). 4 · S2 · (vP − v
(1)
With all the auxiliary and basic landmarks in the ITA contour, we can interpolate them to achieve the full ITA contour. Note that, an initial thin-plate transform has been imposed in the above first step after locating the auxiliary landmarks on the easy-to-detect parts A. We need to apply an inverse thin-plate transform when fitting the estimated ITA contour to the original CT image.
4
Experiments and Results
We collected 14 real instances of the ITA contours for testing the proposed method. Each of the collected contours are extracted from a 3D torso CT image and consist of parts A, B, and C as described above. Particularly, the difficult-to-detect part C’s are manually extracted as ground truth for performance evaluation. We apply a leave-one-out strategy for performance evaluation: taking any 13 out of 14 contours for PDM training and using the remaining one for testing. When testing a contour, we estimate its part C using the proposed method and then compare it with the manual extraction results. In the PDM construction, besides 16 basic landmarks, we identify 16 auxiliary landmarks in parts A and another 16 landmarks in parts C. Finally, complete ITA contours are approximated by interpolating all detected auxiliary landmarks using Catmull-Rom spline [13]. We use two error measures to evaluate the difference between the estimated C part and the ground-truth one: (a) e1 , the average Euclidean distance between each point on the estimated curve and its closest point in the ground truth and (b) e2 , the average distance between their corresponded auxiliary landmarks. Table 1 shows the resulting errors of the proposed method in all 14 leave-one-out cases. Table 1 also shows the errors produced by directly interpolating 14 basic
landmarks (two basic landmarks are not on the ITA contour as shown in Fig. 1) using Catmull-Rom spline. We can see that, by incorporating the shape information of the ITA, the proposed method detects the ITA more accurately. Figure 3 shows the three leave-one-out cases where the proposed method produces the best, median and worst results. Case
Error e1 (in voxels) Direct Interpolation Prop. Method 1 13.6 7.2 2 16.0 10.1 3 8.4 4.9 4 10.1 9.1 5 11.0 6.5 6 13.3 7.6 7 12.5 6.5 8 14.3 12.7 9 13.9 9.2 10 13.2 7.8 11 14.4 10.1 12 24.9 5.7 13 8.6 5.0 14 12.9 9.7 Mean 13.4 8.0
Error e2 (in voxels) Direct Interpolation Prop. Method 49.2 8.9 50.82 14.35 35.46 7.54 49.34 9.73 43.82 11.12 55.66 7.58 46.5 11.1 44.69 12.69 47.07 10.32 41.7 8.62 56.88 9.51 46.19 8.66 39.51 12.35 42.58 11.21 46.4 10.3
Table 1. Experiment results on all 14 leave-one-out cases.
5
Conclusions
In this paper, we introduced a new method to detect the inferior thoracic aperture (ITA) by incorporating the statistical shape model of the ITA. We showed that, by integrating the known parts of the ITA, several additional landmarks on or close to ITA, and the PDM of ITA, the proposed method can estimate ITA much more accurately than a direct curve interpolation. While this method is developed for ITA detection, it can be applied to other similar problems where partial geometric information is available and the desired structures always possess a specific shape.
Acknowledgement This work was funded, in part, by NSF-EIA-0312861.
References 1. Drake, R., Vogl, W., Mitchell, A.W.: Gray’s Anatomy for Students. 39 edn. (ELSEVIER)
Simple Interpolation Proposed Method
Best
Median
Worst
Fig. 3. Three sample ITA detection results. Ground truth is shown by dashed red curve and the detected shape is shown by solid blue curve.
2. Suwatanapongched, T., Gierada, D.S., Slone, R.M., Pilgram, T., Tuteur, P.: Variation in diaphragm position and shape in adults with normal pulmonary function. Chest 123 (2003) 2019–2027 3. Cootes, T., Taylor, C.: Active shape models - ‘smart snakes’. In: BMCV. (1992) 266–275 4. Cootes, T., Hill, A., Taylor, C., Haslam, J.: The use of active shape models for locating structures in medical images. IVC 12 (1994) 355–366 5. Cootes, T., Taylor, C., Cooper, D., Graham, J.: Training models of shape from sets of examples. In: BMCV. (1992) 9–18 6. Davies, R., Twining, C., Cootes, T., Waterton, J., Taylor, C.: A minimum description length approach to statistical shape modeling. IEEE Trans. Medical Imaging 21(5) (2002) 525–537 7. Hill, A., Taylor, C.: A framework for automatic landmark identification using a new method of nonrigid correspondence. PAMI 22(3) (2000) 241–251 8. Thodberg, H.: Adding curvature to minimum description length shape models. In: BMCV. Volume 2. (2003) 251–260 9. Wang, S., Kubota, T., Richardson, T.: Shape correspondence through landmark sliding. In: CVPR. (2004) I–143–150 10. Bookstein, F.: Principal warps: Thin-plate splines and the decomposition of deformations. PAMI 11(6) (1989) 567–585 11. Duchon, J.: Splines minimizing rotation-invariant semi-norms in Sobolev space. In: Constructive Theory of Functions of Several Variables, Lecture Notes in Mathematics, 571. (1977) 85–100 12. Rangarajan, A., Chui, H., Bookstein, F.: The softassign procrustes matching algorithm. In: IPMI. (1997) 29–42 13. Catmull, E., Rom, R.: A class of local interpolating splines. Computer Aided Geometric Design (1974) 317–326
