Focal Cortical Dysplasia Segmentation in 3D Magnetic Resonance Images of the Human Brain Felipe P.G. Bergo Dept. of Neurology – FCM – University of Campinas (UNICAMP) C.P. 6111 – 13083-970 – Campinas – SP – Brazil
[email protected]
Alexandre X. Falc˜ao LIV – Institute of Computing – University of Campinas (UNICAMP) C. P. 6176 – 13083-970 – Campinas – SP – Brazil
[email protected]
Abstract In this work we present an image processing pipeline for automatic segmentation of focal cortical dysplasia lesions in 3D magnetic resonance images of the human brain. Dysplasia lesions are a common cause of refractory epilepsy, especially in children, and their treatment often involve surgical intervention. To achieve this pipeline we developed several new image processing techniques, procedures and algorithms, which led to an automatic FCD segmentation method that is more accurate than the state-of-the-art and that, unlike the state-of-the-art, is applicable to MR images of children.
1. Introduction Focal cortical displasia (FCD) is a malformation of the brain cortex commonly associated to refractory cases of epilepsy, where drug-based treatment does not eliminate the seizures [20]. FCDs are characterized by microscopic abnormalities of the gray matter tissue, which lead to subtle macroscopic variations in the tissue appearance in magnetic resonance images [24, 2]. Detection and localization of FCD lesions are fundamental for treatment planning. These tasks are often difficult, tedious and subjective due to the subtlety of the lesion appearance and to the large amount of data in a typical 3D MR (magnetic resonance) image. In this work we present a method for automatic detection and segmentation of FCD lesions in 3D magnetic resonance images of the human brain. The method combines new image segmentation techniques, automates a previously interactive technique (curvilinear reformatting [2]) and uses
classification by supervised learning [13] to detect and segment lesions with a 100% rate of detection and 76.9% coverage of the lesions [7]. This result is slightly better than the state-of-the-art [10].
2. Related Works The most common FCD diagnosis method in clinical practice is the straightforward visual inspection of MR images by specialists. The reported detection rate for this technique is 50% [20]. Techniques such as multiplanar reconstruction (MPR) and curvilinear reformatting (CR) increase the detection rate to 100% [20]. Curvilinear reformatting [2] consists of visualizing 3D MR images as curved surfaces that follow the shape of the brain. This technique improves the visibility of FCD lesions over MPR [20] and can be computed automatically without human interaction [4]. Several recent works presented automatic FCD detection methods, with detection rates varying from 53% to 85% [18, 1, 23, 10]. Most of these works focus on detection rather than segmentation. To our knowledge, Colliot et al. [10] is the only work that reports segmentation accuracy rate, with a coverage of 73% of the lesional voxels. Some of these works [18, 1, 10] rely on template-based segmentation techniques of the gray-matter (GM) tissue, which are often unreliable for children and patients who underwent brain surgery [9, 12].
3. Contributions The main contribution of this work is an automated procedure for segmentation of FCD lesions in 3D magnetic resonance images of the brain. The procedure attempts to mimic the effective CR-assisted visual inspection technique
used by specialists [20]. The procedure consists of 4 steps: brain segmentation, mid-sagittal plane location, curvilinear reformatting computation, and feature extraction and classification. All steps involve original contributions, detailed in [3]1 The next sections provide an overview of the contributions in each of these steps.
3.1. Brain Segmentation FCD lesions are often diagnosed in childhood [11, 22], and it is also desirable to segment these lesions on images from patients who underwent surgical treatment, to detect, locate and evaluate FCD lesions that were not removed. Many FCD detection/segmentation methods in the literature rely on template-based brain segmentation techniques [18, 1, 10], which are not reliable for images of children or surgical patients [12]. To circumvent this limitation, we developed a graphbased automatic brain segmentation procedure that reliably segments brains from 3D MR images regardless of patient age or anatomical variations. The main contribution in this point was the tree pruning segmentation technique. Tree pruning exploits the Image Foresting Transform (IFT) framework [15] to automatically detect object borders based on graph connectivity properties. In tree pruning segmentation, optimum-path trees are propagated from a set of markers inside the objects of interest. In most images, object borders are non-homogeneous and the forest crosses the objectbackground border through a reduced set of edges where the border is weaker. These are called leaking points and they cause detectable changes in the dynamic of the propagation as the trees reach the background (Figure 1). Unlike traditional segmentation techniques, tree pruning does not require external (background) markers and is completely independent of features in the background such as noise, texture and undesired borders. An interactive version of this idea was first presented in [14]; Improvements and automation were presented in [19]; and an extensive description of the methods, including preprocessing for brain segmentation and evaluation of the method in 8 synthetic MR volumes and 20 real MR volumes, was presented in [5]. Brain segmentation by tree pruning provided an average of 97.6% accuracy in comparison to expert delineations, and a desktop PC can automatically segment a typical MR brain image in about 25 seconds. These results are much better (both in accuracy and performance) than what is achieved with the popular SPM2 [16] template-based segmentation software, used by many works in the literature. Figure 2 shows an example of brain segmentation obtained by tree pruning. 1
This article summarizes the work presented on the Ph.D thesis of the first author. It is available online at: http://libdigi.unicamp. br/document/?code=vtls000439611 and http://www. liv.ic.unicamp.br/∼bergo/bergo-phd.pdf.
3.2. Mid-Sagittal Plane Location A key idea in our approach to FCD segmentation is to exploit the texture asymmetry of FCD lesions. Blurring and intensity shifts are typical signs of FCD lesions, but these features can also be caused by acquisition artifacts, such as partial volume, patient movement and magnetic field ramps. What specialists do when they visually search for lesions is look for asymmetric textures, since there are no records of FCD lesions occurring in both brain hemispheres within the same patient. In order to mimic this behavior, we need to locate a symmetry surface so that we can find a symmetric correspondent to each voxel in the brain. The inter-hemisphere fissure is a liquid-filled region between the brain hemispheres (Figure 3). It can be roughly approximated by a plane and it serves well as symmetry surface.
(a)
(b)
(c)
Figure 3. Brain hemispheres and mid-sagittal plane: (a) mid-sagittal plane shown in 3D renditions of an MR volume with segmented skin and brain. (b) and (c) automatically-located mid-sagittal plane shown in sample 2D slices (axial and coronal, respectively) of an MR volume.
There is a wide selection of mid-sagittal plane (MSP) computation methods in the literature, but many of them
(a)
(b)
(c)
(d)
(e)
(f)
Figure 1. Tree pruning segmentation example: (a) Original image of a chess board. (b) Border image with a marker inside the white knight. (c) and (d) An optimum-path forest is propagated from the marker, with edge costs determined by border intensity. (e) The forest reaches the background through a leaking point (indicated by the arrow), where branch splits at the same point. (f) Segmented object (white border) obtained by pruning the forest at the leaking point.
are not reliable for brains of post-surgery patients, or are too slow and take several minutes to locate the MSP. We developed a new MSP location technique that uses brain segmentation obtained by tree pruning [5] as a starting point and reliably locates the MSP, using an heuristic search algorithm, in about 5 seconds on a desktop PC. This method has been shown to be very reliable for images of patients with surgical cavities. The method has been evaluated on 64 MR images, including 36 images of patients with surgical cavities, and was shown to be accurate to within 1.5 degrees of rotation and 1.5 mm of translation. The method was first presented in [8]. The evaluation was later extended to compare the automatically-located MSPs with specialist delinations, and published in [6].
3.3. Automatic Curvilinear Reformatting Curvilinear reformatting (CR) was originally proposed by [2]. This technique allows specialists to visualize MR images of the brain as surfaces that follow the curvature of the brain, and that minimize partial volume effects and make FCD lesions more evident (Figure 4). This technique has been shown to greatly improve the detection rate of
FCD lesions by visual inspection [20]. But the implementation of the CR computation provided by the authors (in the commercially available software Brainsight [21]) was an interactive process where the operator had to manually delineate the brain curvature on several, but not all, MR slices. The curvature on the non-delineated slices was obtained by interpolation, and this procedure compromised both the repeatability and accuracy of the technique. We have shown that CR can be computed and displayed automatically with an Euclidian distance transform (EDT) from a surface called brain envelope that closely follows the surface of the segmented brain object. The brain envelope can be computed from the brain segmentation using trivial mathematical morphology operations, and the EDT can be efficiently computed using an IFT [15]. These results have been presented in [4].
3.4. Feature Extraction and Classification In this step we extract features from 2D texture patches that are tangent to the CR surfaces. For each voxel p, we extract features from a pair (T1 , T2 ) of texture patches: one centered at the voxel and one centered at the symmetric
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
Figure 2. Automatic brain segmentation: (a) A 2D slice of a 3D MR image of the brain. A typical 3D MR image has from 120 to 200 slices like this. (b) Automatic brain segmentation obtained by tree pruning, shown on a sagittal slice. (c) Another view of the same segmentation, on a coronal slice. (d) Another view of the same segmentation, on an axial slice. (e)–(h) Sample 3D renditions of 4 distinct brains segmented by automatic tree pruning.
voxel on the opposing hemisphere, as illustrated by Figure 5. By doing this, each feature vector stores information about the texture surroundings of the voxel and also about the similarity between the symmetric region in the opposing hemisphere. The key contribution in this step is adding symmetry information to the feature vector, something that was not tried in previously published FCD segmentation methods. To perform segmentation, we use a pattern classifier based on supervised learning. A speciliast manually delineated FCD lesions in MR images of epilepsy patients (Figure 6, first row). These delineations were used to train a Reduced Coulomb Energy network (RCE) [13], which was then used to classify voxels as lesional or not. This intermediary result still contains several isolated false positives spread throughout the image, as well as a minority of false negatives within the lesions (Figure 6, second row), due to the subtlety of FCD appearance. To correct these, we apply a filter that decides the final classification of each voxel based on the density of positively-classified voxels within a fixed neighborhood: for each voxel, we compute the number of lesional and non-lesional voxels (according to the output of the RCE classifier) within a sphere of radius 5 mm. The
voxel is only considered lesional if over 35% of the voxels within the sphere were classified as lesional. The sphere radius and the threshold percentage were determined empirically. This filtering procedure adds spatial coherence to the classification results and effectively removes outliers. Classification results before and after density filtering are illustrated in Figure 6 (second and third rows). While developing this classification procedure we experimented with a variety of feature vectors and techniques for classification by supervised learning. We evaluated several texture features (notably those proposed by Haralick et al. [17]), and how to combine them to evaluate asymmetry (such as using subtraction or division to compute feature asymmetry between T1 and T2 ), as well as different texture patch sizes. We used a feature vector with 16 elements: 4 sharpness features (sharpness in T1 , sharpness in T2 , sharpness difference, sharpness ratio), 2 entropy features, 2 contrast features, 2 homogeneity features, 2 average intensities, average intensity difference, 2 standard deviations of the intensities and the difference of the stadard deviations of intensity. All paired features include the measure in T1 and in T2 . Large texture patches introduce error from
Figure 5. Symmetric pair of texture patches used for feature extraction.
(a)
(b)
Figure 4. (a) Curvilinear reformatting visualizations for an MR volume, at 4 distinct depths. (b) The curvilinear surfaces of (a), plotted on regular 2D MR slices.
non-lesional areas surrounding the evaluated voxel, while smaller patches do not capture the features of large lesions. We experimented with patch sizes varying from 8 × 8mm to 64 × 64mm, and we chose 16 × 16mm since it produced slightly better results than the other sizes, while saving considerable processing time in comparison to computing features in 64 × 64 patches. Larger patches also introduced a
considerable ratio of false negatives, especially in patients with small lesions, since most of the lesional patch tended to present large areas with non-lesional features. Before choosing the RCE classifier, we attempted to perform classification using k-NN classifiers (with k = {1, 3, 5}) and artificial neural networks [13]. Neither provided adequate classification for two reasons: lack of discriminatory power of the feature vector (FCD lesions are subtle and we were unable to find better features to discriminate between lesional and non-lesional tissue) and the class asymmetry: in a patient with a large lesion, 98% of the voxels (or texture patches) are healthy and only the remaining 2% are lesional. Since the feature space is populated with close healthy/lesional samples, a k-NN classifier would pick healthy closer neighbours even when the test sample was very close to lesional samples, due to the higher density of healthy samples. Artifical neural networks were very unstable and their behavior varied considerably depending on the order in which training samples were fed (feeding lesional samples last would bias the network to classify everything as lesional, and vice-versa). It is worthy noting that simply classifying all voxels/patches as healthy with no image analysis easily ensures 98% accuracy (often 99.5% since FCD lesions tend to be small), but such result is clinically irrelevant. This is why the accuracy of lesion detection is measured as a coverage percentage of the lesioned tissue (i.e., how much of those 2% are being correctly classified as lesional, in the above mentioned 98/2 example). The RCE classifier deals with this problem in a simple yet effective way: each training sample is associated to a
Patient 1
Patient 2
Patient 3
Figure 6. Ground truth and segmentation results for 3 sample patients. The first row shows the ground truth provided by an specialist. The second row shows the intermediary classification result without density filtering. The third row shows the final automatic segmentation result. Patients 1 and 3 are children, and patient 1 has a visible anatomical deformity (top left). hypersphere in the feature space (R16 , in our case), whose radius is defined as the minimum radius such that no training samples from other classes are within it. To avoid floating point precision issues, we do not allow hyperspheres with radius below an application-defined ². Figure 7 illustrates an RCE classifier with 2 classes (red/vertical hatch and blue/horizontal hatch) in 2 dimensions: if a test sample falls within a red region (outside any blue sphere), it is surely of the red class. If it falls within a blue region (outside any red sphere), it is surely of the blue class. Purple (cross hatch) regions are ambiguous and to be decided by the application designer. Since in our problem we have few lesional samples, we chose to classify ambiguous test samples as lesional. This generates speckled false positives
(Figure 6, second row), which are eliminated later with the density filter (Figure 6, third row). A test sample falling in a white (uncovered) region would indicate an insufficient amount of training samples, and this situation does not occur on the actual application. The major drawback of the RCE classifier is that training time is quadratic (Θ(N 2 )) on the number of training samples (each sample inserted requires the recalculation of the radii of all hyperspheres of different classes). A test sample can be classified in time linearly proportional to the number of training sample. An evaluation of the complete procedure was presented in [7]. In this article we used datasets from 5 patients for experimental evaluation, obtaining a detection rate of 100%. Detection rate is often mentioned in studies about diag-
(a)
evaluation with 5 patients, the average voxel coverage of the lesions was 76.9%. This coverage was better than the 73% reported by [10]. Unlike [10], our procedure does not use template-based segmentation techniques (unreliable for children) and 2 of the patient datasets in our evaluation were from children. We evaluated the patients using the leaveone-out approach, never using training samples from the patient image being classified. The evaluation was performed on such a low number of patients due to the requirement of 5 different training sets, quadratic time for training and lack of hardware resources at the time. Figure 6 shows sample slices comparing the specialist delineations to the automatic lesion segmentations obtained by our method, as well as the intermediary result from the RCE classifier before density filtering.
4. Conclusions
(b) Figure 7. Reduced Coulomb Energy (RCE) classifier example with 2 classes (red/vertical hatch and blue/horizontal hatch) in 2 dimensions: (a) Simple situation with one training sample of each class. Test samples falling in blue regions belong to the blue class; Test samples falling in red regions belong to the red class; Test samples falling in purple (cross hatch) regions are ambiguous and test samples falling in white regions indicate insufficient training samples. (b) An example with more training samples.
nostic by visual inspection, where lesions can be entirely missed by the specialist [20]. For automatic methods based on supervised learning, such as ours, a detection rate of 100% is expected and a lower rate would most likely mean that the training set lacks information about the appearance of the undetected lesions. A better measurement of delineation quality is the voxel coverage of the lesions. In this
In this doctorate work we have presented several original contributions related to image processing and medical image analysis: tree pruning, a new image processing technique that is fast, does not require background markers and is very tolerant to variations in image background [5]; a mid-sagittal plane location for MR images of the brain that is fast, accurate and works well regardless of patient age and anatomical variations [8, 6]; an automatic technique to compute the curvilinear reformatting of MR images of the brain [4]; a procedure for feature extraction and classification that mimics the behavior of medical specialists to precisely determine the location and extent of focal cortical lesions in MR images of the brain [7]. Treatment of epilepsy patients with FCD lesions often involve surgical removal of the lesions, and MRI is the only imaging technique that can reveal these lesions. The FCD segmentation procedure developed during this work provided better quantitative results than the state-of-the-art, and it is also more widely applicable, since we have chosen techniques that perform well on images of children and of patients with anatomical variations. Extensions of this work include evaluation of a wider variety of texture features, computation of cortex thickness (requires reliable white matter/gray matter segmentation) and minimization of the training set (excluding training samples that do not affect the classification outcome, such as hyperspheres entirely contained by one or more hyperspheres of the same class).
Acknowledgments This work was developed as part of the CinAPCe program (http://www.cinapce.org.br) and was financially supported by FAPESP (Procs. 2009/06274-0 and 2005/56578-4).
References [1] S. B. Antel, D. L. Collins, N. Bernasconi, F. Andermann, R. Shinghal, R. E. Kearney, D. L. Arnold, and A. Bernasconi. Automated detection of focal cortical dysplasia lesions using computational models of their MRI characteristics and texture analysis. NeuroImage, 19(4):1748–1759, Aug 2003. [2] A. C. Bastos, R. M. Comeau, F. Andermann, D. Melanson, F. Cendes, F. Dubeau, S. Fontaine, D. Tampieri, and A. Olivier. Diagnosis of subtle focal dysplastic lesions: Curvilinear reformatting from three-dimensional magnetic resonance imaging. Annals of Neurology, 46(1):88–94, 1999. [3] F. P. G. Bergo. Segmentac¸a˜ o de Displasias Corticais Focais em Imagens de Resonˆancia Magn´etica do C´erebro Humano. PhD thesis, Universidade Estadual de Campinas, Instituto de Computac¸a˜ o, Apr 2008. [4] F. P. G. Bergo and A. X. Falc˜ao. Fast and automatic curvilinear reformatting of MR images of the brain for diagnosis of dysplastic lesions. In Proc. of 3rd Intl. Symp. on Biomedical Imaging, pages 486–489. IEEE, Apr 2006. [5] F. P. G. Bergo, A. X. Falc˜ao, P. A. V. Miranda, and L. M. Rocha. Automatic image segmentation by tree pruning. J Math Imaging and Vision, 29(2–3):141–162, Nov 2007. [6] F. P. G. Bergo, A. X. Falc˜ao, C. Yasuda, and G. C. S. Ruppert. Fast, accurate and precise mid-sagittal plane location in 3D MR images of the brain. In Biomedical Engineering Systems and Technologies (CCIS 25), pages 278–290. Springer, Berlin, Jan 2009. [7] F. P. G. Bergo, A. X. Falc˜ao, C. L. Yasuda, and F. Cendes. FCD segmentation using texture asymmetry of MR-T1 images of the brain. In Proc. of 5th Intl. Symp. on Biomedical Imaging, Paris, France, May 2008. IEEE. [8] F. P. G. Bergo, G. C. S. Ruppert, L. F. Pinto, and A. X. Falc˜ao. Fast and robust mid-sagittal plane location in 3D MR images of the brain. In Proc. BIOSIGNALS 2008 – Intl. Conf. on Bio-Inspired Syst. and Sig. Proc., pages 92–99, Funchal, Portugal, Jan 2008. Springer/IEEE. [9] D. L. Collins, A. P. Zijdenbos, V. Kollokian, J. G. Sled, N. J. Kabani, C. J. Holmes, and A. C. Evans. Design and construction of a realistic digital brain phantom. IEEE Trans. on Medical Imaging, 17(3):463–468, Jun 1998. [10] O. Colliot, T. Mansi, N. Bernasconi, V. Naessens, D. Klironomos, and A. Bernasconi. Segmentation of focal cortical dysplasia lesions on MRI using level set evolution. NeuroImage, 32(4):1621–1630, Oct 2006. [11] N. Colombo, L. Tassi, C. Galli, A. Citterio, G. L. Russo, G. Scialfa, and R. Spreafico. Focal cortical dysplasias: MR imaging, histopathologic, and clinical correlations in surgically treated patients with epilepsy. American Journal of Neuroradiology, 24:724–733, Apr. 2003. [12] M. B. Cuadra, C. Pollo, A. Bardera, O. Cuisenaire, J.G. Villemure, and J.-P. Thiran. Atlas-based segmentation of pathological MR brain images using a model of lesion growth. IEEE Trans Medical Imaging, 23(10):1301–1314, Oct 2004. [13] R. O. Duda, P. E. Hart, and D. G. Stork. Pattern Classification. Wiley-Interscience, 2000.
[14] A. X. Falc˜ao, F. P. G. Bergo, and P. A. V. Miranda. Image segmentation by tree pruning. In XVII Brazilian Symposium on Computer Graphics and Image Processing (SIBGRAPI), pages 65–71. IEEE, Oct 2004. [15] A. X. Falc˜ao, J. Stolfi, and R. A. Lotufo. The image foresting transform: Theory, algorithms, and applications. IEEE Trans. on Pattern Analysis and Machine Intelligence, 26(1):19–29, 2004. [16] R. Frackowiak, K. Friston, C. Frith, R. Dolan, C. Price, S. Zeki, J. Ashburner, and W. Penny. Human Brain Function. Academic Press, 2nd edition, 2003. [17] R. Haralick, K. Shanmugam, and I. Dinstein. Texture parameters for image classification. IEEE Trans. Syst. Man Cyb. 3, pages 610–621, 1973. [18] J. Kassubek, H. J. Huppertz, J. Spreer, and A. SchulzeBonhage. Detection and localication of focal cortical dysplasia by voxel-based 3-D MRI analysis. Epilepsia, 43(6):596– 602, Jun 2002. [19] P. A. V. Miranda, F. P. G. Bergo, L. M. Rocha, and A. X. Falc˜ao. Tree-pruning: A new algorithm and its comparative analysis with the watershed transform for automatic image segmentation. In Proc. of the XIX Brazillian Symposium on Computer Graphics and Image Processing, pages 37–44. IEEE, Oct 2006. [20] M. A. Montenegro, L. M. Li, M. M. Guerreiro, C. A. M. Guerreiro, and F. Cendes. Focal cortical dysplasia: Improving diagnosis and localization with magnetic resonance imaging multiplanar and curvilinear reconstruction. J Neuroimg, 12(3):224–230, Jul 2002. [21] R. Research. BrainSight. http://www.rogueresearch.com/epilepsy.htm. [22] P. M. Ruggieri, I. Najm, R. Bronen, M. Campos, F. Cendes, J. S. Duncan, H. G. Weiser, and W. H. Theodore. Neuroimaging of the cortical dysplasias. Neurology, 62:S27–S29, 2004. [23] S. Srivastava, F. Maes, D. Vandermeulen, W. van Paesschen, P. Dupont, and P. Suetens. Feature-based statistical analysis of structural MR data for automatic detection of focal cortical dysplastic lesions. NeuroImage, 27(2):253–266, Aug 2005. [24] D. C. Taylor, M. A. Falconer, C. J. Bruton, and J. A. Corsellis. Focal dysplasia of the cerebral cortex in epilepsy. J Neurol Neurosurg Psychiatry, 34(4):369–387, Aug 1971.
