Applications of contour tracking techniques Andrea Giachetti
CRS4 - Via N. Sauro 10, Cagliari e-mail
[email protected] [email protected]
Abstract This report describes the results of the application of contour tracking techniques based on Active Contours and image motion estimation on di erent kinds of biomedical images, and the developmente of new user-friendly interfaces for the interactive analysis of the images. A dedicated system similar to the one described in [1, 2] has been developed for the analysis of ultrasound images representing a carotid artery. Another contour-detection and tracking application has been dedicated to the segmentation of MR velocity maps representing the blood ow in the heart. A simple tool for the initialization of the program an the display of the results has been developed without the use of the Xview libraries as in the original case and can therefore be compiled and used on every X window machine. The remote control through a Java Applet communicating with a C program running on the server using the routines developed by David Greco and Marco Perscosolido at CRS4 is going to be experimented.
Introduction This report describes the work realized at CRS4 on the segmentation of moving regions using modi ed versions of the active contour - optical ow technique described in [1, 2]. The modi ed algorithms realized for the images analyzed are described in sections 1. Programs, the user interfaces and instructions on their use are presented in section 2. Experimental results are described in section 3.
1 The contour detection / tracking algorithms The basic algorithm used for the contour detection is a modi cation of the classical snake [3]: i.e. an elastic contour that moves driven or attracted by forces acting on its points and depending also on image features. The contour is represented by a closed curve, parametrized by the arc length s: ~p(s) = (x(s); y (s)), and its elastic behaviour is given by applying to its points the force F~int = ~pss tending to minimize the contour length. This technique provides regular contours that are close to the boundaries of the object desired only if the other forces and the initial position of the contour are correctly chosen. For our purpose, i. e. to nd bright boundaries in an ecochardiographic image Fig. 2 or a bright region in a dark background Fig. 3 a suitable choice is to initialize the contour inside the boundaries or outside the region and to use an in ating (de ating) force like in the balloon model of Cohen & Cohen [4]. This force is described by F~g = fg~ (where ~ (s) is the unit normal vector of the curve) and is directed as the unit vector perpendicular to the contour itself. To have a convergence of the curve close to the desired boundaries it is then necessary to introduce an image force. Being the image very noisy, in order to have a better detection of the borders, a preliminary ltering of the images reducing the high frequency noise should be performed. We applied a Gaussian mask, with = 1:5pixels. We tried di erent kinds of image forces, nding better results by using a "de ating/in ating" force, stopping the in ation/de ation near the higher ~ jrE (~p)j gray level instead of the common edge forces like: F~e(~p) = ?r 1
(where E (~p) is the grey level at the point ~p). Our force is simply proportional to a step function (E ? k) that is zero when the gray level is lower than a xed threshold k and has the value fd when the gray level is above the threshold. The force is directed along the unit vector ^ perpendicular to the curve: F~d = ?fd (E (~x) ? k )~ (1) An edge force can be added or increased after the initialization to have a better detection of the discontinuities. The nal result of the implementation is an elastic curve driven by the image features and controlled with the four parameters fg ; fi; fd; fe and the threshold value k. The use of the in ating and de ating forces, avoiding derivative computations, makes the snake behaviour less in uenced by noise. In order to have the snake evolution stopped near the true boundaries, it is however necessary a careful tuning of the coecients. We have found that an initial tuning of the force coecients is sucient to have a correct behaviour for all the image sequences, but particular attenction should be given to the threshold k of the step function.
1.1 Contour initialization and threshold setting
In our applications the contour is usually initialized as a small model of the object to be detected that the user should place inside the real contours. Of course, the initial contour can be also manually traced near the boundaries. The threshold k can be set by the user, but better and faster results have been obtained with an automatic choice. If the automatic threshold setting is chosen, when the model is posed, for each point the maximum of the gray level signed (the sign is positive when the brighter pixels are outside the contour) variation is searched along the direction perpendicular to the curve in the point and the threshold is taken as the gray-level value corresponding to this maximum (see Fig. 1). The search is performed from a few pixel inside the contour to outside the contour and if during the search a variation larger than a xed threshold respect to the internal gray level is found, the search is stopped and the local gray level is taken as the threshold. 2
p(i) k(i)
Figure 1: Automatic threshold setting: the gray level corresponding to the external region near the larger intensity change from dark (inside) and bright (membrane) pixel along the normal direction is chosen as the threshold k(i).
1.2 Contour tracking
The basic algorithm used for the contour tracking is the siplest one described in the previous works [1, 2]: First the optical ow is computed with a fast corrrelation technique, then the contour is regularized with snake forces (or other techniques like approximating the computed vectors with motion models). For the new applications only a few modi cations to the optical ow algorithm have been performed. The 1d-1d technique of Ancona and Poggio has been dropped and a more accurate one has been introduced. In the following sections the particular modi cations used for the analysis of the new images are describe.
1.3 Tracking the carotid artery
Detecting and tracking the borders of in echo images can be useful to analyze the motion and the contractions of the boundaries of arteries during (e.g. the carotid artery) the cardiac cycle. An example of these images is represented 3
Figure 2: Example of echo image of a carotid artery. in Fig 2. The contour detection is performed initializing the contour as a thin rectangle inside the vessel The snake then undergoes the in ation with the in ating force, the elastic force, the de ating force an a small edge force as described in Section 1. The parameter values have been chosen as: fg = 0:1; fd = 1; Fi = 0:25; fe = 0:05. Slight modi cations of these parameters do not change too much the snake behaviour. The threshold k is usually chosen with the automatic algorithm. As useful parameter computed from the contours we implemented a method to evaluate the contraction of the vessel during a heart beat. The method estimates the vertical distance between the upper and lower points of the contour not too close to the image left and right image borders and consider it as an apptoximated estimate of the diameter of the vessel. The variations of this value are plotted in the display and the percentual variation is given as an estimate of the contraction.
1.4 New constraints
The points at the left and right borders of the image are forced to keep their x coordinates, because the brighter border us, of course, not present in that region. 4
Figure 3: Example of cardiac MR velocity map. In order to have more constraints to have better results when the images are noisy and the contour is not continuous, the displacements of the contour points can be averaged with a model of the motion. This model consists simply in a uniform vertical translation of the lower and the upper bounds of the contour. The tracking is still performed by rst shifting th points with the best local image motion computed with a fast SSD correlation algorithm and then regularizing the contour with the same forces as before, changing only edge force, usually increased to have a contour closer to the edges. Being the motion of the points approximately vertical, the user can choose between two di erent correlation algotithms: the rst is the standard 2D, the second is a 1D vertical correlation. Furthermore, in order to give more stability to the contour "mixed" tracking averaging the ow vectors computed with the vector els resulting tting the computed displacement to a motion model has been introduced as an option. The motion model is no longer a linear one or a statistical model coming from a Pont Distribution model as done in the Left Ventricle tracking system, but a simpler one assuming the two visible sides of the carotid artery moves uniformily upwards and downwards.
5
1.5 Tracking The MR velocity map borders
For the MR velocity maps the problem was slightly di erent: a non-uniform brighter region bust be separated by the surrounding darker zone. The basic idea of the segmentation performed is to change the direction of the forces generating a de ating contour initialized outside the region to detect and stopped in its shrinking by an opposite in ating force generated by the higher gray levels and by the edge force. The automatic threshold choice for the new in ating force has been chosen. The initial contour is usually set by hand. The value of the parameters good for the detection are approximately the same as usual, just changing the sign of the in atinf force and of the de ating force. A good parameters set is then fg = ?0:1; fd = ?1; Fi = 0:25; fe = 0:05 (k automatically taken).
2 Programs realized at CRS4 2.1 Tool for artery tracking
The directory PROGRAMMI contains an Xview tool similar to the one previously realized for the heart tracking with parameters and initialization procedure optimized for the artery tracking. The program can be compiled only on sun-os machines and executed on all SparcStation. The menu allows the user to change the parameters described before; when the botton RUN is pressed, the rst frame of the image sequence is loaded and the contour has to be initialized. If the model initialization is chosen, the user must click the left button to drop the model in the desired point and follow the instructions on the display to modify it if necessary. Clicking the right button, the detection and tracking is started. If the hand initialization is chosen, the user must trace a line by clicking on reference points with the left button. clicking the right button the curve is closed and the detection/tracking started.
6
2.2 The simpler X Windows user interface
The system previously realized for the analysis of Echocardiographic image sequences was realized using the Xwiew libraries limiting the use of the program only on Sun Workstation. A simpli ed user interface has therefore been realized only with the standard X Windows library so it can be compiled with di erent architectures. The parameters are requested initially (if the user wants to change the default ones). A dedicate window menu to change the parameters will be realized in the future. We have realized three version of the program: one for the heart and generic balloon snakes (NEWHEART/heart), another one with the constraints and the initialization for the artery tracking (NEWCARO/caro), and another one with tha added possibility of using images of size lower than 256x256 pixels.
2.3 Use of the program
When the program is started the user is asked to introduce the name of the sequence to be analyzed. The program reads raw unsigned char data, some versions only of 256*256 pixels, other with variable dimension. The Xview version reads also 256*256 mpeg sequences (the option can be introduced easily also in the other versions. The program then ask if the user wants to change some parameters. For the carotid artery tool, the parameters that the user can change and the options that he can select are:
Number labeling the rst image Sequence length Compression algorithm of the data Selection the tracking based on correlation optical ow or based on a simple feature search and snake forces. Selection between 3 di erent way of determining the thresholds k: with a xed value, with the algorithm previously described, as the local grey level outside the contour. 7
Tracking simple or mixed as described in the text. Saving images with contours superimposed. Image dimension (if the option is implemented) Snake parameters fi; Fe; Fg ; Fd Selection of 1d - 2d correlation. Dimensions of the correlation mask. Search space of the correlation algorithm.
The initialization procedure is then identical to the previous version: the contour is initialized by clicking with the mouse in the desired region and then following the instructions on the display. If the contour is initialized as a model, the user is allowed o modify it by clicking the buttons on the display window. The tool used for the MR images is the X windows version of the heart tracking tool, modi ed to allow the use of a de ating contour. The parameter choice is approximately the same, the only di erence is in the model-based algorithms which are not described here and are not useful for these applications.
2.4 The client-server separation
The directory SOCKET contains a modi ed version of caro program, where the input section and the X display are controlled by the "client" program, while the elaborations are performed by a separated process ("server"). The two programs communicate through the socket libraries written by David Greco. When the server is started, it waits for the le information from a client. When the client runs, it asks the parameters in the same way as described before and opens an X display window. The data received by input are sent to the server, that load the rst image and send it back to the client for visualization. 8
Figure 4: Java applet for the remote analysis of echo images.
2.5 Remote analysis with a Java Applet
In order to allows remote applications of the tool, the client side of the aplication must be replaced by a Java Applet. The library used for the communiucation in the java program has been realized by Marco Pescosolido at CRS4. The applet is still under development and some experimental applets are located in the directory HTML/JAVA/WIN.
3 Experimental results
3.1 Carotid artery
The detection and tracking of the vessel boundaries were successful if the images were not too much corrupted by noise. Two sets of images have been analyzed, the rst acquired at video-rate, and the second at the rate of 12 frames/sec. An example of the initialization is represented in Fig. 5. In both cases the tracking algorithm seemed to work well. The 1D correlation 9
Figure 5: Standard initialization and contour detection for the Carotid artery tool. gives approximately the same contour position as the 2D correlation. Some problems can arise only when the image is corrupted and the tissue is not always brighter than the background and the initialization step can fail giving a contour larger than the desired one. A possible solution for these cases is the initialization by hand and a tracking where the threshold k is chosen always as the grey level outside the contour and/or the edge force is more relevant. All the image sequences represented healthy vessels and the computed contraction of the diameter was always found around 8-10%. Fig. 6 represent an example of succesful tracking. A possible to overcome it should be to introduce optimal weights in the mixed tracking and to introduce other constraints for the initialization.
3.2 Skejby
Fig. 7 shows the results of the segmentation of the interesting region in the MR velocity maps, that will be useful for reducing noise in the analysis of 3D data vectors. The contour was initialized as a sort of ellipse outside the reion of interest Fig. 7 A. Then the initial step is the de ation on the rst image to reach the contours. (Fig. 7 B. The trackin is then performed in the 10
Figure 6: Example of tracking of carotid artery contours.
11
usual way.
4 Conclusions The techniques developed seem to provide useful results on the images analyzed and applications of the tools developed seem to be of medical interest.
Acknowledgments This work has been carried out while the author was visiting CRS4 in the summer of 1996. Work carried out with the nancial contribution of Sardinia Regional Authorities.
References [1] A. Giachetti, G. Gigli & V. Torre, \Computer assisted analysis of Echocardiographic Image sequences" proc. CVRMED '95 Nice, apr. 1995. [2] A. Giachetti, M. Cappello, G. Gigli & V. Torre, \Computer assisted analysis of Echocardiographic Image sequences" proc. ICIAP '95 Sanremo, sept. 1995. [3] A. Kass, A. Witkin and D. Terzopoulos, \Snakes: Active contour models," Int. J. of Comp. Vision 1, 321{331 (1988). [4] L.D. Cohen and Isaac Cohen, \A nite element method applied to new active contour models and 3D reconstructions from cross-sections" Proc. of 3rd Int. Conf. on Comp. Vision, pp. 587{591 (1990). [5] T.F. Cootes, A. Hill, C. J. Taylor and J. Haslam \The Use of Active Shape Models For Locating Structures in Medical Images" [6] D. Greco, \A protocol for interactive client-server applications" CRS4 technical Report (in preparation). 12
Figure 7: Tracking the heart moving region in the MR velocity images 13
[7] M. Pescosolido \A Java Implementation of ..." CRS4 technical Report (in preparation). [8] M. Pescosolido \Java classes for interactive graphical clients" CRS4 technical Report (in preparation).
14
