Acta Radiologica 44 (2003) 379â386. [5]Daubechies, I.: Ten Lectures on Wavelets. CBMS-NFS Series Appl. Math., SIAM. (1991). [6] Twellmann, T., Lichte, O., ...
Bielefeld University Faculty of Technology
Content Based Image Retrieval for Dynamic Time Series Data
The Applied Neuroinformatics Group
Birgit Lessmann1,2, Tim W Nattkemper1, Johannes Huth1, Christian Loyek2 Preminda Kessar3, Michael Khazen3, Linda Pointon3, Martin O. Leach3 and Andreas Degenhard2 1Applied
Neuroinformatics and 2Theoretical Physics, University of Bielefeld, Bielefeld, Germany MR Research Group, Institute of Cancer Research, Sutton, Surrey, UK
3Clinical Introduction
Database
• The increasing amount and growth of medical image databases (DB) raises the question how to explore large DB automatically. • For this purpose content based image retrieval (CBIR) systems have been developed [1] to assist in the clinical diagnostic process • These kind of systems allow the user to compare a given dataset with a large DB through retrieving those cases which are most similar • To this end, feature vectors must be computed for all included cases, appropriate to represent the characteristics of the specific case
• 27 time series of DCE-MR images. • the analysis included one image acquired before the injection of the contrast agent (pre-contrast) and four or five images afterwards (post-contrast) • 10 cases → four post-contrast images 17 cases → five post-contrast images • 14 cases → malignant 13 cases → benign
• The retrieval step strongly depends on these computed feature vectors
• Preprocessing: Segmentation of tumour tissue → decomposition of each tumour in several parts
• We have developed a CBIR system for time series of images, i.e. data acquired using dynamic contrast enhanced magnetic resonance imaging (DCE-MRI) [3]
• for each segment an average time series and a corresponding feature vector is computed → database entries
• Different approaches are proposed to handle time series of different length
Methods Dynamic Contrast Enhanced Magnetic Resonance Imaging • Breast Cancer is one of the most common types of cancer in western countries • A diagnosis method with high sensitivity is magnetic resonance imaging (MRI) involving the injection of a contrast agent [2]
• The total database comprises of ca. 3000 entries
• The administration of the contrast agent (Gd-DTPA) is monitored over time for the purpose of tumor detection and classification [3].
Results and Conclusion
• A time series of 3D images is acquired • malignant tumours are considered to show a fast uptake followed by a washout, while benign tumours are characterized by a slow uptake of the contrast agent: signal intensity benign characteristic
malignant characteristic
time
Content Based Image Retrieval • CBIR systems are an active field of research [1] • The idea is, to use these system as an assistant tool for the use of medical databases in diagnostic applications
• a priori probabilities for retrieving a malignant or benign case: 0.61 and 0.39 respectively • the results for malignant cases are better than the a priori value, in contrast to the results for benign cases • the standard deviation is very high → none of the four approaches can be clearly considered as the most appropriate one • in [6] it was shown, that malignant tumours also contain segments with a benign characteristic → many database entries marked as malignant, but showing a benign characteristic → bad results for benign tumour segments
• CBIR systems are based on the concept of query by example (QBE) system input: example case system output: those cases which are most similar to the query case.
1 truncated time series extrapolated time series wavelet features meta features
mean area under precision curve
0.8
Feature Vector Computation
• four approaches: 1. using the averaged time series as feature vector the six-dimensional time series are truncated to five-dimensional ones: t(1)
t(2)
t(3)
t(4)
t(5)
t(1)
t(2)
t(3)
t(4)
t(5)
t(6)
t(1)
t(1)
t(2)
t(2)
t(3)
t(4)
t(5)
t(3)
t(4)
t(5)
t(1)
t(2)
t(3)
t(4)
t(5)
t(2)
t(3)
t(4)
t(5)
t(6)
t(1)
t(2)
t(3)
t(4)
t(5)
t(6)
t(1)
t(2)
t(3)
t(4)
t(5)
t(6)
3. composing the feature vector of computed radiological meta features [4]: t(1)
t(2)
t(3)
t(4)
t(5)
t(1)
t(2)
t(3)
t(4)
t(5)
Five Meta− Features M(1)−M(5)
t(6)
M(1)
M(2)
M(3)
M(4)
M(5)
4. wavelet transform [5] of the time series → five or six wavelet coefficients selecting five coefficients for each feature vector → truncation in the wavelet domain: t(1)
t(2)
t(3)
t(4)
t(5) t(1)
wavelet coefficients
scaling coefficients scale 1
t(2)
wavelet coefficients
t(3)
t(4)
t(5)
t(6)
scaling coefficients
scale 1
scale 2 scale 2
scale 3
0 malignant cases
benign cases
Mean areas under the precision curve for the for different methods. Left: malignant cases, Right: benign cases
• Database Representation of tumours by segments and their averaged time series not sufficient
References
t(6)
2. using the averaged time series as feature vectors extrapolating the five dimensional time series to six-dimensional ones: t(1)
0.4
0.2
• four types of feature vectors → four methods to work with image data acquired using different protocols • feature vectors of equal length must be computed from time series of different length
0.6
scale 3
[1] Mueller, H et al.: A Review of Content-based Image Retrieval Systems in Medical Applications - Clinical Benefits and Future Directions. Int. J. Med. Inf., 73 (2004) pp. 1-23 [2] UK MRI Breast Screening Study Advisory Group: Magnetic resonance imaging screening in women at genetic risk of breast cancer: imaging and analysis protocol for the uk multicentre study. Magn. Res. Imag. 18 (2000) 765–776 [3] Heywang-Koebrunner, S.H., Beck, R.: Contrast-Enhanced MRI of the Breast. SpringerVerlag, Berlin (1996) [4] Szabo, B.K., Aspelin, P., Wiberg, M.K., Bone, B.: Analysis of kinetic and morphologic diagnostic criteria. Acta Radiologica 44 (2003) 379–386 [5] Daubechies, I.: (1991)
Ten Lectures on Wavelets. CBMS-NFS Series Appl. Math., SIAM
[6] Twellmann, T., Lichte, O., Nattkemper, T.W.: An adaptive tissue characterisation network for model-free visualisation of dynamic contrast-enhanced magnetic resonance image data. IEEE Transactions on Medical Imaging 24 (2005) 1256–1266
