Mock article for Studia Informatica

STUDIA INFORMATICA Volume 37

2016 Number 3B (126)

Adam POPOWICZ Silesian University of Technology, Institute of Automatic Control Aleksander R. KUREK Jagiellonian University Astronomical Observatory

AN ALGORITHM FOR JOINT AND BONE LOCALIZATION IN USG IMAGES OF RHEUMATOID ARTHRITIS

Summary. The assessment of degree of joint inflammation using USG imaging is a challenging task, and thus the results may differ between experts. Therefore, the evaluation of images by a computer program may be a vital solution for objective assessment of disease progression. In this paper an original pipeline, developed for precise estimation of bone line and joint location in USG images, is proposed. The presented set of methods provides valuable input for further classification tools, which will aim at identification and rating of the degree of joint inflammation. Keywords: medical image processing, segmentation

ALGORYTM WYZNACZANIA POŁOŻENIA STAWU ORAZ KOŚCI W OBRAZACH USG PALCÓW DŁONI Streszczenie. Ocena stopnia zapalenia stawów palców dłoni z wykorzystaniem obrazów USG jest zwykle trudnym zadaniem i może być różna w zależności od eksperta analizującego obrazy. Z tego względu analiza wykorzystująca program komputerowy stanowić może obiektywny wskaźnik progresu choroby. W artykule zaprezentowano zestaw procedur przetwarzania obrazów, umożliwiający wykrycie położenia kości oraz stawu w obrazach USG. Wyniki zwracane przez algorytm stanowią istotną podstawę do dalszych kroków, mających na celu wykrycie oraz gradację stopnia zapalenia. Słowa kluczowe: przetwarzanie obrazów medycznych, segmentacja

8

A. Popowicz, A.R. Kurek

1. Introduction Ultrasound imaging, which enables detailed insight into internal structures of human body, has also proved its usefulness for precise estimation of a degree of joint inflammation in Rheumatoid Arthritis [1]. By a visual assessment of a USG image, a physician scores the progression of disease from 0 to 3, using the OMERACT definitions [2]. Unfortunately, the rating is usually somehow subjective since it depends on the experience of an examiner and may be affected by the standardized atlases currently utilized [3-4]. Therefore, it was frequently observed that the diagnosis of experts differ one from another. That is why it was decided to initiate a research (MEDUSA project, [5]) on possible automatic detection and rating of joint inflammation using image processing and classification tools. The location and size of synovitis region depends on severity of the disease. Initially, it develops from a joint, extending gradually along a bone toward a skin layer. The pathological region is usually more or less dimmer than the normal tissue (however, it is not a strict rule). Three examples of different degree of joint synovitis are presented in Fig. 1. The marked characteristic parts include inflammation region (green), bone line (red), skin line (violet) and joint (turquoise) as indicated by an experience physician.

Fig. 1. Examples of USG images of joint of various degree (from the left: 1, 2 and 3). Synovitis, bone and joint are marked by green, red and turquoise, respectively Rys. 1. Przykłady obrazów USG stawu o różnym stopniu zapalenia. Obszary zapalenia kości oraz stawu zostały zaznaczone kolorem odpowiednio zielonym, czerwonym i fioletowym

In this paper, a pipeline for automatic detection of joint and bone line is proposed. It was indicated by physicians that the synovitis develops along the bone and starts from the joint, it was concluded, that the proper localization of these two characteristic regions is of the highest importance. In presented algorithm a novel simple method of filtering of USG images is employed. The filter enhances the bones which are further processed by simple morphological operations (dilation, thresholding, calculation of region areas) leading to the estimation of a bone line. A joint position is subsequently obtained by analyzing the profile of bone, looking for its local minima. It should be noted, that there have been already some attempts to localize bones and joint in such USG images [6-8]. However, in contrast to those methods, the pipeline presented in this paper is significantly simpler and requires much less computational effort. It does not

An algorithm for joint and bone localization in USG images of rheumatoid arthritis

9

incorporate sophisticated classifiers which require and, unfortunately, are dependent on selection of a training set. As its operations can be easily understood, the implementation of proposed pipeline is straightforward. Its outcomes can be also utilized in parallel with the results provided by much more sophisticated algorithms, to increase the overall efficiency of such a multi-output detection system.

2. Pipeline 2.1. Detection of bone and joint position The first part of the pipeline aims at the enhancement of regions of finger bones. It is accomplished with a use of a novel filter developed especially for such USG images. The filter is based on the physics of assessment of an acoustic signal on obstacles, e.g. in form of bone. The basic absorption Lambert’s law indicates that the amplitude 𝐽 of acoustic wave moving in a body decreases as: 𝑑

𝐽(𝑑) = 𝐽(0) 𝑒 −𝜎 , (1) where d is the distance, 𝜎 is the parameter of the tissues (the denser the tissue, the lower the 𝜎). The properties of encountered tissues result in various reflections and absorption of incoming sound wave . The sound is reflected strongly by the bones, thus the intensity of this type of objects in USG images is distinctively higher. Due to the reflection, the wave passing through bone is strongly attenuated and therefore the intensity of pixels below bone are much dimmer. Based on this observation, the assumption was made that the bone pixels are the ones which are not proceeded (along the image column) by any brighter pixel. To reveal such regions, the pixels of original image are filtered by reducing its intensity by a maximum intensity of pixels located in a rectangular region below: 𝐼(𝑥0 , 𝑦0 ) = 𝐼(𝑥0 , 𝑦0 ) −

min

𝑦 > 𝑦0 +𝑅, 𝑥 𝜖 {𝑥0 −𝑀 ∶ 𝑥0 + 𝑀}

𝐼(𝑥, 𝑦) ,

(2)

where 𝐼(𝑥, 𝑦) is the intensity of pixel at (𝑥, 𝑦), R and M are parameters. The R parameter can be adjusted to allow for wider (higher R) or shorter (lower R) bone regions. The second parameter M is the width of filtering window and it provides for a possible scattering of acoustic waves in x-direction of an image. To suppress the speckle noise present in USG images, a simple median filter (K×K mask) is employed before the filtration process. The optimization of the values of filter parameters (R, M and K), as well as other tunable parameters included in the pipeline, will be discussed in the next sections of this paper. Exemplary outcomes of the proposed routine are presented in Fig. 2.

10


(a)

(b)

(c)

Fig. 2. Exemplary USG image processed by the proposed filter: (a) original image, (b) median filtered image (11×11 pixels mask), (c) filtered outcome Rys. 2. Przykłady obrazów USG przetworzonych za pomocą proponowanego filtru: (a) obraz oryginalny, (b) wynik filtracji medianowej, (c) wynik filtracji proponowaną techniką

The computational burden of such a filtering schema can be significantly reduced by (1) calculating the intensity maxima progressively in each line and then (2) by considering only the maxima located with a specified shift (R, M) relatively to currently processed pixel. The MATLAB code that allows for such efficient calculation of filter output is given below: function [filtered] = filter_image(image,M,R) [X,Y] = size(image); image = medfilt2(image,[11,11]); local_max = 0*image; for y = 1:Y column_max = 0;

for x = X:-1:1 column_max = max(column_max,image(x,y)); local_max(x,y) = column_max; end end filtered = -inf*image; M_2 = round(M/2-1); for x = 1:X-R for y = M:Y-M filtered(x,y) = max(local_max(x+R,y-M_2:y+M_2)); end end filtered = image – filtered; filtered (filtered

𝐵𝑆(𝑥)

max

− 𝐵𝑆(𝑥)

{𝐵𝑆(𝑥)}

𝑥𝜖{𝑥− 𝑑∶𝑥−1}

(3)

𝐵𝑆𝑓 (𝑥) = 0 if 𝐵𝑆(𝑥) > max {𝐵𝑆(𝑥)} { 𝑥𝜖{𝑥+1∶𝑥+𝑑} where BSf is a filtered bone signal, d is a tunable parameter, which controls the width in x axis within which a local minimum is calculated. The conditional statements in (3) ensure that the average height (i.e. y position) of a bone line on the left and right side is larger than the height of a joint. To further reduce the rate of false detections, only the pixels within a 1

7

limited x range (4 𝑥: 8 𝑥) are considered. Such asymmetrical selection of a region of interest is justified by the fact that joint is usually located slightly on the right side of an image center. It is done to expose the left-side bone which is frequently covered by the inflammation. Eventually, a joint position is determined from the maximum of BSf signal. Exemplary BSf signal calculated for a sample USG image is depicted in Fig. 6. Note the consecutive steps of processing of bone signal: limitation of bone line (green points), filtering (red points) and, finally, finding the position of joint (black circle).


13

Fig. 6. Estimation of a joint position in a sample USG image. Above: USG image with indicated bone line (red) and joint position (green cross). Below: bone signal (blue) with its filtered version (BSf, green) with a joint position indicated (black circle) Rys. 6. Estymacja położenia stawu w przykładowym obrazie USG. Powyżej: obraz USG z zaznaczonym na czerwono położeniem kości oraz zielonym wskaźnikiem stawu. Poniżej: linia kości (niebieska) wraz z jej odfiltrowaną wersją (BSf, kolor czerwony) oraz wskaźnik położenia stawu (czarny okrąg)

3.

Optimization

3.1. Reference data set The reference set of 276 USG images, depicting various degree of joint synovitis, was collected during the realization MEDUSA project, in a public health institution, Helse Forde, Norway [9]. The set consists of USG images with regions of bone line, joint and area of synovitis indicated by an experienced physician. All images depicted only finger crosssections, two types of joints: metacarpophalangeal joint (MCP) and proximal interphalangeal joint (PIP). Additionally, each image was assigned with a given degree of inflammation (0-3, where 0 stands for no visible synovitis). Several examples of images from our dataset are presented in Fig. 1 and Fig. 8. 3.2. Quality measures To assess the proximity between regions indicated by the presented algorithm and the ones marked by an expert, a special method was developed. Since the detection is not a single

14


pixel, but rather a set of pixels, the distance transformation of binary images was applied. First, the bone line indicated by an expert is processed by the distance transformation so that the map of distances to detected line is created. Then, the mean bone distance (Qb) between reference and determined bone lines can be calculated by averaging the distances assigned to the pixels marked as a bone line by the presented pipeline. The distance (Qj) between reference and detected joint position can be obtained similarly, using distance transformation. The distance transformation is a well-known image processing procedure, which allows for estimation of Euclidian distance between each pixel and its nearest indicated object. The algorithm uses the double-scan algorithm and is described in details in [10]. In the case of our images, the joint estimation error is simply the value of distance from the joint indicated by an expert. An example of distances estimated in such a quality assessment are presented in Fig. 7. The distances were calculated for the red, reference points, while the mean distance was calculated be averaging distances assigned to green pixels. 3.3. Parameters optimization The reference set of 276 USG images was processed with the pipeline employing various values of parameters. The optimized parameters were: R, M – parameters of bone filter, K – mask of median filter, T – threshold used for bone selection, and d – width of a window utilized during the search of local minima in filtered bone signal. The range of possible values of the parameters is presented in Table 1. The smallest values of quality indicators (Qf = 8.6 pix. and Qb = 33.9 pix) was achieved for the parameters well within a specified range and are given in Table 2. Several examples depicting various types of USG images with indicated bone and joint by both, expert and by the optimized pipeline are exposed in Fig. 8. The mean distance of automatic and reference detections, as calculated using Q measures, is highlighted with green and yellow, respectively for bone and joint.

4.

Summary

The USG imaging allows for detailed view of joint inflammation in Rheumatoid Arthritis. Unfortunately, the image complexity is so high that the grading of synovitis can differ even between the experts. Therefore, for the purpose of objective and reproducible assessment of such images, a dedicated computer program is being developed in MEDUSA project.


15

Fig. 7. Assessment of the error of pipeline results (Qb=3.5 pix., Qj=10 pix.). Green markers show the output of the proposed pipeline, while the red ones depict the markings indicated by an expert. Pixels intensities correspond to the distances from the reference points Rys. 7. Ocena błędu wyznaczania obszarów (Qb=3.5 pix., Qj=10 pix.). Zielonym markerem zaznaczono wyniki działania algorytmu. Czerwony kolor przedstawia wskazania lekarza. Jasności pikseli odpowiadają dystansowi od wskazań referencyjnych

Table 1 Range of parameters values utilized in the optimization Parameter Range Step R M T K D

10:40 2:10 2:10 11:21 60:120

10 2 2 2 20

Table 2 Optimal values of parameters for joint and bone line estimation Bone estimation Joint estimation Parameter (Qb = 8.6 pix) (Qj = 33.9 pix)

.

R M T K D

20 4 8 11 N/A

20 2 2 15 100

16


Fig. 8. Examples of automatic and manual annotation of bone line and joint position in USG images. Bone and joint indicated by an expert are marked by red and blue points respectively. Green bold line and yellow circle show automatically detected bone line and joint. Green and yellow highlighted regions expose the areas within mean distance as calculated using Q measures Rys. 8. Przykłady manualnej oraz automatycznej detekcji kości oraz stawu w obrazach USG. Kość i staw wskazane przez lekarza zostały zaznaczone kolorem czerwonym oraz niebieskim. Zielona pogrubiona linia oraz żółte kółko przedstawiają wykrytą kość oraz staw. Na zielono oraz żółto podświetlono obszary średniego błędu Qb oraz Qj


17

The first part of such a system includes the precise estimation of bone line and joint position. It is a challenging and crucial task, which directly affects an overall efficiency of automated grading system. For this purpose, a dedicated pipeline was presented in this paper. In a first step of the algorithm, a novel filter, incorporating simple procedures of image processing, was utilized to highlight the regions associated with bones. A computationally efficient implementation of the filter was also provided in this work. For the purpose of localization of joint position, the pipeline performs the search for local minima in estimated bone line. Several modifiable parameters allows for fine-tuning of the algorithm, so that its outputs are consistent with the reference indications provided by an expert. The pipeline, in contrast to the competitive solutions which employ machine-learning methods and complex classification procedures, does not require much computational effort. It is straightforward and may be easily implemented in any programing language. Additionally, this simplicity allows for future modifications and improvements in particular stages of the pipeline. The results of the method may also support the outcomes of much more sophisticated approaches, improving the final efficiency of detection system.

BIBLIOGRAPHY 1. Węgrzyn S.: Podstawy automatyki. PWN, Warszawa 1978. 2. Østergaard M., Ejbjerg B., Szkudlarek M.: Imaging in early rheumatoid arthritis: roles of magnetic resonance imaging, ultrasonography, conventional radiography and computed tomography. Best Practice & Research Clinical Rheumatology, 2005, vol. 19(1), p. 91÷116. 3. Østergaard M., Edmonds J., McQueen F., Peterfy C., Lassere5 M., Ejbjerg B., Bird P., Emery P., Genant H., Conaghan P.: An introduction to the EULAR–OMERACT rheumatoid arthritis MRI reference image atlas. Annals of the Rheumatic Diseases, 2005, vol. 64, p. i3÷i7. 4. Terslev L., Torp-Pedersen S., Savnik A., von der Recke P., Qvistgaard E., DanneskioldSamsøe B., Bliddal H.: Doppler ultrasound and magnetic resonance imaging of synovial inflammation of the hand in rheumatoid arthritis: a comparative study. Arthritis & Rheumatism, 2003, vol. 48(9), p. 2434÷2441. 5. Kulbacki M., Segen J., Habela P., Janiak M., Knieć W., Fojcik M., Mielnik P, Wojciechowski K.: Collaborative Tool for Annotation of Synovitis and Assessment in Ultrasound Images. International Conference on Computer Vision and Graphics ICCVG 2014.

18


6. Mielnik P.: Brief information on research work implemented under the MEDUSA project. Norsk Rheumabulletin, 2013, vol 4, p. 3. 7. Segen J., Kulbacki M., Wereszczyński K.: Registration of Ultrasound Images for Automated Assessment of Synovitis Activity. Intelligent Information and Database Systems, 2015, vol. 9012, p. 307÷316. 8. Segen J., Kulbacki M., Wereszczyński K., Wojciechowski K.: Optimization of Joint Detector for Ultrasound Images Using Mixtures of Image Feature Descriptors. Intelligent Information and Database Systems, Intelligent Information and Database Systems, 2015, vol. 9012, p. 277÷286. 9. Wereszczynski K., Segen J., Kulbacki M., Mielnik P., Fojcik M., Wojciechowski K.: Identifying a joint in medical ultrasound images using trained classifiers. International Conference on Computer Vision and Graphics ICCVG 2014. 10. Kulbacki M., Segen J., Habela P., Janiak M., Knieć W., Fojcik M., Mielnik P., Wojciechowski K.: Collaborative Tool for Annotation of Synovitis and Assessment in Ultrasound Images. International Conference on Computer Vision and Graphics ICCVG 2014. 11. Borgefors G.: Distance transformations in digital images. Computer Vision, Graphics, and Image Processing, 1986, vol. 34, no. 3, p. 344÷371.

Acknowledgment The research leading to these results has received funding from the Norwegian Financial Mechanism 2009-2014 under Project Contract No. Pol-Nor/204256/16/2013 and was performed using the infrastructure supported by POIG.02.03.01-24-099/13 grant: GCONiI Upper-Silesian Center for Scientific Computation.

Omówienie Ocena stopnia zapalenia stawów palców dłoni z wykorzystaniem obrazów USG jest niezwykle trudnym zadaniem. Gradacja stopnia zapalenia może być różna w zależności od eksperta oceniającego obraz. Z tego względu analiza wykorzystująca program komputerowy stanowić może obiektywny wskaźnik progresu choroby. Program taki jest głównym celem projektu MEDUSA. W artykule zaprezentowano zestaw procedur przetwarzania obrazów, umożliwiający wykrycie położenia kości oraz stawu w obrazach USG. Główna część


19

algorytmu wykorzystuje nową technikę filtracji obrazów, dzięki której uwidaczniane i następnie ekstrahowane są obszary obrazu związane z obecnością kości. Poszukiwanie położenia stawu odbywa się dzięki ocenie lokalnych minimów w linii tworzonej przez przekrój kości. Zaprezentowane metody zostały zoptymalizowane dzięki wykorzystaniu kilku modyfikowalnych parametrów, tak aby zmaksymalizować podobieństwo wyników algorytmu oraz referencyjnych wskazań dokonanych przez eksperta. Zestaw algorytmów dostarcza istotnych informacji wejściowych wymaganych do dalszych kroków związanych z detekcją i gradacją stopnia zapalenia stawu.

Addresses Adam POPOWICZ: Silesian University of Technology, Institute of Automatic Control, ul. Akademicka 16, 44-100 Gliwice, Poland, [email protected] Aleksander R. KUREK: Jagiellonian University Astronomical Observatory, ul. Orla 171 30-244 Kraków, Poland, [email protected]