Similarity-based Image Retrieval System Using PIFS Codes. Takanori Yokoyama. Toshinori Watanabe. Ken Sugawara. Graduate School of Information Systems.
Similarity-based Image Retrieval System Using PIFS Codes Takanori Yokoyama Toshinori Watanabe Ken Sugawara Graduate School of Information Systems University of Electro-Communications Chofugaoka 1-5-1, Chofu-shi, Tokyo, 182-8585 JAPAN E-mail: {yokotaka, watanabe, sugawara}@sd.is.uec.ac.jp Abstract We propose a new retrieval system of images using PIFS codes. In PIFS encoding, a compression code contains mapping information between similar regions in the same image. These mapping information can be treated as vectors, and representative vectors can be generated using them. Representative vectors describe the feature of the image. Hence, the similarity between images is calculable from representative vectors directly. This similarity is applicable to image retrieval. In this report, we explain this scheme and demonstrate its usefulness experimentally. Keywords : image retrieval, similarity-based, fractal image compression, PIFS codes
1
Introduction
A lot of image retrieval systems have already been studied, and some of them are used commercially [1]. These many systems use the database of original image set. However, there are a few systems which use the database of compressed image set directly. In this paper, we suggest a new retrieval technique that uses compression codes, especially we use fractal image compression method here. This compression is a relatively recent technique and based on the selfsimilarity of images. Exploiting the robust property of this compression method, we have developed a new similarity-based retrieval system.
2
Related Work
In general, image retrieval process can be described as follows. At first, a retrieval system performs feature extraction from an original image set and creates feature indices of their image sets. These indices are registered in a database and treated as objects.
In retrieval, the system extracts the feature of a query image which a user requested, and compares the feature of the query image with those of the database. As a result, the system shows several candidate images to the user. In this process, the performance of the system greatly depends on the process of feature extraction from original image sets. If a system uses the database of the compressed codes directly, it is relieved of the feature extraction from original image set. Compression operation reduces redundancy in the original data [2]. Recently, it has became to be noticed that compression data can provide some useful feature of the original image. We can find some image researches exploiting compression domain such as Wavelet and DCT [1]. Here, we concentrate on the fractal image compression method.
3 3.1
Similarity-based Image Retrieval Fractal Image Compression
Fractal Image Compression was devised by Michel F. Barnsley [4]. The main idea of this coding is the observation that self-similarity is found within images and is extractable. It employs the affine transformation. The compression is executed by an Iterated Function System (IFS) that consists of a set of these transformations performed on various parts of the image. Therefore, Fractal Image Compression is called as an IFS [3]. Barnsley’s method contains no automated encoding algorithm, so Jacquin improved Barnsley’s method and developed an automatic compression method [4]. As shown in Figure 1, Jacquin defined large regions (which are called as domain regions) and smaller regions(which are called as range regions) in the image. Each range region should be a similar contracted copy of the relevant domain region. Affine transformations are used to represent among their relations. This method is called Partitioned IFS (PIFS) coding
because the image is partitioned in the process of compression [5].
3.3
Image similarity based on PIFS Code
As mentioned before, fractal image compression searches for domain regions which are similar to range regions. Therefore, we can characterize the image by focusing on the mapping vectors from the range to the domain. Important points of this method are : 1. The relation between the domain and the range does not change so long as the image does not change largely. 2. The mapping vectors of an image are different from those of the other image in case that these images are completely largely. Figure 1: Similar Regions. The small region is called range and the large regions is called domain. This method can decode the image of arbitrary resolution from the code. From this property, the system can create the image of arbitrary resolution.
3.2
Representative vectors
In the PIFS code, the relation of similar regions is represented as follows, dx rx ry = wi dy (1) dz rz where, rx and ry represent the position of a range region (the north west corner), and rz represents the brightness value of this region. dx, dy and dz are those of the domain region. wi is the contraction coefficient of the transformation. The PIFS code itself contains some feature of the image. We extract a new feature out of the PIFS code as follows (Figure 2).
The similarity between two images is calculated by the following process. Here we denote two images as A and B, and denote representative vector sets of them as rv(A) and rv(B). 1. Determination of a vector pair A vector is selected from set rv(A), and corresponding vector that satisfies the following formula is searched in rv(B). f (ai ) = arg min ai − b ∈ rv(B) b∈rv(B)
(2)
This process is executed for all vectors in ai ∈ rv(A). 2. Calculation of the similarity From the correspondence of vector sets, the similarity between A and B is calculated by the following formula. s(A, B) =
|rv ∗ (B)| |rv(A)|
(3)
where f (rv ∗ (B)) = {f (ai )|ai ∈ rv(A)} and |X| denotes the cardinal number of X.
1. Mapping vectors We extract range region (R) and domain region (D) correspondence in the form of a vector from R to D.
From this definition, s(A, B) takes the value between 0 and 1. It means that if the two images are completely different, the value of s(A, B) becomes 0, and if they are identical images, the value becomes 1.
2. Representative vectors A lot of vectors are generated by (1). By clustering these vectors, we obtain a new vector set. This is a representative vector set and we consider it as a new feature of the image.
3.4
In our proposal, we use the representative vectors in image retrieval. In the next subsection, this process is described in detail.
Distance between two images
For more accurate retrieval, we define a new distance dsim (A, B) including s(A, B). At first, we calculate the distance between vector set A and corresponding vector set B as follows. (4) d(rv(A), rv(B)) = min ai − b b∈rv(B)
0
0
100
100
200
200
300
300
400
400
500
500
600
600 0
(a) original image
100
200
300
400
500
600
700
800
0
(b) mapping vectors
100
200
300
400
500
600
700
800
(c) representative vectors
Figure 2: Generating a set of representative mapping vectors
Using this distance and the similarity in Equation (3), we define dsim (A, B) as follows. dsim (A, B) = d(rv(A), rv(B))
1 s(rv(A), rv(B))
(5)
0
1.0 1.3 1.6 1.9
100
y
similarity
200
Introduction of this distance enables us to obtain the proper similarity between two images.
300 400 500 0
100 200 300 400 500 x
To confirm the effectiveness of our proposed method, we carried out two types of experiments. The experiments were performed on a PC with dual CPU(Pentium3-550MHz) and its OS was Linux 2.2.17.
4.1
Robustness of the proposed method
At first, we confirmed the robustness of this system for image zooming, rotation and brightness change (histogram shrinking). Figure 3 shows the relation between the magnification rate and the similarity. The similarity remains high even if the image is zoomed. Note that the value keeps high until the magnification is 1.6. We also confirmed the robustness for rotation and brightness change (Figure 4).
4.2
Image retrieval
Next, we will show the result of image retrieval. We prepared a database composed of 960 digital images (8 bit gray scale, 320x240 pixel), which contains natural sceneries, buildings, dolls, and so on. They are preprocessed and described by representative vectors. Table 1 shows three outputs of the image retrieval. Images in the left column are the query images, and 4 images in the right columns are the output images obtained by the proposed method.
(a) representative mapping vectors for various magnification rates
(b) similarity vs. magnification rate
Figure 3: Similarity dependence on magnification rate
0.94 0.92 0.9 0.88 0.86 0.84 0.82 0.8 0.78 0.76 0.74
similarity
Experiments
similarity
4
0.94 0.92 0.9 0.88 0.86 0.84 0.82 0.8 0.78 0.76 0.74 1.0 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 magnification rate
0
5
10 15 20 25 30 35 40 45 degree
(a) rotation
0.94 0.92 0.9 0.88 0.86 0.84 0.82 0.8 0.78 0.76 0.74 256
224
192
160
128
96
maximum value of histogram
(b) brightness change
Figure 4: Similarity dependence on rotation and brightness change
The values under each image denotes the distance dsim and the similarity s between the image and the query image. This result shows that the proposed method works well as an image retrieval system.
5
Conclusion
We proposed a new method for image retrieval using PIFS codes. In our method, we exploit the mapping relations between the range region and domain
Table 1: Retrieval Results Rank Query Image
2
3
4
5
0.0 (1.0)
49.1 (0.80)
55.8 (0.74)
56.1 (0.77)
56.4 (0.77)
0.0 (1.0)
50.6 (0.78)
51.7 (0.75)
53.5 (0.75)
56.0 (0.73)
0.0 (1.0)
48.4 (0.73)
69.1 (0.63)
70.8 (0.67)
74.7 (0.62)
region in an image, and describe the relations by vectors from the range to the domain, we called them as mapping vectors. Next, some representative vectors are extracted out of them. They are used as the feature vectors of the image. In retrieval process, we calculate the similarity between images by using a newly defined similarity measure for two representative vector sets. The effectiveness of the proposed method was confirmed by several experiments. Now we are evaluating the efficiency of the system for large scale of databases.
References [1] T. Huang, Y. Rui (1997), “Image Retrieval: Past, present, and future”, International symposium on Multimedia Information Processing. [2] D. Salomon (2000), Data compression: the complete reference, Springer-Verlag New York. [3] M. Turner, J. Blackledge (1998), Fractal Geometry in Digital Imaging. Academic Press. [4] B. Wohlberg, G. Jager (1999), “A Review of the Fractal Image Coding Literature”, IEEE Transac-
tions on Image Processing, vol. 8, no. 12, December 1999. [5] Y. Fisher (1995), Fractal image compression: theory and application, Springer-Verlag New York. [6] T. Yokoyama, T. Watanabe, K. Sugawara and et al (2002), “A Structural Similarity Extraction Method using PIFS code” (in Japanese), Technical Report of IEICE [7] T. Yokoyama, T. Watanabe, K. Sugawara (2002) “Feature Extraction and Retrieval of Images Using Correlation of PIFS Codes” (in Japanese), ITE Techonical Report
