Image Retrieval Based on a Multipurpose Watermarking Scheme

2 downloads 2641 Views 293KB Size Report
presents a multipurpose watermarking scheme for image retrieval. First, several important ... The experimental results based on a database with 1000 images.
Image Retrieval Based on a Multipurpose Watermarking Scheme Zhe-Ming Lu1,2,*, Henrik Skibbe1, and Hans Burkhardt1,** 1 Institute for Computer Science, University of Freiburg Georges-Koehler-Allee 052, 79110 Freiburg i.Br., Germany {Zheminglu,Hans.Burkhardt}@informatik.uni-freiburg.de [email protected] 2 Department of Automatic Test and Control, Harbin Institute of Technology P.O. Box 339, 150001 Harbin, China [email protected]

Abstract. The rapid development of Internet and multimedia technologies has made copyright protection and multimedia retrieval be the two most important issues in the digital world. To solve these problems simultaneously, this paper presents a multipurpose watermarking scheme for image retrieval. First, several important features are computed offline for each image in the database. Then, the copyright, annotation and feature watermarks are offline embedded into all images in the database. During the online retrieval, the query image features are compared with the exacted features from each image in the database to find the similar images. The experimental results based on a database with 1000 images in 10 classes demonstrate the effectiveness of the proposed scheme.

1 Introduction With the development of computer, multimedia, and network technologies, two important issues have arisen nowadays. The first problem is that the amount of audiovisual information available in digital format has grown exponentially recently, which has resulted in information explosion and has exceeded the limit of human acceptability. Digitization, compression, and archival of multimedia information has become popular, inexpensive and straightforward. Subsequent retrieval of the stored information, however, might require considerable additional work in order to be effective and efficient. The second problem is that there is almost no limit for anyone to make lossless and unlimited copies of digital contents distributed over Internet and via CDROM, which is a major obstacle from the owner’s viewpoint for entering the digital world. Copyright protection has therefore been one of the most important issues in the digital world. Here we concern with the problems in image retrieval and copyright protection. On the first issue, we should use some effective retrieval methods to get the desired images. In typical content-based image retrieval (CBIR) systems [1], the visual contents of the images in the database are extracted and described by multidimensional feature vectors. The feature vectors form a feature database. To retrieve images, users provide the retrieval system with example images or sketched figures. The similarities /distances between the feature vectors of the query example or sketch * **

Room 01-046 Room 01-030

R. Khosla et al. (Eds.): KES 2005, LNAI 3682, pp. 573–579, 2005. © Springer-Verlag Berlin Heidelberg 2005

574

Zhe-Ming Lu, Henrik Skibbe, and Hans Burkhardt

and those of the images in the database are then calculated and retrieval is performed with the aid of an indexing scheme. Recent retrieval systems have incorporated users’ relevance feedback to modify the retrieval process. The existing image retrieval algorithms can be classified into two classes, i.e., low-level based and high-level (semantic) based. The low-level features include color [2], texture [3], shape [4] and spatial relationships [5]. Some retrieval methods based on semantic-level [6] have been proposed recently. On the latter issue, encryption may be one of solutions. However, conventional cryptographic systems permit only valid keyholders access to encrypted data, but once such data is decrypted there is no way to track its reproduction or retransmission. Over the last decade, digital watermarking has been presented to complement cryptographic processes. Invisible watermarks can be broadly classified into two types, robust and fragile (or semi-fragile) watermarks. Robust watermarks [7] are generally used for copyright protection and ownership verification because they are robust to nearly all kinds of image processing operations. In comparison, fragile or semi-fragile watermarks [8] are mainly applied to content authentication because they are fragile to most modifications. To fulfill copyright protection and content authentication together, multipurpose watermarking algorithms based on wavelet transform [9] and Vector Quantization [10] have been presented. In general, the above two issues are taken into account separately. This paper presents a simple multipurpose watermarking scheme to solve these two problems simultaneously. The main idea is to offline embed three watermarks, i.e. the copyright, the denotation, and features into each image in the database. During the online retrieval, we can query based on the copyright, the denotation and the features.

2 Feature Extraction Generally, we should extract the available features as many as possible. However, because the embedding of watermarks will affect the image quality, we should select the best representative features as less as possible. Here, we use three kinds of features, i.e., global invariant feature based on Haar Integral [11], statistical moments of the color histogram and Hu moments, which can be described in detail as follows. 2.1 Global Invariant Feature Invariant features remain unchanged when the image is transformed according to a group of transformations. In [11], a kind of global feature invariant to rotations and translations are presented. Given a gray-scale image M = {M( x ), x = ( x0 , x1 ),0 ≤ x0 < N 0 ,0 ≤ x1 < N1} (1) and an element g ∈ G of the group of image translations and rotations, the transformation can be expressed as:

 cos ϕ ( gM)( x ) = M( x ′) , with x′ =   − sin ϕ

sin ϕ   t0   N  x +    mod 0  cos ϕ   t1    N1 

Based on above definition, an invariant transformation T must satisfy T ( gM ) = T ( M ) , ∀g ∈ G

(2)

(3)

Image Retrieval Based on a Multipurpose Watermarking Scheme

575

For a given gray-scale image M and an arbitrary complex-valued function f (M ) , it is possible to construct such an invariant transformation T by the following Haar integral:

T [ f ]( M ) :=

1 G



G

N

f ( gM )dg =

N



0 1 1 f ( g (t0 , t1 , ϕ )M )dϕdt1dt0 2πN 0 N1 t0∫= 0 t1∫= 0 ϕ∫= 0

(4)

For discrete images, because we choose integers for (t0 , t1 ) and we use K steps for ϕ , we can obtain the following formula:

T [ f ]( M ) ≈

N 0 −1 N1 −1 K −1 1 2πk f ( g (t 0 , t1 ,ϕ = )M ) ∑ t0 =0 ∑t1 =0 ∑k =0 KN 0 N 1 K

(5)

We can apply the Monte-Carlo method for the calculation of multi-dimensional integrals. In this paper, we only use the global feature (i.e. one float-typed value) calculated from luminance component for a color image. 2.2 Statistical Moments of the Color Histogram Generally speaking, texture feature extraction methods can be classified into three major categories, namely, statistical, structural and spectral. In statistical approaches, texture statistics such as the moments of the gray-level histogram, or statistics based on gray-level co-occurrence matrix are computed to discriminate different textures. In this paper, statistical moments of the gray-level histogram are used to describe texture. Let z be a discreet random variable representing gray-levels in the range [0, L − 1] , where L is the maximum gray value. Let p( zi ), i = 0,1,..., L − 1 be a normalized histogram. Then the n-th moment with respect to the mean is given by:

µ n ( z ) = ∑i = 0 ( zi − m ) n p( zi ) L −1

(6)

where m is the mean value of z. The second-order moment, variance, is a measure of gray-level contrast. And the third-order moment is a measure of skewness of the histogram and the fourth-order moment is a measure of its relative flatness. In this paper, we use the mean value m and three moments µ2 ( z ), µ3 ( z ), µ4 ( z ) of each color component’s histogram in RGB color space, i.e., four float-typed values per color-component, as the features to be embedded. 2.3 Hu Moments Hu moments are a set of algebraic invariants that combine regular moments [12]. They are invariant under change of size, translation, and rotation. Hu moments have been widely used in pattern recognition and proved successful in various applications. These moments can be used to describe the shape information of the image. For the digital image M, in this paper, we only use first 4 moments for luminance component (i.e. four float-typed values) as follows:

φ1 = η 20 + η02 , φ2 = (η 20 − η 02 ) 2 + 4η112 φ3 = (η30 − 3η12 ) 2 + (η 03 − 3η 21 ) 2 , φ4 = (η 30 + η12 ) 2 + (η 03 + η 21 ) 2

(7)

576

Zhe-Ming Lu, Henrik Skibbe, and Hans Burkhardt

where η pq =

µ pq , µ pq = ∑ ∑ ( x0 − x 0 ) p ( x1 − x1 ) q M ( x 0 , x1 ) , x 0 = m10 / m00 , µ 00γ x x 0

1

x0

x1

x1 = m01 / m00 and m pq = ∑∑ x0p x1q M( x0 , x1 ) .

3 Offline Multipurpose Watermarking In our system, before we perform the online retrieval, we first embed three watermarks into each image in the database. Because these three watermarks possess different purposes, so we call our watermarking is a multipurpose watermark scheme. The first watermark is a copyright watermark, which is used for copyright protection. The second one is the annotation watermark, which is the name of the image or the semantic meaning of the image. The last one is the feature watermark, which is composed of the extracted features. These watermarks are all robust watermarks In our watermarking system, the famous dither modulation method [13] is used to embed three watermarks into DCT block of each image’s luminance component (we transform from RGB space to YUV space, only the Y component is used, and UV components remaining unchanged). The watermark’s bits are distributed image orientated, so that cropping the image will crop the watermark the same way as the hostimage. The dither modulation algorithm encodes each watermark bit in a middlefrequency DCT coefficient. Assume a DCT coefficient DCTi in an even interval representing the bit ‘0’ and in an odd interval representing the bit ‘1’. An interval is even, if, by a given ∆ , DCTi / ∆  =even, else odd. If a DCT coefficient is already in an interval which represents its bit, we move it into the middle of the interval, else we move it into the middle of the nearest interval next to its current interval. In our system, we embed twelve bits into each 8 × 8 DCT block. The key problem is how to embed a float-typed value into the image. For simplicity, we embed the 4 bytes (i.e., 32 bits) which represents the float value directly into the image. However, a problem arising is the error in bits, even only one bit, may make the extracted value very different from the embedded one. To overcome this problem, we embed each float value multiple times into the image, and during the extraction, we determine the extracted bits by the majority cases. The second problem is how to guarantee the security of the watermarks, especially for the copyright watermark. Here we use the interleaving embedding technique. We randomly select the columns of DCT blocks to embed one of the watermarks, maybe the first column is used to embed the feature watermark, and the second column may be used to embed the annotation watermark, and so on. The selection scheme can be viewed as an embedding key.

4 Online Image Retrieval with Various Query Strategies After embedding the watermarks into all the images in the database, we can perform the online retrieval now. Here we can use several kinds of queries. The most normal one is feature-based. And we can also use copyright-based and annotation-based, or we can use the combination query schemes.

Image Retrieval Based on a Multipurpose Watermarking Scheme

577

For the feature-based query scheme, we first extract the features from the query image. Note that if the query image is without a feature watermark, we should compute the feature online for it. Then we compare the query feature vector with the feature watermark in each database image to find the most desired similar images. For the feature-and-annotation-based query scheme, we can only search the desired images with the same annotation (in general, it is the semantic of the image). For the feature-and-copyright-based query scheme, we can only search the desired images with the same copyright. For the feature-annotation-copyright-based query scheme, we can only search the desired images with the same copyright and with the same annotation.

5 Experimental Results We perform the experiments based on a test database with 1000 images in 10 classes, each class including 100 images. They are with size of 384 × 256 or 256× 384 . During the offline process, we extract and embed three kinds of features. For the global invariant feature, we use n=10000 and f (M ) = M (0,1) ⋅ M ( 2,0) . We embed a 48× 48 sized binary copyright watermark, 17 float-typed feature values and 16bytes annotation text. In the experiment, we view the annotation text as part of the feature watermark. We combine the feature watermark and the copyright watermark to construct the watermark to be embedded for each database image. Then, we embed the watermark to the corresponding image. After we obtain a watermarked image database, we can perform the online retrieval with various queries. For the query based on features, we show an example of retrieval results in Fig. 1. The average precision and recall for each class is shown in Table 1. The average precision and recall for each class can be obtained as follows: First, we randomly select ten images from the class. Then, we use each image to be the query image. For each query image, we get the “recall” by obtaining the ratio of returned images in this class in the first 100 returned images, and find the position of the first returned image which is not in this class and divide it by 100 to obtain the “precision”. After getting ten recalls and ten precisions, we average them to get the average recall and precision. For image retrieval system, the most important operation to which our system should resist is the high-quality compression, while other attacks are not so important. Under the condition that we can extract the watermark more than 80% similar to the original embedded information. In previous sections, we have mentioned that the extracted feature watermark should be no bit loss. With regard to this, in the case of 100% recovery, the experimental results show that the average lowest JPEG compression quality factor to which the feature watermark can resist is 90.

6 Conclusions This paper present an image retrieval system based on multipurpose watermarking scheme. The advantages of this scheme lie in three aspects. First, the system embeds the features in the images, and we need no extra space to save the feature data. Secondly, the image file can be copied to other database without recomputing corre-

578

Zhe-Ming Lu, Henrik Skibbe, and Hans Burkhardt

sponding features. Thirdly, the image is also copyrighted and annotated, and the retrieval system can give the originality of the image and the semantic of the image.

Fig. 1. The retrieval interface of our system Table 1. The average recall and precision for each class Class number 1 2 3 4 5 6 7 8 9 10

Semantic People Beach Building Bus Dinosaur Elephant Flower Horse Mountain Food

Recall 0.27 0.14 0.20 0.24 0.67 0.35 0.24 0.48 0.18 0.20

Precision 0.02 0.01 0.01 0.02 0.29 0.02 0.04 0.12 0.02 0.01

Acknowledgement This work was supported by Alexander von Humboldt Foundation Fellowship(Germany), ID: CHN 1115969 STP and the National Natural Science Foundation of China under grant 60272074 and the Spaceflight Innovation Foundation of China under grant [2002]210-6.

References 1. Long, F. H., Zhang, H. J., Feng, D. D.: Fundamentals of Content-based Image Retrieval, in Multimedia Information Retrieval and Management - Technological Fundamentals and Applications. Editor: Feng, D. D., Siu, W.C., Zhang, H. J., Springer-Verlag, Berlin Heidelberg New York (2003) 2. Smith, J. R.: Color for Image Retrieval. In: Image Databases-Search and Retrieval of Digital Imagery, Castelli V, Beagman L D, eds. John Wiley & Sons, Inc. Ch. 11, (2002)285311

Image Retrieval Based on a Multipurpose Watermarking Scheme

579

3. Sebe, N., Lew, M. S. Texture Features for Content Based Retrieval, In: Principles of Visual Information Retrieval, Lew M S, ed. Springer, Ch. 3, (2001) 51-85 4. Lee, K. M., Street, W. N.: Incremental Feature Weight Learning and Its Application to A Shape Based Query System. PRL, Vol. 23, No.7, (2002)265-274 5. Sciascio, E. D., Donini, F. M., Mongiello, M.: Spatial Layout Representation for Query by Sketch Content-based Image Retrieval, PRL, Vol. 23, No. 13, (2002)1599-1612 6. Lew, M. S.: Features Selection and Visual Learning, In: Principles of Visual Information Retrieval. Lew, M. S., ed. Springer, Ch. 12, (2001)297-318 7. Wang, Y., Doherty, J. F., Van Dyck, R. E.: A Wavelet-based Watermarking Algorithm for Ownership Verification of Digital Images. IEEE Trans. Image Processing, Vol. 11, No.2, (2002)77-88 8. Jaejin, L., Chee, S. W.: A Watermarking Sequence Using Parities of Error Control Coding for Image Authentication and Correction. IEEE Trans. Consumer Electronics, Vol.46, No. 2, (2000) 313-317 9. Lu, C. S., Liao, H.Y.M.: Multipurpose Watermarking for Image Authentication and Protection. IEEE Trans. Image Processing, Vol. 10, No. 10, (2001)1579-1592 10. Lu, Z. M., Xu, D. G. and Sun, S. H.: Multipurpose Image Watermarking Algorithm Based on Multistage Vector Quantization. IEEE Transactions on Image Processing. Vol. 14, No. 6, (To be published). 11. Siggelkow, S., Burkhardt, H. Fast Invariant Feature Extraction for Image Retrieval, in State-of-the-Art in Content-Based Image and Video Retrieval. Veltkamp, R. C., Burkhardt H., Kriegel, H. P. ed. Kluwer Acadenic Publishers (2001) 12. S.O. Belkasim, M. Shridhar, and M. Ahmadi. Pattern recognition with moment invariants: a comparative study. Pattern Recognition, 24(12):1117–1138, 1991. 13. Chen, B., Wornell, G. W.: Digital Watermarking and Information Embedding Using Dither Modulation. IEEE Second Workshop on Multimedia Signal Processing, (1998)273-278