Embedding Fuzzy Logic in Content Based Image Retrieval Constantin Vertan, Nozha Boujemaa INRIA Rocquancourt – Projet IMEDIA Domaine de Voluceau BP105 Rocquancourt, 78153 Le Chesnay Cedex, France Constantin.Vertan,
[email protected]
Abstract
of fuzzy theory used for their definition. Four classes of fuzzy processing algorithms were identified in [15]: This paper focuses on the possible embedding of the un- crude fuzzy, fuzzy paradigm-based, fuzzy aggregational certainty regarding the colors of an image into histogram- and fuzzy inferential. The crude fuzzy techniques (as sugtype descriptors. The uncertainty naturally arises from gested by their name) are characterized by the use of num both the quantization of the color components and the bers in the interval (typically being weights attached human perception of colors. Fuzzy histograms measure to the objects of the problem universe), that are compatthe typicality of each color within the image. We define ible with a semantical description. The fuzzy paradigmvarious fuzzy color histograms following a taxonomy that based techniques use (implicitly or not) a fuzzy model of classifies fuzzy techniques as crude fuzzy, fuzzy paradigm the objects of the problem universe. The fuzzy aggregabased, fuzzy aggregational and fuzzy inferential. For tional techniques use a fuzzy model for the combination these fuzzy sets we must develop appropriate similarity (aggregation) of the individual objects in order to derive measures and distances. We propose a class of such dis- the outcome. Finally, the fuzzy inferential techniques are tances, derived from the fuzzy set equality and which we based on the inference mechanism that is applied with reparticularize according to various T-norms(fuzzy logical spect to a set of fuzzy rules that describe both the object ”or” operators). We also prove that the metric nat- model and the object aggregation. This is the most comurally arises as a distance for fuzzy sets, considering the plex model, that can be viewed as an instance of a rulefuzzy set symmetric difference. based expert system. In this contribution we will revisit the use of color histograms from a fuzzy-logic perspective: the value of each 1 Introduction bin must represent the typicality of the color within the Content-based image retrieval (CBIR) became a must in image rather than its probability. Thus, in section 2, we the last decade. Powered by the explosive development will define new type of histograms as membership funcof the Internet and the Web and the continuously cheaper tions of the colors within the image (and thus as fuzzy digital imagining devices and technologies, applications sets) and some new resemblance measures and distances such as digital libraries, image archives, video-on-demand for fuzzy sets will be introduced in section 3. Finally, secand specific image databases emerge as a real-life fact. tion 4 will present some experiments and conclusions. The basic idea of the CBIR process is to compactly describe an image by a digital signature and then match query images to the most resemblant image within the database according to the similarity of their signatures. 2 Fuzzy image histograms Traditionally, the content description is done (for either global or partial queries) according to the notions of color and texture. Thus the signatures are color distributions As already mentioned in the introduction, the histogram (histograms [13], color moments [9], color coherence vec- (probability density function) of the values within an imtors [10]), second-order, spatially constrained color distri- age (either color or gray-scale) is largely used for the butions (color correlograms [4], edge correlograms [3]) content-based image retrieval. The retrieval means that or classical textural descriptors (Fourier or wavelet coeffi- the images within the databases are selected according to cients, Markov random field models, etc.). the resemblance of their histogram to the histogram of the The search for new, fuzzy color distributions will fol- query image. Given a color image , of size by low the taxonomy proposed in [15] for the classification pixels, characterized by the color at location , i.e. of fuzzy processing techniques according to the amount , the color distribution (histogram) of the color
set. That means that we will associate to any color a Lukasiewicz function, M ?ON 9PQ R and for any color # ! % " $ , + * " $ &' & D of the color universe, M ? D is the resemblance degree
1 2 0 * 4 5 3 7 8 6 (1) of the color D to the color . Further assuming a fuzzy
(*) - (*)/. model that is not machiavelic (as discussed in [1]), we The values are normalized in order to sum to (as re- must logically admit a relation between the color resemquired by the definition of a probability density function) blance degree and the distance that separates the colors and is the Dirac impulse function. The value of each and D , and, more particularly, that the resemblance degree bin is. thus the number of image pixels having the color , decreases as the inter-color distance increases. The natural choice (according to the image processing or, after normalization, is the probability that the color set
is given by
traditions) is to impose a smooth decay of the resemblance function with respect to the inter-color distance. Still, we TSU color space was supposed to may remember that the 2.1 Crude fuzzy histograms offer the equivalence between the perceptual inter-color In order to construct such a function we must provide a distance and the Euclidean distance between their tristimsemantical description of the significance of the numbers ulus representations. Even more, the notion of JND asattached to each color in the color space 9 and a method sures visual equivalence for colors that are closer than 2.3 [12]. Practical considerations and the analytical simplifithat assures that the numbers are well within the range. The immediate approach is to slightly modify the cation of the computational expressions impose the use of construction of the normal color histogram. We may de- an unified formula for the resemblance degree. A linear fine the concept of typicality of a color within the given descent would require little computation but could conimage as the importance of the given color with respect tradict the smooth descent principle. A Gaussian operator to its relative presence. In fact, this description relates the (3) could be a more appropriate choice. \ typicality of the colors to the area that they occupy, and D ? = < ^ _ (3) M D V W XZY%[*\] 0#` XE [*a\ b H thus to their probability of occurrence. But the most typical color must have a typicality degree of 1, regardless its occurrence probability and thus we define this fuzzy This fuzzy color model enables to enlarge the influence of a given color to its neighboring colors, according to histogram as (2): the uncertainty principle (of not being certain that a quan $ tized color has not erroneous changed the original color) (2) :=?A@BC EDF *35 76G9IH and the perceptual similarity. This means that each time a color is found in the image, it will influence all the This is in fact the usual color histogram (1) but with a dif- quantized colors from 9 (and thus all the histogram bins) ferent normalization condition; instead of the probability according to their resemblance to the color . Numeri density function normalization condition ( J ?A@BLK D ), cally, this could be expressed as: we normalize by the mode ( := ? @ BC D ) of the color !d"*$ +"%$ \ c & & & M ? D ghji102 V distribution. The new normalization does not change much the intrinsic properties of the histogram, since it ? @ BLK e (*) f (*) . preserves its shape (including the “holes” or empty bins & ? (4) ?A@BLK D M D H that appeared mainly as quantization effect). We may use this approach as a transitional step in changing the probabilistic description of the color description by a fuzzy The expression in (4) is the linear convolution between description. Also, such a histogram can be useful if fuzzy the usual color histogram and the fuzzy color model (supdistances are to be embedded in the retrieval system, since posing that the model is color-independent, that is M ? the fuzzy distances are defined on fuzzy sets and the nor- M ?@ %35 Ra D 6k9 ). Thus we can compactly write: \ l M ? H mal histogram cannot be viewed as one. (5) appears in the image.
2.2 Fuzzy paradigm-based histograms As ennounced in the description of the taxonomic categories, the fuzzy paradigm-based techniques are constructed according to a fuzzy model of the objects of the universe. In the case of color images, the objects are the colors within the image or the possible colors within the set 9 . The model assumes that any color is a fuzzy
The convolution expresses the histogram smoothing, provided that the color model is indeed a smoothing, lowpass filtering kernel; such an approach was proposed in [7] for gray scale images, but the grays model was a triangular function. The use of the Gaussian shape from (3) as color model produces the smoothed histogram, proposed by many authors (that can be traced back to [8]) as a mean for the reduction of quantization errors.
fuzzy sets defined over the same problem universe , that means {A N P R . We will also denote by The colors are already described by a fuzzy model. As MV {AB ª¡I # £5a5a £*j . Thus, the gensmoothed, paradigm-based, fuzzy histogram; more colors and MsV5 that are actually present in the image (but are close to the eral expression of the fuzzy resemblance degree of the two real colors) have good chances to receive a typicality of 1. sets can be thus expressed as (7). Using m the Zadeh operators, the fuzzy histograms be(7) MV {Azx yB h 0 £5a £5as ExzyB { £5 02{ £*j°, H (9)
S S
For any T-norm ¨ , ¨* if and only if and thus the condition from (9) becomes:
> xzyB
[
2 0 { £5a{ £5a± [
02{ £5a{ £5a±
S
³] ² ´ x¶µZ;>· ] y ¸¸a¹ 35£v6vwH
[16],
In the case of crisp sets, the symmetrical difference is defined as:
Í
(10)
Í
(11)
OÎ#70t tÏ»
