Object Recognition based on Photometric Color Invariants T. Gevers & A. W. M. Smeulders Faculty of Mathematics & Computer Science, University of Amsterdam Kruislaan 403, 1098 SJ Amsterdam, The Netherlands E-mail:
[email protected]
Abstract
Our aim is to analyze and evaluate di erent color models to be used for the purpose of 3-D object recognition by color-metric histogram matching according to the following criteria: invariance to the geometry of the object and illumination circumstances, high discriminative power, and noise robustness. Assuming white illumination and dichromatic re ectance, we propose new color models c1 c2 c3 and l1 l2 l3 invariant to the viewing direction, object geometry and shading. Further, it is shown that l1 l2 l3 is also invariant to highlights. Further a change in spectral power distribution of the illumination is considered to propose a new photometric color invariant m1 m2 m3 for matte objects. To evaluate photometric color invariant object recognition in practice, experiments have been carried out on a database consisting of 500 images taken from 3-D multicolored man-made objects. On the basis of the reported theory and experimental results, it is shown that high object recognition accuracy is achieved by l1 l2 l3 and hue H followed by c1 c2 c3 and normalized colors rgb under the constraint of white illumination. Saturation S and m1 m2 m3 provide slightly worse object recognition accuracy under the same imaging conditions. Finally, it is shown that solely m1 m2 m3 is invariant to a change in illumination color.
1 Introduction Object recognition is an active research area in the eld of computer vision, [1], [5], [8], for example. Most of the work on object recognition is based on matching sets of geometric image features (e.g. edges, lines and corners) to 3-D object models and signi cant progress has been achieved, [5], [6], for example. However, most of the geometry-based schemes can only handle simple man-made objects due to the fact that geometric features are not always adequate for discriminatory and robust 3-D object recognition. Color provides powerful information for object recognition due to the 3-fold higher speci city of each pixel in the image. A simple and e ective recognition scheme is to represent and match images on the basis of color-metric histograms as proposed by Swain and Ballard [8]. The work makes a signi cant contribution in introducing color for object indexing. However, it has the drawback that it is highly dependent on the equality of illumination circumstances for the object and model in the actual recognition process. When the illumination circumstances are not equal, the object recognition accuracy degrades signi cantly. The color-based recognition method has been extended by Funt and Finlayson [3] to become illumination independent by indexing on an illumination-invariant set of color descriptors. The scheme fails, however, when images are heavily contaminated by shadows, shading and highlights. Furthermore, Healey and Slater [4] use illumination invariant moment invariants for object recognition. From the observations above, the choice which color features to use for the purpose of 3-D object recognition depends on the imaging conditions. To that end, in this paper, our aim is to analyze and evaluate various color models to be used for the purpose of object recognition by color-metric histogram matching according to the following criteria: 1. Robustness to a change in viewpoint; 2. Robustness to a change in object orientation; 3. Robustness to a change in the intensity and the direction of the illumination; 4. Robustness to a change in the color of the illumination; 5. High discriminative power; 6. Robustness to noise. The general application is considered of recognition of 3-D multicolored objects from 2-D color images. This paper is organized as follows. In Section 2, the dichromatic re ection model is discussed. The re ection model is used to analyze color models with respect to the rst four above mentioned criteria. From the analysis, new photometric invariant color features are proposed. The performance of photometric invariant color object recognition by color-metric histogram matching di erentiated for the various color features is evaluated and compared on an image database of 500 reference images in Section 3.
Material of this paper can be experienced on-line at http://www.wins.uva.nl/research/isis/zomax/.
2 Photometric Color Invariance
Commonly used well-known color spaces include: RGB , Y IQ, XY Z , I1 I2 I3 , rgb, xyz , U V W , L a b , Luv and ISH . However, a number of these color features are related to intensity I (L , W ), linear combinations of RGB (Y IQ, XY Z and I1 I2 I3 ) or normalized with respect to intensity rgb (xyz , U V , a b , uv). Therefore, in this paper, we concentrate on the following standard, essentially di erent, color features derived from RGB : intensity I (R; G; B ) = R+B +G, RGBp, normalized colors r(R; G; B ) = R+GR+B , g(R; G; B ) = R+GG+B , b(R; G; B ) = R+GB+B , ? G?B ) and saturation S (R; G; B ) = 1 ? min(R;G;B ) . hue H (R; G; B ) = arctan (R?G3()+( R?B ) R+G+B
2.1 The Re ection Model
Consider an image of an in nitesimal surface patch. Using the red, green and blue sensors with spectral sensitivities given by fR (), fG () and fB () respectively, to obtain an image of the surface patch illuminated by a SPD of the incident light denoted by e() , the measured sensor values will be given by Shafer [7]: Z
Z
C = mb (~n;~s) fC ()e()cb ()d + ms (~n;~s;~v) fC ()e()cs ()d
(1)
for C = fR; G; B g giving the C th sensor response. Further, cb () and cs () are the albedo and Fresnel re ectance
respectively. denotes the wavelength, ~n is the surface patch normal, ~s is the direction of the illumination source, and ~v is the direction of the viewer. Geometric terms mb and ms denote the geometric dependencies on the body and surface re ection respectively. Considering dichromatic re ectance and "white" illumination, then e() = e and cs () = cs . The measured sensor values are then:
Cw = emb (~n;~s)kC + ems (~n;~s;~v)cs
Z
fC ()d
(2)
for Cw 2 fRR w ; Gw ; Bw g giving the red, green and blue sensor response under the assumption of a white light source. kC = fC ()cb ()d is a compact formulation depending on the sensors and the surface albedo. If the integrated white condition holds (as we assume throughout the paper): Z
we have:
fR ()d =
Z
fG ()d =
Z
fB ()d = f
(3)
Cw = emb (~n;~s)kC + ems (~n;~s;~v)cs f
(4) One option to achieve integrated white is that the sensitivities of the camera fR (), fG () and fB () are relatively narrow-band lters, with spectral response be approximated by delta functions sR () = ( ? R ), sG () = ( ? G ) and sB () = ( ? B ).
2.2 Re ection with White Illumination
2.2.1 Photometric Invariant Color Features for Matte, Dull Surfaces Consider the body re ection term of eq. ( 4):
Cb = emb (~n;~s)kC
(5) for Cb 2 fRb ; Gb ; Bb g giving the red, green and blue sensor response of a matte, dull surface patch under the assumption of a white light source. According to the body re ection term, the color depends only on kC (i.e. sensors and surface albedo) and the brightness on factor emb (~n;~s). As a consequence, a uniformly painted surface (i.e. with xed kC ) may give rise to a broad variance of RGB values due to the varying circumstances induced by the image-forming process such as a change in object orientation, illumination intensity and position. The same argument holds for intensity I .
In contrast, normalized color rgb is insensitive to surface orientation, illumination direction and intensity as can be seen from: emb (~n;~s)kR r(Rb ; Gb ; Bb ) = em (~n;~ (6) = k + kkR + k s )( k + k + k ) b R G B R G B only dependent on the sensors and the surface albedo. Equal arguments hold for g and b. Saturation S is an invariant for the set of matte, dull surfaces illuminated by white SPD mathematically speci ed by: ~n;~s)kR ; emb (~n;~s)kG ; emb (~n;~s)kB ) = 1 ? min(kR ; kG ; kB ) S (Rb ; Gb ; Bb ) = 1 ? min(emb (em (7) (kR + kG + kB ) b (~n;~s)(kR + kG + kB ) Similarly, hue H is an invariant for matte, dull surfaces:
p
p
? emb (~n;~s)(kG ? kB ) ) = arctan (k ? k3(k)G+?(kkB? (8) H (Rb ; Gb ; Bb ) = arctan em (~n;~s3)(( k ? k ) + ( k ? k )) k ) b R G R B R G R B In fact, any expression de ning colors on the same linear color cluster formed by the body re ection vector in RGB -space are photometric color invariants for the dichromatic re ection model. To that end, we propose the following photometric color invariant model: ?
c1 (R; G; B ) = arctan( maxfRG; B g ); c2 (R; G; B ) = arctan( maxfGR; B g ); c3 (R; G; B ) = arctan( maxfBR; Gg ) (9) The color features c1 , c2 and c3 are the angles of the body re ection vector. Hence, the color features uniquely describe the body re ection vector in RGB -sensor space and consequently being a photometric color invariant for matte, dull objects as follows from: emb (~n;~s)kR kR b (~n;~s)kR c1 (Rb ; Gb ; Bb ) = arctan( maxfem (em ~n;~s)k ; em (~n;~s)k g ) = arctan( em (~n;~s) maxfk ; k g ) = arctan( maxfk ; k g ) (10) b
G
b
B
G B
b
G B
only dependent on the sensors and the surface albedo. Equal arguments hold for c2 and c3 . Obviously, in practice, the assumption of objects composed of matte, dull surfaces is not always realistic. To that end, the e ect of surface re ection (highlights) is discussed in the following section.
2.2.2 Photometric Invariant Color Features for Both Matte and Shiny Surfaces Consider the surface re ection term of eq. ( 4):
Cs = ems (~n;~s;~v)f
(11) for Cs 2 fRs ; Gs ; Bs g giving the red, green and blue sensor response for a highlighted surface patch with white illumination. Note that under the given conditions, the color of highlights is not related to the color of the surface on which they appear, but only on the color of the light source. Thus for the white light source, the surface re ection color cluster is on the diagonal grey axis of the basic RGB -color space corresponding to intensity I . For a given point on a shiny surface, the contribution of the body re ection component Cb and surface re ection component Cs are added together Cw = Cs + Cb . Hence, the observed colors of the surface must be inside the triangular color cluster in the RGB -space formed by the two re ection components. Because H is a function of the angle between the reference line and color point, all possible colors of the same shiny uniformly colored surface have to be of the same hue mathematically speci ed as: arctan
H (Rw ; Gw ; Gw ) =
p3((em (~n;~s)k + em (~n;~s;~v)f ) ? (em (~n;~s)k + em (~n;~s;~v)f )) G s B s b b (emb (~ n;~s)kR + ems (~n;~s;~v)f ) ? (emb (~n;~s)kG + ems (~n;~s;~v)f ) + (emb (~n;~s)kR + ems (~n;~s;~v)f ) ? (emb (~n;~s)kB + ems (~n;~s;~v )f ) =
p
p
? ? 3emb (~n;~s)(kG ? kB ) 3(kG ? kB ) = arctan arctan em (~n;~ (kR ? kG ) + (kR ? kB ) b s)(kR ? kG ) + (kR ? kB )
(12)
Only dependent on the sensors and the surface albedo. Obviously other color features depend on the contribution of the surface re ection component Cs and hence are sensitive to highlights. In fact, any expression de ning colors on the same linear triangular color cluster, formed by the two re ection components in RGB -space, are photometric color invariants for the dichromatic re ection model. To that end, a new photometric color invariant model is proposed uniquely determining the direction of the linear triangular color cluster:
R ? G)2 l1 (R; G; B ) = (R ? G)2 + ((R ? B )2 + (G ? B )2 ;
(13)
R ? B )2 l2 (R; G; B ) = (R ? G)2 + ((R ? B )2 + (G ? B )2 ;
(14)
G ? B )2 l3 (R; G; B ) = (R ? G)2 + ((R ? B )2 + (G ? B )2
(15)
the set of normalized color di erences which is, similar to H , an invariant for the set of matte and shiny surfaces as follows from:
? kG )2 l1 (Rw ; Gw ; Bw ) = (k ? k )2 + ((kkR ? R G R kB )2 + (kG ? kB )2 Equal argument holds for l2 and l3 .
(16)
2.3 Re ection with Colored Illumination 2.3.1 The Re ection Model
We consider the body re ection term of the dichromatic re ection model: Z
Cc = mb (~n;~s) fC ()e()cb ()d
(17)
for C = fR; G; B g where Cc = fRc ; Gc ; Bcg gives the red, green and blue sensor response of a matte in nitesimal surface patch under unknown spectral power distribution of the illumination. Assuming the material's albedo cb () to be independent of the wavelength of the incoming light e(), then cb () = cbC is a constant for the surface patch separated for each of the primary colors and varies from one material to another. Then we have:
Cc = mb (~n;~s)cbC
Z
fC ()e()d
(18)
for Cc = fRc ; Gc ; Bcg giving the red, green and blue sensor response of a matte in nitesimal surface patch. By simply lling in Cc in the color feature equations, we can see that all color feature values change with a change in illumination color. In other words, the distribution of color feature values derived from two images, the test image and the reference image, taken from the same object under di erent illumination color spectra will vary.
2.3.2 Color Constant Feature for Matte, Dull Surfaces
[3] proposes a simple and e ective illumination-independent color feature for the purpose of object recognition. The method runs short, however, when images are contaminated by shading and highlights. To that end, we propose a color constant feature not only independent of the illumination color but also discounting shading cues. The color feature is de ned by: ~x1 ~x2
m(C1~x1 ; C1~x2 ; C2~x1 ; C2~x2 ) = C1~x2 C2~x1 C1 C2
(19)
expressing the color ratio between two neighboring image locations, for C1 ; C2 2 fR; G; B g where ~x1 and ~x2 denote the image locations of the two neighboring pixels. Having three color components of two locations, color ratios obtained from a RGB -color image are: ~x1 G~x2 ~x ~x ~x ~x ~x1 ~x2 ~x1 ; B ~x2 ) = R 1 B 2 ; m3 (G~x1 ; G~x2 ; B ~x1 ; B ~x2 ) = G 1 B 2 (20) m1 (R~x1 ; R~x2 ; G~x1 ; G~x2 ) = R ; m 2 (R ; R ; B ~ x ~ x ~ x ~ x ~ 1 1 2 2 R G R B Gx2 B~x1
For the ease of exposition, we concentrate on m1 based on RG in the following discussion. Without loss of generality, all results derived for m1 will also hold for m2 and m3 . If we assume that the color of the illumination is locally constant (at least over the two neighboring locations from which ratio is computed), the color ratio is independent of the illumination color, and also a change in viewpoint, surface orientation and illumination intensity as follows from: R R (m~yb 1 (~n;~s)c~ybC1 1 fR ()e~y1 ()d)(m~yb 2 (~n;~s)c~ybG2 fG ()e~y2 ()d) (c~ybR1 )(c~ybG2 ) y ~ y ~ y ~ ~ y 2 1 2 1 m1 (Rc ; Rc ; Gc ; Gc ) = y~2 = R R (mb (~n;~s)c~ybR2 fR ()e~y2 ()d)(m~yb 1 (~n;~s)c~ybG1 fG ()e~y1 ()d) (c~ybR2 )(c~ybG1 )
(21)
where ~y1 and ~y2 are two neighboring locations on the object's surface not necessarily of the same orientation. In theory, when ~y1 and ~y2 are neighboring locations on the same uniformly painted surface, the color ratio will be 1. Except along color edges, assuming that the neighboring locations are at either side of the color edge, the value of the color ratio will deviate from 1. Thus, in theory, the number of distinct color ratio values is a measure for the amount of distinct color edges. Taking logarithms of both sides of equation 19 results for m1 in: ln m1 (R~x1 ; R~x2 ; G~x1 ; G~x2 ) = ln R~x1 + ln G~x2 ? ln R~x2 ? ln G~x1 The color ratios can be seen as di erences at two neighboring locations ~x1 and ~x2 in the image domain:
(22)
dm1 (~x1 ; ~x2 ) = ln R~x1 + ln G~x2 ? ln R~x2 ? ln G~x1
(23) When these di erences are taken between neighboring pixels in a particular direction, they correspond to nitedi erence di erentiation. The results obtained so far for m1 hold also for m2 and m3 , yielding a 3-tuple (Gm1 (~x), Gm2 (~x),Gm3 (~x)) denoting the gradient magnitude for every neighborhood centered at ~x in the image. For pixels on a uniformly painted region, in theory, all three components will be zero whereas at least one the three components will be non-zero for pixels on locations where two regions of distinct color meet.
3 Object Recognition: Experiments To evaluate photometric color invariant object recognition in practice, experiments have been carried out on a database consisting of 500 images taken from multicolored man-made objects in real world scenes. The image database and the performance of the recognition scheme can be experienced within the ZOMAX system on-line at http://www.wins.uva.nl/research/isis/zomax/. The following section is organized as follows. The data sets on which the experiments will be conducted are described in Section 3.1. Error measures are given in Section 3.2. Histogram formation and similarity measures are given in Section 3.3.
3.1 Datasets
The database consists of N1 reference images of domestic objects, tools, toys, food cans, art artifacts etc., all taken from two households. Objects were recorded in isolation (one per image) with the aid of the SONY XC-003P CCD color camera (3 chips) and the Matrox Magic Color frame grabber. The digitization was done in 8 bits per color. Objects were recorded against a white cardboard background. Two light sources of average day-light color are used to illuminate the objects in the scene. Objects were recorded at a pace of a few shots a minute. There was no attempt to individually control focus or illumination. They show a considerable amount of noise, shadows, shading, specularities and self occlusion. As a result, recordings are best characterized as snap shot quality, a good
representation of views from everyday life as it appears in home video, the news, and consumer photography in general. A second, independent set (the test set) of recordings was made of randomly chosen objects already in the database. These objects, N2 = 70 in number, were recorded again (one per image) with a new, arbitrary position and orientation with respect to the camera (some recorded upside down, some rotated, some at di erent distances (di erent scale)). In the experiments, all pixels in a color image are discarded with a local saturation and intensity smaller then 5 percent of the total range (this number was empirically determined by visual inspection); otherwise calculation of H , S , rgb, m1 m2 m3 , c1 c2 c3 and l1 l2 l3 becomes unstable. Consequently, the white cardboard background as well as the grey, white, dark or nearly colorless parts of objects as recorded in the color image will not be considered in the matching process.
3.2 Error Measures
For a measure of match quality, let rank rQi denote the position of the correct match for test image Qi , i = 1; :::; N2, in the ordered list of N1 match values. The rank rQi ranges from r = 1 from a perfect match to r = N1 for the worst possible match. Then, for one experiment, the average ranking percentile is de ned by: N2 X
N1 ? rQi )100% 2 i=1 N1 ? 1
r = ( N1
(24)
The cumulative percentile of test images producing a rank smaller or equal to j is de ned as:
X (j ) = ( N1
j X
2
k=1
(rQi == k))100%
(25)
where reads as the number of test images having rank k.
3.3 Similarity Measure and Histogram Formation
Color-metric histograms are created on the basis of each color feature de ned in Section 2 for each reference image in the image database by counting the number of times a discrete color feature occurs in the image. The color-metric histogram from the test image is created in a similar way. Then, the object recognition process is reduced to the problem to what extent histogram HQ derived from the test image Q is similar to a histogram HIk constructed for each reference image Ik in the image database. A similarity function D(HQ ; HIk ) is required returning a numerical measure of similarity between HQ and HIk . For comparison reasons in the literature, in this paper, D is expressed by histogram intersection [8]:
D(HQ ; HIi ) =
Q I ~k=1 minfH (~k ); H i (~k )g Nd
PNd
(26)
where ~k denote the bin index and Nd the number of non zero bins. Histograms are constructed on the basis of di erent color features representing the distribution of discrete color feature values in a n-dimensional color feature space, where n = 3 for RGB , rgb, l1 l2 l3 , c1 c2 c3 and m1 m2 m3 , and n = 1 for I , S and H . Histogram axes are partitioned uniformly with xed intervals. The resolution on the axes follows from the amount of noise and computational eciency considerations. We determined the appropriate bin size for our application empirically. This has been achieved by varying the same number of bins on the axes over q 2 f2; 4; 8; 16; 32; 64; 128; 256g and chose the smallest q for which the number of bins is kept small for computational eciency and large for recognition accuracy. The results show (not presented here) that the number of bins was of little in uence on the recognition accuracy when the number of bins ranges from q = 32 to q = 256 for all color spaces. Therefore, the histogram bin size used during histogram formation is q = 32 in the following. For each test and reference image, 3-dimensional histograms are created for the RGB , l1 l2 l3 , rgb, c1 c2 c3 and m1 m2 m3 color space denoted by HRGB , Hl1 l2 l3 , Hrgb , Hc1 c2 c3 and Hm1 m2 m3 respectively. Furthermore, 1-dimensional histograms are created for I , S and H denoted by HI , HS , and HH .
4 Results In this subsection, we report on the recognition accuracy of the matching process for N2 = 70 test images and N1 = 500 reference images for the various color features. As stated, white lighting is used during the recording of the reference images in the image database and the independent test set. However, the objects were recorded with a new, arbitrary position and orientation with respect to camera. In Fig. 1 accumulated ranking percentile is shown for the various color features. Accumulated ranking percentile for j 20 100 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 2 3 3 2 2 3 2 +2 +2 +2 +2 +2 +2 +2 +2 +2 +2 +2 +2 +2 +2 +2 + + + 3 + + 4 ? ? ? 4 80 ? ?
X (j )
60 40 20
4?
4?
4? b
? ? 4? 4 4
4 4 4? 4? 4? 4? ? ? 4 ? ? ? b b b 4 4 b b
b
b
b
b
b
b b
0
b
4 4 4 b
b
b
b
b
b
Xl1 l2 l3 (j ) Xrgb (j ) XH (j ) Xc1 c2 c3 (j ) Xm1 m2 m3 (j ) XS (j ) XRGB (j )
3 + 2 4 ?
b
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 j
?!
Figure 1: The discriminative power of the histogram matching process di erentiated for the various color features plotted against the ranking j . The cumulative percentile X for H , l1 l2 l3 , c1 c2 c3 , rgb, S , m1 m2 m3 and RGB is given by XH , Xl1 l2 l3 , Xc1 c2 c3 , Xrgb , XS , Xm1 m2 m3 and XRGB respectively. From the results of Fig. 1 we can observe that the discriminative power of l1 l2 l3 , H followed by c1 c2 c3 and rgb is higher then the other color features. Saturation S and color ratio m1 m2 m3 provides slightly worse recognition accuracy. As expected, the discrimination power of RGB has the worst performance due to its sensitivity to varying imaging conditions. Hence, for object recognition purposes under white lighting, color features l1 l2 l3 , H followed by c1 c2 c3 and rgb are most appropriate achieving a probability of respectively 99, 98, 94 and 92 perfect matches out of 100.
4.1 The E ect of a Change in the Illumination Intensity
The e ect of a change in the illumination intensity is equal to the multiplication of each RGB -color by a uniform scalar factor . To measure the sensitivity of di erent color feature in practice, RGB -images of the test set are multiplied by a constant factor varying over 2 f 0:5; 0:7; 0:8; 0:9; 1:0; 1:1; 1:2; 1:3; 1:5g. The discrimination power of the histogram matching process di erentiated for the various color features plotted against illumination intensity is shown in Fig. 2. As expected, RGB and I -color features depend on the illumination intensity. The further illumination intensity deviates from the original value (i.e. = 1), the worse discrimination power is achieved. Note that object are recognized randomly for r = 50. Furthermore, all other color feature are fairly independent under varying intensity of the illumination.
4.2 The E ect of a Change in the Illumination Color
The e ect of multiple light sources of di erent color distributions and a change in the illumination color is equal to the multiplication of each RGB image by an independent scalar factor [2]. In theory, all color features except color ratio m1 m2 m3 are sensitive to changes in the illumination color. To measure the sensitivity of the various color feature in practice with respect to a change in the color of the illumination, the R, G and B -images of the test set are multiplied by a factor 1 = , 2 = 1 and 3 = 2 ? respectively (i.e. 1 R, 2 G and 3 B ) by varying over f0:5; 0:7; 0:8; 0:9; 1:0; 1:1; 1:2; 1:3; 1:5g. The discrimination power of the histogram matching process di erentiated for the various color features plotted against the illumination color is shown in Fig. 3. For < 1 the color is reddish whereas bluish for > 1. As expected, only the color ratio m1 m2 m3 is insensitive to a change in illumination color. From Fig. 3 we can observe that color features H , l1 l2 l3 and c1 c2 c3 , which achieved best recognition accuracy under white illumination,
r
100 3 2 + 95 4? 90 85 80 75 70 c 65 b 60 55 50
Average ranking percentile r against . 2?bc 3 2 + + 3 + 3 + 3 + 3 + 3 3 + 4 ? 4? 4 b c c b
c c
c b
b c b
b
0.6
+ 3 4?
b c
rH (j ) rl1 l2 l3 (j ) rc1 c2 c3 (j ) rrgb (j ) rS (j ) rm1 m2 m3 (j ) rRGB (j ) rI (j )
0.8
?!
1
1.2
3 + 2 4 ?
b c
1.4
Figure 2: The discriminative power of the histogram matching process di erentiated for the various color features plotted against the illumination intensity represented by variation as expressed by the factor .
r
100 b 95 90 85 80 4? 75 70 2 65 + 60 55 50
Average ranking percentile r against tb . b +2b b b 4 ? ? 4 4 ? 2 2 + + 4? 4?
b
b
2 +
2 +
r H (j ) + r l1 l2 l3 (j ) 2 r c1 c2 c3 (j ) rS (j ) 4 rRGB (j ) ? rm1 m2 m3 (j ) b
4 2? +
0.6
0.8
tb
?!
1
1.2
1.4
Figure 3: The discriminative power of the histogram matching process di erentiated for the various color features plotted against the change in the color composition of the illumination spectrum.
see Figures 1 and 2, are highly sensitive to a change in illumination color followed by S and RGB . Even for a slight change in the illumination color, their recognition potential degrades drastically.
5 Discussion and Conclusion In this paper, various color models have been analyzed and evaluated for the purpose of object recognition by color-metric histogram matching under varying illumination circumstances. We theoretically showed that the color features H and the newly proposed l1 l2 l3 are independent of the viewing direction, highlights, surface orientation, illumination intensity and illumination direction but dependent on the illumination color. To that end, a new color constant feature m1 m2 m3 has been proposed measuring color ratios between neighboring pixels discounting the disturbing e ects of the illumination color, and also independent of a change in viewpoint, surface orientation, and illumination intensity. To evaluate the color features for robust and ecient photometric color invariant object recognition, experiments have been carried out on a database consisting of 500 images taken from multicolored man-made objects in real world scenes. On the basis of the above reported theory and experiments, it is concluded that l1 l2 l3 (invariant for both matte and shiny surfaces) followed by H are most appropriate to be used for photometric color invariant object recognition by color-metric histogram matching under the constraint of a white illumination source. When no constraints are imposed on the imaging conditions (i.e. the most general case), the newly proposed color ratio m1 m2 m3 is most appropriate.
References
[1] Besl, P.J. and Jain, R. C, Three-dimensional Object Recognition, ACM Computing Surveys, 17(1):75-145. [2] Finlayson, G. D., Drew M. S., and Funt B. V., Spectral Sharpening: Sensor Transformations for improved Color Constancy J. Opt. Soc. Am., 11(5), pp. 1553-1563, 1994. [3] Funt, B. V. and Finlayson, G. D., Color Constant Color Indexing, IEEE PAMI, 17(5), pp. 522-529, 1995. [4] Healey, G. and Slater D, Global Color Constancy: Recognition of Objects by Use of Illumination Invariant Properties of Color Distributions, J. Opt. Soc. Am. A, Vol. 11, No. 11, pp. 3003-3010, Nov 1995. [5] Rothwell, C. A., Zisserman, A., Forsyth, D. A. and Mundy, J. L., Planar Object Recognition Using Projective Shape Representation, Int. Journal of CV, Vol. 16, pp. 57-99, 1995. [6] Stein, F. and Medioni, G., Structural Indexing: Ecient 2-D Object Recognition, IEEE PAMI, Vol. 14, pp. 1198-1204, 1992. [7] Shafer, S. A., Using Color to Separate Re ection Components, COLOR Res. Appl., 10(4), pp 210-218, 1985. [8] Swain, M. J. and Ballard, D. H., Color Indexing, Int. Journal of CV, Vol. 7, No. 1, pp. 11-32, 1991.
