Oct 28, 2003 - (73) Asslgneer Samsung Electr?nlcs (30-, Ltd- (KR). 6,324,695 ...... x3 y3. Z3. White x0 y0. Z0. In Table 2, the White illuminant indicates a scene.
US006639628B1
(12> Ulllted States Patent
(16) Patent N6.=
Lee et al.
US 6,639,628 B1
(45) Date of Patent:
(54)
APPARATUS AND METHOD FOR PROCESSING COLOR IMAGES
(75)
Inventors:
4,805,010 A 5,495,428 A
_
-
~
_
*
2
i
,
EZEEgPKLm’ U612??? (KR)’ ’
O ang
8/1996
Miyadera ............... .. 348/223.1 51ml - - - - - - - - -
,
----
oge
.
5,691,772 A * 11/1997 Suzuki .................. .. 348/2231
’ _
5,805,213 A
Yang'seock Se_°>Se°u1(KR)> JBOIIg-YBOP K1111, SHWOH (KR) _
2/1989 Shroyer et al. 2/1996 SChWaftZ
5,550,587 A Seong-deok Lee, SuWon
Oct. 28, 2003
*
5,920,358 A * 5,956,015 A *
_
6,160,581 A
(73)
Asslgneer Samsung Electr?nlcs (30-, Ltd- (KR)
(*)
Notice:
9/1998
Spaulding et a1. ..... .. 348/2221
7/1999 Takemura ................. .. 348/655 9/1999 Hino ........................ .. 345/600
* 12/2000
Higashihara et al.
6,459,449 B1 * 10/2002
Subject to any disclaimer, the term of this
U.S.C. 154(b) by 0 days.
..
6,476,877 B2 * 11/2002 Kihara et al. ............. .. 348/650
patent is extended or adjusted under 35
_
_
* cued by examlner
(21) App1_ No; 09/425,674
Primary Examiner—Lin Ye
_
(74) Attorney, Agent, or Firm—Burns, Doane, SWecker &
(22) Flledl
(30)
..... .. 348/364
6,324,695 B1 * 11/2001 Lee et a1. ................... .. 725/38
Oct. 22, 1999
Mathis, LL13~
Foreign Application Priority Data
OCL 22, 1998
(57)
ABSTRACT
(KR) .......................................... .. 98-44351
An apparatus and method for processing Color images are
(51)
Int. Cl? ........................ .. H04N 9/73, H04N 17/00;
provided. The apparatus includes a calarcanvarsian unit for
(52)
US. Cl. .................. .. 348/223.1; 348/655; 348/184;
version unit includes a ?rst White-transforming means for
345/603; 345/604
White-transforming signals in RGB space to signals in a
(58)
Field Of Search ......................... .. 348/223.1 222.1 348/272 277 231 655 577 184 588
predetermined Color Space using N predetermined illumi nant colors, and a second White-transforming means for
6096 5/00
’
’ 23,9 64,9. 34’5/603’ 604; ’
(56)
’
’
White-transforming signals in RGB space. The color con
White-transforming the signals in the predetermined color space to the signals in RGB space using M predetermined
References Cited
color temperatures of the display means, so that a natural color image or an image of a desirable color can be obtained
U-S~ PATENT DOCUMENTS 4,616,253 A
4,685,071 A
Without information With respect to the illuminant.
10/1986 Hashimoto et 81.
8/1987 Lee
28 Claims, 9 Drawing Sheets
/ 32w ———————— ‘A?
F ********* 771
|
| Cont |
l
I FIRST
i-
IMAGE
—
COLOR
—"SECOND
Cont
:R
202+ MEMORY FRAME %-SAMPLING ?-CONVERSIQA/?. FRAME lg UNlT B5 UN|T BO MEMORY __
: Cont
l
COLOR
2
MONITOR
lI | |
Cont
i Cont
206
5‘ 3
U.S. Patent
Oct 28, 2003
r.
l | | I
| I l
.65H
[/l]“ F103
US 6,639,628 B1
Sheet 1 0f 9
@
U.S. Patent
Oct. 28,2003
Sheet 3 0f 9
US 6,639,628 B1
FIG. 3 / 32
-
-
-
\‘l'
-
_
-
-
-
-
-
-
3211,
I Cont
-
-
_
-
-
-
~
'-
1
|
23 |
Cont
RGB—>XYZ
342
I
Cont
| I
RGB-XYZ I Cont
: Cont
25:
-
1/34
l
Cont
-
206
Cont 343
U.S. Patent
Oct. 28,2003
Sheet 4 0f 9
US 6,639,628 B1
/206 442
'
:_____ __*_T_/44
l
l
l I
I I
I I
I I
I
I
I
I
COnt__C—:—- MEMORY Rs 1 GS |
58 I
“T
FIRST ‘
I X :
MEMORY SECOND
MATRIX
I
Y
I
MATRIX
OPERATION
I
Z
I
OPERATION
UNIT
-I——I"'| I
UNIT
COI'I’LJD } R0 I
G0
I 50
; ' T
I______\_______I
L______. _____I
424
444
U.S. Patent
Oct. 28,2003
Sheet 5 0f 9
US 6,639,628 B1
II
I} _ _ _ _ _ _ _ _ ~ _ _
8%1! 88IT 8%1T 8% 85 88IT
.UEm
I m? t
I. I. 0mm
00m
om?
2 rm mm KwPm
cow
HElMOcJ ElNiWHH
“59203;
U.S. Patent
Oct. 28,2003
Sheet 6 6f 9
8?
QI;8mWm VH.
O _ _ _
.28R8%
Eoizmo.lu @ r
US 6,639,628 B1
.2 .2 LE AS
.3 id
SONOWe2.8@dQ0
x
U.S. Patent
Oct. 28,2003
Sheet 7 0f 9
US 6,639,628 B1
ndNdm6#650mdwd
.UEN.
_
x
0m Tc F
|@.@ Iwd Ind |©.O lmio 230 Ind |N.O
o
TO.
o
U.S. Patent
Oct. 28,2003
Sheet 8 0f 9
US 6,639,628 B1
FIG. 8A I
START
)
SELECT A PLUPALITY OF ILLUMINANT COLORS AS RERRENTATIVE
ILLUMINANT COLORS
“T802
806
IS COLOR EMPERATURE OF MONITOR KNOWN?
YES
/808
/81O
SET KNOWN COLOR SET PREDETERMINED THREE COLOR TEMPERATURES AS REPRESENTATIVE TEMPERATURE AS REPRESENTATIVE COLOR COLOR TEMPERATURES OF MONITOR TEMPERATURES OF MONITOR
CALCULATE ROB-to-XYZ WHITE—~ ____82O TRANSFORMATION MATRIX
COEFFICIENTS USING EACH REPRESENTATIVE ILLUMINANT COLOR
PERFORM RGB~tO~XYZ WHITE— TRANSFORMATION USING THE MATRIX COEFFICIENTS
O
$822
U.S. Patent
Oct. 28,2003
Sheet 9 0f 9
US 6,639,628 B1
FIG. 8B
R
CALCULATE XYZ—tO—RGB WHITE TRANSFORMATION MATRIX COEFFICIENTS USINC EACH REPRESENTATIVE COLOR TEMPERATURE OF MONITOR PERFORM XYZ-to-RGB WHITE TRANSFORMATION USING
824
826
MATRIX COEFFICIENTS DISPLAY PLURALITY OF WHITE TRANSFORMED IMAGES IN THE MONITOR
828
SELECT PREFERRED IMAGE
830
US 6,639,628 B1 1
2
APPARATUS AND METHOD FOR PROCESSING COLOR IMAGES
devices. A user obtains color coordinates of the illuminant color emitted from the illumination source by the sensors, and sets camera reference illuminant as White balance mea
sured by the sensor based on the obtained color coordinates
This application claims priority under 35 U.S.C. §§119
to thereby obtain an optimum scene image. HoWever, this
and/or 365 to 98-44351 ?led in Korea on Oct. 22, 1998; the
entire content of Which is hereby incorporated by reference.
method has problems of increased costs due to use of the
BACKGROUND OF THE INVENTION
distant areas Where sensors cannot be installed. Also, there is a restriction that the camera should be a high-class goods,
1. Field of the Invention The present invention relates to an apparatus and method for processing images, and more particularly, to an apparatus
optical sensor, and of dif?culties in adopting to images from
10
images formed by input devices other than a pre-speci?ed camera cannot be accomplished.
and method for processing color images. 2. Description of the Related Art In the ?eld of color image processing and color devices,
A conventional method for estimating illuminant colors 15
When someone photographs a scene using a color image input device such as a camera and reproduces the photo
having independent brightness using specularly re?ected 20
image corresponds With the color of the actual scene. To achieve this, it is an essential point that one obtain the color of an illuminant for illuminating an object (scene
illuminant), then a reference White point of the image input device corresponds With the color of the illuminant, and that
25
one processes the illuminant color in the White point of an
image to be reproduced and in the color temperature of an
display device, Which reproduce the image. In most general cameras, all circuits of the cameras are adjusted using one or tWo reference illumination sources,
30
therefore, there are problems that When an object is photo graphed under illuminants other than the reference
illuminant, the color of the image reproduced by the image output device is different from that of the scene image, and that as the difference of illuminant colors or color tempera 35 tures betWeen the reference illuminant of the camera and the scene illuminant becomes greater, the difference in colors
betWeen the reproduced image and the object becomes of the object corresponds With the reference illuminant of the camera, in the case that the illuminant of the object is different from the color temperature of a cathode ray tube of a monitor on Which the image photographed by the camera is displayed, color variation due to the difference of the color
of an actual scene image With the color of reproduced image, a user adjusts color tone, luminosity and color saturation of the three primary colors of red, green and blue (RGB) on the
monitor With the help of image processing softWare, such as
45
5,495,428. According to the method, image data is projected onto a chromatic space histogram to ?nd a straight line 50
Which is an impetus for determining an illuminant color, straight lines With respect to each of clusters are calculated on the chromatic space, and the chromatic values at posi tions Where the straight lines converge are obtained. Although detecting a change of boundaries in the above
55
method is not complicated method of, straight lines With
image.
respect to all clusters must be calculated Which requires much time. Also, When the colors of objects in the image are similar, the clusters may make a lump to thereby deteriorate precision, and stable illuminant colors require the existence
nants Which used for adjustment of colors, by color adjust ments.
A method for matching the color of the scene illuminant
installed in the camera, or are provided as independent
comparing, and determining must be iterated. Also, there is a problem that various effective highlights for calculation of illuminant colors must exist on the chromaticity. Another conventional method is disclosed in US. Pat. No.
Photoshop of Adobe Co., so that the user can see that the
With the White balance of the image input device, Which are essential for matching the colors of tWo images, is partially adopted by the camera. For instance, a method using an optical sensor is disclosed in US. Pat. Nos. 4,616,253 and 4,805,010. In these patents, optical sensors for detecting illuminant color components irradiating an object are
excessively long processing time.
from various boundary points, linear approximating,
colors of the scene image match those of the reproduced
HoWever, it is generally dif?cult to obtain an optimum RGB rate for color matching betWeen tWo images, and more dif?cult to overcome problems due to imbalance of illumi
light (highlight) from an image, an image is converted to a color coordinate space having a chromaticity coordinate. Then, a color boundary Where both color saturation and color tone are most abruptly changed, is detected, and the illuminant color is detected using data sets around the boundary due to a change of the color saturation. Here, for determining the boundary causes from the change of the color saturation or the change of the color tone, the data sets around both sides of the boundary point are collected, and then the collected data sets are linearly approximated. If the inclinations of the straight lines obtained by the data sets collected from both sides are equal, it is determined that the boundary is caused by the color saturation and uses the data sets for detecting illuminant color, and a variable determined as the illuminant color is obtained by a path of intersecting points of the straight lines obtained from the data sets around the boundary due to the change of the color saturation. But, the above method has a disadvantage that it requires an Also, there are other disadvantages in that it is not easy to collect the data sets from either side of each of the boundary points, and that because the data sets are processed in unit of a boundary point unit, collecting of data on both sides
greater. Also, there is a problem that even if the illuminant
temperature may occur. As a common conventional method of matching the color
regardless of the type of image input device is disclosed in US. Pat. No. 4,685,071. According to this method, illumi nant colors are detected from the color image itself. That is, according to the method, in order to detect a change of colors
graphed image data using an image output device such as a monitor, or When a display image is doWnloaded from the
Internet, it is important that the color of the reproduced
Which is possible to perform a White balance control. Further, there is a disadvantage that color matching of color
60
of strong highlight information in the image. FIG. 1 is a block diagram of the structure of a typical
image processing system for illustrating various processes of generating and reproducing an image. Referring to FIG. 1, a scene of a person standing next to a tree irradiated by an 65
arbitrary illuminant. The scene can be photographed by a camera set to an arbitrary reference illuminant having color
temperature of 5000 K, or 7500 K. Also, an image printed
US 6,639,628 B1 3
4
in the form of a silver salt photograph can be read by an image scanner having an illuminant of color temperature of 2800 K. Also, the image may be doWnload from an Internet site in Which case the image input device cannot be de?ned.
version unit further comprises: a second memory for storing information of the matrices for performing White transformation the signals in the predetermined color space to the signals in a RGB color space using M predetermined
The image data obtained by the above-mentioned methods
color temperature for display means, and outputting the
are displayed on an arbitrary image output device by a computer or an image processing device. Here, color tem
information in response to a second control signal, and a
second matrix operation unit for operating the matrices in accordance With the information output from the second
peratures of monitors may be set differently. That is, the scene image is sole, but images displaying on the monitor may have different colors due to difference from that of the
10
image input device, and a difference in color temperatures betWeen image data having an arbitrary light source and a monitor. SUMMARY OF THE INVENTION
15
It is an object of the present invention to provide an
memory. The predetermined color space is a standard color space and the predetermined space is a color space Which has a relationship that the magnitude of the transformed
signal in the color space and the magnitude of signal in original color space is linear. The N predetermined representative illuminant colors of the color conversion unit are obtained by selecting at least one color temperature from a group consisting of 2800 K,
apparatus for processing color images by Which the differ
4300 K, 5000 K, 5500 K, 6500 K, and 7500 K, Which can
ence in colors betWeen a actual scene and a reproduced
be easily distinguished by person’s eyes, and are frequently used in ordinary.
image of the scene can be stably matched regardless of the
components of the image, and the types of the image input
20
device. It is another object of the present invention to provide a
method for processing color images Which is used in the
apparatus for processing color images. Accordingly, to achieve the ?rst object, the apparatus for
25
processing color images, Which receives RGB signals output from an image input device and processes the received RGB signals to thereby output images to a display means, com prising: a color conversion unit for White-transforming sig nals in RGB space; Wherein, the color conversion unit includes, a ?rst White-transforming means for White transforming signals in RGB space to signals in a predeter
30
transforming the signals in the predetermined color space to the signals in RGB space using M predetermined color
35
Preferably, the color image processing apparatus further received RGB signals and outputting the buffered signals; an
40
image sampling unit for receiving the RGB signals output from the ?rst frame memory means and doWn-sampling the received RGB signals and further outputting a doWn sampled RGB signals; and a second frame memory means
for buffering the doWn-sampled RGB signals and outputting the buffered doWn-sampled RGB signals.
45
It is also preferable that the color conversion unit com
55
information output from the ?rst memory. The color con
The above objects and advantages of the present invention Will become more apparent by describing in detail a pre ferred embodiment thereof With reference to the attached
draWings in Which: FIG. 1 is a block diagram of a structure of a typical image 60
processing system for illustrating various processes of gen erating and reproducing an image; FIG. 2 is a block diagram of a structure of a color image
processing apparatus according to the present invention;
signals in a predetermined color space using N predeter mined illuminant colors, and outputting the information in unit for operating the matrices in accordance With the
mended by CIE, CIEXYZ, CIEUVW, and color spaces adopt UV chromaticity coordinate space and Y, R-Y, B-Y BRIEF DESCRIPTION OF THE DRAWINGS
predetermined color temperature.
response to a ?rst control signal, and a ?rst matriX operation
(a) is a color space Which has a relationship that the
coordinate space.
nant colors.
Preferably, the color conversion unit comprises a ?rst memory for storing information of the matrices for perform ing White-transformation of the signals in RGB space to the
space to the signals in RGB space using M predetermined color temperatures. It is also preferable that the N predetermined representa tive illuminant colors in step (a) are obtained by selecting at least one color temperature from the group consisting of 2800 K, 4300 K, 5000 K, 5500 K, 6500 K, and 7500 K, Which can be easily distinguished by men’s eyes, and are frequently used in routine life. It is further preferable that the predetermined space in step
selected from the group of standard color spaces recom 50
predetermined color space using N predetermined illumi It is further preferable that the color conversion unit further comprises: a second operation unit including M matriX operators connected in parallel to calculate matrices for White-transformation of the signals in the predetermined color space to the signals in RGB space using each M
space using N predetermined illuminant colors; and (a-2) White-transforming the signals in the predetermined color
magnitude of transformed signal in the color space and the magnitude of signal in original color space is linear. The predetermined color space in step (a) is a color space
prises: a ?rst operation unit including N matriX operators connected in parallel to calculate matrices for White transformation of the signals in RGB space to signals in
determining the illuminant color corresponding to the selected image in step (c) as optimum illuminant color.
Preferably, the step (a) includes: (a-1)White-transforming
temperatures of the display means. comprises: a ?rst frame memory means for buffering the
color images includes the steps of: (a) White-transforming signals in RGB space using N predetermined eliminant colors; (b) displaying a plurality of images by the White transformed signal on the display means; (c) selecting the most visually preferred image from the images; and (d)
signals in RGB space to signals in a predetermined color
mined color space using N predetermined illuminant colors; and a second White-transforming means for White
The color conversion unit converts a light source using
Formula 9, When the color temperature of the illuminant is knoWn. To achieve the second object, the method for processing
FIG. 3 is a block diagram of a structure of a color 65
conversion unit of the device of FIG. 2. FIG. 4 is a block diagram of another structure of a color conversion unit of the device of FIG. 2.
US 6,639,628 B1 6
5
The second operation unit 34 includes three matriX opera tors 341, 342 and 343. The matriX operators 341, 342 and 343 are connected in parallel. The matriX operators 341, 342
FIG. 5 shows a spectral distribution of representative
illuminant adopted by a color image processing apparatus according to an embodiment of the present invention; FIG. 6 is a chromaticity graph of color temperatures of
and 343 calculate matrices Which convert a signal in XYZ space to a signal in RGB space using three different color temperatures of a display means.
representative illuminant adopted by the color image pro cessing device according to the embodiment of the present
The matriX operators 321, 322, 323, 324, 325, and 326 and the matriX operators 341, 342, and 343 perform matriX operation in response to the control signal Cont from the
invention; FIG. 7 is a chromaticity graph of color temperatures of
monitors adopted by the color image processing device according to the embodiment of the present invention; and FIG. 8 is a ?oWchart shoWing the major steps of the color image processing method according to the embodiment of the present invention. DESCRIPTION OF THE PREFERRED EMBODIMENTS
10
15
Referring to FIG. 2, a color image processor 20 according to the present invention includes a ?rst frame memory 202,
an image sampling unit 204, a color conversion unit 206, and a second frame memory 208.
In the operation of the color image processor 20, the ?rst frame memory 202 buffers ?rst red, green and blue (RGB) signals Rin, Gin and Bin of the input three primary colors
and outputs the buffered signals. The image sampling unit
preferred image. By this, the illuminant color of illuminant and the color temperature of display means used When 25
204 receives the ?rst RGB signals output from the ?rst frame memory and doWn-samples the received ?rst RGB signals to (height), and the resolution of a color monitor 210 is 1280
images are displayed on the color monitor 210, the doWn
sampling rate of the image sampling unit 204 is determined according to a control signal ContiaddriA of a microcom 35
In other Words, the image sampling unit 204, converts the resolution of the input image to a degree appropriate for the output device and outputs the image With the converted
betWeen RGB space and rg chromaticity space. MeanWhile, It is undesirable that embodied as matriX operators for every matrices are used. FIG. 4 shoWs a more 45
operator 444 With respect to the representative illuminant 55
color are doWnloaded from the second memory 442 in
response to a microcomputer control signal ContiD. Thus, the ?rst matriX operation unit 42 converts signals in RGB space to signals in XYZ space, and the second matriX operation unit 44 converts signals in XYZ space to signal in
stored images to the monitor 210. FIG. 3 shoWs the detailed structure of the color conver
sion unit 206. Referring to FIG. 3, the color conversion unit 206 includes ?rst operation unit 32 and second operation unit 34. The ?rst operation unit 32 includes siX matriX
RGB space.
NoW, it Will be described hoW to select representative illuminants and to perform White-transformation.
operators 321, 322, 323, 324, 325 and 326. The matriX operators 321, 322, 323, 324, 325 and 326 are connected in parallel. The matriX operators 321, 322, 323, 324, 325 and to a signal in XYZ space using the siX representative illuminant colors.
preferred; eXample of the structure of the color conversion unit 206. Referring to FIG. 4, the color conversion unit 206 includes a ?rst operation unit 42 and a second operation unit 44. The ?rst operation unit 42 includes a ?rst memory 422 and a ?rst matriX operation unit 424. The matriX coef?cients for the ?rst matriX operation unit 424 With respect to the representative illuminant color are doWnloaded from the ?rst memory 422 in response to a microcomputer control signal comic, and the matriX coefficients for the second matriX
micro-computer (not-shoWn) respectively, then outputs the
326 calculate matrices Which convert a signal in RGB space
that relationship betWeen a magnitude of transformed signal in a color space and a magnitude of original in a color space is linear. As the color space, CIEUVW color space can be
adopted. Also, as the predetermined color space, color coordinate adopted by Y, R-Y and B-Y, Which have been Widely used for the industry of color images may be used. Alternatively, it is possible to perform White-transformation
resolution to the color conversion unit 206. In the image
The second frame memory 208 stores 18 images in response to the second control signal ContiaddriB from a
are determined to be the optimum illuminant color of
transformation betWeen RGB color space and CIEXYZ color space. HoWever, the signal in RGB color space can be transformed to a signal in a another predetermined color space. Preferably, the predetermined space is a color space
(Width)>
