IEEE Transactions on Consumer Electronics, Vol. 49, No. 4, NOVEMBER 2003

1438

Fast POCS Based Post-Processing Technique for HDTV Yoon Kim, Chun-Su Park, and Sung-Jea Ko, Senior Member, IEEE Abstract — In this paper, we present a novel postprocessing technique based on the theory of the projection onto convex sets (POCS) in order to reduce the blocking artifacts in digital high definition television (HDTV) images. By detecting and eliminating the undesired high-frequency components, mainly caused by blocking artifacts, we propose a new smoothness constraint set (SCS) and its projection operator in the DCT domain. In addition, we propose an improved quantization constraint set (QCS) using the correlation of DCT coefficients between adjacent blocks. In the proposed technique, the range of the QCS is efficiently reduced as close to the original DCT coefficient as possible to yield better performance of the projection onto the QCS. Computer simulation results indicate that the proposed schemes perform better than conventional algorithms. Furthermore, we introduce a fast implementation method of the proposed algorithm. The conventional POCS-based postprocessing techniques require the forward/inverse discrete cosine transform (DCT/IDCT) operations with a heavy computational burden. To reduce the computational complexity we introduce a fast implementation method of the proposed algorithm that does not perform DCT/IDCT operations. Estimates of computation savings vary between 41% and 64% depending on the task. Index Terms — Postprocessing, POCS, SCS, QCS. I.

INTRODUCTION

Image compression techniques based on BDCT have been the popular choice in both still and moving image coding standards, such as JPEG [1], H.26x [2], and MPEG [3]. Even in digital HDTV, image signals are compressed by BDCT and then transmitted via broadcasting channels. For the sake of efficient implementation, an image is partitioned into spatially adjoining blocks that are processed independently without considering inter-block correlations. At low bit rates, this approach inevitably causes two primary visual artifacts in BDCT decompressed images. One is the grid noise in monotone areas, which consists of the introduction of artificial edges at the 8 × 8 DCT block boundaries. The other is the staircase noise that distorts edge shapes. Various algorithms have been proposed to improve the quality of block-coded images in the decoder without increasing the bit rate. Among them, a linear space-invariant low-pass filtering [4] is well known, due to its simplicity. However, since the local characteristics of image vary spatially, it is known that linear, space-invariant filtering is inadequate. Therefore, many adaptive techniques were proposed to solve these problems. Better results were achieved Contributed Paper Manuscript received September 1, 2003

using directional smoothing filters with a stronger effect on the direction orthogonal to the block boundaries [5]. A more sophisticated nonlinear space-variant filter was proposed in [6], where image blocks were classified and each was processed through an appropriate filter. The iterative approaches based on the theory of POCS can be thought of as special cases of image restoration [7]. Zakhor [8] first tried to apply the theory of POCS to the reduction of blocking artifacts of block DCT-coded images and suggested an iterative algorithm. The iterative algorithm can be justified as a method of steepest descent iteration in a constrained minimization problem. She imposed two constraints on the coded image: SCS (Cs), which denotes the set including the images free from blocking artifacts, and QCS (Cq), which denotes the set including the images close to the original image as shown in Fig. 1. Notice in Fig. 1 that the Pq and Ps are the projection operators onto Cq and Cs, respectively. In [8]-[13], POCS-based techniques have been also proposed that are effective in reducing blocking artifacts. In these methods, the projections onto the SCS's are performed through the pixel domain, and the projections onto the QCS's are performed in the DCT domain. These methods require the iterative DCT/IDCT operations with heavy computational burden. Therefore, in spite of better performance than other postprocessing techniques, the POCS-based postprocessing techniques have not been widely used in real applications. f0

k =1

k ≠1

Ps onto Cs

Pq onto Cq

Converge?

No

fk Yes

Fig. 1. Block diagram of the postprocessing technique based on the theory of POCS.

In this paper, we propose a new postprocessing algorithm based on the theory of POCS and its fast implementation in order to reduce the blocking artifacts in HDTV images. Note that a block, which is diagonally shifted one pixel apart from the grid of BDCT, includes the boundary of the original block. If any blocking artifact is introduced along the block boundary, this shifted block will produce frequency characteristics different from those of the original block. Thus, a comparison of frequency characteristics of these two overlapping blocks can detect the undesired high-frequency components, mainly caused by the blocking artifacts. By eliminating these undesired high frequency components, a robust smoothing projection operator can be obtained. Moreover, we propose an improved QCS using the correlation of DCT coefficients between adjacent blocks. In the proposed

0098 3063/00 $10.00 © 2003 IEEE

Y. Kim et al.: Fast POCS Based Post-processing Technique for HDTV

technique, the range of the QCS is efficiently reduced to as close to the original DCT coefficient as possible to yield the better performance of the projection onto the QCS. Furthermore, we introduce a fast implementation method of the proposed algorithm. The conventional POCS-based postprocessing techniques require DCT/IDCT operations with high computational complexity. On the contrary, the proposed method is fully operated in the frequency domain without the need of performing the DCT/IDCT operations. Estimates of computation savings vary between 41% and 64% depending on the task. Moreover, the proposed technique converges within a few iterations. The paper is organized as follows. The proposed postprocessing scheme is presented in Section II. In Section III, the proposed fast implementation and computational complexity are described. Section IV presents and discusses the experimental results. Finally, our conclusions are given in Section V. II. PROPOSED POST-PROCESSING TECHNIQUE

1439

bij ( N − 1, N − 1)} , where bij ( x, y) = f (iN + x + 1, jN + y + 1) . Note that block bij includes both right vertical and bottom horizontal block boundaries of

aij .

Let the DCT of aij ( x, y ) be Aij (u , v ) . Then, for N = 8 , the DCT/IDCT transform is given by

Aij (u , v) = α (u )α (v)∑ x =0 ∑ y =0 aij ( x, y ) 7

× cos[

7

π (2 x + 1)u π (2 y + 1)v ]cos[ ], 16 16

(1)

aij ( x, y ) = ∑ u =0 ∑ v =0 α (u )α (v) Aij (u, v) 7

× cos[

7

π (2 x + 1)u π (2 y + 1)v ]cos[ ], 16 16

(2)

where

A. Proposed SCS and Its Projection In this subsection, we study the blocking artifacts by analyzing frequency characteristics of the original block and the block shifted by one pixel. Then, we introduce a smoothness constraint set and its projection operator in the DCT domain to reduce blocking artifacts by compensating the effects of the added block boundary with the POCS method.

α (k ) =

1 , N 2 , N

k = 0, (3)

otherwise.

In the same manner, bij ( x, y ) and Bij (u , v ) can be

N

formulated. To reduce the blocking artifacts in the decoded image, we first observe the characteristics of Bij , the DCT

aij

block of

bij . The Bij has more nonzero AC coefficients when

compared with

N

bij

Aij , the DCT block of aij , since Bij includes

the high frequency block boundaries (blocking artifacts). In other words, nonzero coefficients of Bij occur at higher orders in the zig-zag scan than highest nonzero valued location of Aij . Figs. 3 (a), (b), and (c) show an example of and the projected element Fig. 2.

aij

and

bij .

First, let us denote an mN × nN input image and its pixel by f and f (⋅, ⋅) , respectively, where m and n are positive integers. In block-based transform coding, f (⋅, ⋅) is composed of nonoverlapping N × N

blocks

{aij (0, 0),L , aij ( N − 1, N − 1)}

aij ( x, y ) =

f (iN + x, jN + y ) , i = 0,1,L , ( m − 2) and j = 0,1,L , ( n − 2) . Fig. 2 shows a block bij , that is diagonally shifted one pixel from the block

aij where bij = {bij (0, 0),L ,

Bij which will be defined later,

respectively. Based on this observation, we establish an algorithm for reducing the blocking artifacts by proposing a smoothness constraint set and its projection operator. Let us define a closed convex set

CBij as

CBij = { f | bij ⊂ f , NZ ( Bij ) ≤ K Bij } ,

aij 's, where aij = with

Aij , Bij ,

(4)

where

K Bij = NZ ( Aij ) + VBij ,

(5)

1440

IEEE Transactions on Consumer Electronics, Vol. 49, No. 4, NOVEMBER 2003

Bij

Bij

Aij 840

0

0

0

0

0

0

0

860 61 -20 12 -12

0

-60

0

0

0

0

0

0

37 -76 14

0

0

0

0

0

0

0

0

-6

5

0

0

0

0

0

0

0

0

9

-8

5

0

0

0

0

0

0

0

0

-7

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

NZ( Aij ) = 5

Highest order ,

-9

4

-2

0

0

860 61 -20 12 0 0 37 -76 14 0

-12 -1

0

1

-1

0

-6

5

0

0

0

0

0

0

-1

4

-2

0

0

9

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

-1

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

VBij

5

(a)

NZ (⋅) represents the order of the last nonzero DCT

coefficient in the zig-zag scan. In Fig. 3 (a), NZ ( Aij ) = 5 . In (5),

0

0

VBij is a factor representing how many DCT coefficients

measure of the local detail at

bij . It is known that noise near

edges is less visible than noise appearing in flat (gradually changing) region. The former should be carefully reduced so that the information about edges will not be damaged, while the latter should be eliminated completely. It is also known that the blocking artifacts are more visible in bright areas rather than in dark areas of the image. Therefore, in bright monotone areas with high

µ and small σ , VBij should be

small such that more high-frequency components are eliminated. On the other hand, since a block

bij with large σ

contains edge areas, the high-frequency components should be preserved. A function that captures this property is

VBij ∝

σ , 1+ µ

(6)

µ in the denominator to avoid mathematical difficulties when µ = 0 . Since µ and σ are where 1 is added to

0

0

0

0

0

proportional to the DC coefficient and a absolute sum of major AC coefficients of the DCT block, respectively, we define

VBij by

VBij = η

∑

and the human perceptual properties.

serves as a measure of the local brightness, and variance is a

0

Aij . (b) Bij . (c) Bij .

VBij should be chosen based on the local statistics of the image bij can be treated as a random variable with mean µ and variance σ . The mean

0

(c)

of Bij do contribute to the blocking artifacts. Thus,

In general, each pixel intensity in

0

KBij 10, NZ( Bij ) = 10

(b) Fig. 3. Example of Ps onto Cs: (a)

and

6

where

5 i =1

| ACi |

1 + DC

,

(7)

η is a scaling factor, and DC and ACi are DCT

coefficients of Bij . From Fig. 3 (b), using (7), we obtain

VBij = 5 and K Bij =10 for

η = 10 . By observing the

smoothness constraint set in (4), the projection operator onto that set can be easily found. By discarding the nonzero coefficients of Bij at higher orders than obtain the projected element

K Bij in (5), we can

Bij as shown in Fig. 3 (c).

B. Proposed QCS In conventional POCS-based postprocessing technique, the QCS ( Cq ) is commonly defined as the quantization interval for the original DCT coefficient. Since the quantization parameters are transmitted from the encoder, the quantization interval for each DCT coefficient is known to the decoder. The task of its projection operator ( Pq ) is to move the DCT coefficient into the quantization interval. The conventional QCS of (i , j ) block in an mN × nN input image is identical to the closure of the quantization region defined as

Y. Kim et al.: Fast POCS Based Post-processing Technique for HDTV

1441

Reconstruction level D ecision level

D ecision level

D C T dom ain

x min

Aij

(u,v)

Aij (u,v)

Aijmax (u,v) Fig. 5. Concept of the proposed QCS in a DCT coefficient location.

Q uantization interval

Fig. 4. Conventional quantization constraint set.

QCSij (u , v) = [ Aijmin (u, v), Aijmax (u, v)] ,

QCS in every coefficient location is proposed and can be expressed as (8)

Qij (u , v) min = − A ( u , v ) A ( u , v ) ( u , v ) , λ ij ij ij 2 Amax (u, v) = A (u , v) + λ (u, v) Qij (u , v) , ij ij ij 2

where

Qij (u , v) min , Aij (u, v) = Aij (u , v) − 2 Amax (u, v) = A (u , v) + Qij (u , v ) , ij ij 2 where of

(11)

(9) where

λij (u, v) =

Aij (u, v) is the reconstruction level at location (u , v )

(i , j ) th block and Qij (u , v) is the quantization step for

M , κ ij (u, v) + M

(12)

λij (u, v) is the value estimated from BDCT

coefficient in each coefficient location as shown in Fig. 4. The

The coefficient

projection onto the QCS in (8) of an input value A%ij (u, v ) is

coefficients in the same location

defined as the clipping operation over that region, and can be written as

neighboring blocks, and determined in the range from 0.5 to 1 . M is the total number of neighboring blocks, 8. The concept of the proposed QCS in a DCT coefficient location is delineated in Fig. 5.

Pij (u, v) Aijmin (u, v), = Aijmax (u, v), % Aij (u , v), where

if A%ij (u , v) ≤ Aijmin (u , v), if Aijmax (u , v) ≤ A%ij (u , v),

(10)

(u , v ) of its eight

DC

DC

DC

DC

DC

DC

otherwise,

Pij (u, v) represents the projection onto the QCS. Since

the value projected onto the SCS deviates from the decoded value, it should be restored as close to an initial value as possible. This is what the projection onto the QCS does. Since the conventional QCS is fixed as a maximum size of the range where an original DCT value can be included, the result of the projection onto the SCS almost exists in the range of the QCS. Thus, the projection onto the conventional QCS often fails to play the role of restraining the divergence caused by that onto the SCS. If the range of the QCS can be sized down efficiently, then the projection onto that QCS can yield better performance. In order to reduce blocking artifacts while preventing overblurring of the postprocessed image, a new QCS with which we can adaptively control the size of the conventional

A ij (u,v) DC

DC

DC

The Blocks of Same Region Fig. 6. Block classification in DCT domain.

1442

IEEE Transactions on Consumer Electronics, Vol. 49, No. 4, NOVEMBER 2003

Partially decoded image

ID C T

P s onto C s in pixeldom ain

Pq

DCT

onto

Cq

C onverge?

in D C T dom ain

Yes

No (a) Shifted block Partially

decoded image

ID C T

DCT

Ps onto C s in D C T dom ain

ID C T

DCT

Pq onto C q in D C T dom ain

C onverge?

Yes

No

Proposed algorithm Fast implementation

Fast implementation

M atrix m ultiplication

M atrix m ultiplication

(b) Fig. 7. Comparison of block diagrams: (a) Conventional algorithm. (b)The proposed algorithm.

To estimate the original value of DCT coefficients, we use the correlation of DCT coefficients between adjacent blocks. Consider a DCT coefficient

Aij (u, v) level at location (u , v )

pixel domain, while the

Pq onto Cq is processed in the DCT

domain, iteratively. Thus, these techniques require DCT/IDCT operations with heavy computational burden. On the contrary, in

of the (i, j ) th block. If the DC coefficient of a neighboring

the proposed algorithm both

block is within the quantization interval of the DC coefficient of the (i, j ) th block, then the neighboring block is classified as the

DCT domain. This algorithm needs DCT/IDCT operations each iteration as shown in Fig. 7 (b). But, the DCT/IDCT operations can be replaced by matrix multiplication, thus fast postprocessing in the DCT domain is possible.

block in same region (Fig. 6). As many equally located data of the same region blocks are within the quantization interval of Aij (u , v ) , the probability that the original DCT coefficient of

Aij (u, v) is close to the center of the interval is high. Therefore, we denote

κ ij (u, v) in (12) the number of DCT

coefficients at the equal location that are within the quantization interval of

Aij (u, v) . As κ ij (u, v) increases, λij (u, v)

decreases and the range of the QCS is sized down adaptively. III. FAST IMPLEMENTATION IN THE DCT DOMAIN Figs. 7 (a) and (b) show block diagrams of conventional and the proposed algorithm, respectively. As shown in Fig. 7 (a), in conventional algorithms the

Ps onto Cs is performed in the

In the proposed method,

Ps and Pq are performed in the

Pq is performed on the DCT

coefficients of blocks with the same grid of the encoded image, and

Ps is performed on the DCT coefficients of diagonally

shifted blocks. In this section, we propose a method that transforms the DCT coefficients for coefficients for

Pq into the DCT

Ps directly, and vice versa. Therefore the

proposed algorithm avoids the implementation of the DCT/IDCT operations and spatial domain processing. Let B

Aα , where α = 1, 2, 3, 4 , denote the shifted block and the original grid overlap blocks with B (see Fig. 8).

and

Y. Kim et al.: Fast POCS Based Post-processing Technique for HDTV

A1

1443

Define the eight-point DCT matrix

A2

S = {s (u, x)}u7 , x = 0 ,

where

s (u , x) = α (u ) cos[ Then, b and

π (2 x + 1)u ]. 16

B can be represented by B = SbS T ,

B

(17)

(18)

where the superscript T denotes matrix transposition. Specifically, since

S T S =I , (12) can be rewritten as 4

b = ∑ cα 1S TSaα S T Scα 2 .

(19)

α =1

Premultiplying both sides of (19) by S and postmultiplying

A3

A4

by

S T provides

Fig. 8. A1, A2, A3, A4, and B.

For simplicity, we consider that IDCT transforms of the block matrix form, let

4

aα and b are the 2-D-

Aα and B , respectively. In a

B = {B (u , v)}u7 ,v = 0 , b = {b(u, v)}u7 ,v =0 ,

Aα = { Aα (u , v)}u7 ,v = 0 , and aα = {aα (u, v)}u7 ,v =0 . Then b can be expressed as a superposition of appropriate windowed and shifted versions of a1 , a 2 , a3 , and a 4 . 4

b = ∑ cα 1aα cα 2 ,

(12)

α =1

where

cαβ ’s, α = 1, 2,3, 4 , β = 1, 2 are sparse 8 × 8

matrices of zeroes and ones that perform window and shift operations accordingly. The matrices are defined by

c11 = c 21 = U 7

0 I7 , 0 0

(13)

0 0 (14) , I7 0 where I 7 is the identity matrix of dimension 7 × 7 . Similarly c12 = c32 = L7

c31 = c41 = L1 ,

(15)

c22 = c42 = U1 .

(16)

and

B = ∑ Cα 1Aα Cα 2 ,

(20)

α =1

where fixed precomputed matrices

Cαβ 's are the DCT of

cαβ 's. The calculation of B can be effectively computed if B can be done even more efficiently rather than fully precomputing Cαβ by

the data is sparse. The computation of

factorizing these matrices into relatively sparse matrices [14]. The proposed implementation method with no use of DCT/IDCT processes can greatly alleviate total computation time required for obtaining B . In a similar fashion, A can be calculated using

Bα for the projection onto the quantization

constraint set. Therefore, the proposed technique is fully operated only in DCT domain. We calculated the number of operations required for our fast implementation method and compared it with that of conventional methods. Estimates of computation savings vary between 41% and 64% depending on the task. IV. EXPERIMENTAL RESULTS In this section, the performance of four postprocessing techniques, namely, Zakhor's [8], Yang's [11], Paek's [12], and the proposed technique is evaluated on various still images. Some images contain low-frequency components and smooth areas, others contain high-frequency components and edge area. The decoded images, with visible blocking artifacts, are obtained by JPEG [1] with the quantization table shown in Table I. Fig. 9 shows the test image LENA with blocking artifacts. As an objective measure of the distance between an original image f ( x, y ) and its reconstructed image g ( x, y ) , peak

1444

IEEE Transactions on Consumer Electronics, Vol. 49, No. 4, NOVEMBER 2003

(a)

(b)

Fig. 9. Test image: (a) Original LENA. (b) LENA with blocking artifacts. TABLE I QUANTIZATION TABLE.

50 60 70 70 90 120 255 255

60 60 70 96 130 255 255 255

70 70 80 120 200 255 255 255

70 96 120 145 255 255 255 255

90 130 200 255 255 255 255 255

operators first on

120 255 255 255 255 255 255 255

255 255 255 255 255 255 255 255

255 255 255 255 255 255 255 255

signal-to-noise ratio (PSNR) is used. For M × N images with [0, 255] gray-level range, PSNR can be defined as

PSNR = 10 log(

M × N × 2552

∑ ∑ M −1 x=0

N −1

( f ( x, y ) − g ( x, y )) 2 y=0

) . (21)

In Fig. 10, the PSNR performance of the proposed technique is compared with those of Zakhor's, Yang's, and Paek's techniques, with increasing iteration. One iteration of postprocessing is carried out by applying two projection

Cs and next on Cq , as described in the

previous section. As shown in Fig. 10, it is observed that the proposed technique converges within a few iterations, regardless of the input images, and can be easily implemented. From Fig. 10, we can see that the Zakhor's technique fails to converge, in terms of the theory of POCS. Also, the performance of the proposed technique is better than that of Yang's and Paek's technique in terms of the PSNR. Thus, it is believed that our technique is robust to the input images, in terms of convergence behavior and performance improvement. To make a comparison of the subjective quality more clear, we also present whole and enlarged pictures of the details of Lena in Figs. 11 and 12. As shown in these figures, although Zakhor's technique alleviates the blocking artifacts, it yields an excessively smoothed image due to the use of a global lowpass filter. And it is observed that the blocking artifacts are alleviated more effectively and edges are better preserved by the proposed techniques than the other two techniques. Similar results are also observed on the other test images. Hence the proposed constraint sets and their projection operators are properly defined, providing good performance in terms of both the PSNR and subjective quality. In Table II, the PSNR performance of the proposed

TABLE II PSNR FOR THE DIFFERENT POSTPROCESSING METHODS.

JPEG Zakhor’s algorithm Yang’s algorithm Paek’s algorithm Proposed

LENA

ZELDA

CAMERA

BRIDGE

PEPPER

BARBARA

CHURCH

27.68

29.81

26.69

24.79

28.84

28.43

28.44

27.85

30.33

26.56

24.98

29.27

29.13

28.77

27.78

29.97

26.56

24.77

28.71

28.96

28.35

27.80

29.99

26.78

24.85

29.01

28.64

28.22

27.98

30.24

26.84

25.00

29.25

28.99

28.76

Y. Kim et al.: Fast POCS Based Post-processing Technique for HDTV

1445

Lena

28.2

Zelda 30.4

28.0

30.2

27.8

30.0

Zakhor Yang Paek Proposed

27.4

27.2

PSNR

PSNR

27.6

29.8 29.6

Zakhor Yang Paek Proposed

29.4

27.0

29.2

26.8

29.0 0

2

4

6

8

10

12

14

16

18

20

0

2

4

6

8

Iterations

(a)

27.0

12

14

16

18

20

(b)

Camera

25.2

Bridge

25.0

26.8

Zakhor Yang Paek Proposed

26.6

24.8

24.6

PSNR

26.4

PSNR

10

Iterations

26.2

24.4

26.0

24.2

25.8

24.0

Zakhor Yang Paek Proposed

23.8

25.6 0

2

4

6

8

10

12

14

16

18

0

20

2

4

6

8

10

12

14

16

18

20

Iterations

Iterations

(c)

(d)

Fig. 10. PSNR performance variation on four decoded images. (a) LENA. (b) ZELDA. (c) CAMERA. (d) BRIDGE.

technique is compared with that of Zakhor's, Yang's, and Paek's techniques. The simulation results show that the performance of the proposed algorithm is better than those of other algorithms. V. CONCLUSIONS In this paper, we proposed a new postprocessing algorithm based on the theory of POCS in order to reduce the blocking artifacts in HDTV images. By detecting and eliminating the undesired high-frequency components, mainly caused by the blocking artifacts, we proposed a new SCS and its projection operator in DCT domain. Moreover, we proposed an improved QCS using the correlation of DCT coefficients between adjacent blocks. In addition, we introduced a fast implementation method of the proposed algorithm. The proposed method is fully operated in the frequency domain without performing DCT/IDCT operations. Computer simulation results indicated that the proposed postprocessing algorithm effectively reduces the blocking effects and preserves the original high-frequency components such as edges, despite of its low complexity. The objective

performance of the proposed algorithm has been measured by the PSNR of the postprocessed images. The measured PSNR of the proposed postprocessing method showed an increase of 0 - 1.0 dB for the various test images. From a subjective point of view, visual improvements of the image qualities were found in the postprocessed images obtained by the proposed algorithm. Moreover, the proposed technique was shown to be robust in terms of convergence. Based on the noniterative nature and fast architecture, the proposed technique can be employed for the postprocessing of a wide variety of images.

REFERENCES [1] [2] [3] [4] [5]

ISO/IEC/JTC1/SC1/WG8 JPEG technical specification, vol. Revision 8, 1990. ITU-T, Video coding for low bit rate communication, ITU-T Recommendation H.263 version 2, Jan. 1998. I. JTC1/SC29 Coding of moving pictures and association audio, vol. Recommendation H.262: ISO/IEC 13818, 1993. M.Liou, “Overview of the p×64kbit/s video coding,” Commun. ACM, vol. 34, pp. 46–58, Apr. 1991. K. H. Tzou, “Post-filtering for transform-coded images,” in Proc. SPIE, Applications of Digital Image Processing XI, vol. 974, San Diego, CA, Aug. 1988, pp. 121–126.

1446

IEEE Transactions on Consumer Electronics, Vol. 49, No. 4, NOVEMBER 2003

(a)

(c)

(b)

(d)

Fig. 11. Comparison of subjective quality on “LENA”. Post-processed images: (a) Zakhor’s algorithm. (b) Yang’s algorithm. (c) Paek’s algorithm. (d) The proposed method. [6]

B. Ramamurthi and A. Gersho, “Nonlinear space-variant postprocessing of block coded images,” IEEE Trans. Acoustics, Speech, and Signal Processing, vol. ASSP-34, pp. 1258–1267, Oct. 1986. [7] D. Youla and H. Webb, “Image restoration by the method of convex projections: part i–theory,” IEEE Trans. Med. Imag., vol. MI-1, pp. 81– 94, Oct. 1982. [8] A. Zakhor, “Iterative procedures for reduction of blocking effects in transform image coding,” IEEE Trans. on Circuits Syst. Video Technol., vol. 2, pp. 91-95, Mar. 1992. [9] S. Minami and A. Zakhor, “An optimization approach for removing blocking effects in transform coding,” IEEE Trans. on Circuits Syst. Video Technol., vol. 5, pp. 74-82, Apr. 1995. [10] Y. Yang, N. P. Galatsanos, and A. K. Katsaggelos, “Regularized reconstruction to reduce blocking artifacts of block discrete cosine transform compressed images,” IEEE Trans. on Circuits Syst. Video Technol., vol. 3, pp. 421-432, Dec. 1993. [11] Y. Yang, N. P. Galatsanos, and A. K. Katsaggelos, “Projection-based spatially adaptive reconstruction of block-transform compressed images,” IEEE Trans. on Circuits Syst. Video Technol, vol. 5, pp. 298304, Aug. 1995. [12] H. Paek, R. C. Kim, and S. U. Lee, “On the pocs-based postprocessing technique to reduce the blocking artifacts in transform coded images,” IEEE Trans. on Circuits Syst. Video Technol., vol. 8, pp. 358-367, June 1998.

[13] S. H. Park and D. S. Kim, “Theory of projection onto the narrow Quantization constraint set and its application,” IEEE Trans. on Image Processing, vol. 8, pp. 1361-1373, Oct. 1999. [14] N. Merhav and V. Bhaskaran, “Fast algorithms for DCT-domain image down-sampling and for inverse motion compensation,” IEEE Trans. Circuits Syst. Video Technol., vol. 7, pp. 468–476, June 1997.

Yoon Kim was born in Korea, on February 16, 1969. He received the B.S. and M.S. degrees in electronic engineering with the Department of Electronic Engineering from Korea University, in 1993 and 1995, respectively. From 1995 to 1999, he was with the LGPhilips LCD Co. where he was involved in research and development on digital image equipments. He is now a Ph.D. candidate in electronic engineering with the Department of Electronic Engineering at Korea University. His research interests are in the areas of video signal processing and multimedia communications.

Y. Kim et al.: Fast POCS Based Post-processing Technique for HDTV

1447

(a)

(b)

(c)

(d)

Fig. 12. Comparison of subjective quality on “LENA”. Post-processed enlarged images: (a) Zakhor’s algorithm. (b) Yang’s algorithm. (c) Paek’s algorithm. (d) The proposed method. Chun–Su Park received the B.S. degree from Korea University, in Electronics Engineering, in 2003. He is now a M.S. candidate in electronic engineering with the Department of Electronic Engineering at Korea University. His research interests are in the areas of Mobile QoS, IP QoS, Handoff, video signal processing and multimedia communications. Sung-Jea Ko (M'88-SM'97) received the Ph.D. degree in 1988 and the M.S. degree in 1986, both in Electrical and Computer Engineering, from State University of New York at Buffalo, and the B.S. degree in Electronic Engineering at Korea University in 1980. In 1992, he joined the Department of Electronic Engineering at Korea University where he is currently a Professor. From 1988 to 1992, he was an Assistant Professor of the Department of Electrical and Computer Engineering at the University of Michigan-Dearborn. From 1986 to 1988, he was a Research Assistant at State University of New York at Buffalo. From 1981 to 1983, he was with Daewoo Telecom where he was involved in research and development on data communication systems.

He has published more than 200 papers in journals and conference proceedings. He also holds over 10 patents on data communication and video signal processing. He is currently a Senior Member in the IEEE, a Fellow in the IEE and a chairman of the Consumer Electronics chapter of IEEE Seoul Section. He has been the Special Sessions chair for the IEEE Asia Pacific Conference on Circuits and Systems (1996). He has served as an Associate Editor for Journal of the Institute of Electronics Engineers of Korea (IEEK) (1996), Journal of Broadcast Engineering (1996 - 1999), the Journal of the Korean Institute of Communication Sciences (KICS) (1997 - 2000). He has been an editor of Journal of Communications and Networks (JCN) (1998 - 2000). He is the 1999 Recipient of the LG Research Award given to the Outstanding Information and Communication Researcher. He received the Hae-Dong best paper award from the IEEK (1997) and the best paper award from the IEEE Asia Pacific Conference on Circuits and Systems (1996).

1438

Fast POCS Based Post-Processing Technique for HDTV Yoon Kim, Chun-Su Park, and Sung-Jea Ko, Senior Member, IEEE Abstract — In this paper, we present a novel postprocessing technique based on the theory of the projection onto convex sets (POCS) in order to reduce the blocking artifacts in digital high definition television (HDTV) images. By detecting and eliminating the undesired high-frequency components, mainly caused by blocking artifacts, we propose a new smoothness constraint set (SCS) and its projection operator in the DCT domain. In addition, we propose an improved quantization constraint set (QCS) using the correlation of DCT coefficients between adjacent blocks. In the proposed technique, the range of the QCS is efficiently reduced as close to the original DCT coefficient as possible to yield better performance of the projection onto the QCS. Computer simulation results indicate that the proposed schemes perform better than conventional algorithms. Furthermore, we introduce a fast implementation method of the proposed algorithm. The conventional POCS-based postprocessing techniques require the forward/inverse discrete cosine transform (DCT/IDCT) operations with a heavy computational burden. To reduce the computational complexity we introduce a fast implementation method of the proposed algorithm that does not perform DCT/IDCT operations. Estimates of computation savings vary between 41% and 64% depending on the task. Index Terms — Postprocessing, POCS, SCS, QCS. I.

INTRODUCTION

Image compression techniques based on BDCT have been the popular choice in both still and moving image coding standards, such as JPEG [1], H.26x [2], and MPEG [3]. Even in digital HDTV, image signals are compressed by BDCT and then transmitted via broadcasting channels. For the sake of efficient implementation, an image is partitioned into spatially adjoining blocks that are processed independently without considering inter-block correlations. At low bit rates, this approach inevitably causes two primary visual artifacts in BDCT decompressed images. One is the grid noise in monotone areas, which consists of the introduction of artificial edges at the 8 × 8 DCT block boundaries. The other is the staircase noise that distorts edge shapes. Various algorithms have been proposed to improve the quality of block-coded images in the decoder without increasing the bit rate. Among them, a linear space-invariant low-pass filtering [4] is well known, due to its simplicity. However, since the local characteristics of image vary spatially, it is known that linear, space-invariant filtering is inadequate. Therefore, many adaptive techniques were proposed to solve these problems. Better results were achieved Contributed Paper Manuscript received September 1, 2003

using directional smoothing filters with a stronger effect on the direction orthogonal to the block boundaries [5]. A more sophisticated nonlinear space-variant filter was proposed in [6], where image blocks were classified and each was processed through an appropriate filter. The iterative approaches based on the theory of POCS can be thought of as special cases of image restoration [7]. Zakhor [8] first tried to apply the theory of POCS to the reduction of blocking artifacts of block DCT-coded images and suggested an iterative algorithm. The iterative algorithm can be justified as a method of steepest descent iteration in a constrained minimization problem. She imposed two constraints on the coded image: SCS (Cs), which denotes the set including the images free from blocking artifacts, and QCS (Cq), which denotes the set including the images close to the original image as shown in Fig. 1. Notice in Fig. 1 that the Pq and Ps are the projection operators onto Cq and Cs, respectively. In [8]-[13], POCS-based techniques have been also proposed that are effective in reducing blocking artifacts. In these methods, the projections onto the SCS's are performed through the pixel domain, and the projections onto the QCS's are performed in the DCT domain. These methods require the iterative DCT/IDCT operations with heavy computational burden. Therefore, in spite of better performance than other postprocessing techniques, the POCS-based postprocessing techniques have not been widely used in real applications. f0

k =1

k ≠1

Ps onto Cs

Pq onto Cq

Converge?

No

fk Yes

Fig. 1. Block diagram of the postprocessing technique based on the theory of POCS.

In this paper, we propose a new postprocessing algorithm based on the theory of POCS and its fast implementation in order to reduce the blocking artifacts in HDTV images. Note that a block, which is diagonally shifted one pixel apart from the grid of BDCT, includes the boundary of the original block. If any blocking artifact is introduced along the block boundary, this shifted block will produce frequency characteristics different from those of the original block. Thus, a comparison of frequency characteristics of these two overlapping blocks can detect the undesired high-frequency components, mainly caused by the blocking artifacts. By eliminating these undesired high frequency components, a robust smoothing projection operator can be obtained. Moreover, we propose an improved QCS using the correlation of DCT coefficients between adjacent blocks. In the proposed

0098 3063/00 $10.00 © 2003 IEEE

Y. Kim et al.: Fast POCS Based Post-processing Technique for HDTV

technique, the range of the QCS is efficiently reduced to as close to the original DCT coefficient as possible to yield the better performance of the projection onto the QCS. Furthermore, we introduce a fast implementation method of the proposed algorithm. The conventional POCS-based postprocessing techniques require DCT/IDCT operations with high computational complexity. On the contrary, the proposed method is fully operated in the frequency domain without the need of performing the DCT/IDCT operations. Estimates of computation savings vary between 41% and 64% depending on the task. Moreover, the proposed technique converges within a few iterations. The paper is organized as follows. The proposed postprocessing scheme is presented in Section II. In Section III, the proposed fast implementation and computational complexity are described. Section IV presents and discusses the experimental results. Finally, our conclusions are given in Section V. II. PROPOSED POST-PROCESSING TECHNIQUE

1439

bij ( N − 1, N − 1)} , where bij ( x, y) = f (iN + x + 1, jN + y + 1) . Note that block bij includes both right vertical and bottom horizontal block boundaries of

aij .

Let the DCT of aij ( x, y ) be Aij (u , v ) . Then, for N = 8 , the DCT/IDCT transform is given by

Aij (u , v) = α (u )α (v)∑ x =0 ∑ y =0 aij ( x, y ) 7

× cos[

7

π (2 x + 1)u π (2 y + 1)v ]cos[ ], 16 16

(1)

aij ( x, y ) = ∑ u =0 ∑ v =0 α (u )α (v) Aij (u, v) 7

× cos[

7

π (2 x + 1)u π (2 y + 1)v ]cos[ ], 16 16

(2)

where

A. Proposed SCS and Its Projection In this subsection, we study the blocking artifacts by analyzing frequency characteristics of the original block and the block shifted by one pixel. Then, we introduce a smoothness constraint set and its projection operator in the DCT domain to reduce blocking artifacts by compensating the effects of the added block boundary with the POCS method.

α (k ) =

1 , N 2 , N

k = 0, (3)

otherwise.

In the same manner, bij ( x, y ) and Bij (u , v ) can be

N

formulated. To reduce the blocking artifacts in the decoded image, we first observe the characteristics of Bij , the DCT

aij

block of

bij . The Bij has more nonzero AC coefficients when

compared with

N

bij

Aij , the DCT block of aij , since Bij includes

the high frequency block boundaries (blocking artifacts). In other words, nonzero coefficients of Bij occur at higher orders in the zig-zag scan than highest nonzero valued location of Aij . Figs. 3 (a), (b), and (c) show an example of and the projected element Fig. 2.

aij

and

bij .

First, let us denote an mN × nN input image and its pixel by f and f (⋅, ⋅) , respectively, where m and n are positive integers. In block-based transform coding, f (⋅, ⋅) is composed of nonoverlapping N × N

blocks

{aij (0, 0),L , aij ( N − 1, N − 1)}

aij ( x, y ) =

f (iN + x, jN + y ) , i = 0,1,L , ( m − 2) and j = 0,1,L , ( n − 2) . Fig. 2 shows a block bij , that is diagonally shifted one pixel from the block

aij where bij = {bij (0, 0),L ,

Bij which will be defined later,

respectively. Based on this observation, we establish an algorithm for reducing the blocking artifacts by proposing a smoothness constraint set and its projection operator. Let us define a closed convex set

CBij as

CBij = { f | bij ⊂ f , NZ ( Bij ) ≤ K Bij } ,

aij 's, where aij = with

Aij , Bij ,

(4)

where

K Bij = NZ ( Aij ) + VBij ,

(5)

1440

IEEE Transactions on Consumer Electronics, Vol. 49, No. 4, NOVEMBER 2003

Bij

Bij

Aij 840

0

0

0

0

0

0

0

860 61 -20 12 -12

0

-60

0

0

0

0

0

0

37 -76 14

0

0

0

0

0

0

0

0

-6

5

0

0

0

0

0

0

0

0

9

-8

5

0

0

0

0

0

0

0

0

-7

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

NZ( Aij ) = 5

Highest order ,

-9

4

-2

0

0

860 61 -20 12 0 0 37 -76 14 0

-12 -1

0

1

-1

0

-6

5

0

0

0

0

0

0

-1

4

-2

0

0

9

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

-1

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

VBij

5

(a)

NZ (⋅) represents the order of the last nonzero DCT

coefficient in the zig-zag scan. In Fig. 3 (a), NZ ( Aij ) = 5 . In (5),

0

0

VBij is a factor representing how many DCT coefficients

measure of the local detail at

bij . It is known that noise near

edges is less visible than noise appearing in flat (gradually changing) region. The former should be carefully reduced so that the information about edges will not be damaged, while the latter should be eliminated completely. It is also known that the blocking artifacts are more visible in bright areas rather than in dark areas of the image. Therefore, in bright monotone areas with high

µ and small σ , VBij should be

small such that more high-frequency components are eliminated. On the other hand, since a block

bij with large σ

contains edge areas, the high-frequency components should be preserved. A function that captures this property is

VBij ∝

σ , 1+ µ

(6)

µ in the denominator to avoid mathematical difficulties when µ = 0 . Since µ and σ are where 1 is added to

0

0

0

0

0

proportional to the DC coefficient and a absolute sum of major AC coefficients of the DCT block, respectively, we define

VBij by

VBij = η

∑

and the human perceptual properties.

serves as a measure of the local brightness, and variance is a

0

Aij . (b) Bij . (c) Bij .

VBij should be chosen based on the local statistics of the image bij can be treated as a random variable with mean µ and variance σ . The mean

0

(c)

of Bij do contribute to the blocking artifacts. Thus,

In general, each pixel intensity in

0

KBij 10, NZ( Bij ) = 10

(b) Fig. 3. Example of Ps onto Cs: (a)

and

6

where

5 i =1

| ACi |

1 + DC

,

(7)

η is a scaling factor, and DC and ACi are DCT

coefficients of Bij . From Fig. 3 (b), using (7), we obtain

VBij = 5 and K Bij =10 for

η = 10 . By observing the

smoothness constraint set in (4), the projection operator onto that set can be easily found. By discarding the nonzero coefficients of Bij at higher orders than obtain the projected element

K Bij in (5), we can

Bij as shown in Fig. 3 (c).

B. Proposed QCS In conventional POCS-based postprocessing technique, the QCS ( Cq ) is commonly defined as the quantization interval for the original DCT coefficient. Since the quantization parameters are transmitted from the encoder, the quantization interval for each DCT coefficient is known to the decoder. The task of its projection operator ( Pq ) is to move the DCT coefficient into the quantization interval. The conventional QCS of (i , j ) block in an mN × nN input image is identical to the closure of the quantization region defined as

Y. Kim et al.: Fast POCS Based Post-processing Technique for HDTV

1441

Reconstruction level D ecision level

D ecision level

D C T dom ain

x min

Aij

(u,v)

Aij (u,v)

Aijmax (u,v) Fig. 5. Concept of the proposed QCS in a DCT coefficient location.

Q uantization interval

Fig. 4. Conventional quantization constraint set.

QCSij (u , v) = [ Aijmin (u, v), Aijmax (u, v)] ,

QCS in every coefficient location is proposed and can be expressed as (8)

Qij (u , v) min = − A ( u , v ) A ( u , v ) ( u , v ) , λ ij ij ij 2 Amax (u, v) = A (u , v) + λ (u, v) Qij (u , v) , ij ij ij 2

where

Qij (u , v) min , Aij (u, v) = Aij (u , v) − 2 Amax (u, v) = A (u , v) + Qij (u , v ) , ij ij 2 where of

(11)

(9) where

λij (u, v) =

Aij (u, v) is the reconstruction level at location (u , v )

(i , j ) th block and Qij (u , v) is the quantization step for

M , κ ij (u, v) + M

(12)

λij (u, v) is the value estimated from BDCT

coefficient in each coefficient location as shown in Fig. 4. The

The coefficient

projection onto the QCS in (8) of an input value A%ij (u, v ) is

coefficients in the same location

defined as the clipping operation over that region, and can be written as

neighboring blocks, and determined in the range from 0.5 to 1 . M is the total number of neighboring blocks, 8. The concept of the proposed QCS in a DCT coefficient location is delineated in Fig. 5.

Pij (u, v) Aijmin (u, v), = Aijmax (u, v), % Aij (u , v), where

if A%ij (u , v) ≤ Aijmin (u , v), if Aijmax (u , v) ≤ A%ij (u , v),

(10)

(u , v ) of its eight

DC

DC

DC

DC

DC

DC

otherwise,

Pij (u, v) represents the projection onto the QCS. Since

the value projected onto the SCS deviates from the decoded value, it should be restored as close to an initial value as possible. This is what the projection onto the QCS does. Since the conventional QCS is fixed as a maximum size of the range where an original DCT value can be included, the result of the projection onto the SCS almost exists in the range of the QCS. Thus, the projection onto the conventional QCS often fails to play the role of restraining the divergence caused by that onto the SCS. If the range of the QCS can be sized down efficiently, then the projection onto that QCS can yield better performance. In order to reduce blocking artifacts while preventing overblurring of the postprocessed image, a new QCS with which we can adaptively control the size of the conventional

A ij (u,v) DC

DC

DC

The Blocks of Same Region Fig. 6. Block classification in DCT domain.

1442

IEEE Transactions on Consumer Electronics, Vol. 49, No. 4, NOVEMBER 2003

Partially decoded image

ID C T

P s onto C s in pixeldom ain

Pq

DCT

onto

Cq

C onverge?

in D C T dom ain

Yes

No (a) Shifted block Partially

decoded image

ID C T

DCT

Ps onto C s in D C T dom ain

ID C T

DCT

Pq onto C q in D C T dom ain

C onverge?

Yes

No

Proposed algorithm Fast implementation

Fast implementation

M atrix m ultiplication

M atrix m ultiplication

(b) Fig. 7. Comparison of block diagrams: (a) Conventional algorithm. (b)The proposed algorithm.

To estimate the original value of DCT coefficients, we use the correlation of DCT coefficients between adjacent blocks. Consider a DCT coefficient

Aij (u, v) level at location (u , v )

pixel domain, while the

Pq onto Cq is processed in the DCT

domain, iteratively. Thus, these techniques require DCT/IDCT operations with heavy computational burden. On the contrary, in

of the (i, j ) th block. If the DC coefficient of a neighboring

the proposed algorithm both

block is within the quantization interval of the DC coefficient of the (i, j ) th block, then the neighboring block is classified as the

DCT domain. This algorithm needs DCT/IDCT operations each iteration as shown in Fig. 7 (b). But, the DCT/IDCT operations can be replaced by matrix multiplication, thus fast postprocessing in the DCT domain is possible.

block in same region (Fig. 6). As many equally located data of the same region blocks are within the quantization interval of Aij (u , v ) , the probability that the original DCT coefficient of

Aij (u, v) is close to the center of the interval is high. Therefore, we denote

κ ij (u, v) in (12) the number of DCT

coefficients at the equal location that are within the quantization interval of

Aij (u, v) . As κ ij (u, v) increases, λij (u, v)

decreases and the range of the QCS is sized down adaptively. III. FAST IMPLEMENTATION IN THE DCT DOMAIN Figs. 7 (a) and (b) show block diagrams of conventional and the proposed algorithm, respectively. As shown in Fig. 7 (a), in conventional algorithms the

Ps onto Cs is performed in the

In the proposed method,

Ps and Pq are performed in the

Pq is performed on the DCT

coefficients of blocks with the same grid of the encoded image, and

Ps is performed on the DCT coefficients of diagonally

shifted blocks. In this section, we propose a method that transforms the DCT coefficients for coefficients for

Pq into the DCT

Ps directly, and vice versa. Therefore the

proposed algorithm avoids the implementation of the DCT/IDCT operations and spatial domain processing. Let B

Aα , where α = 1, 2, 3, 4 , denote the shifted block and the original grid overlap blocks with B (see Fig. 8).

and

Y. Kim et al.: Fast POCS Based Post-processing Technique for HDTV

A1

1443

Define the eight-point DCT matrix

A2

S = {s (u, x)}u7 , x = 0 ,

where

s (u , x) = α (u ) cos[ Then, b and

π (2 x + 1)u ]. 16

B can be represented by B = SbS T ,

B

(17)

(18)

where the superscript T denotes matrix transposition. Specifically, since

S T S =I , (12) can be rewritten as 4

b = ∑ cα 1S TSaα S T Scα 2 .

(19)

α =1

Premultiplying both sides of (19) by S and postmultiplying

A3

A4

by

S T provides

Fig. 8. A1, A2, A3, A4, and B.

For simplicity, we consider that IDCT transforms of the block matrix form, let

4

aα and b are the 2-D-

Aα and B , respectively. In a

B = {B (u , v)}u7 ,v = 0 , b = {b(u, v)}u7 ,v =0 ,

Aα = { Aα (u , v)}u7 ,v = 0 , and aα = {aα (u, v)}u7 ,v =0 . Then b can be expressed as a superposition of appropriate windowed and shifted versions of a1 , a 2 , a3 , and a 4 . 4

b = ∑ cα 1aα cα 2 ,

(12)

α =1

where

cαβ ’s, α = 1, 2,3, 4 , β = 1, 2 are sparse 8 × 8

matrices of zeroes and ones that perform window and shift operations accordingly. The matrices are defined by

c11 = c 21 = U 7

0 I7 , 0 0

(13)

0 0 (14) , I7 0 where I 7 is the identity matrix of dimension 7 × 7 . Similarly c12 = c32 = L7

c31 = c41 = L1 ,

(15)

c22 = c42 = U1 .

(16)

and

B = ∑ Cα 1Aα Cα 2 ,

(20)

α =1

where fixed precomputed matrices

Cαβ 's are the DCT of

cαβ 's. The calculation of B can be effectively computed if B can be done even more efficiently rather than fully precomputing Cαβ by

the data is sparse. The computation of

factorizing these matrices into relatively sparse matrices [14]. The proposed implementation method with no use of DCT/IDCT processes can greatly alleviate total computation time required for obtaining B . In a similar fashion, A can be calculated using

Bα for the projection onto the quantization

constraint set. Therefore, the proposed technique is fully operated only in DCT domain. We calculated the number of operations required for our fast implementation method and compared it with that of conventional methods. Estimates of computation savings vary between 41% and 64% depending on the task. IV. EXPERIMENTAL RESULTS In this section, the performance of four postprocessing techniques, namely, Zakhor's [8], Yang's [11], Paek's [12], and the proposed technique is evaluated on various still images. Some images contain low-frequency components and smooth areas, others contain high-frequency components and edge area. The decoded images, with visible blocking artifacts, are obtained by JPEG [1] with the quantization table shown in Table I. Fig. 9 shows the test image LENA with blocking artifacts. As an objective measure of the distance between an original image f ( x, y ) and its reconstructed image g ( x, y ) , peak

1444

IEEE Transactions on Consumer Electronics, Vol. 49, No. 4, NOVEMBER 2003

(a)

(b)

Fig. 9. Test image: (a) Original LENA. (b) LENA with blocking artifacts. TABLE I QUANTIZATION TABLE.

50 60 70 70 90 120 255 255

60 60 70 96 130 255 255 255

70 70 80 120 200 255 255 255

70 96 120 145 255 255 255 255

90 130 200 255 255 255 255 255

operators first on

120 255 255 255 255 255 255 255

255 255 255 255 255 255 255 255

255 255 255 255 255 255 255 255

signal-to-noise ratio (PSNR) is used. For M × N images with [0, 255] gray-level range, PSNR can be defined as

PSNR = 10 log(

M × N × 2552

∑ ∑ M −1 x=0

N −1

( f ( x, y ) − g ( x, y )) 2 y=0

) . (21)

In Fig. 10, the PSNR performance of the proposed technique is compared with those of Zakhor's, Yang's, and Paek's techniques, with increasing iteration. One iteration of postprocessing is carried out by applying two projection

Cs and next on Cq , as described in the

previous section. As shown in Fig. 10, it is observed that the proposed technique converges within a few iterations, regardless of the input images, and can be easily implemented. From Fig. 10, we can see that the Zakhor's technique fails to converge, in terms of the theory of POCS. Also, the performance of the proposed technique is better than that of Yang's and Paek's technique in terms of the PSNR. Thus, it is believed that our technique is robust to the input images, in terms of convergence behavior and performance improvement. To make a comparison of the subjective quality more clear, we also present whole and enlarged pictures of the details of Lena in Figs. 11 and 12. As shown in these figures, although Zakhor's technique alleviates the blocking artifacts, it yields an excessively smoothed image due to the use of a global lowpass filter. And it is observed that the blocking artifacts are alleviated more effectively and edges are better preserved by the proposed techniques than the other two techniques. Similar results are also observed on the other test images. Hence the proposed constraint sets and their projection operators are properly defined, providing good performance in terms of both the PSNR and subjective quality. In Table II, the PSNR performance of the proposed

TABLE II PSNR FOR THE DIFFERENT POSTPROCESSING METHODS.

JPEG Zakhor’s algorithm Yang’s algorithm Paek’s algorithm Proposed

LENA

ZELDA

CAMERA

BRIDGE

PEPPER

BARBARA

CHURCH

27.68

29.81

26.69

24.79

28.84

28.43

28.44

27.85

30.33

26.56

24.98

29.27

29.13

28.77

27.78

29.97

26.56

24.77

28.71

28.96

28.35

27.80

29.99

26.78

24.85

29.01

28.64

28.22

27.98

30.24

26.84

25.00

29.25

28.99

28.76

Y. Kim et al.: Fast POCS Based Post-processing Technique for HDTV

1445

Lena

28.2

Zelda 30.4

28.0

30.2

27.8

30.0

Zakhor Yang Paek Proposed

27.4

27.2

PSNR

PSNR

27.6

29.8 29.6

Zakhor Yang Paek Proposed

29.4

27.0

29.2

26.8

29.0 0

2

4

6

8

10

12

14

16

18

20

0

2

4

6

8

Iterations

(a)

27.0

12

14

16

18

20

(b)

Camera

25.2

Bridge

25.0

26.8

Zakhor Yang Paek Proposed

26.6

24.8

24.6

PSNR

26.4

PSNR

10

Iterations

26.2

24.4

26.0

24.2

25.8

24.0

Zakhor Yang Paek Proposed

23.8

25.6 0

2

4

6

8

10

12

14

16

18

0

20

2

4

6

8

10

12

14

16

18

20

Iterations

Iterations

(c)

(d)

Fig. 10. PSNR performance variation on four decoded images. (a) LENA. (b) ZELDA. (c) CAMERA. (d) BRIDGE.

technique is compared with that of Zakhor's, Yang's, and Paek's techniques. The simulation results show that the performance of the proposed algorithm is better than those of other algorithms. V. CONCLUSIONS In this paper, we proposed a new postprocessing algorithm based on the theory of POCS in order to reduce the blocking artifacts in HDTV images. By detecting and eliminating the undesired high-frequency components, mainly caused by the blocking artifacts, we proposed a new SCS and its projection operator in DCT domain. Moreover, we proposed an improved QCS using the correlation of DCT coefficients between adjacent blocks. In addition, we introduced a fast implementation method of the proposed algorithm. The proposed method is fully operated in the frequency domain without performing DCT/IDCT operations. Computer simulation results indicated that the proposed postprocessing algorithm effectively reduces the blocking effects and preserves the original high-frequency components such as edges, despite of its low complexity. The objective

performance of the proposed algorithm has been measured by the PSNR of the postprocessed images. The measured PSNR of the proposed postprocessing method showed an increase of 0 - 1.0 dB for the various test images. From a subjective point of view, visual improvements of the image qualities were found in the postprocessed images obtained by the proposed algorithm. Moreover, the proposed technique was shown to be robust in terms of convergence. Based on the noniterative nature and fast architecture, the proposed technique can be employed for the postprocessing of a wide variety of images.

REFERENCES [1] [2] [3] [4] [5]

ISO/IEC/JTC1/SC1/WG8 JPEG technical specification, vol. Revision 8, 1990. ITU-T, Video coding for low bit rate communication, ITU-T Recommendation H.263 version 2, Jan. 1998. I. JTC1/SC29 Coding of moving pictures and association audio, vol. Recommendation H.262: ISO/IEC 13818, 1993. M.Liou, “Overview of the p×64kbit/s video coding,” Commun. ACM, vol. 34, pp. 46–58, Apr. 1991. K. H. Tzou, “Post-filtering for transform-coded images,” in Proc. SPIE, Applications of Digital Image Processing XI, vol. 974, San Diego, CA, Aug. 1988, pp. 121–126.

1446

IEEE Transactions on Consumer Electronics, Vol. 49, No. 4, NOVEMBER 2003

(a)

(c)

(b)

(d)

Fig. 11. Comparison of subjective quality on “LENA”. Post-processed images: (a) Zakhor’s algorithm. (b) Yang’s algorithm. (c) Paek’s algorithm. (d) The proposed method. [6]

B. Ramamurthi and A. Gersho, “Nonlinear space-variant postprocessing of block coded images,” IEEE Trans. Acoustics, Speech, and Signal Processing, vol. ASSP-34, pp. 1258–1267, Oct. 1986. [7] D. Youla and H. Webb, “Image restoration by the method of convex projections: part i–theory,” IEEE Trans. Med. Imag., vol. MI-1, pp. 81– 94, Oct. 1982. [8] A. Zakhor, “Iterative procedures for reduction of blocking effects in transform image coding,” IEEE Trans. on Circuits Syst. Video Technol., vol. 2, pp. 91-95, Mar. 1992. [9] S. Minami and A. Zakhor, “An optimization approach for removing blocking effects in transform coding,” IEEE Trans. on Circuits Syst. Video Technol., vol. 5, pp. 74-82, Apr. 1995. [10] Y. Yang, N. P. Galatsanos, and A. K. Katsaggelos, “Regularized reconstruction to reduce blocking artifacts of block discrete cosine transform compressed images,” IEEE Trans. on Circuits Syst. Video Technol., vol. 3, pp. 421-432, Dec. 1993. [11] Y. Yang, N. P. Galatsanos, and A. K. Katsaggelos, “Projection-based spatially adaptive reconstruction of block-transform compressed images,” IEEE Trans. on Circuits Syst. Video Technol, vol. 5, pp. 298304, Aug. 1995. [12] H. Paek, R. C. Kim, and S. U. Lee, “On the pocs-based postprocessing technique to reduce the blocking artifacts in transform coded images,” IEEE Trans. on Circuits Syst. Video Technol., vol. 8, pp. 358-367, June 1998.

[13] S. H. Park and D. S. Kim, “Theory of projection onto the narrow Quantization constraint set and its application,” IEEE Trans. on Image Processing, vol. 8, pp. 1361-1373, Oct. 1999. [14] N. Merhav and V. Bhaskaran, “Fast algorithms for DCT-domain image down-sampling and for inverse motion compensation,” IEEE Trans. Circuits Syst. Video Technol., vol. 7, pp. 468–476, June 1997.

Yoon Kim was born in Korea, on February 16, 1969. He received the B.S. and M.S. degrees in electronic engineering with the Department of Electronic Engineering from Korea University, in 1993 and 1995, respectively. From 1995 to 1999, he was with the LGPhilips LCD Co. where he was involved in research and development on digital image equipments. He is now a Ph.D. candidate in electronic engineering with the Department of Electronic Engineering at Korea University. His research interests are in the areas of video signal processing and multimedia communications.

Y. Kim et al.: Fast POCS Based Post-processing Technique for HDTV

1447

(a)

(b)

(c)

(d)

Fig. 12. Comparison of subjective quality on “LENA”. Post-processed enlarged images: (a) Zakhor’s algorithm. (b) Yang’s algorithm. (c) Paek’s algorithm. (d) The proposed method. Chun–Su Park received the B.S. degree from Korea University, in Electronics Engineering, in 2003. He is now a M.S. candidate in electronic engineering with the Department of Electronic Engineering at Korea University. His research interests are in the areas of Mobile QoS, IP QoS, Handoff, video signal processing and multimedia communications. Sung-Jea Ko (M'88-SM'97) received the Ph.D. degree in 1988 and the M.S. degree in 1986, both in Electrical and Computer Engineering, from State University of New York at Buffalo, and the B.S. degree in Electronic Engineering at Korea University in 1980. In 1992, he joined the Department of Electronic Engineering at Korea University where he is currently a Professor. From 1988 to 1992, he was an Assistant Professor of the Department of Electrical and Computer Engineering at the University of Michigan-Dearborn. From 1986 to 1988, he was a Research Assistant at State University of New York at Buffalo. From 1981 to 1983, he was with Daewoo Telecom where he was involved in research and development on data communication systems.

He has published more than 200 papers in journals and conference proceedings. He also holds over 10 patents on data communication and video signal processing. He is currently a Senior Member in the IEEE, a Fellow in the IEE and a chairman of the Consumer Electronics chapter of IEEE Seoul Section. He has been the Special Sessions chair for the IEEE Asia Pacific Conference on Circuits and Systems (1996). He has served as an Associate Editor for Journal of the Institute of Electronics Engineers of Korea (IEEK) (1996), Journal of Broadcast Engineering (1996 - 1999), the Journal of the Korean Institute of Communication Sciences (KICS) (1997 - 2000). He has been an editor of Journal of Communications and Networks (JCN) (1998 - 2000). He is the 1999 Recipient of the LG Research Award given to the Outstanding Information and Communication Researcher. He received the Hae-Dong best paper award from the IEEK (1997) and the best paper award from the IEEE Asia Pacific Conference on Circuits and Systems (1996).