High Accuracy Bicubic Interpolation Using Image

29 downloads 0 Views 1MB Size Report
Page 1. High Accuracy Bicubic Interpolation Using Image Local Features. 袁帥 ... fi−1,j+1 fi,j+1 fi+1,j+1 fi+2,j+1 fi−1,j+2 fi,j+2 fi+1,j+2 fi+2,j+2. ⎤. ⎥. ⎥. ⎦. Page 2 ...
Missing:
High Accuracy Bicubic Interpolation Using Image Local Features 袁 帥, 阿部 正英, 田口 亮*, 川又 政征

Shuai YUAN, Masahide ABE, Akira TAGUCHI* and Masayuki KAWAMATA 東北大学大学院工学研究科電子工学専攻 武蔵工業大学電気電子情報工学科*

Department of Electronic Engineering, Tohoku University Department of Electrical and Electronic Engineering, Musashi Institute of Technology* E-mail: [email protected]

Abstract In this paper, we propose a novel bicubic method for digital image interpolation. Since the conventional bicubic method does not consider image local features, the interpolated images obtained by the conventional bicubic method often have a blurring problem. In this paper, the proposed bicubic method adopts both the local asymmetry features and the local gradient features of an image in the interpolation processing. Experimental results show that the proposed method can obtain high accuracy interpolated images.

1.

Introduction

Image interpolation is a prime technique in image processing. It is used in many important applications such as digital high-definition television, big screen display, copy and print machine, medical imaging, end-user equipment and so on. The bicubic interpolation [1], which is linear and easy to be computed parallel, is used widely in many interpolation applications. However, the conventional bicubic interpolation has a blurring problem in the interpolated images, because it ignores the features of the image pixel data, such as the frequency features, the edge features, the features under multi-resolution and so on. For solving the blurring problem in the interpolated images, resolution enhancement (RE) interpolation methods were proposed. Some of the RE interpolation algorithms use multi-resolution pyramids [2] and frequency features [3] of an image to calculate the interpolated image. Another popular type of the RE interpolation algorithms [4], [5] needs a sequence of low-resolution images for producting an interpolated image. Since most of the interpolations based on multi-resolution pyramids or multi-image datas are not linear and difficult to be computed parallel, they can not replace

the bicubic interpolation in many applications. For improving the interpolation accuracy of the bicubic interpolation, some methods [6], [7], which use image local features, are proposed. In [6], the local asymmetry features of image pixels are used to modify the local up-sampling distance in the bicubic interpolation. In [7], the local gradient features of image edges are used to optimize the local interplation weights in the bicubic interpolation. Since both the asymmetry feature and the gradient feature can help to reduce the blurring problem of the bicubic interpolation, in this paper we combine these two kinds of image features to propose a novel high accuacy bicubic algorithm. This paper is organized as follows. The conventional bicubic interpolation is reviewed in Section 2. The bicubic interpolation using the local asymmetry feature is described in Section 3.1. The bicubic interpolation using the local gradient feature is described in Section 3.2. Then, in Section 3.3 we give the proposed algorithm, which adopts both the local asymmetry and the local gradient features. The experimental results and the comparison are shown in Section 4. Finally, the conclusion is included in Section 5.

2.

Conventional Bicubic Interpolation

The conventional bicubic interpolation needs an up-sampling distance S to estimate the unknown pixels for the interpolation processing. At the position (i , j  ), which is shown in Fig. 1, the bicubic interpolation calculates the interpolated pixel as  W−1 (Sy ) W0 (Sy ) W1 (Sy ) W2 (Sy ) ⎡ ⎤ fi−1,j−1 fi,j−1 fi+1,j−1 fi+2,j−1 ⎢ fi−1,j fi,j fi+1,j fi+2,j ⎥ ⎥ ·⎢ ⎣ fi−1,j+1 fi,j+1 fi+1,j+1 fi+2,j+1 ⎦ fi−1,j+2 fi,j+2 fi+1,j+2 fi+2,j+2

fi ,j  =



⎤ W−1 (Sx ) ⎢ W0 (Sx ) ⎥ ⎥ ·⎢ ⎣ W1 (Sx ) ⎦ , W2 (Sx ) ⎡

(1)

where Sy = j  − j, Sx = i − i and fi,j means the pixel value at the position (i, j). The weights W−1 (S), W0 (S), W1 (S), W2 (S) in conventional bicubic interpolation are given as −S 3 + 2S 2 − S , 2 3S 3 − 5S 2 + 2 W0 (S) = , 2 −3S 3 + 4S 2 + S W1 (S) = , 2 S3 − S2 W2 (S) = . 2

W−1 (S) =

3.

position (i , j  ) and upsampling distances Sx , Sy .

図 1: Interpolation

Proposed High Accuacy Bicubic Method

Since the conventional bicubic interpolation has a blurring problem, one modified method was proposed in [6] using the image local asymmetry features, and another modified method was proposed in [7] using the image local gradient features. In this section, we review these two modified bicubic interpolations firstly, and then give the proposed bicubic interpolation, which combines these two kinds of image features.

3.1

Local Asymmetry Feature Based Bicubic Method

The local asymmetry feature based bicubic method uses the modified interpolation value fiA ,j  to replace the conventional interpolation value fi ,j  . In [6] the modified value fiA ,j  is calculated using Eq. (1) by replacing Sy , Sx with Sy , Sx , which is given as fiA ,j  =





W−1 (Sy ) W0 (Sy ) W1 (Sy ) W2 (Sy ) ⎤ fi−1,j−1 fi,j−1 fi+1,j−1 fi+2,j−1 ⎢ fi−1,j fi,j fi+1,j fi+2,j ⎥ ⎥ ·⎢ ⎣ fi−1,j+1 fi,j+1 fi+1,j+1 fi+2,j+1 ⎦ fi−1,j+2 fi,j+2 fi+1,j+2 fi+2,j+2 ⎡ ⎤ W−1 (Sx ) ⎢ W0 (Sx ) ⎥ ⎥ ·⎢ (2) ⎣ W1 (Sx ) ⎦ , W2 (Sx ) ⎡

where Sy =

    Sy1 + (Sy2 − Sy1 )Sx1  − S  )(S  − S  ) , 1 − (Sy2 y1 x2 x1

(3)

Sx =

    Sx1 + (Sx2 − Sx1 )Sy1     ). 1 − (Sy2 − Sy1 )(Sx2 − Sx1

(4)

    In equations (3) and (4) Sx1 , Sx2 , Sy1 , Sy2 are calculated by  = Sx − kAx1 Sx (1 − Sx ), Sx1  Sx2 = Sx − kAx2 Sx (1 − Sx ),  Sy1 = Sy − kAy1 Sy (1 − Sy ),  Sy2 = Sy − kAy2 Sy (1 − Sy ),

where Ax1 , Ax2 , Ay1 and Ay2 are the local asymmetry features defined by |fi+1,j − fi−1,j | − |fi+2,j − fi,j | , L−1 |fi+1,j+1 − fi−1,j+1 | − |fi+2,j+1 − fi,j+1 | , = L−1 |fi,j+1 − fi,j−1 | − |fi,j+2 − fi,j | , = L−1 |fi+1,j+1 − fi+1,j−1 | − |fi+1,j+2 − fi+1,j | , = L−1

Ax1 = Ax2 Ay1 Ay2

in Ax1 , Ax2 , Ay1 and Ay2 , L = 256 for 8-bit luminance images, and then Ax1 , Ax2 , Ay1 and Ay2 are limited in the range [−1, 1]. The positive parameter k decides the warping level. When k is large, S  may be outside of the range [0, 1]. Since S  must be kept in [0, 1] in real image interpolations, the outside-range S  will be clipped to 0 or 1.

3.2

Local Gradient Feature Based Bicubic Method

In digital images, the local gradient features are also one kind of the important image features. In [7], using the local gradient features around the interpolated pixel fi ,j  , four local gradient weights are proposed by the authors. The four local gradient weights Hl , Hr , Vu and Vl are generated from the four pixel masks around the interpolated pixel, where the four masks are shown in Fig. 2 with the dotted line blocks.

図 3: Test images. W0G (Sx ) = Hl W0 (Sx )/DH(Sx ),

W1G (Sx ) = Hr W1 (Sx )/DH(Sx ), W2G (Sx ) = W2 (Sx )/DH(Sx ),

図 2: Gradient masks. The local gradient weights Hl , Hr , Vu and Vl are defined as 1 Hl = 1 + α(|fi,j − fi−1,j | + |fi,j+1 − fi−1,j+1 |) 1 Hr = 1 + α(|fi+1,j − fi+2,j | + |fi+1,j+1 − fi+2,j+1 |) 1 Vu = 1 + α(|fi,j − fi,j−1 | + |fi+1,j − fi+1,j−1 |) 1 Vl = , 1 + α(|fi,j+1 − fi,j+2 | + |fi+1,j+1 − fi+1,j+2 |) where the parameter α is in the range of [0, 1]. Using the local gradient weights Hl , Hr , Vu and Vl , the authors got 8 new interpolation weights WnG to replace the conventional interpolation weights Wn in the bicubic interpolation. Then, the local gradient feature based bicubic method is given as fiG ,j  =



G (Sy ) W0G (Sy ) W−1 ⎡ fi−1,j−1 fi,j−1 ⎢ fi−1,j fi,j ·⎢ ⎣ fi−1,j+1 fi,j+1 fi−1,j+2 fi,j+2 ⎤ ⎡ G (Sx ) W−1 ⎢ W0G (Sx ) ⎥ ⎥ ·⎢ ⎣ W1G (Sx ) ⎦ , W2G (Sx )

 W1G (Sy ) W2G (Sy ) ⎤ fi+1,j−1 fi+2,j−1 fi+1,j fi+2,j ⎥ ⎥ fi+1,j+1 fi+2,j+1 ⎦ fi+1,j+2 fi+2,j+2

in which DV (Sy ) = W−1 (Sy ) + Vu W0 (Sy ) + Vl W1 (Sy ) + W2 (Sy ) and DH(Sx ) = W−1 (Sx ) + Hl W0 (Sx ) + Hr W1 (Sx ) + W2 (Sx ), respectively.

3.3

After the review of the two modified bicubic algorithms, we can see that not only the local asymmetry feature but also the local gradient feature are helpful to solve the bicubic blurring problem. Thus, in the proposed method we would like to adopt both of the two kinds of image features at the same time. To adopt both the local asymmetry feature and the local gradient feature, we give the proposed modified interpolation weights Wnp as

(5)

p G (Sy ) = W−1 (Sy ), W0p (Sy ) = W0G (Sy ), W−1

W1p (Sy ) = W1G (Sy ), W2p (Sy ) = W2G (Sy ),

where the weights WnG (Sy ) are given as G W−1 (Sy ) = W−1 (Sy )/DV (Sy ),

W0G (Sy ) = Vu W0 (Sy )/DV (Sy ), W1G (Sy ) = Vl W1 (Sy )/DV (Sy ),

W2G (Sy ) = W2 (Sy )/DV (Sy ), and the weights

WnG (Sx )

are given as

G W−1 (Sx ) = W−1 (Sx )/DH(Sx ),

Proposed High Accuracy Bicubic Interpolation

and p G (Sx ) = W−1 (Sx ), W0p (Sx ) = W0G (Sx ), W−1

W1p (Sx ) = W1G (Sx ), W2p (Sx ) = W2G (Sx ).

in which Sx , Sy are based on the image local asymmetry features, and WnG are based the image local gradient features. Then, the proposed bicubic interpolation can be written as

(a)

(b) 図 4: (a) MSE surfaces of two times interpolation using the test images (128×128 to 256×256). (b) MSE surfaces of four times interpolation using the test images (64×64 to 256×256).

図 5: Experimental results of the image “Boat” with 256 × 256. (a) Original “Boat” image. (b) Subsampled “Boat” image with 128 × 128. (c) Subsampled “Boat” image with 64 × 64. (d) Result of 2 times conventional bicubic interpolation. (e) Result of 2 times local asymmetry feature based bicubic interpolation. (f) Result of 2 times local gradient feature based bicubic interpolation. (g) Result of 2 times proposed bicubic interpolation. (h) Result of 4 times conventional bicubic interpolation. (i) Result of 4 times local asymmetry feature based bicubic interpolation. (j) Result of 4 times local gradient feature based bicubic interpolation. (k) Result of 4 times proposed bicubic interpolation.

fip ,j  =

4.



p (Sy ) W0p (Sy ) W−1 ⎡ fi−1,j−1 fi,j−1 ⎢ fi−1,j fi,j ·⎢ ⎣ fi−1,j+1 fi,j+1 fi−1,j+2 fi,j+2 ⎡ ⎤ p W−1 (Sx ) ⎢ W p (Sx ) ⎥ 0 ⎥ ·⎢ ⎣ W1p (Sx ) ⎦ . W2p (Sx )

 W1p (Sy ) W2p (Sy ) ⎤ fi+1,j−1 fi+2,j−1 fi+1,j fi+2,j ⎥ ⎥ fi+1,j+1 fi+2,j+1 ⎦ fi+1,j+2 fi+2,j+2 (6)

Parameter Estimation and Experimental Results

Since both the local asymmetry and the local gradient features are used in the proposed method, we have two parameters in the proposed method. One is the parameter k which decides Sx and Sy , the other is the parameter α which decides the local gradient weights Hl , Hr , Vu and Vl . To do a quantitative estimation of the two parameters in the proposed method, we lowpass several different well-known test images (256×256) with Gaussian lowpass filters, and then subsample the lowpassed test images to get the small size images. Then, the subsampled images are interpolated to their original size using the proposed bicubic method. The test images are shown in Fig. 3. Using the proposed bicubic method with different parameter values, the mean square error (MSE) surfaces of each interpolated test image are shown in Fig. 4. From Fig. 4 we can see that all of the minimized MSE points are not at the coordinate origin, the X axis or the Y axis, that means the interpolation accuracy of the proposed bicubic method is better than those of the conventional bicubic, the local asymmetry feature based bicubic and the local gradient feature based bicubic methods. At the coordinate origin, we have k = 0 and α = 0 that means the calculation is the conventional bicubic interpolation. At the X axis, we have α = 0 that means the calculation is the local asymmetry feature based bicubic interpolation. At the Y axis, we have k = 0 that means the calculation is the local gradient feature based bicubic interpolation. Fig. 4 also shows that for both 2 times and 4 times interpolations, the optimum parameters of each test image are almost in the same coordinate range. Thus, we can use the average values of the optimum k, α as the fixed experimental parameters kp , αp for the proposed bicubic interpolation algorithm. Then, we have the experimental parameters kp , αp as kp = 2.1, αp = 0.05. To show the effectiveness of the proposed bicubic interpolation with the experimental parameters, we

give another experiment using a new test image “Boat”. The experimental results are shown in Fig. 5. The proposed results are also compared with the other bicubic results. In [6] the fixed k for the local asymmetry feature based bicubic method is given as 5.5, and in [7] the fixed α for the local gradient feature based bicubic method is given as 0.07. From Fig. 5, we can see that the accuracy of the proposed bicubic method with kp , αp is higher than those of other bicubic methods.

5.

Conclusion

In this paper, we propose a novel bicubic interpolation using both the local asymmetry features and the local gradient features of digital images. Experimental results show that the proposed method can obtain high accuracy interpolated images.

参考文献 [1] W. K. Pratt, Digital Image Processing, New York: Wiley, 1991. [2] H. Greenspan, C. H. Anderson and S. Akber, “Image enhancement by nonlinear extrapolation in frequency space,” IEEE Trans. on Image Processing, vol. 9, no. 6, pp. 1035-1048, June 2000. [3] W. K. Carey, D. B. Chuang and S. S. Hemami, “Regularity-preserving image interpolation,” IEEE Trans. on Image Processing, vol. 8, no. 9, pp. 1293-1297, September 1999. [4] R. Hardie, K. Barnard. and E. Armstrong, “Joint map registration and high resolution image estimation using a sequence of undersampled measured images,” IEEE Trans. on Image Processing, vol. 6, pp. 1621-1632, December 1997. [5] N. R. Shah and A. Zakhor, “Resolution enhancement of color video sequences,” IEEE Trans. on Image Processing, vol. 8, no. 6, pp. 879-885, June 1999. [6] G. Ramponi, “Warped distance for spacevariant linear image interpolation,” IEEE Trans. on Image Processing, vol. 8, no. 5, pp. 629-639, May 1999. [7] J. W. Hwang and H. S. Lee, “Adaptive image interpolation based on local gradient features,” IEEE Signal Processing Letters, vol. 11, no. 3, pp. 359-362, March 2004.