noise on a video sequence, exploiting also the tempo- ral correlation present in the ... Figure 1: Pixels and directions considered by the spa- tial and temporal parts of ... image plus noise; (right) output of the proposed lter. A movement detector ...
DESIGN AND REAL-TIME IMPLEMENTATION OF A 3-D RATIONAL FILTER FOR EDGE PRESERVING SMOOTHING Francesco Cocchia, Sergio Carrato, and Giovanni Ramponi
D.E.E.I., University of Trieste, via Valerio, 10, 34100 Trieste, Italy tel.: +39.40.6767147; fax: +39.40.6763460 ABSTRACT | A nonlinear spatio-temporal lter capable of signi cantly attenuating noise in image sequences without corrupting image details is presented. A real-time implementation of the algorithm for CIF images at 10 frames/s on the TMS320C80 DSP is also described.
previous frame current frame following frame
Introduction
There is presently an increasing interest in nonlinear digital techniques for noise reduction in image sequences, due to their superior capabilities with respect to linear approaches, which are known to blur image details. Many contributions have appeared which are based on order statistics and/or adaptive techniques [1]; in this paper we present a new nonlinear operator based on a rational function, which is shown to e ectively reduce noise while preserving edges. A real-time implementation of the algorithm for CIF images at 10 frames/s on the TMS320C80 DSP is also described.
Noise smoothing with a rational lter
The approach we propose is based on the Rational Filter (RF) [2]. The latter is a nonlinear, edgepreserving smoothing operator which can be interpreted as a lowpass lter with variable cuto frequency; a simple but e ective built-in mechanism senses the presence of image details and accordingly varies the frequency response of the lter making it less selective if a signal transition is supposed to be present. In the one-dimensional case, if a 3-sample mask is considered, the RF may be expressed in the following form: y = x ? k?(xx ?1 +? 2xx ?)2x++1A ?1 +1 being x and y the input and output data, respectively. The edge sensor (in the denominator) is the squared di erence of the two samples at the extremes of the mask: if this value is large, it is assumed that the mask is positioned across a signal transition, and the contribution of the numerator is reduced, so that the overall lowpass response is made less selective. The parameter k is used to control the amount of the nonlinear e ect, while A controls the cut-o frequency of the lter [2]. In this paper we use the same approach to cancel noise on a video sequence, exploiting also the temporal correlation present in the data. The operator we propose may be described with reference to Fig. 1. If x0, belonging to the frame at time t, is the pixel to be n
n
n
t
n
n
n
n
t-1 t t+1 time
Figure 1: Pixels and directions considered by the spatial and temporal parts of the proposed operator. ltered, we apply both a spatial and a temporal lter, as follows: ?x + 2x0 ? x ? ?x ?1 + 2x0 ? x +1 y0 = x0? 2 ?1 +1 2 2 k (x ? x ) + A 2 k (x ? x ) + A where the superscripts t ? 1, t, and t + 1 refer to pixels in the previous, the current, and the following frame, respectively; I is the set of indices of the pixels at time t which are adjacent to x0 along the four main directions (horizontal, vertical, and the two diagonal ones), and J is the set of indices of the pixels at time t ? 1 and t + 1 adjacent to x0 along the nine directions indicated with solid lines in the gure. Finally, k , A k , and A allow for an independent tuning of the spatial and of the temporal part of the lter, respectively. The proposed operator is able to distinguish between image details and noise both in space and along time. As an example, let us consider the case of x0 corrupted by noise, no motion, and presence of some detail in the masks, so that the spatial part of the lter is not allowed to operate: the di erence between the extreme pixels in the rst and in the third frame will be small, so that temporal smoothing will be obtained and the noise will be ltered out, without detail blurring. It has to be noted, however, that this version of the lter may generate artifacts in particular conditions of fast motion. Let us consider for example the case of an object moving quickly on a uniform background: if the object is present in the lter mask at time t, but not at time t ? 1 and t + 1, the value of x0 is considered as impulse noise by the temporal part of the lter, and the object may be deleted. t
t
X
i;j
I
t i
s
t
t i
t j
t j
s
X
i;j
J
t i
t
t j
t i
t j
t
t
t
s
s
t
t
DSP implementation
Figure 2: Performance of the proposed lter in case of Gaussian (top), contaminated Gaussian (middle) and impulsive (bottom) noise: (left) detail of the original image plus noise; (right) output of the proposed lter. A movement detector has consequently to be added, so that in presence of fast motion only the spatial part of the lter is activated. The movement detector we use is similar to the one proposed in [4]. We assume that there is no motion if
jx0?1 ? x0+1j < 1 and jx0 ? (x0?1 + x0+1 )=2j < 2 : t
t
t
t
where 1 e 2 are two suitable thresholds. The rst test checks generally for motion, while the second one speci cally looks for situations similar to the one described above.
Experimental results
Our lter was tested with di erent types of noise: Gaussian noise with 2 = 200; contaminated Gaussian noise [3] with zero mean, 2 = 1000, = 0:1; impulsive noise, 5% contamination percentage. The performances of our operator are shown in Fig. 2, with reference to a detail of a frame of the CIF sequence \Basketball" with added noise. It should be noted that the noise is e ectively removed by our operator, while the sharpness of the image is left una ected. We have favourably compared these results with those obtainable using conventional operators. We have also veri ed, in the real-time implementation described in the next section, that the proposed operator e ectively reduces the noise introduced by a video camera while preserving the image details.
The lter has been implemented on a Texas Instruments TMS320C80 DSP mounted on a board hosted by a PC. The 'C80 has been chosen because of its high computational capability due to its parallel architecture|composed of one Master Processor (MP) and four Parallel Processors (PPs). The algorithm has been implemented according to the following scheme: start of Multitasking Executive software on the MP: start of video capture, preparation to processing, issue of messages to the PPs; waiting for completion of video acquisition; starting of the PPs; each PP, at the same time: { is assigned one fourth of the image; { loads a portion (about 2KB large) of the previous, current and following images in its DataRAM from the VideoRAM (VRAM); performs the image processing task; saves the results in VRAM; repeats with the next portion of the images, until completion; { returns control to the Master Processor; the Master Processor displays the processed frame; repeat. In order not to waste time while loading image data (in our experiments, from a progressive video camera) we operate on a quadruple-bu er scheme: 3 bu ers contain the frames being processed, while the successive frame is loaded into the fourth one. Moreover, in order to reduce the computation time per pixel, the routines for the PP are written in Assembler, to better exploit the ability of these processors to perform several operations in parallel, and a look-up table is used to compute the two divisions. Both the debugging of the code on the simulator and the actual implementation on the board have shown that it is possible to operate on CIF frames (i.e. 288 352) at 10 frames/s, which is appropriate for video-conference and video-telephone applications (e.g., using the H261 standard).
Acknowledgements
This work has been partially supported by the European ESPRIT LTR Project # 20229 \NOBLESSE".
References
[1] I. Pitas, A.N. Venetsanopoulos, \Nonlinear Digital Filters, Principles and Applications", Kluwer Academic Publishers, 1990 [2] Giovanni Ramponi, \The Rational Filter for Image Smoothing", IEEE Signal Processing Letters, vol. 3, No. 3, March 1996 [3] Moncef Gabbouj, Ioan Tabus, \TUT Noisy Image Database v.1.0", Technical Report # 13, Tampere University of Technology, December 1994 [4] R. Simonetti, S. Carrato, G. Ramponi, and A. Polo Filisan, \Deinterlacing of HDTV images for multimedia applications", in Proc. International Workshop on HDTV'92, pages 95.1{95.8, Kawasaki, Kanagawa, Japan, November 1992.
