Numerical schemes for both approaches are given and tested on a real-life image, showing the expected behaviour. When a constraint is added that penalises.
Image Processing with Geometrical and Variational PDEs Arjan Kuijper RICAM - Linz1) http://www.ricam.oeaw.ac.at
Abstract: In this work,two lines of approaches to evolve images are investigated. The first line minimises the general Lp norm of the gradients instead of the L1 norm in the total variation method (known for its denoising and edge-preserving effect) or the L2 norm in Gaussian scale space. The second line evolves according to a weighted combination of second order derivatives in terms of gauge coordinates, incorporating the Mean Curvature Motion (also used for denoising and edge-preserving) and (again) Gaussian scale space. The general methods and their properties are briefly discussed, including relations to known evolutions for specific parameter values. Numerical schemes for both approaches are given and tested on a real-life image, showing the expected behaviour. When a constraint is added that penalises the distance of the results to the input image, one can vary the desired amount of blurring and denoising.
1 Introduction In image processing, Partial Differential Equations play more and more an important role. Some equations arise from minimising some energy functional (like the L 1 norm of the gradient, known as Total Variation [12]). Other are designed using geometrical arguments (like evolution tangent to isophotes, known as Mean Curvature Motion [1]). In this work, a general parameter-driven framework for both approaches is given, that have one specific common element, the Gaussian scale space [9]. For the first set of equation, the L p norm of the gradient is used - with p a free parameter, thus obtaining so-called p-Laplacians [13]. The evolution equation is a PDE that can be simplified using (geometrical) gauge coordinates. It depends on the norm of the gradient and a linear combination of the two second order derivatives in gauge directions. In the second set only weighted combinations (again as free parameters) of the two second order derivatives are regarded. Firstly the p-Laplacians in terms the concepts of gauge coordinates will be derived and related approaches discussed. Secondly, general geometric PDEs depending on second order derivatives in 1) WWTF
(Mathematical Methods for Image Analysis and Processing in the Visual Arts) and FFG (Analyse Digitaler Bilder mit Methoden der Differenzialgleichungen) are gratefully acknowledged for funding
gauge coordinate will be introduced and related approaches discussed. Thirdly, numerical schemes for the general approaches will be presented and tested on both noise-constrained and unconstrained evolution. The focus in on the main ideas of both approaches and many technical and mathematical details are omitted. They can be found in a forthcoming technical report. 1.1 Gauge coordinates An image can be thought of as a collection of curves with equal value, the isophotes. Most isophotes are non-self-intersecting. At extrema an isophote reduces to a point, at saddle points the isophote is self-intersecting. At the non-critical points Gauge coordinates (v, w) (or (η,ξ), or (N ,T )) can be chosen [3, 7]. Gauge coordinates are locally set such, that the v direction is tangent to the isophote and the w direction points in the direction of the gradient vector. Consequently, L v = 0 q and Lw = L2x + L2y . Of interest are the following second order structures: L2x Lyy + L2y Lxx − 2Lx Ly Lxy L2x + L2y L2x Lxx + L2y Lyy + 2Lx Ly Lxy = L2x + L2y
Lvv = Lww
(1) (2)
As Lww is the second order derivative in the direction of the normal vector, it resembles the blurring. Lvv can be regarded as regularising the boundary. Their sum equals the Laplacian.
2 p-Laplacians The functional E(L) =
Z
Ω
1 k∇Lkp d Ω p
(3)
is known as the p-Dirichlet energy integral [13] with accompanying p-Laplacian equation δE = 0, with δE = −∇ · k∇Lkp−2 ∇L . (4)
Using gauge coordinates the integral can be written as E(L) =
1 p
Z
Ω
Lpw d Ω
(5)
and the variational derivative equals (proof omitted) δE(L) = −Lp−2 w ((p − 1)Lww + Lvv ) .
(6)
The p-Laplacian PDE is obtained as a steepest decent evolution towards the solution L t = −δE[L]. For p = 2 we have Gaussian scale space [9]: Lt = L2−2 w ((2 − 2)Lww + 4L) = 4L.
−1 Next, p = 1 gives the Total Variation flow [12]: Lt = L1−2 w ((1 − 1)Lww + Lvv ) = Lw Lvv = κ. In general, it gives a recipe for PDE-driven flow:
(7)
Lt = Lp−2 w ((p − 2)Lww + 4L)
The case p → ∞ is known as the infinity Laplacian, denoted by 4∞ L [2]. This type of flow can be applied to image inpainting [5]. 2.1 Analytic solution The source solution (or Barenblatt [4] solution) for the p-Laplacian equation with p > 2 and 1 < p < 2 reads L=t with
−α
f t
−α n
x , f (ξ) = c − k |ξ|
p (p−1)
p−1 p−2
+
(8)
1
(p − 2) α p−1 n k= , α= . (9) p n n(p − 2) + p R The constant c can be chosen such, that Ω LdΩ = M , where is M is typically the mass in physical R applications; here we can think of M as the total intensity of the image, Ω L0 dΩ.
2.2 Non-linear diffusion In the case of non-linear diffusion, often the choice Lt = ∇ · (g(|∇L|)∇L)
(10)
is made. The function g(.) is chosen such, that it enhances the edges (where |∇L| is large) and deblurs noisy (flat) regions (where |∇L| is small). A classical example is the Perona-Malik equation [11], where 1 g(|∇L|2 ) = . (11) 1 + |∇L|2 /λ2
Note that by taking g(.) = 1, the Gaussian scale space is obtained. A general class of diffusivities is obtained by 1 g(|∇L|) = . (12) |∇L|q
Then the diffusion becomes Lt = ∇ · (|∇L|−q ∇L), and by taking q = −p + 2 the p-Laplacian is obtained. For q = 1 one obtains Total Variation flow, while the case q = 2, is known as balanced forward-backward diffusion. Note that the latter case resembles p = 0, the case where the energy functional (3) would simplify to a constant. This diffusion is obtained by minimising E(L) = R 1 2 R 1 2 2 2 Ω 2 log Lw d Ω, which is close to the PM functional of Eq. (11):E P M (L) = Ω 2 k log k + Lw d Ω. −ω−1 Also the PDE Lt = Lαw ∇ · (Lw ∇L), together with a distance penalty β(L − L0 ) has been investigated. This equals the PDE Lα−ω−1 (−ωLww + Lvv ) , w
(13)
which is called the enhanced TV model for α = ω + 1, i.e. when the L w term vanishes [8]. This type of PDEs will be discussed in a broader setting in the next section.
3 Geometric PDEs: Second order gauge derivatives Obviously, for large values of p, the p-Laplacian gets highly influenced by the term L p−2 w . Given that both sides of the flow should have equal dimensions, one can argue that p = 2. This obviously gives the Gaussian scale space. However, the p-Laplacian shows that the direction of the flow is a combination of the two different second order derivatives in gauge coordinates. So consider Lt = pLvv + qLww .
(14)
For histogram operations [6] several parameter settings are of interest: For (p, q) = (1, 1) we have a Gaussian scale space - and repeated infinitesimal mean filtering, for (p, q) = (1, 0) we have mean curvature motion [1] - and repeated infinitesimal median filtering, and for (p, q) = (1, −2) we have infinitesimal mode filtering. 3.1 A general solution: qualitative behaviour √ A stationary solution for Lww = 0 is given by L(x, y) = a x2 + y 2 , for Lvv = 0 by L(x, y) = L(ax + by), a, b ∈ R. Assume spatial and rotation invariance, and isotropy. Then L(x, y; t) = t
−
p+q 2q
4πq
e−
x2 +y 2 4qt
with a = 4q and b = 1 +
p q
is a solution of (14). The diffusion (q = 1, p = b − 1)
Lt = Lww + (b − 1)Lvv
gives the solution
(15)
−x2 −y 2 1 4t . (16) e 4πtb/2 For b = 2 we have Gaussian filter (since it is the Greens’ function), for b = 1 maximal blurring. Note that a solution can only be obtained when q 6= 0. This implies that the direction L ww must be present in the flow. A filter for pure Lvv flow is given by L(x, y; t) = L [x2 + y 2 + 2t].
3.2 Nonlinear diffusion filtering The general diffusion equation [14] reads Lt = ∇ · (D · ∇L).
(17)
Lt = ∇ · (D · ∇L) = pLww + qLvv
(18)
The diffusion tensor D is a positive definite symmetric matrix. With D = 1 (when D is considered as scalar - i.e. an isotropic flow), or better, D = In , we have Gaussian scale space. When D depends on the image, the flow is nonlinear. For D = Lp−2 we have the p-Laplacian. To force the equality w D must be a matrix that is dimensionless and that contains only first order derivatives. The only possible entries for D are those that result in p = q, yielding the Gaussian scale space.
Figure 1: Original image and a close up showing artifacts at edges. p = 0.75; time = 1000
p = 1.; time = 1000
p = 1.25; time = 1000
p = 1.5; time = 1000
p = 1.75; time = 1000
p = 2; time = 1000
p = 4; time = 1000
p = 10; time = 1000
p = 15; time = 1000
p = 20; time = 1000
Figure 2: Constrained p-Laplacian evolution for several values of p. Here σ is set to 2.5, allowing the flow to evolve from the initial image. The error in the constrain is O(10−7 ).
4 Constraints An extra condition may occur in the presence of noise (assume zero mean, variance σ): I=
Z
Ω
1 (L − L0 )2 d Ω = σ 2 2
(19)
where L0 is the input image and L the denoised one. The solution is obtained by the Euler Lagrange equation δE + λδI = 0 (20) , and < δI, δI >= 2σ 2 . The solution can be reached by an evolution with δI = L − L0 , λ = determined by a steepest decent evolution Lt = −(δE + λδI) When we set λ = 0, an unconstrained blurring process is obtained. Alternatively, λ can be regarded as a penalty parameter that limits the L2 difference between the input and output images. A too small value will cause an evolution that forces the image to stay close to the input image.
p = 0.75; time = 1000
p = 1.; time = 1000
p = 1.25; time = 1000
p = 1.5; time = 1000
p = 1.75; time = 1000
p = 2; time = 1000
p = 4; time = 1000
p = 10; time = 1000
p = 15; time = 1000
p = 20; time = 1000
Figure 3: Unconstrained p-Laplacian evolution for several values of p. For small values cartooning occurs, while large values cause blurring. For very large values, the blurring effect at edges is larger than the denoising effect in regions.
5 Numerical implementation and Discussion The image in Figure 1 is taken to see the effects of the various flows. Convergence of the iteration scheme is obtained when the changes in the update are less than 10 −7 . 5.1 p-Laplacians + − + f with Di+ f = f (i + f kp−2 Di,j A forward Euler scheme is used to compute ∆L = CDi,j kDi,j 1, j) − f (i, j), Di− f = −f (i − 1, j) + f (i, j) and similar for j. The constant is chosen such, that the maximum update is less or equal to 10% of the maximal image intensity to stabilise numerical computation for large p values. The value for λ is computed in accordance with [12]. For the comparison of different values of p ∈ {0.75, 1, 1.25, 1.5, 1.75, 2, 4, , 10, 15}, 1000 iterations are computed. Note that although p-Laplacians are defined for 1 < p < ∞, the value p = 0.75 is numerically possible, allowing small deblurring.
For the constrained evolution, convergence is obtained after approximately 400 iterations. Results are shown in Figure 2. For the unconstrained evolution convergence is obviously not obtained. Results are shown in Figure 3. For large values of p, updates are tempered due to locally large values in the norm of the gradient term. It appears that blurring is achieved without affecting the noise. 5.2 Geometrical evolution The PDE Lt = pLvv + qLww is implemented using Gaussian derivatives [7, 10]. As a consequence, larger time steps can be taken. For Gaussian blurring, the standard value of 41 to guarantee stability
Figure 4: Disk and square, both valued in 0,1 with uniform random noise on 0,1, and the results of a Gaussian √ filter at σ = 128, i.e. t=64. timestep = 1.82857
timestep = 1.77778
timestep = 1.72973
timestep = 1.68421
timestep = 1.64103
timestep = 1.82857
timestep = 1.77778
timestep = 1.72973
timestep = 1.68421
timestep = 1.64103
timestep = 1.82857
timestep = 1.77778
timestep = 1.72973
timestep = 1.68421
timestep = 1.64103
timestep = 1.82857
timestep = 1.77778
timestep = 1.72973
timestep = 1.68421
timestep = 1.64103
timestep = 1.82857
timestep = 1.77778
timestep = 1.72973
timestep = 1.68421
timestep = 1.64103
timestep = 1.82857
timestep = 1.77778
timestep = 1.72973
timestep = 1.68421
timestep = 1.64103
Figure 5: Results of the noisy disk (left) and square (right) for Lvv flow (top row), Gaussian flow (middle row), and Lww flow (bottom row), for various time step ranges around the critical value ∆t = 2et = 2e 21 σ 2 = 1.74.
is enlarged to 2et, with t the scale of blurring. The same value is obtained for L vv flow. For the Lww flow the same line of reasoning can be taken (proof omitted). Examples are given in Figure 4. The scale is chosen as σ = .8, so ∆t = 2et = 2e 21 σ 2 = 1.74. In these examples one clearly sees the change around the critical value. Since relatively much noise is added, the value is a bit lower that the predicted value. If a too large time step is taken, instability artifacts are visible: For Lvv flow the results become peaky, while the Lww flow shows ringing, and the Gaussian blurring is completely disastrous. Note that the rounding effect for the L vv flow is due to the flow, while the peaky results for the Lww flow are due to the forward implementation in x and y coordinates of the flow. The result of applying the Gaussian derivatives implementation of L t = pLvv + qLww for several values of p and q in 50 time steps is shown in Figure 6 (left with constraint and right without one). As one can see in the left block of Figure 6, the choice of p and q enables one to steer between denoising regions and deblurring around edges (where the artifacts occurred). The unconstrained evolution shows the spiky artifacts for p < 0, while q < 0 gives the edges. The diagonal gives Gaussian (de)blurring. Visually, q = 0 gives the best result, although here the number of time steps heavily influences the results.
References [1] L. Alvarez, P.L. Lions, and J.M. Morel. Image selective smoothing and edge detection by nonlinear diffusion. SIAM Journal on Numerical Analysis, 29:845–866, 1992. [2] G. Aronsson. On the partial differential equation u2x uxx + 2ux uy uxy + u2y uyy = 0. Arkiv f¨ur Matematik, 7:395–
8p,q< = 80.,1.
