images as partial functions assuming a slightly more abstract point of view, g[x,y] can also be thought of as a partial function g : R2. {. 0,...,255. } ...
Image Processing Prof. Christian Bauckhage
outline lecture 03 recap representation of digital images (part 2) images as n-way arrays images as functions derivatives of images emboss effect summary
recap
intensity images are recorded using CCD chips which consist of many photosensitive elements
CCD chip
schematic CCD matrix
recap
color images are recorded by means of splitting light beams into basic components . . .
trichroic prism
three separate CCD arrays
recap
or by means of filtering wavelengths prior to recording
Bayer filter
single CCD array
recap
in memory, digital images are stored as pixel arrays their spatial resolution is characterized by width and height of these arrays their color resolution is characterized by the number of bits per color channel typically, images are stored using 256 = 28 intensity levels per channel, this requires 8 bit (= 1 byte) per 3 pixel per channel and allows for 28 = 16, 777, 216 colors per pixel
⇔ digital images are digitized and quantized
representation of digital images (part 2)
images as n-way pixel arrays
in computer memory, digital images are usually represented as n-way arrays of integer values
n-way arrays 1-way array (“vector”)
2-way array (“matrix”)
3-way array (“tensor”)
n-way pixel arrays
intensity image (“matrix”)
color image (“tensor”)
2-way pixel arrays
a pixel p is addressed using its coordinates i and j
i
pij
we also write pij
j
2-way pixel arrays
a pixel p is addressed using its coordinates i and j
i
pij
we also write pij
j
0 6 i < m is the row index and 0 6 j < n is the column index following the convention for matrices, pixel p00 is located in the upper left corner of the image
2-way pixel arrays ⇔ 1-way pixel arrays
pixel p can either be addressed trough two indices i and j denoting its row and column coordinates and we write pij . . . or pixel p can be addressed trough a single index k and we write pk
2-way pixel arrays ⇔ 1-way pixel arrays
pixel p can either be addressed trough two indices i and j denoting its row and column coordinates and we write pij . . . or pixel p can be addressed trough a single index k and we write pk
for an image of resolution m × n, we have the following relationship pij ⇔ p(i·n+j) = pk
pixel neighborhoods
in image processing, we often consider the 4-neighborhood and the 8-neighborhood of a pixel p
p
p
4-neighborhood
8-neighborhood
pixel distances
generally, the spatial distance between two pixels p and q with coordinates i, j and k, l is measured using the Euclidean distance between their coordinates
q
i 2 2 k
d p, q = i−k + j−l
j − l =
note
for the pixels q in the 4-neighborhood of p, the Euclidean distance between p and q amounts to d p, q = 1
note
for the pixels q in the 4-neighborhood of p, the Euclidean distance between p and q amounts to d p, q = 1
for the pixels in the 8-neighborhood of a pixel p, this does not always hold, for instance
d pij , pi+1 j+1 =
q i − (i + 1)
p 12 + 12 √ = 2 =
2
+ j − ( j + 1)
2
question can’t we consider or define a distance that would avoid such “strange behavior” ?
question can’t we consider or define a distance that would avoid such “strange behavior” ?
answer yes, we can (more on this later) but we rather shouldn’t (more on this later)
question why does image processing resorts to pixel arrays where some immediate neighbors are closer than others ? is it because it is impossible to find an arrangement where pixels cover the image plane and have an equal distance to all their immediate neighbors ?
question why does image processing resorts to pixel arrays where some immediate neighbors are closer than others ? is it because it is impossible to find an arrangement where pixels cover the image plane and have an equal distance to all their immediate neighbors ?
answer no, in principle, there are many possibilities to achieve this!
tessellation / tiling
a tessellation is a collection of planar figures that cover the entire plane without overlaps and gaps
tessellation / tiling
interesting tessellations have certain symmetry properties, they are periodic or a-periodic, . . . most notably, there are tessellations where the Euclidean distance between the centers of adjacent tiles is always the same
question then why aren’t these used in digital photography ?
question then why aren’t these used in digital photography ? answer as of today, the memory of computers is typically conceptualized as a (very long) linear sequence of storage cells
question then why aren’t these used in digital photography ? answer as of today, the memory of computers is typically conceptualized as a (very long) linear sequence of storage cells therefore, the simplest approach is to represent the rows of an image as a sequence of storage cells and to represent the image as a sequence of such rows, i.e. as a 1-way pixel array
images as discrete functions
in a gray value image, discrete coordinates are mapped to discrete intensity values a gray value image may therefore also be thought of as a bi-variate discrete function g x, y where g : 0, . . . , n − 1 × 0, . . . , m − 1 → 0, . . . , 255
images as discrete functions
in a gray value image, discrete coordinates are mapped to discrete intensity values a gray value image may therefore also be thought of as a bi-variate discrete function g x, y where g : 0, . . . , n − 1 × 0, . . . , m − 1 → 0, . . . , 255 in contrast to the matrix interpretation of an image, here, index x denotes image columns, y denotes image rows, and coordinate [0, 0] is located in the lower left corner
example
150 0
5
10
15
20
25
30
35
40
45
50
50 45 40 35 30 25 20 15 10 5 0
images as partial functions
assuming a slightly more abstract point of view, g x, y can also be thought of as a partial function g : R2 ; 0, . . . , 255
images as partial functions
assuming a slightly more abstract point of view, g x, y can also be thought of as a partial function g : R2 ; 0, . . . , 255 i.e. while certain tuples x, y ∈ R2 are mapped to gray values in the set 0, . . . , 255 many (or most) possible tuples will not be assigned a value
images as partial functions
assuming a slightly more abstract point of view, g x, y can also be thought of as a partial function g : R2 ; 0, . . . , 255 i.e. while certain tuples x, y ∈ R2 are mapped to gray values in the set 0, . . . , 255 many (or most) possible tuples will not be assigned a value
instead of g x, y we also write g x , x ∈ R2 to express that we understand an image as a partial 2D function
color images as vector fields
in a color image, discrete coordinates are mapped to certain color values which are represented by triples of numbers taken from the set 0, . . . , 255 a color image therefore is a vector valued function a color image can thus be seen as a vector field c x where 3 c : R2 ; 0, . . . , 255
question what do we gain from raising the level of abstraction ?
question what do we gain from raising the level of abstraction ? answer if we are dealing with mathematical functions, we may apply the whole toolbox of mathematical techniques which have been developed over centuries we can apply functions or operators to images, i.e. we can compute with images
question what do we gain from raising the level of abstraction ? answer if we are dealing with mathematical functions, we may apply the whole toolbox of mathematical techniques which have been developed over centuries we can apply functions or operators to images, i.e. we can compute with images
this is the basis of image processing
let’s look at an example
derivative of a continuous uni-variate function f (x)
derivative from above f+0 (x) = lim
ε→0
f (x + ε) − f (x) ε
derivative from below f−0 (x) = lim
ε→0
f (x) − f (x − ε) ε
derivative of a continuous uni-variate function f (x)
derivative from above f+0 (x) = lim
ε→0
f (x + ε) − f (x) ε
derivative from below f−0 (x) = lim
ε→0
f (x) − f (x − ε) ε
f (x) is differentiable in x if f+0 (x) = f−0 (x) = f 0 (x) =
d f (x) dx
example derivatives from above and from below of f (x) = x2 x=2
f+0 :
16
16
16
16
12
12
12
12
8
8
8
4
4
4
0 2
4
0
ε=2
f−0 :
4
0
0 0
8
...
2
0 0
4
2
4
0
16
16
16
16
12
12
12
12
8
8
8
4
4
4
0 2
ε=2
4
8
...
0
0 0
0
2
ε=1
4
4
lim ε → 0
ε = 0.5
ε=1
2
4
0 0
2
ε = 0.5
4
0
2
4
lim ε → 0
note
f (x) may be continuous in x and f+0 (x) and f−0 (x) may exists, but they need not be identical ⇔ f (x) may not be differentiable in x
f (x)
x0
x
derivative of a discrete uni-variate function f [x]
derivative from above f+0 [x] =
f [x + 1] − f [x] = f [x + 1] − f [x] (x + 1) − x
derivative from below f−0 [x] =
f [x] − f [x − 1] = f [x] − f [x − 1] x − (x − 1)
derivative of a discrete uni-variate function f [x]
derivative from above f+0 [x] =
f [x + 1] − f [x] = f [x + 1] − f [x] (x + 1) − x
derivative from below f−0 [x] =
f [x] − f [x − 1] = f [x] − f [x − 1] x − (x − 1)
due to the discrete nature of f [x], it is impossible to consider a limit process ⇒ the minimum of the denominator will always be 1 ⇒ the derivative from above and the derivative from below will generally differ
example derivatives from above and from below of f [x] = x2 x=2
16
16
12
12
8
8
4
4
0
0 0
1
2
f+0 [2]
3
4
0
1
2
f−0 [2]
3
4
“compromise”
when computing the derivative of a discrete function, consider information from below and from above
dx f [x] =
f [x + 1] − f [x − 1] 2
example discrete derivative of f [x] = x2 x=2
16
12
8
4
0 0
1
2
dx f [2]
3
4
gradients
the gradient of a continuous multivariate function f (x1 , x2 , . . . , xn ) is the vector of partial derivatives ∂f ∂x1
∇f (x1 , . . . , xn ) = ... ∂f ∂xn
gradients
the gradient of a continuous multivariate function f (x1 , x2 , . . . , xn ) is the vector of partial derivatives ∂f ∂x1
∇f (x1 , . . . , xn ) = ... ∂f ∂xn
accordingly, we can define “a gradient” of an intensity image g[x, y] as follows 1 g[x + 1, y] − g[x − 1, y] dx g[x, y] ∇g[x, y] = = · dy g[x, y] 2 g[x, y + 1] − g[x, y − 1]
note
the gradient is an operator ∇ : Rn → R → Rn → Rn
note
the gradient is an operator ∇ : Rn → R → Rn → Rn that is, it is a function that operates on functions
note
the gradient is an operator ∇ : Rn → R → Rn → Rn that is, it is a function that operates on functions that is, it takes a function and produces another function
note
the gradient is an operator ∇ : Rn → R → Rn → Rn that is, it is a function that operates on functions that is, it takes a function and produces another function contrast this with functions which take a (tuple of) number(s) and produce another (tuple of) number(s)
note
∇g[x, y] is a discrete, vector valued function, i.e. a discrete vector field
note
∇g[x, y] is a discrete, vector valued function, i.e. a discrete vector field for visualization, we usually consider only its magnitude q
q
2
∇g[x, y] = ∇g[x, y], ∇g[x, y] = ∇g[x, y]
note
∇g[x, y] is a discrete, vector valued function, i.e. a discrete vector field for visualization, we usually consider only its magnitude q
q
2
∇g[x, y] = ∇g[x, y], ∇g[x, y] = ∇g[x, y]
and values exceeding 255 are typically rounded off to 255
example
g[x, y]
dx g[x, y]
dy g[x, y]
∇g[x, y]
example
450 400 350 300 250 200 150 100 50
0
100
200
300
400
500
∇g[x, y] shown as a vector field
600
700
question what is this good for ?
question what is this good for ? answer it detects edges!
question what is this good for ? answer it detects edges! question what is this good for ?
question what is this good for ? answer it detects edges! question what is this good for ? answer we will study numerous applications of this later for now, we consider an artistic image processing example
emboss effect idea transform an input image such that it looks like a copper engraving or an etching
emboss effect idea transform an input image such that it looks like a copper engraving or an etching
approach similar to the computation of derivatives transform given image g into new image ˜ g using
˜g[x, y] = 128 + g[x + 1, y + 1] − g[x − 1, y − 1]
examples
summary
we now know about
mathematical representations of digital images continuous derivatives and gradients discrete derivatives and gradients
exercises
the divergence of a continuous multivariate function f (x1 , x2 , . . . , xn ) is defined as ∆ f = ∇2 f = ∇ · ∇f =
n X ∂2 f i=1
∂xi2
where ∆ is called the Laplace operator write down / devise an expression for the divergence of a discrete 2D function g[x, y]
exercises
assume a discrete 1D function g[x]; what does the following operator do ? AM g[x] =
1 g[x − 1] + g[x] + g[x + 1] 3
what about this one ? WM g[x] =
1 g[x − 1] + 2g[x] + g[x + 1] 4