Pattern Recognition

Pattern Recognition Prof. Christian Bauckhage

outline lecture 02

food for thought

food for thought

what is pattern recognition ?


regression

classification

clustering

IQ tests are all about pattern recognition

example

continue the series 1, 4, 9, 16, 25, 36, ?

example


answer 49 because xn = n2 ,

n ∈ N+

example


example


answer 13 because xn = xn−1 + xn−2

example


example


answer 17 because xn = n-th prime

example

which one does not belong into the set 255, 63, 127, 31, 17, 7

example

which one does not belong into the set 255, 63, 127, 31, 17, 7

answer 17 because 17 6∈ x x = 2n − 1, n ∈ N

example

which one(s) do not belong into the set aa, abba, abbba, aba, abca, abbbbba, abbbb

example

which one(s) do not belong into the set aa, abba, abbba, aba, abca, abbbbba, abbbb

answer abca, abbbb because abca, abbbb 6∈ L(G) where G = (N, T, P, S) and N = S, B T = a, b P = S → aBa, B → ∅, B → bB

example

which one does not belong into the set Stockhausen, Bach, Grieg, Beethoven, Brahms, Wagner

example


answer Stockhausen, because he was not a classical composer

example


answer Stockhausen, because he was not a classical composer Bach, because he was born prior to 1700

example


answer Stockhausen, because he was not a classical composer Bach, because he was born prior to 1700 Grieg, because he was Norwegian

example


answer Stockhausen, because he was not a classical composer Bach, because he was born prior to 1700 Grieg, because he was Norwegian hmmm . . . let’s try this one again!

example

which one does not belong into the set

Stockhausen, DE , Bach, DE , Grieg, NO , Beethoven, DE , Brahms, DE , Wagner, DE

example

which one does not belong into the set

Stockhausen, DE , Bach, DE , Grieg, NO , Beethoven, DE , Brahms, DE , Wagner, DE

answer Grieg, because he was Norwegian

example

complete the relation tiny mouse ↔ large ?

A: grasshopper B: elephant C: courage D: virus

example

complete the relation tiny mouse ↔ large ?

A: grasshopper B: elephant C: courage D: virus

answer B: elephant because . . .

example

cluster these objects into 3 groups of 3

take home messages

the brain is a pattern recognition apparatus

take home messages

the brain is a pattern recognition apparatus pattern ⇔ structure ⇔ analogy ⇔ similarity

take home messages

the brain is a pattern recognition apparatus pattern ⇔ structure ⇔ analogy ⇔ similarity what is a pattern? what is a structure? how to draw analogies? how to measure similarities?

take home messages


pattern recognition requires background knowledge or prior information

take home messages


pattern recognition requires background knowledge or prior information how to represent prior information?

take home messages



automatic pattern recognition ⇔ applied math

take home messages



automatic pattern recognition ⇔ applied math formal languages, geometry, linear algebra, statistics, probability theory, optimization, . . .

wonders of cognition

example

what is this?

example

what is this?

example

read these words aloud (as fast as you can!)

red, green, yellow, blue

example

read these words aloud (as fast as you can!)

red, green, yellow, blue

now, read these words

yellow, blue, green, red

example

read this sentence aloud

The quick brown fox jumps over the lazy dog.

example

read this sentence aloud

The quick brown fox jumps over the lazy dog.

now, read this sentence

The qicuk borwn fox jupms ovre the lzay dog.

take home messages

the brain is the best pattern recognition apparatus out there

take home messages


it even “sees” patterns where there are none, because it incorporates vast amounts of implicit prior knowledge it weighs macroscopic against microscopic patterns it filters unimportant information, i.e. pays attention

take home messages



sometimes, these features can be turned into bugs cognitive dissonance, optical illusions, magic, . . .

take home messages



sometimes, these features can be turned into bugs cognitive dissonance, optical illusions, magic, . . .

the role of complexity


0.014 0.012 0.010 0.008 0.006 0.004 0.002 0.000

0

50

100

150

H = 5.06

200

250


0.6 0.014

0.5

0.012 0.010

0.4

0.008

0.3

0.006

0.2

0.004

0.1

0.002 0.000

0

50

100

150

H = 5.06

200

250

0.0

0

50

100

150

H = 1.04

200

250

let’s play a game . . .


predict what is under the red cover


predict what is under the red cover

round 1

round 1

round 2

round 2

round 3

round 3

incongruity theory of humor

the brain constantly looks out for patterns to learn about the environment if an expectation (an assumed pattern) changes, we are surprised and delighted and laugh, because we learned something


the brain constantly looks out for patterns to learn about the environment if an expectation (an assumed pattern) changes, we are surprised and delighted and laugh, because we learned something ⇒ things are funny because there is an incongruity between what we expected and what happened


the brain constantly looks out for patterns to learn about the environment if an expectation (an assumed pattern) changes, we are surprised and delighted and laugh, because we learned something ⇒ things are funny because there is an incongruity between what we expected and what happened ⇒ comedy is about learning

round 4

round 4

take home messages

pattern recognition ⇔ complexity reduction ⇔ compression

take home messages

pattern recognition ⇔ complexity reduction ⇔ compression how to measure complexity ? entropy fractal dimension description length degree of hierarchy .. .

so, what is pattern recognition really?

a quest for the minimum entropy, whereby our system of categories used in the representation of the data is to be adaptively adjusted and the entropy is suitably defined Watanabe

research and development concerning mathematical and technical aspects of perception Niemann

research and development concerning mathematical and technical aspects of perception Niemann the science that concerns the description or classification of measurements Schalkoff

research and development concerning mathematical and technical aspects of perception Niemann the science that concerns the description or classification of measurements Schalkoff the assignment of physical objects or events to one of several pre-defined categories Duda and Hart

given some examples of complex signals and the correct decision for them, make decisions automatically for a stream of future examples Ripley

given some examples of complex signals and the correct decision for them, make decisions automatically for a stream of future examples Ripley . . . is concerned with the automatic discovery of regularities in data through the use of computer algorithms and with the use of these regularities to take actions such as classifying the data into different categories Bishop

a problem of estimating density functions in a high dimensional space and dividing the space into regions of categories or classes Fukunaga


the problem of giving names Ω to observations x Schurmann ¨


the problem of giving names Ω to observations x Schurmann ¨

. . . concerned with answering the questions: what is this? Morse

tentative word count

category, decision, class, name

|6 ||| |

measurement, signal, object, observation, data

|6 ||| |

classification, assignment

|||

mathematics, algorithm, function

|||

automatic

||

representation

|