Markov models are used in pattern recognition to provide an answer to this question, we recall a didactic example from our lectures on game AI that is, we show ...
Pattern Recognition Prof. Christian Bauckhage
outline additional material for lecture 10
inference with Markov chains
summary
inference with Markov chains
in lecture 10, we discussed maximum likelihood estimation of the (transition) parameters Pij of a Markov chain over a finite set of states a student then asked the very valid question where or how Markov models are used in pattern recognition to provide an answer to this question, we recall a didactic example from our lectures on game AI that is, we show how Markov models allow us to classify sequences of symbols
game AI example
consider a simple game map consisting of 3 rooms assume the rooms form the state space S = A, B, C for discrete Markov chains
A
B treasure
C
game AI example (cont.)
also assume you observe a game agent (player or NPC) moving from room to room in particular, you observe the following sequence of moves O = AABBBBCBBA the obvious question is: what is the agent doing? what kind of behavior would correspond to these movements?
A
B treasure
C
game AI example (cont.)
experience tells us that –on this part of the map– opponent players typically show two behaviors: “patrolling the area” or “guarding the treasure in room B” for both behaviors, we happen to have a Markov model, namely 0.3
0.3
0.8
0.25
0.2
A
0.1
B
0.5
0.5
A 0.25
0.5
0.2
0.2
C
B
0.5
0.1
0.25
0.5
C
0.3
0.25
λ1 ⇔ patrolling
λ2 ⇔ guarding
game AI example (cont.)
under these conditions, the question of what the agent is doing can be answered by evaluation the two probabilities p O λ1 and p O λ2
game AI example (cont.)
for λ1 we have p AABBBBCBBA = p X1 = A · p X2 = A | X1 = A · p X3 = B | X2 = A · · · p X10 = A | X9 = B = 1 · 0.3 · 0.5 · 0.3 · 0.3 · 0.3 · 0.5 · 0.2 · 0.3 · 0.2 ≈ 0.000025
game AI example (cont.)
for λ1 we have p AABBBBCBBA = p X1 = A · p X2 = A | X1 = A · p X3 = B | X2 = A · · · p X10 = A | X9 = B = 1 · 0.3 · 0.5 · 0.3 · 0.3 · 0.3 · 0.5 · 0.2 · 0.3 · 0.2 ≈ 0.000025 for λ2 we have p AABBBBCBBA = 1 · 0.25 · 0.5 · 0.8 · 0.8 · 0.8 · 0.1 · 0.5 · 0.8 · 0.1 ≈ 0.00026
game AI example (cont.)
for λ1 we have p AABBBBCBBA = p X1 = A · p X2 = A | X1 = A · p X3 = B | X2 = A · · · p X10 = A | X9 = B = 1 · 0.3 · 0.5 · 0.3 · 0.3 · 0.3 · 0.5 · 0.2 · 0.3 · 0.2 ≈ 0.000025 for λ2 we have p AABBBBCBBA = 1 · 0.25 · 0.5 · 0.8 · 0.8 · 0.8 · 0.1 · 0.5 · 0.8 · 0.1 ≈ 0.00026 ⇒ according to the two models we have available, it seems as if the opponent is guarding the treasure in room B
note
above we tacitly assumed p X1 = A = 1 as we do not have any information as to what is the most likely symbol for a sequence to begin with, we should acknowledge our ignorance and assume p X1 = A = p X1 = B = p X1 = C of course we could argue that p X1 = A = 1/3 because there are three rooms but whether or not the first factor in the two products on the 1 previous slide not matter; in either case we is /3 or 1 does find p O λ1 < p O λ2
note
if the given sequence of observations is really long, it is numerically much safer to evaluate log p O λi instead of p O λi
note
we can also use Markov models to anticipate the future for instance, now that we “know” that the opponent is guarding the treasure, we might ask: if the opponent is currently in room B, how likely will he stay there for the next τ time steps? to answer this question, we compute p Xt = B, Xt+1 = B, . . . , Xt+τ−1 = B, Xt+τ 6= B = p Xt+1 = B | Xt = B · · · p Xt+τ 6= B | Xt+τ−1 = B = pτ−1 BB · (1 − pBB )
we now know about
how to use (a set of) Markov models to classify sequences of symbols or histories of (discrete) observations
