Probabilistic Acceptors for Languages over Infinite Words Christel ...

Probabilistic Acceptors for Languages over Infinite Words Christel Baier Technical University Dresden joint work with Nathalie Bertrand and Marcus Größer

1 / 148

ω –regular languages

pba-1

• languages over infinite words L ⊆ Σω • arise from ω -regular expressions α1β ω1 + ... + αn β ωn where αi , β i are regular expressions with ε ∈ β i

2 / 148


pba-1

• languages over infinite words L ⊆ Σω • arise from ω -regular expressions α1β ω1 + ... + αn β ωn where αi , β i are regular expressions with ε ∈ β i • recognized by many types of ω -automata, e.g., NBA (nondeterministic B¨ uchi automata) syntax as NFA, but with acceptance criterion “visit infinitely often a final state”

3 / 148


pba-1

• languages over infinite words L ⊆ Σω • arise from ω -regular expressions α1β ω1 + ... + αn β ωn • recognized by many types of ω -automata, e.g., NBA (nondeterministic B¨ uchi automata) • can be used for verifying linear time properties M) ∩ L (A A) = ∅ with graph “check whether L (M algorithm in the product M × A ” where M is the system model, A is an NBA for the bad behaviors 4 / 148

ω -automata

pba-2

NBA

5 / 148

ω -automata

pba-2

NBA

DBA

6 / 148

ω -automata

pba-2

NBA

(a + b)∗ aω DBA

NBA b

1 a

a

2 a

7 / 148

ω -automata

pba-2

♦ F ♦ Hi → ♦ Ki ) NSA/DSA (Streett automata) ( i ♦ ¬Ki ∧ ♦ Hi ) NRA/DRA (Rabin automata) (♦ i .. . (a + b)∗ aω NBA

DBA

NBA b

1 a

a

2 a

8 / 148

ω -automata

pba-2

♦ F ♦ Hi → ♦ Ki ) NSA/DSA (Streett automata) ( i ♦ ¬Ki ∧ ♦ Hi ) NRA/DRA (Rabin automata) (♦ i .. . (a + b)∗ aω NBA

DBA exp

NBA DSA/DRA/... [Safra ’88]

NBA b

1 a

a

2 a

9 / 148

ω -automata

pba-2

♦ F ♦ Hi → ♦ Ki ) NSA/DSA (Streett automata) ( i ♦ ¬Ki ∧ ♦ Hi ) NRA/DRA (Rabin automata) (♦ i .. . (a + b)∗ aω probabilistic B¨ uchi automata ? DBA NBA

exp

NBA DSA/DRA/... [Safra ’88]

NBA b

1 a

a

2 a

10 / 148

Overview pba-3

• • • •

definition of probabilistic Büchi automata (PBA) expressiveness of PBA efficiency of PBA other acceptance conditions (Streett/Rabin, alternative semantics) • composition operators on PBA • decision problems for PBA

11 / 148

Probabilistic B¨ uchi Automaton (PBA)

pba-4

PBA: syntax as for NBA, but all choices are resolved probabilistically

a,

1 3

PBA probabilistic choice

q a,

2 3

a

q a

NBA nondeterministic choice

12 / 148


pba-4

P = (Q, δ , µ , F) over alphabet Σ : • Q finite state space • transition probability function δ : Q × Σ × Q → [0, 1]

13 / 148


pba-4

P = (Q, δ , µ , F) over alphabet Σ : • Q finite state space • transition probability function δ : Q × Σ × Q → [0, 1] s.t. for all q ∈ Q, a ∈ Σ : δ (q, a, p) ∈ {0, 1} ∈Q p∈

14 / 148


pba-4


• initial distribution µ : Q → [0, 1]

15 / 148


pba-4


• initial distribution µ : Q → [0, 1] • set of final states F ⊆ Q

16 / 148

Semantics of PBA

pba-5

P = (Q, δ , µ , F) • Q finite state space • transition probability function δ : Q × Σ × Q → [0, 1] • initial distribution µ • set of final states F ⊆ Q

17 / 148

Semantics of PBA

pba-5

P = (Q, δ , µ , F) • Q finite state space • transition probability function δ : Q × Σ × Q → [0, 1] • initial distribution µ • set of final states F ⊆ Q For x ∈ Σω : Pr Pr(x) = probability for the accepting runs for x

18 / 148

Semantics of PBA

pba-5

P = (Q, δ , µ , F) • Q finite state space • transition probability function δ : Q × Σ × Q → [0, 1] • initial distribution µ • set of final states F ⊆ Q For x ∈ Σω : Pr Pr(x) = probability for the accepting runs for x accepting run: visits F infinitely often

19 / 148

Semantics of PBA P = (Q, δ , µ , F)

pba-5

←

can be viewed as a Markov decision process

• Q finite state space • transition probability function δ : Q × Σ × Q → [0, 1] • initial distribution µ • set of final states F ⊆ Q For x ∈ Σω : Pr Pr(x) = probability for the accepting runs for x probability measure in the infinite Markov chain induced by x viewed as a scheduler 20 / 148

Semantics of PBA P = (Q, δ , µ , F)

pba-5

←

can be viewed as a Markov decision process

• Q finite state space • transition probability function δ : Q × Σ × Q → [0, 1] • initial distribution µ • set of final states F ⊆ Q For x ∈ Σω : Pr Pr(x) = probability for the accepting runs for x P ) = x ∈ Σω : Pr accepted language: L(P Pr(x) > 0 21 / 148

Example for a PBA a, 12

1

pba-6

2

b,1 a,

1 2

a,1

final state

non-final state 22 / 148

Example for a PBA a, 12

1

pba-6

2

b,1 a,

1 2

a,1

input word x = aabaωω final state

non-final state 23 / 148

Example for a PBA a , 12

1

pba-6

1 1

2

b,1 a,

1 2

1 2 1 2

1 2

1

a,1

1

input word aabaω

x = aaba

1 1 .. .

1 2

1 2 1 2

.. .

1

a

2 b

1 2

1 2

a 2

2

1

1 2

a 2

1 2

2 .. .

1

a

1

a .. .

2 1

.. .

24 / 148


1

pba-6

1 1

2

b,1 a,

1 2

1

x = aaba

1 4

1 1 .. .

1 2

1 2 1 2

.. .

a 1

a

2 b

1 2

1 2

1 2

2 2

1

aabaω

1 1 Pr(x) = 18 + 16 + 32 + ...

1 2

1

a,1

input word

=

1 2 1 2

a 2

1 2

2 .. .

1

a

1

a .. .

2 1

.. .

25 / 148


1

pba-6

1 1

2

b,1 a,

1 2

1

x = aaba

1 1 .. .

1 2

1 2 1 2

.. .

a 1

a

2 reject

1 2

1 2

1 2

2 2

1

aabaω

1 1 Pr(x) = 18 + 16 + 32 + ...

1 2

1

a,1

input word

= 1-Pr(xx is rejected)

1 2 1 2

b a

2

1 2

2 .. .

1

a

1

a .. .

2 1

.. .

26 / 148

Example for PBA a,

1

1 2

pba-7

accepted language: 2

(a + b)∗aω

b,1 a,

1 2

a,1

27 / 148

Example for PBA a,

1

1 2

pba-7


(a + b)∗aω

b,1 a,

1 2

a,1

Thus: PBA are strictly more expressive than DBA

28 / 148

Example for PBA a,

1

pba-7

1 2


(a + b)∗aω

b,1 a,

1 2

a,1


b 2

c

1 a,

1 2

b a,

1 2

3

29 / 148

Example for PBA a,

1

pba-7

1 2


(a + b)∗aω

b,1 a,

1 2

a,1


b 2

c

1 a,

1 2

b a,

1 2

3

NBA accepts ((ac)∗ ab)ω 30 / 148

Example for PBA a,

1

pba-7

1 2


(a + b)∗aω

b,1 a,

1 2

a,1


b 2

c

1 a,

1 2

b a,

1 2

accepted language: (ab + ac)∗(ab)ω

3

but NBA accepts ((ac)∗ab)ω 31 / 148

Expressiveness of PBA

pba-8a

32 / 148

Expressiveness of PBA

pba-8

Theorem: PBA are strictly more expressive than NBA

33 / 148

PBA are strictly more expressive than NBA

pba-8a

from NBA to PBA: NBA

NBA deterministic in = PBA limit Courcoubetis/ Yannakakis ’95

34 / 148


pba-8a


NBA deterministic in = PBA limit Courcoubetis/ Yannakakis ’95 d

d

a a b

c b 35 / 148


pba-8a


NBA deterministic in = PBA limit Courcoubetis/ Yannakakis ’95 d

d

a a b

c b

deterministic 36 / 148


pba-8a


NBA deterministic in = PBA limit Courcoubetis/ Yannakakis ’95 d, 12

d d

a a b

= c b

d, 12

a, 12 a, 12

c b b

deterministic 37 / 148


pba-8

• from NBA to PBA: via NBA that are deterministic in limit ω -regular languages • PBA can accept non-ω

38 / 148


pba-8

• from NBA to PBA: via NBA that are deterministic in limit ω -regular languages • PBA can accept non-ω b 1 a , 12

2 a,

1 2

a

39 / 148


pba-8


2 a,

1 2

a

accepted language: ak11bak2 bak3 b . . . :

40 / 148


pba-8


2 a,

1 2

a

accepted language: ∞

1 k i k11 k2 k3 a ba ba b . . . : 1− >0 2 i=1 i=1

41 / 148


pba-uniform


2 a,

1 2

a

• uniform PBA cover exactly the class of ω -regular languages

42 / 148


pba-uniform


2 a,

1 2

a

• uniform PBA cover exactly the class of ω -regular languages where “uniformity” is a probabilistic condition on end components 43 / 148

Efficiency of PBA

pba-12

44 / 148

Efficiency of PBA

pba-12

Theorem: There exist uniform PBA Pn with |Pn | = O(n) s.t. each equivalent NSA An has Ω(2n /n) states. NSA: nondeterministic Streett automaton

45 / 148

Efficiency of PBA

pba-12

Theorem: There exist uniform PBA Pn with |Pn | = O(n) s.t. each equivalent NSA An has Ω(2n /n) states. Proof: consider the ω -regular languages Ln = xyω : x, y ∈ {a, b}∗ , |y| = n

46 / 148

Efficiency of PBA

pba-12

Theorem: There exist uniform PBA Pn with |Pn | = O(n) s.t. each equivalent NSA An has Ω(2n /n) states. Proof: consider the ω -regular languages Ln = xyω : x, y ∈ {a, b}∗, |y| = n show that 1. Each any NSA for Ln has ≥ 2n /n states. 2. There is a uniform PBA with O(n) states. 47 / 148

Efficiency of PBA

pba-12

Theorem: There exist uniform PBA Pn with |Pn | = O(n) s.t. each equivalent NSA An has Ω(2n /n) states. Proof: consider the ω -regular languages Ln = xyω : x, y ∈ {a, b}∗ , |y| = n Each any NSA for Ln has ≥ 2n /n states

48 / 148

Efficiency of PBA

pba-12

Theorem: There exist uniform PBA Pn with |Pn | = O(n) s.t. each equivalent NSA An has Ω(2n /n) states. Proof: consider the ω -regular languages Ln = xyω : x, y ∈ {a, b}∗ , |y| = n Each any NSA for Ln has ≥ 2n /n states as the “accepting loops” for y1 ≡ y2 , |y1| = |y2| = n are disjoint where for y1 = c1 c2 . . . cn and y2 = d1 d2 . . . dn : y1 ≡ y2 iff ∃i s.t. c1 c2 . . . cn = di . . . dn d1 . . . di−1 49 / 148

Efficiency of PBA

pba-13

Thm: There exist uniform PBA Pn with |Pn | = O(n) s.t. each equivalent NSA An has Ω(2n /n) states. Proof: consider the ω -regular languages Ln = xyωω : x, y ∈ {a, b}∗ , |y| = n show that 1. each NSA for Ln has ≥ 2n /n states

√

2. there exists a uniform PBA with 2n states

50 / 148

Efficiency of PBA

pba-13

Thm: There exist uniform PBA Pn with |Pn | = O(n) s.t. each equivalent NSA An has Ω(2n /n) states. Proof: consider the ω -regular languages Ln = xyωω : x, y ∈ {a, b}∗ , |y| = n PBA for Ln mit 2n states: 1a

2a

3a

...

na

1b

2b

3b

...

nb 51 / 148

Efficiency of PBA

pba-13

Thm: There exist uniform PBA Pn with |Pn | = O(n) s.t. each equivalent NSA An has Ω(2n /n) states. Proof: consider the ω -regular languages Ln = xyωω : x, y ∈ {a, b}∗ , |y| = n PBA for Ln mit 2n states: a a a

1a

2a a

1b

a

a

3a

...

na

3b

...

nb

a 2b

a

X 52 / 148

Efficiency of PBA

pba-13

Thm: There exist uniform PBA Pn with |Pn | = O(n) s.t. each equivalent NSA An has Ω(2n /n) states. Proof: consider the ω -regular languages Ln = xyωω : x, y ∈ {a, b}∗ , |y| = n PBA for Ln mit 2n states: a a b b a 1a 2 3a a a a a a a 1b

2b

3b

a b ... a ...

b a

na

nb

a

X 53 / 148

Efficiency of PBA

pba-13

Thm: There exist uniform PBA Pn with |Pn | = O(n) s.t. each equivalent NSA An has Ω(2n /n) states. Proof: consider the ω -regular languages Ln = xyωω : x, y ∈ {a, b}∗ , |y| = n PBA for Ln mit 2n states: a a b b a 1a 2 3a a a a a a a 1b

2b

3b

a b ... a ...

b a

na b

X

a

X

nb

54 / 148

Efficiency of PBA

pba-13

Thm: There exist uniform PBA Pn with |Pn | = O(n) s.t. each equivalent NSA An has Ω(2n /n) states. Proof: consider the ω -regular languages Ln = xyωω : x, y ∈ {a, b}∗ , |y| = n PBA for Ln mit 2n states: a a b b a 1a 2 3a a a ba b aa a b b 3b b 1b a 2b a b

b

a b ... a

b a

na b

X

b ... a

b a

nb

a

X

b

55 / 148

uPBA for L3 = {xyω : x, y ∈ {a, b}∗ , |y| = 3 } a a b

b 1b

a

a a b

1a

a b b

pba-14

2a b a 2b

a b

3a

a b

3b

uniform distributions

b

56 / 148


b 1b

a

a a b

1a

a b b

pba-14

2a b a 2b

a b

3a

a b

3b


b

1. Let z = c1c2 . . . ∈ {a, b}ω s.t. ∞

∃ i with ci = a and ci+3 = b

57 / 148


b 1b

a

a a b

1a

a b b

pba-14

2a b a 2b

a b

3a

a b

3b


b

1. Let z = c1c2 . . . ∈ {a, b}ω s.t. ∞

∃ i with ci = a and ci+3 = b c11...cii−1 −1a

ci +1

ci +3

b

→ with prob. 1: ∃i s.t. −−−−−→ 1a −−→ 2a −−→= 3a − 58 / 148


b 1b

a

a a b

1a

a b b

pba-14

2a b a 2b

a b

3a

a b

3b


b

1. Let z = c1c2 . . . ∈ {a, b}ω s.t. ∞

∃ i with ci = a and ci+3 = b c11...cii−1 −1a

ci +1

ci +3

b

→ with prob. 1: ∃i s.t. −−−−−→ 1a −−→ 2a −−→= 3a − i.e. z will be rejected almost surely

59 / 148

uPBA for L3 = {xyω : x, y ∈ {a, b}∗ , |y| = 3 }

a a b

b 1b

a

a a b

1a

a b b

pba-15

2a

a b

3a

2b

a b

3b

b a


b

2. Let z = xyω ∈ L3 where x = c1 . . . ci .

60 / 148


a a b

b 1b

a

a a b

1a

a b b

pba-15

2a

a b

3a

2b

a b

3b

b a


b

2. Let z = xyω ∈ L3 where x = c1 . . . ci . All runs for z that start with the prefix c22 ci c1 1a − → 1c11 − → ... − → 1ci are infinite 61 / 148


a a b

b 1b

a

a a b

1a

a b b

pba-15

2a

a b

3a

2b

a b

3b

b a


b

2. Let z = xyω ∈ L3 where x = c1 . . . ci . All runs for z that start with the prefix c22 ci c1 1a − → 1c11 − → ... − → 1ci are infinite (and accepting). 62 / 148


a a b

b 1b

a

a a b

1a

a b b

pba-15

2a

a b

3a

2b

a b

3b

b a


b

2. Let z = xyω ∈ L3 where x = c1 . . . ci . All runs for z that start with the prefix c22 ci c1 1a − → 1c11 − → ... − → 1ci are infinite (and accepting). 1 i Hence: Pr(z) ≥ ( 2 ) > 0 63 / 148

Efficiency of PBA

pba-16

Thm: There exist uniform PBA with of size O(n) s.t. each equivalent NSA has Ω(2n /n) states Thm: There exist NBA A n of size O(n) s.t. each equivalent uniform PBA has Ω(2n ) states

64 / 148

Efficiency of PBA

pba-16

Thm: There exist NBA A n of size O(n) s.t. each equivalent uniform PBA has Ω(2n ) states Proof: consider the language ((a + b)∗a(a + b)n−1c)ω

65 / 148

Efficiency of PBA

pba-16

Thm: There exist NBA A n of size O(n) s.t. each equivalent uniform PBA has Ω(2n ) states Proof: consider the language ((a + b)∗a(a + b)n−1c)ω NBA: 0 a, b

a

1

a, b

2

a, b . . .

a, b

n−1 c

a, b

n

66 / 148

Efficiency of PBA

pba-16

Thm: There exist NBA A n of size O(n) s.t. each equivalent uniform PBA has Ω(2n ) states Proof: consider the language ((a + b)∗a(a + b)n−1c)ω NBA: 0 a, b

a

1

a, b

2

a, b . . .

a, b

n−1 c

a, b

n

PBA: has to “remember” all a ’s of the last n symbols

67 / 148

Probabilistic ω -automata

pba-17

accepted language: P ) = x ∈ Σ ω : PrP ( accepting runs for x )} > 0 L (P acceptance conditions: • Büchi: ♦ F ♦Hi ∧ ♦¬Ki ) • Rabin: ( 1≤i≤k ♦ Hi → ♦ Ki ) ( • Streett: 1≤i≤k

68 / 148


pba-17


poly

PRA −−→ PBA as for nondeterministic automata exp PSA −→ PBA preserving uniformity 69 / 148


pba-17


poly

PRA −−→ PBA as for nondeterministic automata exp PSA −→ PBA preserving uniformity poly

PSA −−→ PBA possibly non-uniform 70 / 148

Poly transformation from PSA to PBA PSA P with acc. condition

pba-18

♦ Hi → ♦ Ki ) (

1≤i≤m

PBA consists of several copies of P

71 / 148


pba-18

♦ Hi → ♦ Ki ) (

1≤i≤m

PBA: P init P accept

72 / 148


pba-18

♦ Hi → ♦ Ki ) (

1≤i≤m

PBA: P init P accept Ki Pi

Hi

73 / 148


pba-18

♦ Hi → ♦ Ki ) (

1≤i≤m

PBA: P init P accept Ki Pi

Hi Hj

74 / 148


pba-18

♦ Hi → ♦ Ki ) (

1≤i≤m

PBA: P init P accept Ki Pi P i,j i = j

Hi Hj

Ki Hj 75 / 148


pba-18

♦ Hi → ♦ Ki ) (

1≤i≤m

PBA: P init P accept Ki Pi P i,j i = j

Hi Hj

Pj

P j,k j = k

Ki Hj

76 / 148


pba-18

♦ Hi → ♦ Ki ) (

1≤i≤m

PBA: P init

Ki Pi P i,j i = j

Hi Hj

Hj

P accept Kj Hi

P j,k j = k

Ki Hj

Hi

Pj

Kj 77 / 148


pba-18

♦ Hi → ♦ Ki ) (

1≤i≤m

PBA: P init

Ki Pi P i,j i = j

Hi Hj

Hj

P accept

size: O(m2|P|) Kj Hi

P j,k j = k

Ki Hj

Hi

Pj

Kj 78 / 148

Alternative semantics for PBA

pba-19

standard semantics: P ) = x ∈ Σω : PrP (xx) > 0 L (P

79 / 148

Alternative semantics for PBA

pba-19

standard semantics: P ) = x ∈ Σω : PrP (xx) > 0 L (P alternative semantics: • threshold semantics • almost-sure semantics

80 / 148

Threshold semantics for PBA

pba-19

for PBA P and λ ∈]0, 1[ P ) = x ∈Σ Σωω : PrP (xx) > λ L>λ (P

81 / 148


pba-19

for PBA P and λ ∈]0, 1[ P ) = x ∈Σ Σωω : PrP (xx) > λ L>λ (P Results: P ) = L (P P) • ∀ PBA P ∀λ ∈]0, 1[ ∃ PBA P s.t. L>λ (P

82 / 148


pba-19

for PBA P and λ ∈]0, 1[ L>λ (P P ) = x ∈Σ Σωω : PrP (xx) > λ Results: P ) = L (P P) • ∀ PBA P ∀λ ∈]0, 1[ ∃ PBA P s.t. L>λ (P

P

ε, λ ε, 1 − λ

accept

a∈Σ

P

P is uniform if P is uniform 83 / 148


pba-19

for PBA P and λ ∈]0, 1[ P ) = x ∈Σ Σωω : PrP (xx) > λ L>λ (P Results: P ) = L (P P) • ∀ PBA P ∀λ ∈]0, 1[ ∃ PBA P s.t. L>λ (P P ) cannot be • ∃ PBA P ∃λ ∈]0, 1[ s.t. L>λ (P recognized by a standard PBA

84 / 148


pba-19

for PBA P and λ ∈]0, 1[ P ) = x ∈Σ Σωω : PrP (xx) > λ L>λ (P Results: P ) = L (P P) • ∀ PBA P ∀λ ∈]0, 1[ ∃ PBA P s.t. L>λ (P P ) cannot be • ∃ PBA P ∃λ ∈]0, 1[ s.t. L>λ (P recognized by a standard PBA P ) is not • ∃ uniform PBA P ∃λ ∈]0, 1[ s.t L>λ (P ω -regular

85 / 148

Almost-sure semantics of PBA P ) = x ∈ Σω : Pr(x) = 1 L=1 (P

pba-20a

86 / 148


pba-20a

Theorem: For each PBA P there exists a PBA P P ) = L (P P ). such that L=1 (P

87 / 148


pba-20a

Theorem: For each PBA P there exists a PBA P P ) = L (P P ). such that L=1 (P Proof sketch. PBA P for input word x • simulates P with input x • guesses at random a word position i • checks whether ¬F holds with positive probability from position i • if so, P rejects; otherwise P accepts. 88 / 148


pba-20a

Theorem: For each PBA P there exists a PBA P P ) = L (P P ). such that L=1 (P P ) is Theorem: There exists a PBA P such that L (P not recognizable by a PBA with the almost-sure semantics.

89 / 148


pba-20a

Theorem: For each PBA P there exists a PBA P P ) = L (P P ). such that L=1 (P P ) is Theorem: There exists a PBA P such that L (P not recognizable by a PBA with the almost-sure semantics. example: PBA for the ω-regular language (a + b)∗aω

90 / 148

PBA with standard and alternative semantics

pba-20

PBA

DBA

91 / 148


pba-20

PBA ω -regular languages uniform PBA DBA

92 / 148


pba-20

PBA with thresholds PBA ω -regular languages uniform PBA DBA

93 / 148


pba-20

PBA with thresholds PBA ω -regular languages uniform PBA DBA

PBA almost sure semantics

94 / 148


pba-20

PBA with thresholds (a+b)∗aω



95 / 148


pba-20



{ak11bak2 b. . . :

∞


(1 − ( 12 )kii ) > 0}

i=1

96 / 148


pba-20


PBA ω -regular languages uniform PBA PBA almost sure semantics

DBA

{ak11bak2 b. . . :

∞

(1 − ( 12 )kii ) > 0}

i=1

{ak11bak22b. . . :

∞

(1 − ( 12 )ki ) = 0}

97 / 148


pba-20


{ak11bak2 b. . . :

PBA, PSA, PRA ω -regular languages uniform PBA PBA almost sure semantics DBA ∞

(1 − ( 12 )kii ) > 0}

i=1

{ak11bak22b. . . :

∞

(1 − ( 12 )ki ) = 0}

98 / 148


pba-20


{ak11bak2 b. . . :

PBA, PSA, PRA and 0/1-PRA ω -regular languages uniform PBA PBA almost sure semantics DBA ∞

(1 − ( 12 )kii ) > 0}

i=1

{ak11bak22b. . . :

∞

(1 − ( 12 )ki ) = 0}

99 / 148

0/1-PRA

pba-20b

0/1-PRA: probabilistic Rabin automaton P s.t. ∀x ∈ Σω : PrP (x) ∈ {0, 1}

100 / 148

0/1-PRA

pba-20b

0/1-PRA: probabilistic Rabin automaton P s.t. ∀x ∈ Σω : PrP (x) ∈ {0, 1} Theorem: For each (standard) PBA P there exists a P ) = L (P PR ). 0/1-PRA PR such that L (P

101 / 148

0/1-PRA

pba-20b

0/1-PRA: probabilistic Rabin automaton P s.t. ∀x ∈ Σω : PrP (x) ∈ {0, 1} Theorem: For each (standard) PBA P there exists a P ) = L (P PR ). 0/1-PRA PR such that L (P Corollary: The almost-sure semantics for PRA is as powerful as the standard semantics. The same holds for PSA, but not for PBA.

102 / 148

0/1-PRA

pba-20b

0/1-PRA: probabilistic Rabin automaton P s.t. ∀x ∈ Σω : PrP (x) ∈ {0, 1} Theorem: For each (standard) PBA P there exists a P ) = L (P PR ). 0/1-PRA PR such that L (P idea: 0/1-PRA PR • generates up to n = |Q| sample runs of P (as representatives for all runs in P ) • and checks whether at least one of them is accepting

103 / 148

From PBA P to 0/1-PRA P R pairwise distinct states in P

pba-20b

powerset construction

state in P R : q1, q2 , . . . , qi , R

104 / 148

From PBA P to 0/1-PRA P R pairwise distinct states in P state in P R : q1, q2 , . . . , qi , R a a ... a q1 , q2, . . . , qi ,

δ

pba-20b


a δ (R, a)

= transition function in P 105 / 148

From PBA P to 0/1-PRA P R pairwise distinct states in P state in P R : q1, q2 , . . . , qi , R a a ... a

pba-20b


a

. . . qk , δ (R, a) q1 , q2, . . . , qi , qi+1

where {qi+1 , . . . , qk } = F ∩ δ (R, a)\{q1 , . . . , qi }

F = set of accept states in P δ

= transition function in P 106 / 148


pba-20b


a

. . . qk , δ (R, a) q1 , q2, . . . , qi , qi+1

, . . . , qk , δ (R, a) next state q1 , q3 , . . . qi ,qi+1

shift

107 / 148


pba-20b


a

. . . qk , δ (R, a) q1 , q2, . . . , qi , qi+1

, . . . , qk , δ (R, a) next state q1 , q3 , . . . qi ,qi+1

shift

acceptance condition: ♦ “no shift in component j” ∧ ♦ F) (♦ j

108 / 148

Complementation of PBA

pba-21

Theorem: The class of PBA-recognizable languages is closed under complementation.

109 / 148


pba-21


PBA P

110 / 148


pba-21


PBA −→ 0/1-PRA P

PR

111 / 148


pba-21

Theorem: The class of PBA-recognizable languages is closed under complementation. compl.

PBA −→ 0/1-PRA −−−→ 0/1-PSA P

PR

112 / 148


pba-21


PBA −→ 0/1-PRA −−−→ 0/1-PSA P

PR

=

PS

113 / 148


pba-21


PBA −→ 0/1-PRA −−−→ 0/1-PSA −→ PBA P

PR

=

PS

P

114 / 148


pba-21

Theorem: The class of PBA-recognizable languages is closed under complementation. exp

compl.

poly

PBA −−→ 0/1-PRA −−−→ 0/1-PSA −−→ PBA P

PR

=

PS

P

115 / 148

Operators on PBA

pba-21

Theorem: The class of PBA-recognizable languages is closed under complementation, union, intersection. complementation: exp

compl.

poly

PBA −−→ 0/1-PRA −−−→ 0/1-PSA −−→ PBA union and intersection: • union: random choice between two PBA • intersection: via generalized PBA (as for NBA)

116 / 148

Decision problems for PBA

pba-22

The emptiness problem for PBA given: PBA P P ) = ∅ hold? question: does L (P

117 / 148


pba-22

The emptiness problem for PBA given: PBA P P ) = ∅ hold? question: does L (P is undecidable.

118 / 148


pba-22

The emptiness problem for PBA given: PBA P P ) = ∅ hold? question: does L (P is undecidable. proof by a reduction from the emptiness problem for probabilistic finite automata (PFA) [Paz’71, Madani/Hanks/Condon’03]

119 / 148


pba-22

The emptiness problem for PBA is undecidable. Hence, the following problems are undecidable too:

120 / 148


pba-22

The emptiness problem for PBA is undecidable. Hence, the following problems are undecidable too: • universality: given a PBA P , P ) = Σω hold? does L (P

121 / 148


pba-22

The emptiness problem for PBA is undecidable. Hence, the following problems are undecidable too: • universality: given a PBA P , P ) = Σω hold? does L (P • equivalence: given two PBA P 1 , P 2 , P 1 ) = L (P P 2 ) hold? does L (P

122 / 148


pba-22

The emptiness problem for PBA is undecidable. Hence, the following problems are undecidable too: P ) = Σω hold? • universality: does L (P P 1 ) = L (P P 2 ) hold? • equivalence: does L (P • model checking finite transition systems against PBA-specifications

123 / 148


pba-22

The emptiness problem for PBA is undecidable. Hence, the following problems are undecidable too: P ) = Σω hold? • universality: does L (P P 1 ) = L (P P 2 ) hold? • equivalence: does L (P • model checking finite transition systems against PBA-specifications given a finite TS T and PBA P , does there exist P )? a path π in T s.t. trace(π) ∈ L (P

124 / 148


pba-22

The emptiness problem for PBA is undecidable. Hence, the following problems are undecidable too: P ) = Σω hold? • universality: does L (P P 1 ) = L (P P 2 ) hold? • equivalence: does L (P • model checking finite transition systems against PBA-specifications given a finite TS T and PBA P , does there exist P )? a path π in T s.t. trace(π) ∈ L (P given a finite TS T and PBA P , does P ) hold for all paths π in T ? trace(π) ∈ L (P 125 / 148


pba-22

The following problems are undecidable: • emptiness for PBA • universality for PBA • equivalence for PBA • model checking finite TS against PBA-specifications • verification of observation-based stochastic games formalized by partially-observable MDPs (POMDPs) against ω -regular specifications:

126 / 148


pba-22

The following problems are undecidable: • emptiness for PBA • universality for PBA • equivalence for PBA • model checking finite TS against PBA-specifications • verification of observation-based stochastic games formalized by partially-observable MDPs (POMDPs) against ω -regular specifications: ♦ F) > 00? * does there exist a strategy S s.t. Pr S ( ♦ F) = 11? * does there exist a strategy S s.t. Pr S(♦ 127 / 148

Decision problems for POMDPs

pba-22a

Theorem: The following instances of the verification problem for finite POMDPs against ω -regular specifications is undecidable: ♦ F) > 00? * does there exist a strategy S s.t. Pr S( ♦ F) = 11? * does there exist a strategy S s.t. Pr S(♦

128 / 148


pba-22a

Theorem: The following instances of the verification problem for finite POMDPs against ω -regular specifications is undecidable: ♦ F) > 00? * does there exist a strategy S s.t. Pr S( ♦ F) = 11? * does there exist a strategy S s.t. Pr S(♦ but the following problems are decidable: F) > 00? * does there exist a strategy S s.t. Pr S( [de Alfaro’99]

129 / 148


pba-22a

Theorem: The following instances of the verification problem for finite POMDPs against ω -regular specifications is undecidable: ♦ F) > 00? * does there exist a strategy S s.t. Pr S( ♦ F) = 11? * does there exist a strategy S s.t. Pr S(♦ but the following problems are decidable: F) > 00? * does there exist a strategy S s.t. Pr S( [de Alfaro’99] ♦F) = 11? * does there exist a strategy S s.t. Pr S(♦ 130 / 148


pba-22a

Theorem: The following instances of the verification problem for finite POMDPs against ω -regular specifications is undecidable: ♦ F) > 00? * does there exist a strategy S s.t. Pr S( ♦ F) = 11? * does there exist a strategy S s.t. Pr S(♦ but the following problems are decidable: F) > 00? * does there exist a strategy S s.t. Pr S( ♦F) = 11? * does there exist a strategy S s.t. Pr S(♦ ♦ F) = 11? * does there exist a strategy S s.t. Pr S( ♦ F) > 00? * does there exist a strategy S s.t. Pr S(♦ 131 / 148


pba-22a

Theorem: The following instances of the verification problem for finite POMDPs are decidable: .. . ♦ F) = 11? does there exist a strategy S s.t. Pr S ( .. .

132 / 148


pba-22a

Theorem: The following instances of the verification problem for finite POMDPs are decidable: .. . ♦ F) = 11? does there exist a strategy S s.t. Pr S ( .. . Corollary: The emptiness problem for PBA with the almost-sure semantics is decidable.

133 / 148

... undecidability results for general PBA, but ...

pba-23

• emptiness problem for PBA with the almost-sure semantics is in EXPTIME

134 / 148


pba-23

• emptiness problem for PBA with the almost-sure semantics is in EXPTIME • emptiness problem for uniform PBA is in EXPTIME

135 / 148


pba-23

• emptiness problem for PBA with the almost-sure semantics is in EXPTIME • emptiness problem for uniform PBA is in EXPTIME exp

via uPBA −−→ NSA

136 / 148


pba-23

• emptiness problem for PBA with the almost-sure semantics is in EXPTIME • emptiness problem for uniform PBA is in EXPTIME • model checking Markov chains against uPBA-spec given: finite Markov chain M , uniform PBA P L(P P )) > 0 hold? question: does PrM (L

137 / 148


pba-23

• emptiness problem for PBA with the almost-sure semantics is in EXPTIME • emptiness problem for uniform PBA is in EXPTIME • model checking Markov chains against uPBA-spec given: finite Markov chain M , uniform PBA P L(P P )) > 0 hold? question: does PrM (L L(P P )) denotes the probability measure where PrM (L π ) ∈ L (P P) of the set of paths π in M s.t. trace(π 138 / 148


pba-23

• emptiness problem for PBA with the almost-sure semantics is in EXPTIME • emptiness problem for uniform PBA is in EXPTIME • model checking Markov chains against uPBA-spec given: finite Markov chain M , uniform PBA P L(P P )) > 0 hold? question: does PrM (L is in PTIME as L(P P )) > 0 iff PrM×P ( ♦ F) > 0 PrM (L

139 / 148


pba-23

• emptiness problem for PBA with the almost-sure semantics is in EXPTIME • emptiness problem for uniform PBA is in EXPTIME • model checking Markov chains against uPBA-spec given: finite Markov chain M , uniform PBA P L(P P )) > 0 hold? question: does PrM (L is in PTIME as L(P P )) > 0 iff PrM×P ( ♦ F) > 0 PrM (L ↑ analysis of the SCCs in the product Markov chain 140 / 148

Conclusion

pba-conc

• expressiveness: PBA are more expressive than NBA and almost-sure PBA,

141 / 148

Conclusion

pba-conc

• expressiveness: PBA are more expressive than NBA and almost-sure PBA, while uniform PBA have equal power than NBA

142 / 148

Conclusion

pba-conc

• expressiveness: PBA are more expressive than NBA and almost-sure PBA, while uniform PBA have equal power than NBA • efficiency: PBA can be exp smaller than NSA

143 / 148

Conclusion

pba-conc

• expressiveness: PBA are more expressive than NBA and almost-sure PBA, while uniform PBA have equal power than NBA • efficiency: PBA can be exp smaller than NSA • polynomial transformation PSA PBA

144 / 148

Conclusion

pba-conc

• expressiveness: PBA are more expressive than NBA and almost-sure PBA, while uniform PBA have equal power than NBA • efficiency: PBA can be exp smaller than NSA • polynomial transformation PSA PBA • undecidability results for PBA and POMDPs

145 / 148

Conclusion

pba-conc

• expressiveness: PBA are more expressive than NBA and almost-sure PBA, while uniform PBA have equal power than NBA • efficiency: PBA can be exp smaller than NSA • polynomial transformation PSA PBA • undecidability results for PBA and POMDPs • decidability results for almost-sure PBA and POMDPs

146 / 148

Conclusion

pba-conc

• expressiveness: PBA are more expressive than NBA and almost-sure PBA, while uniform PBA have equal power than NBA • efficiency: PBA can be exp smaller than NSA • polynomial transformation PSA PBA • undecidability results for PBA and POMDPs • decidability results for almost-sure PBA and POMDPs • application: run-time verification (probabilistic monitoring) [Sistla et al] 147 / 148

Conclusion • • • • •

pba-conc

expressiveness ... efficiency .... polynomial transformation PSA PBA (un)decidability results for PBA and POMDPs application: run-time verification [Sistla et al]

many open problems: • transformations LTL PBA or MSO PBA • alternative semantics for PBA • variants: NPBA, QBA, ... 148 / 148