The Complexity of Probabilistic versus Deterministic Finite Automata Andris Ambainis Institute of Mathematics and Computer Science University of Latvia Raina bulv. 29, Riga, Latvia e-mail:
[email protected] No Institute Given
Abstract. We show that there exists probabilistic finite automata with an isolated cutpoint and n states such that the smallest equivalent deterministic finite automaton contains Ω(2
n
log log n log n
) states.
Keywords: Automata theory, the complexity of finite automata, probabilistic finite automata.
1
Introduction
Rabin[Ra63] proved that arbitrary language which is accepted by a probabilistic finite automaton with an isolated cutpoint is also accepted by a deterministic finite automaton. However, in the process of transition from a probabilistic to a deterministic finite automaton, the complexity increases. Rabin[Ra63] showed that for probabilistic automaton with n states, r accepting states, and isolation radius δ there exists an equivalent deterministic finite automaton with at most (1 + rδ )n states. Later this estimate was slightly improved by Paz[Paz66] and Gabbasov and Murtazina[GM79]. However, even after these improvements the estimate remained exponential, only the base of the exponent became smaller. Construction of concrete languages for which deterministic automata are more complex than probabilistic ones appeared to be quite a complicated problem. Freivalds[Fr82] constructed probabilistic finite automata with n√states for which the smallest equivalent deterministic automaton contains Ω(2 n ) states. The problem of constructing probabilistic automata with n states such that any equivalent deterministic automaton contains an states still remains open. In this paper we present a language such that – it is accepted by a probabilistic automaton with n states, and – any deterministic automaton accepting this language has Ω(2n/ log n ) states.
2
Results
We shall use the standard definitions of probabilistic and deterministic finite automata[TB73]. Theorem 1 There exists a probabilistic finite automaton with n states such n log log n that the smallest equivalent deterministic finite automaton contains Ω(2 log n ) states. Proof. Consider the language Lm having the alphabet {a1 , a2 , . . . , am } and consisting of all words that contain each of the letters a1 , . . . , am exactly m times. Lemma 1 If a deterministic finite automaton accepts Lm , it has at least (m + 1)m states. Proof. The automaton needs to remember the number of a1 ’s in the part of the input word read, the number of a2 ’s, . . . , the number of am ’s. The number of a1 ’s can assume (m + 1) possible values: 0, 1, . . . , m. So, there are (m + 1)m possible values for the combination of the number of a1 ’s, the number of a2 ’s and so on. 2 Lemma 2 There exists a probabilistic finite automaton with isolated cutpoint log2 m which accepts Lm and has O(m log log m ) states. Proof. We shall construct an automaton accepting Lm with the probability 3/4. (For other accepting probabilities the construction is similar.) In our construction we shall use automata constructed by Freivalds[Fr82] as subroutines. Freivalds[Fr82] considered automata in a one-letter alphabet and proved the following result: Lemma 3 [Fr82] There exists a probabilistic finite automaton with an isolated cutpoint that recognizes whether the word consists of exactly n words and has log2 n O( log log n ) states. Let p be the smallest prime larger than 6m. From prime number distribution theorems we have that p = 6m + o(m). Let k = pm2 . Ui denotes the probabilistic automaton that recognizes (with 9 accepting probability at least 10 ) whether the word consists of exactly i letters. si denotes the number of states in Ui and s denotes max(s1 , . . . , sk ). The states of Ui are denoted by Ai,1 , Ai,2 , . . . , Ai,si . Without the loss of generality, we assume that the starting state of Ui is Ai,1 . Informally, we shall consider an automaton which, at the beginning, chooses i ∈ {1, . . . , 6m} equiprobably, and then counts the sum m X j=1
(ij mod p)nj
where n1 is the number of letters a1 in the word, n2 is the number of a2 ’s and so on. For the counting of this sum probabilistic automata by Freivalds[Fr82] are used. A more formal desription follows: Automaton U has 1 + 6ms states: 1. Starting state S; 2. States Qi,j for i ∈ {1, . . . , 6m} and j ∈ {1, . . . , s}. From the starting state S, the automaton without reading any input passes to one of the states Qi,1 equiprobably (with probability 1/6m). If the automaton is in one of states Qi,j , it reads the symbol and passes to one of states Qi,1 , . . . , Qi,s . The probability of moving from state Qi,j to state Qi,j 0 after reading at is equal to the probability of moving from Ai0 ,j to Ai0 ,j 0 after reading (it−1 mod p) symbols in automaton Ui0 where i0 =
m X
m(ij−1 mod p).
j=1
The state Qi,j is defined to be an accepting state iff the corresponding state Ai0 ,j in automaton Ui0 is accepting. Proof of correctness. We use Lemma 4 If i1 , . . . , im ∈ {1, 2, . . . , 6m} are pairwise different, then the vectors v1 , . . . , vm where vj = (i0j mod p, i1j mod p, . . . im−1 mod p) j are linearly independent. Proof. The proof is similar to the proof of linear independence for Vandermonde’s determinant. Consider the matrix 0 i1 mod p i11 mod p . . . i1m−1 mod p i0 mod p i1 mod p . . . im−1 mod p 2 2 2 ... ... ... ... m−1 0 1 im mod p im mod p . . . im mod p Rows of this matrix (i.e. vj ’s) are linearly independent iff the columns of it are independent. By the way of contradiction assume that some of columns are dependent. Then c1 multiplied by the first column plus c2 multiplied by the second col→ − umn and so on, plus cm multiplied by the last column is equal to 0 where c1 , c2 , . . . , cm are some constants which are not all equal to 0. It follows that, for all j ∈ {1, . . . , m} c1 + c2 (ij mod p) + c3 (i2j mod p) + . . . + cm (im−1 mod p) = 0 j
c1 + c2 ij + c3 i2j + . . . + cm im−1 =0 j
(mod p)
c1 + c2 x + c3 x2 + . . . + cm xm−1 = 0
(mod p)
So, an equation
has m solutions: i1 , i2 , . . . , im . By definition, p > 6m. Hence all m solutions i1 , . . . , im are different modulo p. However, an equation of degree m − 1 can have only m − 1 different solutions. Contradiction. 2 Lemma 5 If the described automaton starts from Qi,1 , the probability of accepting a word u is 1. greater or equal than 9/10 if n X
nj (ij−1 mod p) =
j=1
n X
m(ij−1 mod p)
j=1
2. less or equal than 9/10, otherwise. Proof. By definition of our automaton, the probabilities of passing from one state to another after reading the symbol aj are equal to similar probabilities in automaton Ui0 after reading (ij−1 mod p) symbols. Hence, the probability of passing from Qi,1 to an accepting state after reading the word consisting of n1 symbols a1 , n2 symbols a2 , . . ., nm symbols am is equal to the probability of passing from Ai0 ,1 to the accepting state in automaton Ui0 after reading m X
nj (ij−1 mod p)
j=1
symbols. By definition, Ui0 accepts the word consisting of i0 =
m X
m(ij−1 mod p)
j=1
symbols with a probability greater or equal to 9/10 and rejects other words with a probability greater or equal to 9/10. 2 Consider two cases: 1. The automaton receives a word u ∈ Lm . Then n1 = n2 = . . . = nm = m. From Lemma 5 we have that the automaton starting from any of states Q1,1 , . . . , Q6m,1 accepts u with a probability 9/10. Starting from its normal starting state S, the automaton equiprobably chooses one of states Q1,1 , . . . , Q6m,1 and moves to it. So, it accepts the word u ∈ Lm with a probability at least 9/10.
2. Automaton receives a word u ∈ / Lm . Then equality m X
nj (ij−1 mod p) =
j=1
m X
m(ij−1 mod p)
j=1
holds for at most (m − 1) different i ∈ {1, . . . , 6m}. (If this equality holds for m different i ∈ {1, . . . , 6m} then, from m X (nj − m)(ij−1 mod p) = 0 j=1
and from the linear independence of vectors vi (Lemma 4) it follows that nj − m = 0 for all j. This means that u ∈ Lm .) So, with a probability of at least 1 − state S to state Qi,1 such that m X
m−1 6m
nj (ij−1 mod p) 6=
j=1
> 5/6 the automaton moves from
m X
m(ij−1 mod p).
j=1
After starting from such state Qi,1 the word is rejected with a probability of at least 9/10 (Lemma 5). Hence, the probability of rejecting the word u∈ / Lm is at least 5/6 ∗ 9/10 = 3/4. We have shown that words belonging to Lm are accepted by an automaton with a probability of at least 9/10 and the words which do not belong to Lm are rejected with a probability of at least 3/4. The number of states in automaton U . The automaton U consists of 1 + 6ms states. We have log2 k s=O , and k = pm2 = (6m + o(m))m2 = O(m3 ). log log k Hence s=O
log2 m log log m
.
The number of states is log2 m . 1 + 6ms = O m log log m 2 Denote
log2 m n=O m . log log m
Then, the language Lm can be accepted by a probabilistic finite automaton whith n states (Lemma 2). Any deterministic finite automaton accepting Lm has at least (m + 1)m states (Lemma 1). We have n log log n (m + 1)m ≈ 2m log m = Ω 2 log n This proves the theorem. 2 Acknowledgements. I would like to thank R¯ usi¸nˇs Freivalds for suggesting the problem and help during this research. Research was supported by Latvia’s Science Council Grant No. 93.599.
References [Fr82] R. Freivalds, On the growth of the number of states in result of determinization of probabilistic finite automata, Avtomatika i Vicislitelnaja Tehnika, 1982, N.3, 39-42 (in Russian) [GM79] N. Z. Gabbasov, T. A. Murtazina, Improving the estimate of Rabin’s reduction theorem, Algorithms and Automata, Kazan University, 1979, 7-10 (in Russian) [Paz66] A. Paz, Some aspects of probabilistic automata, Information and Control, 9(1966) [Ra63] M. O. Rabin, Probabilistic automata, Information and Control, 6(1963), 230245 [TB73] B. A. Tracktenbrot, Ya. M. Barzdin’, Finite Automata: Behaviour and Synthesis. North-Holland, 1973
