A LOWER BOUNDTO ... - Stanford InfoLab - Stanford University
A LOWER BOUNDTO PALINDROMERECOGNITION BY PROBABILSTICTURINGMACHINES
Andrew C. Yao
STAN-CS-77-647 DECEMBER 1977
COMPUTER SCIENCE DEPARTMENT School of Humanities and Sciences STANFORD UNIVERSITY
A Lower Bound to Palindrome Recognition by Probabilistic Turing Machines ..
Andrew Chi-Chih Yao Computer Science Department Stanford University Stanford, California 94305
Abstract. We call attention to the problem of proving lower bounds on probabilistic Turing machine camgutations. It is shown that any probabilistic Turing machine recognizing the language L =
{W
4 w 1
WE
[O,l)*) with error h < l/2 must take n(n log n)
time.
a
Keywords:
computational complexity, crossing sequences, lower bound, palindrome, probabilistic Turing machine.
This research was supported in part by National Science Foundation grants MCS-72-03752 A03 and MCS-77905313.
1
1.
Introduction. The idea of including controlled stochastic moves in algorithms has
received considerable attention recently [4,9,10,12]. The demonstration by Rabin [g] and Solovay and Strassen [lo], that fast tests for prime numbers can be done probabilisticaJ.ly with a small error, raised the hope that many more problems may allow similar fast algorithms. As in deterministic computations, a challenging problem is to prove lower bounds to the computational complexity of specific problems for such probabilistic algorithms. was initiated in Yao [12],
An investigation for decision tree type models The techniques used there [12], however,
are not applicable t-o Turing machine computations, for which a number of lower bound results are known for the deterministic computations ( see, e.g. [5]). In this paper, we call attention to proving lower bounds in probabilistic Turing machines, by proving a non-linear bound to a palindrome-like language. It is well known that it takes any deterministic one-tape Turing machine
nb2)
steps to recognize the language L = {w 4 w 1
WE
(O,l)*)
interesting result of Freivald [3], cited in ( see, e.g. [5]). A very . Gill's paper [4], states that L can be recognized with a small error by a one-tape probabilistic Turing machine in time O(n(log n)2) . Recently, this bound was improved to .
O(n log n) by Nick Pippenger [7].
This seems to be the only example known in a Turing machine model where a provable speed-up is achieved, in an order-of-magnitude sense, by allowing stochastic decisions.
The purpose of the present paper is to
show that n(n log n) -time is also a lower bound to any one-tape probabilistic Turing machine recognizing
2
L with a small error (Theorem 4.1).
2.
.
Definitions and Notations. We first give an informal description of probabilistic Turing machines,
the readers are referred to Gill [k] (and references therein) for more detailed discussions.
A probabilistic one-tape Turing machine (l-Pm) M
consists of a finite control, a read-write head on an infinite l-dimensional tape, and a randam symbol generator (RSG) capable of generating integers i between 1 and i. a random symbol i
with fixed probabilities pi .
Before each move,
is generated by the RSG, and the action of Turing
machine M depends on the current state of M , the symbol on the tape being read, and the random symbol i . states
go
.,
cl-i
9
and
92
l
There are three distinguished
The machine starts in
and if it halts, it must halt in either ql (the rejecting state). For a given input
go (initial state),
(the accepting state) or q2
v t it is possible that M
will not halt for some infinite sequences of random symbols that are generated by RSG. given input
v , halts with probability 1 (it maq still not halt for
sane sequences).
e
We shall restrict ourselves to M which, for any
For each input v ) let pi(v) be the probability
that M halts in state . i = 1,2) . A language L is recognized . % ( by M with error A (0 < h < l/2) , if pi(v) > 1-h for each VEL , and B2(v) 2 1-h for each v{L . Intuitively, given an input v - if we accept or reject is
on M,
v when M halts depending on whether the state
s, or 92 9 then we would be wrong at most with probability h .
We shall now introduce some notations, and give a formal definition of terms involving probability described in the last paragraph. an input word, and
Let v be
1 and i.
c a finite sequence of integers between
generated by RSG that leads to the halting of M , i.e., M halts CT rs generated.
immediately after the last element in
Let A(v) denote the set of all
is a decision sequence for v on M .
v , and Ai c A(v) (i = 1,2) be the subsets
decision sequences for consisting of
We say that c
(J leading to state
qi when M halts. We use c[j]
for the j-th element in the sequence c , and \cI for the length of c . . the probability that the first
Define P(a) = paLl] paL23
IuI
l
**
Pu[
(up
’
random symbols generated by RSG form the sequence c .
restate some terms defined earlier in these notations. that M halts for any v with probability 1 means
We now
The condition c P(Q) = 1 ;
TV E A(v) the quantities pi(v) are
c
p(a) for i = 1,2 . Also, the
acAi(v) expected number of steps
FM(V) =
c
M will make for input v is equal to
ceA(v) since
I0 I
generated.
p
l
2
Proof.
The lemma is obviously true if yM(v) = 03 ; we therefore assume
that FM(v) is fini t e.
For each ceA(v) , the nwnber of steps taken is
at least as large as the sum of the lengths of all crossing sequences. Thus, -.
ph W> 1 . By definition,
where in the last step we have changed the order of summation of an absolutely convergent double series (see, e.g. [XL, p. 28, Example 11). Signatures and Patterns. --. Let Q = 1~Y91'Q2'...'Qr] of tape symbols used by M .
be the set of states, and r be the set
Denote Q - (s,, %'J by Q' .
Suppose during the camputation process of M , the following configuration is encountered. j+l -st cell to
A word uer*
is on the tape from the
(j+luI) -th cell, and all cells to the right of u
are blank (see Figure 1 top). The machine M is in state s , and its head is just crossing from the j-th to the j+l -st cell.
cell j
word u f
a
cell j
-
-
blanks -
blanks word z \\e
I
I
I
I I I I I I I
7 Figure 1.
Illustration for g(s,u;t,z) . 6
0
We are interested in the situation when M cCrlIles back crossing from the j+l -St cell to the j-th cell for the first time.
As M is not
deterministic, the state M is in and the contents of cells at this time may not be unique.
We shall use. g(s,u;t,z) , where teQ , zer* ,
to denote the probability that M is in state t and the contents of cells from the j+l -St cells on are the word z followed by blanks (see Figure 1, bottom). The function g is independent of the contents in cells z t,
i 5 j , and the explicit value of j . Clearly
g(w;t,q
l. The k-th-order left signature
2*(r-l)2k-1 -tuple of numbers,
. l l , Sk, $)
c g(sl,u;tl’zl) xg(s~‘Zl;t2’z2) X”* d’k’zk_l;tk’zk) t ,...,ZkEr+ z1Jz2 (3)
for each
s 1,
t
1,S2,t2,-,skEQ1
We shall show that
G (k)
ad
+
(qys,3
l
are well defined by (3) and in fact, satisfy
OSG 04 (w 1Yt 1" . . ..Sk.tk) 5 1 .
(4)
As all terms in the summation (3) are non-negative, it is sufficient to prove that, for every finite subset v s r* , and every uey* , slytly...yskEQ' , tke {qlyq2) ,the following is true:
7
c
dsl,u;tl,zl)
Xds2,zl;t2’Z2)
x
“*
xg(sk,Zk-l;tk~zk)
5
l
l
z1,z2~...,Zkd.f
..
This can be proved by induction on k ; we have
c
1
z g(s2,z;t2,z2) x '** xgbk,zk-l;tk'Zk) z2, z3". .., Zk E v
l.
The k-th-order right signature
of x is the (r-1)2k-1 -tuple, H 04 (x;s 1' + ‘23 $2
=
’ ’ l , ‘k-1,
$1, “k)
-. Y+$, l -,Yk E r*
ho(x;slYYl) Xh(tlPYl;S2JY2) Xh(t2YY2;s3YY3)
x ‘*’ x h(tk,l’Yk,l;sk”Yk)
,
where all si,ti EQ' . As in the case of satisfy
G 04 , the numbers
-v. 04 0 < H (X;sl,tl,...,sk)
l/5 , contradicting'the fact that V'W" $! Ln
(thus Bl(v'w') = 1-p*(vw") 5 l/10 ).
Let d =
r
$ iog, n
Claim 4.3 leads to
I Ln
- F
1
0
. Then, for all sufficiently large n ,
> 2n- exp(32d3r4d In r) -2X2n > ' nrj w 1 -
1 = z
Lnl ’
Thus, for each 2n+l >
c iM(v) VE Ln
VE Ln
1 c rLn - r *n+l<j 0 ; thus one can
17
decide if wl = w2
with only a small chance of error by comparing
wl(mod p) with w,(mod p)
for a fixed number of such randoti primes p
generated by, for example, the method used in (a). These ideas imply that we can check the equation xxy E z Smith only a small chance of error by generating a few m-bit random primes
p , computing x(mod p) ,
y(mod p) and z(mod p) , and checking equations (x(mod p)*y(mod p))(mod p) = z(mod p) . The running time is dominated by the computing of x,y,z(mod p) , which takes 0 (iM(m))
time
(cf. [ll) 0 To end this section, we remark that the bound in the corollary to Theorem 4.1 is the best possible.
By a slight adaptation of Pippenger's
l-PTM for recognizing [wtiw') [7], one can construct a l-PTM recognizing rl-" # ln 1 n > 1) with a small error in time
O(n log log n) , thus
achieving the lower bound stated in the corollary.
18
6.
Conclusions. The subject of proving lower bounds for probabilistic Turing machines
offers many challenging problems, of which only one is solved in this paper.
It seems to be most fruitful to consider problems where good
bounds exist in the deterministic case.
We believe such studies will
provide insights to probabilistic computations beyond the framework of Turing machine models.
We mention only two such problems for further
research.
(i >
With a read-only input tape and several working tapes, is the extra space requirement for recognizing {w#w) probabilistically (with error) !&log n) ? (See [3, p. 154, Exercise 10.33 for the deterministic analogue.
( ii )
Can Rabin's language defined in [8] be recognized in real time by a probabilistic Turing machine with one working tape? (Deterministically it cannot [8].) Finally we like to mention that the overlap argument for on-line
multiplication (Cook and Aanderaa [2], also Paterson, Fischer, and e
Meyer [6]) can be extended to the probabilistic case [13].
Acknowledgments.
I wish to thank John Gill for a stimulating conversation
and for c ommunicating Pippenger's result [7] to me.
19
p
References
Dl
A. V. Aho, J. E. Hopcroft, and J. D. UUman, The Design and Analysis
PI
of Computer Algorithms, Addison-Wesley, Reading, Mass,, 1974. S. A. Cook and S. 0. Aanderaa, "On the minim computation time of functions," Trans. Amer. Math. Sot. 142 (1969), 291-314.
131
R. V. Freivald, "Fast computation by probabilistic Turing machines," Theory of Algorithms and Programs, no. 2, Latvian State University, Riga, 1975, 201-205 (in Russian).
i
VI
J. T. Gill III, "Computational complexity of probabilistic Turing machines,"
[51
SIAM
J. on Computing 6 (1917), 675-695.
J. E. Hopcroft and J. D, Ul&nan, Formal Languages and Their Relation to Automata, Addison-Wesley, Reading, Mass., 1969.
El
M. S. Paterson, M. J. Fischer, and A. R, Meyer, "An improved overlap argument for on-line multiplication," SIAM-AMS Proc., vol. 7,
171
Amer. Math. SOC., Providence, R.I., 1974, 97-111. N. Pippenger, private cammunication, November 1977.
I81
M. 0. Rabin, "Real-time computation," Israel J. Math. 1 (1963), 203-211.
L91
M. 0. Rabin, "Probabilistic algorithms," in Algorithms and Caaplexity: New Directions and Recent Results, J. F, Traub, ed., Academic Press, New York, 1976, 21-39,
DOI
R. Solovay and V. Strassen, "A fast Monte-Carlo test for primslity,"
D-U
J. on Computiq 6 (l-977), 84-85. E. T, Whittaker and G, N. Watson, A Course of Modern Analysis, 4th edition, Cmbridge University Press, 1958.
cm
A. C. Yao, "Probabilistic computations -- toward a unified measure
SIAM
- of complexity," Proc. 18th Annual Symp. on Foundations of Computer Science, 1977, 222-227. ' [13 1 A. C. Yao, unpublished.
20
Andrew C. Yao
STAN-CS-77-647 DECEMBER 1977
COMPUTER SCIENCE DEPARTMENT School of Humanities and Sciences STANFORD UNIVERSITY
A Lower Bound to Palindrome Recognition by Probabilistic Turing Machines ..
Andrew Chi-Chih Yao Computer Science Department Stanford University Stanford, California 94305
Abstract. We call attention to the problem of proving lower bounds on probabilistic Turing machine camgutations. It is shown that any probabilistic Turing machine recognizing the language L =
{W
4 w 1
WE
[O,l)*) with error h < l/2 must take n(n log n)
time.
a
Keywords:
computational complexity, crossing sequences, lower bound, palindrome, probabilistic Turing machine.
This research was supported in part by National Science Foundation grants MCS-72-03752 A03 and MCS-77905313.
1
1.
Introduction. The idea of including controlled stochastic moves in algorithms has
received considerable attention recently [4,9,10,12]. The demonstration by Rabin [g] and Solovay and Strassen [lo], that fast tests for prime numbers can be done probabilisticaJ.ly with a small error, raised the hope that many more problems may allow similar fast algorithms. As in deterministic computations, a challenging problem is to prove lower bounds to the computational complexity of specific problems for such probabilistic algorithms. was initiated in Yao [12],
An investigation for decision tree type models The techniques used there [12], however,
are not applicable t-o Turing machine computations, for which a number of lower bound results are known for the deterministic computations ( see, e.g. [5]). In this paper, we call attention to proving lower bounds in probabilistic Turing machines, by proving a non-linear bound to a palindrome-like language. It is well known that it takes any deterministic one-tape Turing machine
nb2)
steps to recognize the language L = {w 4 w 1
WE
(O,l)*)
interesting result of Freivald [3], cited in ( see, e.g. [5]). A very . Gill's paper [4], states that L can be recognized with a small error by a one-tape probabilistic Turing machine in time O(n(log n)2) . Recently, this bound was improved to .
O(n log n) by Nick Pippenger [7].
This seems to be the only example known in a Turing machine model where a provable speed-up is achieved, in an order-of-magnitude sense, by allowing stochastic decisions.
The purpose of the present paper is to
show that n(n log n) -time is also a lower bound to any one-tape probabilistic Turing machine recognizing
2
L with a small error (Theorem 4.1).
2.
.
Definitions and Notations. We first give an informal description of probabilistic Turing machines,
the readers are referred to Gill [k] (and references therein) for more detailed discussions.
A probabilistic one-tape Turing machine (l-Pm) M
consists of a finite control, a read-write head on an infinite l-dimensional tape, and a randam symbol generator (RSG) capable of generating integers i between 1 and i. a random symbol i
with fixed probabilities pi .
Before each move,
is generated by the RSG, and the action of Turing
machine M depends on the current state of M , the symbol on the tape being read, and the random symbol i . states
go
.,
cl-i
9
and
92
l
There are three distinguished
The machine starts in
and if it halts, it must halt in either ql (the rejecting state). For a given input
go (initial state),
(the accepting state) or q2
v t it is possible that M
will not halt for some infinite sequences of random symbols that are generated by RSG. given input
v , halts with probability 1 (it maq still not halt for
sane sequences).
e
We shall restrict ourselves to M which, for any
For each input v ) let pi(v) be the probability
that M halts in state . i = 1,2) . A language L is recognized . % ( by M with error A (0 < h < l/2) , if pi(v) > 1-h for each VEL , and B2(v) 2 1-h for each v{L . Intuitively, given an input v - if we accept or reject is
on M,
v when M halts depending on whether the state
s, or 92 9 then we would be wrong at most with probability h .
We shall now introduce some notations, and give a formal definition of terms involving probability described in the last paragraph. an input word, and
Let v be
1 and i.
c a finite sequence of integers between
generated by RSG that leads to the halting of M , i.e., M halts CT rs generated.
immediately after the last element in
Let A(v) denote the set of all
is a decision sequence for v on M .
v , and Ai c A(v) (i = 1,2) be the subsets
decision sequences for consisting of
We say that c
(J leading to state
qi when M halts. We use c[j]
for the j-th element in the sequence c , and \cI for the length of c . . the probability that the first
Define P(a) = paLl] paL23
IuI
l
**
Pu[
(up
’
random symbols generated by RSG form the sequence c .
restate some terms defined earlier in these notations. that M halts for any v with probability 1 means
We now
The condition c P(Q) = 1 ;
TV E A(v) the quantities pi(v) are
c
p(a) for i = 1,2 . Also, the
acAi(v) expected number of steps
FM(V) =
c
M will make for input v is equal to
ceA(v) since
I0 I
generated.
p
l
2
Proof.
The lemma is obviously true if yM(v) = 03 ; we therefore assume
that FM(v) is fini t e.
For each ceA(v) , the nwnber of steps taken is
at least as large as the sum of the lengths of all crossing sequences. Thus, -.
ph W> 1 . By definition,
where in the last step we have changed the order of summation of an absolutely convergent double series (see, e.g. [XL, p. 28, Example 11). Signatures and Patterns. --. Let Q = 1~Y91'Q2'...'Qr] of tape symbols used by M .
be the set of states, and r be the set
Denote Q - (s,, %'J by Q' .
Suppose during the camputation process of M , the following configuration is encountered. j+l -st cell to
A word uer*
is on the tape from the
(j+luI) -th cell, and all cells to the right of u
are blank (see Figure 1 top). The machine M is in state s , and its head is just crossing from the j-th to the j+l -st cell.
cell j
word u f
a
cell j
-
-
blanks -
blanks word z \\e
I
I
I
I I I I I I I
7 Figure 1.
Illustration for g(s,u;t,z) . 6
0
We are interested in the situation when M cCrlIles back crossing from the j+l -St cell to the j-th cell for the first time.
As M is not
deterministic, the state M is in and the contents of cells at this time may not be unique.
We shall use. g(s,u;t,z) , where teQ , zer* ,
to denote the probability that M is in state t and the contents of cells from the j+l -St cells on are the word z followed by blanks (see Figure 1, bottom). The function g is independent of the contents in cells z t,
i 5 j , and the explicit value of j . Clearly
g(w;t,q
l. The k-th-order left signature
2*(r-l)2k-1 -tuple of numbers,
. l l , Sk, $)
c g(sl,u;tl’zl) xg(s~‘Zl;t2’z2) X”* d’k’zk_l;tk’zk) t ,...,ZkEr+ z1Jz2 (3)
for each
s 1,
t
1,S2,t2,-,skEQ1
We shall show that
G (k)
ad
+
(qys,3
l
are well defined by (3) and in fact, satisfy
OSG 04 (w 1Yt 1" . . ..Sk.tk) 5 1 .
(4)
As all terms in the summation (3) are non-negative, it is sufficient to prove that, for every finite subset v s r* , and every uey* , slytly...yskEQ' , tke {qlyq2) ,the following is true:
7
c
dsl,u;tl,zl)
Xds2,zl;t2’Z2)
x
“*
xg(sk,Zk-l;tk~zk)
5
l
l
z1,z2~...,Zkd.f
..
This can be proved by induction on k ; we have
c
1
z g(s2,z;t2,z2) x '** xgbk,zk-l;tk'Zk) z2, z3". .., Zk E v
l.
The k-th-order right signature
of x is the (r-1)2k-1 -tuple, H 04 (x;s 1' + ‘23 $2
=
’ ’ l , ‘k-1,
$1, “k)
-. Y+$, l -,Yk E r*
ho(x;slYYl) Xh(tlPYl;S2JY2) Xh(t2YY2;s3YY3)
x ‘*’ x h(tk,l’Yk,l;sk”Yk)
,
where all si,ti EQ' . As in the case of satisfy
G 04 , the numbers
-v. 04 0 < H (X;sl,tl,...,sk)
l/5 , contradicting'the fact that V'W" $! Ln
(thus Bl(v'w') = 1-p*(vw") 5 l/10 ).
Let d =
r
$ iog, n
Claim 4.3 leads to
I Ln
- F
1
0
. Then, for all sufficiently large n ,
> 2n- exp(32d3r4d In r) -2X2n > ' nrj w 1 -
1 = z
Lnl ’
Thus, for each 2n+l >
c iM(v) VE Ln
VE Ln
1 c rLn - r *n+l<j 0 ; thus one can
17
decide if wl = w2
with only a small chance of error by comparing
wl(mod p) with w,(mod p)
for a fixed number of such randoti primes p
generated by, for example, the method used in (a). These ideas imply that we can check the equation xxy E z Smith only a small chance of error by generating a few m-bit random primes
p , computing x(mod p) ,
y(mod p) and z(mod p) , and checking equations (x(mod p)*y(mod p))(mod p) = z(mod p) . The running time is dominated by the computing of x,y,z(mod p) , which takes 0 (iM(m))
time
(cf. [ll) 0 To end this section, we remark that the bound in the corollary to Theorem 4.1 is the best possible.
By a slight adaptation of Pippenger's
l-PTM for recognizing [wtiw') [7], one can construct a l-PTM recognizing rl-" # ln 1 n > 1) with a small error in time
O(n log log n) , thus
achieving the lower bound stated in the corollary.
18
6.
Conclusions. The subject of proving lower bounds for probabilistic Turing machines
offers many challenging problems, of which only one is solved in this paper.
It seems to be most fruitful to consider problems where good
bounds exist in the deterministic case.
We believe such studies will
provide insights to probabilistic computations beyond the framework of Turing machine models.
We mention only two such problems for further
research.
(i >
With a read-only input tape and several working tapes, is the extra space requirement for recognizing {w#w) probabilistically (with error) !&log n) ? (See [3, p. 154, Exercise 10.33 for the deterministic analogue.
( ii )
Can Rabin's language defined in [8] be recognized in real time by a probabilistic Turing machine with one working tape? (Deterministically it cannot [8].) Finally we like to mention that the overlap argument for on-line
multiplication (Cook and Aanderaa [2], also Paterson, Fischer, and e
Meyer [6]) can be extended to the probabilistic case [13].
Acknowledgments.
I wish to thank John Gill for a stimulating conversation
and for c ommunicating Pippenger's result [7] to me.
19
p
References
Dl
A. V. Aho, J. E. Hopcroft, and J. D. UUman, The Design and Analysis
PI
of Computer Algorithms, Addison-Wesley, Reading, Mass,, 1974. S. A. Cook and S. 0. Aanderaa, "On the minim computation time of functions," Trans. Amer. Math. Sot. 142 (1969), 291-314.
131
R. V. Freivald, "Fast computation by probabilistic Turing machines," Theory of Algorithms and Programs, no. 2, Latvian State University, Riga, 1975, 201-205 (in Russian).
i
VI
J. T. Gill III, "Computational complexity of probabilistic Turing machines,"
[51
SIAM
J. on Computing 6 (1917), 675-695.
J. E. Hopcroft and J. D, Ul&nan, Formal Languages and Their Relation to Automata, Addison-Wesley, Reading, Mass., 1969.
El
M. S. Paterson, M. J. Fischer, and A. R, Meyer, "An improved overlap argument for on-line multiplication," SIAM-AMS Proc., vol. 7,
171
Amer. Math. SOC., Providence, R.I., 1974, 97-111. N. Pippenger, private cammunication, November 1977.
I81
M. 0. Rabin, "Real-time computation," Israel J. Math. 1 (1963), 203-211.
L91
M. 0. Rabin, "Probabilistic algorithms," in Algorithms and Caaplexity: New Directions and Recent Results, J. F, Traub, ed., Academic Press, New York, 1976, 21-39,
DOI
R. Solovay and V. Strassen, "A fast Monte-Carlo test for primslity,"
D-U
J. on Computiq 6 (l-977), 84-85. E. T, Whittaker and G, N. Watson, A Course of Modern Analysis, 4th edition, Cmbridge University Press, 1958.
cm
A. C. Yao, "Probabilistic computations -- toward a unified measure
SIAM
- of complexity," Proc. 18th Annual Symp. on Foundations of Computer Science, 1977, 222-227. ' [13 1 A. C. Yao, unpublished.
20
