Borel Normality and Algorithmic Randomness - Department of ...
C. Calude. Borel normality and algorithmic randomness, in G.Rozenberg, A.Salomaa (eds.). Developments in Language Theory, World Scienti c, Singapore, 1994, 113-129.
Borel Normality and Algorithmic Randomness Cristian Calude
y
Abstract
We prove that all random sequences (in Chaitin-Martin-Lof sense) and almost all random strings (in both Kolmogorov-Chaitin and Chaitin senses) satisfy various conditions of normality ( rst introduced by Borel). All proofs are constructive. Keywords: Random strings, random sequences, Borel normality
1 Introduction Borel ([1, 2]) introduced the normality property for reals and sequences (see also [21, 24]) and was the rst author to explicitly study randomness (see [18] for an historical overview). Although Borel's de nition of randomness is not satisfactory (e.g. there exist recursive sequences satisfying Borel's de nition of randomness; the rst such example was discovered in [15]) his criterion of normality has been proved to be extremely useful in various areas of mathematics (see, for instance, [24, 21, 31]). Also, it is now clear that every 'true random' sequence should be Borel normal (see, for instance, the discussion in [35]). Working in a non necessarily binary framework we prove that every random sequence in Chaitin-Martin-Lof's sense (see [29]) is Borel normal and that almost all random strings (in both Kolmogorov-Chaitin and Chaitin senses) do satisfy a Borel normality-like property. In this way we extend results obtained in [9, 8, 11, 34] (for sequences) and [28, 23] (for strings); related results are proved in [26]. All proofs are constructive. A large part of this paper has been done during the visits of the author at Turku University, Finland (Spring, 1991) and The University of Western Ontario, London, Canada (Fall, 1992). A preliminary version of this paper has been presented at the Second International Colloquium on Words, Languages and Combinatorics, Kyoto, Japan, August 1992. The work has been supported by Auckland University Research Grant A18/XXXXX/62090/3414012. y Computer Science Department, The University of Auckland, Private Bag 92109, Auckland, New Zealand; email: c [email protected].
The paper is organised as follows. Some notation and basic notions are introduced in Section 2. In Section 3 one discusses random sequences, while in Section 4 one studies random strings. The main results are announced in Theorem 3.4, Theorem 4.2, Theorem 4.3 and Theorem 4.4.
2 Prerequisites
Denote by N and R, respectively, the sets of natural numbers and real numbers; N+ = N n f0g and R+ = fx 2 R j x 0g. If S is a nite set, then #S denotes the cardinality of S . We shall use the following functions : i) rem(m; i), the remainder of the ,integral division of m by i(m; i 2 N+), ii) [ ], the integral part of the real , iii) nk , the binomial coecient, iv) logQ , the base Q logarithm, ln, the natural logarithm and log = [log2 ]. By j we denote the divisibility predicate. Fix A = fa1 ; : : : ; aQ g; Q 2, a nite alphabet. By A we denote the free monoid generated by A (under concatenation). The elements of A are called strings; is the empty string. For x in A ; jxj is the length of x(jj = 0). Every ordering on A, say a1 < a2 < < aQ , induces a quasi-lexicographical order on A : < a1 < < aQ < a1 a1 < < a1 aQ < < aQ aQ < < a1 a1 a1 < . For m in N, Am = fx 2 A j jxj = mg. In case m 1 we consider the alphabet B = Am and construct the free monoid B = (Am ) . Every x 2 B belongs to A , but the converse is false. For x 2 B we denote by jxjm the length of x (according to B ) which is exactly m,1 jxj. By A1 we denote the set of all sequences x = x1 x2 : : : xn : : : with elements
xi in A. The set A1 is no longer a monoid, but it comes equipped with an interesting probabilistic structure, which will be used in Section 3. For x 2 A1 and n 2 N+ ; put x(n) = x1 : : : xn 2 A . For S A ; SA1 = fx 2 A1 j x(n) 2 S; for some natural n 1g; xA1 = fxgA1 ; x 2 Ao. We shall work with partial recursive (p.r.) functions ' : A ! A (we adopt the notations from [3]). For such a ' one associates the Kolmogorovo N; K (x) = minfjy j j y 2 Chaitin (blank-endmarker) complexity K' : A ! ' A ; '(y) = xg, with the convention min = 1. In case the domain of ' is pre x-free (i.e. there is no pair of distinct strings x; y in the domain of ' such that x is a pre x of y), then ' is called a Chaitin computer ; in such a case we write H' = K' (H' is called the Chaitin's complexity induced by '). The basic result obtained independently by Kolmogorov and Chaitin (called the Invariance Theorem) states the existence of a p.r. function (called universal computer) such that for every p.r. function ' there exists a constant c (depending upon and ') such that K (x) K'(x)+ c, for all x 2 A . The same result holds true for Chaitin's complexity, [9, 8, 11]. For this paper we x a universal computer and a universal Chaitin computer and denote by K and H , respectively, the induced complexities.
3 The in nite case We study the Borel normality of the (in nite) random sequences over a nite alphabet A having Q 2 elements. Borel was working with the interval [0; 1] endowed with Lebesgue measure; a criterion, equivalent to the one described in De nition 3.1, de nes the 'normal numbers'; his main result states that almost all real numbers in [0; 1] are normal (see [1, 2]). For every 1 i Q denote by Ni (x) the number of occurrences of the letter ai in the string x 2 A (in other words, Ni : A ! N is the unique morphism of monoids - where N comes equipped with the addition - acting on the generators as follows: Ni (ai ) = 1 and Ni (aj ) = 0, for j 6= i). Accordingly, the ratio Ni (x)=jxj is the relative frequency of the letter ai in the string x. Fix now an integer m > 1 and consider the alphabet B = Am (#B = m Q ). For every 1 i Qm denote by Nim the integer-valued function de ned on B by Nim (x) = the number of occurrences of yi in the string x 2 B ; B = fy1; : : : ; yQm g. For example, take A = f0; 1g; m = 2; B = A2 = f00; 01; 10; 11g = fy1 ; y2; y3 ; y4 g; x = y1 y3 y3 y4 y3 2 B (x = 0010101110 2 A ). It is easy to see that jxj2 = 5; jxj = 10; N12(x) = 1; N22(x) = 0; N32(x) = 3; N42(x) = 1. Note that the string y2 = 01 appears three times into x, but not on the right positions. Not every string x 2 A belongs to B . However, there is a possibility "to approximate" such a string by a string in B . We proceed as follows. For x 2 A and 1 j jxj we denote by [x; j ] the pre x of x of length jxj, rem(jxj; j ) (i.e. [x; j ] is the longest pre x of x whose length is divisible by j ). Clearly, [x; 1] = x and [x; j ] 2 (Aj ) . We are now in a position to extend the functions Nim from B to A : put Nim (x) = Nim ([x; m]), in case jxj is not divisible by m. Similarly, jxjm = j[x; m]jm . For x 2 A1 and n 1, x(n) = x1 x2 : : : xn 2 A , so Ni (x(n)) counts the number of occurrences of the letter ai in the pre x of length n of x .
De nition 3.1.
a) The sequence x is called Borel m-normal (m 1) in case for every 1 i Qm one has: Nim (x(n)) = Q,m: lim n] n!1 [ m b) The sequence x is called Borel normal if it is Borel m-normal, for every natural m 1.
Remark. In case m = 1, the property of Borel 1-normality can be written lim Ni (x(n)) = Q,1 ;
in the form
n for every 1 i Q. It corresponds to the Law of Large Numbers (see [19, 35]). n!1
We start with some preliminary results. i i PQLetxiQ=1,2forandall(xnn)n1.1; 1 i Q, be Q sequences such that xn 0 and i=1 n
Lemma 3.2. The following assertions are equivalent: i) For all 1 i Q, lim infn xin = Q,1, ii) For all 1 i Q, lim infn xin Q,1.
Proof. Suppose, by absurdity, that lim infn xin > Q,1 , for some 1 i Q. One has: Q X 1 = limninf(x1n + x2n + + xQn ) limninf xjn > 1; j =1 a contradiction. 2 Lemma 3.3. If for every 1 i Q, lim infn xin = Q,1, then for all 1 i Q, limn!1 xin = Q,1 . Proof. Assume, by absurdity, that lim inf n xin 6= lim supn xin , for some 1 i Q, i.e. there exists a > 0 such that lim supn xin = Q,1 + . Since lim infn (,xin ) = , lim supn xin it follows that
limninf(1 , xin ) = 1 + limninf(,xin )
On the other hand,
= 1 , lim sup xin = Q Q, 1 , : n
0 Q 1 X jA limninf(1 , xin ) = limninf @ xn j =1;j 6=i
= a contradiction.
Q X j =1;j 6=i
limninf xjn = Q Q, 1 > Q Q, 1 , ;
2
First we are dealing with the case m = 1. For every sequence x 2 A1 we consider the sequences Ni(x(n) ; i = 1; : : : ; Q n n1 which satisfy the conditions in Lemma 3.2 and Lemma 3.3. So, in order to prove that lim N (x(n)) = Q,1 ; n!1 i
whenever x is random, it suces to show that lim infn Ni (xn(n)) Q,1 , for every 1 i Q. Assume, by absurdity, that there exists an i; 1 i Q, such that limninf Ni (xn(n)) < Q,1 : An elementary reasoning shows that the set fn 1 j Q1 , Ni (xn(n)) > "g is in nite, for some rational, small enough " > 0. Consider now the recursive set S A N+ : S = f(y; n) j y 2 An ; n 1; Q1 , Nin(y) > "g: (1) Clearly, x 2 Sn A1 , for in nitely many n (here Sn = fy 2 A j (y; n) 2 S g). We are using now the following charaterization of random sequences in ChaitinMartin-Lof sense due to Solovay (see [9]): Theorem 3.4. A Psequence x 2 A1 is random i for every r.e. set S A1 N+ such that n1 (Sn A1 ) < 1, there exists a natural N (depending upon x and S ) such that x 62 Sn A1 , for all n > N . Here is the product measure on A1 induced by the unbiased discrete measure on A, h(R) = #QR , for every R A. It is seen that is a probabilistic measure de ned on all Borel subsets of A1 and (xA1 ) = Q,jxj; for all x 2 A . Furthermore, x 2 Sn A1 (where Sn A ) means that x has a pre x in Sn , i.e. x(m) 2 Sn, for some m 1. (A detailed construction appears in [4].) Returning to our proof, it is clear that all it remains to show at this stage reduces to the convergence of the series
X
n1
(Sn A1 );
when S comes from (1). A combinatorial argument (Sn An ) shows that n X (Sn A1 ) = Q,n : (Q , 1)n,k k 0k 0 and m 1. a) We say that a non-empty string x 2 A is ("; m)-limiting if
for every 1 i Qm.
N m(x) i , Q,m "; jxjm
b) A non-empty string x 2 A is called Borel ("; m)-normal if x is ("; j )-limiting, for every 1 j m.
De nition 4.3.
i) A non-empty string x 2 limiting, i.e.
A is called m-limiting if x is
q log
N m(x) s log jxj Q i , Q,m jxjm jxj ;
Q jxj ; m
jxj
-
for every 1 i Qm. ii) If for every natural m; 1 m logQ logQ jxj; x is m-limiting, then we say that x is Borel normal.
The main result of this section states that almost all random strings are Borel normal (and, consequently, Borel ("; m)-normal).
Fact 4.4 ([32]). For all naturals i; m 0 and real x > 0 m i X
(k , ix)2 xk (1 , x)i,k = ix(1 , x): k k=0
Lemma 4.5. For every " > 0; 1 m M and 1 i Qm : N m(x) M ,2m (Qm , 1) M i , m #fx 2 A j jxj , Q > "g Q "2 [M=m ] : m Proof. In (4) put x = Q,m ; i = [M=m]: 2 [M=m X ] [M=m] k ,m [M=m]2 (Qm , 1)[M=m],k , Q [M=m] k k=0
= [M=m]Qm[M=m],2m(Qm , 1):
On one hand:
i (x) , Q,m > "g #fx 2 AM j N[M=m ]
=
m
[M=m X]
#fx 2 AM jNim (x) = kg k ,Q,m >" k=0; M=m [
]
(4)
(5)
[M=m X]
= Qrem(M;m): =
M
m
#fx 2 A jNi (x) = kg k , m k=0; M=m ,Q >" [M=m] [M=m X] rem ( M;m ) (Qm , 1)[M=m],k : Q : k k ,Q,m >" k=0; M=m [
]
[
]
On the other hand: [M=m]Qm[M=m],2m(Qm , 1)
[M=m X]
k ,Q,m >" k=0; [M=m ]
] (Qm , 1)[M=m],k "2 [M=m]2 [M=m k
m = "2 [M=m]2 Q,rem(M;m)#fx 2 AM j Ni (x) , Q,m > "g: jxjm
Remark. For every 1 m M; 1 i Qm:
m #fx 2 AM j Njixj(x) m
r log M MQM ,2m(Qm , 1) Q M g [M=m] logQ M :
, Q,m >
2 (6)
Comment. In case m = 1 and 1 i Q; Ni1(x) = Ni(x), formula (5) becomes M ,2 #fx 2 AM j Ni (x) , Q,1 > "g Q (Q , 1) ; and the inequality (6)
M
M
M"2
r log M QM ,2(Q , 1) Q M g logQ M :
#fx 2 AM j Ni (x) , Q,1 >
In view of De nition 4.3, a string x 2 AM is not Borel normal in case N m(x) r i , Q,m > logQ M ; [M=m] M for some 1 m logQ logQ M; 1 i Qm . Lemma 4.6. We can e ectively compute a natural N such that for all naturals M N M #fx 2 AM jx is not Borel normal g p Q : (7) logQ M
Proof. Using formula (6) we perform the following computation:
#fx 2 AM jx is not Borel normalg
X
Qm X MQM ,2m (Qm , 1)
1mlogQ logQ M;m2N i=1
X
[M=m] logQ M
MQM ,m(Qm , 1) 1mlogQ logQ M;m2N [M=m] logQ M M X m2 (1 , Q,m ) logQ M Q 1mlogQ logQ M;m2N M logQ M (logQ logQ M )3 Q M Q plog M ; Q for suciently large M .
2
Corollary 4.7. There exists a natural N (which can be e ectively computed) such that for all M N one has M +3 #fx 2 A jN jxj M; x is not Borel normal g pQ : logQ M
(8)
Proof. By Lemma 4.6 we get a bound N for which the inequality (7) is true. Accordingly,
#fx 2 A jN jxj M; x is not Borel normal g M +3
pQlog M : Q
(the last inequality can be proved by induction on M .)
M X Qi plog Qi i=N
2
Theorem 4.8. We can e ectively nd two natural constants c and M such that every x 2 A with jxj M and which is not Borel normal satis es the inequality: (9) K (x) jxj , 12 logQ logQ jxj + c:
Proof. Consider the (primitive) recursive bijection string : N ! A given by string(n) = the n-th string according to the quasi-lexicographic order on A . Next de ne the recursive function f : N+ ! A by f (t) = the t , th string x (according to the same quasi-lexicographical order) which is not Borel normal and has the length greater than N . (N comes from Corollary 4.7.) In view of (8), jxj+3 t qQ ; (10) logQ jxj
provided f (x) = t. Finally, de ne the p.r. function : A !o A by
(u) = f (string,1(u)) (11) and consider b the constant comming from the Invariance Theorem (applied to and ). If x = f (t), then (string(t)) = f (t) = x and (by (10)): K (x) K (x) + b jstring(t)j + b logQ (t + 1) + b 0 1 jxj+3 Q logQ @ q + 1A + b jxj , 21 logQ logQ jxj + c: logQ jxj 2 Theorem 4.9. For every natural t 0 we can e ectively compute a natural number Mt (depending upon t) such that every Kolmogorov-Chaitin t-random string of length greater than Mt is Borel normal. Proof. Let c, M be the constants comming from Theorem 4.8 and put
Mt = max M; QQ
t c
2( + )
:
If x is not Borel normal, then x satis es the inequality (9) (Mt N ) : K (x) jxj , 12 logQ logQ jxj + c < jxj , 12 logQ logQ Mt + c jxj , t; a contradiction.
2
Corollary 4.10. Almost all Kolmgorov t-random strings are m-limiting, Borel ("; m)-normal and "-limiting.
Proof. Every Kolmgorov t-random string x which is m-limiting and satis es the inequality logQ jxj "2 jxj is also Borel ("; m)-normal. 2
As a direct consequence we get the following result proved by Kramosil [23]:
Corollary 4.11. Let (xn )n1 be a sequence of Kolmgorov t-random strings such that jxn j = n. Then, for every natural n 1 Nim(xn ) = Q,m ; lim n!1 [n=m] for every 1 i Qm.
Remark. Due to a classical result of Chaitin (see Section 10 in [7] and Theorem B in [9]) (for an alternative approach see Martin-Lof [30, 5], Calude, Chitescu [5]) there is no sequence x 2 A whose pre xes are all KolmogorovChaitin t-random. We nish this section with an analysis of random strings in Chaitin's sense. Chaitin [13] pointed out that if K dips t below its maximum, then H must dip t,O(log t), i.e. H (x) < jxj+H (string(jxj)),t+O(log t) provided K (x) < jxj,t: This shows that every maximally H complex string must be also a maximally K complex string. To state this result we needo some more tools. Consider Chaitin's computers C : A A ! oA such that for every v 2 A , the domain of Cv is pre x-free, where Cv : A ! A ; Cv (x) = C (x; v), for all x 2 A . Fix a universal Chaitin's computer U : A A !o A . Chaitin's conditional complexity induced by C is de ned by HC (x=v) = minfjyj j y 2 A ; C (y; v ) = xg; where
v = minfu 2 A jU (u; ) = vg is the canonical program generating v. Finally put HC (x; y) = HC (< x; y >), where : A A ! A is a pairing function.
Theorem 4.12. (Chaitin [14]) For all x 2 A and t 2 N, if K (x) < jxj, t, then H (x) < jxj + H (string(jxj)) , t + O(logQ t): Proof. We start noticing that the set f< x; t >2 A NjK (x) < jxj, tg is r.e., thus if K (x) < jxj , t, then we can eventually discover this. Moreover, there are less than Qn,t =(Q , 1) strings of length n having this property. Thus if we are given jxj = n and t we need to only know n , t digits to pick out any particular string x 2 An with this property. I.e., as the rst x that we discover has this property, the second x that we discover has this property,..., the ith x that we discover has this property, and i < Qn,t =(Q , 1). It follows that any x 2 An that satis es the inequality
K (x) < jxj , t
has the property that
H (x= < string(n); string(t) >) < n , t + O(1): So, by the relation (for a proof see Theorem I1 in Chaitin [9]):
H (x) H (y) + H (x=y) + O(1) we get
H (x) < H (x= < string(n); string(t) >) + H (< string(n); string(t) >) + O(1) < n , t + H (string(n); string(t)) + O(1) < n , t + H (string(n)) + H (string(t) + O(1) < n , t + H (string(n)) + O(logQ t); since in general H (string(m)) < O(logQ m):
2
Corollary 4.13. For every t 2 N and every Chaitin t-random string x, one has K (x) jxj , T , whenever T , O(logQ T ) t. Proof. Fix t 2 N and pick x such that H (x) (jxj) , t. If K (x) < jxj , T , then by Theorem 4.12
H (x) < (jxj) , T , O(logQ T ); which means T , O(logQ T ) < t.
2
In view of Corollary 4.13, Theorem 4.9 applies and we get:
Theorem 4.14. For every natural t 0 we can e ectively nd a natural Mt (depending upon t) such that every Chaitin's t-random string x with jxj Mt is Borel normal.
Acknowledgement Arto Salomaa drew our attention to the paper [28] and suggested the study of normality for random strings. Greg Chaitin kindly permitted to insert his Theorem 4.12 (proved twenty years ago, but never published) and suggested Theorem 4.14. Ion Chitescu has communicated us Lemma 3.2. Helmut Jurgensen gave valuable comments on the manuscript. After completion of the paper, Ming Li pointed out the related results from [26]. I express them all my gratitude.
References
[1] E . Borel. Les probabilites denombrables et leurs applications arithmetiques, Rend. Circ. Mat. Palermo 27(1909), 247-271. [2] E . Borel. Lecons sur la theorie des fonctions, Gauthier-Villars, Paris, 2nd ed., 1914. [3] C. Calude. Theories of Computational Complexity, North-Holland, Amsterdam, 1988. [4] C. Calude. Information and Randomness: An Algorithmic Perspective, Springer-Verlag (in press; draft version: Elementary Algorithmic Information Theory, Bucharest University, 1991, 90 pp). [5] C. Calude, I. Chitescu. Upper limitation of Kolmogorov complexity and universal P.Martin-Lof tests, J. Computational Math. 7(1989), 61-70. [6] C. Calude, I. Chitescu. Qualitative properties of P.Martin-Lof random sequences, Boll. Unione Mat. Ital. (7) 3 - B(1989), 229-240. [7] G. J. Chaitin. On the length of programs for computing nite binary sequences, J. Assoc. Comput.Mach. 13(1966),547-569. (Reprinted in: Chaitin [10], 369-410.) [8] G. J. Chaitin. On the length of programs for computing nite binary sequences : statistical considerations, J. Assoc. Comput. Mach. 16 (1969), 145-159. (Reprinted in [10], 245-260.) [9] G. J. Chaitin. Algorithmic Information Theory, Cambridge University Press, Cambridge,1987. (third printing 1990) [10] G. J. Chaitin. Information, Randomness and Incompleteness, Papers on Algorithmic Information Theory, World Scienti c, Singapore, 2nd ed., 1990. [11] G. J. Chaitin. Information-Theoretic Incompleteness, World Scienti c, Singapore, 1992. [12] G. J. Chaitin.Randomness in Arithmetic and the Decline and Fall of Reductionism in Pure Mathematics, IBM Research Report RC-18532 (November 1992), 26 pp. [13] G. J. Chaitin. Electronic Mail to C.Calude, December 12, 1992. [14] G. J. Chaitin. Electronic Mail to C.Calude, March 25, 1993. [15] D. G. Champernowne. The construction of decimals normal in the scale of ten, The Journal of London Mathematical Society 8 (1933), 254-260.
[16] A. H. Copeland, P. Erdos. Note on normal numbers, Bull. Amer. Math. Soc. 52(1946), 857-860. [17] T. M. Cover, P. Gacs, R. M. Gray. Kolmogorov's contributions to information theory and algorithmic complexity, The Annals of Probability 17(1989), 840-855. [18] C. Dellacherie. Nombres au hazard. De Borel a Martin-Loef, Gazette des Math., Soc. Math. France 11(1978), 23-58. [19] W. Feller. An Introduction to Probability Theory and Its Aplications, Vol. 1, Chapman & Hall, London, John Wiley & Sons, New York, 2nd ed., 1958. [20] G. H. Hardy, E. M. Wright. An Introduction to the Theory of Numbers, Clarendon Press, Oxford, 5th ed., 1979. [21] D. E. Knuth. The Art of Computer Programming, Vol. 2, Seminumerical Algorithms, Addison-Wesley, Reading, Ma., 2nd ed., 1981. [22] A. N. Kolmogorov. Three approaches for de ning the concept of \information quantity", Problemy Peredachi Informatsii 1(1965), 3-11. [23] I. Kramosil. In nite pseudo-random sequences of high algorithmic complexity, Kybernetika (Prague) 20(1984), 429-437. [24] L. Kuipers, H. Niederreiter. Uniform Distribution of Sequences, JohnWiley & Sons, New York, 1974. [25] M. Li, P. M. Vitanyi. Kolmogorov complexity and its applications, in J. van Leeuwen (ed.). Handbook of Theoretical Computer Science, Vol. A, North-Holland, Amsterdam, MIT Press, Boston, 1990, 187-254. [26] M. Li, P. M. Vitanyi. Combinatorics and Kolmogorov complexity, Proc. 6th IEEE Structure in Complexity Theory Conference, 1991, 154-163. [27] M. Li, P. M. Vitanyi. An Introduction to Kolmogorov Complexity and Its Applications, Report, May 1992, 593 pp. [28] H. B. Marandijan. Selected topics in recursive function theory in computer science, ID - TR: 1990 - 75, Dept. Comput. Sci., Technical University of Denmark,Lyngby, 1990, 93 pp. [29] P. Martin-Lof. The de nition of random sequences, Inform. and Control 6(1966), 602-619. [30] P. Martin-Lof. Complexity oscillations in in nite binary sequences, Zeitschrift fur Wahrscheinlichkeitstheorie und Verwantdte Gebiete 19(1971), 225-230.
[31] M. Mendes France. Suites de nombres au hasard (d'apres Knuth), Seminaire de Theorie des Nombres, Expose 6, 1974, 1-11. [32] I. P. Natanson. Theory of Functions of A Real Variable, Frederick Ungar Publishing Co., New York, 1955. [33] I. Niven, H. S. Zuckerman. On the de nition of normal numbers, Paci c J.Math. 1(1951), 103-110. [34] C. P. Schnorr. Zufalligkeit und Wahrscheinlickeit, Lecture Notes Math. Vol. 218, Springer - Verlag, Berlin, 1981. [35] R. von Mises. Probability, Statistics and Truth, G.Allen and Unwin Ltd., London, Macmillan, New York, 2nd Revised English Edition Prepared by Hilda Geiringer, 1961.
Borel Normality and Algorithmic Randomness Cristian Calude
y
Abstract
We prove that all random sequences (in Chaitin-Martin-Lof sense) and almost all random strings (in both Kolmogorov-Chaitin and Chaitin senses) satisfy various conditions of normality ( rst introduced by Borel). All proofs are constructive. Keywords: Random strings, random sequences, Borel normality
1 Introduction Borel ([1, 2]) introduced the normality property for reals and sequences (see also [21, 24]) and was the rst author to explicitly study randomness (see [18] for an historical overview). Although Borel's de nition of randomness is not satisfactory (e.g. there exist recursive sequences satisfying Borel's de nition of randomness; the rst such example was discovered in [15]) his criterion of normality has been proved to be extremely useful in various areas of mathematics (see, for instance, [24, 21, 31]). Also, it is now clear that every 'true random' sequence should be Borel normal (see, for instance, the discussion in [35]). Working in a non necessarily binary framework we prove that every random sequence in Chaitin-Martin-Lof's sense (see [29]) is Borel normal and that almost all random strings (in both Kolmogorov-Chaitin and Chaitin senses) do satisfy a Borel normality-like property. In this way we extend results obtained in [9, 8, 11, 34] (for sequences) and [28, 23] (for strings); related results are proved in [26]. All proofs are constructive. A large part of this paper has been done during the visits of the author at Turku University, Finland (Spring, 1991) and The University of Western Ontario, London, Canada (Fall, 1992). A preliminary version of this paper has been presented at the Second International Colloquium on Words, Languages and Combinatorics, Kyoto, Japan, August 1992. The work has been supported by Auckland University Research Grant A18/XXXXX/62090/3414012. y Computer Science Department, The University of Auckland, Private Bag 92109, Auckland, New Zealand; email: c [email protected].
The paper is organised as follows. Some notation and basic notions are introduced in Section 2. In Section 3 one discusses random sequences, while in Section 4 one studies random strings. The main results are announced in Theorem 3.4, Theorem 4.2, Theorem 4.3 and Theorem 4.4.
2 Prerequisites
Denote by N and R, respectively, the sets of natural numbers and real numbers; N+ = N n f0g and R+ = fx 2 R j x 0g. If S is a nite set, then #S denotes the cardinality of S . We shall use the following functions : i) rem(m; i), the remainder of the ,integral division of m by i(m; i 2 N+), ii) [ ], the integral part of the real , iii) nk , the binomial coecient, iv) logQ , the base Q logarithm, ln, the natural logarithm and log = [log2 ]. By j we denote the divisibility predicate. Fix A = fa1 ; : : : ; aQ g; Q 2, a nite alphabet. By A we denote the free monoid generated by A (under concatenation). The elements of A are called strings; is the empty string. For x in A ; jxj is the length of x(jj = 0). Every ordering on A, say a1 < a2 < < aQ , induces a quasi-lexicographical order on A : < a1 < < aQ < a1 a1 < < a1 aQ < < aQ aQ < < a1 a1 a1 < . For m in N, Am = fx 2 A j jxj = mg. In case m 1 we consider the alphabet B = Am and construct the free monoid B = (Am ) . Every x 2 B belongs to A , but the converse is false. For x 2 B we denote by jxjm the length of x (according to B ) which is exactly m,1 jxj. By A1 we denote the set of all sequences x = x1 x2 : : : xn : : : with elements
xi in A. The set A1 is no longer a monoid, but it comes equipped with an interesting probabilistic structure, which will be used in Section 3. For x 2 A1 and n 2 N+ ; put x(n) = x1 : : : xn 2 A . For S A ; SA1 = fx 2 A1 j x(n) 2 S; for some natural n 1g; xA1 = fxgA1 ; x 2 Ao. We shall work with partial recursive (p.r.) functions ' : A ! A (we adopt the notations from [3]). For such a ' one associates the Kolmogorovo N; K (x) = minfjy j j y 2 Chaitin (blank-endmarker) complexity K' : A ! ' A ; '(y) = xg, with the convention min = 1. In case the domain of ' is pre x-free (i.e. there is no pair of distinct strings x; y in the domain of ' such that x is a pre x of y), then ' is called a Chaitin computer ; in such a case we write H' = K' (H' is called the Chaitin's complexity induced by '). The basic result obtained independently by Kolmogorov and Chaitin (called the Invariance Theorem) states the existence of a p.r. function (called universal computer) such that for every p.r. function ' there exists a constant c (depending upon and ') such that K (x) K'(x)+ c, for all x 2 A . The same result holds true for Chaitin's complexity, [9, 8, 11]. For this paper we x a universal computer and a universal Chaitin computer and denote by K and H , respectively, the induced complexities.
3 The in nite case We study the Borel normality of the (in nite) random sequences over a nite alphabet A having Q 2 elements. Borel was working with the interval [0; 1] endowed with Lebesgue measure; a criterion, equivalent to the one described in De nition 3.1, de nes the 'normal numbers'; his main result states that almost all real numbers in [0; 1] are normal (see [1, 2]). For every 1 i Q denote by Ni (x) the number of occurrences of the letter ai in the string x 2 A (in other words, Ni : A ! N is the unique morphism of monoids - where N comes equipped with the addition - acting on the generators as follows: Ni (ai ) = 1 and Ni (aj ) = 0, for j 6= i). Accordingly, the ratio Ni (x)=jxj is the relative frequency of the letter ai in the string x. Fix now an integer m > 1 and consider the alphabet B = Am (#B = m Q ). For every 1 i Qm denote by Nim the integer-valued function de ned on B by Nim (x) = the number of occurrences of yi in the string x 2 B ; B = fy1; : : : ; yQm g. For example, take A = f0; 1g; m = 2; B = A2 = f00; 01; 10; 11g = fy1 ; y2; y3 ; y4 g; x = y1 y3 y3 y4 y3 2 B (x = 0010101110 2 A ). It is easy to see that jxj2 = 5; jxj = 10; N12(x) = 1; N22(x) = 0; N32(x) = 3; N42(x) = 1. Note that the string y2 = 01 appears three times into x, but not on the right positions. Not every string x 2 A belongs to B . However, there is a possibility "to approximate" such a string by a string in B . We proceed as follows. For x 2 A and 1 j jxj we denote by [x; j ] the pre x of x of length jxj, rem(jxj; j ) (i.e. [x; j ] is the longest pre x of x whose length is divisible by j ). Clearly, [x; 1] = x and [x; j ] 2 (Aj ) . We are now in a position to extend the functions Nim from B to A : put Nim (x) = Nim ([x; m]), in case jxj is not divisible by m. Similarly, jxjm = j[x; m]jm . For x 2 A1 and n 1, x(n) = x1 x2 : : : xn 2 A , so Ni (x(n)) counts the number of occurrences of the letter ai in the pre x of length n of x .
De nition 3.1.
a) The sequence x is called Borel m-normal (m 1) in case for every 1 i Qm one has: Nim (x(n)) = Q,m: lim n] n!1 [ m b) The sequence x is called Borel normal if it is Borel m-normal, for every natural m 1.
Remark. In case m = 1, the property of Borel 1-normality can be written lim Ni (x(n)) = Q,1 ;
in the form
n for every 1 i Q. It corresponds to the Law of Large Numbers (see [19, 35]). n!1
We start with some preliminary results. i i PQLetxiQ=1,2forandall(xnn)n1.1; 1 i Q, be Q sequences such that xn 0 and i=1 n
Lemma 3.2. The following assertions are equivalent: i) For all 1 i Q, lim infn xin = Q,1, ii) For all 1 i Q, lim infn xin Q,1.
Proof. Suppose, by absurdity, that lim infn xin > Q,1 , for some 1 i Q. One has: Q X 1 = limninf(x1n + x2n + + xQn ) limninf xjn > 1; j =1 a contradiction. 2 Lemma 3.3. If for every 1 i Q, lim infn xin = Q,1, then for all 1 i Q, limn!1 xin = Q,1 . Proof. Assume, by absurdity, that lim inf n xin 6= lim supn xin , for some 1 i Q, i.e. there exists a > 0 such that lim supn xin = Q,1 + . Since lim infn (,xin ) = , lim supn xin it follows that
limninf(1 , xin ) = 1 + limninf(,xin )
On the other hand,
= 1 , lim sup xin = Q Q, 1 , : n
0 Q 1 X jA limninf(1 , xin ) = limninf @ xn j =1;j 6=i
= a contradiction.
Q X j =1;j 6=i
limninf xjn = Q Q, 1 > Q Q, 1 , ;
2
First we are dealing with the case m = 1. For every sequence x 2 A1 we consider the sequences Ni(x(n) ; i = 1; : : : ; Q n n1 which satisfy the conditions in Lemma 3.2 and Lemma 3.3. So, in order to prove that lim N (x(n)) = Q,1 ; n!1 i
whenever x is random, it suces to show that lim infn Ni (xn(n)) Q,1 , for every 1 i Q. Assume, by absurdity, that there exists an i; 1 i Q, such that limninf Ni (xn(n)) < Q,1 : An elementary reasoning shows that the set fn 1 j Q1 , Ni (xn(n)) > "g is in nite, for some rational, small enough " > 0. Consider now the recursive set S A N+ : S = f(y; n) j y 2 An ; n 1; Q1 , Nin(y) > "g: (1) Clearly, x 2 Sn A1 , for in nitely many n (here Sn = fy 2 A j (y; n) 2 S g). We are using now the following charaterization of random sequences in ChaitinMartin-Lof sense due to Solovay (see [9]): Theorem 3.4. A Psequence x 2 A1 is random i for every r.e. set S A1 N+ such that n1 (Sn A1 ) < 1, there exists a natural N (depending upon x and S ) such that x 62 Sn A1 , for all n > N . Here is the product measure on A1 induced by the unbiased discrete measure on A, h(R) = #QR , for every R A. It is seen that is a probabilistic measure de ned on all Borel subsets of A1 and (xA1 ) = Q,jxj; for all x 2 A . Furthermore, x 2 Sn A1 (where Sn A ) means that x has a pre x in Sn , i.e. x(m) 2 Sn, for some m 1. (A detailed construction appears in [4].) Returning to our proof, it is clear that all it remains to show at this stage reduces to the convergence of the series
X
n1
(Sn A1 );
when S comes from (1). A combinatorial argument (Sn An ) shows that n X (Sn A1 ) = Q,n : (Q , 1)n,k k 0k 0 and m 1. a) We say that a non-empty string x 2 A is ("; m)-limiting if
for every 1 i Qm.
N m(x) i , Q,m "; jxjm
b) A non-empty string x 2 A is called Borel ("; m)-normal if x is ("; j )-limiting, for every 1 j m.
De nition 4.3.
i) A non-empty string x 2 limiting, i.e.
A is called m-limiting if x is
q log
N m(x) s log jxj Q i , Q,m jxjm jxj ;
Q jxj ; m
jxj
-
for every 1 i Qm. ii) If for every natural m; 1 m logQ logQ jxj; x is m-limiting, then we say that x is Borel normal.
The main result of this section states that almost all random strings are Borel normal (and, consequently, Borel ("; m)-normal).
Fact 4.4 ([32]). For all naturals i; m 0 and real x > 0 m i X
(k , ix)2 xk (1 , x)i,k = ix(1 , x): k k=0
Lemma 4.5. For every " > 0; 1 m M and 1 i Qm : N m(x) M ,2m (Qm , 1) M i , m #fx 2 A j jxj , Q > "g Q "2 [M=m ] : m Proof. In (4) put x = Q,m ; i = [M=m]: 2 [M=m X ] [M=m] k ,m [M=m]2 (Qm , 1)[M=m],k , Q [M=m] k k=0
= [M=m]Qm[M=m],2m(Qm , 1):
On one hand:
i (x) , Q,m > "g #fx 2 AM j N[M=m ]
=
m
[M=m X]
#fx 2 AM jNim (x) = kg k ,Q,m >" k=0; M=m [
]
(4)
(5)
[M=m X]
= Qrem(M;m): =
M
m
#fx 2 A jNi (x) = kg k , m k=0; M=m ,Q >" [M=m] [M=m X] rem ( M;m ) (Qm , 1)[M=m],k : Q : k k ,Q,m >" k=0; M=m [
]
[
]
On the other hand: [M=m]Qm[M=m],2m(Qm , 1)
[M=m X]
k ,Q,m >" k=0; [M=m ]
] (Qm , 1)[M=m],k "2 [M=m]2 [M=m k
m = "2 [M=m]2 Q,rem(M;m)#fx 2 AM j Ni (x) , Q,m > "g: jxjm
Remark. For every 1 m M; 1 i Qm:
m #fx 2 AM j Njixj(x) m
r log M MQM ,2m(Qm , 1) Q M g [M=m] logQ M :
, Q,m >
2 (6)
Comment. In case m = 1 and 1 i Q; Ni1(x) = Ni(x), formula (5) becomes M ,2 #fx 2 AM j Ni (x) , Q,1 > "g Q (Q , 1) ; and the inequality (6)
M
M
M"2
r log M QM ,2(Q , 1) Q M g logQ M :
#fx 2 AM j Ni (x) , Q,1 >
In view of De nition 4.3, a string x 2 AM is not Borel normal in case N m(x) r i , Q,m > logQ M ; [M=m] M for some 1 m logQ logQ M; 1 i Qm . Lemma 4.6. We can e ectively compute a natural N such that for all naturals M N M #fx 2 AM jx is not Borel normal g p Q : (7) logQ M
Proof. Using formula (6) we perform the following computation:
#fx 2 AM jx is not Borel normalg
X
Qm X MQM ,2m (Qm , 1)
1mlogQ logQ M;m2N i=1
X
[M=m] logQ M
MQM ,m(Qm , 1) 1mlogQ logQ M;m2N [M=m] logQ M M X m2 (1 , Q,m ) logQ M Q 1mlogQ logQ M;m2N M logQ M (logQ logQ M )3 Q M Q plog M ; Q for suciently large M .
2
Corollary 4.7. There exists a natural N (which can be e ectively computed) such that for all M N one has M +3 #fx 2 A jN jxj M; x is not Borel normal g pQ : logQ M
(8)
Proof. By Lemma 4.6 we get a bound N for which the inequality (7) is true. Accordingly,
#fx 2 A jN jxj M; x is not Borel normal g M +3
pQlog M : Q
(the last inequality can be proved by induction on M .)
M X Qi plog Qi i=N
2
Theorem 4.8. We can e ectively nd two natural constants c and M such that every x 2 A with jxj M and which is not Borel normal satis es the inequality: (9) K (x) jxj , 12 logQ logQ jxj + c:
Proof. Consider the (primitive) recursive bijection string : N ! A given by string(n) = the n-th string according to the quasi-lexicographic order on A . Next de ne the recursive function f : N+ ! A by f (t) = the t , th string x (according to the same quasi-lexicographical order) which is not Borel normal and has the length greater than N . (N comes from Corollary 4.7.) In view of (8), jxj+3 t qQ ; (10) logQ jxj
provided f (x) = t. Finally, de ne the p.r. function : A !o A by
(u) = f (string,1(u)) (11) and consider b the constant comming from the Invariance Theorem (applied to and ). If x = f (t), then (string(t)) = f (t) = x and (by (10)): K (x) K (x) + b jstring(t)j + b logQ (t + 1) + b 0 1 jxj+3 Q logQ @ q + 1A + b jxj , 21 logQ logQ jxj + c: logQ jxj 2 Theorem 4.9. For every natural t 0 we can e ectively compute a natural number Mt (depending upon t) such that every Kolmogorov-Chaitin t-random string of length greater than Mt is Borel normal. Proof. Let c, M be the constants comming from Theorem 4.8 and put
Mt = max M; QQ
t c
2( + )
:
If x is not Borel normal, then x satis es the inequality (9) (Mt N ) : K (x) jxj , 12 logQ logQ jxj + c < jxj , 12 logQ logQ Mt + c jxj , t; a contradiction.
2
Corollary 4.10. Almost all Kolmgorov t-random strings are m-limiting, Borel ("; m)-normal and "-limiting.
Proof. Every Kolmgorov t-random string x which is m-limiting and satis es the inequality logQ jxj "2 jxj is also Borel ("; m)-normal. 2
As a direct consequence we get the following result proved by Kramosil [23]:
Corollary 4.11. Let (xn )n1 be a sequence of Kolmgorov t-random strings such that jxn j = n. Then, for every natural n 1 Nim(xn ) = Q,m ; lim n!1 [n=m] for every 1 i Qm.
Remark. Due to a classical result of Chaitin (see Section 10 in [7] and Theorem B in [9]) (for an alternative approach see Martin-Lof [30, 5], Calude, Chitescu [5]) there is no sequence x 2 A whose pre xes are all KolmogorovChaitin t-random. We nish this section with an analysis of random strings in Chaitin's sense. Chaitin [13] pointed out that if K dips t below its maximum, then H must dip t,O(log t), i.e. H (x) < jxj+H (string(jxj)),t+O(log t) provided K (x) < jxj,t: This shows that every maximally H complex string must be also a maximally K complex string. To state this result we needo some more tools. Consider Chaitin's computers C : A A ! oA such that for every v 2 A , the domain of Cv is pre x-free, where Cv : A ! A ; Cv (x) = C (x; v), for all x 2 A . Fix a universal Chaitin's computer U : A A !o A . Chaitin's conditional complexity induced by C is de ned by HC (x=v) = minfjyj j y 2 A ; C (y; v ) = xg; where
v = minfu 2 A jU (u; ) = vg is the canonical program generating v. Finally put HC (x; y) = HC (< x; y >), where : A A ! A is a pairing function.
Theorem 4.12. (Chaitin [14]) For all x 2 A and t 2 N, if K (x) < jxj, t, then H (x) < jxj + H (string(jxj)) , t + O(logQ t): Proof. We start noticing that the set f< x; t >2 A NjK (x) < jxj, tg is r.e., thus if K (x) < jxj , t, then we can eventually discover this. Moreover, there are less than Qn,t =(Q , 1) strings of length n having this property. Thus if we are given jxj = n and t we need to only know n , t digits to pick out any particular string x 2 An with this property. I.e., as the rst x that we discover has this property, the second x that we discover has this property,..., the ith x that we discover has this property, and i < Qn,t =(Q , 1). It follows that any x 2 An that satis es the inequality
K (x) < jxj , t
has the property that
H (x= < string(n); string(t) >) < n , t + O(1): So, by the relation (for a proof see Theorem I1 in Chaitin [9]):
H (x) H (y) + H (x=y) + O(1) we get
H (x) < H (x= < string(n); string(t) >) + H (< string(n); string(t) >) + O(1) < n , t + H (string(n); string(t)) + O(1) < n , t + H (string(n)) + H (string(t) + O(1) < n , t + H (string(n)) + O(logQ t); since in general H (string(m)) < O(logQ m):
2
Corollary 4.13. For every t 2 N and every Chaitin t-random string x, one has K (x) jxj , T , whenever T , O(logQ T ) t. Proof. Fix t 2 N and pick x such that H (x) (jxj) , t. If K (x) < jxj , T , then by Theorem 4.12
H (x) < (jxj) , T , O(logQ T ); which means T , O(logQ T ) < t.
2
In view of Corollary 4.13, Theorem 4.9 applies and we get:
Theorem 4.14. For every natural t 0 we can e ectively nd a natural Mt (depending upon t) such that every Chaitin's t-random string x with jxj Mt is Borel normal.
Acknowledgement Arto Salomaa drew our attention to the paper [28] and suggested the study of normality for random strings. Greg Chaitin kindly permitted to insert his Theorem 4.12 (proved twenty years ago, but never published) and suggested Theorem 4.14. Ion Chitescu has communicated us Lemma 3.2. Helmut Jurgensen gave valuable comments on the manuscript. After completion of the paper, Ming Li pointed out the related results from [26]. I express them all my gratitude.
References
[1] E . Borel. Les probabilites denombrables et leurs applications arithmetiques, Rend. Circ. Mat. Palermo 27(1909), 247-271. [2] E . Borel. Lecons sur la theorie des fonctions, Gauthier-Villars, Paris, 2nd ed., 1914. [3] C. Calude. Theories of Computational Complexity, North-Holland, Amsterdam, 1988. [4] C. Calude. Information and Randomness: An Algorithmic Perspective, Springer-Verlag (in press; draft version: Elementary Algorithmic Information Theory, Bucharest University, 1991, 90 pp). [5] C. Calude, I. Chitescu. Upper limitation of Kolmogorov complexity and universal P.Martin-Lof tests, J. Computational Math. 7(1989), 61-70. [6] C. Calude, I. Chitescu. Qualitative properties of P.Martin-Lof random sequences, Boll. Unione Mat. Ital. (7) 3 - B(1989), 229-240. [7] G. J. Chaitin. On the length of programs for computing nite binary sequences, J. Assoc. Comput.Mach. 13(1966),547-569. (Reprinted in: Chaitin [10], 369-410.) [8] G. J. Chaitin. On the length of programs for computing nite binary sequences : statistical considerations, J. Assoc. Comput. Mach. 16 (1969), 145-159. (Reprinted in [10], 245-260.) [9] G. J. Chaitin. Algorithmic Information Theory, Cambridge University Press, Cambridge,1987. (third printing 1990) [10] G. J. Chaitin. Information, Randomness and Incompleteness, Papers on Algorithmic Information Theory, World Scienti c, Singapore, 2nd ed., 1990. [11] G. J. Chaitin. Information-Theoretic Incompleteness, World Scienti c, Singapore, 1992. [12] G. J. Chaitin.Randomness in Arithmetic and the Decline and Fall of Reductionism in Pure Mathematics, IBM Research Report RC-18532 (November 1992), 26 pp. [13] G. J. Chaitin. Electronic Mail to C.Calude, December 12, 1992. [14] G. J. Chaitin. Electronic Mail to C.Calude, March 25, 1993. [15] D. G. Champernowne. The construction of decimals normal in the scale of ten, The Journal of London Mathematical Society 8 (1933), 254-260.
[16] A. H. Copeland, P. Erdos. Note on normal numbers, Bull. Amer. Math. Soc. 52(1946), 857-860. [17] T. M. Cover, P. Gacs, R. M. Gray. Kolmogorov's contributions to information theory and algorithmic complexity, The Annals of Probability 17(1989), 840-855. [18] C. Dellacherie. Nombres au hazard. De Borel a Martin-Loef, Gazette des Math., Soc. Math. France 11(1978), 23-58. [19] W. Feller. An Introduction to Probability Theory and Its Aplications, Vol. 1, Chapman & Hall, London, John Wiley & Sons, New York, 2nd ed., 1958. [20] G. H. Hardy, E. M. Wright. An Introduction to the Theory of Numbers, Clarendon Press, Oxford, 5th ed., 1979. [21] D. E. Knuth. The Art of Computer Programming, Vol. 2, Seminumerical Algorithms, Addison-Wesley, Reading, Ma., 2nd ed., 1981. [22] A. N. Kolmogorov. Three approaches for de ning the concept of \information quantity", Problemy Peredachi Informatsii 1(1965), 3-11. [23] I. Kramosil. In nite pseudo-random sequences of high algorithmic complexity, Kybernetika (Prague) 20(1984), 429-437. [24] L. Kuipers, H. Niederreiter. Uniform Distribution of Sequences, JohnWiley & Sons, New York, 1974. [25] M. Li, P. M. Vitanyi. Kolmogorov complexity and its applications, in J. van Leeuwen (ed.). Handbook of Theoretical Computer Science, Vol. A, North-Holland, Amsterdam, MIT Press, Boston, 1990, 187-254. [26] M. Li, P. M. Vitanyi. Combinatorics and Kolmogorov complexity, Proc. 6th IEEE Structure in Complexity Theory Conference, 1991, 154-163. [27] M. Li, P. M. Vitanyi. An Introduction to Kolmogorov Complexity and Its Applications, Report, May 1992, 593 pp. [28] H. B. Marandijan. Selected topics in recursive function theory in computer science, ID - TR: 1990 - 75, Dept. Comput. Sci., Technical University of Denmark,Lyngby, 1990, 93 pp. [29] P. Martin-Lof. The de nition of random sequences, Inform. and Control 6(1966), 602-619. [30] P. Martin-Lof. Complexity oscillations in in nite binary sequences, Zeitschrift fur Wahrscheinlichkeitstheorie und Verwantdte Gebiete 19(1971), 225-230.
[31] M. Mendes France. Suites de nombres au hasard (d'apres Knuth), Seminaire de Theorie des Nombres, Expose 6, 1974, 1-11. [32] I. P. Natanson. Theory of Functions of A Real Variable, Frederick Ungar Publishing Co., New York, 1955. [33] I. Niven, H. S. Zuckerman. On the de nition of normal numbers, Paci c J.Math. 1(1951), 103-110. [34] C. P. Schnorr. Zufalligkeit und Wahrscheinlickeit, Lecture Notes Math. Vol. 218, Springer - Verlag, Berlin, 1981. [35] R. von Mises. Probability, Statistics and Truth, G.Allen and Unwin Ltd., London, Macmillan, New York, 2nd Revised English Edition Prepared by Hilda Geiringer, 1961.
