Cornucopia of Randomness

Verónica BecherDepartamento de Computación

Universidad de Buenos Aires

Joos Heintz’s 60th birthday – October 2005

Tossing a fair coin?

11111111111111111111111111111111111

0101010101010101010101010101010101

0010100010010101000001011111001001

Towards a definition of randomness

A random sequence…

should be unpredictable,

should be lawless,

should lack structure,

should avoid distinguishing properties,

should pass all conceivable statistical randomness tests.

Chaitin’s notion of randomness

“lack of structure”

Any structure or regularity can be used to compress the sequence, so,

the initial segments of a random infinite sequence are incompressible.

Chaitin’s definition (1975)

x{0,1} is random if its initial segments are algorithmically incompressible.

x is random if c n H(x | n ) > n-c

H is prefix- program size-complexity (variant of Kolmogorov complexity)

Proposition. Random sequences are Borel normal and not computable.

G.Chaitin “A theory of program size formally identical to Information theory”, J ACM 1975

Another definition of randomness?

random sequences should have no distinguishing properties…

A naive idea:

a sequence x{0,1} is random if it is in no set of Lebesgue measure 0.

Of course, since singletons have measure 0, there is no such sequence.

Martin Löf’s definition (1966)Random sequences are those that avoid every effectively presented measure 0 set of a certain kind (effective G of measure 0).

Formalizes the idea that a random sequence should pass every conceivable statistical test.

Corollary. The set of Martin Löf random reals has measure 1.

Martin Löf “The definition of random sequences” Information and Control 9, 1966.

The two definitions coincide

Theorem (Schnorr): Chaitin random Martin Löf random.

As with Church’s thesis for the definition of an algorithm,this equivalence can be regarded as supporting the definition of randomness.

The definition extends immediately to R, identifying R with {0,1}

Dyadic rationals may have two representations, but this is not important for the measure-theoretic considerations made in this work.

Almost all real numbers are random.

How can we come up with specific examples?

R

?

Examples of not random reals

rational numbers

computable numbers (e.g., , e, )

Liouville numbers

characteristic functions of r.e. sets

Computable reals are not random

They can be dramatically compressed: there is an algorithm (encoded with a fixed number of bits) for their whole fractional expansion.

e.g. Rationals, , 0.010101...

Pseudo-random numbers

"Anyone who considers arithmetical methods of producing random digits is, of course, in a state of sin.”

John von Neumann

Chaitin (1975) gave the first example of a random real

U = 2 -|p| = (domain(U){0,1} ) U(p) halts

where U:{0,1}*-> {0,1}* is any optimal (hence universal) function

with prefix-free domain.

Also known as a self-delimiting optimal Turing machine

Proposition. 0 < U < 1

U is the probability that the machine U with an arbitrary input halts.

R

U

-numbers

There are countably many self-delimiting optimal Turing machines

U1, U2 , U3 ...

Thus, there are countably many halting probabilities

1, 2 , 3 ...

R

2

1

3

Computably enumerable and random reals

Definition. A real is left c.e. if its left cut is recursively enumerable.

Proposition. -numbers are left c.e.

Theorem (Calude et al. and Kučera-Slaman 2001) -number left computably enumerable and random.

R

random computable c.e

?

Degrees of randomness

Turing machines provide the primary notion of effective computability. When machines are equipped with an oracle we have relative computability.

This induces a definition of randomness relative to some oracle.

Definition. A real is random in B iff its initial segments are algorithmically incompressible even with the help of oracle B.

random is 1-randomrandom in ’ is 2-random

random in (n-1) is n-random

and the obvious examples are

= the halting probability of U, is 1-random’

’ = the halting probability of U’, is 2-random’’ = the halting probability of U’’ , is 3-random

.

.

.

R

’

’’

’’’

Except for , all these examples are defined using oracles.

Cornucopia of randomness

Joint work with Serge Grigorieff (Université Paris 7)

Grigorieff’s conjecture

Given a non-empty set, the probability that an optimal Turing machine produces an output in this set is random.

Moreover, “the harder” the set, “the more random” the probability.

( If the set is 0n (0

n)-complete, then such probability is n-random )

Grigorieff’s conjectureGiven a non-empty set, the probability that an optimal Turing machine produces an output in this set is random.Moreover, “the harder” the set, “the more random” the probability.

( If the set is 0n (0

n) complete, then such probability is n-random)

Positive instances (1-random)

If the set is {0,1}*, the above probability is . For infinite r.e. sets, the above probability is random (Chaitin, 1988). Finite sets lead to randomness for some class of optimal machines (B-G)

0n complete sets have random probability (Miller 2005).

Grigorieff’s conjectureGiven a non-empty set, the probability that an optimal Turing machine produces an output in this set is random.Moreover, “the harder” the set, “the more random” the probability.

( If the set 0n (0

n) complete then such probability is n-random)

Negative instances

The conjecture fails for some 0n sets with rational probability (Miller 2004)

0n complete sets do not give n-randomness.

The 01 case is still open.

R

Randomness from infinite computations

Consider the space of finite and infinite sequences {0,1}

(or more generally the set of increasing sequences of elements from a computable partially ordered set).

We consider the “upper-cone” topology on this set (Beware!).

Theorem (B-G 2003) Monotone Turing machines U: {0,1} -> {0,1} are exactly the effective continuous maps for the usual topology on the Cantor space and the upper-cone topology on the target space.

Theorem (B-G 2004, 2005)

Let O be 0n and effectively hard for the class 0

n ({0,1} ). Then,

the probability that an optimal monotone machine U produces

an output in O, i.e.,

(U-1(O)),

is n-random.

If O is … then, the probability that a monotone machine performing infinite computations gives an output in O is ….

1-random[Space R, topology of semi-intervals (0,q) and (q,), O=R]

2-random[Space {0,1}, O ={0,1}*][ Space (N), O= finite sets]

3-random[Space (N), O = recursive sets O = r.e. sets O = cofinite sets]

Cornucopia of randomness

If O is 0n and effectively hard for the class 0

n ({0,1} )

we give a method to define sets that are 0n+m

and 0n+m

effectively

hard, for every m.

We use classes S= Rec, Fin, Inf, Cof, Coinf, Exists, All

rules for combining them given by certain operators .

Theorem (B-G 2005)

Let O be 0n and effectively hard for the class 0

n ({0,1}).

Then (U-1( O )) is n+m -random.

R

Happy birthday dear Joos!

Émile Borel

B= b1 b2 b3 b4...

bi= 0 if the i-th string is not a question in French. 1 if it is a question with a positive answer 2 if it is a question with a negative answer

“La connaissance de ce nombre donnerait donc la solution à toutes les énigmes, en infinité dénombrable, qui peuvent être posées dans le domaine de la science, de la curiosité,

de l´histoire et de la metaphysique.”

Émile Borel, La Définition en Mathematiques, 1948

Émile Borel

B= b1 b 2 b3 b4...

“Tout cela est pure fantaisie”É. Borel, La Définition en Mathematiques, 1948

We can not prove the properties of this number.

Chaitin’s definition of randomness

x is random iff c n H(x n) > n - c

Program size complexity

H(s) = min{ |p| : U(p) = s}

A = a1 a2 a3 ....

ai = 1 iff U(pi) halts

is not random

Indicate how many programs halt among the first n,

and there is an algorithm that tells you which ones!

The characteristic number of

Turing’s Halting Problem

U(p1)= b11 b12 b13 ....U(p2)= b21 b22 b23 ....U(p3)= b31 b32 b33 ....

.....

: b11 b22 b33 ....

is not in the table! Why not?

Turing’s application to the Entscheidungsproblem

It is undecidable whether the i-th program prints an i-th digit

Turing’s application to the Entscheidungsproblem

It is undecidable whether the i-th program prints an i-th digit

It is undecidable whether the i-th program will ever print 0.

It is undecidable whether the i-th program halts.

encodes the Halting problem

very compactly!

Knowing the first n digits of we can algorithmically

determine all halting programs up to length n.

Enumerate the halting programs until their contribution

reaches the first n digits of . Any halting program p not

yet enumerated contributes to with 2-|p|. Thus, |p| > n.

The characteristic number of the Halting problem is not random

A = a1 a2 a3 ....

where ai = 1 iff i(i) halts

(i)i is a recursive enumeration of the partial recursive functions

Proposition: A is not computable, not random

Algorithm: tell me how many among the first n halt and I will find them.The first n digits can be compressed to log n digits plus a constant.

Chaitin’s example of random real (1975)

U = 2 -|p| U(p) halts

where U:{0,1}*-> {0,1}* partial recursive, prefix-free domain, optimal.

(U is a self-delimiting universal Turing machine)

Proposition: 0 < U < 1

Proposition: U = ( domain(U){0,1} )= P(U halts)

Theorem : U is not computable and random.