Towards a Mathematical Understanding of Intelligence

V. Kosoy

What is Intelligence?

• AI dates back at least to the 1950s• There is no accepted definition of what

intelligence is• Turing test doesn’t count: empirical rather

than mathematical

What is Intelligence?

• Most every day objects don’t have mathematical definitions

• My opinion: intelligence must be different• It seems natural to seek for this definition in

the domain of computation theory

Characteristics of Intelligence

• Pattern recognition• Prediction• Universal problem solving

Pattern Recognition

• Can be formalized as the Kolmogorov complexity problem

• What is the shortest program producing given data (string of bits)?

• Random string: has to be hardcoded• 010101010…01 has compact description

Pattern Recognition

• Kolmogorov complexity is uncomputable due to Berry paradox!

Solomonoff Induction

• Bayesian inference for a universe generated by a random program

• This distribution favors low Kolmogorov complexity: formalization of Occam’s razor!

• Making best guess with this prior allows predicting any computable sequence

• This procedure is uncomputable!

Imperfect Prediction

• Shane Legg ‘06• No Universal

Predictors• Predicted

complexity vs. predictor complexity

• Unprovability

Universal Problem Solving

• Arguably we can only solve problems for which the solution can be efficiently verified

• Corresponds to non-deterministic algorithms• Efficiently computing non-deterministic

algorithms is possible iff P = NP

Legg-Hutter Intelligence

• Shane Legg, Marcus Hutter 2007• Quantitative rather than qualitative• Black boxes with input, output and utility• Average utility in a random program universe• No complexity considerations

Legg-Hutter Intelligence

Goedel Machine

• Juergen Shmidhuber 2003• Essentially the same black box framework• Reprograms itself using (Levin) proof search• Metalearner: everything is reprogrammable• Universe prior can be e.g. Solomonoff• Limited by axiom system• The degenerate environment problem

Asymptotic Computation

• Kolmogorov complexity and universal prediction are asymptotically computable

• This is a realistic model of intelligence: we can’t be sure we found the best model

P vs. NP revisited

• NP oracle allows efficient pattern recognition• NP oracle allows efficient prediction• NP oracle allows universal problem solving• Maybe the problem is hard because its

solution is key to understanding the nature of intelligence and even reality itself

Summary

• There is no satisfactory definition yet• Connection between properties of intelligence

and natural concepts in computer science is ominous

• Solomonoff induction, the works of Hutter, Shmidhuber and Legg provide important pieces for the puzzle

• Further progress will come from P vs. NP