Data Evolution in General Formal Problem Statement

Data Evolution inGeneral Formal Problem Statement

A General Approach to Evolutionary Problem Solving, Implemented in the Computer Language GENETICA

Background

CLASSICAL PROBLEM SOLVING

According to the general notion of "problem solving" there is a set of states (blue dots) …

Search Space Solution Space… within which is defined a subset of states (red dots) having some desired properties.

The former set—structured by an appropriate search procedure—is called "the search space"…

… while the subset is called "the solution space".

The objective of a problem solving procedure is to reach an element of the solution space, given the search space.


ConfirmationThis is the case of "confirmation" problems, i.e. problems where a state either belongs to the solution space or not.


In another class of problems, membership in the solution space is considered as quantitative: all the elements of the search space belong to the solution space, each one having a different membership value which expresses the acceptability of the element as a solution. Then the problem becomes one of finding a state having as high membership value as possible.OptimizationThis is the case of the optimization problems


Although the current Artificial Intelligence (AI) research has focused on different issues, such as "behavioral" intelligence, adaptability, cellular or connectionist systems, evolutionary systems etc, the classical notion of "problem solving" remains valid as—in its most abstract and general form—it is encountered in virtually any AI system. For instance, adaptability can be conceived as convergence to goal-states, while "behaviors" can be viewed as states within a search space.

Due to the critical role of "problem solving" in various AI methods, the development of AI systems can be effectively supported by general problem solving tools. Such tools should offer both expressive and creative power. The former has been traditionally dealt with by means of formal languages, within the field of symbolic AI, while the latter—viewed as performance in searching within complex and "unpredictable" search spaces—has been extensively investigated within the field of Evolutionary Computation (EC). As any AI system, even if it is not "algorithmic" in nature, is subjected to a formal description (otherwise it could not be developed in the computer), while several forms of "creative" behavior could be based on (or enhanced by) evolutionary dynamics, the development of AI systems could be effectively supported by tools that integrate formal expression and evolution methods. According to this direction, the computer language GENETICA is proposed here as a general tool for the development of AI systems.

First we are going to identify some key-features of the original approaches to "expressive" and "creative" problem solving, respectively represented by symbolic AI and EC methods.

A weak point of classical AI problem-solving is that exhaustive search suffers from the "combinatorial explosion" effect which could make solution practically inefficient.

Classical AI problem-solving systems are based on "physical symbol structures".

Such systems include symbols, relations and "processes", where symbols and relations form a symbolic language, while "processes" generate the elements of the search space.AI languages include predicate logic, which makes possible to use any logical expression concerning

countable classes in the problem representation. Given this representation, AI problem solvers scan the search space until a solution is reached. A solution is a "symbol structure" that confirms the problem statement.

SYMBOLIC ARTIFICIAL INTELLIGENCE

Physical symbol structures

Symbols

Relations

Processes

Symbolic Language

Search

Exhaustive Search

Expressive power

: Complexity limitations

Low Creativity

The strong point is expressive power, which allows reduction of the search space through the incorporation of problem specific knowledge in the problem formulation.

However, this approach reduces "creativity", i.e. the potential of the AI system to create innovative solutions unpredictable in terms of existing problem specific knowledge.Furthermore, AI systems based on symbol structures cannot cope with optimization problems where the search space is continuous.

A major class of problem solving systems falls into the framework of symbolic Artificial Intelligence

Unfortunately, even if the implementation of problem-specific knowledge is desirable (in order to achieve a more accurate problem statement or a more effective solution procedure) it is often very hard to be attained.

Computational efficiency supports creativity, as the need for search space reduction through problem-specific knowledge becomes less intensive.

The strong point of evolutionary problem-solving is computational efficiency: evolution may result in solutions, in cases where this would be practically inefficient by exhaustive search.

A "population" of individuals, i.e. formal representations of potential solutions, is evolved…

EVOLUTIONARY COMPUTATION

Fitness evaluation

Low expressive power

High computational efficiency

in a population of individuals

subjected to genetic operations

High creativity

Evolutionary problem solving methods evolve elements of the search space aiming to the optimization of "fitness": a magnitude representing the "desirable attribute" of a solution.

…as "genetic operations" affect individuals, while emerging high-fitness individuals substitute lower-fitness ones in the population.

This is because the problem statement is not supported by a universal formal language (such as predicate logic). As a consecuence, evolutionary methods tend to be highly "problem-specific".Furthermore, the quantitative nature of "fitness" makes hard for these methods to cope with general confirmation problems.

Other problem solving systems include optimization methods based on either numerical techniques or stochastic search. Within this framework, Evolutionary Computation systems exhibit a remarkable combination of generality of application and effectiveness.


Genetic Algorithms

Binary strings

Major fields of investigation within the framework of Evolutionary Computation are Genetic Algorithms (GAs) and Genetic Programming (GP)

Classical GAs represent to-be-evolved objects as fixed-length binary strings, the latter viewed as genotypes.


1 1 0 0 1 0 1 0

1 0 0 1 0 0 0 1

Consider a population of genotypes, each one presented here as a colored dot.

Colors represent fitness values: higher fitness genotypes (biased to red) are shown at the top of the oval shape, while lower fitness genotypes (biased to blue) are shown at the bottom.

Each genotype is a binary string

A new genotype could emerge by the alteration of a limited number of bits within a string.

Such alterations constitute "genetic operations" on genotypes, known as "mutations".

1 1 0

1 0 0 1 0 1 1 0


1 1 0 0 0 0 0 1

1 0 0 1 1 0 1 0Homologous bits of different genotypes can be interchanged. This is a genetic operation, named "crossover", which results in new genotypes.

The application of either "crossover" or "mutation" on high fitnesss genotypes may result in higher fitness genotypes.


Such genotypes are introduced in the population……substituting low-fitness genotypes.As this process is repeated, the population is successively enriched in higher-fitness genotypes.


Genetic Algorithms

Binary strings

Genetic Programming

Tree-structured genotypes

Computer programs

As binary strings have very limited expressive power…

…the expressive power of genetic methods has been enhanced by more sophisticated representations, such as tree-structured genotypes, which reflect domain-specific knowledge.

Genetic Programming (GP) considers computer programs as the objects to be evolved. This largely improves expressiveness, as the expressive power of a computer language allows to represent virtually anything as a computer program.

P Q R F

NOT (F)

P AND Q x (R)

(P AND Q) OR ( x (R))

((P AND Q) OR ( x (R))) AND (NOT (F))

EVOLUTIONARY COMPUTATIONConsider the syntax structure of a computer program (schematically represented here as a predicate logic expression).

F

NOT (F)

P Q

RP OR Q

(P OR Q) AND R

((P AND Q) OR ( x (R))) AND (NOT (F))


F

NOT (F)

P Q

RP OR Q

(P OR Q) AND R

((P OR Q) AND R) AND (NOT (F))

EVOLUTIONARY COMPUTATIONParts of this structure can be interchanged with syntactically compatible parts of the syntax structure of another computer program. This can be viewed as an extension of the "crossover" operation. Partial regeneration of the structure can be viewed as an extension of the "mutation" operation.

GP methods use such operations to evolve a population of computer programs, where the fitness of each program is calculated by the program execution.

A class of evolutionary methods—some of them using GP techniques—consider abstract objects, which reflect the syntax of a fixed predicate logic expression, as the objects to be evolved. The logical expression can be viewed as a problem statement, while the objects to be evolved can be viewed as solution instances. In this case the problem representation is supported by the expressive power of predicate logic.

Although the programming language of a GP application makes possible high expressiveness in solution representation, GP does not provide an equally effective representation of the problem to be solved: in classical GP the user is only allowed to define the elements of the programming language (i.e. the function set and the terminal set), the fitness function, and potentially some aspects of the program architecture. Implementation of problem-specific knowledge has been attempted through specifications of the solution representation language, which results in highly problem-specific GP methods.

Another limitation of expressiveness concerns the quantitative nature of "fitness", which severely limits the possibility of a general GP approach to confirmation problems.


Evolution in formal logic

Evolutionary machine learning

Evolutionary abstract interpretations

Evolutionary theorem provingApplications have been developed within the fields of machine learning, abstract interpretation and theorem proving. These applications exhibit high expressiveness in problem representation, while they can cope with confirmation problems. However, they are not appropriate for general problem solving tasks, as their application domain is limited, while they do not exhibit creativity, as they are strictly based on existing knowledge.

SOME REMARKS ON COMPUTER AIDED PROBLEM SOLVING

Elementary decisions

Selection in

Countable classes

Continuous classes

Selection method

Deterministic

Stochastic

Evaluation method

Logical

Numerical

Under a general conceptualization, problem-solving in the computer can be viewed as a process of construction and evaluation of data structures during the program execution. Construction of different data structures creates the search space while evaluation selects the elements of the solution space.

The elementary decisions leading to data construction concern selection of an element of either a countable class (which is represented either as a list, an array or an integer interval) or a continuous class (which is represented as a real interval).

Selection may be either deterministic (e.g. exhaustive enumeration of potential solutions in AI systems) or stochastic (e.g. random selections encountered in stochastic optimization or genetic methods).

Data structures may be evaluated either in a Boolean manner (confirmation criteria) or in a numerical manner (optimization criteria).

A general application of evolutionary heuristics in problem solving could be achieved if stochastic elementary decisions within (arbitrarily complex) formal expressions could be treated as genes, while either logical or numerical evaluation of (arbitrarily complex) data structures—the latter viewed as potential solutions—could be formulated as a fitness function.

Continued in Section_2.PPS

Documents

Data Evolution in General Formal Problem Statement