Structures and Typologies of Computation through the Prism of Computational Models

Mark Burgin 1, Gordana Dodig-Crnkovic 2

1 Department of Mathematics, UCLA, Los Angeles, USAmburgin@math.ucla.edu

2 Mälardalen University, Västerås, Swedengordana.dodig-crnkovic@mdh.se

Abstract. We need much better understanding of information processing and its primary form – computation than we have now. As there is no information without (physical) representation, the dynamics of information is implemented on different levels of granularity by different physical processes, including the level of computa-tion performed by computing machines and living organisms. There are a lot of open problems related to the nature of information and essence of computation, as well as to their relationships. How is information dynamics represented in computa-tional systems, in machines, as well as in living organisms? Are computers pro-cessing only data or information and knowledge as well? What do we know of computational processes in machines and living organisms and how these processes are related? What can we learn from natural computational processes that can be useful for information systems and knowledge management?

1 Introduction

Many researchers have asked the question “What is computation?” trying to find a universal definition of computation or, at least, a plausible description of this important type of processes (cf., for example, (Turing, 1936)(Kolmogorov, 1953)(Copeland, 1996)(Burgin, 2005)(Denning, 2010)(Burgin & Dodig-Crnkovic, 2011) (Zenil, 2012)). Some did this in an informal setting based on computational and research practice, as well as on philosophical and methodological considerations. Others strived to build exact mathemat-ical models to comprehensively describe computation, and when Turing machine was constructed and accepted as a universal computational model, they imagined achieving the complete and exact definition of computation. However, the absolute nature of a Tur-ing machine was disproved and in spite of all efforts, the conception of computation re-mains too vague and ambiguous.

2 Mark Burgin 1, Gordana Dodig-Crnkovic 2

This vagueness of foundations has resulted in a variety of approaches, including ap-proaches that contradict each other. For instance, (Copeland, 1996) writes “to compute is to execute an algorithm.” Active proponents of the Church-Turing Thesis, such as (Fort-now, 2012) claim computation is bounded by what Turing machines are doing. For them the problem of defining computation was solved long ago with the Turing machine model. At the same time, Wegner and Goldin insist that computation is an essentially broader concept than algorithm (Goldin, Smolka, & Wegner, 2006) and propose interac-tive view of computing. At the same time (Conery, 2012) argues that computation is symbol manipulation, thus disregarding analog computers computing over continuous signals. Neuroscientists on the contrary describe sub-symbolic computation in neurons. (Angelaki et al. 2004)

Existence of various types and kinds of computation, as well as a variety of ap-proaches to the concept of computation, shows the complexity of understanding of com-putation in the holistic picture of the world.

To work out the situation, we analyzed processes of concept clarification in science and mathematics when attempts were made at finding comprehensive definitions of basic scientific and mathematical concepts. For instance, mathematicians tried to define a num-ber for millennia. However, all the time new kinds of numbers were introduced changing the comprehension of what a number is.

Looking back we see that at the beginning, numbers came from counting and there was only a finite amount of numbers. Then mathematicians found a way to figure out the infinite set of natural numbers, constructing it with the number 1 as the building block and using addition as the construction operation. As the number 1 played a specific role in this process, for a while, mathematicians excluded 1 from the set of numbers. At the same time, mathematicians introduced fractions as a kind of numbers. Later they under-stood that fractions are not numbers but only representations of numbers. They called such numbers rational as they represented a rational, that is, mathematical, approach to quantitative depiction of parts of the whole. Then a number zero was discovered. Later mathematicians constructed negative numbers, integer numbers, real numbers, imaginary numbers and complex numbers. Complex numbers allowed mathematicians to find solu-tions for all algebraic equations, becoming very useful in mathematics and beyond. So, mathematicians started to think that all kinds of numbers had already been found.

However, the rigorous representation of complex numbers as vectors in a plane gave birth to diverse number-like mathematical systems and objects, such as quaternions, oc-tanions, etc. Even now only few mathematicians regard these objects as numbers.

A little bit later, the great mathematician Cantor introduced transfinite numbers, which included cardinal and ordinal numbers (Cantor, 1883). Thus, the family of numbers was augmented by an essentially new type of numbers and this was not the end. In the 20 th

century, (Robinson, 1961) introduced nonstandard numbers, which included hyperreal and hypercomplex numbers. Later (Conway, 1976) founded surreal numbers and (Burgin, 2012a) established hypernumbers, which included real and complex hypernumbers. This process shows that it would be inefficient to restrict the concept of a number by the cur -rent state of the art in mathematics. This history helps us to come to the conclusion that it would be unproductive to restrict the concept of computation by the current situation in computer science and information theory.

Structures and Typologies of Computation through the Prism of Computational Models 3

In this paper, we present methodological and historical analysis of the conception of computation before and after electronic computers were built and computer science emerged, demonstrating that history brings us to the conclusion that efforts in building such definitions by traditional approaches would be inefficient, while an effective methodology is to find essential features of computation with the goal to explicate its nature and to build adequate models for research and technology.

In addition, we perform structural analysis of the conception of computation, explicat-ing various structures intrinsically related to computation.

Consequently, we study computation in the historical perspective, demonstrating the development of this concept on the practical level related to operations performed by people and computing devices, as well as on the theoretical level where computation is represented by abstract (mostly mathematical) models and processes. This allows us to discover basic structures inherent for computation and to develop a multifaceted typology of computations.

The paper is organized in the following way. In Section 2, we present methodological and historical analysis of the conception of computation. In Section 3, we study the struc-tural context of computation, illuminating the Computational Triad, the Shadow Compu-tational Triad and several other triads and pyramids intrinsically related to computation. In Section 4, we develop computational typology, which allows us to extract basic char-acteristics of computation and separate fundamental computational types. The suggested system of classes allows us to reflect a natural structure in the set of computational pro-cesses. In Section 5 we present the development of computational models, and particu-larly natural computing. Finally, we summarize our findings in Section 6.

2 Historical Development of the Notion of Computation

If you ask now who is a computer and what is computation, 99 people out of a hundred will tell that computer is not who but what because it is an electronic device, while com-putation is what computers are doing. However, for a long time, the term computer was associated with a human being. As Parsons and Oja (1994) write, “if you look in a dictio-nary printed anytime before 1940, you might be surprised to find a computer defined as a person who performs calculations. Although machines performed calculations too, these machines were related to as calculators, not computers.”

This helps us to understand the words from the famous work of Turing (1936). Ex-plaining first how his fundamental model, which later was called a Turing machine, works, Turing writes: “We may now construct a machine to do the work of this com-puter.” Here a computer is a person and not a machine.

Even in a recently published book (Rees, 1997), we can read, “On a chi-chhou day in the fifth month of the first year of the Chih-Ho reign period (July AD 1054), Yang Wei-Te, the Chief Computer of the Calendar – the ancient Chinese counterpart, perhaps, of the English Astronomer Royal – addressed his Emperor …” It does not mean that the Em-peror had an electronic device for calendar computation. A special person, who is called Computer by the author of that book, performed necessary computations.

4 Mark Burgin 1, Gordana Dodig-Crnkovic 2

Calculations of people-computers were governed by definite rules and in the medieval Europe these rules acquired the name algorithms. The word “algorithm” has an interest-ing historical origin. It derives from the Latin form of the name of the famous medieval mathematician Muhammad ibn Musa al-Khowarizmi. He was born sometimes before 800 A.D. and lived at least until 847. His last name suggests that his birthplace was in the Middle Asia, somewhere in the territory of modern Uzbekistan. He was working as a scholar at the House of Wisdom in Baghdad when around the year 825, he wrote his main work Al-jabr wa'l muqabala (from which our modern word "algebra" is derived) and a treatise on Hindu-Arabic numerals. The Arabic text of the latter book was lost but its Latin translation, Algoritmi de numero Indorum, which means in English Al-Khowarizmi on the Hindu Art of Reckoning, introduced to the European mathematics the Hindu place-value system of numerals based on the digits 1, 2, 3, 4, 5, 6, 7, 8, 9, and 0. The first intro-duction to the Europeans on the use of zero as a placeholder in a positional base notation was probably also due to al-Khowarizmi in this work. Various methods for arithmetical calculation in a place-value system of numerals were given in this book. In the twelfth century, his works were translated from Arabic into Latin. Methods described by al-Khowarizmi were the first to be called algorithms following the title of the book, which begins with the name of the author. For a long time, algorithms meant the rules for people to use in making calculations although as such people were called computers, their opera-tion with numbers was also called computation. Thus, algorithms were the first models of computation.

Over time, the meaning of the word algorithm has extended more and more (cf., for example, Barbin, et al, 1994). Originating in arithmetic, it was explained as the practice of algebra in the 18th century. In the 19th century, the term came to mean any process of systematic calculation. In the 20th century, Encyclopaedia Britannica described algorithm as a systematic mathematical procedure that produces – in a finite number of steps – the answer to a question or the solution of a problem.

With the advent of electronic computers, people started to call processes performed by these computers by the name computation. In addition, it was discovered that algorithms determine everything that electronic computers can do. In such a way, the name Al-Khowarizmi became imprinted into the very heart of information technology, computer science, and mathematics.

Now three levels of generality in understanding the phenomenon of computation exist (Burgin and Dodig-Crnkovic, 2011).):

On the top level, computation is perceived as any transformation of information and/or information representation.

On the middle level, computation is distinguished as a discrete process of transfor-mation of information and/or information representation.

On the bottom level, computation is recognized as a discrete process of symbolic transformation of information and/or symbolic information representation.

However, even before electronic computers came into being, models of computation became mathematical as mathematicians tried to capture the notion of algorithm by rigor-ous formal symbolic constructions.

Structures and Typologies of Computation through the Prism of Computational Models 5

The first model was λ-calculus built by Church (1932/33). Creating λ-calculus, Church was developing a logical theory of functions and suggested a formalization of the notion of computability by means of λ-definability. It is interesting to know that the theory of Frege (1893) actually contains λ-calculus. So, there were chances to develop a theory of algorithms and computability in the 19 th century. However, at that time the mathematical community did not feel a need in such a theory and probably, would not accept it if some-body created it.

The next model of computation, recursive functions, was introduced by Gödel (1934) in his 1934 Princeton lectures. According to Gödel, this concept was based on ideas of Herbrand communicated in a letter, while the construction was equivalent to the current notion of general recursive function. As it is stated in (Barbin, et al, 1999), the structure of recursive function formalizes double recursion, while a function defined by double recursion appeared in the works of Ackermann in 1920. Hilbert presented this function in 1925 in a lecture, so as to prove the continuum hypothesis, and Ackermann studied it in 1928.

Gödel expected that all effectively computable functions are general recursive, but was not convinced of this (nor of the first version of the Church's thesis of 1934 that stated that effective computability is equivalent to -computability) until he became acquainted with the work of Turing (1936), in which Turing introduced and studied ordinary Turing machines. Approximately at the same time, similar construction was developed by Post (1936).

Besides, Church (1936) brought in recursive functions and Kleene (1936) presented partial recursive functions. Later Kleene (1936a) demonstrated how λ-definability is related to the concept of recursive function, and Turing (1937) showed how λ-definability is related to the concept of Turing machine.

After this, a diversity of mathematical models of algorithms and computational archi-tectures has been suggested. They include: a variety of Turing machines: multihead, multitape Turing machines, Turing machines

with n-dimensional tapes, nondeterministic, probabilistic, alternating and reflexive Turing machines, Turing machines with oracles, Las Vegas Turing machines,.

neural networks of various types have been proposed – fixed-weights, unsupervised, supervised, feed forward, and recurrent neural networks.

different kinds of automata: von Neumann automata and general cellular automata; fi-nite automata of different forms – automata without memory, autonomous automata, automata without output or accepting automata, deterministic, nondeterministic, proba-bilistic automata, etc.;

variety of theoretical computing machines: Minsky machines; Storage Modification Machines or simply, Shönhage machines; Random Access Machines (RAM) and their modifications - Random Access Machines with the Stored Program (RASP), Parallel Random Access Machines (PRAM);

Petri nets (mathematical modeling language in a form of a directed bipartite graph for the description of distributed systems) of various types – ordinary and ordinary with restrictions, regular, free, colored, and self-modifying Petri nets, etc.;

computing architectures such as: vector machines; array machines; multidimensional structured model of computation and computing systems; systolic arrays; hardware modification machines;

6 Mark Burgin 1, Gordana Dodig-Crnkovic 2

new types of algorithms such as normal Markov algorithms and Kolmogorov algo-rithms.

varieties of production-rule based formalisms: Post productions; formal grammars of many forms – regular, context-free, context-sensitive, phrase-structure, etc.; and so on.

In addition, direct models of computational processes were introduced. Examples are: the CSP (communicating sequential processes) model of Hoare (1985), the ACP (algebra of communicating processes) model (Bergstra and Klop, 1984; Baeten and Bergstra, 1991), the CCS (calculus of communicating systems) model of Milner (1989), ESP (event-signal-process) model (Lee and Sangiovanni-Vincentelli (1996), the VCR model (Smith, 2000), the EVCR (extended view-centric reasoning) model (Burgin and Smith, 2006, 2007), and the EAP (event-action-process) model (Burgin and Smith, 2010).

The variety of models of algorithms and computational processes resulted in the corre-sponding assortment of types and kinds of computations, which are classified in Section 4 of this paper.

3 Structural Framework of Computation

The first intrinsic structure is the Computational Dyad (cf. Figure 1), was introduced in (Burgin, Mark and Eberbach, 2012)

Figure 1. The Computational Dyad

The Computational Dyad reflects the existing duality between computations and algo-rithms. According to (Denning, 2010), in the 1970s Dijkstra defined an algorithm as a static description of computation, which is a dynamic state sequence evoked from a ma-chine by the algorithm.

Later a more systemic explication of the duality between computations and algorithms was elaborated. Namely, computation is a process of information transformation, which is organized and controlled by an algorithm, while an algorithm is a system of rules for a computation (Burgin, 2005). In this context, an algorithm is a compressed informational/structural representation of a process.

Structures and Typologies of Computation through the Prism of Computational Models 7

Note that a computer program is an algorithm written in (represented by) a program-ming language. This shows that an algorithm is an abstract structure and it is possible to realize one algorithm as different programs (in different programming languages). More-over, many people think that neural networks perform computations without algorithms. However, this is not true because neural networks algorithms have representations that are very different from traditional representations of algorithms as systems of rules/in-structions. The neural networks algorithms are represented by neuron weights and con-nections between neurons. This is similar to hardware representation/realization of algo-rithms in computers (analog computing).

However, the Computational Dyad is incomplete because there is always a system that uses algorithms to organize and control computation. This observation shows that the Computational Dyad has to be extended to the Triangular Basic Computational Triad (cf. Figure 2).

Algorithm

Device/Agent Computation

Figure 2. The Triangular Basic Computational Triad (TBCT)

The Triangular Basic Computational Triad (TBCT) reflects that computation is per-formed by a device, such as a computer, or by an agent, such as a human being or bac-teria. Functioning of the device or work of the agent that/who performs computation is directed or controlled by an algorithm embodied in a program, plan, scenario or hard-ware. Consequently, the process of computation is also directed/controlled by an al-gorithm.

Note that the computing device can be either a physical device, such as a computer, or an abstract device, such as a Turing machine, or a programmed (virtual or simulated) device when a program simulates some physical or abstract device. For instance, neural networks and Turing machines are usually simulated by programs in conventional com-puters or a Java virtual machine can be run on different operating systems and is proces-sor- and operating system- independent.

Besides, with respect to architecture, it can be an embracing device, in which computa-tion is embodied and exists as a process, or an external device, which organize and con-trol computation as an external process.

It is also important to understand the difference between algorithm and its representa-tion implementation or embodiment. An algorithm is an abstract structure, which can be represented implemented in a multiplicity of ways: as a computer program, a control schema, a graph, a system of cell states in the memory of a computer, a mathematical system, such as an abstract finite automaton, etc.

8 Mark Burgin 1, Gordana Dodig-Crnkovic 2

The Computational Triad precisely reflects the structure of the world represented by the Existential Triad (Burgin, 2012). The Existential Triad develops the tradition of Plato, Aristotle and Popper, and is presented in Figure 3, giving a consistent understanding and scientifically grounded interpretation of Plato ideas.

World of Structures

Physical World Mental World

Figure 3. The Existential Triad of the World

Here the Physical (material) World is interpreted as the physical reality studied by natural sciences, the Mental World encompasses different levels of mentality, and the World of Structures consists of various forms and types of structures.

There is a natural correspondence between the Computational Triad and the Existential Triad. Indeed, the computing device belongs to the physical world, algorithm is the struc-tural representation of computation and computation itself goes in the Mental World, which according to existential triad theory (Burgin, 2012), is understood in a much broader way than in the traditional interpretation of mentality.

In addition to the components of the Computational Triad, there are other objects es-sentially related to computation. Computation always goes in some environment and within some context. Computation always works with data, performing data transforma-tions. Besides, it is possible to assume that computation performs some function and has some goal (for some agent) even if we don’t know this goal. The basic function of com-putation is information processing.

These considerations bring us to a new structure called the Triangular Shadow Compu-tational Triad (cf. Figure 4). In this triad, data come from the environment, their meaning and processing depend on the context, while the whole computation is function-oriented or goal-oriented. Computations initiated or performed by people are goal-oriented, while computational processes in nature are function-oriented.

Figure 4. The Triangular Shadow Computational Triad (TSCT)

Structures and Typologies of Computation through the Prism of Computational Models 9

Thus, the Triangular Shadow Computational Triad complements the Triangular Basic Computational Triad reflecting that any computation has a goal, goes on in some context, which includes environment, and works with data. In a computation, information is pro-cessed by data transformations.

In addition, the Triangular Shadow Computational Triad is homological (in the biolog-ical sense) to the Triangular Basic Computational Triad. Namely, data correspond to al-gorithm as in some computational situations; data play the role of an algorithm, or more exactly, the embodiment of the algorithm in the form of a program, while the embodi-ment of algorithm plays the role of data. An example of this situation is (traditional) sim -ulation testing of software when program is performed with different datasets to check the results of program functioning.

In the homological correspondence between computational triads, the triad component context/environment corresponds to the triad component device. More exactly, context corresponds to abstract devices, such as finite automata or Turing machines, while envi-ronment corresponds to physical devices, such as computers.

The last, but not the least, component of the Triangular Shadow Computational Triad, function/goal, corresponds to the component computation of the Triangular Basic Com-putational Triad. More exactly, function corresponds to computations that go on in nature, while goal corresponds to artificial devices, such as computers, which are oriented at achieving some goal in their computations.

10 Mark Burgin 1, Gordana Dodig-Crnkovic 2

To control computers and other computing devices, an algorithm has to be embodied into a program. This brings us to one more computational triad called the Triangular Ac-tion Computational Triad (cf. Figure 5).

program

device data

Figure 5. The Triangular Action Computational Triad (TACT)

In addition to triangular computational triads, there are also synthetic computational triads. The first such a triad (cf. Figure 6) was discovered by Wirth (1975) who called his book Algorithms + Data Structures = Programs. In this triad, algorithms and data struc-tures are synthesized into programs.

algorithm program

data structure

Figure 6. The Synthetic Computational Triad of Wirth (SCTW)

A similar synthetic computational triad (cf. Figure 7) was explicated by Michalewicz (1996) who called his book Genetic Algorithms + Data Structures = Evolution Programs. In this triad, genetic algorithms and data structures are synthesized into evolution pro-grams.

genetic algorithm evolution program

data structure

Figure 7. The Synthetic Computational Triad of Michalewicz (SCTM)

Structures and Typologies of Computation through the Prism of Computational Models 11

Here we introduce one more synthetic computational triad (cf. Figure 8), in this which operations applied to the data domain generate a process in general and computation in particular.

operations process/computation

data domain

Figure 8. The Synthetic Computational Triad of Processes (SCTP)

All triads considered above are flat structures. At the same time, further development of these triads gives us spatial structures. Namely, computation brings a new dimension to the Triangular Action Computational Triad (TACT) transforming it into the Action Computational Pyramid (ACP) presented in Figure 9.

Computation

Data Program

Device/Agent

Figure 9. The Action Computational Pyramid (ACP)

12 Mark Burgin 1, Gordana Dodig-Crnkovic 2

Combination of the Triangular Action Computational Triad with the Synthetic Compu-tational Triad of Wirth generates one more special structure called the Plain Computa-tional Pyramid (PCP) and presented in Figure 10.

Program/Plan/Scenario

Algorithm Data Structure

Data Device/Agent

Figure 10. The Basic Computational Pyramid (TBCP)

Context/environment brings a new dimension to the Action Computational Pyramid (TACP) making it the Four-dimensional (Superspatial) Action Computational Pyramid (SSACP). Here we cannot draw a picture of a four-dimensional pyramid. So, we present only its three-dimensional faces in Figure 11.

Computation

Data Program

Device

a) The first face

Structures and Typologies of Computation through the Prism of Computational Models 13

Context/environment

Data Program

Device

b) The second face

Computation

Data Context/environment

Device

c) The third face

Computation

Context/environment Program

Device

14 Mark Burgin 1, Gordana Dodig-Crnkovic 2

d) The fourth face

Computation

Data Program

Context/environment

e) The fifth face

Figure 11. The Superspatial Action Computational Pyramid (SSACP)

Exploring the external structure of computation in the context of information process-ing, we come to the Basic Information Processing Triad Presented in Figure 12. It encom-passes three basic information processes (Burgin, 2005).

storage

computation communication

Figure 12. The Basic Information Processing Triad (BIPT)

4 Computational Typology

There are many types and kinds of computations utilized by people and known to peo-ple. The structure of the world (Burgin, 2012) implies the following classification.

Structures and Typologies of Computation through the Prism of Computational Models 15

4.1[2.1] Existential/substantial typology of computations

1. Physical or embodied computations.2. Abstract or structural computations.3. Mental or impersonalized computations.

According to contemporary science, abstract and mental computations are always represented by some embodied computations.

The existential types from this typology have definite subtypes. There are three known types of physical/embodied computations.

1.1. Technical computations.

1.2. Biological computations.

1.3. Chemical computations.

Researchers discern three types of structural/abstract computations.

2.1 Symbolic computations.

2.2 Subsymbolic computations.

2.3 Iconic computations.

There are connections between these types. For instance, as (Bucci, 1997) suggests, the principle of object formation may be an example of the transition from a stream of massively parallel subsymbolic microfunctional events to symbol-type, serial processing through subsymbolic integration.

In addition to the existential typology, there are other typologies of computations.

3.2 Spatial (Organizational) typology of computations

1. Centralized computations where computation goes controlled by a single algorithm.

2. Distributed computations where there are separate algorithms that control computa-tion in some neighbourhood. Usually a neighbourhood is represented by a node in the computational network. Note that a neighbourhood can consist of a single com-putational device.

3. Clusterized computations where there are separate algorithms that control computa-tion in clusters.

Turing machines, partial recursive functions and limit Turing machines are models of centralized computations.

Neural networks, Petri nets and cellular automata are models of distributed computa-tions.

Grid automata in which some nodes represent networks with the centralized control (Burgin, 2005) and the World Wide Web are systems that perform clusterized computa-tions. Note that grid automaton is the most advanced abstract model of distributed com-puting systems, which perform concurrent computations, while is a physical distributed computing system.

16 Mark Burgin 1, Gordana Dodig-Crnkovic 2

3.3 Temporal (Operational) typology of computations

1. Sequential computations, which are performed in linear time.2. Parallel or branching computations, in which separate steps (operations) are syn-

chronized in time.3. Concurrent computations, which do not demand synchronization in time.

Note that while parallel computation is completely synchronized, branching computa-tion is not completely synchronized because separate branches acquire their own time and become synchronized only in interactions.

Classical models of computation, such as the classical Turing machine or partial recur-sive function, perform only sequential computations. Models that appeared later, such as Turing machines with several heads and tapes or cellular automata, provide means for parallel computations.

There are also various models for concurrent computations, such as Petri nets (Petri, 1962; Reisig, 1985), Kahn process networks (Kahn, 1974), discrete event simulators (Fishman, 1978), the CSP model of Hoare (1985), the Linda model (Gelernter, 1985), the ACP model (Bergstra and Klop, 1984; Baeten and Bergstra, 1991), the Actors model (Agha, 1986), the CCS model of Milner (1989), dataflow process networks (Lee and Parks, 1995), ESP (event-signal-process) model (Lee and Sangiovanni-Vincentelli, 1996), the VCR model (Smith, 2000), grid automata (Burgin, 2005), the EVCR (extended view-centric reasoning) model (Burgin and Smith, 2006), and the EAP (event-action-process) model (Burgin and Smith, 2010). All these models form three classes – device models (Petri nets, Kahn process networks, dataflow process networks discrete event simulators, grid automata, the Linda model and the Actors model), operational models (ACP model, ESP model, VCR model, EVCR model and ESP model) and process mod-els (CSP model and CCS model).

In this context, the most advanced device model with respect to spatial organization is grid automaton, while the most advanced model with respect to temporal operation, is the EAP (event-action-process) model, which also is a process model.

3.4 Representational typology of computations

1. Discrete computations, which include interval computations.2. Continuous computations, which include fuzzy continuous computations.3. Mixed computations, that include both discrete and continuous processes.

Digital computer devices and the majority of computational models, such as finite automata, Turing machines, recursive functions, inductive Turing machines, and cellular automata, perform discrete computations.

Examples of continuous computations are given by abstract models, such as general dynamical systems (Bournez, 1977) and hybrid systems (Gupta, V.; Jagadeesan, R.; Saraswat, 1998), and special computing devices, such as the differential analyzer (Shan-non, 1941; Moore, 1996).

Structures and Typologies of Computation through the Prism of Computational Models 17

Mixed computations include piecewise continuous computations, combining both discrete computation and continuous computation. Examples of mixed computations are given by neural networks (McCulloch & Pitts, 1990), finite dimensional machines and general machines of (Blum, Cucker, Shub, & Smale, 1996).

It is possible to refine the representational typology in the following way obtaining three additional classifications.

3.5 Data-oriented typology of computations

1. The domain of computation is discrete and data are finite. For instance, data are words in some alphabet.

2. The domain of computation is discrete but data are infinite. For instance, data are -words in some alphabet. This includes interval computations because real numbers traditionally are represented as -words.

3. The domain of computation is continuous

3.6 Operational typology of computations

1. Operations in computation are discrete and they transform discrete data ele-ments. For instance, addition or multiplication of whole numbers.

2. Operations in computation are discrete but they transform (operate with) con-tinuous sets. For instance, addition or multiplication of all real numbers or of real functions.

3. Operations in computation are continuous. For instance, integration of real func-tions.

3.7 Process typology of computations

1. The process of computation is discrete, i.e., it consists of separate steps in the discrete domain, and it transforms discrete data elements. For instance, computa-tion of a Turing machine or a finite automaton.

2. The process of computation is discrete but it employs continuous operations. An example is given by analogue computations (Shannon, 1941; Scarpellini, 1963; Moore, 1996).

3. .The process of computation is continuous but it employs discrete operations.. For instance, computation of a limit Turing machine (Burgin, 2005).

3.8 Hierarchy of computation levels

In (Burgin & Dodig-Crnkovic, 2011) three generality levels of computations are introduced.

18 Mark Burgin 1, Gordana Dodig-Crnkovic 2

1. On the top and most abstract/general level, computation is perceived as any transformation of information and/or information representation.

2. On the middle level, computation is distinguished as a discretized process of transformation of information and/or information representation.

3. On the bottom, least general level, computation is recognized as a discretized process of symbolic transformation of information and/or symbolic information representation.

There are also spatial levels or scales of computations:

1. The macrolevel includes computations performed by electromechanical devices, devices based on vacuum tubes and/or transistors, as well as mechanical calcu-lators.

2. The microlevel includes computations performed by integrated circuits.3. The nanolevel includes computations performed by fundamental parts that are

not bigger than a few nanometers.4. The molecular level includes computations performed by molecules.5. The quantum level includes computations performed by atoms and subatomic

particles.

There are no commercially available nanocomputers, molecular or quantum computers in existence at present. However, current chips produced by nanolithography are close to computing nanodevices because their basic elements are less than 100 nanometers in scale. In natural computing, addressed in the next chapter, this hierarchy of levels of computation corresponds to levels of physical organization of matter – from quantum computing to the universe as a network of networks of computational processes.

4 Natural Computing

Structures and Typologies of Computation through the Prism of Computational Models 19

The development of computing, both machinery and its models, continues. We are used to quick increase of computational power, memory and usability of our computers, but the limit of miniaturization is approaching as we are getting close to quantum dimensions of hardware. One of the ideals of computing ever since time of Turing is intelligent com-puting which would besides mechanical include even intelligent problem solving. Cur-rently, there is a development of cognitive computing aimed towards human abilities to process/organize/understand information. At the same time development of computa-tional modelling of human brain has for a goal to reveal the exact mechanisms of human brain function that will help us understand not only how humans actually perform symbol processing when they follow an algorithm, but also how humans create algorithms or models. Those new developments can be seen as a part of the research within the field of natural computing, where natural system performing computation is human brain. How-ever, natural computing has much broader scope. According to the Handbook of Natural Computing (Rozenberg, Bäck, & Kok, 2012) natural computing is “the field of research that investigates both human-designed computing inspired by nature and computing tak-ing place in nature.” In particular, it addresses: computational models inspired by the natural systems, computation performed by natural materials and computational nature of processes taking place in (living) nature. It includes among others areas of cellular auto-mata and neural computation, evolutionary computation, molecular computation and quantum computation and nature-inspired algorithms and alternative models of computa-tion.

An important characteristic of the research in natural computing is that knowledge is generated bi-directionally, through the interaction between computer science and natural sciences. While the natural sciences are rapidly absorbing ideas, tools and methodologies of information processing, computer science is broadening the notion of computation, recognizing information processing found in nature as natural computation. (Rozenberg & Kari, 2008) (Stepney et al., 2005)(Stepney et al., 2006) That led Denning (Denning, 2007) to argue that computing today is a natural science.

This new concept of computation with inspiration in natural information processing allows among others learning about nondeterministic complex computational systems with self-* properties (self-organization, self-configuration, self-optimization, self-heal-ing, self-protection, self-explanation, and self-awareness. Natural computation thus provides a basis for a unified understanding of phenomena of embodied cognition, intelli-gence and knowledge generation. (Dodig-Crnkovic & Mueller, 2009)(Wang, 2009)

In the development of unconventional algorithms (Dodig-Crnkovic & Burgin, 2012), which emphasizes the complementarity of axiomatics and construction, both elements being necessary for the progress in our understanding of computation. New powerful tools are brought forth by local mathematics, local logics, logical varieties and the axio-matic theory of algorithms, automata and computation. Further work includes study of natural computation by unconventional algorithms and constructive approaches.

5 Conclusions

20 Mark Burgin 1, Gordana Dodig-Crnkovic 2

We need much better understanding of information processing and its primary form – computation than we have now. As there is no information without (physical) representa-tion, the dynamics of information is implemented on different levels of granularity by different physical processes, including the level of computation performed by computing machines and living organisms. There are a lot of open problems related to the nature of information and essence of computation, as well as to their relationships. How is informa-tion dynamics represented in computational systems, in machines, as well as in living organisms? Are computers processing only data or information and knowledge as well? What do we know of computational processes in machines and living organisms and how these processes are related? What can we learn from natural computational processes that can be useful for information systems and knowledge management?

Present account of models of computation highlights several topics of importance for the development of new understanding of computing and its role: natural computation and the relationship between the model and physical implementation, interactivity as fundamental for computational modelling of concurrent information processing systems such as living organisms and their networks, and the new developments in modelling needed to support this generalized framework.

Structures and Typologies of Computation through the Prism of Computational Models 21

References

Bergstra, J. A. and Klop, J. W. (1984). Process algebra for synchronous communication. Informa-tion and Control, Vol. 60, Issue 1-3, pp. 109—137

Abramsky, S. (2008). Information, Processes and Games. In J. Benthem van & P. Adriaans (Eds.), Philosophy of Information (pp. 483–549). Amsterdam, The Netherlands: North Holland.

Blum, L., Cucker, F., Shub, M., and Smale, S. (1996). Complexity and Real Computation: A Manifesto. International Journal Of Bifurcation And Chaos, 6(1), 3–26. doi:10.1142/S0218127496001818

Bournez, O. (1977). Achilles and the Tortoise climbing up the hyper-arithmetical hierarchy. Theoretical Computer Science, 210, 210–211.

Bucci, W. (1997). Psychoanalysis and Cognitive Science: A Multiple Code Theory. New York: Guilford Press.

Burgin, M. (1990). Hypermeasures and hyperintegration. Notices of the National Academy of Sciences of Ukraine, 6, 10–13.

22 Mark Burgin 1, Gordana Dodig-Crnkovic 2

Burgin, M. (2005). Super-Recursive Algorithms (pp. 1–320). New York: Springer-Verlag New York Inc.

Burgin, M. (2012). Structural Reality. New York: Nova Science Publishers.Burgin, M.: Hypernumbers and Extrafunctions: Extending the Classical Calculus, Springer, New

York (2012a)Burgin, M., and Dodig-Crnkovic, G. (2011). Information and Computation – Omnipresent and

Pervasive. Information and Computation (pp. vii –xxxii). New York/London/Singapore: World Scientific Pub Co Inc.

Burgin, M. and Eberbach, E. (2012). Evolutionary Computation and Processes of Life. Ubiquity, (August), 1–13.

Cantor, G. (1883). Über unendliche lineare Punktmannigfaltigkeiten, 5. Grundlagen einer allgemeinen Mannigfaltigkeitslehre. Ein mathematisch-philosophischer Versuch in der Lehre des Unendlichen. Leipzig: Teubner.

Conery, J. S. (2012). Computation is symbol manipulation. The Computer Journal, 55(7), 814–816.Conway, J. H. (1976). On numbers and games. London, New York, San Francisco: Academic

Press.Copeland, B. J. (1996). What is computation? Synthese, 108(3), 335–359.Denning, P. (2007). Computing is a natural science. Communications of the ACM, 50(7), 13–18.

Retrieved from http://cs.gmu.edu/cne/pjd/PUBS/CACMcols/cacmJul07.pdfDenning, P. (2010). What is computation?: Editor’s Introduction. Ubiquity, (October), 1–2.Dodig-Crnkovic, G., and Burgin, M. (2012). Unconventional Algorithms: Complementarity of

Axiomatics and Construction. Entropy, 14(11), 2066–2080.Dodig-Crnkovic, G., and Mueller, V. (2009). A Dialogue Concerning Two World Systems: Info-

Computational vs. Mechanistic. (G. Dodig Crnkovic & M. Burgin, Eds.)Information and Computation. Singapore: World Scientific Pub Co Inc. Retrieved from http://arxiv.org/abs/0910.5001

Fortnow, L. (2012). The enduring legacy of the Turing machine. The Computer Journal, 55(7), 830–831.

Gupta, V.; Jagadeesan, R.; and Saraswat, V. A. (1998). Computing with continuous change. Science of Computer Programming, 30(1-2), 3–49.

Kolmogorov, A. N. (1953). On the Concept of Algorithm. Russian Mathematical Surveys, 8(4), 175–176.

McCulloch, W. S., and Pitts, W. (1990). A logical calculus of the ideas immanent in nervous activity. 1943. Bulletin of Mathematical Biology, 52(1-2), 99–115; discussion 73–97. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/2185863

Moore, C. (1996). Recursion theory on the reals and continuous-time computation. Theoretical Computer Science, 162(1), 23–44. doi:10.1016/0304-3975(95)00248-0

Robinson, A. (1961). Non-standard analysis. Indagationes Mathematicae, 23, 432–440.Rozenberg, G., Bäck, T., and Kok, J. N. (Eds.). (2012). Handbook of Natural Computing. Berlin

Heidelberg: Springer.Rozenberg, G., and Kari, L. (2008). The many facets of natural computing. Communications of the

ACM, 51, 72–83.Schachter, V. (1999). How Does Concurrency Extend the Paradigm of Computation? Monist,

82(1), 37–58.Shannon, C. (1941). Mathematical Theory of the Differential Analyzer. J. Math. Physics, 20, 337–

354.Stepney, S., Braunstein, S. L., Clark, J. A., Tyrrell, A. M., Adamatzky, A., Smith, R. E., Addis, T.

R., et al. (2005). Journeys in Non-Classical Computation I: A Grand Challenge for Computing Research. Int. J. Parallel Emerg. Distr. Syst., 20, 5–19.

Stepney, S., Braunstein, S. L., Clark, J. A., Tyrrell, A. M., Adamatzky, A., Smith, R. E., Addis, T. R., et al. (2006). Journeys in Non-Classical Computation II: Initial Journeys and Waypoints. Int. J. Parallel Emerg. Distr. Syst., 21, 97–125.

Structures and Typologies of Computation through the Prism of Computational Models 23

Turing, A. M. (1936). On computable numbers, with an application to the Entscheidungs problem. Proceedings of the London Mathematical Society, 42(42), 230–265. doi:10.1112/plms/s2-42.1.23

Wang, Y. (2009). On Abstract Intelligence: Toward a Unifying Theory of Natural, Artificial, Machinable, and Computational Intelligence. Int. J. of Software Science and Computational Intelligence, 1(1), 1–17. Retrieved from http://enel.ucalgary.ca/People/wangyx/Publications/Papers/aI/IJSSCI-1101-AbstractInt.pdf

Barbin, E., Borowczyk, J., Chabert, J.-L., Guillemot, M., Michel-Pajus, A., Djebbar, A., and Martzloff, J.-C. (1999) A History of Algorithms: From the Pebble to the Mi-crochip, (Chabert, J.-L., Ed.), Springer, New York/ Heidelberg/ Berlin

Burgin, M. and Smith, M. L. (2006) Compositions of Concurrent Processes, in Communi-cating Process Architectures, Concurrent Systems Engineering Series, IOS Press, Amsterdam, pp. 281-296

Burgin, M. and M.L. Smith, A (2007) Unifying Model of Concurrent Processes, In Pro-ceedings of the 2007 International Conference on Foundations of Computer Science (FCS'07), CSREA Press, Las Vegas, Nevada, USA, pp. 321-327

Burgin, M. and M.L. Smith, A (2010) A Theoretical Model for Grid, Cluster and Internet Computing, in Selected Topics in Communication Networks and Distributed Sys-tems, World Scientific, New York/London/Singapore, pp. 485 - 535

Michalewicz, Z. Genetic Algorithms + Data Structures = Evolution Programs, Springer-Verlag, New York, 1996

Parsons, J.J. and Oja, D. (1994) New Perspectives on Computer Concepts, Course Tech-nology, Inc., Cambridge, MA

Rees, M. (1997) Before the Beginning: Our Universe and Others, Simon&Schuster, Lon-don

Scarpellini, B. (1963) Zwei unentscheidbare Probleme der Analysis, Z. Math. Logik Grundlagen Math., v. 9, pp. 265-289

Wirth, N. Algorithms + Data Structures = Programs, Prentice Hall, Englewood Cliffs, NJ, 1975Zenil, H. (Ed.) A Computable Universe, Singapore: World Scientific Pub Co Inc., 2012