Titel - Research in Innovation, Design and Engineering ... · Web viewConcurrent computations, which do not demand synchronization in time. Note that while parallel computation is

The Nature of Computation andThe Development of Computational Models

Mark Burgin 1, Gordana Dodig-Crnkovic 2

1 Department of Mathematics, UCLA, Los Angeles, [email protected]

2 Mälardalen University, Västerås, [email protected]

Abstract. We need much better understanding of information processing and its primary form – computation than we have now. As there is no information without (physical) representation, the dynamics of information is implemented on different levels of granularity by different physical processes, including the level of computation performed by computing machines and living organisms. There are a lot of open problems related to the nature of information and es-sence of computation, as well as to their relationships. How is information dy-namics represented in computational systems, in machines, as well as in living organisms? Are computers processing only data or information and knowledge as well? What do we know of computational processes in machines and living organisms and how these processes are related? What can we learn from natural computational processes that can be useful for information systems and know-ledge management?

1 Introduction

Many researchers have asked the question “What is computation?” trying to find a universal definition of computation or, at least, a plausible description of this impor-tant type of processes (cf., for example, (Turing, 1936; Kolmogorov, 1953; Copland, 1996; Burgin, 2005; Denning, 2010; Burgin and Dodig-Crnkovic, 2011)). Some did this in an informal setting based on computational and research practice, as well as on philosophical and methodological considerations. Others strived to build exact mathe-matical models to comprehensively describe computation, and when Turing machine was constructed and accepted as a universal computational model, they imagined achieving the complete and exact definition of computation. However, the absolute nature of a Turing machine was disproved and in spite of all efforts, the conception of computation remains too vague and ambiguous.

This vagueness of foundations has resulted in a variety of approaches, including approaches that contradict each other. For instance, Copland (1996) writes “to compute is to execute an algorithm.” Active proponents of the Church-Turing Thesis, such as Fortnov (2010), claim computation is bounded by what Turing machines are doing. For them the problem of defining computation was solved long ago with the

2 Mark Burgin 1, Gordana Dodig-Crnkovic 2

Turing machine model. At the same time, Wegner and Goldin insist that computation is an essentially broader concept than algorithm (Goldin et al 2006) and propose interactive view of computing. At the same time Conery (2010) explains argues that computation is symbol manipulation. Neuroscientists on the contrary describe sub-symbolic computation in neurons. (Angelaki et al. 2004)

Existence of various types and kinds of computation, as well as a variety of ap-proaches to the concept of computation, shows complexity of understanding computa-tion in the holistic picture of the world.

To work out the situation, we analyzed processes of concept clarification in science and mathematics when attempts were made at finding comprehensive definitions of basic scientific and mathematical concepts. For instance, mathematicians tried to de-fine a number for millennia. However, all the time new kinds of numbers were intro-duced changing the comprehension of what a number is.

Looking back we see that at the beginning, numbers came from counting and there was only a finite amount of numbers. Then mathematicians found a way to figure out the infinite set of natural numbers, building it with 1 as the building block and using addition as the construction operation. As 1 played a specific role in this process, for a while, mathematicians excluded 1 from the set of numbers. At the same time, mathe-maticians introduced fractions as a kind of numbers. Later they understood that frac-tions are not numbers but only representations of numbers. They called such numbers rational as they represented a rational, that is, mathematical, approach to quantitative depiction of parts of the whole. Then a number zero was discovered. Later mathemati-cians constructed negative numbers, integer numbers, real numbers, imaginary num-bers and complex numbers. It looked like as if all kinds of numbers had been already found.

However, the rigorous representation of complex numbers as vectors in a plane gave birth to diverse number-like mathematical systems and objects, such as quater-nions, octanions, etc. Even now only few mathematicians regard these objects as numbers.

A little bit later, the great mathematician Georg Cantor (1883) introduced transfi-nite numbers, which included cardinal and ordinal numbers. So, the family of num-bers was augmented by an essentially new type of numbers and this was not the end. In the 20th century, Abraham Robinson (1961) introduced nonstandard numbers, which included hyperreal and hypercomplex numbers. Later Conway (1976) intro-duced founded surreal numbers and Burgin (1990) introduced established hypernum-bers, which included real and complex hypernumbers. This process shows that it would be inefficient to restrict the concept of a number by the current situation in mathematics. This history helps us to come to the conclusion that it would be ineffi-cient unproductive to restrict the concept of computation by the current situation in computer science and information theory.

In this paper, we present historical analysis of the conception of computation be-fore and after electronic computers were built and computer science emerged, demon-strating that history brings us to the conclusion that efforts in building such definitions by traditional approaches would be inefficient, while an effective methodology is to find essential features of computation with the goal to explicate its nature and to build adequate models for research and technology.

The Nature of Computation andThe Development of Computational Models 3

Consequently, we study computation in the historical perspective, demonstrating the development of this concept on the practical level related to operations performed by people and computing devices, as well as on the theoretical level where computa-tion is represented by abstract (mostly mathematical) models and processes. This al-lows us to discover basic structures inherent for computation and to develop a multi-faceted typology of computations.

The paper is organized in the following way. In Section 2, we study the structural context of computation, explicating the Computational Triad and the Shadow Compu-tational Triad. In Section 3, we develop computational typology, which allows us to extract basic characteristics of computation and separate fundamental computational types. The suggested system of classes allows us to reflect a natural structure in the set of computational processes.

In Section 4 we present the historical development of computational devices, from the oldest Antikythera mechanism that was an ancient analog device for computing of astronomical data, to present day digital computers. Section 5 addresses the develop-ments beyond conventional computing machinery in the form of unconventional/ nat-ural computation. The development of computing machinery is followed by the devel-opments of computational models which in the next step again affect the development of next generation of computational devices. Thus the Section 6 is devoted to the de-velopment of computational models, while Section 7 discusses current developments and prospect of natural computation, corresponding devices and models. We present the view of computing as natural science. Finally, we summarize our findings in Sec-tion 8.

2 Structural Context of Computation

The first intrinsic structure is the Computational Dyad (cf. Figure 1), was introduced in (Burgin and Eberbach, 2012). (cf. Figure 1).

Fig. 1. The Computational Dyad

The Computational Dyad reflects the existing duality between computations and algorithms. According to Denning (2010), in the 1970s Dijkstra defined an algorithm as a static description of computation, which is a dynamic state sequence evoked from a machine by the algorithm.

Later a more systemic explication of the duality between computations and algo-rithms was elaborated. Namely, computation is a process of information transforma-tion, which is organized and controlled by an algorithm, while an algorithm is a sys-tem of rules for a computation (Burgin, 2005). In this context, an algorithm is a com-pressed informational/structural representation of a process.

Note that a computer program is an algorithm written in (represented by) a pro-gramming language. This shows that an algorithm is an abstract structure and it is


possible to realize one algorithm as different programs (in different programming lan-guages). Moreover, many people think that neural networks perform computations without algorithms. However, this is not true because neural networks algorithms have representations that are very different from traditional representations of algo-rithms as systems of rules/instructions. The neural networks algorithms are repre-sented by neuron weights and connections between neurons. This is similar to hard-ware representation/realization of algorithms in computers (analog computing).

However, the Computational Dyad is incomplete because there is always a system that uses algorithms to organize and control computation. This observation shows that the Computational Dyad has to be extended to the Computational Triad (cf. Figure 2).

Fig. 2. The Computational Triad

Note that the computing device can be either a physical device, such as a computer, or an ab-stract device, such as a Turing machine, or a programmed (virtual or simulated) device when a program simulates some physical or abstract device. [I would use the word virtual instead of programmed device because ordinary computers are also programmed devices.] For instance, neural networks and Turing machines are usually simulated by programs in conventional com-puters. Or Java virtual machine can be run on different operating systems and is completely processor and operating system independent.

Besides, with respect to architecture, it can be an embracing device, in which com-putation is embodied and exists as a process, or an external device, which organize and control computation as an external process.

It is also important to understand the difference between algorithm and its repre-sentation or embodiment. An algorithm is an abstract structure, which can be repre-sented in a multiplicity of ways: as a computer program, a control schema, a graph, a system of cell states in the memory of a computer, a mathematical system, such as an abstract finite automaton, etc.

In addition, there are other objects essentially related to computation. Computation always goes in some environment and within some context. Computation always works with data performing data transformations. Besides, it is possible to assume that computation performs some function and has some goal (for some agent) even if we don’t know this goal. The basic function of computation is information process-ing.


These considerations bring us to the Shadow Computational Triad (cf. Figure 3).

Fig. 3. The Shadow Computational Triad

Thus, the Shadow Computational Triad complements the Computational Triad re-flecting that any computation has a goal, goes on in some context, which includes en-vironment, and works with data. In a computation, information is processed by data transformations.

[Would it be possible to characterize this triad as Computational Agent-centric Triad while Figure 2 is Computational Process-centric triad? If we call it “shadow” it somehow indicates it is not equally important while in fact it is. That Process-centric vs. Agent-centric nomenclature is closer to the classification of Ch 3. Does it make difference that in Figure 3 the connections are by lines not by double arrows?]

3 Computational Typology

There many types and kinds of computations utilized by people and known to peo-ple. The structure of the world (Burgin, 2012) implies the following classification.

3.1 Existential/substantial typology of computations

1. Physical or embodied computations.

2. Abstract or structural computations.

3. Mental or impersonalized computations.

According to contemporary science, abstract and mental computations are always represented by some embodied computations.

The existential types from this typology have definite subtypes. There are three known types of physical/embodied computations.


a. Technical computations.

b. Biological computations.

c. Chemical computations.

Researchers discern three types of structural/abstract computations.

a. Symbolic computations.

b. Subsymbolic computations.

c. Iconic computations.

There are connections between these types. For instance, as Bucci (1997) suggests, the principle of object formation may be an example of the transition from a stream of massively parallel subsymbolic microfunctional events to symbol-type, serial process-ing through subsymbolic integration.

In addition to the existential typology, there are other typologies of computations.

3.2 Spatial typology of computations

1. Centralized computations where computation goes controlled by a single al-gorithm.

2. Distributed computations where there are separate algorithms that control computation in some neighborhood. Usually a neighborhood is represented by a node in the computational network.

3. Clusterized computations where there are separate algorithms that control computation in clusters of neighborhoods.

Turing machines, partial recursive functions and limit Turing machines are models of centralized computations.

Neural networks, Petri nets and cellular automata are models of distributed compu-tations.

Grid automata in which some nodes represent networks with the centralized con-trol (Burgin, 2005) and the World Wide Web are systems that perform clusterized computations.

3.3 Temporal typology of computations

1. Sequential computations, which are performed in linear time.2. Parallel or branching computations, in which separate steps (operations) are

synchronized in time.3. Concurrent computations, which do not demand synchronization in time.

Note that while parallel computation is completely synchronized, branching com-putation is not completely synchronized because separate branches acquire their own time and become synchronized only in interactions.


3.4 Representational or operational typology of computations

1. Discrete computations, which include interval computations.2. Continuous computations, which include fuzzy continuous computations.3. Mixed computations, some processes in which are discrete, while others are

continuous.Digital computer devices and the majority of computational models, such as finite

automata, Turing machines, recursive functions, inductive Turing machines, and cel-lular automata, perform discrete computations.

Examples of continuous computations are given by abstract models, such as gen-eral dynamical systems (Bournez, 1999) and hybrid systems (Gupta, et al, 1999), and special computing devices, such as the differential analyzer (Shannon, 1941; Moore, 1996).

Mixed computations include piecewise continuous computations, combining both discrete computation and continuous computation. Examples of mixed computations are given by neural networks (McCulloch and Pitts, 1943), finite dimensional ma-chines and general machines of Blum, Shub, and Smale (1989).

3.5 Hierarchy of levels of computations

In (Burgin and Dodig-Crnkovic, 2011), three generality levels of computations are introduced.

1. On the top and most abstract/general level, computation is perceived as any transformation of information and/or information representation.

2. On the middle level, computation is distinguished as a discretized process of transformation of information and/or information representation.

3. On the bottom, least general level, computation is recognized as a discretized process of symbolic transformation of information and/or symbolic informa-tion representation.

There are also spatial levels or scales of computations:

1. The macrolevel includes computations performed by electromechanical devices, devices based on vacuum tubes and/or transistors.

2. The microlevel includes computations performed by integrated schemas.3. The nanolevel includes computations performed by fundamental parts that

are not bigger than a few nanometers.4. The molecular level includes computations performed by molecules.5. The quantum level includes computations performed by atoms and subatomic

particles.

There are no commercially available nanocomputers, molecular or quantum com-puters in existence at present. However, current chips produced by nanolithography are close to computing nanodevices because their basic elements are less than 100 nanometers in scale.


4 Computational Devices up to Electronic Computers

The oldest computational devices were analog. The earliest calculating tools were fingers (Latin "digit") and pebbles (Latin “calculus”) that can be considered to be simple means of extended human cognition. Tally stick, counting rods and abacus were the first steps towards mechanization of calculation. The historical development led to increasingly more sophisticated machines.

The ancient Greek geared astronomical calculator, Antikythera mechanism, dated to the end of second century BC, was designed to calculate the motions of stars and planets. The device used a differential gear arrangement with over 30 gears, and is re-markable for the complexity comparable to that of 18th century astronomical clocks. (Marchant 2006)

Among the first known constructors of mechanical calculators was Leonardo da Vinci, around year 1500. One notable early calculating machine was Schickard’s cal-culating clock designed in 1623. In 1645 Pascal invented the mechanical calculator Pascaline that could add and subtract two numbers directly, and multiply and divide by repetition. Leibniz improved the work of Pascal by adding direct multiplication and division to his calculator the Stepped Reckoner, about 1673. Describing to astronomers the value of his calculating machine, Leibniz argued:

“It is unworthy of excellent men to lose hours like slaves in the labour of calculation which could safely be relegated to anyone else if machines were used”. Leibniz, as quoted in D. E. Smith, A Source Book in Mathematics (1929), see (Blok and Downey 2003)

Traditionally, computation was understood as synonymous with calculation. The first computing machines were thus constructed simply to calculate. The first recorded use of the word "computer" was in 1613 to denote a person who carried out calculations, or computations, and the word retained the same meaning until the middle of the 20th century. From the end of the 19th century, the word "computer" started to assume its current meaning, describing a machine that performs computations.

In 1837 Babbage was the first to design a programmable mechanical computer, the general purpose Analytical Engine, based on Jacquard's punched cards for its program storage. 1880 Hollerith was the first to use a punched-card system for data storage – a technology that he sold to the company that will later became IBM.

The first electronic digital computer was built in 1939 by Atanasoff and Berry and marks the beginning of the era of digital computing. In 1941 Zuse designed the first programmable computer Z3 capable of solving complex equations. This machine was also the first based on the binary system instead of the earlier used decimal system. 1943 a special-purpose electronic machine Colossus was built by Turing to decode secret messages by performing logical and not usual arithmetical operations. In 1950, the UNIVAC was the first computer that was capable of storing and running a pro-gram from memory. The first minicomputer PDP was built in 1960 by DEC. Since 1960s the extremely fast growth of computer use was based on the technology of in-tegrated circuit (or microchip), which triggered the invention of the microprocessor, by Intel in 1971. For the details of historical developments of computing devices, the


interested reader is referred to the e.g. The History of Computing Project web page. http://www.thocp.net/index.html.

5 Beyond Contemporary Conventional Computing Machinery: Natural Computing

The development of computing, both machinery and its models of course cannot stop at the level of machinery and theoretical understanding that we have today – progress continues. We are used to quick increase of computational power, memory and usab-ility of our computers, but the limit of miniaturization is approaching as we are get -ting close to quantum dimensions of hardware. Moreover, parallelism of networked, communicating asynchronous computational processes calls for new models, more suitable than Turing machines. There is a development of cognitive computing based on human abilities to process/organize/understand information. At the same time computational modelling of human brain aims to reveal the exact mechanisms of hu-man brain function that will help us understand not only how humans actually per-form symbol processing when they follow an algorithm, but also how humans create algorithms or models. Those new developments belong to research within the field of natural computing. According to the Handbook of Natural Computing (Rozenberg et al. 2012) Natural Computing is “the field of research that investigates both human-de-signed computing inspired by nature and computing taking place in nature.” In partic-ular, the book addresses:

Computational models inspired by the natural systems such as neural compu-tation, evolutionary computation, cellular automata, swarm intelligence, arti-ficial immune systems, artificial life systems, membrane computing and amorphous computing.

Computation performed by natural materials such as bioware in molecular computing or quantum-mechanical systems in case of quantum computing.

Computational nature of processes taking place in (living) nature, such as: self-assembly, developmental processes, biochemical reactions, brain pro-cesses, bionetworks and cellular processes.

The Handbook covers the following: Cellular Automata and Neural Computation; Evolutionary Computation; Molecular Computation and Quantum Computation; and Nature-Inspired Algorithms and Alternative Models of Computation.

In the next section we will come back to the development of models of computa-tion. Important in the context of research in Natural Computing is that knowledge is generated bi-directionally, through the interaction between computer science and the natural sciences. While the natural sciences are rapidly absorbing ideas, tools and methodologies of information processing, computer science is broadening the notion of computation, recognizing information processing found in nature as natural compu-tation. (Rozenberg and Kari 2008) (Stepney et al. 2005) (Stepney et al. 2006) This new concept of computation allows for nondeterministic complex computational sys-tems with self-* properties (self-organization, self-configuration, self-optimization,


self-healing, self-protection, self-explanation, and self-awareness. Natural computa-tion understood as information processing provides a basis for a unified understanding of phenomena of embodied cognition, intelligence and knowledge generation. (Dodig Crnkovic and Müller 2009)

6 The Development of Computational Models

As we have seen in Section 4, computational machinery evolved historically from mechanical computers (calculators) to electronic machines with vacuum tubes and then transistors, and to integrated circuits and eventually microprocessors. During this remarkable development of hardware technologies towards ever smaller, faster and cheaper devices, the computational principles remained similar: an isolated machine calculating a function, executing an algorithm that can be represented by the Turing machine model.

However, since the 1950s computational machinery has been used to exchange in-formation and computers gradually started to connect in networks and communicate. In the 1970s ARPANET was used by US research institutions to link their computers via telecommunications.

The emergence of networking involved a redefinition of the nature and boundaries of a computer. Computer operating systems and applications were modified to be able to access the resources of other computers on the network. European organization for particle physics CERN created the World Wide Web in 1991, resulting in computer networking becoming a part of everyday life for people. By the end of 2011 an estimated 35% of Earth's population used the Internet (according to Wikipedia article Global Internet usage).

With the development of computer networks, two important characteristics of computing systems have become increasingly important: parallelism and openness – both based on communication between computational units.

Hewitt [2] characterizes the Turing machine model as an internal (individual) framework and the Actor model of concurrent computation as an external (sociologi-cal) model of computing.

“If computation is understood as a physical process, if nature computes with physical bodies as objects (informational structures) and physical laws govern process of computation, then the computation necessarily appears on many differ-ent levels of organization. Natural sciences provide such a layered view of nature. One sort of computation process is found on the quantum-mechanical level of ele-mentary particles, atoms and molecules; yet another on the level of classical phys-ical objects. In the sphere of biology, different processes (computations = informa-tion processing) are going on in biological cells, tissues, organs, organisms, and eco-systems. Social interactions are governed by still another kind of communica-tive/interactive process. If we compare this to physics where specific “force carri-ers” are exchanged between elementary particles, here the carriers can be complex


chunks of information such as molecules or sentences and the nodes (agents) might be organisms or groups—that shows the width of a difference.” [3]

6.1 Computation as Interaction and Concurrent Interactive Computing

Interactive computation (Wegner 1998) involves interaction, or communication, with the environment during computation, contrary to traditional algorithmic compu-tation which goes on in an isolated system. The interactive paradigm includes concur-rent and reactive computations, agent-oriented, distributed and component-based computations, (Goldin and Wegner 2002).

The paradigm shift from algorithms to interactive computation follows the technol-ogy shift from mainframes to networks, and intelligent systems, from calculating to communicating, distributed and often even mobile devices. A majority of the comput-ers today are embedded in other systems and they are continuously communicating with each other and with the environment. The communicative role has definitely out-weighed the original role of a computer as an isolated, fast calculating machine.

The following characteristics distinguish this new, interactive notion of computa-tion (Goldin, Smolka and Wegner eds. 2006):

- Computational problem is defined as performing a task, rather than (algorithmi-cally) producing an answer to a question.

- Dynamic input and output modeled by dynamic streams which are interleaved; later values of the input stream may depend on earlier values in the output stream and vice versa.

- The environment of the computation is a part of the model, playing an active role in the computation by dynamically supplying the computational system with the in-puts, and consuming the output values from the system.

- Concurrency: the computing system (agent) computes in parallel with its environ-ment, and with other agents that may be in it.

- Effective non-computability: the environment cannot be assumed to be static or effectively computable; it may include humans, or other elements of the real world. We cannot always pre-compute input values or predict the effect of the system's out-put on the environment.

Even though practical implementations of interactive computing are several decades old, a foundational theory, and primarily the semantics and logic of interac-tive computing is only in its beginnings. A theoretical foundations analogous to what Turing machines are for algorithmic computing, is under development (Wegner 1998, Abramsky 2003). Hewitt [2]

Computational logic is a tool that both supports computation modeling and reason-ing about computation. (Goldin and Wegner 2002) argue e.g. that computational logic must be able to model interactive computation, that classical logic does not suffice and that logic must be paraconsistent, able to model both a fact and its negation, due to the role of the environment and incompleteness of interaction.


If the semantics for the behavior of a concurrent system is defined by the func-tional relationship between inputs and outputs, as within the Church-Turing frame-work, then the concurrent system can be simulated by a Turing machine. The Turing machine is a special case of a more general computation concept.

The added expressiveness of a concurrent interactive computing may be seen as a consequence of the introduction of time within the perspective. Time seen from a sys-tem is defined through the occurrence of external events, i.e. through interaction with the environment. In a similar way, spatial distribution, (between an inside and an out-side of the system, also between different systems) gets its full expression through in -teraction. Different distributed agents, with different behaviors, interact with different parts of the environment. In interactive computing, time distribution and generally also (time-dependent) spatial distribution are modeled in the same formalism (Milner 1989) and (Wegner 1998).

The advantages of concurrency theory in the toolbox of formal models used to sim-ulate observable natural phenomena are according to (Schachter 1999) that:

“it is possible to express much richer notions of time and space in the concurrent interactive framework than in a sequential one. In the case of time, for example, in-stead of a unique total order, we now have interplay between many partial orders of events--the local times of concurrent agents--with potential synchronizations, and the possibility to add global constraints on the set of possible scheduling. This requires a much more complex algebraic structure of representation if one wants to "situate" a given agent in time, i.e., relatively to the occurrence of events originated by herself or by other agents.“

Theories of concurrency are partially integrating the observer into the model by permitting limited shifting of the inside-outside boundary. By this integration, theo-ries of concurrency might bring major enhancements to the computational expressive toolbox.

“An important quality of Petri’s conception of concurrency, as compared with “linguistic” approaches such as process calculi, is that it seeks to explain fundamental concepts: causality, concurrency, process, etc. in a syntax-independent, “geometric” fashion. Another important point, which may originally have seemed merely eccen-tric, but now looks rather ahead of its time, is the extent to which Petri’s thinking was explicitly influenced by physics (…).

Living systems are essentially open and in constant communication with the envi-ronment. New computational models must be interactive, concurrent, and asynchro-nous in order to be applicable to biological and social phenomena and to approach richness of their information processing repertoire.

Present account of models of computation highlights several topics of importance for the development of new understanding of computing and its role: natural compu-tation and the relationship between the model and the physical implementation, inter-activity as fundamental for computational modeling of concurrent information pro-cessing systems (such as living organisms and their networks), and new developments in logic needed to support this generalized framework. Computing understood as in-formation processing is closely related to natural sciences; it helps us recognize con-


nections between sciences, and provides a unified approach for modeling and simulat-ing of both living and non-living systems. [4]

6.2 Concurrency vs. Symbolic Computation of Function Values

In his article: What is computation? Concurrency versus Turing's Model, Hewitt [2] makes the following very apt analysis of the relationship between Turing ma-chines and concurrent computing processes:

“Concurrency is of crucial importance to the science and engineering of com-putation in part because of the rise of the Internet and many-core architectures. However, concurrency extends computation beyond the conceptual framework of Church, Gandy [1980], Gödel, Herbrand, Kleene [1987], Post, Rosser, Sieg [2008], Turing, etc. because there are effective computations that cannot be per-formed by Turing Machines. In the Actor model [Hewitt, Bishop and Steiger 1973; Hewitt 2010], computation is conceived as distributed in space where computa-tional devices communicate asynchronously and the entire computation is not in any well-defined state. (An Actor can have information about other Actors that it has received in a message about what it was like when the message was sent.) Tur-ing's Model is a special case of the Actor Model.” Hewitt [2] (emphasis added)

According to natural computationalism/pancomputationalism [4] every physical system is computational, but there are many different sorts of computations going on in nature seen as a network of agents/actors exchanging ”messages”. The simplest agents communicate with simplest messages such as elementary particles (with 12 kinds of matter and 12 anti-matter particles) exchanging 12 kinds of force-communi-cating particles. Next example from physics is Yukawa’s theory of strong nuclear force modeled as exchange of mesons, which explained the interaction between nu-cleons. Complex agents like humans communicate through languages which use very complex messages for communication.

Natural computational systems as networks of agents exchanging messages are in general asynchronous concurrent systems. Conceptually, agent-based models and ac-tor model are closely related, and as mentioned, understanding of interactions (forces) between elementary particles as exchanges of elementary particles fits well in those frameworks thus connecting the simplest and the most complex agent systems into a common framework.

6.3 Physical Computing - New Computationalism. Embodied Networks and Symbolic Representation

Trenholme [32] describes the relationship of analog vs. symbolic simulation:


“Symbolic simulation is thus a two-stage affair: first the mapping of inference structure of the theory onto hardware states which defines symbolic computation; second, the mapping of inference structure of the theory onto hardware states which (under appropriate conditions) qualifies the processing as a symbolic simu-lation. Analog simulation, in contrast, is defined by a single mapping from causal relations among elements of the simulation to causal relations among elements of the simulated phenomenon.” Trenholme [32] p.119.

Both symbolic and sub-symbolic (analog) simulations depend on causal/analog/physical and symbolic type of computation on some level but in the case of symbolic computation it is the symbolic level where information processing is observed . Simi-larly, even though in the analog model symbolic representation exists at some high level of abstraction, it is the physical agency and its causal structure that define com-putation (simulation).

Freeman characterizes accurately the relationship between physical/sub-symbolic and logical/symbolic level in the following:

“Human brains intentionally direct the body to make symbols, and they use the symbols to represent internal states. The symbols are outside the brain. Inside the brains, the construction is effected by spatiotemporal patterns of neural activity that are operators, not symbols. The operations include formation of sequences of neural activity patterns that we observe by their electrical signs. The process is by neurodynamics, not by logical rule-driven symbol manipulation. The aim of simu-lating human natural computing should be to simulate the operators. In its simplest form natural computing serves for communication of meaning. Neural operators implement non-symbolic communication of internal states by all mammals, in-cluding humans, through intentional actions. (…) I propose that symbol-making operators evolved from neural mechanisms of intentional action by modification of non-symbolic operators.“ [33] (Emphasis added)

Consequently, our brains use non-symbolic computing internally in order to manip-ulate relevant external symbols/objects!

The central characteristics of new forms of natural computers, accompanied with the generalized concept of computing, is a BEHAVIOR. As long as computers only calculated functions, behavior was fixed. Nowadays however computing machinery can respond with variety of behaviors.

Now the question of behavior becomes increasingly important in many fields of computation such as embedded systems, robotics, (especially cognitive robotics), con-trol and automation systems or generally any type of application that directly commu-nicates with the physical world.

Behavior (as defined in Wiki) refers to the actions of a system or organism, usually in relation to its environment, which includes the other systems or organisms around as well as the physical environment. It is the response of the system or organism to


various stimuli or inputs, whether internal or external, conscious or subconscious, overt or covert, and voluntary or involuntary.

Behavior as used in computing denotes activities carried out by a computer, computer application, or computer code in response to stimuli, such as user input, sensor data or input from some other computing device.


7 Developments and Prospects of Natural Computation. Computing as Natural Science

When we talk about natural computation by “nature” we mean everything that physically exists – not only living organisms, animals, plants and microorganisms, ge-ological formations, astronomical objects but also machines, humans and human soci-eties (understood as physical systems) – in other words all that can be described as ex-isting in terms of matter/energy and space/time. On different levels of physical orga-nization we find different types of natural computation: on quantum level, there is quantum computation going on in nature, on the molecular level there is molecular computation, higher up in hierarchy we find nano-computation, networks of proteins are computing in living organisms, DNA code governs variety of computational pro-cesses in cells, metabolic processes are at the same time information processing and they are constitutive of life and autopoiesis (Maturana and Varela equate cognition with life), nervous systems computations resemble neural network models, living or-ganisms as wholes are regulated on variety of levels and so are ecologies. All that in-formation processing going on in physical world can ultimately be represented as computation – some of it performed on continuous flow of signals, some of it on dis-crete signals or symbols, some within living agents without conscious control, while other like symbol manipulation in different languages require living organisms for in-formation of that complexity to be processed.

7.1 Physical Computation/Natural Computation vs. Turing Machine Model

So in what way is physical computation/natural computation important vis-à-vis Turing machine model? One of the central questions within computing, cognitive sci-ence, AI and other related fields is about computational modeling (and simulating) of intelligent behaviour. What can be computed and how? It has become obvious that we must have richer models of computation, beyond Turing machines, if we are to effi-ciently model and simulate biological systems. What exactly can we learn from nature and especially from intelligent organisms?

It has taken more than sixty years from the first proposal of Turing test he called the ”Imitation Game", described in Turing [34] p. 442, to the Watson machine win-ning Jeopardy. That is just the beginning of what Turing believed one day will be pos-sible - a construction of computational machines capable of generally intelligent be-havior as well as the accurate computational modeling of the natural world. So there are several classes of problems that deserve our attention when talking about comput-ing nature.

To ”compute” nature by any kind of computational means, is to model and/or simu-late the behaviors of natural systems by computational means. Watson is a good ex-ample. We know that we do not function like Watson or like chess playing programs that take advantage of brute force algorithms to search the space of possible states.


We use our ”gut feeling” and ”fingertip-feeling”/ ”fingerspitzengefühl” that can be understood as embodied, physical, sub-symbolic information processing mechanisms we acquire by experience and use when necessary as automatized hardware-based recognition tools.

To compute nature means to interpret natural processes, structures and objects as a result of natural computation which is in general defined as information processing. This implies understanding and modeling of physical agents, starting from the very fundamental level of quantum computing via several emergent levels of chemistry, bi-ology, cognition and extended cognition (social and augmented by computational ma-chinery).

At the moment we have bits and pieces of the picture – COMPUTING nature, that is computational modeling of nature and computing NATURE, that is nature under-stood in itself as a computational network of networks.

7.2 The Unreasonable Ineffectiveness of Mathematics in Biology, Mathem-aticians Bias and How to Advance

Mathematician’s contribution to the development of the idea of computing nature is central. Turing as an early proponent of natural computing put forward a machine model that is still in use. How far can we hope to go with Turing machine model of computation?

In the context of computing nature, living systems are of extraordinary importance as up to now science haven’t been able to model and simulate the behavior of even the simplest living organisms. “The unreasonable effectiveness of mathematics” ob-served in physics (Wigner) is missing for complex phenomena like biology that today lack effective mathematical models (Gelfand), see Chaitin [11].

Not many people today would claim that human cognition (information processing going on in our body, including thinking) can be adequately modeled as a result of computation of one Turing machine, however complex function it might compute. In the next attempt, one may imagine a complex architecture of Turing machines run-ning in parallel as communicating sequential processes (CSPs) exchanging informa-tion. We know today that such a system of Turing machines cannot produce the most general kind of computation, as truly asynchronous concurrent information processing going on in our brains.

However, one may object that IBM’s super-computer Watson, the winner in man vs. machine "Jeopardy!" challenge, runs on contemporary (super)computer which is claimed to be implementation of the Turing machine. Yet, Watson is connected to the Internet. And Internet is not a Turing machine equivalent. It is not even a network of Turing machines. Information processing going on throughout the entire Internet in-cludes signaling and communication based on complex asynchronous physical pro-cesses that cannot be sequentionalized. (Hewitt, Sloman) As an illustration see Barabási et al. article [12] on parasitic computing that implements computation on the communication infrastructure of the Internet, thus using communication for computa-tion.


If we want to generalize the idea of computation so to be able to encompass more complex operations than mechanical execution of an algorithm, simulating not only a person executing strictly mechanical procedure, but the one constructing a new the-ory, we must go back to underlying mathematics.

Dodig-Crnkovic and Burgin, in [ ] analyze methodological and philosophical im-plications of algorithmic aspects of unconventional/natural computation that extends the closed classical universe of computation of the Turing machine type. The new model constitute an open world of algorithmic constellations, allowing increased flex-ibility and expressive power, supporting constructivism and creativity in mathemati-cal modeling and enabling richer understanding of computation, see [14]. (Dodig-Crnkovic and Burgin 2012, Entropy)

Cooper in his article Turing's Titanic Machine? [17] diagnoses the limitations of the Turing machine model and identifies the following ways of overcoming those lim-itations, by introducing:

Embodiment invalidating the `machine as data' and universality paradigm. The organic linking of mechanics and emergent outcomes delivering a

clearer model of supervenience of mentality on brain functionality, and a reconciliation of different levels of effectivity.

A reaffirmation of experiment and evolving hardware, for both AI and ex-tended computing generally.

The validating of a route to creation of new information through interaction and emergence.

Related article by the same author elucidates the role of physical computation vs. universal symbol manipulation: The Mathematician's Bias and the Return to Embod-ied Computation, in [18].

The underlying logic of Turing’s “logical calculating machine” is fully consistent standard logic. Turing machine is assumed always to be in a well defined state. [2]

In contemporary computing machinery, however, we face both states that are not well defined (in the process of transition) and states that contain inconsistency:

“Consider a computer which stores a large amount of information. While the computer stores the information, it is also used to operate on it, and, crucially, to infer from it. Now it is quite common for the computer to contain inconsistent in-formation, because of mistakes by the data entry operators or because of multiple sourcing. This is certainly a problem for database operations with theorem-provers, and so has drawn much attention from computer scientists. Techniques for removing inconsistent information have been investigated. Yet all have limited applicability, and, in any case, are not guaranteed to produce consistency. (There is no algorithm for logical falsehood.) Hence, even if steps are taken to get rid of


contradictions when they are found, an underlying paraconsistent logic is desirable if hidden contradictions are not to generate spurious answers to queries.” Priest and Tanaka [22]

Open, interactive and asynchronous systems have special requirements on logic. Goldin and Wegner [23], and Hewitt [2] argue e.g. that computational logic must be able to model interactive computation, and that classical logic must be robust towards inconsistencies i.e. must be paraconsistent due to the incompleteness of interaction.

As Sloman [24] argues, concurrent and synchronized machines are equivalent to sequential machines, but some concurrent machines are asynchronous, and thus not equivalent to Turing machines. If a machine is composed of asynchronous concur-rently running subsystems, and their relative frequencies vary randomly, then such a machine cannot be adequately modeled by Turing machine, see also [4].

Turing machines are discrete but can in principle approximate machines with con-tinuous changes, yet cannot implement them exactly. Continuous systems with non-linear feedback loops may be chaotic and impossible to approximate discretely, even over short time scales, see [26] and [2]. Clearly Turing machine model of computa-tion is an abstraction and idealization. In general, instead of idealized, symbol-manip-ulating models, more and more physics-inspired modeling is taking place.

Theoretical model of concurrent (interactive) computing corresponding to Turing machine model of algorithmic computing is under development. (Abramsky, Hewitt, Wegner) From the experience with present day networked concurrent computation it becomes obvious that Turing machine model can be seen as a special case of a more general computation. During the process of learning from nature how to compute, we both develop computing and at the same time improve understanding of natural phe-nomena.

7.3 Unconventional Algorithms: Complementarity of Axiomatics and Con-struction

In this paper, we analyze axiomatic and constructive issues of unconventional compu-tations from a methodological and philosophical point of view. We explain how the new models of algorithms and unconventional computations change the algorithmic universe, making it open and allowing increased flexibility and expressive power that augment creativity. At the same time, the greater power of new types of algorithms also results in the greater complexity of the algorithmic universe, transforming it into the algorithmic multiverse and demanding new tools for its study. That is why we analyze new powerful tools brought forth by local mathematics, local logics, logical varieties and the axiomatic theory of algorithms, automata and computation. We demonstrate how these new tools allow efficient navigation in the algorithmic multi-verse. Further work includes study of natural computation by unconventional al-gorithms and constructive approaches. (Gordana Dodig Crnkovic and Mark Burgin, Entropy 2012)


7.4 Interactive Computation vs. Algorithmic Computation. Open vs. Computational Closed Systems

In this new view of computation as found in natural world, an open computational system (an agent or a system of agents) communicates with its environment by mes-sage passing (information exchange). Thus we have the following:

Fig. 4. The Computational Dyad of Physical Computation / Natural Comutation

An underlying assumption is that there are entities (agents) responsible for this communication/interaction. This model can be seen as agent computing, where com-putation is modeled as message exchange (information exchange) between agents.

Computation is a process that is a result of exchanging messages among agents.

Fig. 5. The Process- centred Computational Triad

From a point of view of an agent, it performs a function or searches the goal in a given context (environment).

Fig. 6. The Agent-centred Computational Triad

Very simple agents in nature can be ascribed very simple “goals”, while complex agents like living organisms have non-trivial function and goals.


8 Conclusions

We need much better understanding of information processing and its primary form – computation than we have now. As there is no information without (physical) repres-entation, the dynamics of information is implemented on different levels of granular-ity by different physical processes, including the level of computation performed by computing machines and living organisms. There are a lot of open problems related to the nature of information and essence of computation, as well as to their relationships. How is information dynamics represented in computational systems, in machines, as well as in living organisms? Are computers processing only data or information and knowledge as well? What do we know of computational processes in machines and living organisms and how these processes are related? What can we learn from natural computational processes that can be useful for information systems and knowledge management?

“The increased interactivity and connectivity of computational devices along with the spreading of computational tools and computational thinking across the fields, has changed our understanding of the nature of computing. In the course of this develop-ment computing models have been extended from the initial abstract symbol manipu-lating mechanisms of stand-alone, discrete sequential machines, to the models of nat-ural computing in the physical world, generally concurrent asynchronous processes capable of modelling living systems, their informational structures and dynamics on a symbolic and sub-symbolic information processing levels.

Present account of models of computation highlights several topics of importance for the development of new understanding of computing and its role: natural compu-tation and the relationship between the model and physical implementation, inter-activity as fundamental for computational modelling of concurrent information pro-cessing systems such as living organisms and their networks, and the new develop-ments in modelling needed to support this generalized framework. Computing under-stood as information processing is closely related to natural sciences; it helps us re -cognize connections between sciences, and provides a unified approach for modeling and simulating of both living and non-living systems.” M&M

1 References

1. Blum, L., Cucker, F., Shub, M., and Smale, S. (1998) Complexity of Real Computation, Springer, New York

2. Bournez, O. (1999) Achilles and the tortoise climbing up the hyper-arithmetical hierarchy, Theoretical Computer Science, v. 210, No.1, pp. 21-71

3. Bucci, W. (1997). Psychoanalysis and Cognitive Science: A multiple code theory. NY: Guilford Press.4. Burgin, M. (1990) Hypermeasures and hyperintegration, Notices of the National Academy of Sciences

of Ukraine, v. 6, pp. 10-13 (in Russian and Ukrainian)5. Burgin, M. (2005) Super-recursive Algorithms, Springer, New York/Heidelberg/Berlin 6. Burgin, M. (2012) Structural Reality, Nova Science Publishers, New York 7. Burgin, M. and Dodig-Crnkovic, G. (2011) Information and Computation – Omnipresent and Pervasive,

in Information and Computation, in Information and Computation, World Scientific, New York/Lon-don/Singapore

8. Burgin, M. and Eberbach, E. Evolutionary Computation and Processes of Life (Ubiquity Symposium Opening Statement), Ubiquity, August 2012


9. Burks, Arthur W. (1961) Computation, behavior, and structure in fixed and growing automata: Behav Sci. Jan; 6:5-22.

10. Cantor, G.: Grundlagen einer allgemeinen Mannigfaltigkeitslehre: Ein mathematisch-philosophischer Versuch in der Lehre des Unendlichen, Teubner, Leipzig (1883)

11. Conery, J.S. Computation is symbol manipulation, Ubiquity, November 201012. Conway, J.H. (1976) On Numbers and Games, Academic Press, London/ New York 13. Copeland, J. (1996) What is computation? Synthese, v. 108, No. 3, pp 335-35914. Denning, P. (2010) What is computation? Opening Statement, Ubiquity 15. Dodig-Crnkovic, G. (2009) Information and Computation Nets. Investigations into Info-computational

World, VDM Verlag16. L. Fortnow. 2010. The enduring legacy of the Turing machine. Ubiquity 2010, December, pages.

DOI=10.1145/1895419.1921573 http://doi.acm.org/10.1145/1895419.1921573 17. Goldin, D. Smolka, S. and Wegner P., Eds. (2006). Interactive Computation: The New Paradigm,

Springer-Verlag18. Gupta, V., Jagadeesan, R., and Saraswat, V.A. (1999) Computing with Continuous Change, Science of

Computer programming, v. 30, No. 1/2, pp. 3-4919. Kolmogorov, A.N. (1953) On the Concept of Algorithm, Russian Mathematical Surveys, v. 8, No. 4,

pp. 175-17620. Blok A. and Downey G. J., Uncovering Labour in Information Revolutions, 1750-2000 (2003), 93.21. McCulloch, W. S., and Pitts, E. (1943) A Logical Calculus of the Ideas Immanent in Nervous Activity,

Bulletin of Mathematical Biophysics, v. 5, pp.115-13322. Jo Marchant In search of lost time, Nature 444, #7119 (30 November 2006), pp. 534–538 http://

www.nature.com/nature/journal/v444/n7119/full/444534a.html 23. Moore, C. (1996) Recursion Theory on the Reals and Continuous-time Computation: Real numbers and

computers, Theoretical Computer Science, v. 162, No. 1, pp. 23-44 24. Robinson, A. (1961) Non-standard analysis, Indagationes Mathematicae, v. 23, pp. 432-440 25. Shannon, C. (1941) Mathematical Theory of the Differential Analyzer, J. Math. Physics, MIT, v. 20, pp.

337-354 26. Turing, A. (1936) On Computable Numbers with an Application to the Entscheidungs-problem, Proc.

Lond. Math. Soc., Ser.2, v. 42, pp. 230-265

27. Dodig-Crnkovic G., Significance of Models of Computation from Turing Model to Natural Computa-tion. Minds and Machines, Volume 21, Issue 2 (2011), pp. 301-322. http://www.springerlink.com/open-url.asp?genre=article&id=doi:10.1007/s11023-011-9235-1

28. Dodig-Crnkovic G., The Info-computational Nature of Morphological Computing, in Müller V. C. (ed.), Theory and Philosophy of Artificial Intelligence (SAPERE; Berlin: Springer), 2012. (Selected contribu-tions from PT-AI conference @ACT) pp. 59-68.

29. Dodig-Crnkovic G., Info-computationalism and Morphological Computing of Informational Structure, in Integral Biomathics, Simeonov, P., Smith, L. and Ehresmann, A. (Eds.). Springer Serie on Computa-tional Intelligence and Complexity, Table of Contents, 2012. (Selected contributions from Conference on Integral Biomathics Stirling University, Scotland), pp. 97-105.

30. Dodig-Crnkovic G. and Burgin M., Unconventional Algorithms: Complementarity of Axiomatics and Construction, Special issue of the journal Entropy, 14(11), 2066-2080; "Selected Papers from the Sym-posium on Natural/Unconventional Computing and its Philosophical Significance", doi:10.3390/e14112066 http://www.mdpi.com/journal/entropy/special_issues/unconvent_computing, 2012.

31. Dodig-Crnkovic G. and Giovagnoli R. (Eds) COMPUTING NATURE, (book introduction) Springer, Heidelberg, SAPERE book series, forthcoming 2013. http://arxiv.org/abs/1210.7784

32. Dodig-Crnkovic G. and Giovagnoli R., Natural/Unconventional Computing and its Philosophical Signi-ficance, Special issue of the journal Entropy, 14, 2408-2412; doi:10.3390/e14122408 http://www.mdpi.-com/journal/entropy/special_issues/unconvent_computing, 2012.

14. Dodig Crnkovic G. and Burgin M.: Unconventional Algorithms: Complementarity of Axiomatics and Construction, Entropy, 14(11), 2066-2080; doi:10.3390/e14112066 http://www.mdpi.com/journal/entropy/special_issues/unconvent_computing, 2012

http://www.nature.com/nature/journal/v444/n7119/full/444534a.html

http://www.nature.com/nature/journal/v444/n7119/full/444534a.html


15. Burgin M. and Dodig Crnkovic G., From the Closed Classical Algorith-mic Universe to an Open World of Algorithmic Constellations, In: COM-PUTING NATURE, Springer, Heidelberg, SAPERE book series, forth-coming 2013. arXiv:1211.4547

17. Cooper S. B.: Turing's Titanic Machine? Communications of the ACM, Vol. 55 No. 3, Pages 74-83. (2012)

18. Zenil H. (Ed.): A Computable Universe, Understanding Computation & Exploring Na-ture As Computation, World Scientific Publishing Company/Imperial College Press, (2012)

Angelaki DE, Shaikh AG, Green AM, Dickman JD. Neurons compute internal models of the physical laws of motion. Nature. 2004 Jul 29; 430(6999):560-4

Marcia Barinaga A New Look at How Neurons Compute, Science 29 September 2000: Vol. 289 no. 5488 pp. 2256-2257

ALL LANGUAGES CANNOT BE DESCRIBED BY ALGORITHMS.

All languages form a powerset a set of sets of strings which is uncountable

Turing machines are countable There are infinitely many more languages than Turing machines! There are some languages not accepted by Turing Machines. All languages cannot be described by algorithms.

As far as I understand that depends on the difference between two infinities – set of natural numbers (that TM as finite effective procedures place) and set of real number where powerset can be found.

Moreover, real numbers have the same cardinality as the power set of natural numbers. Using the Cantor–Bernstein–Schröder theorem, it is easy to prove that there exists a bijection between the set of reals and the power set of the natural numbers. The cardinality of the continuum is

http://en.wikipedia.org/wiki/Power_set

http://arxiv.org/abs/1211.4547

http://arxiv.org/ftp/arxiv/papers/1211/1211.4547.pdf

http://arxiv.org/ftp/arxiv/papers/1211/1211.4547.pdf

Documents

Titel - Research in Innovation, Design and Engineering ... · Web viewConcurrent computations, which do not demand synchronization in time. Note that while parallel computation is