Complex Systems Engineering SwE 488 Artificial Complex Systems Prof. Dr. Mohamed Batouche Department of Software Engineering CCIS – King Saud University

Prof. Dr. Mohamed Batouche

Department of Software EngineeringCCIS – King Saud University

Riyadh, Kingdom of Saudi [email protected]

Cellular Automata

2

Purpose

3

• In Theory:• Computation of all computable functions• Construction of (also non-homogenous)

automata by other automata, the offspring being at least as powerful (in some well-defined sense) as the parent

• In Practice: • Exploring how complex systems with

emergent patterns seem to evolve from purely local interactions of agents. I.e. Without a “master plan!”

Cellular Automata• A cellular automata is a family of

simple, finite-state machines that exhibit interesting, emergent behaviors through their interactions in a population

4

The famous BOIDS model shows how flocking behavior can emerge from a collection of agents following a few simple rules.

Emergent Behavior

5

• Original concept of CA is most strongly associated with John von Neumann who was interested in the connections between biology and the new study of automata theory

• Stanislaw Ulam suggested to von Neumann the use a cellular automata as a framework for researching these connections.

• The original concept of CA can be credited to Ulam, while the early development of the concept is credited to von Neumann.

• Ironically, although von Neumann made many contributions and developments in CA, they are commonly referred to as “non-von Neumann style”, while the standard model of computation (CPU, globally addressable memory, serial processing) is know as “von Neumann style”.

6

7

Cellular Automata (CAs)

• Have been used as:• massively parallel computer architecture• model of physical phenomena (Fredkin, Wolfram)

• VLSI Testing• Data Encryption• Error Correcting Code Correction• Testable Synthesis• Generation of hashing Function

• Currently being investigated as model of quantum computation (QCAs)

• Grid• Mesh of cells.• Simplest mesh is one dimensional.

• Cell• Basic element of a CA.• Cells can be thought of as memory

elements that store state information.• All cells are updated synchronously

according to the transition rules.

• Rules 8

• Local interaction leads to global dynamics.• One can think of the behavior of a

cellular automata like that of a “wave” at a sports event.

• Each person reacts to the state of his neighbors (if they stand, he stands).

9

• Rule Application• Next state of the core cell is related to the states

of the neighboring cells and its current state.• An example rule for a one dimensional CA: 011-

>x0x• All possible states must be described.

• Next state of the core cell is only dependent upon the sum of the states of the neighboring cells.• For example, if the sum of the adjacent cells is

4 the state of the core cell is 1, in all other cases the state of the core cell is 0.

10

11

Structure• Discrete space (lattice) of regular cells

• 1D, 2D, 3D, …• rectangular, hexagonal, …

• At each unit of time a cell changes state in response to (Time advances in discrete steps):• its own previous state • states of neighbors (within some “radius”)

• All cells obey same state update rule, depending only on local relations• an FSA

• Synchronous updating (parallel processing)

Structure: Neighborhoods

12

1-DIMENSIONAL AUTOMATA

13

One-Dimensional CA’s

• Game of Life is 2-D. Many simpler 1-D CAs have been studied

• For a given rule-set, and a given starting setup, the deterministic evolution of a CA with one state (on/off) can be pictured as successive lines of colored squares, successive lines under each other

14

Neighborhoods

15

• 3 Black = White• 2 Black = Black• 1 Black = Black• 3 White = White

Now make your own CA 16

http://www.ncf.edu/patterns/CreateYourOwnCellularAutomaton.doc

“A New Kind of Science”

Stephen WolframISBN 1-57955-008-8

www.wolframscience.com

http://www.wolframscience.com/

1-D CA Example

18

cell# 1 2 3 4 5 6 7 8 9 10 11 12 13

time = 1 time = 2 time = 3 time = 4 time = 5 time = 6 time = 7

Rules

0 0 1 1 0 1 1 0

Rule# = 2 + 4 + 16 + 32 = 54

Wolfram Model

1 1 1 1 1 1 1 0

Rule #126 = 64 + 32 + 16 + 8 + 4 + 2 + 0 = 126

Most of the rules are degenerate, meaning they create repetitive patterns of no interest.

However there are a few rules which produce surprisingly complex patterns that do not repeat themselves.

1 1 1 1 1 1 0 0

Rule #124 = 64 + 32 + 16 + 8 + 4 + 0 + 0 = 124

19

Wolfram Model

we can view the state of the model at any time in the future as long as we step through all the previous states. 20

21

Hundred generations of Rule 30

CA Example: Rule 30

111 110 101 100 011 010 001 000

0 0 0 1 1 1 1 0

Rule (0 + 0 + 0 + 16 + 8 + 4 + 2 +0 ) = 30

22

Conus Textile pattern

23

The pattern is neither regular nor completely random.

It appears to have some order, but is never predictable. 24

See Demo - NetLogo

Rule #45=32+8+4+1

= 0 27+ 0 26+ 1 25+ 0 24+1 23+ 1 22+ 0 21+ 1 20

=0 0 1 0 1 1 0 1

Rule #30=16+8+4+2

= 0 27+ 0 26+ 0 25+ 1 24+1 23+ 1 22+ 1 21+ 0 20

=0 0 0 1 1 1 1 0

This naming convention of the 256 distinct update rules is due to Stephen Wolfram. He is one of the pioneers of Cellular Automata and author of the book a New Kind of Science, which argues that discoveries about cellular automata are not isolated facts but have significance for all disciplines of science.

Wolfram Model

25

0 1 0 1 1 0 1 0Rule (0 + 2 + 0 + 8 + 16 + 0 + 64) = 90

Wolfram Rule 90

26

Wolfram Rule 110

Proven to be Turing Complete - Rich enough for universal computation

interesting result because Rule 110 is an extremely simple system, simple enough to suggest that naturally occurring physical systems may also be capable of universality

27

Wolfram Rule 99

28Rule# 99 = 0 27 + 1 26 + 1 25 + 0 24 + 0 23 + 0 22 + 1 21 + 1 20

0 + 64 + 32 + 0 + 0 + 0 + 2 + 1

29

Mollusc Pigmentation Patterns

30

Wolfram’s CA classes 1,2

From observation, initially of 1-D CA spacetime patterns, Wolfram noticed 4 different classes of rule-sets. Any particular rule-set falls into one of these:-:

CLASS 1: From any starting setup, pattern converges to all blank -- fixed attractor

CLASS 2: From any start, goes to a limit cycle, repeats same sequence of patterns for ever. -- cyclic attractors

31

Wolfram’s CA classes 3,4

CLASS 3: Turbulent mess, chaos, no patterns to be seen.

CLASS 4: From any start, patterns emerge and continue without repetition for a very longtime (could only be 'forever' in infinite grid)

Classes 1 and 2 are boring, Class 3 is messy, Class 4 is 'At the Edge of Chaos' - at the transitionbetween order and chaos -- where Game of Life is!. 32

2-DIMENSIONAL AUTOMATA

33

2-dimensional cellular automaton consists of an infinite (or finite) grid of cells, each in one of a finite number of states. Time is discrete and the state of a cell at time t is a function of the states of its neighbors at time t-1.

34

Neighborhoods

35

Von Neumann Moore margolus

• Cryptography use, Rule 30• Simulations

• Gas behaviour.• Forest fire propagation.• Urban development.• Traffic flow.• Air flow.• Crystallization process.

• Alternative to differential equations

36

37

Snowflakes

Bead-sort• Bead-Sort is a method of ordering a set of

positive integers by mimicking the natural process of objects falling to the ground, as beads on an abacus slide down vertical rods. The number of beads on each horizontal row represents one of the numbers of the set to be sorted, and it is clear that the final state will represent the sorted set.

38

[10 8 6 2 4 12] [2 4 6 8 10 12]

Bead-sort

39

Bead-sort - CA implementation

40

[10 8 6 2 4 12]

[2 4 6 8 10 12]

[10

8 6

2 4

12

]

[2 4

6 8

10

12

]

See Demo - Netlogo

the natural process of objects falling to the ground

Bead-sort Extended CA implementation

• The Bead-Sort can be modeled by the two-dimensional cellular automaton (CA) rule, .

• For the Antigravity Bead-Sort, we add a second rule: .

41

Bead-sort Extended

42

The "extended" (anti-gravity) mode allows the inclusion of all integers, with "negative beads" rising while "positive beads" fall.

43

Example:Conway’s Game of Life

• Invented by Conway in late 1960s• A simple CA capable of universal

computation• Structure:

• 2D space• rectangular lattice of cells• binary states (alive/dead)• neighborhood of 8 surrounding cells (&

self)• simple population-oriented rule

44


Cell

State = dead/off/0

State = alive/on/1

• A cell dies or lives according to some transition rule

• The world is round (flips over edges)

• How many rules for Life? 20, 40, 100, 1000?

T = 0 T = 1

transition

rules


45

46

State Transition Rule

• Live cell has 2 or 3 live neighbors stays as is (stasis)

• Live cell has < 2 live neighbors dies (loneliness)

• Live cell has > 3 live neighbors dies (overcrowding)

• Empty cell has 3 live neighbors comes to life (reproduction)

47


• Survive with 2 or 3 live neighbors Live cell stays as is (stasis)otherwise dies from loneliness or overcrowding

• Generate with 3 live neighbors Empty cell comes to life (reproduction)

48


Three simple rules:

• dies if number of alive neighbor cells =< 1 (loneliness)

• dies if number of alive neighbor cells >= 4 (overcrowding)

• generate alive cell if number of alive neighbor cells = 3 (procreation)

Examples of the rules

• loneliness (dies if #alive =< 1)

• overcrowding (dies if #alive >= 4)

• procreation (lives if #alive = 3)


49

CA: Discrete Time, Discrete Space

Initial Setup

Number of Neighbors

After Pass 1

After Pass 2

50

T = 0

neighboring values

T = 1

Game of Life: 2D Cellular Automata using simple rules

Emergent pattern: Blinker

51

Emergent patternsConway automaton can simulate a number of different effects that can be found in the evolution of a living population.

Equilibria Oscillation Movement

square

2 steps

diagonal

beehive

2 steps

horizontal

boat

3 steps

instability

(all the space is filled up by horizontal lines)

ship 15 steps

instability

toast

chaos?

52

Game of Life: emergent patterns

53gliders: patterns that moves constantly across the grid

Conway’s Rules: Game of Life

Survive with 2 – 3 living neighbors

Generate with 3 living neighbors

Gosper’s glider gun : emits glider stream

Emergent Patterns

54

Emergent Patterns

Oscillators-objects that change from step to step, but eventually repeat themselves. These include, but are not limited to, period 2 oscillators, including the blinker and the toad.

Blinker

Toad

55

Emergent Patterns: A Clock

56See Demo: Game of Life

Emergent Patterns:


Oscillator

Barber’s Pole

Pulsar SpaceShip

Beacon

Glider

Emergent Patterns: Gosper’s Glider Gun


Emergent Patterns: Puffer train

59

Emergent Patterns: Double-Barreled Gun

60

Emergent Patterns: Edge Shooter

61

Emergent Patterns: Evolution of a breeder ...

62

Game of Life - implications

Typical Artificial Life, or Non-Symbolic AI, computational paradigm:• bottom-up• parallel• locally-determined

Complex behavior from (... emergent from ...) simple rules.

Gliders, blocks, traffic lights, blinkers, glider-guns, eaters, puffer-trains ...

63

Game of Life as a Computer ?

Higher-level units in Game of Life can in principle be assembled into complex 'machines' -- even into a full computer, or Universal Turing Machine.

'Computer memory' held as 'bits' denoted by 'blocks‘ laid out in a row stretching out as a potentially infinite 'tape'. Bits can be turned on/off by well-aimed gliders.

64

Game of Life as a Computer ?

Bits can be turned on/off by well-aimed gliders:

65

This is a Turing Machine implemented in Conway's Game of Life.

http://rendell-attic.org/gol/tm.htm

66

This is a Universal Turing Machine (UTM) implemented in Conway's Game of Life.

Designed by Paul Rendell 10/February/2010.

http://rendell-attic.org/gol/utm/index.htm

67

http://rendell-attic.org/gol/utm/picture.htm

68

Foundations Turing Machine -> Computer

•The theory of computation and the practical application it made possible — the computer — was developed by an Englishman called Alan Turing.

Turing machinesTuring machines (TM’s) were introduced by

Alan Turing in 1936They are more powerful than both finite

automata or pushdown automata. In fact, they are as powerful as any computer we have ever built.

The main improvement from PDA’s is that they have infinite accessible memory (in the form of a tape) which can be read and written to.

Basic design of Turing machine

Control

Infinite tape

Tape head

Usual finite state control

The basic improvement from the previous automata is the infinite tape, which can be read and written to.

The input data begins on the tape, so no separate input is needed (e.g. DFA, PDA).

Turing Machine

. . .

Infinite tape: Γ*Tape head: read current square on tape,

write into current square,move one square left or right

FSM: like PDA (PushDown Automata), except: transitions also include direction (left/right) final accepting and rejecting states

FSM

72

A Turing Machine

............

Tape

Read-Write head

Control Unit (FSM: Finite Sate Machine)

73

The Tape

............

Read-Write head

No boundaries -- infinite length

The head moves Left or Right

74

............

Read-Write head

The head at each time step:

1. Reads a symbol 2. Writes a symbol 3. Moves Left or Right

75

Turing MachineEG Successor Program

Sample Rules:

If read 1, write 0, go right, repeat.If read 0, write 1, HALT!If read �, write 1, HALT!

Let’s see how they are carried out on a piece of paper that contains the reverse binary representation of 47:

76


If read 1, write 0, go right, repeat.If read 0, write 1, HALT!If read , write 1, HALT!

1 1 1 1 0 1

77



0 1 1 1 0 1

78



0 0 1 1 0 1

79

A Thinking MachineEG Successor Program


0 0 0 1 0 1

80



0 0 0 0 0 1

81



0 0 0 0 1 1

82


So the successor’s output on 111101 was 000011 which is the reverse binary representation of 48.

111101 = 1 + 2 + 4 +8 + 0 + 32 = 47000011 = 0 + 0 + 0 + 0 + 16 + 32 =

48

83



1 1 1 1 1 1 1

84



0 1 1 1 1 1 1

85



0 0 1 1 1 1 1

86



0 0 0 1 1 1 1

87



0 0 0 0 1 1 1

88



0 0 0 0 0 1 1

89



0 0 0 0 0 0 1

90



0 0 0 0 0 0 0

91



0 0 0 0 0 0 0 1

The successor of 127 is 128 !!!

03/03/2000 FLAC: Reductions 92

Turing machines as functions

TM

Input word

Output word

f

A function f is a computable function if aTM with word w as input halts with f(w) as output.

w

f(w)


First computers: custom computing machines

1950 -- Eniac: the control is hardwired manually foreach problem.

Control

Input tape (read only)

Output tape (write only)

Work tape (memory)


TMs can be described using textInput tape (read only)

Control

Output tape (write only)

Work tape (memory)

Program•n states•s letters for alphabet•transition function:

•d(q0,a) = (q1, b, L)•d(q0,b) = (q1, a, R)•….

Consequence:There is a countable number of Turing Machines

The Universal Turing Machine

We can build a Turing Machine that can do the job of any other Turing Machine. That is, it can tell you what the output from the second TM would be for any input!

This is known as a universal Turing machine (UTM).

A UTM acts as a completely general-purpose computer, and basically it stores a program and data on the tape and then executes the program.


There is a TM which can simulate any other TM(Alan Turing, 1930)

Input tape

UniversalTM

Output tape

Work tape (memory)

Interpreter

Program


The Digital Computer: a UTM

Machine code

UniversalTM

Output

Memory + disk

Machine-code Interpreter

Input

Palindrome Example - UTM

Figure 2 – Palindrome Example of a TM computation

Palindrome Example


Palindrome Example


Palindrome Example


Palindrome Example


Palindrome Example


Palindrome Example


Palindrome Example


Palindrome Example


Palindrome Example


Palindrome Example


109

Can Machines Think?

•In “Computing machinery and intelligence,” written in 1950, Turing asks whether machines can think.

•He claims that this question is too vague, and proposes, instead, to replace it with a different one.

•That question is: Can machines pass the “imitation game” (now called the Turing test)? If they can, they are intelligent.

•Turing is thus the first to have offered a rigorous test for the determination of intelligence quite generally.

110

Self-reproducing CAsvon Neumann saw CAs as a good framework for studying the necessary and sufficient conditions for self-replication of structures.

von N's approach: self-rep of abstract structures, in the sense that gliders are abstract structures.

His CA had 29 possible states for each cell (compare with Game of Life 2, black and white) and his minimum self-rep structure had some 200,000 cells.

111

Self-rep and DNA

This was early 1950s, pre-discovery of DNA, butvon N's machine had clear analogue of DNA whichis both:used to determine pattern of 'body'

interpretedand itself copied directly

copied without interpretation as a symbol string

Simplest general logical form of reproduction (?)

How simple can you get?

112

Langton’s LoopsLangton’s LoopsChris Langton formulated a much simpler form of self-rep structure - Langton's loops - with only a few different states, and only small starting structures.

[ Further developments -- eg 'Wireworld'. ]

3-Dimensional CA

113

One difficulty with three-dimensional cellular automata is the graphical representation (on two-dimensional paper or screen)

114

Conclusions

115

Conclusions

• This topic is very hot and has widespread implications• Biology• Chemistry• Computer science• Complexity

• We’ve seen the basic concepts …• But we’ve only scratched the surface!

From now on, Think Biology, Emergence, Complex Systems …

References

116

References• Jay Xiong, New Software Engineering Paradigm Based on

Complexity Science, Springer 2011.• Claudios Gros : Complex and Adaptive Dynamical Systems.

Second Edition, Springer, 2011 .• Blanchard, B. S., Fabrycky, W. J., Systems Engineering and

Analysis, Fourth Edition, Pearson Education, Inc., 2006.• Braha D., Minai A. A., Bar-Yam, Y. (Editors), Complex Engineered

Systems, Springer, 2006 • Gibson, J. E., Scherer, W. T., How to Do Systems Analysis, John

Wiley & Sons, Inc., 2007.• International Council on Systems Engineering (INCOSE) website (

www.incose.org).• New England Complex Systems Institute (NECSI) website (

www.necsi.org).• Rouse, W. B., Complex Engineered, Organizational and Natural

Systems, Issues Underlying the Complexity of Systems and Fundamental Research Needed To Address These Issues, Systems Engineering, Vol. 10, No. 3, 2007.

117

http://www.incose.org/

http://www.necsi.org/

References• Wilner, M., Bio-inspired and nanoscale integrated

computing, Wiley, 2009.

• Yoshida, Z., Nonlinear Science: the Challenge of Complex Systems, Springer 2010.

• Gardner M., The Fantastic Combinations of John

Conway’s New Solitaire Game “Life”, Scientific American 223 120–123 (1970).

• Nielsen, M. A. & Chuang, I. L. ,Quantum Computation

and Quantum Information, 3rd ed., Cambridge Press, UK, 2000.

118

119

Documents

Complex Systems Engineering SwE 488 Artificial Complex Systems Prof. Dr. Mohamed Batouche Department of Software Engineering CCIS – King Saud University