Breaking the Von NeumannBottleneck
And the future of Computing
Bruce Damer for the Singularity UniversityJuly 8, 2009
Understanding what is going to be possible with computing in the future is inseparable with knowing its origins in the past. The birth of modern digital computing in the 1930s through the 1950s has placed an indelible mark on how digital spaces are structured today. The vast majority of computing is locked into the von Neumann architecture, originating from John von Neumann's work on Electronic Computer Project at the Institute for Advanced Study in Princeton, New Jersey six decades ago. Today, challenges in advanced computing where we wish to simulate and observe emergent phenomena that model nature at the quantum, chemical and higher levels run hard up against the insidiously hidden but still built-in von Neumann Bottleneck. Overcoming this bottleneck may simply be a matter of throwing more computers, cores or pipelines at the problem, or it might require a re-thinking of computing architectures and software entirely.
In this presentation, we will take a whirlwind visual tour of the history of computing, beginning with the Colossus code-breaking machine of World War Two, through Von Neumann’s ECP project, the birth of interactive personal computing in the 1960s and 1970s, the coming of the network in the 1980s and 1990s, 3D virtual worlds and the GPU as a breakaway UI and computing paradigm of the 1990s, and multi-core, grid and cloud computing of the 2000s. We will then take a look at a recently launched visionary computing challenge called the EvoGrid and how it maps onto these architectures and suggest where we might be headed (or not headed).
The Tour to Come
Where did the Von Neumann Machine Come from?
Why and how did von Neumann’s architecture come to dominate the Computing World?
How Von Neumann Machines May or May Not be able to Compute Nature (in which case concepts like a singularity or other
bio-inspired phenomena in digital technology are off the table)
The EvoGrid: An Origin of Artificial Life Project in the Von Neumann digital space
An Alternate View: Non-Von Neumann digital space: Systemic
Computing
WWII Bletchley Park CodebreakersThe Colossus: a Non-Von Neumann Digital Computer
1944 – The Colossus Rebuild
1946 – Eniac, programming by plug board
Enter John von Neumann
Birth of the First True Digital Computer The Institute for Advanced Study, Princeton NJ - 1946
Von Neumann Architecture
All Programming done in commoninstruction/data memory (cybernetic feedback)
Architecture of Contingency: Memory Clock
Public Unveiling of Electronic Computer, 1952
Robert OppenheimerJohn von Neumann
Alan Richards photographer. Courtesy of The Shelby White and Leon Levy Archives Center, Institute for Advanced Study, Princeton, NJ, USA
The ECP: Compact, fast, stored program,world changing
Forty Williams Tubes for Cache Memory
Programs and data loaded and stored on IBMPunched Cards
ECP punched card
ECP Program #3: Experiments in biometric evolution… 1953 report by Nils Baricelli
Baricelli Report & EvoGrid Design
Baricelli “Blueprints”
Baricelli “Blueprints”
Punched Card Output Photo Captures first directvisual output of a program state (memory)
The world’s first Artifical Life program
Why and How did von Neumann’s architecture come to dominate the
computing world?
Answer: it was open sourced!
Entering the IAS Director’s Files
The meteoric rise and dominance of von Neumann’s architecture
Interactive *personal* computing beginsthe LINC - 1962
The ARPANET and the creation of the first networked multi-user graphical space – Maze War
1974
Maze War in Action
The Xerox Alto and the coming of the Graphical User Interface and Networked Computing - 1973
Alto Screen (Draw) courtesy Al Kossow
The birth of the modern networked computing environment Xerox Star - 1981
Xerox Star: The Coming of the Desktop Metaphor
Star Desktop Environment
Star Desktop EnvironmentThe Explosion of visual interfaces
Apple Lisa, 1983Apple Lisa
Apple Macintosh, 1984Apple Macintosh
In Walk the Avatars, Lucasfilm Habitat - 1986
Explosion of Social Virtual Worlds platforms(Book Avatars by Damer – 1997)
AlphaWorld “satellite map” 1995-96
Elements of Avatars98
Some Large Systems – Cray 1, Q2, LINC, TR-48, S-100 MultiCray Supercomputers: 1970s-80s (Digbarn)
Some Large Systems – Cray 1, Q2, LINC, TR-48, S-100 MultiGrid Supercomputers: 1990s-2000s (NASA ARC)
Some Large Systems – Cray 1, Q2, LINC, TR-48, S-100 MultiToday: Multi-Core Revolution, GPUs, Clouds, Grids
Computer history brought to you by the Digibarn
2008: A Grand Scientific Challenge takes on the Von Neumann Bottleneck: The EvoGrid,
an “Origin of Artificial Life”
Origins of Life: Archaean to Cambrian1997: Digital Burgess - quest for life’s algorithmic
origins in the “Cambrian Explosion”, Biota.org
Early exemplar: Karl Sims’ Evolving Virtual Creatures (1991-4)
“Soft” Artificial Life Through the Ages: field named in the 1980s, progress through the
1990s, 2000s
Evolving Virtual Creatures by Karl SimsInspired a generation of Soft Alife developers in the 1990s-2000s
Karl Sims: Evolving Virtual Creatures
Early exemplar: Karl Sims’ Evolving Virtual Creatures (1991-4)
State of the art of “Soft” Artificial Life
The Dawn of “Wet” ALife Protocells (Monnard, Rasmussen, Bedau et al)
Micelle SimulationExploring Life’s Origins Project (Harvard)
Micelle Division
Real Biology in Action: Ribosomes
Visit to FLiNT: Fundamental Living Technology Laboratory
University of Southern Denmark, Odense
Prebiotic Chemistry Fellermann (FLiNT, Univ Southern Denmark)
Models of Prebiotic Chemistry Monnard: Complexification & Formation of a Protocell
Mesoscale Molecular Simulation Fellermann: Formation of Simulated Membranes
Million Atom Satellite Tobacco Mosaic Virus simulation(NAMD and VMD, University of Illinois
Theoretical and Computational Biophysics Group
Enter the EvoGrid Could Artificial Life arise spontaneously from
Artificial Non-Lifeand could this shed light on
the Origins of Life from Non-Life?
New Book: Divine Action
and Natural Selection Gordon: Hoyle’s ChallengeDamer: The God Detector
The Vision: EvoGrid The MovieA Thought Experiment - Storyboards
EvoGrid The Movie
Farther Future Vision of the EvoGridProjecting life into the Solar System
Could we Artificially EvolveFreeman Dyson’s Trees?
Fanciful Concepts of Dyson Trees
Near Earth Objects: Threat or Future of Life in the Solar System?
Design for a human mission to a NEO
EvoGrid the Movie: The Asteroid Eaters
EvoGrid the Movie: The Asteroid Eaters
Building the EvoGrid
Variants
Building the EvoGrid
Ratcheting up the Complexity
Building the EvoGrid
Conceptual architecture
EvoGrid Prototype 2009bond formation in GROMACS
EvoGrid Prototype 2009movie
Final Question: Is digital technology based on the Von Neumann architecture up to the task of the EvoGrid or any substantial bio-inspired
computing as suggested by visions of the Singularity?
Answer: Probably not, but it is worth a try (get ready to work for decades)
Or: do we need to start thinking about Non-Von Neumann digital spaces as Von Neumann did 60 years ago?
Conventional vs Natural ComputationSystemic Computer model by Peter J. Bentley, UCL, Digital Biology Group
Conventional Natural Deterministic Stochastic Synchronous Asynchronous
Serial Parallel Heterostatic Homoestatic
Batch Continuous Brittle Robust
Fault intolerent Fault tolerant Human-reliant Autonomous
Limited Open-ended Centralised Distributed
Precise Approximate Isolated Embodied
Linear causality Circular causality
Table 1 Features of conventional vs Natural computation
Non-living natural world supports a massive number of parallel interactions but they are finite, bounded
Living natural world supports infinitely repeatable computations in a massively parallel fashion
Can this kind of machine do that?
Definitely not
Low level approximations (overhead)
How about a lot of these?
Perhaps… for the equivalent of a small volume of aqueous
chemicals
You need this…. to originate and evolve complex life (and civilization)
Penny Boston, CONTACT Conference 2009, NASA Ames
• Self-replication is perhaps one of the simplest forms of repeatable computations, and self-replicating systems in a systemic computer would therefore necessarily require simple metabolisms and the formation of a simple ecology in order for their behaviour to be maintained indefinitely.
• When the natural world is viewed from the perspective of systemic computation, the distinction between living systems and non-living systems becomes immediately clear: life is (or has the potential to be) an infinitely repeatable computation, non-living systems (for example, crystal growth) are bounded or finitely repeated computations.
Systemic Computing model by Peter J. Bentley, UCL, Digital Biology Group
Systemic computation
Everything is a system
Systems may comprise or share other nested systems.
Systems can be transformed but never destroyed
Interaction between systems may cause transformation of those systems, where the nature of that transformation is determined by a contextual system.
All systems can potentially act as context and affect the interactions of other systems, and all systems can potentially interact in some context.
The transformation of systems is constrained by the scope of systems.
Computation is transformation
Thanks EvoGrid External Advisors
•Prof. Richard Gordon, Professor-University of Manitoba•Tom Barbalet, director and community leader-Biota.org•Freeman Dyson-Institute for Advanced Study•Bob Taylor (retired), formerly of DARPA, Xerox PARC, DEC•Steen Rasmussen, Marin Hanczyc-FLiNT, Univ S. Denmark•Prof. Tom Ray, University of Oklahoma•Doron Lancet-Weizmann Institute, Israel•Dr. Nick Herbert, physicist and author•Larry Yaeger, Professor, University of Indiana•Brian Allen-MAGIX Lab, UCLA•Karl Sims, GenArts, Inc., Cambridge MA, USA•Dr. Ben Goertzel, CEO, Novamente, Silicon Valley, CA•Dan Miller, President/CTO, Singular Robotics, CA•Neil Datta, Imperial College London
Resources and Acknowledgements Project EvoGrid at: http://www.evogrid.orgProject Biota & Podcast at: http://www.biota.org DigitalSpace 3D simulations and all (open) source code at: http://www.digitalspace.com
We would also like to thank NASA and many others for funding support for this work. Other acknowledgements include: Dr. Richard Gordon at the University of Manitoba, Tom Barbalet, DM3D Studios, Peter Newman, Ryan Norkus, SMARTLab, Peter Bentley, University College London, FLiNT, Exploring Life’s Origins Project, Scientific American Frontiers, DigiBarn Computer Museum, The Shelby White and Leon Levy Archives Center, Institute for Advanced Study, Princeton, NJ, USA, and S. Gross.
Closing Thought