EMERGENCE ORIENTED PROGRAMMING AND …...•Using wind as motive force removes the need for motors, transmissions, power supplies etc. •Balloon like structure prov ides natural mechanism

4/29/2016

1

EMERGENCE ORIENTED PROGRAMMING AND BIOMIMICRY

Dr. Marc KirschenbaumMathematics and Computer Science DepartmentJohn Carroll University

YOUR MISSION, IF YOU CHOOSE TO ACCEPT, IS TO BUILD A

1/2 mile tall skyscraper (320+ stories) No time spent planning the project No blueprints No foremen No building codes Built by 1000’s of untrained citizens Includes cost-free climate control

MISSION IMPOSSIBLE!!!

4/29/2016

2

~2 m

~6 m

Termite: ~1 cm

THE EASTGATE IN CENTRAL HARARE, ZIMBABWEBIOMIMICRY

Designed by Mick Pearce with Arup engineers

No conventional air-conditioning or heating

Stays regulated year round with dramatically less energy consumption

Use design methods inspired by indigenous Zimbabwean masonry and self-cooling mounds of African termites!

4/29/2016

3

EMERGENCE DEFINITION

“A more nuanced definition is higher-order complexity arising out of chaos in which novel, coherent structures coalesce through interactions among the diverse entities of a system. Emergence occurs when these interactions disrupt, causing the system to differentiate and ultimately coalesce into something novel.”- Peggy Holman

SWARM CHARACTERISTICS

Large numbers of Agents

Each agent is very simple

Agents chose actions based on local info

No knowledge of the goal

Decentralized control

Relies on randomness to succeed

4/29/2016

4

EMERGENT BEHAVIOR CHARACTERISTICS

Global behavior emerging from local actions

Complexity arises from the random interaction of many known behaviors

Requires large numbers of agents

More than just a “speed-up” factor, extends capabilities

EMERGENT BEHAVIOR AS A PROBLEM SOLVING TECHNIQUE

Complex problems require complex solutions swarms allow the complexity to emerge

Flexible easy to reconfigure

Robust distribution of work - no single point of failure

Inexpensive lots of small deployments are cheaper

4/29/2016

5

WHAT ARE SOME APPLICATIONS?

Collectives of Autonomous Robots

Searching large solution spaces

Simulations of multi-agent activities

Large collaborations Human Swarms

APPLICATIONS OF SWARMS




Large collaborations

4/29/2016

6

COLLECTIVES OF AUTONOMOUS ROBOTS

Military applications

Space exploration

Hazardous environment access

MILITARY APPLICATIONS

Battlefield communication Ad hoc networks, ground radio

Covert Reconnaissance Sensor fusion networks

Unmanned Vehicle operations Target acquisition, destruction, verification Battlefield assessment

Urban Warfare Data acquisition and communication Battlefield dynamic mission components

Explosives Detection

4/29/2016

7

SPACE EXPLORATION

Micro-satellite constellations Several small satellites replace single, large one

Individual components used as needed

Multi-vehicle exploration missions Modular development: less expensive

Planetary rovers Decentralized control of robot collective: robust

MARTIAN ROVERS

Artist’s conception of Sojourner bouncing across Mars surface during landing (left)

Mars Exploration Rover – MER (above)

4/29/2016

8

THE TUMBLEWEED ROVER

•Must be compressible to fit within rocket fairing

•Must minimize mass

•Must survive Martian Environment

•Using wind as motive force removes the need for motors, transmissions, power supplies etc.

•Balloon like structure provides natural mechanism for compression and encapsulating delicate sensors etc.

BIOLOGICAL INSPIRATION

The Russian Thistle (salsola tragus) provides biological inspiration for wind driven rover design

4/29/2016

9

HAZARDOUS OR INACCESSIBLE ENVIRONMENT ACCESS

Rescue missions

Natural and man-made disaster rescue

Hostage situation information gathering

Dangerous sites

Radioactive site inspection and evaluation

Hostile environments – volcanic, vacuum, etc.

Maintenance bots

Sewer tracking

IN JAPAN ROBOTS GO WHERE HUMANS CAN'T. BY BOONSRI DICKINSON, INNOVATION

4/29/2016

10






ANT COLONY OPTIMIZATION (ACO) CREATED BY MARCO DORIGO, ANNUAL CONFERENCE IN BELGIUM, ANTS Artificial ant colonies

using pheromones to solve general optimization problems

Uses ants’ desire for high concentrations of pheromones and randomness to produce positive feedback loops and behavior reinforcement

Goal of the “programmer” is to create the proper conditions under which the pheromones are released and sensed to bring about an emergent solution

4/29/2016

11

Food Nest

From: Ant colonies for the traveling salesman problem by Dorigo and Gambardella

Pheromone trail

Food Nest


Obstacle appears

4/29/2016

12

Food Nest


Ants randomly explore both alternate directions

Food Nest


Path reconnected on shorter side of obstacle first, Thus faster application of additional pheromones

4/29/2016

13

Food Nest


Reinforcement of pheromones establishes shorter path before longer path can gain a critical mass. Eventually all ants use Shorter path






4/29/2016

14

MULTI-AGENT SIMULATIONS

Emergency exits Previously modeled as a liquid

Improved survival with agent-based models

Urban, traffic, public transportation, sporting venue planning and design

Physical properties of the “Wave” at stadiums

MORE MULTI-AGENT SIMULATIONS

Predator-Prey studies

Disease transmission and infection vectors

Shipboard logistics and supply storage balancing

Ancient civilizations, simulated archeology

Cultural algorithms

Social networks

4/29/2016

15

SWARM PROGRAMMING

Writing Agent Rules Low level actions

Realizing Global Behavior High level actions

Most Difficult Aspect of Swarms





Large collaborations Human Swarms

4/29/2016

16

ULTIMATE BIO-INSPIRATION: HUMANS

Trial and error approaches to swarm algorithm development

Apply biological inspiration: tell a swarm of humans what to do, then observe

Reverse-engineer their algorithms and apply to swarm control problem.

PHYSICAL HUMAN SWARMS

4/29/2016

17

LARGE COLLABORATIONS -EXAMPLES OF EMERGENT HUMAN BEHAVIOR

“The Wave” at sporting events Smart Mobs AMBER Alert Systems e-Political movements World Wide Web Wikipedia Open Source software development UNU collective intelligence platform

SWARM ALGORITHM MINING

Observing Human Swarms can produce decentralized control strategies

Can test existing algorithms

Can watch as the swarm generates new ones

Once extracted, can be implemented in simulation or on robots

Color grouping Algorithm Example

4/29/2016

18

POSSIBLE APPLICATION

Human Swarming Behavior applied to the development of collective control architectures for small quadruped robots Proposed jointly by Dan Palmer, Marc

Kirschenbaum, Ravi Vaidyanathan, and Roger Quinn

SWARM OF MULTI-PURPOSE ROBOTS

Capable Emergent Behavior

Simple Relatively inexpensive Relatively easy to program

Flexible Many different applications

Robust Generally no single points of failure

4/29/2016

19

ROBOT SWARM CATCH-22

Must know robot capabilities to write algorithms

Must know algorithms’ requirements to spec out robot

Configure and Build Swarm of Robots First?

Too expensive if you get it wrong…

…even once

ROBOT SWARM CATCH-22 Design Algorithms First and Verify with Simulations?

Make assumptions about robot capabilities Approximate/write possible swarm algorithms

Bio-inspiration Writing detailed simulations is time consuming Repeat for different:

Robot capabilitiesApplication scenariosDifferent possible algorithms

Very time consuming

4/29/2016

20

NEED TO BREAK THE INTERDEPENDENCE

Fast, reprogrammable system Interactive programming interface Describe agent program in terms of swarm goals

Evaluate and suggest programming possibilities

Provide feedback on robot/scenario success Inexpensive

SOLUTION: HUMAN SWARMS Humans are instantly (re)programmable

Programming interface is easy: articulate goal

Source of Bio-inspiration for algorithms Can potentially mine swarm algorithms

Can provide a computational “upper-bound” If a swarm of humans can’t do it…

Can be inexpensive if “fun” enough

4/29/2016

21

BIO-INSPIRATION: ALGORITHM MINING

Give the human swarm a goal

Record their actions towards the solution

Analyze recorded data

Extract human swarm algorithms

Implement algorithms autonomously

Verify the problem-solving success

Human Swarm: Color Grouping

4/29/2016

22

HIGH-LEVEL SWARM COMMANDS

4/29/2016

23

PROBLEMS WITH HUMAN SWARMS

Difficult to control humans in a large group

Limited to physical environment

Subtle algorithmic factors hard to extract, replicate

Human capabilities exceed robot’s Sensor, communication, computation, ….

How to find motivated volunteers when needed?

4/29/2016

24

EMERGENCE-ORIENTED PROGRAMMING

EOP Complete

Restart Cycle

Restart Cycle

Start Cycle

No Improvement

Improvement

Human Observation of Swarm Behavior

Baseline Swarm System

Evaluate Hybrid Performance

Codify Human Assistance Becomes New Baseline

Roll Back to Previous Swarm System

Goal Emergent Behavior

Human Assists/interacts with Swarm

HUMAN SWARM FEEDBACK LOOP - EBST

Humans generate Finite State Automata (FSA) to program swarms Get immediate feedback of swarm in action Refine control rules on the fly

4/29/2016

25

EMERGENT BEHAVIOR SIMULATION TOOL (EBST)

Online, interactive swarm programming system

Users generate Finite State Automata (FSA) to program swarms

Users set priorities of agent sensors and size of the experiment

Users can view a real-time animated simulation of their programmed swarm

EBST - EOP

Two programming scenarios Multi-goal ants

Task-oriented robots

User writes a single program that all agents execute

User observes results and modifies program

4/29/2016

26

ANT BRIDGES (SCENARIOS 1 )

http://www.emergencemarketing.com

• Ants use their bodies to traverse large gaps

• An ant tries to go as far as possible on a path

• When it reaches the edge, it strives to reach out further

• Subsequent ants trying to go as far as possible, climb on them

• Each ant follows an individual goal, the bridge emerges.

CONCEPTS OF ANTS, BRIDGES AND FOOD SCENARIO

Trade-offs and alternatives Many possible ways to produce a solution

Communication needed for emergent behavior

Hungry ants require balancing multiple goalsBuild bridge(s)

Don’t starve

More behavioral building blocks = more interesting swarm programming

4/29/2016

27

4/29/2016

28

EBST

DUAL MOBILITY ROBOTS (SCENARIO 2) Robots can work effectively in sand and underwater Robots contain a reservoir that can be filled with sea water Weight increase supports in water mobility, weight

decrease supports movement in sand Cost associated with switching

From: Case Western Reserve University’s BioRobotics Lab

4/29/2016

29

57 of 29

ADDING COMPLEXITY: ROBOTS

Scenario 2, a surf zone, mine sweeping experiment, adds more: Sensors Actions Communication capabilities Evaluation criteria

Goal: Program Robot swarm to make a surf zone landing “Safe” Disarm, detonate, mark mines Identify safe pathways

Naval Post-graduate SchoolJohn Carroll University

EIAE '07

58 of 29

Initial deployment

4/29/2016

30

CURRENT RESEARCH SWARM/HUMAN BLENDED INTELLIGENCE

DESIRABLE

Scalable

High-level, short-duration Interaction

Intuitive

AVOID

Micromanagement

OTHER APPROACHES

Beacons

Human as swarm agent

Swarm “leader”

OUR APPROACH

Population Ratio Control

DRIVING PROBLEM CHARACTERISTICS

Swarm

Parallel component

Randomness useful

Emergent

Can solve autonomously

Humans

Big Picture component

Engage human visual system

Contextual knowledge

Can solve independently

4/29/2016

31

HUMAN-SWARM HYBRID SYSTEM

Digital Jigsaw Puzzle

Agent Population Ratio Control

Humans address

“big picture”

Swarms leverage parallelism and randomness

HASS – Human Assisted Swarm Simulation

4/29/2016

32

AGENT TYPES

Connectors

Sorters Color

Edge

“Shape”

Finishers

DEMONSTRATE PROGRAM:

USER INTERFACESWARM COMPOSITION HASSDOWNLOAD FROM HTTPS://WWW.ERICMUSTEE.COM/

4/29/2016

33

SCALABILITY

Previous work showed Blended Intelligence out performed both a human and a swarm working independently

Single Human can direct several different swarms simultaneously

Compared the number of swarm moves necessary to complete a puzzle for: Human and a Single Swarm Human with Two Swarms Human with Four Swarms

RESULTS

OnePuzzle

TwoPuzzles

TwoPuzzles

FourPuzzles

FourPuzzles

FourPuzzles

FourPuzzles

Mean 13866.16 13904.16 13390.4 13023.36 13060.52 12877.32 13904.16

Min 9642 9467 11284 9346 10150 9671 9467

Q1 12252.5 12441.5 12111.5 11886.5 12200 11666.5 13904.16

Median 13909 13471 13305 14399 14108 14125 15561

Q3 15105 15561 14328 14399 14108 114125 15561

Max 18458 18379 17881 16395 15869 15739 18379

4/29/2016

34

BLENDED INTELLIGENCE

Jigsaw Puzzle Human Alone Averaged a little over 2 Hours to

solve a puzzle

Swarm Alone Averaged around 10 minutes to solve a puzzle

Human/Swarm Blended Intelligence around 5 minutes to solve a puzzle.

Building a Skyscraper?

4/29/2016

35

EOP AND DIGITAL JIGSAW PUZZLE DESIGN

Baseline, reactionary swarm consisting of only connector agents

Added sorters

Basic strategies to autonomously address ratios – no human intervention

Child agents

Finisher agentsYou can watch animated examples of each improvement to HASS in chronological order at: http://tinyurl.com/l24vef3

SHBI2016 WORKSHOP Swarm/Human Blended Intelligence

Dates 10/21-10/23

Check out SHBI2015 at

http://shbi2015.org

Artist: Rebecca Borrelli

4/29/2016

36

ACKNOWLEDGEMENTS Dr. Daniel Palmer

Dr. Roger Quinn

John Carroll University Undergraduates

Huntington Foundation

CAS - John Carroll University

Naval Post-graduate School

Documents

EMERGENCE ORIENTED PROGRAMMING AND …...•Using wind as motive force removes the need for motors, transmissions, power supplies etc. •Balloon like structure prov ides natural mechanism