physiology, robotics, and computational biologyaliceabr/EI_talk_fall_2007.pdf · * serves...

Evolving “Physical” Intelligence:

physiology, robotics, and

computational biology

By Bradly Alicea

EI Meeting, DevoLab, Michigan State University, Fall 2007

Introduction

Research Question: how do we uncover and represent the adaptive and

phylogenetic processes behind “physical” intelligent behavior (e.g.

movement, kinetics, control)?

* examples focus on autonomous physical

intelligence in vertebrates (lampreys

to humans), may generalize to design of

machines (biomimetics).

* paradigm focuses on motility related to

propulsion and “work”; interaction of

multiple physical elements.

* requires approximating a physiological

control system. Application domains:

biomechatronics, robotics, even micro-

machines.

* look at morphology alone, nervous system

alone, and morphology and nervous system

together.

Introduction (con’t)

Jeff Hawkins (Redwood Neuroscience Institute, Palm Technologies): “On

Intelligence”:

Intelligence is an internal mechanism:

* serves “pattern prediction” function

* memory-based, adaptive, hierarchical

* has an effect on behavior, not behavior

in and of itself.

* his focus is on “neocortex”, which is a

specific physiological system.

* idea can be generalized; formalized as

a control system.

Design principles (or principles of

evolvability)Principle #1 – modeling physical intelligence takes into account:

* physical sensory receptors: proprioceptors, nociceptors, muscle spindles. Capture the

collective activity of excitable cell populations.

* adaptability of morphology (e.g. muscle, bone): hypertrophy, fatigue, stress/strain,

regeneration.

Principle #2 – tetanic stimulation, physical exercise, environmental

training = “triad” of inducing adaptability (e.g. physiological plasticity):

* tetanic stimulation: deliver a tetanus (rapid electrical pulse) to muscle, neuronal tissue.

Results in LTP, “virtual” training

* physical exercise: Kaatsu (restrict blood flow to limb, stress muscles in that limb),

Fartlek (alternate intensity of training).

* alternate and extreme environment training: 0-g, force field adaptation, environmental

switching, H2S respiration (reduced metabolic baseline), ischemic preconditioning.

Design principles (or principles of

evolvability) – (con’t)Principle #3 – structural modular

intelligence:

* custom prosthetics (C-leg, foot-ankle

prosthetics, brain-machine interfaces)

replicate “intelligence” locally.

* adaptive walking, reaching, motility,

even thinking.

* function regulated by nervous system,

other morphological systems, environment.

Andy Clark (Natural-Born Cyborgs, 2004),

transformative potential of prosthetics.

Limbs > cells (e.g. living heart valve).

* due to role of proprioception, induces

locally adaptive changes in cell populations

(Smith et.al, Tissue Engineering, 7(2),

131, 2001.

1) Passive Dynamic Walkers (PDWs).

2) stability enforcement mechanisms for intelligent physical behavior.

3) the intelligence of “physical” intelligence.

Part I: Morphology

by Itself

Morphology by itself: PDWs

Andy Ruina and friends: Passive Dynamic Walking (PDW)

* inverted pendulum model: given a stochastic input (simple oscillator, finite

energetic input), stable gait can be physically approximated.

* bipedal: hindlimbs – human gait,

forelimbs – gibbon brachiation.

* no neuromuscular or cognitive

feedback, no mechanotransduction

(e.g. efference copy).

* when environmental conditions

are variable, gait is not stable.

Morphology by itself: PDWs (con’t)

Honda‟s ASIMO: demonstrates basic application of how bipedal gait is

regulated (also falls down a lot).

* afferent signal (tells legs to move)

* morphology reinforces efficiency

of movement.

* efference copy (feedback from

environment)

No “biological” component (e.g. muscle plasticity, neuroplasticity, learning

and memory).

* what would a “biological” controller look like?

Morphology by itself: PDWs (con’t)

Key feature of PDWs: behavior for

“free”.

* bipedal gait = zero net energy

expenditure given constant

movement (no adaptive adjustments).

Stable state discovery: Sherrington

(Integrative Action of the Nervous

System, 1947):

* amputate one limb, insect finds new 'stable phase' for motility.

* robotics/postural sway work: „internal‟ mechanisms perform relevant

computations.

Morphology by itself: stability

enforcement mechanisms

Mechanism #1: “static” allometry:

*controls the size of limbs relative to one

another and body size.

* basis for metabolic efficiency (cost of

locomotion decreases as body weight

increases in quadrupeds).

Body weight + limb shape +

forces in environment = cost of

transport.

* linear function, true for many varieties of

quadruped (see graph).

* cost of transport ~ muscle power (output)

needed for specific tasks and environments.From: Herr et.al (J. Experimental Biology,

205, 2005)

Morphology by itself: stability

enforcement mechanisms (con’t)

From: Bejan and Marden, J. of

Experimental Biology, 209, 238.

“Constructal” effects across phylogeny

(energy needs during locomotion = strong

positive selection on morphology):

* vary environment (air, water; variables = Reynolds

number, surface reaction forces)

* vary mode of locomotion (running, swimming;

variables embodied in velocity, frequency, force).

* linear scaling for all verts/inverts. Swimming

(fishes), flying (birds, bats, insects), running

(mammals, reptiles) all “cluster” along same trend

line (force production vs. body mass).

Morphology by itself: stability

enforcement mechanisms (con’t)Mechanism #2: matched volumes. MacIver‟s

simulation of Apternotus albifrons (Nelson and

MacIver, J. Experimental Biology, 202(10),

1999):

Weakly electric fish have a

special sensory modality called

electroreception.

* “active” (e.g. field generated around

organism).

* originated from neuromuscular system,

important in navigation.

* “map” at right is the electrosensory

field as it overlaps with “short-time

motor volume”.

Morphology by itself: stability

enforcement mechanisms (con’t)

Active sensing in context of set matching:

* actively sense at time t; at every t, iteratively

create vol(x) based on current environment.

* fill in space vol(x) with form(y); shift ith set

of motor commands towards leading front of

movement and exploration (optimize degree of

isomorphy).

* tail bending behavior (Behrend, Neuroscience,

13, 171-178, 1984); introduces "critical"

exploration points.

* electrodermal potential changes during tail

bending, potentially shifts the phase of short-

time motor volume.

* a "memory" of interaction (sensory inputs|limb size x muscle power); acts as an integrator

mechanism (allometric scaling in development and evolution ensures control).

1) biological A.I. (hybrots = cortical cells for computational environments)

2) neural coding (movement vector) and applicability to A.I. problems.

3) future advances: molecular models.

Part II: Nervous System

by Itself

Nervous system by itself: biological

AI = hybrots

In experiments by Reger et.al (Artificial Life, 10, 2000), hindbrain of

lamprey explanted and connected to Khepera robot.

* artificial photoreceptors from robot body provided input channel to Muller

cells, play the role of sensorimotor integration in lamprey brainstem.

* sensors on the robot's body = inputs to neural system. Resulting control

loop allows for adaptive behavior.

“Brain-in-a-dish”: collective

output, environmental

feedback (simulation).

* at left is an example of an

adaptive flight control

system.

* software is used to find

“taxic” information in neural

output.

* signals “mapped” to degrees

of freedom in the simulation

(roll, pitch, and yaw).

Nervous system by itself: biological

AI = hybrots (con’t)

DeMarse and Dockendorf, IEEE International Joint

Conference on Neural Networks, 3, 1548-1551,

2005.

Nervous system by itself: biological

AI = hybrots (con’t)

Control systems called hybrots have been

used to map neural signals to “skilled”

behaviors, such as drawing on an easel.

* cell culture of cortical neurons that

selectively grow connections between

neurons and show postsynaptic

modification (neuroplasticity).

* systems inform general processes behind

learning and memory in systems where

biology and machines are tightly coupled.

Nervous system by itself: applied

neural codingPole balancing (neural integrator keeps pole

from falling due to inertia or gravity):

* 1 DOF, “toy” problem.

* reinforcement learning methods solve this problem well

(actor-critic model).

* perceptron can be used to calculate and encode information

for movement direction, velocity, etc.

* does not approximate complex physiologically-based

functions (dampening, rate limiting).

See Broussard and Kassardjian, Learning

and Memory, 11, 127, 2004.

In mammals, neurons in premotor and motor cortex (PMC) contribute

to planning and directionality of movement:

* activity onset is 1-2 seconds before actual behavior.

* a "population code" (collective encoding of single behavioral events by

neuronal cell populations) has been found to exist.

* population coding may be important for other functions (memory encoding,

satiety states, etc).

Movement vector: Georgeopoulos et al (Journal of Neuroscience, 2, 1987):

* single cell activity in premotor and motor cortex predicts direction

of movement, mental rotation, force and velocity parameters.

Nervous system by itself: applied

neural coding (con’t)

The collective activity of cells results

in the encoding of desired behavioral

states.

* average activity of a population is greatest

in a certain direction(e.g. 45, 90, 155 degrees

from straight ahead).

* used as the driving

force behind Brain-

Machine Interface

(BMI) technology.

Nervous system by itself: applied

neural coding (con’t)

Nervous System by itself: future

advances -- molecular modelsMechanostimulation:

* activates stress pathway in cell populations.

* within minutes of stimulation, series of genes

upregulated (enhanced expression).

* in preconditioning, low levels of perturbation

increase robustness of system to acute shocks.

* depending on stimulus (environmental setting),

different regulatory patterns should result.

* patterns not well understood: what are the

effects of environmental switching, mutation of

genes involved in stimulus response, long-term

adaptation?

Nervous System by itself: future

advances -- molecular models (con’t)

Signaling pathways in memory-associated plasticity in brain (left - CREB)

and hypertrophy-associated plasticity in muscle (right - IGF):

Activity of pathways change across

training, interaction with environment.

* One emergent property of gene

expression and regulation = change in

morphology and internal state (figures:

http://www.biocarta.com).

Presence of hormone receptors, proteins and mRNAs in specific concentrations

(activity-dependence). Contributes to plasticity outcome (“increased/decreased

capacity” of tissues).

How do we piece together the interesting aspects of morphology and neural

systems into one unified framework/approach?

1) functional allometry/epigenetic matching

2) neurobiological control theory

Part III: Morphology and

Brain Together

Allometry: different anatomical segments are genetically “linked”. Consequences

for growth regulation and function within and between species.

Y = ax + b, Y = axb, Y = -Ax2 + Bx – c

Functional effects of allometry:

Herr et.al (J. Experimental Biology, 205, 2005):

* allometric scaling is a feature of "optimal“

locomotion and goal-directed behavior. Limb

length, circumference, brain size, metabolic

rate ~ body mass.

* provides a mechanism for determining

"optimal" scaling.

* Collins et.al (Science, 307, 1996) have

found that there is an optimal ratio of 1.06

between the length of the shank and thigh

in human bipedalism.

Morphology + Brain: functional

allometry/epigenetic matching

Morphology + Brain: functional

allometry/epigenetic matching (con’t)Epigenetic Matching: motorneuron population ~ target tissue (allometry and

growth regulation of target tissues ~ evolution and adaptability of nervous

system):

Streidter (Principles

of Brain Evolution,

Sinauer, 2006)

* finite pool of

motorneurons,

finite volume

of muscle target

tissue (myocytes).

* if axon from

motorneuron does

not innervates target

tissue = apoptosis.

Morphology + Brain: functional

allometry/epigenetic matching (con’t)

Katz and Lasek (PNAS USA, 75(3), 1977): Type I and Type II evolution.

* Type I: “linkage” (neuron-to-myocyte matching; innervational “linkage”

between two sets of cells).

* conservation via hormone action, high degree of epistasis, high degree of

evolution (no developmental constraint).

Type II: no autonomous preservation of axonally-mediated matches (no

innervational linkage between two sets of cells).

* depends on function of interactome, serves as evolutionary constraint

(unless mutation introduced for both motorneuron pool and muscle mass,

complexity remains low).

Morphology + Brain:

neurobiological control theory

Computational Neurobiology of Reaching and

Pointing: Reza Shadmehr (Johns Hopkins) and

Steven Wise.

* internal states not a black box, play an

important role in regulating behaviors

(normal and pathological).

* internal “model” is a statistical mechanism

(others are more interested in the internal

model as anatomical ROI).

* internal model = memory-based

displacement mechanism. Updates =

incoming physical sensory information,

visual information, and prior states.

Morphology + Brain: neurobiological

control theory (con’t)Reaching involves contributions from

both the CNS and constraints imposed

by limb geometry (230 and 137):

*anatomical stiffness ~ constraints.

Stiffness = stability.

* disease states (e.g. Parkinson‟s):

represents perturbation of neural

mechanisms involved with “normal”

movement (135).

* cerebellar, basal ganglia components

of learning system = nuclei, synapses

mediated by neurotransmitters (456).

Reinforcement learning mechanism.

Morphology + Brain: neurobiological

control theory (con’t)Internal Model: Computational function of

cerebellum:

* internal model is highly

conserved across vertebrates.

* general (innate) and specific

(acquired) internal models.

* innate: general limb

movements, environmental

resistance.

* specific: single and related

sets of objects.

How does evolution of the nervous system and morphology (as a unified

system) proceed phylogenetically?

* what “strategies” (e.g. combination of mutations, adaptations) are used to achieve a

derived form?

* three slides with hypothetical phylogenies only suggestive (focus on locomotive gait --

could have happened many different ways, and actually has in terms of convergent

evolution).

Postscript: “solutions” for

evolving physical

intelligence

Phylogenetic “solutions” application

domain: morpho-functional machines

Defined as the co-evolution of morphology and control unit:

* change functionality by changing control parameters and shape.

* evolve whole system in pieces, or modules (specialized substructures or

distinct behaviors).

* evolve morphology (morphogenesis) semi-independently from neural

controller.

* evolution of both morphology and neural mechanisms define a particular

evolutionary derivation (but multiple evolutionary “strategies”).

Phylogenetic “solutions” to evolving

physical intelligence

At left: how Type I and II

evolution may proceed:

Cladogenesis requires

generalized capacity for

plasticity.

* one mutation, may trigger

endocrine plasticity.

Anagenetic taxa may

require two specialized

mutations.

* morphology and nervous

system specialized but not

evolvable.

Phylogenetic “solutions” to evolving

physical intelligence (con’t)

At left: how static allometry

in hindlimb evolves along

mode of gait.

* gene controlling thigh

plasticity evolves before

common ancestor of C,

D, E, and F.

* bipedalism evolves in F

(requires other associated

mutations).

* genes “unlinked” by thigh

plasticity mutation, “relinked”

when bipedalism arises.

Phylogenetic “solutions” to evolving

physical intelligence (con’t)

At left: how to move from

one physically intelligent

mode to another in

evolution:

* three behavior-related

mutations to go from

specialized quadruped to

a biped (probably more).

* also anatomical changes

(joint morphology, spinal

cord alignment).

* behavioral mutations >

anatomical mutations

(which come first)?

Conclusions“Physically” Intelligent Systems:

1) Consider morphology and physiology together

* provides a mechanism for dynamic behavior

* emergent features of physiological interactions – constrained by morphology

2) Dissociate morphology and physiology for purposes of understanding

phylogeny

* shared derived characters (changes in phylogeny required for behavior,

match phenotype?)

* possible control mechanisms (morphology, genes, regulatory mechanisms)

3) Computational Principles

* What else is needed? What other tools can be deployed?

Additional Notes:

Comparative function and main neural centers:

Each brain center has a

specific computational

function:

* integration, acquisition,

encoding, and recall of

information.

* work together as an

anatomical network

to send feedforward

information to limbs.

* cross-talk between

networks.

Interacting Neural Systems and

Crosstalk: an “inconvenient truth”

Notes on Passive Sensing

(according to me)Passive sensing in the context of moving a limb towards a target:

* uncontrolled manifold hypothesis (Domkin et.al,

Experimental Brain Research, 163(1), 2005). Arm

has many DOFs with which it can potentially

reach an object.

* no finite sensory envelope, dynamic opposition

of forces from environment determine manifold

for movement.

* lots of behavioral variability as compared

with orthogonal manifold (set of solutions

chosen by CNS).

* scaling (geometry) of limbs important to

constrain what functional manifolds look

like in adulthood (also limits mathematical

solutions for SI|LS x MP).

* motor primitives in spinal cord (see Mussa-Ivaldi and Arbib) – combinatorially

put together to drive outputs based on current environmental demands.

Bayesian-Systems Model of

Adaptation via Molecular Pathways

A preliminary “model” of signal

transduction in a cell w.r.t. motor

performance (mechanotransduction

and control).

* expression of genes in tissues ~

properties of tissues. Each set of

relationships for single cell, many of

these in parallel ~ tissue.

* may be able to approximate emergent

changes in tissues ~ changes in

performance, morphological adaptation

(ability to encode adaptive changes).