Upload
ralph-lane
View
214
Download
0
Tags:
Embed Size (px)
Citation preview
Contact [email protected]
Controlling Mobile Robots with Distributed
Neuro-Biological SystemsSebastian Gutierrez-Nolasco (UCI)
Nalini Venkatasubramanian (UCI)
Alfredo Weitzenfeld (ITAM)
Biologically Inspired Robotic Systems
Nature has always been a source of inspiration in the development of autonomous robotic systems Ethology
Animal behavior-based simulation Interaction with the environment is usually oversimplified
Lack of strong biological basis for their working assumptions Lack of any formal underpinnings for the simulation results
Neuroethology behavior related to neurobiological structure
Replicate brain models to provide credible and general animal behavior Provide inspiration for further robotics architectures
More complex and accurate than ethology systems Enable experimentation Experimentation requires real-time performance
Neuroethological robotic systems
Super Robots Incorporate extensive
processing capabilities Bulky Expensive
Inexpensive Robots Smaller and inexpensive robots
connected to a network of processing nodes
Concerns Real-time performance Unpredictable communication
environment affects robot performance
A neural model may take hours of processing time Simulation of multiple neural networks require a distributed
processing environment A typical retina model may consist of more than 100,000 neurons and
500,000 interconnections
Biologically inspired robotics demand sophisticated image processing techniques Communication intensive tasks are required
Autonomous robotic agents have real-time and processing restrictions, as well as power awareness requirements Battery usage is a major concern in mobile robots
Challenges of Biologically Inspired Autonomous Robots
Developing software for autonomous mobile robots is complex Highly heterogeneous methods for capturing and processing
sensor information Multiple sensory input devices Sensory input multi-granularity
Communication is error-prone due to unpredictable interference and failures partial and complete failures Unreliability and disconnection Varying available bandwidth
Developing Biologically Inspired Robot Architectures
Our Approach
Develop an embedded architecture capable of conducting neuroethological robotic experimentation
Inexpensive small robots communicate (wireless) with distributed computational resources
Neural models are distributed in multiple processing nodes
Adaptive robotic middleware optimizes robot communication in response to varying network conditions
Structure of the Talk
1. Neuroethological Modeling Study animal behavior and corresponding neural structure
as inspiration to robotic architectures
2. Embedded Mobile Robots Develop distributed wireless robot architectures capable of
efficient neural processing
3. Adaptive Middleware Achieve real-time computation and adapt embedded
architecture to varying network conditions
4. Internet Based Robotics Enable remote robot task development and
experimentation
1. Neuroethological Modeling Study animal behavior and corresponding neural structure
as inspiration to robotic architectures
2. Embedded Mobile Robots Develop distributed wireless robot architectures capable of
efficient neural processing
3. Adaptive Middleware Achieve real-time computation and adapt embedded
architecture to varying network conditions
4. Internet Based Robotics Enable remote robot task development and
experimentation
PSMate
Mate-PairS
PSPrey
Prey-AcqS
PSPred
Pred-AvS
Moving-ObjectS
PSNMO
Find-LocS
Non-Moving-ObjectS
PSMO
+ ++
-
--
Perceptual Schema (PS)
Main Schema (S)
Behavior:Frog and Toad - Rana Computatrix [Arbib 1987, Cervantes 1990]
Mobile visual stimulus in lateral visual field(monocular perception)
Mobile visual stimulus in binocular visual field(short distance)
Mechanic stimulusin mouth and pharynxreceptors
Orientation
Binocularfixation
Attack
Snap
Clean
Stimulus Response
Behavior:Toad Prey Acquisition [Cervantes 1985]
Behavior:Toad Prey Acquisition with Detour Behavior Before and After Learning [Corbacho and Arbib 1995]
20cm Barrier Before learning 20cm Barrier After learning
10cm Barrier
Schema Level 1 data indata out
Other Processes
Schema Level 2
Schema Level
Neural Level......... ...
din1
dinn
dout1
doutm
Schema
Neural
Schema Computational Model
R4
Visual
R1-R2
R3
Retina
Tectum
PreTectum/Thalamus
Motor Heading Map
Static ObjectRecognizer
Prey Recognizer
TH10
T5_2
Static ObjectAvoidance
Prey Approach
Forward
Orient
Sidestep
Backward
Tactile Schema Level
Neural Level
Moving Stimulus Selector
R1-R2
R3
R4MaxSelector
PredatorRecognizer Predator Avoid
R1-R2
R3
R4
Depth
Stereo
Neural based Behavior : Toad Prey Acquisitions and Predator Avoidance
1
2
3
4
5
67
1189
1012
1314
1516
10cm barrier 20cm barrierBefore learning
20cm barrierAfter learning
Toad Prey Acquisition with Detour:Simulation Results
1. Neuroethological Modeling Study animal behavior and corresponding neural structure
as inspiration to robotic architectures
2. Embedded Mobile Robots Develop distributed wireless robot architectures capable of
efficient neural processing
3. Adaptive Middleware Achieve real-time computation and adapt embedded
architecture to varying network conditions
4. Internet Based Robotics Enable remote robot task development and
experimentation
Sensors Actuators Vision Tact
Legs Wheels
LEGO
OOPIC
Embedded Mobile Robots:Robot Hardware
Autonomous Robot 1
Autonomous Robot N
Inte
rnet
Wireless
InternetServer
... ......
RemoteComputaional
System
Instance 1
Instance N
RemoteComputational
System
Embedded Mobile Robots:Distributed Embedded Architecture
Time consuming processes are carried out in the (neural) computational system Neural processing Image processing
Limited task are carried out in the robot hardware Sensory input Motor output Default behavior
Communication and data transformation is managed by the adaptive middleware
Embedded Mobile Robots:Distributed Embedded Architecture
motormotor
servo
camera
CPU(OOPic)
Transceiver
Power stage
Frame Grabber
Sensors(tact)
TransceiverPC
Remote Computaional System
Robot
Wireless
Embedded Mobile Robots:Distributed Embedded Architecture
NSL NSL
NSL NSL
NSL NSL
NSL NSL
ASL ASL
ASLASL
VideoServer
Video/Image
Processingcamera
Robot
Wireless
Remote Computational System
TactileServer
Motor Server
tactile
motortran
scei
ver transceiver
NSL – Neural Simulation LanguageASL – Abstract Schema Language
Embedded Mobile Robots:Distributed Embedded Architecture
Video capture
Video processing
Model simulation
Model output
Navigation control
(d , r , c)
Embedded Mobile Robots:Processing cycle
1. Neuroethological Modeling Study animal behavior and corresponding neural structure
as inspiration to robotic architectures
2. Embedded Mobile Robots Develop distributed wireless robot architectures capable of
efficient neural processing
3. Adaptive Middleware Achieve real-time computation and adapt embedded
architecture to varying network conditions
4. Internet Based Robotics Enable remote robot task development and
experimentation
Distributed Systems Middleware
Enables the modular interconnection of distributed software Abstract over low level mechanisms used to implement resource
management services
Concurrent Object Oriented Model Separation of concerns and reuse of services
Customizable, Composable Middleware Frameworks Provide for dynamic network and system customizations,
dynamic invocation/revocation/installation of services Concurrent execution of multiple resource management policies
Core Resource Management Services
Core Services - basic services where interactions between the application and system can occur. Building blocks for other services Reduce interactions among many services to interactions
between a few simple services
Choosing core services - commonly observed patterns Recreation of data/services at a remote site Capturing approximation of distributed state at multiple sites Interactions with a global repository
TLAM: The Two Level Meta-architecture
DistributedSnapshot
Remote Creation
Directory Services
Replication
Migration
DGC
Check-pointing
AccessControl
System (Meta) Level
Application (Base) Level
Adaptive Robotic Middleware (ARM)
Extends the TLAM to Optimize information flow between robots and the computational
system Determine how, when and what information should be modified in
order to match fluctuations in the communication environment
Compose communication protocols to obtain the combined benefits - conflicting requirements Explicit knowledge of how communication protocols compose and
interact is required
Adapt protocols and mechanisms to changing communication and power constraints
ARM
ARM
NSL NSL
NSL NSL
NSL NSL
NSL NSL
ASL ASL
ASLASL
VideoServer
Video/Image
processingcamera
Wireless
TactileServer
Motor Server
tactile
motortran
scei
ver transceiver
Robot
Remote Computational System
NSL – Neural Simulation LanguageASL – Abstract Schema Language
ARM: Distributed Embedded Architecture
Communication manager Provide and enforce application level requirements Components
Oracle determine most suitable protocol implementation in terms of coverage and efficiency
Set of communication protocols Protocol installer/uninstaller Resident ARM module running in the robot (resident evil)
Adaptation manager Provide adaptation and monitor mechanisms operating at different levels of
abstraction Reactive
Triggered when failure to achieve intended communication goal is detected Proactive
Triggered when a more efficient communication can be achieved under the current environment conditions
Adaptation Repository Determine most suitable adaptation strategy to be applied
ARM: Components
Protocol Oracle
ARM manager
Protocol(un)installer
Protocolloader
Secure compressed video capture
SecurityProtocol
CompressionProtocol
TimelinessProtocol
set of protocols
Robot’sresident
ARM
AdaptationProfiler
Battery Monitors
Communication Performance Monitor
AdaptationRepository
Communication Manager
Adaptation Manager
Communication Requirement
Protocol Oracle
Protocol Oracle
ARM manager
Protocol(un)installer
Protocolloader
Secure compressed video capture
Secure compressed video capture
SecurityProtocol
CompressionProtocol
TimelinessProtocol
set of protocols
Robot’sresident
ARM
AdaptationProfiler
Battery Monitors
Communication Performance Monitor
AdaptationRepositoryAdaptationRepository
Communication Manager
Adaptation Manager
Communication Requirement
ARM: Example
1. Neuroethological Modeling Study animal behavior and corresponding neural structure
as inspiration to robotic architectures
2. Embedded Mobile Robots Develop distributed wireless robot architectures capable of
efficient neural processing
3. Adaptive Middleware Achieve real-time computation and adapt embedded
architecture to varying network conditions
4. Internet Based Robotics Enable remote robot task development and
experimentation
Interned based Robotics: Web Access
Experimental Results: 2 Preys
Experimental Results: 2 Preys and Predator
(A) (B)
(C) (D)
Embedded Mobile Robots:Experimental Results: Prey Acquisition with 10 cm Barrier
(A) (B) (C) (D)
(E) (F) (G) (H)
Embedded Mobile Robots:Experimental Results: Prey Acquisition with 20 cm Barrier
Neural based Behavior: Prey Acquisition (10cm barrier)
• Barrier (PreTectum)
• Prey (Tectum)
• Integrated (MHM)
• Heading (MHM)
•Tactile
Visual Fields
Neural based Behavior: Prey Acquisition (20cm barrier before bumping)
• Barrier (PreTectum)
• Prey (Tectum)
• Integrated (MHM)
• Heading (MHM)
•Tactile
Visual Fields
Neural based Behavior: Prey Acquisition (20cm barrier after bumping)
• Barrier (PreTectum)
• Prey (Tectum)
• Integrated (MHM)
• Heading (MHM)
•Tactile
Visual Fields
Neural based Behavior: Prey Acquisition (20cm barrier after learning)
• Barrier (PreTectum)
• Prey (Tectum)
• Integrated (MHM)
• Heading (MHM)
•Tactile
Visual Fields
Future Work
Complete Internet based System Develop middleware adaptation capabilities Build smaller robotic systems Extend to multiple robot tasks Extend vision system to “true” moving forms Extend biological models
Video
Bonus Section
Neuroscience(Experiments)
RoboticsBrain Theory(Modeling)
New HypothesisGaps in Knowledge
Data, Hypotheses Formal Models
New Ideas(Results from
Experiments withPhysical Devices)
New Hypotheses
Research Cycle
T - Temporal, D - Dorsal, N - Nasal, V - VentralO - Optic Tectum, B - Nucleus of BelonciC - Lateral Geniculate Nucleus,P - Thalamic Pretectal NeuropilX - Basal Optic Root[Scalia and Fite 1974]
Neural Maps
mp
neuron
s mf
input output
mp - membrane potential : dmp(t)/dt = f(s,mp,t) mf - firing rate : mf(t) =(mp(t)) Leaky Integrator : dm(t)/dt = -m(t) + s
Neuron Model
TP – ThalamusPreTectumGL – GlomerelusSN – Stellate NeuronsSP – Small PearLP – Large PearPY - Pyramidal
TP
LP
SN
SP
GL
PY
R4R3R2
++
++
+
+-
+
++ +
+
-
+
+-
+
-
+
Output
Retina
+-
+
-
Excitation
Inhibition
Synapsis
Input
Retina-Thalamus-Tectum
vp
vf
up0up1 upi upn-1
ufn-1uf0 uf1
w0
ufi
w1wi
wn-1
s0 s1si sn-1
........
wu0wu1 wui
wun-1
wm
........
imiuii
u shvgwufwudt
tdu 1
v
dv
dt v wn f ui
i 1
n
h2
f (ui ) 1 ui 0
0 ui 0
g(v) v v 0
0 v 0
Max Selector [Didday 1976]
uf
Ulayer
vf
Vlayer
u_inv_in
s_inin
outMaxSelector
s_out
MaxSelectorStimulus
MaxSelectorModel
MaxSelectorOutput
nslModel MaxSelectorModel () extends NslModel(){
private MaxSelector maxselector(10);private MaxSelectorStimulus stimulus(10);private MaxSelectorOutput output();
public void initSys() {system.setRunTime(10.0);system.setRunDelta(0.1);
}public void makeConn() {
nslConnect(stimulus.s_out,maxselector.s_in);nslConnect(stimulus.s_out,output.s_in);nslConnect(maxselector.out, output.uf);
}}
uf s_in
Max Selector Model
uf
Ulayer
vf
Vlayer
u_in
v_in
s_inin
out
MaxSelector
nslModule MaxSelector (int size) extends NslModule() {
public Ulayer u1(size);public Vlayer v1(size);public NslDinDouble1 in(size);public NslDoutDouble1 out(size);
public void makeConn(){nslRelabel(in,u1.s_in);nslConnect(v1.vf,u1.v_in);nslConnect(u1.uf,v1.u_in);nslRelabel(u1.uf,out);
}}
Max Selector Module
nslModule Ulayer(int size) extends NslModule () {
public NslDinDouble1 s_in(size); public NslDinDouble0 v_in();public NslDoutDouble1 uf(size);private NslDouble1 up(size);private NslDouble0 hu();private double tau;
public void simRun() { up =0; uf = 0; hu = 0.1; tau =1.0;
} public void simRun() {
up = nslDiff(up,tau, -up + uf - v_in – hu + s_in); uf = nslStep(up,0.1,0.1.0);
}}
nslModule Vlayer(int size) extends NslModule () {
public NslDinDouble1 u_in(size);public NslDoutDouble0 vf();private NslDouble0 vp();private NslDouble0 hv();private double tau;
public void initRun() { vp =0; vf = 0; hv=0.5; tau=1.0;
} public void simRun() {
vp = nslDiff(vp,tau,-vp+nslSum(u_in) – hv); vf = nslRamp(vp);
}}
uf
Ulayer
v_in
s_invf
Vlayer
u _in
Ulayer and Vlayer Modules
Axon
Neuron
Dendrite
Soma
Synapse
Spine
Synaptic Cleft
AxonTerminal
Receptor
InteracellularElement
Vescicle
CalciumMechanism
NMDAReceptor
AMPAReceptor
PumpDiffusionChannel Buffer
DendritesAxon
Synapses
Soma
Neuron (detailed)
Neuroscience:Autonomous Biological Agents
Sensors Actuators Vision Sound Smell Touch
Legs Wings Fins
Robotics:Autonomous Robotic Agents
Sensors Actuators Vision Sound Smell Touch
Legs Wings FinsWheels