Christian LAUGIER – e-Motion Team-Project 1
The The e-Motione-Motion Team-Project Team-Project
« Geometry and Probability for motion and action »« Geometry and Probability for motion and action »
Inria Rhône-Alpes & Gravir laboratory (UMR 5527)
Scientific leader : Christian LAUGIER
Christian LAUGIER – e-Motion Team-Project 2
Team-Project membersTeam-Project members (year 2004)(year 2004)
• Permanent staffPermanent staff– Christian Laugier, DR2 Inria (Scientific leader)– Emmanuel Mazer, DR2 Cnrs (External collaborator currently in our start-up « Probayes »)– Pierre Bessiere, CR1 Cnrs– Thierry Fraichard, CR1 Inria– Sepanta Sekhavat, CR1 Inria (Currently in Iran for 2 years)– Olivier Aycard, MC UJF– Anne Spalanzani, MC UPMF
• Invited researchers, Postdocs, and EngineersInvited researchers, Postdocs, and Engineers– Juan-Manuel Ahuactzin– Olivier Malrait– Kamel Mekhnacha– Jorge Hermosillo– Christophe Coué
• PhD studentsPhD students– 7 theses defended in 2003: J. Diard, C. Mendoza, R. Garcia, J. Hermosillo, F. Large, C. Coué, K. Sundaraj– 8 PhD students: C. Pradalier, C. Koike, M. Amavicza, R. Lehy, F. Colas, D. Vasquez, P.Dangauthier, M. Yguel– 3 PhD students co-directed : M. Kais (rocquencourt), S. Petti (rocquencourt), B. Rebsamen (NUS)– 5 Master students : Christopher Tay, Alejandro Vargas, Ruth Lezama, Christophe Braillon, Julien Burlet
Christian LAUGIER – e-Motion Team-Project 3
Objectives & motivationsObjectives & motivations
• Scientific challenge :Scientific challenge : To develop new models for constructing « artificial systems » having sensing, decisional, and acting capabilities sufficiently efficient and robust for making them really operational in open (i.e. large & weakly structured) and dynamic environments
• Practical objective :Practical objective : To built operational (i.e. scalable) systems having the capability to « share our living space » in some selected application domains (e.g. transportation, personal robots …).
• Current technological context :Current technological context : (1) Continuous & fast growing of computational power; (2) Fast development of micro & nano technologies (mechatronics); (3) Increasing impact of information & telecom technologies on our everyday life (ambiant intelligence)
• Motivations & difficulties :Motivations & difficulties : Instead of promises & impressive advances in robotics in the last decade, almost no advanced robots are currently evolving around us !
=> Not reliable enough, Weak reactivity and low efficiency (real time constraints), Exhibit “cybernetic” behaviors, Quite difficult to program (with no real learning capabilities)
Christian LAUGIER – e-Motion Team-Project 4
A large spectrum of potential applicationsA large spectrum of potential applications
Virtual autonomous agents Virtual autonomous agents (natural interaction with humen)
Rehabilitation & Medical robotsRehabilitation & Medical robots
Future carsFuture cars(driving assistance & autonomous driving)
““Personal Robot Assistant”Personal Robot Assistant”
Main considered applications
=> Transport & logistics, Services, Rehabilitation & Medical care, Entertainment …
Christian LAUGIER – e-Motion Team-Project 5
Cooperation & ContractsCooperation & Contracts
• International cooperationInternational cooperationJapan (Riken), Singapore (NTU, NUS, SIMTech), USA (UCLA, Stanford, MIT), Mexico (IESTM Monterrey, UDLA Puebla), Europe (EPFL, Univ college of
London)
• Industrial cooperationIndustrial cooperationRobosoft, Renault, PSA, XL-Studio, Aesculap-BBrown, Teamlog, KelkooStartups: ITMI, Getris Images, Aleph Technologies, Aleph Med, Probayes (oct. 2003)
• R&D contractsR&D contracts– Industrial : ACI « Protege », Priamm « Visteo », RNTL « Amibe », Kelkoo– National : Robea (EVbayes, Parknav), Predit (Arcos, MobiVip, Puvame)– Europe : NoE « Euron », IST-FET « Biba », IST « Cybercars », IST « Prevent»
Christian LAUGIER – e-Motion Team-Project 6
SScientific approachcientific approach
• Our approachOur approach To focus on « complexity » and « incompleteness » problems (scalability & real world ) We guess that this can be achieved by combining geometrical and probabilistic approaches
• Main difficultiesMain difficulties– Previous approaches on AI & Robotics have shown their limitations
=> Logics (70’s), Geometry (80’s), Random search (90’s), purely Reactive Architectures (90’s)– The real world is too complex for being fully modeled using classical tools (inparticular: incompleteness & uncertainty)
=> Additional methods are required (e.g. probabilistic programming)=> Biologic inspiration could bring some help (sensori-motors systems, internal representationsfor motions… which seems mostly based on probabilistic laws)
• Required technological breakthroughsRequired technological breakthroughs– Motion & action autonomy in a complex dynamic world
=> Incremental world modeling, time-space dimension, prediction & estimation of obstacles motions– Increased robustness & safety of navigation systems (perception & control)
=> Dealing with incompleteness & uncertainty– “Easy” programming & system adaptativity
=> Self learning capabilities & behaviors coding and mixing
Christian LAUGIER – e-Motion Team-Project 7
Two complementary reasoning processesTwo complementary reasoning processes
Mastering the complexity by using the right reasoning level & incremental approaches
Taking explicitly into account the hidden variables at the reasoning level
Dynamic worldDynamic world
Space & MotionSpace & MotionModelsModels
Analytical & Statistical dataSensing data
Geometric & Kinematic reconstruction,SLAM
Motion prediction
Motion planMotion plan& Navigation controls& Navigation controls
Constrained Motion PlanningDifferential Flatness,Velocity Space, ITP
IncompletnessIncompletness
UncertaintyUncertainty
Preliminary Knowledge+
Experimental Data=
Probabilistic Representation
Maximum EntropyPrinciple
ii PP log
Queries &Queries &Decision ProcessDecision Process
Bayesian Inference(NP-Hard [Cooper 90]Heuristics & Optimization)
P(AB|C)=P(A|C)P(B|AC)=P(B|C)P(A|BC)P(A|C)+P(¬A|C) = 1
Christian LAUGIER – e-Motion Team-Project
Bayesian Programming (principle)Bayesian Programming (principle)D
escr
ipti
onD
escr
ipti
onQ
ues
tion
Qu
esti
on
Bay
esia
n P
rogr
amB
ayes
ian
Pro
gram
• Specification Specification => Preliminary Knowledge – Set of relevant probabilistic variables
– Parametric forms assigned to some of the terms appearing in the decomposition (gaussian, uniform …)
– Decomposition of the joint distribution (=> efficient way to compute it)
nXXX ,...,, 21
kkn RLRLLXXX |P...|P|P|...P 22121
iR
ii LRL i ,
f|P
• Identification Identification (of some of the terms of the decomposition)
=> Experimental Data P Li | Ri P Li | Ri
=>Inference (i.e. answer to the question) carried out by an “inference engine” (symbolic simplification + numerical computation)
Unknown
nXXXKnownSearch |...P1
|P 21
e.g. using ProBT software (Probayes)
Christian LAUGIER – e-Motion Team-Project 9
Research axesResearch axes• Multi-modal modeling of space & motionMulti-modal modeling of space & motion
– Incremental world modeling– Prediction & estimation of obstacles motions– Sensori-motors maps
Sensori-motors space(bayesian maps)Dynamic SLAM
q
q
t
0,, ii qq tqq gg ,,
• Motion planning in a dynamic worldMotion planning in a dynamic world– Iterative trajectory planning (ITP)– Instantaneous escaping trajectories (Velocity space)– States of unavoidable collisions (State-time space)
NH constraints
Space-Time constraints
Sensory data fusion(situation analysis)
Learning reactive behaviors
• Decision in an uncertain world (Bayesian inference)Decision in an uncertain world (Bayesian inference)– Bayesian programming tools– Automatic learning & Entropy maximization– Biological inspiration
Christian LAUGIER – e-Motion Team-Project 10
Autonomous navigation & Easy robot programmingAutonomous navigation & Easy robot programming
SLAMSLAM++
NH Motion planningNH Motion planning++
Reactive nReactive navigationavigation
Several functionalities (learned and downloaded) have to be combinedIncremental world modeling & localization + Motion planning + Autonomous sensor-based navigation
[Pradalier & Hermosillo 03]
Christian LAUGIER – e-Motion Team-Project 11
Bayesian reactive behaviorsBayesian reactive behaviors
Current work: More complex situations & Learning, Better integration with Control
=> Controlling the vehicle using a probability distribution on v e.g. reducing speed and/or modifying steering angle for avoiding a pedestrian or a car
Gaussian functionCommand fusionCommand fusion
)(411
xe
Sigmoïde f
V Area 1
Area 2
Area 8
8
181 )/()()...(
iii VDPVPDDVP
Uniform)( VP
iDiiiiii
iiiiiiii DPDVPDP
DPDVPDPVDP
)/()/()(
)/()/()()/(
where :
Joint distribution for the fusion :
Probabilistic joint distribution for area i
[Pradalier et al. 03]
Tracking a given trajectory while avoiding sensed obstacles
Christian LAUGIER – e-Motion Team-Project 12
Bayesian programming : some experimental resultsBayesian programming : some experimental results
Reactive trajectory tracking (Cycab) Target following (Koala)
Christian LAUGIER – e-Motion Team-Project 13
V-Obstacles & ITP (dynamic environment)V-Obstacles & ITP (dynamic environment)
• Instantaneous escaping trajectories (V-obstacles) => Strategies for avoiding moving obstacles• Iterative Trajectory Planning => Complete navigation strategy
Relative VelocityCone (RVC)
)(ˆ)()(
)(ˆ)()(
)(
)()(:)()(
)(
tct
ritctvo
tct
ritctvo
etc
etdtctrajectoryObstacle
lvl
lvr
tittd
v
ti
Current work: More complex geometry & dynamics, Perception, Uncertainty
V-obstacles
V-obstacles+
ITP
[Large et al. 03]
Christian LAUGIER – e-Motion Team-Project 14
Robust obstacles tracking Robust obstacles tracking (Bayesian Occupancy Filter approach)(Bayesian Occupancy Filter approach)
Occupiedspace
Freespace
Unobservable space
Concealed space
(“shadow” of the obstacle)
1 sensor & 1 object
2 sensors & 3 objects
z1,1 = (5.5, -4, 0, 0) z1,2 = (5.5, 1, 0, 0)z2,1 = (11, -1, 0, 0) z2,2 = (5.4, 1.1, 0,0)
P([Ec=1] | z1,1 z1,2 z2,1 z2,2 c)c = [x, y, 0, 0]
P( [Ec=1] | z c)c = [x, y, 0, 0] and z=(5,2,0,0)
Occupancy grid
– Variables :• C : cell• EC : cell occupancy (EC=1 means“occupied”)
• Z1:S : observations• M1:S : association (1 for each sensor)
– Parametric form :• P(EC C) : a priori uniform• P(Z1:S | EC C) : sensor models
Pro
gram
Pro
gram
Des
crip
tion
Des
crip
tion
Qu
esti
onQ
ues
tion
• UtilizationUtilization
• SpecificationSpecification
• IdentificationIdentification => Calibration
– Decomposition :
For all c : P(Ec | Z1:S c)
S
s
s
O
iSCis
S
ssC
SSC
MCEZPMPCEP
ZMECP
1,
1
:1:1
)( )()()(
)(
Christian LAUGIER – e-Motion Team-Project 15
Robust obstacles tracking & avoidance Robust obstacles tracking & avoidance Experimental results with the CycabExperimental results with the Cycab
Prediction
Estimation
Bayesian Occupancy Filter (BOF)Bayesian Occupancy Filter (BOF)
Pedestrian avoidance using the BOFPedestrian avoidance using the BOF
Christian LAUGIER – e-Motion Team-Project 16
Automatic reconstruction of a dynamic mapAutomatic reconstruction of a dynamic map(Parkview : Multi-camera system)(Parkview : Multi-camera system)
Dynamic Map
Static components Mobile components Changing components
Fusion
[Helin 03]
Christian LAUGIER – e-Motion Team-Project 17
Trajectory prediction for moving obstaclesTrajectory prediction for moving obstacles
22
),(2
1
2
1)|(
kpp d
k
kp eCdP
2/1
0
2)()(1
),(
dttdtdT
ddpT
t
ipp
jpp
Learning phase : Learning phase : Clustering
kN
ii
kk td
Nt
1
)(1
)(
2/1
1
2),(1
kN
iki
kk d
N
Mean trajectory :
Standard deviation :
Prediction phase (at each time step) :Prediction phase (at each time step) :Calculating the likelihood thatCalculating the likelihood that kp Cd
Partial distance :
Likelihood estimation :
[Vasquez 04]
Christian LAUGIER – e-Motion Team-Project 18
Some previous results ofSome previous results ofour research teamour research team
• Interactive medical simulationInteractive medical simulation
• Autonomous navigation for Virtual Reality Autonomous navigation for Virtual Reality applicationsapplications
Christian LAUGIER – e-Motion Team-Project 19
Echographic simulatorEchographic simulator (c(coop. TimC & UC-Berkeley & LIRMM)[Daulignac & Laugier 00-01]
Stress-strain curves(measured)
)(
)(
linearnonbxa
xF
linearxkF
Geometric model
Measured data
+
Separating the Gall-bladderfrom the liver
[Boux & Laugier & Mendoza 01]
Cutting a flag using an haptic device[Boux & Laugier 00]
Interactive medical simulationInteractive medical simulation
Christian LAUGIER – e-Motion Team-Project 20
• Dynamic path planning : Dynamic path planning : Adriane’s Clew Algorithm [Ahuactzin 94]
• Reactive navigation : Reactive navigation : => => Path tracking & Obstacle avoidance [Raulo &Laugier 00] => Bayesian behaviors [Lebeltel 99, Raulo 01]
Solving : P(Solving : P(Vrot Vtrans | | px0 px1 … … lm7 veille feu … … per))
Question : P( M | S Cp_Surveil)
._P....
SurveilCp
permldmnvdnv
vtrans_cproxGdirGproxdir tourtempo1-td_t1-tach_tengobj?feuveillelm7lm0px7px0
VtransVrot
Autonomous navigation for Virtual Reality Autonomous navigation for Virtual Reality applicationsapplications
Christian LAUGIER – e-Motion Team-Project 21
Autonomous drivingAutonomous driving& &
Driving assistanceDriving assistance
Christian LAUGIER – e-Motion Team-Project 22
The « CyberCars » approachThe « CyberCars » approach Door to door, 24 hours a dayDoor to door, 24 hours a day Small (urban size), silent Small (urban size), silent User friendly interface User friendly interface Automatic manoeuvresAutomatic manoeuvres => parking, platooning
… up to fully automated
CyberCars are focusing on historical city centres
Praxitele : Real experiment in SQY (97-99)Praxitele : Real experiment in SQY (97-99)
Industrial siteTrain station
Shopping centre
Tramway Metro
TGV
Bus
Private Car
Wal
k
FarNear
Low
High
Distance
Cap
aciy
....
> 500 m
BikeRoller
CyCab dual-mode vehicleCyCab dual-mode vehicleCommercialized by Robosoft
Private tracksLocal tracks
Pedestrian zonesCalm zones
Suburban tracks
Intercity tracksFull driving automation on :
Christian LAUGIER – e-Motion Team-Project 23
Some CyberCars projectsSome CyberCars projects
ParkShuttle (Frog, Netherlands)ParkShuttle (Frog, Netherlands) Serpentine (Switzerland)Serpentine (Switzerland) PraxitelePraxitele ( (France)France)
European European CybercarsCybercars project (2001-05) project (2001-05) 10 industrial partners 10 industrial partners (Fiat, Yamaha, Frog …),(Fiat, Yamaha, Frog …), 7 research institutes 7 research institutes (Inria, Inrets,
Ensmp …), 12 cities involved , 12 cities involved (Rome, Rotterdam, Lausanne, Antibes …)(Rome, Rotterdam, Lausanne, Antibes …) 10 M €10 M €
E-Cab (Yamaha)E-Cab (Yamaha)CyCab (Robosoft)CyCab (Robosoft) ParkShuttle (Frog)ParkShuttle (Frog)
Serpentine (SSA)Serpentine (SSA)
Christian LAUGIER – e-Motion Team-Project 24
The The « Automotive »« Automotive » approach approachADAS : Advanced Driver AssistanceADAS : Advanced Driver Assistance
European Prometheus project (86-94)European Prometheus project (86-94)
R&D programR&D program ( (on-board and off-board systemson-board and off-board systems) for increasing safety & driving confort) for increasing safety & driving confort
ACCStop&Go
Stop&Go ++
Rural Drive Ass.
Urban Drive Ass.
Full DrivingAutomation
Longitudinal ControlLongitudinal Control
+ Lateral Control+ Lateral Control
Current projects: Carsense, Arcos, PreventCurrent projects: Carsense, Arcos, Prevent
Carsense (car manufacturers & suppliers)Carsense (car manufacturers & suppliers)Sensor fusion for danger estimationSensor fusion for danger estimation
French Arcos project: French Arcos project: Vehicle-Infrastructure-Driver systems for road safetyVehicle-Infrastructure-Driver systems for road safety
Christian LAUGIER – e-Motion Team-Project 25
Experimental vehicles at INRIA Rhône-AlpesExperimental vehicles at INRIA Rhône-Alpes
• electric vehicle with front driven and steering wheels• abilities of human or computer-driven motion• control system: VME CPU-board, transputer net• sensor : odometry, ultrasonic sensors, linear CCD camera
Ligier vehicleLigier vehicle
CycabCycab
Models for controlling the CycabModels for controlling the Cycab
22
2cos
BA
fLM
0 2
322
2 ''cos
uBuA
uBuAuAuBufLN
fffB
ffA
sincos'sincos
cos'cos 22
• Existence (Differential flatness) [ Sekhavat, Hermosillo, 99]
• Necessary conditions (« flat outputs ») [Sekhavat, Rouchon, Hermosillo 01]
• Analytic determination of the « flat outputs » [Sekhavat, Hermosillo, Rouchon 01]
• Motion planning & control (trajectory tracking) [Hermosillo 03] [Hermosillo & Pradalier 04]
)(
)()(1
cos'cos
tan)(),,(:
f
AB
ufuft
ttF withframeTurning
21
1
0
0
0
0cos
sinsin
cos
uu
L
ff
f
y
x
R
R
Differential Flatness of the CycabDifferential Flatness of the Cycab
Decisional & Control architectureDecisional & Control architecture
continuous curvature profile + upper-boundedcurvature & curvature derivative
continuous curvature profile + upper-boundedcurvature & curvature derivative
Kinodynamic Motion Planning(Dynamic constraints ...)
q
t
Graph
Trajectory
[Fraichard 92 ]Planning CC-paths
(kinematic constraints ...)
x
y
)(tan 1 w
w
[Scheuer & Laugier 98 ]
3-layered control architecture[Laugier et al. 98 ]
Decision module
Reactive mechanism=> Control the executionof the selected skills
Real-time “Skills” => Close-loop controls & Sensor processing
Platooning [Parent & Daviet 96] Automatic Parallel Parking [Paromtchik & Laugier 96]
Lane Changing & Obstacle avoidance [Laugier et al. 98]
Christian LAUGIER – e-Motion Team-Project 28
« Platooning »« Platooning »
Electronic « Tow-bar »
CCD Linear camera + Infrared target(high rate & resolution)
[Parent & Daviet 96]
Christian LAUGIER – e-Motion Team-Project 29
Automatic parking maneuversAutomatic parking maneuvers
Start location specification
[Paromtchik & Laugier 96]
( ) ( ),
( ) ( ),, , ,
( )
,
cos ,
,
, ,
( ) . cos ,
max
maxmax max
*
**
t k A t t T
v t v k B t t Tv k k
A t
t tt t
Tt t T t
T t t T
tT T
T T
B t t T
vv
,
0
00 0 1 1
1 0
12
05 1 4 0 t T
=> On-line motion planning using sinusoidal controls (t) and v(t)(search for control parameters T and max)
On-line local world reconstruction& Incremental motion planning