Upload
dinhdiep
View
222
Download
2
Embed Size (px)
Citation preview
Development of Autonomy for
Unmanned Surface Vehicles
Satyandra K. Gupta, Bob Kavetsky, Steve Lubard,
Dana Nau, Eric Raboin, Brual C. Shah, Max
Schwartz, Petr Svec, and Atul Thakur
University of Maryland, College Park
Sponsor: Office of Naval Research
Challenges in Achieving USV Autonomy
• USSV will need to perform a large
number of tasks
– Different tasks require different
behaviors for achieving autonomy
– Missions may involve intelligent
adversaries
– Handling contingencies requires
additional behaviors
• The process of humans
developing the logic for autonomy
components and coding it
requires significant effort!
Unmanned Sea Surface Vehicle
(USSV)
Planning and Control Software for
Autonomous Operations
Task Planner
Behavior Selector
Trajectory Planner
Context
Dependent
Vehicle
Capability
Model
Behaviors
Nominal Planner
Refinement Planner
Actuator
Commands
Sensor Data
Maneuvers
USSV ONBOARD COMPUTING
Manually generating
these components
takes long time!
Exception
Exception
USSV
• Automatically generate behaviors and context dependent vehicle capability models for USV from a large number of simulations and demonstrations
– Simulations are getting faster and more accurate with faster computers
– Behaviors can be audited and edited to improve if distance < 20 then
rudder angle = 30;
throttle = 40;
...
...
USV finds the right
behavior
USV explores possible actions to
limit the operation of an intruder
in the virtual environment
Project Goal
Actuator
Commands
Maneuvers Behaviors
Automatically Generated Software Libraries
Learning from
Demonstration
Simulation in Virtual Environment
Synthesis by Evolutionary
Computation Reasoning
Context Dependent Vehicle
Capability Models
Computationally
Efficient
High-Fidelity
Simulations
Task Planner Behavior Selector Trajectory Planner
Planning Software
Overview of Approach
Feedback
Control
Layer
Behavior
Layer
Trajectory
Planning
Layer
Task Planning
Layer Mission
Motion Goal
Task Sub
Goals Queue
Task Planner
Behavior
Selection
Model-Predictive
Nominal
Trajectory Planner
Task Sub
Goals
Dispatcher Exceptio
n
Exceptio
n
Exceptio
n
Atomic
Maneuvers Maneuvers
Traversability
Map
S
E
N
S
O
R
S
S
T
A
T
E
E
S
T
I
M
A
T
O
R
Traversability
Model
Behaviors
ACTUATORS
Model-Predictive
Trajectory
Refinement
Planner
Behavior Executor
Planning
Heuristics
Motion Control
System Controller
Parameters
Trajectory
Tracking
Layer
Reference
Computing
Algorithm
Waypoint
Management
Control
Allocation
USV
Guidance System
Control System
Navigation
System
Waves, wind,
currents
Actuator commands
Control
forces
𝜏
1) Developed a virtual environment – Stereo visualization
– Human vs. human and human vs. autonomy
– Enables evaluation of behaviors
2) Developed physics based meta-models – Six DOF dynamics model
– Geometric model simplification techniques
to speed up computations by factor of 28
3) Developed trajectory planners and target
following capability – Safe, efficient, and dynamically feasible
trajectory planning and target following in high sea states
4) Synthesized USV blocking behavior
against a human-competitive intruder – Intruder with probabilistic actions
– Synthesized behavior was comparable to
a hand-coded behavior
5) Synthesized strategies and behaviors
for multi-USV team to protect asset from
threat of possible adversaries – USVs had to deal with uncertainty in recognition of
intruder boats
Summary of Accomplishments
Buffer zone
Danger zone
Friendly certified
boats
Target USV
Intruder
• Developed interactive rate
physics-based virtual simulation
environment for USVs
– Modeled interactions of waves and
the vehicle
– Real-time ocean rendering
– Stereo visualization
– Intuitive user interface
– Modeling of boats, terrain, and
shorelines
– Modular software design
• Can be used for measuring
performance of hand coded and
computer synthesized behavior
P. Svec, A. Thakur, D.K. Anand, S.K. Gupta, and M. Schwartz. A simulation based framework for discovering planning logic for
autonomous unmanned surface vehicles. ASME Engineering Systems Design and Analysis Conference, Istanbul, Turkey, July, 2010.
Accomplishment #1: Virtual Environment
1) A. Thakur, and S.K. Gupta, Real-time dynamics simulation of unmanned sea surface vehicle for virtual environments. Journal of
Computing and Information Science in Engineering, 11(3), 2011, 2-11.
2) A. Thakur, P. Svec, and S. K. Gupta. GPU based generation of state transition models using simulations for unmanned sea
surface vehicle trajectory planning, Robotics and Autonomous Systems, In review.
Accomplishment #2: Physics-Based Meta-Models
Boat
geometry Ocean
Model
Simplify based on
clustering
Simplify based on
temporal coherence
Parallelization on CPU
cores using OpenMP
Parallelize Monte Carlo
runs
Parallelize each run
across wet facets
Simplify based on
temporal coherence
CPU based simplification GPU based acceleration
Initial
State
• Developed high-fidelity simulation model and
corresponding physics based meta-model
– Six DOF dynamics model
– Geometric model simplification techniques to
speed up computations by factor of 42
– Combined with GPU computing for further
speed up
0
10
20
30
40
50
1 4 16 64 256
Sp
eed
up
facto
r o
ver
CP
U b
aseli
ne
Number of simultaneous runs GPU acceleration
CPU based simplification (τ=0.07, C=50 Thakur and Gupta 2011) GPU and temporal coherence based simplification τ=0.075
Accomplishment #3: Trajectory Planning
1. Manually designed
control actions
Au
Cu
Bu
'B
u
'C
u
Au
Cu
Bu
'B
u
'C
u
Au
Cu
Bu
'B
u
'C
u
2. Monte Carlo simulations of
control actions
3. Manually designed
probabilistic transitions
4. 3D lattice based trajectory
planning over space of (x, y, θ)
5. Localized search in space of
probabilistic transitions
Expected
collision
probability:
0.001 1.0
0.0 0.0
Collision
probability
0.1
0.01
0.1
0.8
0.1
Motion Commands Computed Control Set Variation in Final Orientation
• A state transition maps a given
state and action combination to
another state
• Motion commands are
implemented as position tracking
using PID controller
• Position uncertainty is captured by
running sufficient number of
simulation trials θ =2π
θ = 0
si
X Nominal
trajectory
obtained for
action u with
no ocean
disturbance
se
sj X X X
X
Ending state
for kth
simulation
run X XX
X X
XX
X X
X XX
X
X
XX X
X X X X
X
Accomplishment #3.1: Computing State Transition
Probability Using High-Fidelity Simulations
A. Thakur, P. Svec, and S. K. Gupta. GPU Based Generation of State Transition Models Using Simulations for Unmanned Sea
Surface Vehicle Trajectory Planning, Robotics and Autonomous Systems, In review.
Trajectory for sea state 3
(avg. wave height 0.5 to 1.25 m)
Trajectory for sea state 4
(avg. wave height 1.25 to 2.5 m)
Value iteration used for obtaining optimal feedback plan
A. Thakur, P. Svec, and S.K. Gupta. Generation of State Transition Map Using Unmanned Sea Surface Vehicle (USSV) Simulation,
ASME 2011 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference
(IDETC/CIE ‘11), 2011.
Accomplishment #3.2: Computing Trajectories For
Various Sea States
P. Svec, M. Schwartz, A. Thakur, and S.K. Gupta. Trajectory Planning with Look-Ahead for Unmanned Sea Surface Vehicles to
Handle Environmental Uncertainties, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS '11), San
Francisco, September 25-30, 2011.
Accomplishment #3.3: Incorporating
Look-Ahead Into Trajectory Planning
• Standard stochastic dynamic programming
algorithms too slow to handle motion
uncertainty for nonholonomic vehicles
• A* or D* based heuristic search alone does
not handle motion uncertainty
• Developed a look-ahead based algorithm
for trajectory planning under motion
uncertainty for USVs
• Motion uncertainty is modeled
and explicitly used during the computation
of the trajectory
• Contingency plan is generated to efficiently
handle sudden large deviations from the
intended trajectory
• Developed follow behavior for tracking
moving target in complex environment
with obstacles
‒ Tracking influenced by USV dynamics,
motion characteristics of moving target,
and complexity of environment
‒ USV does not know trajectory of target
in advance
‒ USV predicts future pose of target based
on its motion characteristics and
configuration of obstacles
‒ USV keeps target within user-specified
distance range (𝑟𝑚𝑖𝑛, 𝑟𝑚𝑎𝑥)
‒ USV computes dynamically feasible
trajectory to reach target
‒ USV applies velocity and position
feedback control to reject disturbances
due to ocean waves
Accomplishment #3.4: Target Following with
Motion Prediction for USVs Operating in Cluttered Environments
Accomplishment #3.4: Target Following with
Motion Prediction for USVs Operating in Cluttered Environments (Cont.)
Buffer zone
Danger zone
Friendly certified
boats
Target USV
Intruder
• Case study: USV supposed to
block the advancement of an
intruder boat
─ Needs a reactive action selection
policy to compute motion goals
• Intruder exhibits a human-
competitive, deceptive
attacking behavior
─ USV cannot exploit regularity in
the intruder’s behavior due to its
probabilistic actions
Accomplishment #4: Action Selection Policy
Synthesis For Blocking Task
P. Svec and S.K. Gupta. Automated Synthesis of Action Selection Policies for Unmanned Vehicles Operating in Adverse
Environments. Autonomous Robots, Accepted for publication.
• Representative portions of
intruder’s attacking behavior
Target
Target
Accomplishment #4: Action Selection Policy
Synthesis For Blocking Task (Cont.)
P. Svec and S.K. Gupta. Automated Synthesis of Action Selection Policies for Unmanned Vehicles Operating in Adverse
Environments. Autonomous Robots, Accepted for publication.
Termination
criterion
met?
No
Yes
Find representative
set of states of
exception REPE
S,
Evolve policy
component to
handle REPE
S,
i
Optimize and
integrate into
the overall policy i
U
Evaluate policy U
States of
failure F
S
Representative
set of states of
exception REPE
S,
Policy
component
ASii:
Local
test cases
Default policy
Evolved
part
Hand-coded
part
D
Test cases T
Generate
local test
cases i
T
iT
USV policy
UUAS :
Return U
Intruder policy
IIAS :
World
Coordinate
Space
1) Detect failure state sF
2) Find
exception
state sE
by going
back in
time
Iterative State Space
Partitioning
3) Synthesize action Ai for
sE and its neighboring
states
sE
Accomplishment #4: Action Selection Policy
Synthesis For Blocking Task (Cont.)
P. Svec and S.K. Gupta. Automated Synthesis of Action Selection Policies for Unmanned Vehicles Operating in Adverse
Environments. Autonomous Robots, Accepted for publication.
Automated
Policy Synthesis
Approach
Accomplishment #4: Action Selection Policy
Synthesis For Blocking Task (Cont.)
P. Svec and S.K. Gupta. Automated Synthesis of Action Selection Policies for Unmanned Vehicles Operating in Adverse
Environments. Autonomous Robots, Accepted for publication.
Hand-
coded
average
Accomplishment #4: Action Selection Policy
Synthesis For Blocking Task (Cont.)
P. Svec and S.K. Gupta. Automated Synthesis of Action Selection Policies for Unmanned Vehicles Operating in Adverse
Environments. Autonomous Robots, Accepted for publication.
Example of a run in which the USV managed to block the intruder for additional 45
seconds. The start position of the USV is marked as 1.1 while the start position of
the intruder is marked as 2.1.
• Synthesized strategies and
behaviors for multi-USV team to
protect asset from the threat of
possible adversaries
‒ USVs must deal with uncertainty about
which boats are adversaries
‒ Difficult task allocation due to multi-task
vehicles, multi-vehicle tasks, and time-
extended assignment
• Online simulation-based task
allocation
‒ Used predictive simulation for
task allocation evaluation
• Behavior learning through
parameter optimization
‒ Must learn behaviors for each
task
Danger
Zone
Asset
USV
2 Intruder
b2
P(b2 is intruder)=0.8
P(b1 is intruder)=0.1
USV1
USV3
Boat b1 Intruder b3
P(b3 is intruder)=0.4
• Team of 5 USSVs
• 6-8 passing boats
• 2-3 intruders,
Result of 1000
randomly generated
trials
• Predictive is 26%
better than baseline
Accomplishment #5: Generating Behaviors to
Support Multi-USSV Guarding