Models, Algorithms, and Evaluation for Autonomous Mobility-On-Demand Systems

Marco PavoneAutonomous Systems Laboratory Department of Aeronautics and AstronauticsStanford University

Seminar @Stanford Platform Lab

January, 31 2017

Autonomous Systems LabASL

Research Synopsis

Stochastic Optimal ControlReal-Time Control/Optimization

Distributed Computing

Enabling Technologies

Research Areas

Controlling Individual AS:- Robotic Motion Planning

- Risk-Sensitive Decision-Making

Deploying AS:- Autonomous Spacecraft/Space Robots- Autonomous Transportation Networks

Autonomous Systems (AS)Key Approach:

Optimization-Based Control

Controlling Networked AS:- Task-Based Coordination of Robots

- Complexity Theory for Robotic Networks

Controlling Individual AS

• Value of time (productivity/leisure)4 = $1.3T

• Throughput:• Economic cost of congestion (time/fuel

wasted)2 = $160B• Health cost of congestion (pollution)3 =

$15B• Enabling carsharing on a massive

scale4 = $402B

Potential Benefits of Self-Driving Cars (US Market)

• Safety1:• Value of a statistical life = $9.1M• Economic cost of traffic accidents = $242B• Societal harm of traffic accidents (loss in

lifetime productivity) = $594B

1 [Blincoe et al., NHTSA Report, ’15]2 [Schrank et al., Texas A&M Transportation Institute, ’15]3 [Levy et al., Environmental Health, ’10]4 [Spieser, Treleaven, Zhang, Frazzoli, Morton, Pavone, Road Vehicle Automation, ’14]

A New Paradigm for Personal Urban MobilityVehicle Autonomy Car Sharing

+"Autonomous Mobility-On-Demand (AMoD)

Research objectives: 1. Modeling: stochastic models for tractable analyses2. Control: real-time routing of autonomous vehicles at a city-wide scale3. Applications: case studies and technology infusion

How to Control a Fleet of Autonomous Vehicles?

Problem falls under the general class of networked, heterogeneous, stochastic decision problems with uncertain information:• Problem Data / Model: travel demand, road network• Control inputs: vehicle routing, passenger loading/unloading• Outputs: customer waiting times, customer queue lengths, etc.

Key features:Static version NP-hard Dynamics add queueing phenomena

Closed system:cascade feedback effects

Closed-loop control policies aimed at optimal throughput

SystemController

Previous Work and Its Limitations• Traffic modeling:

• Static models [Wardrop ’52]• Simulation models [Treiber, Hennecke, Helbing, ‘00]• Queueing models [Osorio, Bierlaire, ‘09]

Focus on analysis rather than on control

• Dynamic Traffic Assignment [Janson ’91]: does not address some operational constraints (e.g., rebalancing)

• Dynamic vehicle routing (pickup-and-delivery problems) [Psaraftis ’88], [Berbeglia, Cordeau, Laporte ’10], [Pavone ’10], [Treleaven, Pavone, Frazzoli ’13]: results only available for light-load or heavy-load situations; difficult to assess quality-of-service.

• Fluidic models of AMoD [Pavone ’12]: does not account for stochastic nature of the system

Need approach that (1) allows both analysis and control, (2) enables the inclusion of operational constraints, and (3) takes into account stochasticity

7

A Family of Models for Control and Evaluation

Spatial queueing-theoretical models

Jackson queueing-theoretical models

Stochastic MPC models

Macroscopic Microscopic

Analytical Computational

[Pavone, MIT ’10], [Treleaven, Pavone and Frazzoli, TAC ’13]

[Pavone, Smith, Frazzoli, Rus, IJRR ’12],[Zhang, Pavone, IJRR ’15][Zhang, Rossi, Pavone, ‘16]

[Zhang, Rossi, Pavone, ICRA ’16], [Chow, Yu, Pavone, IJCAI, ‘16]

• Complements simulation-based methods such as [Osorio, Bierlaire, OR ’13], [Fagnant, Kockelman, TR C ’14], [Fagnant, Kockelman, Bansal, JTRB ’15], [Shen, Lopes, PRIMA ’15].

• Leverages vast literature on VRP [Toth, Vigo, ’14], DTA [Janson, TR B, ’91], and queueing theory [Larson, Odoni, ’81], [Gelenbe, Pujolle, Nelson, ‘98], [Osorio, Bierlaire, OR ’09]

Models and Controls

Jackson Queueing-Theoretical Model

[Zhang, Pavone, RSS ’14 - best paper award finalist], [Zhang, Pavone, IJRR ’15]

⇡i =X

j

⇡jpji

Model:• Stochastic model with passenger loss• - arrival rate of customers• - routing probabilities• - travel times• Balance Equations:• Stationary distribution:

�i

pij

Tij

P(X) =1

G(m)

|N |Y

j=1

⇡xj

j

xjY

n=1

µj

(n)�1

�i = ⇡i/�i• Relative utilization:

Ai(m) = �iG(m� 1)

G(m)Performance metric: vehicle availability

�

�� ����

���� ����

����

��������

�2infinite-server queue

�1

�3

10

Vehicle Rebalancing

Key idea: Stochastic “rebalancing-promoting” policy using “virtual” customers with arrival rates and routing probabilities i ↵ij

minimize

i,↵ij

X

i,j

Tij ↵ij i

subject to �i = �jX

j

↵ij = 1

↵ij � 0, i � 0 i, j 2 {1, . . . , N}

Optimal Rebalancing Problem (ORP): Given an autonomous MOD system modeled as a closed Jackson network, solve

where �i =⇡i

�i + i

[Zhang, Pavone, IJRR ’15]

Results

• Provides theoretical justification to earlier fluidic approximations [Pavone, Smith, Frazzoli, Rus, IJRR ’12]

• Rather flexible model: BCMP extension [Iglesias, Rossi, Zhang, Pavone, WAFR ‘16], different objectives, e.g., [Thai, Chenyang, Bayen, ’16], etc.

0 50 100 150 2000

0.2

0.4

0.6

0.8

1

Number of vehicles

Prob

abilit

y at

leas

t 1 v

ehic

le is

ava

ilabl

e

Unbalanced System

0 50 100 150 2000

0.2

0.4

0.6

0.8

1

Number of vehicles

Prob

abilit

y at

leas

t 1 v

ehic

le is

ava

ilabl

e

Balanced System

Real-Time Control Algorithms

• Optimal rebalancing policy found by solving a linear program

minimize

�ij

X

i,j

Tij�ij

subject to

X

j 6=i

(�ij � �ji) = ��i +

X

j 6=i

pji�j

�ij � 0

• In practice, routing problem solved with an MPC-like algorithm

minimize

numij

X

i,j

Tijnumij

subject to vei (t) +X

j 6=i

(numji � numij) � vdi (t) for all i 2 S

numij 2 N for all i, j 2 S, i 6= j

[Zhang, Pavone, IJRR ’15], [Zhang, Rossi, Pavone, ICRA ’16]

System-Level Control of MoD Systems

[Zhang, Pavone, IJRR ’15]

Evaluation

Evaluation: Case Study of Manhattan

Key result: fleet size reduced by ~40%!

• Trip data collected on March 1, 2012, consisting of 439,950 trips within Manhattan

• ≈ 13,300 taxis• 100 stations for robotic MoD

[Zhang, Pavone, IJRR ’15]

0 2000 4000 6000 8000 10000 12000 140000

0.2

0.4

0.6

0.8

1

m (Number of vehicles)

Vehi

cle

avai

labi

lity

Peak demandLow demandAverage demand

0 5 10 15 200

5

10

15

20

25

Time of day (hours)

Expe

cted

wai

t tim

e (m

in)

6000 Vehicles7000 Vehicles8000 Vehicles

Evaluation: Case Study of Singapore

• Three complementary data sources: HITS survey, Singapore taxi data, Singapore road network

• 779,890 passenger vehicles operating in Singapore• 100 stations for robotic MoD

0 5 10 15 20 230

10

20

30

40

50

60

70

Time of day (hours)

Aver

age

wai

t tim

e (m

in)

200,000 vehicles250,000 vehicles300,000 vehicles

COS COT TMC

Traditional 0.96 0.76 1.72

AMoD 0.66 0.26 0.92

Mobility-related costs (USD/km)

[Spieser, Treleaven, Zhang, Frazzoli, Morton, Pavone, Road Vehicle Automation, ’14]

Key result: total mobility cost cut in half!

Current Research Directions

sisj

ti

tj

Does AMoD Increase congestion?

{(sm, tm,�m)}m

G(V,E)

Network flow model:• Road network represented by a graph with road capacities

(threshold model)• Trip requests represented by set of origins/destinations/rates• Vehicles’ motion represented as flows on edges:

• customer flows:• rebalancing flows:

fm(u, v), u, v 2 V

c(u, v), u, v 2 V

fR(u, v), u, v 2 V

sisj

ti

tj

[Zhang, Rossi, Pavone, RSS ’16], [Iglesias, Rossi, Zhang, Pavone, WAFR ‘16]

It has been argued that the presence of many rebalancing vehicles may contribute to an increase in congestion, e.g., [Templeton, ’15], [Barnard, ’16], [Levin, Li, Boyles, Kockelman, TRB ‘16]

19

Existence of Congestion-Free FlowsCapacity-symmetric (C-S) networks:• For all network cuts pS, S̄qC

out

= Cin

Capacity-symmetric networkFeasible customer flows Feasible rebalancing flows

Feasibility of rebalancing [Zhang, Rossi, Pavone, RSS ‘16]

Implications:1. Theory: In C-S networks rebalancing does not increase congestion2. Practice: If goal is to maximize customer satisfaction, customer flows and

rebalancing flows are decoupled and can be computed separately20

On Cut Symmetry

Urban center Avg. frac. capacity disparity Std. dev.

Chicago, IL 1.2972∙10-4 1.003∙10-4

New York, NY 1.6556∙10-4 1.304∙10-4

Colorado Springs, CO 3.1772∙10-4 2.308∙10-4

Los Angeles, CA 0.9233∙10-4 0.676∙10-4

Mobile, AL 1.9368∙10-4 1.452∙10-4

Portland, OR 1.0769∙10-4 0.778∙10-4

Conclusion:• In practice, very high degree of cut-

symmetry (even with many one-way streets)

[Zhang, Rossi, Pavone, RSS ’16]

Additional Research Directions

• More sophisticated algorithms: stochasticity, robustness, & decentralization

• Heterogeneity: multi-modal, smart grid, & society

• Technology infusion and advising policy makers

Charging demandEnergy storage

Electricity pricesEnergy provision

Power network

Transportation network

Conclusions

1. Autonomous driving might lead to a transformational paradigm for personal urban mobility (to improve or sustain current mobility needs)

2. Integration of system-wide coordination and autonomous driving gives rise to an entirely new class of problems at the interface of robotics and transportation research

3. Solutions to these problems are key to enable autonomous MoD and to carefully evaluate their value proposition

Acknowledgements: NSF CAREER Award, Car2Go, Toyota, collaborators at Stanford (Ramon Iglesias, Federico Rossi, and Rick Zhang), MIT, and Singapore

Contact: pavone@stanford.edu

23