Safety Pilot Model Deployment - UMTRIumtri.umich.edu/content/2014.GlobalSymposium.Gay.pdf · Safety Pilot Model Deployment Lessons Learned 2014 Global Symposium on Connected Vehicles

Safety Pilot Model Deployment Lessons Learned

2014 Global Symposium on Connected Vehicles and Infrastructure

April 22, 2014

Kevin Gay

USDOT / Volpe

2 U.S. Department of Transportation

Objective of Presentation

Discuss Key Lessons Learned from Safety Pilot Focus on Four Interrelated Areas □ Roadside Infrastructure Deployed □ In-Vehicle Devices Deployed □ Experimental Design □ Data Collected


Roadside Infrastructure Deployment • RSEs Deployed • SpaT Updates • IPv6 Installation Challenges / Radio

Coverage Ranges


Planned Deployment of RSEs

RSE Installed / Actuated Traffic Signal RSE Installed / SPaT-enabled Traffic Signal Originally proposed RSE freeway sites


Actual Deployment of RSEs

RSE site removed due to lack of connectivity RSE connected to AA City backhaul


Fuller Road – Projected SPaT Range (300m)


Fuller Road – Actual SPaT Range (>300m)


What Did We Learn?

1. IPv6 connectivity was one of the biggest challenges to the RSE deployment and required the shifting of freeway sites

2. DSRC radio coverage ranges for the RSEs were much larger than expected

3. RSEs collected a much larger volume of data than was estimated due to the overlaps of coverage

Roadside Equipment specification needed an update (version 4.0)


In-Vehicle Device Deployment • Types of Devices Deployed • Device Testing • Number and Types of Updates Needed on

Devices


Vehicles & Devices Deployed

2,362 VADs □ Light Vehicles □ Trucks □ Buses

289 ASDs (Light Vehicles) 64 Integrated Light Vehicles 19 Integrated / Retrofit Heavy

Vehicles 3 Retrofit Transit Vehicles


Testing Conducted

Multiple Stages of Testing □ USDOT Testing (QPL) □ Interoperability (4 rounds) □ Pre-Model Deployment

8 Vehicle manufacturers (CAMP) Multiple hardware vendors

□ Savari □ Cohda

Multiple vehicle platforms/products

□ Light, heavy, and transit vehicles

□ Denso □ Arada

Presenter

Presentation Notes

Another major accomplishment in the SPMD is the interoperability testing. There were three separate stages of interoperability testing in the Safety Pilot, conducted in June, August, and December. Each stage focused on different functional capabilities of the devices (i.e. BSM transmission, traveler information messages/SPaT, and security functions). Each phase of the testing included bench testing in a laboratory environment as well as field testing where the devices were temporarily installed in vehicles and driven around the MD environment. The objectives of this testing was to verify that these devices produced by different manufacturers could work cooperatively together. Furthermore, this testing was also able to provide additional information that can be feed back into the USDOT requirements and industry standards for next generations of these devices.


Device Monitoring & Updates

Approaches to Device Health Monitoring □ Data Acquisition Systems – 183 devices/vehicles □ Mining BSM data collected by RSEs – 2,554

devices/vehicles

Every device deployed needed at least 1 software update

Types of Device Updates □ GPS Parameter □ Path History □ Security Credential Parameters □ Device Stability

Replacement of Damaged Units

Presenter

Presentation Notes

Image #: 181736890


What Did We Learn?

1. Deploying 2,700+ devices required different installation configurations for each vehicle class and type

2. Efficient end-of-line testing (configuration/quality) and baseline performance characterization testing would have been beneficial

3. Monitoring the health of the devices in the field was challenging as not all devices had health monitoring capabilities

4. Recalling and updating deployed devices was a labor-intensive and time-intensive process

Deploying, testing, and managing 2,700+ deployed devices required an extensive amount of planning.


Experimental Design • Power Analysis • Traffic Simulation Modeling • Interaction Monitoring / Results

Presenter

Presentation Notes

Item Number: 470186355


Scoping the Model Deployment – Power Analysis Prior to selection of Ann Arbor as Model Deployment Geographic Area What should the scope of the Model Deployment be to gather enough

data? □ Participants □ Duration □ Vehicles

Conducted analysis using prior field test results

Variable Recommended Actual

Test Subjects 108 128

Duration 5 months 6 months

Integrated Vehicles 55 64

Equipped Vehicles 2,500 – 3,000 2,772


Simulation of Estimates Utilized a traffic simulation model

to estimate expected number of interactions

Interactions are communications between integrated light vehicles and other equipped vehicles within close proximity

Real-Time Monitoring All vehicles with data acquisition

systems were remotely monitored via cell link

Total Interactions Observed interactions were within

15% of expected interactions

Estimating the V2V Interactions

50,088

66,933

990 3,796

9,410

16,030

23,700

28,704

37,486

44,942

54,281

56,541

-

10,000

20,000

30,000

40,000

50,000

60,000

70,000

1 2 3 4 5 6 7 8 9 10 11 12

Interaction Targets

Observed Interactions


Actual Frequency of V2V Interactions

UMTRI selected drivers with large number of trips in the Model Deployment Geographic Area

Majority of drivers had interactions with other V2V equipped vehicles 4 out of 5 trips

0

2

4

6

8

10

12

14

16

18

35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%

No.

of S

ubje

cts

% of Trips with V2V Interactions


What Did We Learn?

1. Previous field test data provided a good data point for developing an estimate for procurement (vehicles, drivers, time)

2. More detailed analysis (traffic simulation) after site selection allowed the development of performances measures (V2V interactions) and helped refine the experimental design

3. Selecting drivers based on trips in the Model Deployment Area instead of overall mileage or other metrics resulted in higher numbers of V2V interactions

A relatively small number of total equipped vehicles can generate a large

number of V2V interactions with a multi-faceted experimental design.


Data Collection and Evaluation • Different Types & Sources of Data Collected • Volumes of Data Collected

Presenter

Presentation Notes

Item Number: 481135637


Virginia Tech/CAMP

Model Deployment Database

Southwest Research

Battelle

University of

Michigan

Safety Pilot Data Sources


Types of Safety Pilot Data

Numerical Data □ In-vehicle □ Wireless/GPS □ External Sensors

Volume of Data □ 2.3B database records

Video Data □ In-vehicle cameras

Volume of Data □ 63,000 hours of video

Presenter

Presentation Notes

In vehicle data: All information off the vehicle’s computer like speed/acceleration, driver input to the vehicle V2V data: All data relevant to how the connected vehicle system is working, GPS, communication logs, and application alerts External sensors: info about vehicle surroundings, serve as a ground truth


What Did We Learn?

1. Manually removing and replacing the hard drives was a labor intensive process for harvesting data

2. Volume of data generated was larger than any previous naturalistic driving field tests conducted by the USDOT

3. Sharing the data with entities outside of the project team (i.e. Research Data Exchange) required detailed analysis and cleansing of the data

It is vital to have well-defined and tested processes for data harvesting, storage and sharing


Questions

Kevin Gay US DOT / Volpe [email protected]

mailto:[email protected]

Documents

Safety Pilot Model Deployment - UMTRIumtri.umich.edu/content/2014.GlobalSymposium.Gay.pdf · Safety Pilot Model Deployment Lessons Learned 2014 Global Symposium on Connected Vehicles