Data Science in Practice: Developing a Real-time …...Data Science in Practice: Developing a Real-time Prediction Model of Driver Behavior at Intersections Using Kinematic Time Series

Data Science in Practice:Developing a Real-time Prediction Model of

Driver Behavior at Intersections UsingKinematic Time Series Data

Michael Elliott1

1Department of BiostatisticsUniversity of Michigan

1 / 22

Mixing Autonomous and Human-Driven Vehicles

There will be a long period, likelyseveral decades, where partially or fullyautonomous vehicles share the roadwith human drivers.

Predicting the behavior of human-controlled vehicles isthus a major challenge to making such systems practical.Here I describe a project to predict whether or not a driverwill stop before turning left across oncoming traffic lanes(joint work with Vincent Tan of Biostatistics and CarolFlannagan of the University of Michigan TransportationResearch Institute).

Use kinematic behavior of human-driven vehicle starting100 m before the intersection to predict stopping behaviorover this interval.

2 / 22



Predicting the behavior of human-controlled vehicles isthus a major challenge to making such systems practical.Here I describe a project to predict whether or not a driverwill stop before turning left across oncoming traffic lanes(joint work with Vincent Tan of Biostatistics and CarolFlannagan of the University of Michigan TransportationResearch Institute).

Use kinematic behavior of human-driven vehicle starting100 m before the intersection to predict stopping behaviorover this interval.

3 / 22



Predicting the behavior of human-controlled vehicles isthus a major challenge to making such systems practical.Here I describe a project to predict whether or not a driverwill stop before turning left across oncoming traffic lanes(joint work with Vincent Tan of Biostatistics and CarolFlannagan of the University of Michigan TransportationResearch Institute).Use kinematic behavior of human-driven vehicle starting100 m before the intersection to predict stopping behaviorover this interval.

4 / 22


Naturalistic driving data obtained from 108 licensed driversin Michigan (12 days of unsupervised driving).n =1,823 left turns.Data consists of speed (km/sec) in 10 msec intervalsstarting 100 m before the intersection, continuing to thecenter line of the intersection.

5 / 22


Convert to speed at 1m intervals and use principalcomponents analysis (PCA) to predict the probability of afuture stop at each 1m interval.

6 / 22


Outcome is a binary indicator, at each 1m interval, ofwhether the car will stop in the future.Use Bayesian additive regression trees (BART) to predictbased on 3 PCs.

Bayesian ensemblemethod developed byChipman et. al. (2010) thatmodels the mean outcomegiven predictors by a sumof regression trees andincorporates additiveeffects of predictors.

7 / 22


Best prediction usesa 6m movingwindow of speeds toestimate PCs.BART produces anAUC of 0.75 -95mout; reaches 0.9 by-25m out.Results imply thatROC shifts to the leftas vehicleapproaches thecenter ofintersection.

−80 −60 −40 −20 0

0.5

0.6

0.7

0.8

0.9

1.0

Distance from reference (m)

AU

C

Estimate95% C.I.

8 / 22


Solid line is captureratio (CR) ≡autonomous carguesses correctly thatthe human drivenvehicle stops. Wantthis high.

Dotted line is falsepostive ratio (FPR) ≡autonomous carguesses incorrectlythat the human drivenvehicle stops⇒CRASH! Want this low.

−80 −60 −40 −20 0

0.0

0.4

0.8

10% cut−off

Distance (m)

CRFPR

−80 −60 −40 −20 0

0.0

0.4

0.8

20% cut−off

Distance (m)

CRFPR

−80 −60 −40 −20 0

0.0

0.4

0.8

30% cut−off

Distance (m)

CRFPR

−80 −60 −40 −20 0

0.0

0.4

0.8

40% cut−off

Distance (m)

CRFPR

−80 −60 −40 −20 0

0.0

0.4

0.8

50% cut−off

Distance (m)

CRFPR

−80 −60 −40 −20 0

0.0

0.4

0.8

60% cut−off

Distance (m)

CRFPR

−80 −60 −40 −20 0

0.0

0.4

0.8

70% cut−off

Distance (m)

CRFPR

−80 −60 −40 −20 0

0.0

0.4

0.8

80% cut−off

Distance (m)

CRFPR

−80 −60 −40 −20 0

0.0

0.4

0.8

90% cut−off

Distance (m)

CRFPR

9 / 22


Open issuesAdd predictors such as presence of lead vehicle anddistance when driver first used signal were not used.

Joint modeling of PCA and BART.Similar drivers and intersection road types suggest thatthere could be intra-driver or intra-road type correlation.CR and FDR have uncertainty associated with them.Consider using quantiles to determine cutoffs rather than(posterior) mean.Attach different costs to CR and FPR and use numericalmethods to find the optimal cut-off at each distance.Consider other outcomes (an ensemble of outcomes?)Work with human factors and engineering experts toimprove understanding of input/output values of interest,model formation, etc.

10 / 22


Open issuesAdd predictors such as presence of lead vehicle anddistance when driver first used signal were not used.Joint modeling of PCA and BART.

Similar drivers and intersection road types suggest thatthere could be intra-driver or intra-road type correlation.CR and FDR have uncertainty associated with them.Consider using quantiles to determine cutoffs rather than(posterior) mean.Attach different costs to CR and FPR and use numericalmethods to find the optimal cut-off at each distance.Consider other outcomes (an ensemble of outcomes?)Work with human factors and engineering experts toimprove understanding of input/output values of interest,model formation, etc.

11 / 22


Open issuesAdd predictors such as presence of lead vehicle anddistance when driver first used signal were not used.Joint modeling of PCA and BART.Similar drivers and intersection road types suggest thatthere could be intra-driver or intra-road type correlation.

CR and FDR have uncertainty associated with them.Consider using quantiles to determine cutoffs rather than(posterior) mean.Attach different costs to CR and FPR and use numericalmethods to find the optimal cut-off at each distance.Consider other outcomes (an ensemble of outcomes?)Work with human factors and engineering experts toimprove understanding of input/output values of interest,model formation, etc.

12 / 22


Open issuesAdd predictors such as presence of lead vehicle anddistance when driver first used signal were not used.Joint modeling of PCA and BART.Similar drivers and intersection road types suggest thatthere could be intra-driver or intra-road type correlation.CR and FDR have uncertainty associated with them.Consider using quantiles to determine cutoffs rather than(posterior) mean.

Attach different costs to CR and FPR and use numericalmethods to find the optimal cut-off at each distance.Consider other outcomes (an ensemble of outcomes?)Work with human factors and engineering experts toimprove understanding of input/output values of interest,model formation, etc.

13 / 22


Open issuesAdd predictors such as presence of lead vehicle anddistance when driver first used signal were not used.Joint modeling of PCA and BART.Similar drivers and intersection road types suggest thatthere could be intra-driver or intra-road type correlation.CR and FDR have uncertainty associated with them.Consider using quantiles to determine cutoffs rather than(posterior) mean.Attach different costs to CR and FPR and use numericalmethods to find the optimal cut-off at each distance.

Consider other outcomes (an ensemble of outcomes?)Work with human factors and engineering experts toimprove understanding of input/output values of interest,model formation, etc.

14 / 22


Open issuesAdd predictors such as presence of lead vehicle anddistance when driver first used signal were not used.Joint modeling of PCA and BART.Similar drivers and intersection road types suggest thatthere could be intra-driver or intra-road type correlation.CR and FDR have uncertainty associated with them.Consider using quantiles to determine cutoffs rather than(posterior) mean.Attach different costs to CR and FPR and use numericalmethods to find the optimal cut-off at each distance.Consider other outcomes (an ensemble of outcomes?)

Work with human factors and engineering experts toimprove understanding of input/output values of interest,model formation, etc.

15 / 22


Open issuesAdd predictors such as presence of lead vehicle anddistance when driver first used signal were not used.Joint modeling of PCA and BART.Similar drivers and intersection road types suggest thatthere could be intra-driver or intra-road type correlation.CR and FDR have uncertainty associated with them.Consider using quantiles to determine cutoffs rather than(posterior) mean.Attach different costs to CR and FPR and use numericalmethods to find the optimal cut-off at each distance.Consider other outcomes (an ensemble of outcomes?)Work with human factors and engineering experts toimprove understanding of input/output values of interest,model formation, etc.

16 / 22


What makes this a “big data” problem, besides the obvious(thousands of observations across hundred(s) of drivers)?

What is the best way to obtain information from speedtrajectories in a fashion that would make this applicable tothe real-world prediction problem.Visualization to understand and interpret results.Need to be able to “scale” in a practical fashion.Opportunity to keep learning and improving prediction, atboth population and individual vehicle level.

Privacy and social/political issues.

Opportunity for collaboration

17 / 22



What is the best way to obtain information from speedtrajectories in a fashion that would make this applicable tothe real-world prediction problem.

Visualization to understand and interpret results.Need to be able to “scale” in a practical fashion.Opportunity to keep learning and improving prediction, atboth population and individual vehicle level.



18 / 22



What is the best way to obtain information from speedtrajectories in a fashion that would make this applicable tothe real-world prediction problem.Visualization to understand and interpret results.

Need to be able to “scale” in a practical fashion.Opportunity to keep learning and improving prediction, atboth population and individual vehicle level.



19 / 22



What is the best way to obtain information from speedtrajectories in a fashion that would make this applicable tothe real-world prediction problem.Visualization to understand and interpret results.Need to be able to “scale” in a practical fashion.

Opportunity to keep learning and improving prediction, atboth population and individual vehicle level.



20 / 22






21 / 22






22 / 22

fd@rm@0:

Documents

Data Science in Practice: Developing a Real-time …...Data Science in Practice: Developing a Real-time Prediction Model of Driver Behavior at Intersections Using Kinematic Time Series