Detecting Flight Trajectory Anomalies and Predicting Diversions in Freight Transportation

Detecting Flight Trajectory Anomalies and Predicting Diversions in Freight TransportationClaudio Di Ciccio, Han van der Aa, Cristina Cabanillas, Jan Mendling, andJohannes PrescherEMISA 2016, Vienna, Austria

claudio.di.ciccio@wu.ac.at

Business processes intransport domain

SEITE 2

Continuous taskmonitoring

SEITE 3

Continuous taskmonitoring

SEITE 4

Continuous task monitoringin multimodal transport

SEITE 5

Continuous task monitoringin multimodal transport

SEITE 6

Diversion

Diversion airport

Dealing with flight diversionsA real-life scenario

Start

End

SEITE 7

©

Dealing with flight diversionsA real-life scenario

SEITE 8

©

Dealing with flight diversionsA real-life scenario

SEITE 9

©

Dealing with flight diversionsA real-life scenario

SEITE 10

©

Dealing with flight diversionsA real-life scenario

SEITE 11

©

Dealing with flight diversionsA real-life scenario

SEITE 12

©

Dealing with flight diversionsA real-life scenario

SEITE 13

©

Dealing with flight diversionsA real-life scenario

SEITE 14

©

Dealing with flight diversionsA real-life scenario

SEITE 15

©

Dealing with flight diversionsA real-life scenario

SEITE 16

©

Dealing with flight diversionsA real-life scenario

SEITE 17

©

Dealing with flight diversionsA real-life scenario

SEITE 18

©

Dealing with flight diversionsA real-life scenario

SEITE 19

©

Dealing with flight diversionsA real-life scenario

SEITE 20

©

Dealing with flight diversionsA real-life scenario

SEITE 21

©

Dealing with flight diversionsA real-life scenario

Modern technologycomes

into play

SEITE 22

©

Dealing with flight diversionsA real-life scenario

SEITE 23

©

Objective:monitor the

continuous taskand, in case of anomalies,

raise an alertat this time:

not at this time:

Motivation

SEITE 24

©

Flight diversion

Flight diversion is an example ofcontinuous task execution anomaly

Flight diversion

Flight diversion is an example ofcontinuous task execution anomaly

Which is going to be diverted?

Source: http://www.flightradar24.com/

SEITE 27

Motivation

Objective:monitor the

continuous taskand, in case of anomalies,

raise an alertat this time:

not at this time:

… with an automated integrated system

SEITE 28

Solution sketch

Gather and buffer flight data information Slice data into time-based intervals Extract flight features (deltas) representing the

flight in the interval

SEITE 29

Interval-basedprogress features

Features are extracted out of data Clustered into fixed-length time intervals

SEITE 30

Gather flight data events along a time interval

Interpolate attribute values

Redo

Solution sketch

Gather and buffer flight data information Slice data into time-based intervals Extract flight features (deltas) representing the

flight in the interval Let an automated classifier establish whether

the features are anomalous In our implementation:

Support Vector Machines (SVMs) After a given number of consecutive

anomalous intervals, raise an alert

SEITE 31

Interval-basedchecking

latitude

longitude

velocity (speed)

height (altitude)

timestamp

<lat,lon,v,h,t>

SEITE 32

Interval-basedchecking

<lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t> ⟨∆𝑑gain ,∆𝑑

cmpl ,∆𝑑ph ,∆𝑣 ,∆h ⟩

[features] [SVM]

SEITE 33

Interval-basedchecking

<lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t> ⟨∆𝑑gain ,∆𝑑

cmpl ,∆𝑑ph ,∆𝑣 ,∆h ⟩

⟨∆𝑑gain ,∆𝑑cmpl ,∆𝑑

ph ,∆𝑣 ,∆h ⟩ <lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t>

SEITE 34

Interval-basedchecking

<lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t> ⟨∆𝑑gain ,∆𝑑

cmpl ,∆𝑑ph ,∆𝑣 ,∆h ⟩

⟨∆𝑑gain ,∆𝑑cmpl ,∆𝑑

ph ,∆𝑣 ,∆h ⟩ ⟨∆𝑑gain ,∆𝑑

cmpl ,∆𝑑ph ,∆𝑣 ,∆h ⟩

<lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t>

SEITE 35

Interval-basedchecking

<lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t> ⟨∆𝑑gain ,∆𝑑

cmpl ,∆𝑑ph ,∆𝑣 ,∆h ⟩

⟨∆𝑑gain ,∆𝑑cmpl ,∆𝑑

ph ,∆𝑣 ,∆h ⟩ ⟨∆𝑑gain ,∆𝑑

cmpl ,∆𝑑ph ,∆𝑣 ,∆h ⟩

⟨∆𝑑gain ,∆𝑑cmpl ,∆𝑑

ph ,∆𝑣 ,∆h ⟩

<lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t>

SEITE 36

Interval-basedchecking

<lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t> ⟨∆𝑑gain ,∆𝑑

cmpl ,∆𝑑ph ,∆𝑣 ,∆h ⟩

⟨∆𝑑gain ,∆𝑑cmpl ,∆𝑑

ph ,∆𝑣 ,∆h ⟩ ⟨∆𝑑gain ,∆𝑑

cmpl ,∆𝑑ph ,∆𝑣 ,∆h ⟩

⟨∆𝑑gain ,∆𝑑cmpl ,∆𝑑

ph ,∆𝑣 ,∆h ⟩ ⟨∆𝑑gain ,∆𝑑

cmpl ,∆𝑑ph ,∆𝑣 ,∆h ⟩

<lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t>

SEITE 37

Interval-basedchecking

<lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t> ⟨∆𝑑gain ,∆𝑑

cmpl ,∆𝑑ph ,∆𝑣 ,∆h ⟩

⟨∆𝑑gain ,∆𝑑cmpl ,∆𝑑

ph ,∆𝑣 ,∆h ⟩ ⟨∆𝑑gain ,∆𝑑

cmpl ,∆𝑑ph ,∆𝑣 ,∆h ⟩

⟨∆𝑑gain ,∆𝑑cmpl ,∆𝑑

ph ,∆𝑣 ,∆h ⟩ ⟨∆𝑑gain ,∆𝑑

cmpl ,∆𝑑ph ,∆𝑣 ,∆h ⟩

⟨∆𝑑gain ,∆𝑑cmpl ,∆𝑑

ph ,∆𝑣 ,∆h ⟩

<lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t>

SEITE 38

Interval-basedchecking

<lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t> ⟨∆𝑑gain ,∆𝑑

cmpl ,∆𝑑ph ,∆𝑣 ,∆h ⟩

⟨∆𝑑gain ,∆𝑑cmpl ,∆𝑑

ph ,∆𝑣 ,∆h ⟩

⟨∆𝑑gain ,∆𝑑cmpl ,∆𝑑

ph ,∆𝑣 ,∆h ⟩ ⟨∆𝑑gain ,∆𝑑

cmpl ,∆𝑑ph ,∆𝑣 ,∆h ⟩

⟨∆𝑑gain ,∆𝑑cmpl ,∆𝑑

ph ,∆𝑣 ,∆h ⟩ ⟨∆𝑑gain ,∆𝑑

cmpl ,∆𝑑ph ,∆𝑣 ,∆h ⟩

⟨∆𝑑gain ,∆𝑑cmpl ,∆𝑑

ph ,∆𝑣 ,∆h ⟩

<lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t>

SEITE 39

Interval-basedchecking

<lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t> ⟨∆𝑑gain ,∆𝑑

cmpl ,∆𝑑ph ,∆𝑣 ,∆h ⟩

⟨∆𝑑gain ,∆𝑑cmpl ,∆𝑑

ph ,∆𝑣 ,∆h ⟩ ⟨∆𝑑gain ,∆𝑑

cmpl ,∆𝑑ph ,∆𝑣 ,∆h ⟩

⟨∆𝑑gain ,∆𝑑cmpl ,∆𝑑

ph ,∆𝑣 ,∆h ⟩ ⟨∆𝑑gain ,∆𝑑

cmpl ,∆𝑑ph ,∆𝑣 ,∆h ⟩

⟨∆𝑑gain ,∆𝑑cmpl ,∆𝑑

ph ,∆𝑣 ,∆h ⟩ ⟨∆𝑑gain ,∆𝑑

cmpl ,∆𝑑ph ,∆𝑣 ,∆h ⟩

⟨∆𝑑gain ,∆𝑑cmpl ,∆𝑑

ph ,∆𝑣 ,∆h ⟩

<lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t>

AlertSEITE 40

Interval-basedchecking

<lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t> ⟨∆𝑑gain ,∆𝑑

cmpl ,∆𝑑ph ,∆𝑣 ,∆h ⟩

⟨∆𝑑gain ,∆𝑑cmpl ,∆𝑑

ph ,∆𝑣 ,∆h ⟩ ⟨∆𝑑gain ,∆𝑑

cmpl ,∆𝑑ph ,∆𝑣 ,∆h ⟩

⟨∆𝑑gain ,∆𝑑cmpl ,∆𝑑

ph ,∆𝑣 ,∆h ⟩ ⟨∆𝑑gain ,∆𝑑

cmpl ,∆𝑑ph ,∆𝑣 ,∆h ⟩

⟨∆𝑑gain ,∆𝑑cmpl ,∆𝑑

ph ,∆𝑣 ,∆h ⟩ ⟨∆𝑑gain ,∆𝑑

cmpl ,∆𝑑ph ,∆𝑣 ,∆h ⟩

⟨∆𝑑gain ,∆𝑑cmpl ,∆𝑑

ph ,∆𝑣 ,∆h ⟩

<lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t>

AlertSEITE 41

Interval-basedchecking

<lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t> ⟨∆𝑑gain ,∆𝑑

cmpl ,∆𝑑ph ,∆𝑣 ,∆h ⟩

⟨∆𝑑gain ,∆𝑑cmpl ,∆𝑑

ph ,∆𝑣 ,∆h ⟩ ⟨∆𝑑gain ,∆𝑑

cmpl ,∆𝑑ph ,∆𝑣 ,∆h ⟩

⟨∆𝑑gain ,∆𝑑cmpl ,∆𝑑

ph ,∆𝑣 ,∆h ⟩ ⟨∆𝑑gain ,∆𝑑

cmpl ,∆𝑑ph ,∆𝑣 ,∆h ⟩

⟨∆𝑑gain ,∆𝑑cmpl ,∆𝑑

ph ,∆𝑣 ,∆h ⟩ ⟨∆𝑑gain ,∆𝑑

cmpl ,∆𝑑ph ,∆𝑣 ,∆h ⟩

⟨∆𝑑gain ,∆𝑑cmpl ,∆𝑑

ph ,∆𝑣 ,∆h ⟩

<lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t><lat,lon,v,h,t>

AlertSEITE 42

Evaluation:Flight data

Flight data gathered from FlightStats.com and FlightRadar24.com July-August 2013 (Semi-)publicly available

K-fold cross validation

Area Diverted Regular OverallEU 46 746 792US 22 316 338

Total 68 1,062 1,130

* Thanks to Han van der Aa for his contributionSEITE 43

Evaluation:Train & validation (tuning)

F-score, Precision, Recall F-Score v. time-to-predict

* Thanks to Han van der Aa for his contributionSEITE 44

Evaluation:Test results

SEITE 45

Further reading

Claudio Di Ciccio, Han van der Aa, Cristina Cabanillas, Jan Mendling, and Johannes Prescher (2016)Detecting flight trajectory anomalies and predicting diversions in freight transportation Decision Support Systems, 88, 1 - 17http://dx.doi.org/10.1016/j.dss.2016.05.004

Cristina Cabanillas, Claudio Di Ciccio, Jan Mendling, andAnne Baumgrass (2014)Predictive Task Monitoring for Business ProcessesBPM 2014, Springerhttp://dx.doi.org/10.1007/978-3-319-10172-9_31

Anne Baumgrass, Cristina Cabanillas, and Claudio Di Ciccio (2015)A Conceptual Architecture for an Event-based Information Aggregation Engine in Smart LogiticsEMISA 2015 (GI)http://subs.emis.de/LNI/Proceedings/Proceedings248/109.pdf

SEITE 46

Detecting Flight Trajectory Anomalies and Predicting Diversions in Freight TransportationClaudio Di Ciccio, Han van der Aa, Cristina Cabanillas, Jan Mendling, andJohannes PrescherEMISA 2016, Vienna, Austria

claudio.di.ciccio@wu.ac.at

Detecting Flight Trajectory Anomalies and Predicting Diversions in Freight Transportation

Extra slides

System Architecture:Which component does what

!

Further reading

Claudio Di Ciccio, Han van der Aa, Cristina Cabanillas, Jan Mendling, and Johannes Prescher (2016)Detecting flight trajectory anomalies and predicting diversions in freight transportation Decision Support Systems, 88, 1 - 17http://dx.doi.org/10.1016/j.dss.2016.05.004

Cristina Cabanillas, Claudio Di Ciccio, Jan Mendling, andAnne Baumgrass (2014)Predictive Task Monitoring for Business ProcessesBPM 2014, Springerhttp://dx.doi.org/10.1007/978-3-319-10172-9_31

Anne Baumgrass, Cristina Cabanillas, and Claudio Di Ciccio (2015)A Conceptual Architecture for an Event-based Information Aggregation Engine in Smart LogiticsEMISA 2015 (GI)http://subs.emis.de/LNI/Proceedings/Proceedings248/109.pdf

SEITE 50