Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad

1  

Last Updated: Jan. 2014

VP  Pla&orm  Evangelism  @cobiacomm  on  Twi6er  

h6p://blog.cobia.net/cobiacomm/  

Chris  Haddad  

Tracking  a  Soccer  Game  with  Big  Data  

Big  Data  Improves  Team  Performance  

๏  Sports  Teams  

๏  Your  Business  Team  3  

Improvement  Requires  A  Feedback  Loop  

4  

Internet  of  Things  (IoT)    Bridges  Physical  and  Virtual  Worlds  

๏  Sensors,  actuators  everywhere  

๏  Internet  connecOvity  

Improvement  =  IoT,  Apps,  and  APIs  

๏  VisualizaOon  Apps  

๏  Control  APIs  

DEBS  Challenge  

•  Bridge  game  play  to  Internet  of  Things  

•  Sensors  in  shoes,  ball,  goalie  hands  

•  Analyze  Soccer  Game  

•  SpaOal  and  Temporal  Processing  

•  Visualize  Game  Play  

•  AcOvity  Heat  maps  

•  Recommend  Performance  Improvements  

http://srinathsview.blogspot.com/2013/05/solving-debs-2013-grand-challenge-with.html

Bridging  Game  Play  to  IoT  

7  

Temporal  Tracking  

SpaOal  Tracking  

•  Each  event  includes  the  locaOon  (x,y,z),  Ome  stamp,  velocity  and  acceleraOon  

•  Throughput  15,000  events  per  second  

Analyze  Game  

8  

๏  BALL  POSSESSION  STATE  DIAGRAM  

๏  SHOT  ON  GOAL  STATE  DIAGRAM  

Analyze  Game  

9  

๏  RUNNING  LOCATION  STATES  

10  

SoluOon  Architecture  ๏  Capture  millions  of  events  with  

Common  Events  Collector    

๏  Derive  real-­‐Ome  decisions  with  temporal  processing      

๏  Further  opOmize  recommendaOons  with  spaOal  Map-­‐Reduce  processing  analyOcs  

๏  View  feedback  dashboards  

Batch  AnalyOc  FoundaOon  

11  

Business  AcOvity  Monitor  

Realizing  Real-­‐Ome  AnalyOcs  ๏  Processing  Data  on  the  fly,  while  

storing  a  minimal  amount  of  informaOon  and  responding  fast  (from  <1  ms  to  few  seconds)  

๏  Idea  of  Event  streams    

๏  A  series  of  events  in  Ome    

๏  Enabling  technologies    

๏  Stream  Processing  (Storm)  

๏  Complex  Event  processing    (Siddhi)  

Complex  Event  Processing  

Complex  Event  Processing  

Stream  Processing  

• SQL  like  language  • Supports  powerful  temporal  operators  (e.g.  windows,  event  paberns)    • Focus  on  speed  • Harder  to  scale    • e.g.  WSO2  CEP,  Streambase,  Esper      

• Operators  connected  in  a  network,  but  you  have  to  write  the  logic  • Distributed  by  design  • Focus  on  reliability  (do  not  loose  messages),  has  transacOons  • e.g.  Storm,  S4    

Complex  Event  Processing  Operators  ๏  Filters  or  transformaOons  (process  a  single  event)  

๏  from Ball[v>10] select .. insert into ..

๏  Windows  +  aggregaOon  (track  window  of  events:  Ome,  length)  

๏  from Ball#window.time(30s) select avg(v) ..

๏  Joins  (join  two  event  streams  to  one)  

๏  from Ball#window.time(30s) as b join Players as p on p.v < b.v

๏  Paberns  (state  machine  implementaOon)  

๏  from Ball[v>10], Ball[v<10]*,Ball[v>10] select ..

๏  Event  tables  (map  a  database  as  an  event  stream)  

๏  Define table HitV (v double) using .. db info ..

Usecase  1:  Running  Analysis  ๏  Detect  when  speed  crosses  threshold  limits    

define partition player by Players .id; from s = Players [v <= 1 or v > 11] , t = Players [v > 1 and v <= 11]+ , e = Players [v <= 1 or v > 11] select s.ts as tsStart , e.ts as tsStop ,s.id as playerId , ‘‘trot" as intensity , t [0].v as instantSpeed , (e.ts - s.ts )/1000000000 as unitPeriod insert into RunningStats partition by player;

Usecase  2:  Ball  possession  ๏  Ball  possession  

๏  Defined  as  possessing  the  ball  from  Ome  you  hit  it  unOl  someone  else  hit  it  or  ball  leaves  the  ground)    

Usecase  3:  Heatmap  of  AcOvity    

๏  Show  where  acOons  happened  (via  cells  defined  by  a  grid  of  64X100  etc.),  need  updates  once  every  second  

๏  Can  solved  via  cell  change  boundaries,  but    does  not  work  if  one  player  stays  more  than  1  sec  in  the  same  cell.  So  need  to  join  with  a  Omer.    

Usecase  4:  Detect  kicks  on  goal    

๏  Main  Idea:  Detect  kicks  on  the  ball,  calculate  direcOon  aker  1m,  and  keep  giving  updates  as  long  as  it  is  in  right  direcOon  

Results  for  DEBS  Scenarios  

Big  Data  and  IoT  Changes  the  Game  ๏  Real-­‐Ome  Status  Dashboard    

๏  Player  game  play  

๏  Momentum  indicators  

๏  PredicOons  about  the  next  move    

๏  Improvement  Dashboard  

๏  Study  of  game  and  players  effecOveness    

๏  Monitor  player  health  and  body  funcOons    

Success  Target  

Photo by John Trainoron Flickr http://www.flickr.com/photos/trainor/2902023575/, Licensed under CC

SituaOonal  Context:  Traffic  Example  

{incidents: [ {impacting: false,

eventCode: 0, iconURL: "http://...",

lat: 38.743999, shortDesc: "Exit ramp closed on I 95...",

type: 1, severity: 0,

fullDesc: "In NEWINGTON exit ramp closed on I-95 ...",

startTime: "2010-02-21T00:14:07", lng: -77.188004, id: "368598263",

endTime: "2010-02-27T05:04:19" },

Image source: http://www.directoryofnewyorkcity.com/blog/2009/05/how-to-find-parking-in-new-york-city/ Real-time traffic map: http://www.mapquestapi.com/traffic/

Accelerate  interac6ons  inside  and  outside  the  organiza6on  

Reduce  interac6on  

fric6on  and  cost    

Increase  engagement  and  

enhance  produc6vity        

Sense    business  ac6vity  

and    automa6cally  

adapt    

Become  a  More  Connected  Business  

http://wso2.com/landing/enabling-the-connected-business

Connected  Business    Reference  Architecture  

Conceptual  Architecture  

CEP  High  Availability  

Siddhi  Storm  Bolt  (next  Major  Release)  

๏  We  have  wriben  a  Siddhi  bolt  that  would  let  users  run  distributed  Siddhi  Queries  using  Storm  

SiddhiBolt siddhiBolt1 = new SiddhiBolt( .. siddhi queries .. ); SiddhiBolt siddhiBolt2 = new SiddhiBolt( .. siddhi queries .. ); TopologyBuilder builder = new TopologyBuilder(); builder.setSpout("source", new PlayStream(), 1); builder.setBolt("node1", siddhiBolt1, 1) .shuffleGrouping("source", "PlayStream1"); .. builder.setBolt("LeafEacho", new EchoBolt(), 1) .shuffleGrouping("node1", "LongAdvanceStream"); .. cluster.submitTopology("word-count", conf, builder.createTopology());

31  

Contact  us  !  

33  

About  the  Presenter  

๏  Chris  Haddad  

๏  VP  Plaoorm  Evangelism  

๏  Learn  more  about  me  

๏  www.linkedin.com/in/cobiacomm/  

๏  Follow  me  

๏  @cobiacomm  on  Twiber  

๏  hbp://blog.cobia.net/cobiacomm  

๏  On  Google+  too