1. 1. GMMCP-Tracker: Globally Optimal Generalized Maximum Multi Clique Problem for Multiple Object Tracking 2015 / 7 / 26 (Fri.) : @hokkun_cv GMMCP-Tracker: Globally Optimal Generalized Maximum Multi Clique Problem for Multiple Object Tracking 1 Afshin Dehghan, Shayan Modiri Assari, Mubarak Shah University of Central Florida
2. 2. GMMCP-Tracker: Globally Optimal Generalized Maximum Multi Clique Problem for Multiple Object Tracking About me M2 2014/5CVCNN 2 Preferred Networks @tabe2314
3. 3. GMMCP-Tracker: Globally Optimal Generalized Maximum Multi Clique Problem for Multiple Object Tracking Multiple Object Tracking (MOT) YouTube (GMMCP) 3
4. 4. GMMCP-Tracker: Globally Optimal Generalized Maximum Multi Clique Problem for Multiple Object Tracking Multiple Object Tracking CVPR2015 Target Identity-aware Network Flow for Online Multiple Target Tracking (Ph.D2 4
5. 5. GMMCP-Tracker: Globally Optimal Generalized Maximum Multi Clique Problem for Multiple Object Tracking H. Possegger et al., In Defense of Color-based Model-free Tracking detection based) T. Liu et al., Real-time part-based visual tracking via adaptive correlation lters S. Tang et al., Subgraph Decomposition for Multi- Target Tracking 5
6. 6. GMMCP-Tracker: Globally Optimal Generalized Maximum Multi Clique Problem for Multiple Object Tracking Data Association (Navest) 6 Frame n Frame n+1 Bipartite Matching Problem
7. 7. GMMCP-Tracker: Globally Optimal Generalized Maximum Multi Clique Problem for Multiple Object Tracking Tracking 7 Detection Data Association http://crcv.ucf.edu/projects/GMMCP-Tracker/CVPR15_GMMCP_Presentation.pptx
8. 8. GMMCP-Tracker: Globally Optimal Generalized Maximum Multi Clique Problem for Multiple Object Tracking Tracking 8 Detection Data Association http://crcv.ucf.edu/projects/GMMCP-Tracker/CVPR15_GMMCP_Presentation.pptx
9. 9. GMMCP-Tracker: Globally Optimal Generalized Maximum Multi Clique Problem for Multiple Object Tracking Data Association (Navest) 9 Frame n Frame n+1 Bipartite Matching Problem
10. 10. GMMCP-Tracker: Globally Optimal Generalized Maximum Multi Clique Problem for Multiple Object Tracking Data Association (Navest) 10 Frame n Frame n+1 Bipartite Matching Problem
11. 11. GMMCP-Tracker: Globally Optimal Generalized Maximum Multi Clique Problem for Multiple Object Tracking Data Association (Network Flow) 11 Frame n Frame n+1 Frame n+2 Frame n+3 sources sinks minimum-cost maximum-ow problem incorporating motion feature multi-commodity network
12. 12. GMMCP-Tracker: Globally Optimal Generalized Maximum Multi Clique Problem for Multiple Object Tracking 12 Frame1 Frame2 Frame3
13. 13. GMMCP-Tracker: Globally Optimal Generalized Maximum Multi Clique Problem for Multiple Object Tracking However, Data association with network ow is simplied formulation of this problem Assuming no simplication is closer to the tracking scenario in real world. 13
14. 14. GMMCP-Tracker: Globally Optimal Generalized Maximum Multi Clique Problem for Multiple Object Tracking Data Association (Not Simplify) 14 Frame n Frame n+1 Frame n+2 Frame n+3 =0.95 =0.10
15. 15. GMMCP-Tracker: Globally Optimal Generalized Maximum Multi Clique Problem for Multiple Object Tracking Preliminary: clique () 2 see wikipedia in detail 1 OK 15
16. 16. GMMCP-Tracker: Globally Optimal Generalized Maximum Multi Clique Problem for Multiple Object Tracking Data Association (Not Simplify) 16 Frame n Frame n+1 Frame n+2 Frame n+3 Input: k-partite complete graph (k) A person form a clique maximum clique problem
17. 17. GMMCP-Tracker: Globally Optimal Generalized Maximum Multi Clique Problem for Multiple Object Tracking GMCP Tracker[1] The same team s ECCV 2012 paper They formulate MOT as generalized maximum clique problem. (cf. former page) 17[1] Amir Roshan Zamir et al., GMCP-Tracker: Global Multi-object Tracking Using Generalized Minimum Clique Graphs, ECCV, 2012.
18. 18. GMMCP-Tracker: Globally Optimal Generalized Maximum Multi Clique Problem for Multiple Object Tracking However (2), Due to complexity of the model, these approaches have been solved by approximate solutions. GMCP Tracker also used a greedy local neighborhood search, which is prone to local minima. GMCP Tracker doesn t follow a joint optimization for all the tracks simultaneously (one by one). 18
19. 19. GMMCP-Tracker: Globally Optimal Generalized Maximum Multi Clique Problem for Multiple Object Tracking Contribution 1. this approach doesn t involve any simplication neither in formulation nor in optimization (Binary Integer Problem). 2. they propose a more ecient occlusion handling strategy, which can handle long-term occlusions (e.g. 150 frames) and can speed-up the whole algorithm. 19
20. 20. GMMCP-Tracker: Globally Optimal Generalized Maximum Multi Clique Problem for Multiple Object Tracking Contribution 1. this approach doesn t involve any simplication neither in formulation nor in optimization (Binary Integer Problem). 2. they propose a more ecient occlusion handling strategy, which can handle long-term occlusions (e.g. 150 frames) and can speed-up the whole algorithm. 20
21. 21. GMMCP-Tracker: Globally Optimal Generalized Maximum Multi Clique Problem for Multiple Object Tracking 21 Low-level Tracklets Segment 01 Segment 05 Segment 06 Segment 10 Mid-level Tracklets Final Trajectories GMMCP GMMCP Input Video Human Detection Detected Humans
22. 22. GMMCP-Tracker: Globally Optimal Generalized Maximum Multi Clique Problem for Multiple Object Tracking 22 Low-level Tracklets Segment 01 Segment 05 Segment 06 Segment 10 Mid-level Tracklets Final Trajectories GMMCP GMMCP Input Video Human Detection Detected Humans
23. 23. GMMCP-Tracker: Globally Optimal Generalized Maximum Multi Clique Problem for Multiple Object Tracking Step 0: Low-level Tracklet In GMCP, the nodes at rst step are each detections. 23 Frames1-10 In GMMCP, the nodes are (low-level) tracklet How to nd: bounding boxes that overlap more than 60% between two frames are regarded as being connected.
24. 24. GMMCP-Tracker: Globally Optimal Generalized Maximum Multi Clique Problem for Multiple Object Tracking 24 Low-level Tracklets Segment 01 Segment 05 Segment 06 Segment 10 Mid-level Tracklets Final Trajectories GMMCP GMMCP Input Video Human Detection Detected Humans
25. 25. GMMCP-Tracker: Globally Optimal Generalized Maximum Multi Clique Problem for Multiple Object Tracking Step 1: Mid-level Tracklet 25 Frames1-10 Frames11-20 Frames21-30 Frames31-40 Frames41-50 Frames51-60
26. 26. GMMCP-Tracker: Globally Optimal Generalized Maximum Multi Clique Problem for Multiple Object Tracking Step 1: Mid-level Tracklet 26 = () + () Frames1-10 Frames11-20 Frames21-30 Frames31-40 Frames41-50 Frames51-60
27. 27. GMMCP-Tracker: Globally Optimal Generalized Maximum Multi Clique Problem for Multiple Object Tracking Step 1: Mid-level Tracklet 27 Frames1-10 Frames11-20 Frames21-30 Frames31-40 Frames41-50 Frames51-60
28. 28. GMMCP-Tracker: Globally Optimal Generalized Maximum Multi Clique Problem for Multiple Object Tracking 28 Low-level Tracklets Segment 01 Segment 05 Segment 06 Segment 10 Mid-level Tracklets Final Trajectories GMMCP GMMCP Input Video Human Detection Detected Humans
29. 29. GMMCP-Tracker: Globally Optimal Generalized Maximum Multi Clique Problem for Multiple Object Tracking Step 2: Final Trajectories The another but similar problem with step 1. They solve GMMCP: Nodes are Mid-level Tracklet For appearance feature, they use median (or average) feature among detections in each frame For motion feature, they use middle point of mid-level tracklet as the location of each node 29
30. 30. GMMCP-Tracker: Globally Optimal Generalized Maximum Multi Clique Problem for Multiple Object Tracking Appearance Anity Feature: Invariant Color Histogram [2] Deformation and viewpoint invariant Anity: Histogram Intersection 30[1] J. Domke et al., Deformation and Viewpoint Invariant Color Histogram, BMVC, 2006 min(H1[i], H2[i])
31. 31. GMMCP-Tracker: Globally Optimal Generalized Maximum Multi Clique Problem for Multiple Object Tracking Motion Anity 31[1] J. Domke et al., Deformation and Viewpoint Invariant Color Histogram, BMVC, 2006
32. 32. GMMCP-Tracker: Globally Optimal Generalized Maximum Multi Clique Problem for Multiple Object Tracking Optimization GMMCP is NP Hard, but they solve without any simplication. They formulate GMMCP as Binary Integer Problem (BIP, 0-1) 32
33. 33. GMMCP-Tracker: Globally Optimal Generalized Maximum Multi Clique Problem for Multiple Object Tracking 33http://www.dais.is.tohoku.ac.jp/ shioura/teaching/dais08/dais02.pdf
34. 34. GMMCP-Tracker: Globally Optimal Generalized Maximum Multi Clique Problem for Multiple Object Tracking 34http://www.dais.is.tohoku.ac.jp/ shioura/teaching/dais08/dais02.pdf
35. 35. GMMCP-Tracker: Globally Optimal Generalized Maximum Multi Clique Problem for Multiple Object Tracking Optimization GMMCP is NP Hard, but they solve without any simplication. They formulate GMMCP as Binary Integer Problem (BIP, 0-1) 35 cf. 0-1
36. 36. GMMCP-Tracker: Globally Optimal Generalized Maximum Multi Clique Problem for Multiple Object Tracking BIP in this case C is weight matrix (?) x is boolean column vector the elements of x is all of edges and nodes Ax = b is equality constraints Mx