Upload
dyan
View
77
Download
5
Tags:
Embed Size (px)
DESCRIPTION
Sketch Tokens: A Learned Mid-level Representation for Contour and Object Detection. Joseph J. Lim (MIT), C. Lawrence Zitnick (MSR), Piotr Dollár (MSR). Contour Detection (BSDS 500). Object Detection on PASCAL2007. Overview. Method. - PowerPoint PPT Presentation
Citation preview
Sketch Tokens: A Learned Mid-level Representation for Contour and Object Detection
Joseph J. Lim (MIT), C. Lawrence Zitnick (MSR), Piotr Dollár (MSR)
Overview
Method ODS OIS AP Speed
Human 0.80 0.80 -
Canny 0.60 0.64 0.58 1/15s
gPb 0.73 0.76 0.73 240s
SCG 0.74 0.76 0.77 280s
Sketch tokens 0.73 0.75 0.78 1s
200x faster
!
Sketch Tokens
Goal: learn and detect local contour-based representation for mid-level features
Sketch Tokens:• Local edge structures (e.g. straight lines, t-junctions, y-junctions)• Discovered from human-generated image sketches
We demonstrate our approach on both top-down and bottom-up tasks.
• State-of-the-art result on contour detection, while 200x faster• Large improvements on object and pedestrian detection.
Method
Contour Detection (BSDS 500) Object Detection on PASCAL2007
Conclusion
MATLAB code is available on the website
We are given a set of images, I, and its corresponding set of binary contour images, S.
Defining Sketch Tokens
Sketch Tokens are clusters of extracted patches from the binary contour images S.
- Each patch has a fixed size of 35x35, and its center pixel must be on a labeled contour- 150 clusters are extracted using K-means on Daisy descriptors computed on binary patches.
Detecting Sketch TokensGiven a set of sketch token classes, our goal is to detect them in color images.
Each color patch’s ground truth class is assigned to one of Sketch Token or background class.
We used random forest classifier with various features (e.g. CIE-LUV intensity, orientation, and self-similarity).
Method # channels miss rate
LUV+M+O 10 17.2%
ST 151 19.5%
ST+LUV+M+O 161 14.7%
INRIA Pedestrian Detection
Method plane bike bird boat bottle bus car cat chair cow
HOG 19.7 43.9 2.2 4.8 13.4 36.6 40.2 5.4 10.9 15.7
ST 17.8 41.1 4.8 5.7 11.1 31.9 33.8 5.1 10.8 16.1
HOG+ST 21.9 48.5 6.3 6.4 14.6 41.5 43.3 6.1 15.7 19.2
Method table dog horse moto person plant sheep sofa train tv
HOG 7.5 2.1 41.9 30.9 23.9 3.4 9.3 14.8 26.9 32.4
ST 7.4 3.1 32.9 27.0 20.9 4.6 8.6 10.4 18.9 26.3
HOG+ST 14.2 3.8 46.1 34.5 30.9 8.1 15.3 18.9 30.3 36.6
We used Sketch Token responses (150 st + 1 bg dimension) on images as additional features to the deformable parts model detector.
On average, we improved 3.8 AP.
In addition to standard features used in Dollár et. al.’s implementation, we added Sketch Token responses.
t1
t2 t3
t4 t5 t6 t7
t8 t9 t14 t15