Upload
mohamed-elawady
View
114
Download
3
Tags:
Embed Size (px)
DESCRIPTION
Reading Group Activity - December 2013 B31XM Advanced Image Analysis Module Heriot-Watt University VIBOT Promotion 7 (2012-2014)
Citation preview
B31XM Advanced Image Analysis 1
Saliency Detection: A Boolean Map Approach IEEE International Conference on Computer
Vision (ICCV), 2013Jianming Zhang , Stan Sclaroff
Department of Computer Science, Boston University
Team Members : H.Kidane, I.Sadek, M.Elawady Heriot Watt University
School of Electrical and Physical Sciences
B31XM Advanced Image Analysis 2
Outline
• Introduction• Related work• Methodology• Experiments• Conclusion• Future Work
B31XM Advanced Image Analysis 3
Outline
• Introduction• Related work• Methodology• Experiments• Conclusion• Future Work
B31XM Advanced Image Analysis 4
• Goal– Proposing a Boolean map based saliency . An image is represented in a set
of binary images by randomly thresholding the image’s color channel. based on figure ground segregation
• What is saliency !– Saliency at a given location = how different this location is from its
surround color, orientation, motion, depth etc (Koch an Ullman, 1985, Itti et al. 1998). This is usually called (Bottom up saliency)
• Applications– Image segmentation – Object recognition– Visual tracking
Introduction
B31XM Advanced Image Analysis 5
Introduction
Feature Search
Fast & Effortless
Conjunction Search
Slow& Effortful
Vis
ual
In
pu
t
Sal
ien
cy M
ap
Visual Saliency Map
B31XM Advanced Image Analysis 6
Reaction time vs. number of distractors
Introduction
Feature Search
Conjunction Search
Rea
ctio
n t
ime
to f
ind
th
e ta
rget
# of distractors
(Koch & Ullman 1985, Wolfe et al 1989, Itti & Koch 2000)
B31XM Advanced Image Analysis 7
• Figure ground segregation:
– It is known as identifying the figure
from the background
• This image can be perceived as:– a vase shape in front of a black
background– two black faces on
a white background
Introduction
Rubin's Vase
B31XM Advanced Image Analysis 8
Outlines
• Introduction• Related work• Methodology• Experiments• Conclusion• Future Work
B31XM Advanced Image Analysis 9
Related Work
Method Limitation
Center surround difference
Cannot detect large salient region efficiently
Scale variant
The negative logarithm of probability
Hierarchical decomposition
Spectral domain analysis
Machine learning
Methods based on topological structure information
Strong influence on visual attentionScale invariant
B31XM Advanced Image Analysis 10
Outline
• Introduction• Related work• Methodology• Experiments• Conclusion• Future Work
B31XM Advanced Image Analysis 11
Methodology
Input Image
Boolean Maps
Attention Maps
Saliency
Mean of Attention Maps
B31XM Advanced Image Analysis 12
It is generated by randomly thresholding an input image I
Where
donates feature map of I
Randomly generated threshold in the range [0, 255]
CIE lab color space (perceptual uniformity)
Boolean MAP
)),(( ITHRESHBi
)(I
Methodology
B31XM Advanced Image Analysis 13
• the attention map A(B) is computed based on Gestalt Principle for figure-ground segregation from B
• Gestalt Principle: surrounded regions are more likely to be perceived as figure
Attention Map
Methodology
B31XM Advanced Image Analysis 14
• Given a Boolean map B, and attention Map A(B), the saliency is modeled by the mean attention map given by
A
dBIBpBAA
)/()(
Methodology
B31XM Advanced Image Analysis 15
Methodology
B31XM Advanced Image Analysis 16
Outline
• Introduction• Related work• Methodology• Experiments• Conclusion• Future Work
B31XM Advanced Image Analysis 17
Experiments
B31XM Advanced Image Analysis 18
ExperimentsEye Fixation Prediction
Implementationdetails
Width of image resizing 600
Width of kernel for opening operation 5
Sampling step size 8
Width of kernel for dilation operation 7
Width of kernel for opening operation(before Gaussian blurring)
23
Standard deviation of Gaussian blurring
20
Removing Small peaks on mean attention map
http://mentormate.com/
B31XM Advanced Image Analysis 19
Datasets
MIT 1003
Toronto 120
Kootstra 100
Cerf 181
ImgSal 235
http://www.cse.cuhk.edu.hk
Evaluation Metric
AUC
Shuffled AUC
Border CutCenter-Bias
Sampling
ExperimentsEye Fixation Prediction
B31XM Advanced Image Analysis 20
Original
GT
BMS
ΔMQDCT
SigSal
LG AWS
HFT
CAS
Judd
AIMGBVS
Itti
ExperimentsEye Fixation Prediction
B31XM Advanced Image Analysis 21
Original
GT
BMS
ΔMQDCT
SigSal
LG AWS
HFT
CAS
Judd
AIMGBVS
Itti
ExperimentsEye Fixation Prediction
B31XM Advanced Image Analysis 22
Original
GT
BMS
ΔMQDCT
SigSal
LG AWS
HFT
CAS
Judd
AIMGBVS
Itti
ExperimentsEye Fixation Prediction
B31XM Advanced Image Analysis 23
ExperimentsEye Fixation Prediction
B31XM Advanced Image Analysis 24
ExperimentsEye Fixation Prediction
B31XM Advanced Image Analysis 25
ExperimentsEye Fixation Prediction
B31XM Advanced Image Analysis 26
ExperimentsEye Fixation Prediction
B31XM Advanced Image Analysis 27
ExperimentsEye Fixation Prediction
B31XM Advanced Image Analysis 28
Optimal average shuffled-AUC with
corresponding Gaussian blur STD
2012 2012 2012 2011 2013 2012 2009 2009 2007 1998
Less Background Distraction
No Multi-Scale Processing
ExperimentsEye Fixation Prediction
B31XM Advanced Image Analysis 29
Runtime performance
Programming Language
C++
Image Size 600x400
Processor 2.5 GHz
Running OS Windows
Memory 2 GB
http://runtime.bordeaux.inria.fr
CAS 78 LG 13
AWS 10 Judd 6.5
AIM 4.8 GBVS 1.1
ΔQDCT 0.49 Itti 0.43
HFT 0.27 SigSal 0.12
BMS 0.38
ExperimentsEye Fixation Prediction
ExperimentsSalient Object Detection
High Blurred
BinaryImage
Width of kernel for opening operation for boolean maps is modified to 13
Turning off the dilation operation for attention maps
Post-processing for mean attention map using (opening, closing) operations with kernel size (5)
Binarizing saliency map at a fixed threshold
Object Level Segmentation
30B31XM Advanced Image Analysis
B31XM Advanced Image Analysis 31
Original
GT
BMS
GSSP
HSal
RC
FT
CAS
HFT
ExperimentsSalient Object Detection
B31XM Advanced Image Analysis 32
ExperimentsSalient Object Detection
ASDDataset
B31XM Advanced Image Analysis 33
ExperimentsSalient Object Detection
ASDDataset
B31XM Advanced Image Analysis 34
Outlines
• Introduction• Related work• Methodology• Experiments• Conclusion• Future Work
B31XM Advanced Image Analysis 35
Conclusion
BMS has powerful advantage in surroundence
aspect•Helpful in figure-ground segregation
BMS is only model that consistently achieves the state-of-art performance
•Best results in different five eye-tracking datasets
•Proper results in salient object detection
B31XM Advanced Image Analysis 36
Outlines
• Introduction• Related work• Methodology• Experiments• Conclusion• Future Work
B31XM Advanced Image Analysis 37
Future Work
• Not only color channels• Feature channels (i.e. orientation, depth,
and motion)
Improve the effectiveness of BMS
• Integrating more saliency cues (i.e. convexity, symmetry, and familiarity) instead of current one (eliminating regions that touch image borders)
Improve the attention map computation
B31XM Advanced Image Analysis 38