View
216
Download
1
Category
Tags:
Preview:
Citation preview
A Novel Method for Generation of Motion Saliency
Yang Xia, Ruimin Hu, Zhenkun Huang, and Yin Su
ICIP 2010
Outline
• Introduction• Itti’s model• Proposed Visual Saliency– Generation of motion feature map– Enhancement of motion sub-saliency map
• Experiment Results• Conclusion
Introduction
• Visual saliency– Bottom up saliency– Top down saliency– Applications• Image segmentation, motion detection, image/video
compression……
• Motion saliency– Motion object is more salient to human vision
system(HVS) than spatial contrast in video.
Itti’s Model
• For Image– Spatial features• Color, Intensity, and Orientation
– Feature maps– Combining the normalized activation maps
• Visual Saliency Model For Video– Temporal features• Flicker and Motion
Itti’s Model
• Motion feature map– Computed by spatially-shifted differences between Gabor
pyramids from the current frame n and previous frame n-1– Motion feature
– The minimum captured object velocity at scale
: motion feature map for scale and orientation : Gabor pyramid of original frame n : the shifted Gabor pyramid of original frame n
dx, dy : horizontal and vertical shift distancef : the frame rate
Itti’s Model
• Drawback– Inaccurate when the objects move slowly• when the velocity is smaller than , none of the
pyramidal scales can capture the movement • group into the background
– Only the edge of object is labeled salient• Using spatially-shifted differences
Multi reference frame
Enhance motion saliency map by spatial saliency information
Proposed Visual Saliency
• Generation of motion feature map– Multi reference frames to enhance the ability to
capture object movement
– Motion feature map
– Processed by graph theory to form the activation map[1]
[1] J. Harel, C. Koch, and P. Perona, “graph-based visual saliency,” in Advances in Neural Information Processing Systems 19, Cambridge, MA: MIT Press, 2007
(Reference frame) np
add two velocity profiles about and
Proposed Visual Saliency
• Enhancement of motion sub-saliency map– Spatial sub-saliency map
– Find the point which belongs to salient object• Check un-salient point is near a salient point which has
large saliency value both in motion and spatial sub-saliency maps, and its spatial saliency value is close to that of the salient point.
– Update the motion saliency– Generate the whole saliency map
Enhancement of Motion Sub-saliency Map
{𝑆𝑖 } {𝑀𝑖 }
Top 25% of locations which have larger saliency values in SMS
Top 5% of locations which have larger saliency values in SMM
𝑑(𝑝𝑖)
𝑝𝑖
𝑛𝑞𝑖If and the difference of the spatial saliency values between and
New saliency location set
Motion saliency points
Whole saliency map +
Experimental Results
• Dataset in CAVIAR—ThreePastShop1cor– ROC(Relative Operating Characteristic) score between estimated
saliency maps(ESMs) and ground-truth saliency maps(GSMs)
Anchor1: Itti’s modelAnchor2: Itti’s model using activation operator based on graph theorySMRF: saliency model with the multi-reference framesSMRF+STE: plus spatio-temporal enhancement
Experimental Results
motion channel
five channel
anchor1
SMRF+STE
anchor2
anchor1
anchor2
SMRF
SMRF+STE
Recommended