Context-Aware Saliency Detection

Stas GofermanLihi Zelnik-Manor

Ayellet Tal

Technion

1. Introduction 2. Principles of context-aware saliency 3. Detection of context-aware saliency 4. Result 5. Applications

Outline

Aim at detecting the image regions that

represent the scene. This definition differs from previous definitions

whose goal is to either identify fixation points or detect the dominant object

Context-aware saliency

What most people think is important or salient

1. Introduction 2. Principles of context-aware saliency 3. Detection of context-aware saliency 4. Result 5. Applications

Outline

1. Local low-level considerations, including factors such as contrast and color.

Areas that have distinctive colors or patterns should obtain high saliency

Conversely, homogeneous or blurred areas should obtain low saliency values

2. Global considerations, which suppress frequently- occurring features, while maintaining features that deviate from the norm.

Four Principles(1/2)

3. Visual organization rules, which state that

visual forms may possess one or several centers of gravity about which the form is organized.

The salient pixels should be grouped together, and not spread all over the image

4. High-level factors, such as human faces.

Implemented as post-processing.

Four Principles(2/2)

1. Local low-level (b) 2. Global (c) 3. Visual organization rules about (b) + (c) 4. High-level factors (post-processing)

Four principles

1. Introduction 2. Principles of context-aware saliency

3. Detection of context-aware saliency 3.1 Local-global single-scale saliency 3.2 Multi-scale saliency enhancement 3.3 Including the immediate context 3.4 High-level factors 4. Result 5. Applications

Outline

Local-global single-scale saliency

dcolor(pi,pj) is the Euclidean distance between the vectorized patches pi and pj in CIE L*a*b color space.dposition(pi,pj) is the Euclidean distance between the positions of patches pi and pj .

Dissimilarity measure between a pair of patches as:

Only considering the K most similar patches in the local measurement.

Single-scale saliency value of pixel i at scale r is defined as:

Multi-scale saliency enhancement

Background patches are likely to have similar patches at multiple scales, Searching K most similar patches in the local measurement in scale R1 = {r,0.5r,0.25r}

Representing each pixel by the set of multi-scale image patches centered at it. The saliency at pixel i is taken as the mean of its saliency at different scales

The steps of our saliency estimation algorithm *Multiple scales foreground *Few scales background

Steps

1: A pixel is considered attended if its saliency value exceeds a certain threshold ( Si > 0.8). 2: The saliency of a pixel is redefined as

Let dfoci(i) be the Euclidean positional distance between pixel i and the closest focus of attention pixel, normalized to the range [0,1]

Including the immediate context

Final , face detection or recognized objects

High-level factors

face detection algorithm of [23], which generates 1 for face pixels and 0 otherwise.

1. Introduction 2. Principles of context-aware saliency 3. Detection of context-aware saliency 4. Result 5. Applications

Outline

Result(1)

Result(2)

Comparing the saliency map in the paper with [13].Top: Input images. Middle: the bounding boxes obtained by [13] capture a single main object. Bottom: the saliency map convey the story

[13] T. Liu, J. Sun, N. Zheng, X. Tang, and H. Shum. Learning to Detect A Salient Object. In CVPR, 2007.

1. Introduction 2. Principles of context-aware saliency 3. Detection of context-aware saliency 4. Result 5. Applications 5.1. Image retargeting 5.2. Summarization through collage creation

Outline

Seam Carving for Content-Aware Image Resizing

Image retargeting aims at resizing an image by

expanding or shrinking the non-informative regions.

Image retargeting

Summarization

1.Computing the saliency maps

2.Extracting ROI by considering both the saliency and the image-edge information

3.Assemble the non-rectangular ROIs, allowing slight overlaps

Summarization through collage creation

(b)The collage summarization