Project Motivation
• Thumbnails used to quickly select a specific a specific image from a set (when lacking appropriate metadata), but…
• Larger sets require smaller, less detailed thumbnails.
Two subproblems
• How do we determine the salient portions of the image?
• Knowing the above, how should we crop the image?
Determining Image Saliency
• F. Liu, M. Gleicher, Region enhanced scale-invariant saliency detection, in: Proceedings of IEEE International Conference on Multimedia & Expo (ICME), 2006, pp. 1477-1480.
• Two-part method:– Scale-invariance makes this method suitable for
determining salient features of many sizes.– Region-enhancement aids in making logical crops
later on.
Scale-invariant Saliency
• Image transformed to perceptually uniform color space (Lu*v*).– Difference in color values proportional to perceived difference.
• Gaussian pyramid created and a contrast map is calculated for each level.– The contrast of each pixel is the sum of its distances from its
neighbors (in Lu*v* space).– Pixels closer to the center are given more weight.
• The contrast maps are summed to find scale invariant saliency.
• Algorithm implemented in Matlab.
Region-enhanced Saliency
• Regions extracted from the image using any preferred method.– Mean shift method used in paper (and in this
project).– C++ code from EDISON, Matlab wrapper function
from shawnlankton.com.• Scale-invariant saliency is averaged over each
region.
Saliency-based Cropping
• B. Suh, H. Ling, B. B. Bederson, and D. W. Jacobs, "Automatic Thumbnail Cropping and Its Effectiveness," in 16th Annual ACM Symposium on User Interface Software and Technology. 2003, pp. 95-104.
• What is the smallest crop that exceeds a specified saliency threshold (proportion of the total image saliency)? – Can be calculated by brute force, but inefficient
A greedy approach
• User specifies a target output size and saliency threshold.
• Select the most salient region.• Until we reach our threshold:
– Expand selection to include next most salient region.– Adjust selection to match aspect ratio of output size.
• Not guaranteed optimal crop– But much faster– Guaranteed to include whole regions
Determining Saliency Threshold Automatically
• Find the point of diminishing returns: when does increasing the crop area not become worth it?
• One simple method: select the image coming after the biggest increase in saliency.
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1Saliency vs. Area
Saliency (percent of total)
Are
a (p
erce
nt o
f to
tal)