Spatiograms Versus Histograms for Region-Based TrackingSTAN BIRCHFIELD AND SRIRAM RANGARAJAN

CLEMSON UNIVERSITY

Abstract

Histograms and spatiograms

An illustrative insight Tracking results

Conclusion

We introduce the concept of a spatiogram, which is a generalization of a histogram that includes potentially higher order moments. A histogram is a zeroth-order spatiogram, while second-order spatiograms contain spatial means and covariances for each histogram bin. This spatial information still allows quite general transformations, as in a histogram, but captures a richer description of the target to increase robustness in tracking. We show how to use spatiograms in kernel-based trackers, deriving a mean shiftprocedure in which individual pixels vote not only for the amount of shift but also for its direction. Experiments show improved tracking results compared with histograms, usingboth mean shift and exhaustive local search.

• The spatial histogram, or spatiogram, captures some spatial information about the target:• is the spatial mean of all the pixels that contribute to the bin• is the spatial covariance matrix of all the pixels that contribute to the bin

• Spatiograms are between histograms (which contain no spatial information) and specific geometric models like SSD-based translation or affine (which maintain precise spatial information)

• Introduction of a novel concept: a higher-order histogram that captures a limited amount of Introduction of a novel concept: a higher-order histogram that captures a limited amount of spatial information (spatiogram) spatial information (spatiogram)• Derivation of a mean shift procedure for spatiogramsDerivation of a mean shift procedure for spatiograms• Demonstration of improved tracking results when compared to histogramsDemonstration of improved tracking results when compared to histograms

Tracking by mean shift

To compare histograms and spatiograms, three experiments were conducted.

Likelihood function:Likelihood function:

Three poses of a head

Image generated from histogram

Image generated from spatiogram

Histogram(no spatial information)

Spatiogram(some spatial Information)

modelmodel targettarget

Experiment #1 Using mean shift. Spatiogram is slightly better, but both lose the target when the Experiment #1 Using mean shift. Spatiogram is slightly better, but both lose the target when the head jerks quickly.head jerks quickly.

Experiment #2 Using local exhaustive search (6 x 6 x 1 in x, y, and scale), with gradient dot product.Experiment #2 Using local exhaustive search (6 x 6 x 1 in x, y, and scale), with gradient dot product. Spatiogram is less distracted by the background, but both succeed in maintaining the target. Spatiogram is less distracted by the background, but both succeed in maintaining the target.

Σ

µ

A discrete function (an image):A discrete function (an image):

Binary 2D formulation:Binary 2D formulation:

The The i i th moment:th moment:

number of binsnumber of bins

target locationtarget location

HISTOGRAMSHISTOGRAMS SPATIOGRAMSSPATIOGRAMS

Mean shift update:Mean shift update:

Experiment #2 Using local exhaustive search (6 x 6 x 1 in x, y, and scale), with gradient dot product. Experiment #2 Using local exhaustive search (6 x 6 x 1 in x, y, and scale), with gradient dot product. Spatiogram succeeds, while histogram fails.Spatiogram succeeds, while histogram fails.

SPATIOGRAM HISTOGRAM