Computer Science Department, Duke University PhD Defense TalkMay 4, 2005

Fast Extraction of Feature Salience Maps for Rapid Video Data Analysis

Nikos P. Pitsianis and Xiaobai Sun

Fast Extraction of Feature Salience Maps for Rapid Video Data Analysis

• FSMs are used in – separation or integration– automatic or assisted visual search tasks – target indication, object recognition, tracking

• Salient information in multiple feature dimensions– color, edge orientation, shape, texture, motion– selective tuning, feedback, attentional or intentional guidance

• High volume and rate of video data, frame by frame• Involves many filtering steps at multiple spatial scales

Feature Salience Maps (FSMs)

Sept 15, 2010HPEC 2010 2

MUNDHENK, T. N., ITTI, L. Computational modeling and exploration of contour integration for visual saliency. Biological Cybernetics (2005).

Fast Extraction of Feature Salience Maps for Rapid Video Data Analysis

• Direct domain– Filter-centric– Image-centric

• Fourier domain– Based on the convolution

theorem

Processing of feature maps on GPU

Sept 15, 2010HPEC 2010 3

• Using CUDA SDK 3.1

– NVIDIA Tesla C1060

• 240 processing cores @ 1.3GHz

• 4GB or GDDR3

– CUFFT CUDA FFT library

– Asynchronous I/O and Streaming

5x5 10x10 15x15 20x200.05

0.1

0.15

0.2

0.25

0.3

NVIDIA Tesla C1060 @1.3GHz, CUDA v3.12D Convolutions of 10 512x512 frames and 16 filters/frame

Template Size

Exe

cutio

n W

allc

lock

Tim

e (s

ec)

Spatial domain

Fourier domain

5x5 10x10 15x15 20x2020

40

60

80

100

120

140

NVIDIA Tesla C1060 @1.3GHz, CUDA v3.12D Convolution of 512x512 frames and 16 filters/frame

Template Size

Fra

mes

Per

Sec

Spatial domain

Fourier domain

Fast Extraction of Feature Salience Maps for Rapid Video Data Analysis

• The extraction and use of salient information from static or dynamic images are recent and active research topics

• The computation based on an extraction model serves two purposes– Test and validate the underlying neurobiological model for certain

visual function in the visual system of the primate brain– Exploit the new understanding and model(s) for developing and

improving artificial vision systems. • Challenges :

– generation of motion features, which are much more computation intensive

– visual tasks at the higher levels• segmentation, object recognition, tracking of moving targets. • data representation at higher levels, sparse and irregular, but still

structured– Efficiency of high-level processing steps on GPUs

Discussion

Sept 15, 2010 4HPEC 2010