Human Detection Human Detection

Mikel RodriguezMikel Rodriguez

OrganizationOrganization

1.1. Moving Target Indicator Moving Target Indicator (MTI)(MTI)

Background modelsBackground models Moving region detectionMoving region detection Target chip generation Target chip generation ResultsResults

Input Frame

Object Detectio

n

Target Chips

Wavelet Features

SVM Classifier

MTI Classification

2.2. Target Classification Target Classification (Human Detection)(Human Detection)

Target featuresTarget features Support vector machinesSupport vector machines ResultsResults

Moving Target IndicatorMoving Target Indicator

Moving target indicator (MTI) identifies moving Moving target indicator (MTI) identifies moving objects which can be potential targetsobjects which can be potential targets

MTI MotivationMTI Motivation

Becoming increasingly important in military Becoming increasingly important in military and civilian applicationsand civilian applications

To minimize human involvementTo minimize human involvement ExpensiveExpensive Short attention spansShort attention spans

Computerized monitoring systemComputerized monitoring system Real-time capabilityReal-time capability 24/724/7

MTI ChallengesMTI Challenges

Different sensor modalitiesDifferent sensor modalities LADAR, IR, EOLADAR, IR, EO

Targets with different dynamicsTargets with different dynamics Small targetsSmall targets Weather conditionsWeather conditions

Illumination changes, shadows…Illumination changes, shadows…

Input Video

MTIMTI

BackgroundModeling

Intensitymodels

Gradient models

Moving Target Detection

BackgroundSubtraction

dynamic update

Targets

Chips Position

Hierarchical Approach to Background Hierarchical Approach to Background ModelingModeling Pixel levelPixel level Region levelRegion level Frame levelFrame level

Pixel LevelPixel LevelBackground FeaturesBackground Features

Intensity, heat indexIntensity, heat index

GradientGradient 2D: magnitude, orientation2D: magnitude, orientation

20 40 60 80 100 120 140 160 180 200

20

40

60

80

100

120

140

160

20 40 60 80 100 120 140 160 180 200

20

40

60

80

100

120

140

160

IREO

Magnitude Orientation

Pixel LevelPixel Level Background FeaturesBackground Features

Intensity, heat indexIntensity, heat index Per-pixel mixture of Per-pixel mixture of

Gaussians.Gaussians.

Gradient based subtractionGradient based subtraction Gradient feature vector Gradient feature vector

=[=[mm, , dddd]]

22yxm ff

)(tan 1

x

yd f

f

Pixel LevelPixel LevelMoving Region DetectionMoving Region Detection

Mark pixels that are different from the background intensity Mark pixels that are different from the background intensity modelmodel

Mark pixels that are different from the background gradient Mark pixels that are different from the background gradient modelmodel

Color basedImage Gradient

Region LevelRegion LevelFusion of Intensity & Gradient ResultsFusion of Intensity & Gradient Results

• For each color based region, presence of“edge difference” pixels at the boundaries is checked.

• Regions with small number of edge difference pixel are removed, color model is updated.

Final

Frame LevelFrame LevelModel UpdateModel Update

Performs a high level analysis of the scene Performs a high level analysis of the scene componentscomponents

If more > 50% of the intensity based

background subtracted image becomes

foreground.

Frame level processing issues an alert

Intensity based subtraction results

are ignored

Structure of the MTI ClassStructure of the MTI ClassMTI

Background

ConnectedComponents()BoundaryEdges()SetNumGaussians()SetAlpha()SetRhoMean()SetWeightThresh()SetActiveRegion()GetNumGaussians()GetAlpha()GetRhoMean()GetWeightThresh()GetActiveRegion()

Object

SetBoundingBox()SetRhoLocation()SetCentroid()GetBoundingBox()GetRhoLocation()GetCentroid()IsFalseDetection()

Chips

Centroid()ObjectArea()Height()Width()

ResultsResults

Target ClassificationTarget Classification

Classification of objects into two classes: Classification of objects into two classes: humans humans and and othersothers, from target chips generated by MTI, from target chips generated by MTI

ChallengesChallenges

Small sizeSmall size Obscured targetsObscured targets Background clutterBackground clutter Weather conditionsWeather conditions

Classifier FlowClassifier Flow

FeatureExtraction

Wavelet

Testing

MTI Chips

Neg

ativ

e

Pos

itive Training

SVM

Support

Vectors

DecisionDecision

Wavelet Based Target FeaturesWavelet Based Target FeaturesBlurred

Vertical

Horizontal

Diagonal

Feature ExtractionFeature Extraction

Apply 2D Wavelet TransformApply 2D Wavelet Transform Daubechies waveletsDaubechies wavelets

Apply Inverse 2D Wavelet Transform to each Apply Inverse 2D Wavelet Transform to each of the coefficient matrices individuallyof the coefficient matrices individually

Rescale and vectorize output matricesRescale and vectorize output matrices

Why Wavelets?Why Wavelets?

Separability among samplesSeparability among samples Humans can be separated from can be separated from cars and background

Correlation using gray levels Correlation using gradient mag.

Why Wavelets?Why Wavelets?

Person 11 - DB3 Wavelet Correlation

Support Vector Machines (SVM)Support Vector Machines (SVM)

Classification of data into two classesClassification of data into two classes NN dimensional data. dimensional data. Linearly separableLinearly separable

If not transform data into a higher dimensional If not transform data into a higher dimensional spacespace

Find separating Find separating NN dimensional hyperplane dimensional hyperplane

SVMSVMLinear ClassifierLinear Classifier

0bTxw

0bTxw

0bTxwhyperplane equation

N dimensional data point xi

w

xw br

T

Sample distance to hyperplane

SVMSVMBest Hyperplane?Best Hyperplane?

Infinite number of hyperplanes.Infinite number of hyperplanes. Minimize Minimize rrii over sample set over sample set xxii

Maximize margin Maximize margin around hyperplane around hyperplane Samples inside the margin are the support vectorsSamples inside the margin are the support vectors

0

0

0

22

11

b

b

b

T

T

T

xw

xw

xw

w

2

SVMSVMTraining SetTraining Set

Let Let =1=1,A training set is a set ,A training set is a set of tuples: of tuples: {({(xx11,,yy11),(),(xx22,,yy22),…),…

((xxmm,,yymm)})}..

For support vectors inequality For support vectors inequality becomes equalitybecomes equality

Unknowns are Unknowns are ww and and bb

1 if1

1 if1

jjT

iiT

yb

yb

xw

xw

SVMSVMLinear SeparabilityLinear Separability

Linear programming, Linear programming, Separator line in 2D Separator line in 2D ww11xxii,1,1+w+w22xxii,2,2++bb=0=0..

Find Find ww11,, ww22,, b b such that such that is maximized is maximized

Find Find ww11,, ww22,, b b such that such that ((ww)=)=wwTTww is minimized is minimized

1by iT

i xw

SVMSVMSolutionSolution

Has the following form:Has the following form:

Non-zero Non-zero ii indicates indicates xxii is support vector is support vector

Classifying function is:Classifying function is:

iT

iiiii yby xwxw and

byf Tiii xxx )(

Classification ClassClassification ClassHuman Classification

TrainingFunction

ReadPositiveImages()ReadNegativeImages()AssemblePositive()AssembleNegative()AssembleMatrices()

TestingFunction

LoadSVM()ReadImages()

ExtractFeatures

ConvertToGray()ApplyWaveletFilter()ApplyInverseTrans()ResizeInverse()VectorizeInverse()Concatenate()

TrainSVM

LIBSVM

TestSVM

LIBSVM

Classification Baseline AnalysisClassification Baseline Analysis

Run time for 3.0GHz dualcore, 2GB RAMRun time for 3.0GHz dualcore, 2GB RAM Training:Training: 276 training samples 8.015 seconds 276 training samples 8.015 seconds Testing:Testing: 24.087 chips (25 by 25) per second 24.087 chips (25 by 25) per second

Classifier sizeClassifier size Depends on diversity of imagesDepends on diversity of images For 276 training samples of 25x25, classifier For 276 training samples of 25x25, classifier

size is 1.101 MBsize is 1.101 MB

Classification Baseline AnalysisClassification Baseline Analysis

Memory requirementsMemory requirements Requires entire set of support vectorsRequires entire set of support vectors

Current classifierCurrent classifier

pixelsof #bits32tscoefficien wavelet4SV of # M

MbbitsbitM 3.22400000252512830

ExperimentsExperiments

Vivid DatasetVivid Dataset UCF DatasetUCF Dataset

ResultsResults

Training setTraining set 300 target chips300 target chips

TestingTesting 3872 human chips3872 human chips 5605 vehicle and background chips5605 vehicle and background chips

PerformancePerformance 2.4% false positive 2.4% false positive (others classified as pedestrians)(others classified as pedestrians)

3.2% false negative 3.2% false negative (pedestrian classified as others)(pedestrian classified as others)

Future directionsFuture directions

MTIMTI Detection by partsDetection by parts Motion clusteringMotion clustering

ClassificationClassification Various kernels for SVMVarious kernels for SVM Better target featuresBetter target features

Motion, steerable pyramids, shape features Motion, steerable pyramids, shape features (height, width)(height, width)

Local wavelet coefficients Local wavelet coefficients Adaboost Adaboost