View
218
Download
0
Category
Preview:
Citation preview
8/14/2019 MotionTracking ACV2009 Winter
1/38
Saturday, December 12, 2009
8/14/2019 MotionTracking ACV2009 Winter
2/38
AGENDA
Object Detection
Object Tracking
Moving Camera
Saturday, December 12, 2009
8/14/2019 MotionTracking ACV2009 Winter
3/38
OBJECT DETECTION
How do we segment
the image into objects?
Background Subtraction
Frame Difference
Saturday, December 12, 2009
8/14/2019 MotionTracking ACV2009 Winter
4/38
BACKGROUND SUBTRACTION
Background must be determined
Select a frame
Average over frames
Statistical method
Assumes images are grey
Can be done with color images, but needs a morecomplex method to do difference (other than -)
Saturday, December 12, 2009
8/14/2019 MotionTracking ACV2009 Winter
5/38
BACKGROUND DIFFERENCESELECT A FRAME
Pick one Frame
+ Easy
+ Fast
background = im2double(imread(background.jpg))frame = im2double(imread(current_frame.jpg))mask = abs(frame background)
- What happens when the frame changes?
Saturday, December 12, 2009
8/14/2019 MotionTracking ACV2009 Winter
6/38
BACKGROUND DIFFERENCESELECT A FRAME
Saturday, December 12, 2009
8/14/2019 MotionTracking ACV2009 Winter
7/38
BACKGROUND DIFFERENCEAVERAGE FRAMES
Average the image values over multiple frames
+ Easy
%%Determine Background
imname = 'image_%03d.jpg' %
back = double(zeros(size(im1))); %blank imagefor idx = 20:30
new_im = im2double(imread(sprintf(imname,idx)));
back = back + new_im;
endback = back./11
%%Difference
frame = im2double(imread(current_frame.jpg))mask = abs(frame background)
- Slower
- How many frames is enough? 10? 20? 30?
How often do we need to do this?
Saturday, December 12, 2009
8/14/2019 MotionTracking ACV2009 Winter
8/38
BACKGROUND DIFFERENCEAVERAGE FRAMES
10 Frames 30 Frames 60 Frames
Saturday, December 12, 2009
8/14/2019 MotionTracking ACV2009 Winter
9/38
BACKGROUND DIFFERENCESTATISTICAL
Determine standard deviation over range of frames
If the difference between the current frame and the average is greater than the standarddeviation, then it is a foreground pixel
+ More accurate
+ Less effected by noise
%%Determine Backgroundimname = 'image_%03d.jpg' %all_back = repmat(zeros(size(im1)),[1 1 11]); %blank imagefor idx = 20:30
new_im = im2double(imread(sprintf(imname,idx)));all_back(:,:,idx19) = new_im
endavg_back = mean(all_back,3);std_back = std(all_back,0,3);
%%Differenceframe = im2double(imread(current_frame.jpg))diff = abs(frame background)diff(diff < std_back) = 0
- Additional Processing
Needs to be constantly updated
Saturday, December 12, 2009
8/14/2019 MotionTracking ACV2009 Winter
10/38
BACKGROUND DIFFERENCESTATISTICAL
Average
Standard Deviation
Frame
Difference of Frame & Average > std dev
Saturday, December 12, 2009
8/14/2019 MotionTracking ACV2009 Winter
11/38
BACKGROUND DIFFERENCE
+ Based on simple operations (-)
+ Can be very effective in controlled situations
- Hard to determine what is the background
- Real cameras introduce noise
- Changes in lighting & introduction of objectsrequire updates to background
Saturday, December 12, 2009
8/14/2019 MotionTracking ACV2009 Winter
12/38
FRAME DIFFERENCE
Ignore determining what the background is
Look at strict frame to frame differences
+ No need for background updatingimname = 'image_%03d.jpg' %delta = 2;im_prev = im2double(imread(sprintf(imname,idxdelta)));im = im2double(imread(sprintf(imname,idx)));im_next = im2double(imread(sprintf(imname,idx+delta)));prev_diff = abs(im im_prev);
next_diff = abs(im im_next);combined_diff = cat(3,prev_diff,next_diff);diff = min(combined_diff,[],3)
- Have to look forward and backward in time
Saturday, December 12, 2009
8/14/2019 MotionTracking ACV2009 Winter
13/38
FRAME DIFFERENCE
Saturday, December 12, 2009
8/14/2019 MotionTracking ACV2009 Winter
14/38
BLOB DETECTION
Difference images are double values
Objects need to be detected as logical values
Difference images need to be thresholded
What is the right threshold?
variable?
static?
Saturday, December 12, 2009
8/14/2019 MotionTracking ACV2009 Winter
15/38
BLOB DETECTION
> 0.01
> 0.05
> 0.1Saturday, December 12, 2009
8/14/2019 MotionTracking ACV2009 Winter
16/38
NOISE MANAGEMENT
Need to remove noise from a thresholded image
Morphological processingopen
close
dilate
erode
Smoothing & Adaptive filters
Saturday, December 12, 2009
8/14/2019 MotionTracking ACV2009 Winter
17/38
MORPHOLOGICAL PROCESSING
diff_image > 0.05
Saturday, December 12, 2009
8/14/2019 MotionTracking ACV2009 Winter
18/38
MORPHOLOGICAL PROCESSING
diff_image > 0.05
bwareaopen(image,30)
Saturday, December 12, 2009
8/14/2019 MotionTracking ACV2009 Winter
19/38
MORPHOLOGICAL PROCESSING
diff_image > 0.05
bwareaopen(image,30)
imclose(opened_image,strel('disk',5))
Saturday, December 12, 2009
8/14/2019 MotionTracking ACV2009 Winter
20/38
8/14/2019 MotionTracking ACV2009 Winter
21/38
SMOOTH THE INPUTSIGNAL
No smoothing of theinput image
Smooth & reduce input
image with 5x5
gaussian low pass filter
Saturday, December 12, 2009
8/14/2019 MotionTracking ACV2009 Winter
22/38
MORE SMOOTHING
Difference image
processed with [10 7]average filter
fspecial(average,[10 7])
Thresholded at optimal value
(0.0196)
Note: No morphologicalprocessing has been done yet
Saturday, December 12, 2009
8/14/2019 MotionTracking ACV2009 Winter
23/38
Look at the histogram of the
difference image
OPTIMALTHRESHOLDING
Saturday, December 12, 2009
8/14/2019 MotionTracking ACV2009 Winter
24/38
Look at the histogram of the
difference image
OPTIMALTHRESHOLDING
Analyze the cumulative sum
Saturday, December 12, 2009
8/14/2019 MotionTracking ACV2009 Winter
25/38
Look at the histogram of the
difference image
OPTIMALTHRESHOLDING
Analyze the cumulative sum
Use second derivative to findlast inflection point
Saturday, December 12, 2009
8/14/2019 MotionTracking ACV2009 Winter
26/38
OPTIMALTHRESHOLDING
Threshold can be calculated for each frame
no need to guess
less morphological processing (which is also aguess)
Unfortunately only works on video with noise
(expects noise)
Saturday, December 12, 2009
8/14/2019 MotionTracking ACV2009 Winter
27/38
BLOB DETECTION
bwlabel
regionprops
stats =regionprops(mask,'basic')
Each stat represents ablob
Saturday, December 12, 2009
8/14/2019 MotionTracking ACV2009 Winter
28/38
OBJECT TRACKING
Segment foreground into independent blobs
Find correlation between blobs of previous frames tocurrent frame
Blobs can merge (and split)
If a blob travels multiple frames, track it (show trail)
Saturday, December 12, 2009
8/14/2019 MotionTracking ACV2009 Winter
29/38
BLOB TRACKING &MERGING
Match Blobs
Look for forward/backward matches
Only Consider blobs that overlap mask
based on linear prediction
Matching based on:
Difference = w0*|distance| + w1*Area + w2*Color
Match previous frame blobs to current frame based on thisdifference (minimize difference)
Saturday, December 12, 2009
8/14/2019 MotionTracking ACV2009 Winter
30/38
BLOB MATCHING
T=1 T=2
Saturday, December 12, 2009
8/14/2019 MotionTracking ACV2009 Winter
31/38
BLOB TRACKING &MERGING CONTD.
If multiple blobs match to the same new blob in thecurrent frame, merge
When drawing bounding boxes, draw boxes aroundsub blobs instead of the merged blobs
Before matching for next frame, decompose all blobs
to base components
not merged blob, but original matched sub blobs
Saturday, December 12, 2009
8/14/2019 MotionTracking ACV2009 Winter
32/38
MOVING CAMERA
Detect and realign video frame based on cameramotion
Find static points
Align images based on points
Saturday, December 12, 2009
8/14/2019 MotionTracking ACV2009 Winter
33/38
POINT MATCHING
Saturday, December 12, 2009
8/14/2019 MotionTracking ACV2009 Winter
34/38
POINTS
Harris Corners
Find the corners
Match the corners
SIFT
Scale Invariant Feature Transform
Saturday, December 12, 2009
8/14/2019 MotionTracking ACV2009 Winter
35/38
HARRIS CORNERS
Harris Corner Detection
Finds corners in an image
Two Options:
Find x shift, y shift for all pixels & determine highestcorrelation of x & y shift to determine image shift
faster, less accurate
Use cross correlation to match region around corner to findmatch
slower, more accurate
Saturday, December 12, 2009
8/14/2019 MotionTracking ACV2009 Winter
36/38
EXAMPLES
Saturday, December 12, 2009
8/14/2019 MotionTracking ACV2009 Winter
37/38
EXAMPLES
Saturday, December 12, 2009
8/14/2019 MotionTracking ACV2009 Winter
38/38
QUESTIONS?
?
Recommended