December 9, 2014Computer Vision Lecture 23: Motion Analysis 1 Now we will talk about… Motion Analysis

December 9, 2014 Computer Vision Lecture 23: Motion Analysis

1

Now we will talk about… Motion Analysis


2

Motion analysisMotion analysis is dealing with three main groups of motion-related problems:• Motion detection• Moving object detection and location.• Derivation of 3D object properties.

Motion analysis and object tracking combine two separate but inter-related components:• Localization and representation of the object of interest (target).• Trajectory filtering and data association.

One or the other may be more important based on the nature of the motion application.


3

Motion analysis


4

Differential Motion Analysis

A simple method for motion detection is the subtraction of two or more images in a given image sequence.

Usually, this method results in a difference image d(i, j), in which non-zero values indicate areas with motion.

For given images f1 and f2, d(i, j) can be computed as follows:

d(i, j) = 0 if | f1(i, j) – f2(i, j) |

= 1 otherwise


5



6

Difference Pictures

Another example of a difference picture that indicates the motion of objects ( = 25).


7

Difference Pictures

Applying a size filter (size 10) to remove noise from a difference picture.


8

Difference Pictures

The differential method can rather easily be “tricked”.

Here, the indicated changes were induced by changes in the illumination instead of object or camera motion (again, = 25).


9


In order to determine the direction of motion, we can compute the cumulative difference image for a sequence f1, …, fn of more than two images:

|),(),(|),(2

1

n

kkkcum jifjifajid

Here, f1 is used as the reference image, and the weight coefficients ak can be used to give greater weight to more recent frames and thereby highlight the current object positions.


10

Cumulative Difference Image|),(),(|),(

21

n

kkkcum jifjifajid

Example: Sequence of 4 images:0 1 1 0

0 1 1 0

0 1 1 0

0 0 0 0

a2 = 1

0 0 0 0

0 1 1 0

0 1 1 0

0 1 1 0

0 0 0 0

0 0 0 0

0 1 1 0

0 1 1 0

0 0 0 0

0 0 0 0

0 0 0 0

0 1 1 0

a3 = 2 a4 = 4

Result: 0 7 7 0

0 6 6 0

0 4 4 0

0 7 7 0


11


Generally speaking, while differential motion analysis is well-suited for motion detection, it is not ideal for the analysis of motion characteristics.

Computer Vision Lecture 23: Motion Analysis

12December 9, 2014

Optical flow

Optical flow reflects the image changes due to motion during a time interval dt which must be short enough to guarantee small inter-frame motion changes.

The optical flow field is the velocity field that represents the three-dimensional motion of object points across a two-dimensional image.

Optical flow computation is based on two assumptions:• The observed brightness of any object point is constant over time.• Nearby points in the image plane move in a similar manner (the velocity smoothness constraint).


13December 9, 2014

Optical flow


14December 9, 2014

Optical Flow

The basic idea underlying most algorithms for optical flow computation:• Regard image sequence as three-dimensional (x, y, t) space• Determine x- and y-slopes of equal-brightness pixels along t-axis

The computation of actual 3D gradients is usually quite complex and requires substantial computational power for real-time applications.


15December 9, 2014

Optical FlowLet us consider the two-dimensional case (one spatial dimension x and the temporal dimension t), with an object moving to the right:

x

t


16December 9, 2014

Optical FlowInstead of using the gradient methods, one can simply determine those straight lines with a minimum of variation (standard deviation) in intensity along them:

x

t

Flow undefined in these areas


17December 9, 2014

Optical flow

Optical flow computation will be in error if the constant brightness and velocity smoothness assumptions are violated.

In real imagery, their violation is quite common.

Typically, the optical flow changes dramatically in highly textured regions, around moving boundaries, at depth discontinuities, etc.

Resulting errors propagate across the entire optical flow solution.


18December 9, 2014

Optical flow

Global error propagation is the biggest problem of global optical flow computation schemes, and local optical flow estimation helps overcome the difficulties.However, local flow estimation can introduce large errors for homogeneous areas, i.e., regions of constant intensity.One solution of this problem is to assign confidence values to local flow estimations and consider those when integrating local and global optical flow.


19December 9, 2014

Optical flow


20December 9, 2014

Optical flow


21December 9, 2014

Detecting Motion Pattern in Optical Flow Fields

We can compute the local speed gradient to detect certain basic motion patterns.The speed gradient is defined as the direction of the steepest speed increase, regardless of the direction of motion.


22December 9, 2014


There are four different basic motion patterns: counterclockwise and clockwise rotation, contraction, and expansion. They can be combined to form spiral motion.

counterclockwise clockwise contraction expansion

movement speed gradient

Idea: A consistent angle between the direction of movement and the orientation of the speed gradient indicates a specific motion pattern.


23December 9, 2014


direction of movement

orientation of speed gradient

Angles other than 0, 90, 180, and 270 degrees correspond to spiral motion patterns.


24December 9, 2014

Feature Point CorrespondenceFeature point correspondence is another method for motion field construction. Velocity vectors are determined only for corresponding feature points.

Object motion parameters can be derived from computed motion field vectors.

Motion assumptions can help to localize moving objects. Frequently used assumptions include, like discussed above:• Maximum velocity• Small acceleration• Common motion


25December 9, 2014

Feature Point CorrespondenceThe idea is to find significant points (interest points, feature points) in all images of the sequence—points least similar to their surroundings, representing object corners, borders, or any other characteristic features in an image that can be tracked over time. Basically the same measures as for stereo matching can be used.

Point detection is followed by a matching procedure, which looks for correspondences between these points in time. The main difference to stereo matching is that now we cannot simply search along an epipolar line, but the search area is defined by our motion assumptions.

The process results in a sparse velocity field.

Motion detection based on correspondence works even for relatively long interframe time intervals.


26December 9, 2014

Feature Point Correspondence


27December 9, 2014

And this is… The End

Documents

December 9, 2014Computer Vision Lecture 23: Motion Analysis 1 Now we will talk about… Motion Analysis