Ter Haar Romeny, EMBS Berder 2004 How can we find a dense optic flow field from a motion sequence in 2D and 3D? Many approaches are taken: - gradient based

ter Haar Romeny, EMBS Berder 2004

How can we find a denseoptic flow field from a motion sequence in 2D and 3D?

Many approaches are taken:

- gradient based (or differential);- phase-based (or frequency domain);- correlation-based (or area);- feature-point (or sparse data) tracking.

Multi-scale optic flow


Reichardt detector

In the visual front-end retinal receptive fields are organized inpairs, tuned to a specific velocity and direction. The pairs arecoupled by a delay cell, possibly the amacrine cell.

Neurons act as temporalcoincidence detectors

This leads to a redundantrepresentation, all velocities and directionsare measuredat all scales.


Amacrine cellsare found nextto ganglion cellbodies

Similar RF pairsare present inboth eyes fordisparitydetection


Calibration

We generate a test sequence with a warping vector field, sowe know the absolute displacement of each pixel


The isophote landscape of an image changes drastically when we change our aperture size. This happens when we move away or towards the scene with the same camera. Left: observation of an image with = 1 pix, isophotes L=50 are indicated. Right: same observation at a distance twice as far away. The isophotes L=50 have now changed.


scalarflow densityflow

Scalar images: intensity is kept constant with the divergenceDensity images: intensity ‘dilutes’ with the divergence

Two types of images need to be considered:


The Lie derivative (denoted with the symbol v) of a

function Fg with respect to a vectorfield v is defined as

vFg. The optic flow constraint equation (OFCE) states

that the luminance does not change when we take the

derivative along the vectorfield of the motion:

vFg 0


vFg F.v

v Div v v. 0

Multi-scale optic flow constraint equation:

For scalar images:

For density images:

The velocity field is unknown, and this is what we want to recover from the data. We like to retrieve the velocity and its derivatives with respect to x, y, z and t. We insert this unknown velocity field as a truncated Taylor series, truncated at first order.


Multi-scale density flow: in each pixel 8 equations of third order and8 unknowns:


Scale selection:

The condition number of the coefficient matrix exhibits an optimumover scale in many pixels, given the local density of texture.

0 5 10 15 20scaleindex

0.1

1

10

100

norm


0.8

3.0

Artificially created test image sequencefor validation purposes

Scale selection map


A. Suinesiaputra, UMCL / TUE, MICCAI 2002

Documents

Ter Haar Romeny, EMBS Berder 2004 How can we find a dense optic flow field from a motion sequence in 2D and 3D? Many approaches are taken: - gradient based