Topics in Vision and GraphicsRichard Hartley

I am not going to talk about this stuff …

Sensors:• FaceLab• CeDAR active head• 4 x Sony NTSC cameras• 6 DOF INS• Tail shaft encoder• Steering potentiometer• Strain gauges on steering wheel

Actuation:• Throttle through cruise control• Steering drive motor• Brake linear actuator

Experimental Vehicle

Speed Sign Detection& Recognition

High reliability detectors for speed and traffic signs.

Example of hyperspectral imagery in the detection of fruit damage.

Image of an imaged cylinder, mapped onto a 3D surface of that object (CAD model).

Hyperspectral and IR imagery

Spectral Imagery

Automatic Anatomical Structure ExtractionAnalysis of MR brain images

Specularity detection

a. Input b. Normalized image

c. Diffuse component

c. Specular Component

Color Constancy

Reflection Separation

Chief Researcher: Robby Tan

New View Synthesis

•Generation of new-viewpoint images from sets of images or Video

•Possible application in making 3D video from 2D.

Key researcher: Hongdong Li

Radial Distortion Correction

Improved methods of detecting and correcting radial distortion

Key researchers: Hongdong Li, Richard Hartley

E-med project – Collaboration with Lions Eye Institute

• Focus on ophthalmologicalapplications– Diabetic retinopathy– Glaucoma detection through

stereo imaging– Tear film detection.

Key researcher: Richard Hartley (+ grad student)

Ubiquitous Flying Insect Robot

Design and use of a small mobile robot and investigation of insect-based vision

Key researchers: Nick Barnes, M. Srinivasan

Background Subtraction and Applications

But I am going to talk about this …

Original Video

Our original algorithm

Synopsis Video

More difficult example

Shyjan Algorithms

Show cook fountain video

Synopsis Video

Synopsis with Overlay

Indoor Sequence

Tracking People

Show Soccer Video

Applications for Surveillance

Measuring dimensions and trajectory

•The estimated heights are: •Tiberio: 1.79m•Peter: 1.89m•Richard: 1.95m

Trajectory

Robust to change in scale and view point

Recognition of People in Videos

Show Soccer TrackingKylie

How do you do this?

Computer Vision in Hollywood

Courtesy Oxford Visual Geometry group

Courtesy Oxford Visual Geometry group

Courtesy Oxford Visual Geometry group

What is the mathematics behind this?

Projective Geometry

Felix Klein: Erlangen Program (1872)

View of geometry as an analysis of what is invariant under different groups of transforms.

i.e. What is unchanged about the plane when it undergoes a particular type of transformation (motion, distortion, change).

Euclidean GeometryThe geometry of rigid transformations (congruence).

Invariants are:

• Angles, distances, areas

• Circles remain circles

• Straight lines remain lines

Concepts of Euclidean Geometry•Circle

•Length

•Area

•Congruent triangles

•Parallelism

•Angles

All these are invariant (unchanged) under rigid transformations.

Projective GeometryThe geometry of projective transforms.

What transformations does a plane undergo when you take an image of it with a pinhole camera?

Parallel lines are not preserved

Neither are angles, areas, distance-ratios, lengths

Circles are projected as ellipses

Fisheye imageProjective Image

• Circles are not preserved under projective transformations.

• They can be seen as ellipses (including circles), parabolas, hyperbolas, (second degree curves).

• The right concept in projective geometry is a “conic”.

• Conics are preserved by projective transformations.

“Second Order” curves.

Any two lines meet in exactly one point

Line at infinity

Projective TransformationAny mapping of the projective plane to itself that preserves lines is called a “projective transformation”, or “collineation”, or “homography”

Mapping: A mapping from points in the plane to points in the plane.

Preserves lines: If three points lie on a line, then their corresponding points under the mapping lie on a line.

Tracked Points

Billboard: Image in an image

Building Models from Video

Courtesy Marc Pollefeys

Courtesy Marc Pollefeys

Single View Reconstruction

Original Painting

Synthetic Views

Interactive Video

Interactive video navigation using a game-controller.

Navigation is done using a game controller.

The End