Content Analysis and Restoration of Marine Video

Student :- Ken Sooknanan

Supervisor :- Professor Naomi Harte in collaboration with Prof. Jim Wilson

(Dept. of Zoology), the Marine Institute Ireland, and Prof. Anil Kokaram (Google)

Goals

1) Content Analysis:-

(a) Identify Nephrop burrow complexes

(b) Identify when major changes along the seabed occur.

(c) Rocky (d) Muddy Video 2) Restoration:-

(a) Improve Visibility by correcting illumination degradations due to light source.

(a) Complex (b) Nephrop Video

Light Footprint

Improving Visibility

(2) S, c estimated with ratio of pt. correspondences

(1) Degradations modelled as mixture

optimized alternately in Bayesian Framework

G(x) )(5.3 2

)( xrexI

r ≤ rf)(xIr > rf

= );()()(2 cxScxxr T

s1 s2

s2 s3S =

cx cy

; c = )(xI )(5.3 2

)( xrexI G(x)

21

1

22

2

5.3

5.3

reI

reI

2

1

G

G 21

22

1

2 5.3ln rrG

G

cn+1=arg max p(c|G1,G2,Sn);

Sn+1=arg max p(S|G1,G2,cn+1);

(3) Footprint estimated as region where largest

percentage increase in degradation occurred G1(r1)

G2(r2)(c) A(rx) = (d) A(rx) > μ. Split into

regions, footprint (red)

Frame 2

G2

c

r2G1

c

r1

Frame 1

Results

(a) Original with

footprint in blue

(b) Our Result (c) Seon Joo

Original Result

(d) Seon Joo Result

with our radii est.

IMAGES :-

VIDEO :-

[1] Seon Joo, et al.., “Robust radiometric calibration and vignetting correction," Pattern Analysis and Machine Intelligence, 2008

(a) Sample Video Frame

(b) Generated Mosaic

(using 1st 50 Frames)

Identifying Nephrop Clusters

• Exploring the use of a Mosaic, as it:-

- Eliminates tracking

- Fixes geometric distortion.

- Computations reduced to a single image

- Provides a wide view of the seabed.

- Easy to spot clusters.

• Problem broken up into two parts:-

- identifying all burrows

- clustering

• Currently working on identification part

Algorithm overview

- Cascade Classifier,

p(f) > (T =0.5) SHAPE

(1) Highlight all dark regions

- DOG

4) Classify based on shape, colour and shading features, f,

2) Segment and label DOG – Intensity based (Bayesian framework - ICM).

3) Identify and Split multiple burrows regions – shape modelling with GMMs

(a) Original (b) Segmentation (c) Est. GMM (d) Split Region

COLOUR SHADDING

25.0

)(

f

efp

Initial Results

(a) Original Mosaic (b) Identified Burrows

Exp. Video Results Mosaic Results

GroundTruth

% CorrectDetected

GroundTruth

Recall %

Precision %

1 50 90 50 90.0 84.0

2 63 85.7 60 82.5 88.3

3 84 88.1 83 88.1 85.5

Results obtained from 3 Video

Sequences (10 min., 15000

Frames) Compared with

Ground Truth from:-

(1) Video (existing method)

(2) Mosaic

Accomplishments

(1) Gave a presentation in Google, San Francisco on 23rd Jan. 2012

(2) Published a paper [1] in SPIE conference, San Francisco Jan. 2012

(3) Submitted a paper [2] on Burrow Detection algorithm in ICIP 2012.

(4) Submitted an abstract [3] on Mosaicing Algorithm in Oceans conference 2012.

[1] Sooknanan, et al.., “Improving Underwater Visibility using Vignetting correction," in proceedings of SPIE, 2012[2] “A Bayesian Framework for Detection of Nephrop Burrows for Seabed Video Analysis,” ICIP 2012 [3] “A Bayesian Framework for Mosaic Creation of the Seabed from Underwater Video For Nephrop Burrow Detection”, Oceans, 2012

Year 1:- Attended and passed 3 courses to gain the necessary 15-credits

Year 2:- Wrote year-1 transfer report and passed viva to enter PhD roster.

Year 3:-

(5) Established good collaboration with the Marine Institute Galway

- keep in contact with Jennifer Doyle approximately once every month.

Future Work(1) Present work in Study Group on Nephrops Survey (SGNEPS) Meeting in

Italy in 8th March 2012.

(2) Do more testing with Burrow detection algorithm

(3) Move onto the Burrow clustering part of the problem

(4) Move onto detecting when major changes in seabed type occur

(5) Write papers on (3) and (4)

(6) Write up Thesis.