Detection, tracking and sizing of fish of in data from DIDSON multibeam sonars

1

Detection, tracking and sizing of Detection, tracking and sizing of fish of in data from DIDSON fish of in data from DIDSON multibeam sonars multibeam sonars

Helge Balk1, Torfinn Lindem1, Jan Kubečka2

1 Department of Physics, University of Oslo, PO.Box.1048. Blindern, NO-0317 Oslo, Norway email: helge.balk@fys.uio.no, Torfinn.lindem.@fys.uio.no

2 Biology Centre of Czech Academy of Sciences, Institute of Hydrobiology, Na sadkach 7, CZ 37005 Ceske Budejovice, Czech Republic. e-mail: kubecka@hbu.cas.cz,

2

CFD AND CFD AND DIDSONDIDSON

tracking

3D approach

Detection methods

Echogram approach

Introduction

Conclusion

Inc.Video methods

3

Placing Norway on the mapPlacing Norway on the mapUniversity of Oslo No

Biological institute Cz

Our main interest Our main interest

As usual to find out abot the fish

How many How big What are they doing

4

Equipment that may be usedEquipment that may be used

Resons-SeabatCoda Octopus EchoscopeDIDSON

Simrad MS70Simrad SM2000 Split beam

DIDSONDIDSON

Dual frequency Identification SONnar Developed for military underwater tasks like

diver night vision and mine searching

Become popular for fish studies Identification ability Can see pictures of the fish. Fish size from geometry, not from TS

6

Our aimOur aim

Develoop a target detector for DIDSON data

Can vi use the Cross Filter Detector CFD develooped for ordinary echogram

If not, can we optimise it to fit the DIDSON data

Or is there something to learn from the video world

7

8

Dual-Frequency Identification Dual-Frequency Identification Sonar (DIDSON)Sonar (DIDSON)

DIDSON problemsDIDSON problems

Low snr,

Low dynamic span,

Not calibrated,

Not veldefined sample volume

Only x,z, but no y position information

9

10

DIDSON insideDIDSON inside

11

Examples of dataExamples of data

12

CFD AND CFD AND DIDSONDIDSON

Tracking

Echogram approach3D approach

Detection methods

Aim, material and methods

Introduction

Conclusion

Detection theory - methodsDetection theory - methods

Edgebased Gradient operators Linking Edge

Thresholding Constant, Addaptive,

Stastistical Relaxation

If this is a fish pixel, then…

13

Cross Filter Detector (CFD) Cross Filter Detector (CFD) aFilter 1

Variancec

Comparator

Filter 2 bEvaluator Traces

Signal a

Signal b

Signal c

Combine

Evaluator

Filter direction

CFD –Addaptive thresholdingCFD –Addaptive thresholdingMain challenge: Find the optimal threshold

signal

threshold

Detection methodsDetection methods

16

Foreground filter

Background filter

Comparator

variance

Evaluator

Background

Modelling

Comparator EvaluatorVideo

Echogram

Crossfilter detector

Common video processing

How to fit the Crossfilter to video like data?Can we learn something from the video world?

Background modelling. Background modelling. – the most important part. – the most important part.

Recursive Approximated median

Kalmann filter

Mixture of Gausians

Non recursive Previous picture Median Linear predictive Nonparametric

Background

Modelling

Comparator EvaluatorVideo

Common video processing

Background modelling. Background modelling. – the most important part. – the most important part.

Three best

1 Mixture of Gausians

2 Median

3 Approximated median

18

Ching , Cheung and Kamath found

Not much difference App. Median much faster

and simpler than the others

Sen-Ching S. Cheung and Chandrika Kamath Center for Applied Scientic Computing Lawrence Livermore National Laboratory, Livermore, CA 94550

ComparatorComparator

19

Background

Modelling

Comparator EvaluatorVideo

Common video processing

EvaluatorEvaluator

Morfological filter Recognise fish on size and shape May use higher order statistics Connect parts of targets

20

Background

Modelling

Comparator EvaluatorVideo

Common video processing

21

CFD AND CFD AND DIDSONDIDSON

Tracking

3D approach

Detection methods

Echogram approach

Introduction

Summary

Inc.Video methods

22

Echogram approachEchogram approach

AmplitudeDetector

Gain96-Ch

Multi beam-viewer

Amp-Echogram

Multi 1 beamEchogram generator

23

Generate echograms and apply the Generate echograms and apply the Cross-FilterCross-Filter

a) Mean echogram At each range bin extract mean values from a selected number

of beams. Like an ordinary transducer with controllable opening angle

b) Max Intensity At each range bin, select the sample from the beam with

highest intensity

How to combine many beams into one ?

24

Generating Echograms from multi beamGenerating Echograms from multi beam

Data recorded by Debby Burwen

a) Averaging a number of beams 10x12 deg b) Pick the beam with strongest intensity

Many beams 1 beam

25

Testing the CFD on many to 1 Testing the CFD on many to 1 beam echogramsbeam echograms

Echogram approach

26

Echogram approach works well Echogram approach works well until density becomes too highuntil density becomes too high

We want to push the density limit

Echogram approach

27

CFD AND CFD AND DIDSONDIDSON

Tracking

Echogram approach3D approach

The original Cross filter

Aim, material and methods

Introduction

Summary

28

Adding a third dimensionAdding a third dimension Work directly on the multi beam data

Want to detect more than one target in the same range bin

3d-trace2d-trace

time

time

width

rangerange

3D approach

29

We added the beam dimension to We added the beam dimension to the filters the filters

DDF

New

Running window operators

2D 3D

Beam. nr

Range

Ping

Ping

Range

3D approach

Test foreground Test foreground filterfilter

FrameBeam Range

1

1

1

35

35

35

operator size

1

Test Background Test Background filterfilter

FrameBeam Range 5

1

1

1525

operator size

32

Testing cross filter on a small Testing cross filter on a small trout in Fisha Rivertrout in Fisha River

Max Intensity echogram

CFD with filters CFD with filters and thresholdand threshold

Forefilt 3 x 3 x 3Back filt 3 x 3 x 3

Threshold Offset=20

34

Evaluator can take away Evaluator can take away unwanted targetsunwanted targets

35

36

CFD AND CFD AND DIDSONDIDSON

Tracking

3D approach

Detection methods

Echogram approach

Introduction

Summary

Inc.Video methods

37

Extended the background filter Extended the background filter with an approximated median with an approximated median operatoroperator

(N. McFarlane and C. Schoeld 1995)

ddfQ

1

1

BRBRFBR

BRBRFBR

AMthenAMQIf

AMthenAMQIf

38

And extended the And extended the comparator with comparator with

alternatives alternatives

Background

Foreground If ( a - b )>T )a

b

detectionThreshold

Background Background subtractionsubtraction

Forefilt 3 x 3 x 3Back filt 3 x 3 x 3

App.MedianThreshold Offset=20

40

CFD AND CFD AND DIDSONDIDSON

Tracking

3D approach

Detection methods

Echogram approach

Introduction

Summary

Inc.Video methods

41

The initial idea was to detect The initial idea was to detect traces directly by clustering traces directly by clustering

Cluster of overlapping fish pictures

( Work well in the echogram approach )

42

But data often showed traces But data often showed traces split up in individual fish picturessplit up in individual fish pictures

Clustering worked for big slow fish

Tracker needed for fast fish

Center of gravity

track

43

Special predictor can be made Special predictor can be made for multi beam datafor multi beam data

Special predictor can be formed from the DIDSON fish picture

In addition to traditional predictors are available such as Alpha Beta and Kalman

Fish center line

predictor

44

CFD AND CFD AND DIDSONDIDSON

Tracking

3D approach

Detection methods

Echogram approach

Introduction

Summary

Inc.Video methods

SummarySummary

45

Background

Modelling

Comparator EvaluatorVideo

Common video processing

Foreground filter

Background filter

Comparator

variance

EvaluatorEchogram

Crossfilter detector

DIDSON

Best method

Tracker3D-Foreground filter Comparator Evaluator

Background

Modelling

SummarySummary

46

DIDSON

Best method for moving targets

Tracker3D-Foreground filter

Background

Modelling

Comparator Evaluator

Needed in most cases Need for various predictors

depending on data

Improved foreground signal

Approximated Median

( a - b )>T )

a

b

3D better than 2DOptimise on improving foreground

Run demo now if timeRun demo now if time

47

And that was it! Thanks for And that was it! Thanks for the attention! Questions? the attention! Questions?

48

CFD AND CFD AND DIDSONDIDSON

Tracking

3D approach

Detection methods

Echogram approach

Introduction

Summary

Inc.Video methods