Dave Mellinger, Sharon Nieukirk, Sara Heimlich, Curtis Lending,
Liz Ferguson, Tom Norris
Detection Workshop
Exploring automatic detection
capabilities with Ishmael
Contents Features slide No. Automatic Detection: review 3 - 8 Introduction to Ishmael 9,10 Open and view a sound file in Ishmael 11 Spectrogram and waveform 11 Equalization & Normalization 12 Sound Playback & Saving sound 13 Preferences files: load and save 14 Multi-channel files 15 Unknown file formats 16 Acoustic Measurements 17 Converting sound file formats 18 Filtering sound files- getting rid of noise 19 Principles of Detection: Detection and Classification- definitions and differences 20 - 24 Detection Thresholds 25 Detector: Energy Sum Detector 26 Improving the detection function: spectrogram effects 27 Detector: Spectrogram correlation 28 - 31 Detections: Manual verification 32 Detector: Matched Filter 33 Detection: Repetitive calls 34,35 Downloading detectors 36 Detector: Whistle and Moan 37 Performance Evaluation: ROC and DET curves 38 - 43 Matlab interface with Ishmael 44 Localization 45 - 50
Why?
• basic research – understanding animals
• management / mitigation
• Endangered Species Act
• Marine Mammal Protection Act
• National Environmental Policy Act
What is measured?
• population
• absolute, relative, or trend
• distribution
• daily, seasonal, year-to-year
• movement
Automatic detection: Why?
Most acoustic surveys are done by manual spectrogram scans
looking for individual calls or examining long-term spectral averages
Data volumes are now too large for manual scanning
• many groups are using autonomous acoustic instruments
• fixed moorings and mobile vehicles
• cabled arrays: CTBTO, ocean observing
systems (OOS), SOSUS
Automated methods becoming commonplace
Automatic detection: Why?
Detection is one step of a much larger process for
learning about marine (or terrestrial) life
• Establishing study/monitoring goals
• Study/monitoring design
• Data collection
• instrument preparation, deployment, operation,
recovery
• data curation: quality assurance, archiving,
dissemination
Data → Information → Knowledge → Wisdom
Automatic Detection in Context
Detection is one step of a much larger process for
learning about marine (or terrestrial) life
• Data analysis: from data to information
• data exploration
• species/call determination
• levels of analysis
• species
• call type
• other groups…
Automatic Detection in Context
Data → Information → Knowledge → Wisdom
Detection is one step of a much larger process for
learning about marine (or terrestrial) life
• Data analysis: from data to information
• data processing – automatic detection
• checking of results
• performance analysis
• summary statistics
• comparative statistics
Automatic Detection in Context
Data → Information → Knowledge → Wisdom
Detection is one step of a much larger process for
learning about marine (or terrestrial) life
• Putting information in context
• animal life history
• animal population(s)
• impacts, natural and human
• Population Consequences of Disturbance (PCOD)
Automatic Detection in Context
Data → Information → Knowledge → Wisdom
What is Ishmael? A tool for analyzing sound
• viewing
• recording
• measurement
• call detection & verification
• localization
• annotation
Introduction to Ishmael
Recording
Viewing
Automatic
recognition
Localization
& tracking Ishmael
Real-time input Sound file input
Beamforming
0101010
1001000
1010101
0111101
0101010
0001000
1010101
0101101
0101010
1001000
1010101
0111101
) ) )
Orca1.wav
Orca2.wav
Orca3.wav
• • •
Annotation log
Sound clips song note at 10:53:01
weird noise at 10:55:08
song note at 10:57:20
song note at 10:58:31
song note at 10:59:45
finished for today; restart at
11:00:00
song note at 11:00:57
song note at 11:02:10
song note at 11:03:22
song note at 11:04:37
. . . .
Measurement
Open and view a sound file
• test.wav – spectrogram and waveform
• brightness, contrast
• vertical and horizontal scaling
• spectrogram parameters
• color
Using Ishmael
Equalization / normalization
• MBay 05190920.wav
• equalization, floor, ceiling
• readjust brightness and contrast
Color
Using Ishmael
Sound playback
• MBay 05190920.wav
• select & play
• change playback rate
Saving sound
• select & save
Now you try it
• energy sum\Fin-93-001-1217.ch04.wav
Using Ishmael
Preferences files
• full load/save
• partial save
• default preferences (IshDefault.ipf)
• .ipf as text file
Using Ishmael
• Multi-channel files
• 4channel-DB-100709-090600.wav
• channels to read in
• channels to view
• channels for detection
• channels to save
Using Ishmael
Unknown file formats
• 00016945.DAT
• unknown file extensions
• can handle wide variety of files
• but not files with “records”
Using Ishmael
• Acoustic measurements
• making measurements
• choosing what to measure
• saving measurements
• noise-resistant features
• 1pAB1-Mellinger-FeatureExtractionInIshmael.pptx
• Mellinger and Bradbury 2007 in articles folder
Using Ishmael
Converting sound file formats
• Blue-wh-NEP-etp-ne-00095-1500-short.wav
• convert to .aif
Using Ishmael
Filtering
• to eliminate noise
• test.wav [500 Hz]
• decimation to change sample rate [x2]
• energy sum\[email protected]
Using Ishmael
Detection: finding potential sounds of interest in a signal
• often operates on a continuous signal
Classification: assigning these sounds to categories
• often operates on discrete segments of sound
No firm difference
• many techniques do some of both
The focus in Ishmael is on detection (so far)
Principles of Detection
The decision criterion
• separates detections from non-detections, or one class from
another
• every detector/classifier has one or more
Principles of Detection
Detection example
• decision criterion is a threshold line
Principles of Detection
time, s
(a)
Examples of classifiers
Principles of Detection
Decision criterion:
nearness to the cluster
centers
Decision criterion:
position on the
projection line
Multi-class example
Principles of Detection
x x x x
x
x x x
o o o
o
o
o + + +
+ +
+
* *
* * * * * * * * *
+
Factor 1
Fa
cto
r 2
x x
x x
* o
o
o
o
Decision criterion: position with respect to
the class separation lines
Detection: Thresholds
• higher thresholds give fewer wrong detections but more missed
calls
• vice versa for lower thresholds
Choice of threshold depends on task
• finding rare species low threshold
• don’t miss any calls
• index of abundance for common species high threshold
• few wrong detections; accurate index
• sometimes it depends on how much time you have to check
results
Principles of Detection
Detection: Energy sum
• s010723-141142L-onewhale.wav
• spectrogram [512]
• energy sum [1.5-11 kHz]
• threshold
• edit the action – logging
• Fin-93-001-1217.ch04.wav
Using Ishmael
Detection: Improving the detection function
• Detection neighborhood [1.5 s]
• Max/min duration (@ 33 s) [0.1 – 4 s]
• Smoothing [0.5 s]
• Sharpening
• s010723-141142L-onewhale.wav
Using Ishmael
Detection: Spectrogram correlation
• what is it?
Using Ishmael
time, s
de
tec
tio
n s
co
re
*
=
fre
qu
en
cy,
Hz
Detection: Spectrogram correlation
• BlueDemo-etp-ne-00069-1046+NOISE.wav
• 2 Hz contour, [0->9, 51->48.5]
• saving calls
• time before/after [10 10] or [5 20]
• logging
• BlueDemo-etp-ne-00069-1046.wav
Using Ishmael
Detection: Manual verification
• blue whale detector, save detections as files
• BlueDemo-etp-ne-00069-1046+NOISE.wav
Using Ishmael
Detection: matched filter
• cross-correlation w/kernel
• matched filter\BlueWhale-etp-ne-00069-1046.wav
• and kernel
• natural vs. synthetic kernel
• kernel CuviersDetector-MatchedFiltKernel-v3.wav
Using Ishmael
Detection: repetitive calls
• how does it work?
• Mellinger-AsaFall05-RegularSeqDet-v2.ppt
Using Ishmael
Detection: repetitive calls
• repeating calls\cuskeel_ptown_1.wav
• energy detector [1-2 kHz]
• equalization [3 s]
• regular sequences [0.01-0.3 s, 1 s window]
• Minke-93-001-2321.ch13.wav
Using Ishmael
Detection: downloading detectors
• fin whale - Atlantic
• Fin-080103-235959-obs16H.wav
• minke whale – Pacific (boing)
• minke_DCL5_locT123\27Apr09_174921_026_p1.wav
Using Ishmael
Detection: Whistles and Moans
• how does it work?
• Mellinger-Whistle Detection-DCL 09 Pavia.ppt
• tonals\s2k-000629-214906-dolphin.wav
• Beluga.wav
• 27Apr09_174921_026_p1-minke.wav
Using Ishmael
How well is a detector is doing?
• Compare to human detections (“ground truth”)
• Compare to other detectors
Performance always depends on
• noise level
• more generally, environmental conditions
Performance is always evaluated on a certain data set
• might be different for other data sets, other times/dates, other
noise environments
Performance evaluation
How well is a detector is doing?
• Compare to human detections (“ground truth”)
• Compare to other detectors
Performance always depends on
• noise level
• more generally, environmental conditions
Performance is always evaluated on a certain data set
• might be different for other data sets, other times/dates, other
noise environments
Performance evaluation
Performance evaluation
Receiver Operating Characteristic (ROC) curve
Performance evaluation
False positive rate
Tru
e p
ositiv
e r
ate
Detection Error Tradeoff (DET) curve
Performance evaluation
False positive rate (%)
Fals
e n
egative r
ate
(%
)
Detection: Performance evaluation
• Minke v1.ipf
• batch run minke_DCL5_locT123, make MinkeBoingDet09.log
• MATLAB
• go to ROC_DET_Matlab_code
• detector_eval.m
Using Ishmael
Detection: MATLAB interface
• MATLAB
• cd plugins/Energy Sum, startup
• Ishmael: Stubb->Energy Sum
• GPL detector demo
Using Ishmael
Background:
• Mellinger-Signal Processing.ppt
Dolphin whistle localization
• open “dolphin whistle @ 20,20.wav”
• load array file “dolphin whistle @ 20,20.arr”
• can also look at this file in Notepad
• make a spectrogram
• select part of the whistle
• click “hyperbolic location” button
• hydrophones are red dots
• location is a white dot
Localization
Localization: what happens after you locate something?
• can send it to whale tracking software
• WhalTrak (Jay Barlow)
• WhaleTrack II (Glenn Gailey)
• WILD (Whale Identification, Logging, and Display – Chris
Kyburg)
Localization
Tracking:
WILD
Localization
Tracking: WhalTrak
Localization
Sperm whale click bearings:
• open “sperm2001-010723-153000.wav”
• Localize->Loc options, load array file “JGordonArray.arr”
• check “Show results graphically”, uncheck “intermediate results”
• make a spectrogram, set up equalization
• make a detector, set a threshold
• Detect->Detection options->Saving calls (tab)
• set “before start of call” and “after end of call” to 0
• edit the action, click “calculate phone pair bearing”
Localization
Detection: Your own dataset!
… or use a data set from the file server.
Using Ishmael