Embed Size (px)
Citation preview
Dave Mellinger, Sharon Nieukirk, Sara Heimlich, Curtis Lending,
Liz Ferguson, Tom Norris
Detection Workshop
Exploring automatic detection
capabilities with Ishmael
Contents Features slide No. Automatic Detection: review 3 - 8 Introduction to Ishmael 9,10 Open and view a sound file in Ishmael 11 Spectrogram and waveform 11 Equalization & Normalization 12 Sound Playback & Saving sound 13 Preferences files: load and save 14 Multi-channel files 15 Unknown file formats 16 Acoustic Measurements 17 Converting sound file formats 18 Filtering sound files- getting rid of noise 19 Principles of Detection: Detection and Classification- definitions and differences 20 - 24 Detection Thresholds 25 Detector: Energy Sum Detector 26 Improving the detection function: spectrogram effects 27 Detector: Spectrogram correlation 28 - 31 Detections: Manual verification 32 Detector: Matched Filter 33 Detection: Repetitive calls 34,35 Downloading detectors 36 Detector: Whistle and Moan 37 Performance Evaluation: ROC and DET curves 38 - 43 Matlab interface with Ishmael 44 Localization 45 - 50
Why?
• basic research – understanding animals
• management / mitigation
• Endangered Species Act
• Marine Mammal Protection Act
• National Environmental Policy Act
What is measured?
• population
• absolute, relative, or trend
• distribution
• daily, seasonal, year-to-year
• movement
Automatic detection: Why?
Most acoustic surveys are done by manual spectrogram scans
looking for individual calls or examining long-term spectral averages
Data volumes are now too large for manual scanning
• many groups are using autonomous acoustic instruments
• fixed moorings and mobile vehicles
• cabled arrays: CTBTO, ocean observing
systems (OOS), SOSUS
Automated methods becoming commonplace
Automatic detection: Why?
Detection is one step of a much larger process for
learning about marine (or terrestrial) life
• Establishing study/monitoring goals
• Study/monitoring design
• Data collection
• instrument preparation, deployment, operation,
recovery
• data curation: quality assurance, archiving,
dissemination
Data → Information → Knowledge → Wisdom
Automatic Detection in Context
Detection is one step of a much larger process for
learning about marine (or terrestrial) life
• Data analysis: from data to information
• data exploration
• species/call determination
• levels of analysis
• species
• call type
• other groups…
Automatic Detection in Context
Data → Information → Knowledge → Wisdom
Detection is one step of a much larger process for
learning about marine (or terrestrial) life
• Data analysis: from data to information
• data processing – automatic detection
• checking of results
• performance analysis
• summary statistics
• comparative statistics
Automatic Detection in Context
Data → Information → Knowledge → Wisdom
Detection is one step of a much larger process for
learning about marine (or terrestrial) life
• Putting information in context
• animal life history
• animal population(s)
• impacts, natural and human
• Population Consequences of Disturbance (PCOD)
Automatic Detection in Context
Data → Information → Knowledge → Wisdom
What is Ishmael? A tool for analyzing sound
• viewing
• recording
• measurement
• call detection & verification
• localization
• annotation
Introduction to Ishmael
Recording
Viewing
Automatic
recognition
Localization
& tracking Ishmael
Real-time input Sound file input
Beamforming
0101010
1001000
1010101
0111101
0101010
0001000
1010101
0101101
0101010
1001000
1010101
0111101
) ) )
Orca1.wav
Orca2.wav
Orca3.wav
• • •
Annotation log
Sound clips song note at 10:53:01
weird noise at 10:55:08
song note at 10:57:20
song note at 10:58:31
song note at 10:59:45
finished for today; restart at
11:00:00
song note at 11:00:57
song note at 11:02:10
song note at 11:03:22
song note at 11:04:37
. . . .
Measurement
Open and view a sound file
• test.wav – spectrogram and waveform
• brightness, contrast
• vertical and horizontal scaling
• spectrogram parameters
• color
Using Ishmael
Equalization / normalization
• MBay 05190920.wav
• equalization, floor, ceiling
• readjust brightness and contrast
Color
Using Ishmael
Sound playback
• MBay 05190920.wav
• select & play
• change playback rate
Saving sound
• select & save
Now you try it
• energy sum\Fin-93-001-1217.ch04.wav
Using Ishmael
Preferences files
• full load/save
• partial save
• default preferences (IshDefault.ipf)
• .ipf as text file
Using Ishmael
• Multi-channel files
• 4channel-DB-100709-090600.wav
• channels to read in
• channels to view
• channels for detection
• channels to save
Using Ishmael
Unknown file formats
• 00016945.DAT
• unknown file extensions
• can handle wide variety of files
• but not files with “records”
Using Ishmael
• Acoustic measurements
• making measurements
• choosing what to measure
• saving measurements
• noise-resistant features
• 1pAB1-Mellinger-FeatureExtractionInIshmael.pptx
• Mellinger and Bradbury 2007 in articles folder
Using Ishmael
Converting sound file formats
• Blue-wh-NEP-etp-ne-00095-1500-short.wav
• convert to .aif
Using Ishmael
Filtering
• to eliminate noise
• test.wav [500 Hz]
• decimation to change sample rate [x2]
• energy sum\[email protected]
Using Ishmael
Detection: finding potential sounds of interest in a signal
• often operates on a continuous signal
Classification: assigning these sounds to categories
• often operates on discrete segments of sound
No firm difference
• many techniques do some of both
The focus in Ishmael is on detection (so far)
Principles of Detection
The decision criterion
• separates detections from non-detections, or one class from
another
• every detector/classifier has one or more
Principles of Detection
Detection example
• decision criterion is a threshold line
Principles of Detection
time, s
(a)
Examples of classifiers
Principles of Detection
Decision criterion:
nearness to the cluster
centers
Decision criterion:
position on the
projection line
Multi-class example
Principles of Detection
x x x x
x
x x x
o o o
o
o
o + + +
+ +
+
* *
* * * * * * * * *
+
Factor 1
Fa
cto
r 2
x x
x x
* o
o
o
o
Decision criterion: position with respect to
the class separation lines
Detection: Thresholds
• higher thresholds give fewer wrong detections but more missed
calls
• vice versa for lower thresholds
Choice of threshold depends on task
• finding rare species low threshold
• don’t miss any calls
• index of abundance for common species high threshold
• few wrong detections; accurate index
• sometimes it depends on how much time you have to check
results
Principles of Detection
Detection: Energy sum
• s010723-141142L-onewhale.wav
• spectrogram [512]
• energy sum [1.5-11 kHz]
• threshold
• edit the action – logging
• Fin-93-001-1217.ch04.wav
Using Ishmael
Detection: Improving the detection function
• Detection neighborhood [1.5 s]
• Max/min duration (@ 33 s) [0.1 – 4 s]
• Smoothing [0.5 s]
• Sharpening
• s010723-141142L-onewhale.wav
Using Ishmael
Detection: Spectrogram correlation
• what is it?
Using Ishmael
time, s
de
tec
tio
n s
co
re
*
=
fre
qu
en
cy,
Hz
Detection: Spectrogram correlation
• BlueDemo-etp-ne-00069-1046+NOISE.wav
• 2 Hz contour, [0->9, 51->48.5]
• saving calls
• time before/after [10 10] or [5 20]
• logging
• BlueDemo-etp-ne-00069-1046.wav
Using Ishmael
Detection: Manual verification
• blue whale detector, save detections as files
• BlueDemo-etp-ne-00069-1046+NOISE.wav
Using Ishmael
Detection: matched filter
• cross-correlation w/kernel
• matched filter\BlueWhale-etp-ne-00069-1046.wav
• and kernel
• natural vs. synthetic kernel
• kernel CuviersDetector-MatchedFiltKernel-v3.wav
Using Ishmael
Detection: repetitive calls
• how does it work?
• Mellinger-AsaFall05-RegularSeqDet-v2.ppt
Using Ishmael
Detection: repetitive calls
• repeating calls\cuskeel_ptown_1.wav
• energy detector [1-2 kHz]
• equalization [3 s]
• regular sequences [0.01-0.3 s, 1 s window]
• Minke-93-001-2321.ch13.wav
Using Ishmael
Detection: downloading detectors
• fin whale - Atlantic
• Fin-080103-235959-obs16H.wav
• minke whale – Pacific (boing)
• minke_DCL5_locT123\27Apr09_174921_026_p1.wav
Using Ishmael
Detection: Whistles and Moans
• how does it work?
• Mellinger-Whistle Detection-DCL 09 Pavia.ppt
• tonals\s2k-000629-214906-dolphin.wav
• Beluga.wav
• 27Apr09_174921_026_p1-minke.wav
Using Ishmael
How well is a detector is doing?
• Compare to human detections (“ground truth”)
• Compare to other detectors
Performance always depends on
• noise level
• more generally, environmental conditions
Performance is always evaluated on a certain data set
• might be different for other data sets, other times/dates, other
noise environments
Performance evaluation
How well is a detector is doing?
• Compare to human detections (“ground truth”)
• Compare to other detectors
Performance always depends on
• noise level
• more generally, environmental conditions
Performance is always evaluated on a certain data set
• might be different for other data sets, other times/dates, other
noise environments
Performance evaluation
Performance evaluation
Receiver Operating Characteristic (ROC) curve
Performance evaluation
False positive rate
Tru
e p
ositiv
e r
ate
Detection Error Tradeoff (DET) curve
Performance evaluation
False positive rate (%)
Fals
e n
egative r
ate
(%
)
Detection: Performance evaluation
• Minke v1.ipf
• batch run minke_DCL5_locT123, make MinkeBoingDet09.log
• MATLAB
• go to ROC_DET_Matlab_code
• detector_eval.m
Using Ishmael
Detection: MATLAB interface
• MATLAB
• cd plugins/Energy Sum, startup
• Ishmael: Stubb->Energy Sum
• GPL detector demo
Using Ishmael
Background:
• Mellinger-Signal Processing.ppt
Dolphin whistle localization
• open “dolphin whistle @ 20,20.wav”
• load array file “dolphin whistle @ 20,20.arr”
• can also look at this file in Notepad
• make a spectrogram
• select part of the whistle
• click “hyperbolic location” button
• hydrophones are red dots
• location is a white dot
Localization
Localization: what happens after you locate something?
• can send it to whale tracking software
• WhalTrak (Jay Barlow)
• WhaleTrack II (Glenn Gailey)
• WILD (Whale Identification, Logging, and Display – Chris
Kyburg)
Localization
Tracking:
WILD
Localization
Tracking: WhalTrak
Localization
Sperm whale click bearings:
• open “sperm2001-010723-153000.wav”
• Localize->Loc options, load array file “JGordonArray.arr”
• check “Show results graphically”, uncheck “intermediate results”
• make a spectrogram, set up equalization
• make a detector, set a threshold
• Detect->Detection options->Saving calls (tab)
• set “before start of call” and “after end of call” to 0
• edit the action, click “calculate phone pair bearing”
Localization
Detection: Your own dataset!
… or use a data set from the file server.
Using Ishmael
