View
27
Download
1
Category
Tags:
Preview:
DESCRIPTION
6 New England Executive Park, Burlington, MA 01803 781-273-3388 3811 N. Fairfax Dr., Arlington, VA 22203 703-524-6263 4445 Eastgate Mall, San Diego, CA 92121 858-812-7874. A. L. P. H. A. T. E. C. H. ,. I. n. c. - PowerPoint PPT Presentation
Citation preview
ALPHATECH, Inc.
Comparison of Support Vector Machines and Multilayer Perceptron Networks in Building Mine Classification Models
Dr. Martin G. Bello
ALPHATECH Inc.
6 New England Executive Park
Burlington, Mass. 01803
August 29, 2003
(Research Funded by ONR as Part of the 6.2 MCM Program Element, with Associated Technical Agent: NSWC Coastal Systems Station, Panama City, FL.)
6 New England Executive Park, Burlington, MA 01803 781-273-33883811 N. Fairfax Dr., Arlington, VA 22203 703-524-6263
4445 Eastgate Mall, San Diego, CA 92121 858-812-7874
3811 N. Fairfax Dr., Arlington VA 22203 703-524-62636 New England Executive Park, Burlington MA 01803 781-273-3388
ALPHATECH, Inc.
4445 Eastgate Mall, San Diego CA 92121 858-812-7874
PRESENTATION OVERVIEW
• Mine Countermeasures Overview
• Mine Hunting Algorithm Structure
• Overview of Multilayer Perceptron, Support Vector Machine Classifier Construction Methodologies
• Alternative Classifier Construction Performance Result Comparisons
• Conclusions
3811 N. Fairfax Dr., Arlington VA 22203 703-524-62636 New England Executive Park, Burlington MA 01803 781-273-3388
ALPHATECH, Inc.
4445 Eastgate Mall, San Diego CA 92121 858-812-7874
Mine Countermeasures Overview
SONAR MISSION TAPE
NEW POST MISSION ANALYSIS CONCEPT-ICoastal Systems Station
PMA 2000
Bluefin BPAUV / Klein Sonar
WHOI / REMUS UUV & MSTL Sonar
CSSCAD/CAC
AlphatechCAD/CAC
LockheedCAD/CAC
Display,Analysis,
& Fusion ofContacts
PAYOFF OF FUSING MULTIPLE CAD / CAC ALGORITHMS
• REDUCED FALSE ALARM RATES• ENVIROMENTALLY ROBUST• DIVERSE ALGORITHMS HAVE FEW FALSE ALARMS IN COMMON
SONAR MISSION TAPE
MULTIPLE COMPUTER-AIDED DETECTION & CLASSIFICATION (CAD / CAC) ALGORITHMS
OBJECTIVEINCREASE SPEED & ROBUSTNESSOF POST MISSION ANALYSIS
NEW POST MISSION ANALYSIS CONCEPT-IICoastal Systems Station
PMA 2000
RaytheonCAD/CAC
Display,Analysis,
& Fusion ofContacts
NEW POST MISSION ANALYSIS CONCEPT-IIICoastal Systems Station
PMA 2000
SonarData
CSSCAD/CAC
LockheedCAD/CAC
OperatorMarks
AlphatechCAD/CAC
Bluefin BPAUV / Klein sonar
Mine-Like Objects (MLO’s)
3811 N. Fairfax Dr., Arlington VA 22203 703-524-62636 New England Executive Park, Burlington MA 01803 781-273-3388
ALPHATECH, Inc.
4445 Eastgate Mall, San Diego CA 92121 858-812-7874
AGGREGATE DETECTION/CLASSIFICATION ALGORITHM STRUCTURE
• Normalization Algorithm Enforces More Uniform “Local” Background
• Anomaly Screening Extracts Blobs/Tokens Corresponding to Mine-Like(ML) Target Candidates
• Features are “Local” Functionals of Image Calculated for each Blob/Token
• Feature Vector Multilayer Perceptron Neural NetworkLog-Likelihood Calculation or Alternatively Feature VectorSVM score calculation
Input Side-Scan SonarImagery
Image NormalizationAlgorithm
AnomalyScreeningAlgorithm
Feature Calculationfor
Screener Tokens
Token Log-Likelihood
RatioCalculation
TokenRanking/Thresholding
3811 N. Fairfax Dr., Arlington VA 22203 703-524-62636 New England Executive Park, Burlington MA 01803 781-273-3388
ALPHATECH, Inc.
4445 Eastgate Mall, San Diego CA 92121 858-812-7874
ANOMALY-SCREENING ALGORITHM STRUCTURE
• Anomaly Statistic Quantifies Deviation from “Local” Background Characteristics
• (Distinct Highlight(H) and Highlight/Shadow(HS) and Shadow(S) Contrast Statistics Have Been Conceived)
• MP, PC = Blob Anomaly Statistic Maximum Intensity and Pixel Count (PC)
• rMP, rPC= Ranks Associated with MP, PC
• Blob Filtering Identifies Candidate ML-Tokens
• Current Screening Algorithm Employs both H, HS, and S Based Segmentation Statistics, Deriving the Final Collection of Screened Tokens as those HS-blobs which Intersect either a H- or S-blob
AnomalyStatistic
CalculationAlgorithm
Image Histograming,Extraction of Top p% Pixels,
and Region Labeling
Calculation of MP,PC Featuresand Associated
Ranks rMP,rPC for eachBlob/Token
Blob/Token FilteringBased on Size andAggregate Rank
Statistic min(rMP,rPC)
3811 N. Fairfax Dr., Arlington VA 22203 703-524-62636 New England Executive Park, Burlington MA 01803 781-273-3388
ALPHATECH, Inc.
4445 Eastgate Mall, San Diego CA 92121 858-812-7874
BASELINE CLASSIFICATION FEATURE VECTOR -f DEFINITIONS-I
• f1= PC for HS-segmentation
• f2= min(rPC, rMP) for HS-segmentation
• f3= “Local” Blob Pixel Count for HS-segmentation
• f4= Mean Blob Anomaly Statistic Intensity for HS-segmentation
• f5= Standard Deviation of Blob Anomaly Statistic Intensity for HS-segmentation
• f6= “Local” Blob Count for HS-segmentation
• f7= MP for HS-segmentation
• f8 = (1,0) Valued Indicator for Existence of Intersecting H-segmentation Blob
• f9 = MP for Intersecting H-Segmentation Blob
• f10 = PC for Intersecting H-Segmentation Blob
• f11 = (1,0) Valued Indicator for Existence of Intersecting S-segmentation Blob
• f12 = PC for Intersecting S-Segmentation Blob
Original Feature Set-1992
Highlight Related
Shadow Related
3811 N. Fairfax Dr., Arlington VA 22203 703-524-62636 New England Executive Park, Burlington MA 01803 781-273-3388
ALPHATECH, Inc.
4445 Eastgate Mall, San Diego CA 92121 858-812-7874
BASELINE CLASSIFICATION FEATURE VECTOR -f DEFINITIONS-II
• f13 = Mean Normalized Image Over HS-Segmentation Blob
• f14 = Maximum Normalized Image Over HS-Segmentation Blob
• f15 = Standard Deviation of Normalized Image Over HS-segmentation Blob
• f16 = Skewness Coefficient of Normalized Image over HS-Segmentation Blob
• f17 = Kurtosis Coefficient of Normalized Image Over HS-Segmentation Blob
• f18 = Perimeter of HS-Segmentation Blob
• f19 = (16*PC)/(Perimeter*Perimeter) for HS-Segmentation Blob
• f20 = Perimeter/(2*(Bounding-box-width + Bounding-box-height)) for HS-Segmentation Blob
• f21 = (Major-axis – Minor-axis)/ (Major-axis + Minor-axis) for HS-Segmentation Blob
• f22 = Major-axis
• f23 = Orientation of HS-Segmentation Blob
Statistical Intensity Distribution Related
Shape Related
3811 N. Fairfax Dr., Arlington VA 22203 703-524-62636 New England Executive Park, Burlington MA 01803 781-273-3388
ALPHATECH, Inc.
4445 Eastgate Mall, San Diego CA 92121 858-812-7874
DISCRETE COSINE TRANSFORM AND PSEUDO-ZERNIKE MOMENT FEATURE DEFINITION
• Discrete Cosine Transform(DCT) Features Defined on Window Centered on HS-Segmentation Blob,
is a vector of quantities obtained by stacking rows of the below defined matrix…
1
0
1
0)/)(*)(*2(),(
Q
m
Q
nQvcucvuG
))*2/(**)1*2cos((*))*2/(**)1*2cos((*),( QvnQumnmg
• Pseudo-Zernike Moment Features(PZM) Defined on Window Centered on HS-Segmentation Blob…
pqMp
rdrdrgrp
D
iq
qpqp eSZ
0,...0
),()()1(
,,
)()( ,,22
, ZZ iqp
rqpqp
)0(),0(,1,,1,1,,,
pqforpqforqpqpqpqpqpqpPZM
f
fDCT
3811 N. Fairfax Dr., Arlington VA 22203 703-524-62636 New England Executive Park, Burlington MA 01803 781-273-3388
ALPHATECH, Inc.
4445 Eastgate Mall, San Diego CA 92121 858-812-7874
OVERVIEW OF COMPARED CLASSIFIER CONSTRUCTION STRATEGIES-I
• Traditional Classifier Construction first involves a “Feature Selection” stage where an Information Theoretic Measure, or actual Discrimination Performance of a simple classifier are optimized
• Multilayer Perceptron(MLP) Based Training Algorithm Optimizes Mutual Information Between Feature Vector and “True” Class Using a Recursive “Backpropagation-like” approach
• Cross-Validation Using a Test Set is Employed to Terminate Training when a specified objective function corresponding to the integral over a Test Set Derived Receiver Operating Characteristic Curve(ROC), is maximized
• The above steps may be repeated for on the order of 50-100 network optimizations to arrive at a “best” solution
3811 N. Fairfax Dr., Arlington VA 22203 703-524-62636 New England Executive Park, Burlington MA 01803 781-273-3388
ALPHATECH, Inc.
4445 Eastgate Mall, San Diego CA 92121 858-812-7874
OVERVIEW OF COMPARED CLASSIFIER CONSTRUCTION STRATEGIES-II
• Support Vector Machine(SVM) Implementation Adopted is SVMlight , developed by Professor Thorsten Joachims of Cornell University
• The Linear SVM “Soft-Margin” Training Formulation Is defined as a Quadratic Programming Problem, Optimizing a sum of two terms related to the squared norm of the classifier inner-product related parameter vector, and a weighted sum of “slack” variables related to miss-classification of training samples
• SVMlight employs an iterative “working-subset” strategy to solve the dual of the above described Quadratic Programming Problem, avoiding the excessive memory and time requirements of off the shelf Quadratic Programming Implementations, for large training data sets.
3811 N. Fairfax Dr., Arlington VA 22203 703-524-62636 New England Executive Park, Burlington MA 01803 781-273-3388
ALPHATECH, Inc.
4445 Eastgate Mall, San Diego CA 92121 858-812-7874
COMPARISON OF SVM AND MLP RESULTS-I
• 415F= Baseline 23F Set + 256F (DCT Transform Related) + 136F (PZM Related)
• 48F = Baseline 23F Set + 25F Selected from DCT, PZM Related Features using Genetic Algorithm Based Approach (NeuralWare Predict Algorithm)
• 6F= Feature Set Selected from 48F using Genetic Algorithm Based Approach (NeuralWare Predict Algorithm)
• 6F,48F, 23F Results Using MLP networks are superior to SVM based Results using Aggregate 415F Set
0.800.810.820.830.840.850.860.870.880.890.900.910.920.930.940.950.960.970.980.991.00
PC
D
0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0
FAI
Legend
PCD_415F_SVML
PCD_415F_SVML_U
PCD_415F_SVML_L
PCD_48F_MLP
PCD_6F_MLP
PCD_23F_MLP
Average False Alarms per Image
Low False Alarms Desired
3811 N. Fairfax Dr., Arlington VA 22203 703-524-62636 New England Executive Park, Burlington MA 01803 781-273-3388
ALPHATECH, Inc.
4445 Eastgate Mall, San Diego CA 92121 858-812-7874
COMPARISON OF SVM AND MLP RESULTS-II
• 6F,48F, 23F SVM Results are superior to SVM based Results using the Aggregate 415F Set
• Baseline 23F Result using MLP network classifier is the best
0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0
FAI
0.800.810.820.830.840.850.860.870.880.890.900.910.920.930.940.950.960.970.980.991.00
PCD
Legend
PCD_415F_SVML
PCD_415F_SVML_U
PCD_415F_SVML_L
PCD_48F_SVML
PCD_6F_SVML
PCD_23F_SVML
Average False Alarms per Image
3811 N. Fairfax Dr., Arlington VA 22203 703-524-62636 New England Executive Park, Burlington MA 01803 781-273-3388
ALPHATECH, Inc.
4445 Eastgate Mall, San Diego CA 92121 858-812-7874
CONCLUSIONS
• MLP and SVM based classifier construction strategies frequently achieve similar performance. In this study, the MLP approach more consistently resulted in the best performance for a feature set of limited size
• There is an advantage to employing the GA based feature selection technique first, as opposed to the blind use of an aggregate collection of features
• SVM Implementation Needs to be generalized to Incorporate Cross-Validation over Weighting Parameter Associated with Miss-Classification Terms
Recommended