Road Extraction fromRemote Sensing Images using SVM combined with FCM and MRF
Jiawei Xu MULTIMEDIA PROCESSING LABORATORYDepartment of Electronic Engineering, Hallym UniversityE-mail: [email protected]
Contents
Introduction A brief review of previous methods Multi-times SVM Proposed algorithm Experimental results Performance evaluation Conclusion
A brief review of previous methods
Mathematical morphology Hough transform P-value segmentation Genetic algorithm Markov random field
MATHEMATICAL MORPHOLOGY
Structure element: square, rectangle, ball, disk and line…Morphology: Erosion, dilation, open operator, close operator
Original image 2D median filter open operator erosion & thinning
Flowchart of the proposed algorithm in mathematical morphology
Hough transform
Theta (degree)
Pho (
pix
els
)
Accumulation Array from Hough Transform
0 20 40 60 80 100 120 140 160 180
-300
-200
-100
0
100
200
300
400
The Hough transform is most commonly used for the detection of regular curves such as lines, circles, ellipses, etc.Parameter setting in this thesis:Detection sensitivity: 0.15The smaller the value is, the more features in the image will be considered as lines.
P-value segmentation
This figure shows the entire procedure of road extraction using P value segmentation. In figure (a) is an input image with some selected road parts on it, (b) is the cumulative gray-image histogram distribution , (c) is the result of P value segmentation. Application of some morphological operators such as region property, open operator to (c) results in (d), (e) is the road net image using thinning function, (f) shows the input image with the extracted road net overlaid on it
P-value segmentation
Post processing incurs an information loss. (a) is the original image, (b) is the P value diagram, (c) is the result of P value segmentation and (d) is the application of some morphological operators, such as region property, open operator based on (c).
Compare with Genetic algorithm threshold segmentation
Best fitness and best threshold with generation increase
Markov random field
Mean value and variance of selected area
2)(
1
1
Sss
Sss
fS
fS
Original image MRF processing Speckle-like noise removal Road net
Linear SVMs
Support vector machine, alias maximum margin classifiers.
belong to a family of generalized linear classifiersminimize the empirical classification errormaximize the geometric margin
Φ: x → φ(x)
Conversion of Nonlinear Classifier to Linear Classifier via Mapping
Two classes(red dots for class 1,blue dots for class 2) can be separated by a circle, and it is not a linear classifier, which means SVM can not be used.
To slove such a problem is to import a nonlinear mapping function. If data sets in lower dimensional space are transformed to a higher dimensional space and successfully separated by a linear hyperplane as we showed in right figure ,we can utilize SVM.
linear optimal separating hyper-plane
nonlinear optimal separating hyper-plane
Multi-times SVM classification
1. Easy to be implemented, using LIBSVM2. Polynomial kernel, RBF kernel and Sigmoid kernel were
experimented but due to their similar results, here we only illustrate polynomial kernel function, the formula is as follows:
• Relatively lower accuracy• Complex post processing
( , ) ( ), ( ) (1 , )di j i j i jK x x x x x x
Fig.1:Pre-trained model is constituted by 20 circles from road and 20 asterisks from non-road, respectively.
Advantages:
Disadvantages:
Multi-times SVM( binary classifica-tion)
(a)RS image
(b) first-timeSVM
(c)Second-timeSVM
(d)Road extraction
Block diagram of our approach
FCM classifer
MRF regularizer
SVM classifer
RS image
Road image
FCM preprocessor
Partitioninto a collection of c fuzzy clusters with a list of c cluster centers V , such that
and a partition matrix
where is a numerical value in [0,1] that tells the degree to which the element belongs to the i-th cluster.
1 2,...,{ , }nX x x x
, 1, 2,...,iV v i c
, 1,..., , 1,..,ijU u i c j n
iju
jx
1. Calculate the fuzzy cluster centers
by using and the new partition matrix
by using
2. Update to
3. Stop iteration if otherwise set
and return to step 2
1 | 1, 2,...,iv i c
( )rU( 1)rU
1 | 1, 2,...,iv i c
( )rU ( 1)rU
( 1) ( ) 1maxij
r r r rij ijU U U u u
( 1) ( )r rU U 1r r
non-road-like non-road-like road-like image1 image 2 image 3
Algorithm
FCM processing result
(a)Original image(b)~(d)cluster 1,2,3
(a) (b)
(c) (d)(Road-like image used for further processing)
Clusters number 3
exponential weight 2
Maximum iterations 200
Termination value 1e-5
Why we use FCM before SVM?
Because of unbalanced dataIf class A(non-road) samples distributed over a
large areaClass B(road) samples distributed over a small
area,Supposing we use SVM directly, the coefficient Matrix will be more close to
the class that has a large distribution (which means hyper-plane is close to the large-distributed class because of the property of SVM)
An extreme case is : one-class SVM(to detect the outlier)If samples amounts is very few and distributed in an extremely small area,
SVM will recognize it as outlier,(in road extraction,for extreme case, road part will be neglected as the outlier)i.e., the hyperplane is unbal-ancedly assigned, which will lead to misclassification.
FCM can display the each cluster features more obviously,alternatively speaking, enlarge the difference/distance be-tween different clusters, which more or less decrease the misclassifica-tion rate.
Test Images & Reference Models to evaluate our performance
Performance evaluation criteria
Complete:
Correct:
Rank distance:
Quality :
TP
TP+FN100%
TP
TP+FP100%
2 2complete correct
2
TP
TP+FP+FN100%
True positive (TP): both the processed model and the reference scene model classify the pixel belonging to road.
True negative (TN): both the processed model and the reference scene model classify the pixel as belonging to the background.
False positive (FP): processed model classifies the pixel as belonging to road, but the reference scene model classifies the pixel as belonging to the background.
False negative (FN): the processed model classifies the pixel as belonging to the background, but the reference scene model classifies the pixel as belonging to road
Comparison between SVM and FCM+SVM
Beijing SVM Beijing FCM+SVM
Shanghai SVM Shanghai FCM+SVM
Vancouver SVM Vancouver FCM+SVM
0
10
20
30
40
50
60
70
80
90
Quality Percentage Rank distance
RS imagesBeijing.bmpShanghai.bmpVancouver.bmp
MRF regularizer
Initial temperature 5
isomorphic parameter 0.75
thermo regulation fac-tor
0.7
Ss
sfS
1
2)(
1
Ss
sfS
is the pixel set of the corresponding area, is the total number of the pixel set, is the intensity value of pixel, other values were decided empirically. iteration ,global E and temperature
S S
Selected area
Non-road part 1 86.2 153.6
Road part 2 153.2 236.9
Road part 3 162.3 248.4
Road part 4 157.8 241.5
Mean value:
Variance:
sf
Experimental results
We intentionally selected RS images with different characteristics: Beijing, shanghai, Vancouver… (ALL images from http://maps.google.com/)original images FCM-SVM processing MRF regularizer output images
Multi-classes SVM(one against all)
RS image road-class image lake-class image
1. expectation value initialization2. classification of samples to be recognized by SVM3. plot result image
Rank distance of K-means, SVM and our method
(%)
We do not list out morphology, Hough transform…because rank distance, quality percentage and other values are much lower than these approaches
Quality percentage of K-means, SVM and proposed method
(%)
Comparison of FCM+K-means with FCM+SVM
Comparison of FCM+K-means with FCM+SVM (a) results of FCM clustering; (b) FCM followed by K-means clustring; (c) results of FCM followed by SVM.
(a)
(b) (c)
(b) (c)
(a)
Comparison of FCM+K-means with FCM+SVM
shanghai.bmp FCM+K-means shanghai.bmp FCM+SVM Vancouver.bmp FCM+K-means Vancouver.bmp FCM+SVM0
10
20
30
40
50
60
70
80
90
Quality percentage Rank distance
GUI IMPLEMENTATION FOR ROAD Extraction: example 1
Example 2
Example 3
Executable file contains DOS win-dow
Road.exe:Classified the functions two categories:One is for manual control methods and the other is for machine learning methodsMeanwhile, we can also conceal DOS window
Conclusion
This thesis has proposed a new road extraction method based on SVM classification combined with FCM clustering and MRF regularization.
In terms of rank distance, and quality percentage, the proposed method is superior to SVM, morphological approaches, Hough transforms and K-means.
Using FCM clustering to separate road-like cluster and the other clusters increases the SVM classification accuracy.
We used MRF regularization to remove speckle-like noise then we could extract the fine road net.
Experimental results with several images with various features show that the proposed method gives us a higher accuracy and strong robustness regardless of input characteristics.
Future research
Batch processing of remote sensing image
Fuzzy SVM application in RS images
One against all SVM computational duration reduc-
tion
ANN algorithm optimization
References
[1] Stefan Hinz, “Automatic extraction of urban road networks from multi-view aerial imagery”, Technische University 2003
[2] Vladmir Vapnik, ”Statistical Learning Theory”, JOHN WILELY & SONS, Inc.1998
[3] Curt H.Davis, “An integrated system for automatic road mapping from high-resolution multi-spectral satel-lite imagery by information fusion”, Elsevier Inc. 2004
[4] Yair moshe, “GUI with Matlab” Department of Electronic Engineering, Columbia University , May 2004.
[5] http://maps.google.com/
[6] Yang Li, “A new validity function for fuzzy clustering”, School of mathematical sciences, Beijing normal university, 2005
[6] Xu Yong and Shaoguang Zhou, “Markov random field for road extraction applications in remote sensing images”, Department of Surveying and Mapping Engineering, Hohai University, 2008
[7] David M.McKeown ,“Performance evaluation for automatic feature extraction ”, Computer Science Depart-ment, Carnegie Mellon University 2000
[8] Patrick Perez, “Markov random fields and images”, Campus Beauileu, 1999
[9] Drs. Trani and Rahka, “MATLAB Graphic user interfaces(GUI) computer applications in civil engineering ”, Spring, 2000
Other fields
From 2009.01 to 2009.12 OpenCV1.0+VC6.0 OpenGL+VC6.0 OpenCV2.0+VS2008 Java3d on Myeclipse 7.5
OpenCV1.0+VC6.0
HaarcascadeCascade: Stage1:
Classifier11:
Feature11
Classifier12:
Feature12 ... Stage2: Classifier21:
Feature21
CvHaarFeature, CvHaarClassifier, CvHaarStageClassifier, CvHaarClassifierCascade Boosted Haar ------Tree structure
Here, we import cvLoadHaarClassifierCascade
OpenGL+VC6.0
3D effect by using Anaglyph glasses
OpenCV2.0+Visual studio2008
Configuration(CMake setup…) Debug/Release Rebuild Options -> Projects and Solutions -> VC++ Directories Car plate recognition
Soft matting
Function: Image enhancementBoxfilter.m: Equivalent to the function: colfilt(imSrc, [2*r+1, 2*r+1], 'sliding', @sum) - But much fasterGuidefilter.m% - guidance image: I% - filtering input image: p (should be a gray-scale/single channel image)% - local window radius: r% - regularization parameter: eps
THE END
Q&A