Presentation for korea multimedia(in english)

Road Extraction fromRemote Sensing Images using SVM combined with FCM and MRF

Jiawei Xu MULTIMEDIA PROCESSING LABORATORYDepartment of Electronic Engineering, Hallym UniversityE-mail: abyssecho@hallym.ac.kr

Contents

Introduction A brief review of previous methods Multi-times SVM Proposed algorithm Experimental results Performance evaluation Conclusion

A brief review of previous methods

Mathematical morphology Hough transform P-value segmentation Genetic algorithm Markov random field

MATHEMATICAL MORPHOLOGY

Structure element： square, rectangle, ball, disk and line…Morphology: Erosion， dilation， open operator, close operator

Original image 2D median filter open operator erosion & thinning

Flowchart of the proposed algorithm in mathematical morphology

Hough transform

Theta (degree)

Pho (

pix

els

)

Accumulation Array from Hough Transform

0 20 40 60 80 100 120 140 160 180

-300

-200

-100

0

100

200

300

400

The Hough transform is most commonly used for the detection of regular curves such as lines, circles, ellipses, etc.Parameter setting in this thesis:Detection sensitivity: 0.15The smaller the value is, the more features in the image will be considered as lines.

P-value segmentation

This figure shows the entire procedure of road extraction using P value segmentation. In figure (a) is an input image with some selected road parts on it, (b) is the cumulative gray-image histogram distribution , (c) is the result of P value segmentation. Application of some morphological operators such as region property, open operator to (c) results in (d), (e) is the road net image using thinning function, (f) shows the input image with the extracted road net overlaid on it

P-value segmentation

Post processing incurs an information loss. (a) is the original image, (b) is the P value diagram, (c) is the result of P value segmentation and (d) is the application of some morphological operators, such as region property, open operator based on (c).

Compare with Genetic algorithm threshold segmentation

Best fitness and best threshold with generation increase

Markov random field

Mean value and variance of selected area

2)(

1

1

Sss

Sss

fS

fS

Original image MRF processing Speckle-like noise removal Road net

Linear SVMs

Support vector machine, alias maximum margin classifiers.

belong to a family of generalized linear classifiersminimize the empirical classification errormaximize the geometric margin

Φ: x → φ(x)

Conversion of Nonlinear Classifier to Linear Classifier via Mapping

Two classes(red dots for class 1,blue dots for class 2) can be separated by a circle, and it is not a linear classifier, which means SVM can not be used.

To slove such a problem is to import a nonlinear mapping function. If data sets in lower dimensional space are transformed to a higher dimensional space and successfully separated by a linear hyperplane as we showed in right figure ,we can utilize SVM.

linear optimal separating hyper-plane

nonlinear optimal separating hyper-plane

Multi-times SVM classification

1. Easy to be implemented, using LIBSVM2. Polynomial kernel, RBF kernel and Sigmoid kernel were

experimented but due to their similar results, here we only illustrate polynomial kernel function, the formula is as follows:

• Relatively lower accuracy• Complex post processing

( , ) ( ), ( ) (1 , )di j i j i jK x x x x x x

Fig.1:Pre-trained model is constituted by 20 circles from road and 20 asterisks from non-road, respectively.

Advantages:

Disadvantages:

Multi-times SVM（ binary classifica-tion）

(a)RS image

(b) first-timeSVM

(c)Second-timeSVM

(d)Road extraction

Block diagram of our approach

FCM classifer

MRF regularizer

SVM classifer

RS image

Road image

FCM preprocessor

Partitioninto a collection of c fuzzy clusters with a list of c cluster centers V , such that

and a partition matrix

where is a numerical value in [0,1] that tells the degree to which the element belongs to the i-th cluster.

1 2,...,{ , }nX x x x

, 1, 2,...,iV v i c

, 1,..., , 1,..,ijU u i c j n

iju

jx

1. Calculate the fuzzy cluster centers

by using and the new partition matrix

by using

2. Update to

3. Stop iteration if otherwise set

and return to step 2

1 | 1, 2,...,iv i c

( )rU( 1)rU

1 | 1, 2,...,iv i c

( )rU ( 1)rU

( 1) ( ) 1maxij

r r r rij ijU U U u u

( 1) ( )r rU U 1r r

non-road-like non-road-like road-like image1 image 2 image 3

Algorithm

FCM processing result

(a)Original image(b)~(d)cluster 1,2,3

(a) (b)

(c) (d)(Road-like image used for further processing)

Clusters number 3

exponential weight 2

Maximum iterations 200

Termination value 1e-5

Why we use FCM before SVM?

Because of unbalanced dataIf class A(non-road) samples distributed over a

large areaClass B(road) samples distributed over a small

area,Supposing we use SVM directly, the coefficient Matrix will be more close to

the class that has a large distribution (which means hyper-plane is close to the large-distributed class because of the property of SVM)

An extreme case is : one-class SVM(to detect the outlier)If samples amounts is very few and distributed in an extremely small area,

SVM will recognize it as outlier,(in road extraction,for extreme case, road part will be neglected as the outlier)i.e., the hyperplane is unbal-ancedly assigned, which will lead to misclassification.

FCM can display the each cluster features more obviously,alternatively speaking, enlarge the difference/distance be-tween different clusters, which more or less decrease the misclassifica-tion rate.

Test Images & Reference Models to evaluate our performance

Performance evaluation criteria

Complete:

Correct:

Rank distance:

Quality :

TP

TP+FN100%

TP

TP+FP100%

2 2complete correct

2

TP

TP+FP+FN100%

True positive (TP): both the processed model and the reference scene model classify the pixel belonging to road.

True negative (TN): both the processed model and the reference scene model classify the pixel as belonging to the background.

False positive (FP): processed model classifies the pixel as belonging to road, but the reference scene model classifies the pixel as belonging to the background.

False negative (FN): the processed model classifies the pixel as belonging to the background, but the reference scene model classifies the pixel as belonging to road

Comparison between SVM and FCM+SVM

Beijing SVM Beijing FCM+SVM

Shanghai SVM Shanghai FCM+SVM

Vancouver SVM Vancouver FCM+SVM

0

10

20

30

40

50

60

70

80

90

Quality Percentage Rank distance

RS imagesBeijing.bmpShanghai.bmpVancouver.bmp

MRF regularizer

Initial temperature 5

isomorphic parameter 0.75

thermo regulation fac-tor

0.7

Ss

sfS

1

2)(

1

Ss

sfS

is the pixel set of the corresponding area, is the total number of the pixel set, is the intensity value of pixel, other values were decided empirically. iteration ,global E and temperature

S S

Selected area

Non-road part 1 86.2 153.6

Road part 2 153.2 236.9

Road part 3 162.3 248.4

Road part 4 157.8 241.5

Mean value:

Variance:

sf

Experimental results

We intentionally selected RS images with different characteristics： Beijing， shanghai， Vancouver… (ALL images from http://maps.google.com/)original images FCM-SVM processing MRF regularizer output images

Multi-classes SVM(one against all)

RS image road-class image lake-class image

1. expectation value initialization2. classification of samples to be recognized by SVM3. plot result image

Rank distance of K-means, SVM and our method

(%)

We do not list out morphology, Hough transform…because rank distance, quality percentage and other values are much lower than these approaches

Quality percentage of K-means, SVM and proposed method

(%)

Comparison of FCM+K-means with FCM+SVM

Comparison of FCM+K-means with FCM+SVM (a) results of FCM clustering; (b) FCM followed by K-means clustring; (c) results of FCM followed by SVM.

(a)

(b) (c)

(b) (c)

(a)

Comparison of FCM+K-means with FCM+SVM

shanghai.bmp FCM+K-means shanghai.bmp FCM+SVM Vancouver.bmp FCM+K-means Vancouver.bmp FCM+SVM0

10

20

30

40

50

60

70

80

90

Quality percentage Rank distance

GUI IMPLEMENTATION FOR ROAD Extraction: example 1

Example 2

Example 3

Executable file contains DOS win-dow

Road.exe:Classified the functions two categories:One is for manual control methods and the other is for machine learning methodsMeanwhile, we can also conceal DOS window

Conclusion

This thesis has proposed a new road extraction method based on SVM classification combined with FCM clustering and MRF regularization.

In terms of rank distance, and quality percentage, the proposed method is superior to SVM, morphological approaches, Hough transforms and K-means.

Using FCM clustering to separate road-like cluster and the other clusters increases the SVM classification accuracy.

We used MRF regularization to remove speckle-like noise then we could extract the fine road net.

Experimental results with several images with various features show that the proposed method gives us a higher accuracy and strong robustness regardless of input characteristics.

Future research

Batch processing of remote sensing image

Fuzzy SVM application in RS images

One against all SVM computational duration reduc-

tion

ANN algorithm optimization

References

[1] Stefan Hinz, “Automatic extraction of urban road networks from multi-view aerial imagery”, Technische University 2003

[2] Vladmir Vapnik, ”Statistical Learning Theory”, JOHN WILELY & SONS, Inc.1998

[3] Curt H.Davis, “An integrated system for automatic road mapping from high-resolution multi-spectral satel-lite imagery by information fusion”, Elsevier Inc. 2004

[4] Yair moshe, “GUI with Matlab” Department of Electronic Engineering, Columbia University , May 2004.

[5] http://maps.google.com/

[6] Yang Li, “A new validity function for fuzzy clustering”, School of mathematical sciences, Beijing normal university, 2005

[6] Xu Yong and Shaoguang Zhou, “Markov random field for road extraction applications in remote sensing images”, Department of Surveying and Mapping Engineering, Hohai University, 2008

[7] David M.McKeown ,“Performance evaluation for automatic feature extraction ”, Computer Science Depart-ment, Carnegie Mellon University 2000

[8] Patrick Perez, “Markov random fields and images”, Campus Beauileu, 1999

[9] Drs. Trani and Rahka, “MATLAB Graphic user interfaces(GUI) computer applications in civil engineering ”, Spring, 2000

Other fields

From 2009.01 to 2009.12 OpenCV1.0+VC6.0 OpenGL+VC6.0 OpenCV2.0+VS2008 Java3d on Myeclipse 7.5

OpenCV1.0+VC6.0

HaarcascadeCascade: Stage1:

Classifier11:

Feature11

Classifier12:

Feature12 ... Stage2: Classifier21:

Feature21

CvHaarFeature, CvHaarClassifier, CvHaarStageClassifier, CvHaarClassifierCascade Boosted Haar ------Tree structure

Here, we import cvLoadHaarClassifierCascade

OpenGL+VC6.0

3D effect by using Anaglyph glasses

OpenCV2.0+Visual studio2008

Configuration(CMake setup…) Debug/Release Rebuild Options -> Projects and Solutions -> VC++ Directories Car plate recognition

Soft matting

Function: Image enhancementBoxfilter.m: Equivalent to the function: colfilt(imSrc, [2*r+1, 2*r+1], 'sliding', @sum) - But much fasterGuidefilter.m% - guidance image: I% - filtering input image: p (should be a gray-scale/single channel image)% - local window radius: r% - regularization parameter: eps

THE END

Q&A