8/13/2019 iscsct09p458

1/4

458

Image Semantic Classification Using SVM InImage Retrieval

Xiaohong Yu 1, and Hong Liu21 College of Computer Science & Information Enginerring , Zhejiang Gongshuang University

No.18,Xuezheng Str.,Xiasha University Town,Hangzhou, ChinaEmail:XHYU@mail.zjgsu.edu.cn

2 College of Computer Science & Information Enginerring , Zhejiang Gongshuang UniversityNo.18,Xuezheng Str.,Xiasha University Town,Hangzhou, China

Email: LLH @mail.hzic.edu.cn

Abstract There is a gap between low-level descriptions of

image content and the semantic understanding of users to

query image databases in the content-based image retrieval.

In this paper, we put forward a method of classifying imageregions hierarchically using their semantics and that

resembles peoples perception more than using low-level

features. The experiments show, the better precision of

semantic classification justifies the feasibility of our method.

It uses in image retrieval field further and get better index

effect.

Index Terms Image classification, semantic classification,

image retrieval, Super Vector Machine, keyword-based

retrieval

I.INTRODUCTION

The growth of the World Wide Web have led to the

huge online digital images and videos, so there is a strongdemand for developing an efficient technique for imageretrieval to exploit maximum benefit from this hugeamount of digital information. In traditional system, thekeywords of image in database are labeled manually andthen it utilizes text-based retrieval system to index theimage. As the image increases, this technique becomesvery inefficient and insufficient to describe the details ofan image, so the content-based image retrieval systemshave been the major subject for recent decades. Manyimages retrieval systems have finished, such as QBIC,Visual SEEK, Netra and MARS and so on. They indexand retrieval of image based on low-level features ofimage, such as color, texture and shape. Content-basedimage retrieval techniques based on similarity matchingof features. Take Color similarity for example, colorsimilarity of image can measure by pixel luminancematching, but pixel matching is highly sensitive to noiseand small distortions like rotation, further, it is reallytime-consuming. Most of above systems have theadvantage of being automatic, but they are hard to use fornovice because of the semantic gap that exists betweenuser perception and system requirements.

As a matter of fact, novice prefer to retrieval imageusing image semantic elements, such as land, sky,mountain, snow and grass, which are closer to theirperception than low-level features. If the system adopthierarchical semantic to organize and index the image,the gap between the low-level descriptions of image and

the users semantic needs reduce. That is, to reduce thesemantic gap, we need to classify image regions based ontheir semantics. Let novice queries desired images

intuitively.In this paper, we put forward a method of the

automatic hierarchical classification of image regions intoa more detailed classification hierarchy based on thesemantics of the region content by using SVM, and the

paper also give a experiment to prove the method that canperform well in the image retrieval field.

We organize this paper as follows. In Sec. 2, wedescribe the HSV color space that is more suitable forhuman perception. In Sec. 3, we pay attention to imagesegmentation, extraction of region features, and simplydiscuss how to build the SVM and classify the imageregions by using SVM. In Sec. 4, the experimental results

are presented and finally, the conclusions are given in Sec.5.

.HSVCOLOR SPACE

Although the process followed the human brain inperceiving and interpreting color is a psychologicalproblem that is not yet fully comprehended. The purposeof color model ( also call color space) is to facilitate thespecification of colors in some standard. In fact, a colorspace is a specification of a coordinate system where eachcolor is represented by a single point

Most color space in use today are oriented either

toward hardware or toward applications, the hardware-oriented space most commonly used in practice are theRGB (Red, Green, Blue) space, the HSV (Hue ,Saturation, Value) space is more suitable for human

perceive , so in this paper we use HSV space for studying.

We map the image into the HSV color space.

++=

+

+=

++=

)],,[min(

3

1

)])(()[(

)]()[(2

1

arccos

3

2/12

BGRBGR

s

BGBRGR

BRGR

H

BGRv

For each image, we find the minimum and maximumvalues of each of the three colors components to set up

ISBN 978-952-5726-07-7 (Print), 978-952-5726-08-4 (CD-ROM)Proceedings of the Second Symposium International Computer Science and Computational Technology(ISCSCT 09)

Huangshan, P. R. China, 26-28,Dec. 2009, pp. 458-461

2009 ACADEMY PUBLISHERAP-PROC-CS-09CN005

8/13/2019 iscsct09p458

2/4

459

the coordinate of the HSV space. Each axis runs fromminimum to maximum values. These values normalize sothat the minimum value equal zero and the maximumvalue is one. Then, the H, S and V components arequantized to 16, 8 and 8, respectively, within the

minimum-maximum range of each component. The Hcomponent quantize into more levels, as compared toboth S and V, to reflect the diversity of colors in theimage database. The values can change by user ifnecessary to suit a specific image collection.

.IMAGE CLASSIFICATION BY REGION-BASED ONSEMANTIC

A. Image segmentation

Image segmentation is a process of dividing an imageinto coherent, uncovered and significative regions.

Generic, complete and to the pixel accurate unsupervisedsegmentation regard as it is virtually impossible, so inthis paper, we just want to get a method to segment animage, which is satisfied with the following condition:

(1) First, the extracted regions are coherent.(2) Segmentation should give satisfactory results on

general image data without knowledge assumed.(3) Segmentation process should be unsupervised.In our experiment, we use hill-climbing method to

segment the image, which can be satisfied with abovecondition.

The hill-climbing algorithm summarizes as follows:(1) Compute the HSV color histogram of the image.

(2) Start at a non-zero bin of the color histogram andmake uphill moves until reaching a peak.(3) Choose another unclimbed bin and re-perform step

2 to find another peak. Repeat this step until all non-zerobins of the color histogram climbed.

(4) The peaks we get from above represent the firstnumber of clusters of the input image, and, these peakssaved.

(5) In the end, neighboring pixels that have same peakput together, that is associating every pixel with one ofthe identified peaks. Consequently, the segments of theinput image formed.

The segmentation results shows in Figure 1.

Figure 1. The result of Image segmentation

B. Extract feature of region

We use different way to extract the feature of region.1) color histogram in the HSV spaceIn our classification experiments, we found that we

could achieve better classification accuracy if werepresent the color content of each region with only the Hand S components. Thus, we eliminate the V component.A color histogram contains the H component, which is

quantized to 6 values, and the S component, which isquantized to 4 values. Hence, the color histogram isrepresented by a 24-dimensional vector.

2) Edge direction histogramEdge Direction Histogram of the image shape is one of

feature extraction methods, the algorithm extractedfeature vectors satisfies scale, translation and rotationchange.

There are many different types of edge detectoroperators. We use the popular Canny edge detector.Experiment proved that the method for a single

background, the shape characteristics of clear image withbetter research results.

C. Support Vector Machine(SVM) Classifier

1) Review of support vector classifiers theoryThe way of constructing a hyperplane to get binary

classifiers done that can separate members of one class

from others, but most real data hardly separate becausethe hyperplane that can successfully separate themembers of the two classes in most case does not exist.One measure to solve this problem is to map the data intoa higher dimensional space, where the members of thetwo classes can separate by a hyperplane. However, thetraditional classifier is not good at in high dimensionalvector. It is extremely expensive in terms of memory andtime.

Support Vector Machines can solve this problem.SVM avoid overfitting the data by choosing a hyperplanefrom the many that can separate the data. That maximizesthe minimum distance from the hyperplane to the closest

training point. Such a hyperplane call the maximummargin hyperplane. Another advantage of the SVM is thecompact representation of the decision boundary, so thenumber of support vectors is small as compared to thenumber of points in the training set.

In this, we simply introduce Support Vector Machinefor binary classification

The given training data set for binary classificationproblem is :

)},(),...,,),...(,{( 11 llii yxyxyx (3.1)

whered

i Rx and }1,1{iy are training patternvectors and their corresponding labels, and l indicates the

number of training pattern vectors.Let us also define a linear decision surface by the

equation0)( =+= bxwxf (3.2)

Where w is normal to the hyperplane, wb / is the

distance from the distance from the origin to thehyperplane.

If the following formulation exists:

11

11

=+

=+

i

i

ybxw

ybxw (3.3)

It means the training date set can be separated in linear.Using a nonlinear transform

, these pattern vectorsin Eq. (2.1) can be mapped from the original input space

dR into high dimensional feature space

nR , the

transform shown in Figure. 2.

8/13/2019 iscsct09p458

3/4

460

ndRxRx )( (3.4)

In the feature spacen

R , SVM aims at constructing alinear discriminant function of the form,

))(()( bxwsignxf += (3.5)

Where w and b imply the weight vector and threshold;and denotes the inner product.

According to structural risk minimization principle,SVM is to solve a problem as follows,

=

+l

i

i

TCww

12

1min (3.6)

li

bxwyts

i

ii

T

i

,,10

1))((..

=

+

Where C is the regularization parameter which cancontrol the tradeoff between the number of errors and thecomplexity of model, and the slack variable

1>i corresponds to some misclassified training sample.

=

= ==

=

=

=

=

l

i

ii

i

l

i

l

j

jijiji

l

i

i

jijiji

l

i

iD

ya

Cats

xxKyyaaa

xxyyaaaL

1

1 11

1

0

0..

),(2

1

)()(2

1

max

(3.7)

Where liai ,,1,0 = are Lagrangian multipliers to

solve. )()(),( jiji xxxxk = is kernel function, Some

of the classical SVM kernels are reported in Table 1.So, the discriminant function and parameter b are:

=

+=l

i

iii bxxKaysignxf1

)),()(~

(3.8)

=JNx Jx

ijjji

NSV i j

xxKyayN

b )),((1

Where NSVN is the number of standard support vectors,

JN is the set of standard support vectors, J is the set ofsupport vectors.

Figure 2. A linear hyperplane separating the members of two classes.The support vectors are circled which is technique used by most radial

basis function classifiers.2) Region ClassificationIn order to classify image regions into semantic classes

in which humans can understand easily, we manually

defined a hierarchy organization that reflects thesemantics in the Nature images and based on human

judgement subjectively, as shown in Figure. 3. Thehierarchy organization is not complete by itself, but it is areasonable organization to simplify image retrieval.

TABLE I.CLASSICAL COMMON KERNELS (A IN KMODIS A NORMALIZATION

CONSTANT

Kernel Formula

Linear yxyxk =),( Polynomial d

byaxyxk )(),( += RBF

)/exp(),( 22

yxyxk =

KMOD

)1)(exp(),(22

2

+

=

yxayxk

The selection of classes based on having general

concepts to give meaningful associations in normalcomprehension.

Figure 3. A class organization used the SVM to distinguish the membersof a class from others

3) Learning and classification stageLearning the semantics for each class through using

SVM based on different features of training sampleregions of each class. We get the SVM classifiers isP(classified | notMemberOfClass) .These binaryclassifiers achieve good class separation under theconstraint that each region belongs to only one, or none,of the classes.

After training the SVM, binary classifiers that canclassify image regions based on their semantics create.Then, we use these binary classifiers to classify ourdatabase of image regions leading to the classificationshown in Figure 3 to determine the class of an input

image region and this image region map into its class inthe semantic class hierarchy in Figure.3. In the semanticclass hierarchy, all scenery regions are classified into

Nature regions or Artificial regions. Since in this paperwe concentrate on Nature regions, thus we furtherclassify the Nature regions into three subclasses: Sky,Land and Water, Each one of these subclasses furtherdivided into sub-subclasses. The Sky subclass dividedinto Night, Sunset, Clouds and BlueSky. Next, the Watersubclass divided into Waterfall, BlueSea, WhiteWave andRiver. Then, the Land subclass divided into Mountain,Sand, Greenground, and Snow. The Greenground sub-subclass further divided into Grass and Forest.

Thus, each image can be represented by a set ofkeywords that are the name of class based on semanticclassification of image regions. The choice of keyword-

based method allow for highly intuitive query interface,

8/13/2019 iscsct09p458

4/4

461

so the novice can use the semantic to retrieval image bytheir understanding.

.EXPERIMENTS

We do experiments mainly on images testing set, suchas nature scenery, flowers, flags and winter about twothousands images. We divide each image into 5 regionson the average. We got a database with about tenthousands regions as a result of segmentation. Weselected 600 regions from above database as a training setfor training the SVM. That is about 50 images per class.Then, the extracting feather use color histogram and Edgedirection histogram in different classes.

To classify the image regions, we tried differentgrouping of classes and different features before wefinally decided on the grouping and features in Figure. 3,which gives the best classification precision. At is

experiment we performed the classification Natureregions and Artifical regions using EDH to extract thefeature of region. Then we tried to group the Natureimage regions into 3 classes using color histogram featurebecause it gets high precision than using EDH. Finally,we use EDH feature to group the Water image regionsinto 4 classes. We get the experimental result shown inFigure. 4 when the user input the keyword waterfall.

Figure 4. Result of the a query of waterfall

.CONCLUSION

We put forward a method to classify image regionsbased on their semantics. It can reduce the gap betweenhumans perception and description of image content.Because the pre-defined semantic class hierarchy reflectsin the semantics by humans subject, so it is flexible andintuitive query by novice.

The use of the binary SVM classifiers that classifyimage regions using different features at different levelsin the hierarchy were the main reasons behind the highclassification precision that we achieved in ourexperiments. Currently, we are looking adding morefeature extraction methods to get high precision and putmore classifiers to include more classes into the system.

ACKNOWLEDGMENT

This work was supported by Natural ScienceFoundation of Zhejiang province (No:Y1080565)

REFERENCES

[1] N.E.Ayat, M.Cheiet, C.Y.Suen, Automatic model selectionfor the optimization of SVM kernels-pattern recognition1733-1745 (2005)

[2] Zaher Al Aghbari, Region-based semantic imageclassification International Journal of Image and GraphicsVol.6. No.3 (3006) 357-375

[3] Pawan Jain, S.N.Merchant, Wavelet-based multiresolutionhistogram for fast image retrieval. International journal ofWavelets, Multiresolution and Information processing(2004)

[4] M. Flickner, H. Sawhney, J. Ashley, Q. Huang, B. Dom, M.Gorkani, J. Hafner, D.Lee, D. Petkovic, D. Steele and P.Yanker, Query by Image and Video Content:The QBICSystem, IEEE Computer Magazine (1995).

[5] S. Mehrotra, Y. Rui, M. Ortega and T. S. Huang,Supporting Content-Based Queriesover Images in MARS Proc. IEEE Int. Conf. On Multimedia ComputingandSystems (1997).

[6] Mihalcea, R. and Moldovan, D.: Semantic indexing usingWordNet senses. In Proceedings of ACL Workshop on IR& NLP, Hong Kong, October 2000.

[7] Miller, G. Wordnet: A lexical database. Communication ofthe ACM, 38(11):39--41, (1995).

[8] 7. Joon Ho Lee, Myong Ho Kim, and Yoon Joon Lee."Information retrieval based on conceptual distance in IS-A hierarchies". Journal of Documentation, 49(2):188{207,June 1993.

[9] Haav, H. M 5. A. Natsev, R. Rastogi and K. Shim,WARLUS: a similarity retrieval algorithmfor image

database, IEEE Transaction on Knowledge and DataEngineering 16(3),(March 2004).[10]R. Krishnapuram and S. Medasani, Content-based image

retrieval based on a fuzzyapproach, IEEE Transaction onKnowledge and Data Engineering 16(10), (October2004).

[11]F. Jeng, M. Li, H.-J. Zhang and B. Zhang, An efficientand effective region-basedimage retrieval framework,IEEE Transaction on Image Processing 13(5), (May2004).

[12]J.-H. Lim, Explicit query formulation with visualkeywords, Proc. ACM Multimedia(October 2000).

[13]B. Bradshaw, Semantic based image retrieval: aprobabilistic approach, Proc. ACMMultimedia (October2000).Region-Based Semantic Image Classification 375

[14]Minka, T.P., Picard, R.W.: Interactive learning using asociety of models. Pattern Recogn. 30, 565581 (1997)

[15]Wood, M.E., Campbell, N.W., Thomas, B.T.: Iterativerefinement by relevant feedback in content based digitalimage retrieval. In: Proceedings of the InternationalConference on Multimedia, pp.1320 (1998)

[16]Jing, F., Li, M.J., Zhang, H.J., Zhang, B.: Learning regionweighting from relevance feedback in image retrieval. Proc.IEEE Int. Conf. Acoust. Speech Sign. Process. 4, 40884091 (2002)

[17]Jing, F., Li, M.J., Zhang, H.J., Zhang, B.: Region-basedrelevancefeedback in image retrieval. Proc. IEEE Int.Symp. Circ. Syst. 4,145148 (2002)

[18]Jing, F., Li, M.J., Zhang, H.J., Zhang, B.: Support vectormachines for region-based image retrieval. Proc. IEEE Int.Conf. Multimedia Expo. 2, 2124 (2003)