1

VISHVESHWARAIAH TECHNOLOGICAL UNIVERSITY

S.D.M COLLEGE OF ENGINEERING AND TECHNOLOGY

A seminar report on

CONTENT-BASED IMAGE RETRIEVAL

Submitted by

SAMEER KULKARNI

2SD06CS083

8th semester

DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

2009-10

2

VISHVESHWARAIAH TECHNOLOGICAL UNIVERSITY

S.D.M COLLEGE OF ENGINEERING AND TECHNOLOGY

DEPARTMENT OF COMPUTER SCIENCE ENGINEERING

CERTIFICATE

Certified that the seminar work entitled “content-based image retrieval” is a bonafide work

presented by Sameer Kulkarni bearing USN NO 2SD06CS083 in a partial fulfillment for the

award of degree of Bachelor of Engineering in Computer Science Engineering of the

Vishveshwaraiah Technological University, Belgaum during the year 2009-10. The seminar

report has been approved as it satisfies the academic requirements with respect to seminar

work presented for the Bachelor of Engineering Degree.

Staff in charge H.O.D CSE

Name: SAMEER KULKARNI

USN: 2SD06CS083

3

INDEX

1. INRODUCTION 4

2. BAKGROUND 4

3. NEED FOR CBIR 5

4. TECHNIQUE USED IN CBIR

o CBIR TECHNIQUES 6

o COLOR RETRIEVAL 7

o TEXTURE RETRIEVAL 8

o SHAPE RETRIEVAL 9

o OTHERS TYPES OF FEATURES 10

5. SEGMENTATION 11

6. REFERENCE FEEDBACK 12

7. CO-OCCURRENCE MATRIX AND R I 12

8. APPLICATIONS OF CBIR 15

9. ADVANTAGES OF CBIR OVER TBIR 16

10. ISSUES RELATED TO CBIR 16

11. DRAWBACKSOF CBIR 16

12. CONCLUSION 17

13. REFERENCES 18

4

1 INTRODUCTION:

Content-based image retrieval is a part of image processing and is also

comes under artificial intelligence. We know that interest in digital images is

growing day by day; Users in many professional fields are exploiting the

opportunities offered by the ability to access and manipulate remotely-stored

images in all kinds of new and exciting ways. The problems in image retrieval is

becoming widely recognized, and the search for solutions an increasingly active

area for research and development. So here is the technique Content-based image

retrieval (CBIR), also known as query by image content (QBIC) and content-

based visual information retrieval (CBVIR) is the application of computer vision

to the image retrieval problem, that is, the problem of searching for digital images

in large databases.

2 BACKGROUNDS:

The use of digital images and pictures in communication is new – what

olden days cave-dwelling ancestors did was they use to paint pictures on the

walls of their caves, and the use of maps and building plans to convey information

almost certainly dates back to pre-Roman times. But the twentieth century has

witnessed unparalleled growth in the number, availability and importance of

images in daily life. They now play an important role in fields as diverse as

medicine, journalism, advertising, design, education and entertainment.

The term CBIR seems to have originated in 1992, when it was used by

T. Kato to describe experiments into automatic retrieval of images from a

database, based on the colors and shapes present. Since then, the term has been

used to describe the process of retrieving desired images from a large collection on

the basis of syntactical image features. The techniques, tools and algorithms that

are used originate from fields such as statistics, pattern recognition, signal

processing, and computer vision.

5

3 NEED FOR CBIR:

In today’s world more and more multimedia Information is stored in

databases like images audio and video. An image may be better or more effective

than a substantial amount of text. It also aptly characterizes the goals of

visualization where large amounts of data must be absorbed quickly and it is a

picture is worth of thousand words. If we use text for retrieving/searching an

image what happens is-we can’t write whole description of image this is one thing

and if the image contains geographical data then manual keyword description are

not possible ex: Google earth contains geographical data. So then why to go for

this is because- It is cultural language dependent and is not possible to describe

every image in database. Keyword matching will not give most relevant images.

Fig1

6

4 TECHNIQUES USED IN CBIR:

CBIR techniques

CBIR operates on a totally different principle, retrieving/searching stored

images from a collection by comparing features automatically extracted from the

images themselves. The commonest features used are mathematical measures of

color, texture or shape (basic). A system (CBIR) allows users to formulate

queries by submitting an example of the type of image being sought (input),

though some offer alternatives such as selection from a palette or sketch input we

can also select color textures or any other visual information. The system then

identifies those stored images whose feature values match those of the query most

closely (right side), and displays thumbnails of these images on the screen like in

fig1.

Fig2

The above shown Activity Diagram explains the basic steps in any of the CBIRS

(systems).

7

Colour retrieval

Each image added to the collection is analyzed to compute a ‘color

histogram’ where ‘color histogram’ is a representation of the distribution of colors

in an image. For digital images, it is basically the number of pixels that have colors

in each of a fixed list of color ranges, which span the image's color space, the set of

all possible colors. This shows the proportion of pixels of each color within the

image. The color histogram for each image is then stored in the database. During

searching time, the user can either specify the desired proportion of each color

(75% olive green and 25% red, for example)it depends on the system to give that

option or not(design and options), or submit an example image from which a color

histogram is calculated. Matching process then retrieves those images whose color

histograms match those of the query most closely as described above in

percentages. The results from some of these systems can look quite impressive.

One more concept is ‘Gray scale image’ which is simply one in which the

only colors are shades of gray (fig 3 right side). It is different from any other sort

of color image is that less information needs to be provided for each

pixel(maximum information of image color). In fact a `gray' color is one in which

the red, green and blue components all have equal intensity in RGB space, and so it

is only necessary to specify a single intensity value for each pixel, as opposed to

the three intensities needed to specify each pixel in a full color image.

Fig3

8

Texture retrieval

What if the images are of same color? This will be answered by textures. The

ability to retrieve images on the basis of texture similarity may not seem very

useful-But the ability to match on texture similarity can often be useful in

distinguishing between areas of images with similar color (such as sky and sea, or

leaves and grass). A variety of techniques has been used for measuring texture

similarity; the best-established rely on comparing values of what are known as

second-order statistics calculated from query and stored images. Essentially, these

calculate the relative brightness of selected pairs of pixels from each image. From

these it is possible to calculate measures of image texture such as the degree of

contrast, coarseness, homogenality and regularity [Tamura et al, 1978], or

periodicity, correlation and entropy [Liu and Picard, 1996]. Alternative methods

of texture analysis for retrieval include the use of Gabor filters this is the widely

used technique now a days. Texture queries/specifying can be formulated in a

similar manner to color queries, by selecting examples of desired textures from a

palette, or by supplying an example query image. The system then retrieves images

with texture measures similar in value. A recent technique is the texture thesaurus

developed by Ma and Manjunath [1998], which retrieves textured regions in

images on the basis of similarity to automatically-derived codeword’s representing

important classes of texture within the collection.

Contrast is the dissimilarity or difference between things(color, brightness

etc).

Homogeneity means "being similar throughout"(like same color can be said

to one part segmentation can also be done through this).

Entropy is a measure of the uncertainty associated with a random variable.

Fig4

9

Shape retrieval

Shape is the most obvious requirement at the primitive level. Unlike texture,

shape is a fairly well-defined concept – and there is considerable evidence that

natural objects are primarily recognized by their shape .A number of features

characteristic of object shape (but independent of size or orientation) are computed

for every object identified within each stored image. Queries are then answered by

computing the same set of features for the query image, and retrieving those stored

images whose features most closely match those of the query. Two main types of

shape feature are commonly used – global features such as aspect ratio,

circularity and moment invariants [Niblack et al, 1993] and local features

such as sets of consecutive boundary segments [Mehrotra and Gary, 1995].

Alternative methods proposed for shape matching have included elastic

deformation of templates (Pentland et al [1996], del Bimbo et al [1996]),

comparison of directional histograms of edges extracted from the image and

shocks, skeletal representations of object shape that can be compared using graph

matching techniques.

Fig5

10

Retrieval by other types of primitive feature

One of the oldest-established means of accessing pictorial data is retrieval by

its position within an image. Techniques have been applied to image collections,

allowing users to search for images containing objects in defined spatial

relationships. Improved algorithms for spatial retrieval are still being proposed.

Spatial indexing is seldom useful on its own, though it has proved effective in

combination with other cues such as colour (Stricker and Dimai [1996], Smith and

Chang [1997a]) and shape [Hou et al, 1992].

Several other types of image feature have been proposed as a basis for

CBIR. The well-researched technique of this kind uses the wavelet transform to

model an image at several different resolutions. Promising retrieval results have

been reported by matching wavelet features computed from query and stored

images .Another method giving interesting results is retrieval by appearance.

The advantage of using these are they can describe an image at varying levels

of detail (useful in natural scenes where the objects of interest may appear in a

variety of guises), and avoid the need to segment the image into regions of interest

before shape descriptors can be computed. Despite recent advances in techniques

for image segmentation ,this remains a troublesome problem. Now we will see one

of the technique that is edge detection that comes under shape retrieval.

fig6

The above marked points in image are called spatial points.

11

5 Segmentation:

Segmentation refers to the process of dividing the a digital image into

multiple segments or a set of pixel it is also sometimes known as super pixels.

Why we go for segmentation? - is to simplify and/or change the representation of

an image into something that is more meaningful and easier to analyze. Image

segmentation is typically used to locate objects and boundaries, lines and edges in

images. Precisely image segmentation is the process of assigning a label to

every pixel in an image such that pixels with the same label share certain

visual characteristics.

Fig7

12

6 RELEVANCE FEEDBACKS (FURTHER):

Fig8

After retrieval of results, the irrelevant images also may found. User give

their feedback about the results to avoid images that are not related to the given

query. Relevant, Not-Relevant, No-Idea After that the image which is irrelevant is

not included for that query in future.

7 Co-occurrence Matrix and Rotational Invariance:

Co-occurrence matrix method is an essential requirement in the field of

image processing using pattern recognition and has been used extensively. A co-

occurrence matrix, also referred to as a co-occurrence distribution, is defined over

an image to be the distribution of co-occurring values at a given offset. Any matrix

or pair of matrices can be used to generate a co-occurrence matrix, though their

main applicability has been in the measuring of texture in images.

13

Let I be a gray scale image which is coded on ‘m’ gray levels. S=(x, y) is the

position of a pixel in the image and T be the translational vector. The co-

occurrence matrix Mt is an m*m matrix whose (i, j) th element is the number of

pairs of pixels separated by the translation vector T that have the pair of gray levels

(i, j).

An example of co-occurrence matrix is shown in figure.

Table 1:Image matrix

In the output GLCM, element (1,1) contains the value 1 because there is only one

instance in the input image where two horizontally adjacent pixels have the values

1 and 1, respectively. glcm(1,2) contains the value 2 because there are two

instances where two horizontally adjacent pixels have the values 1 and 2. Element

1 2 0 0 1 0 0 0

0 0 1 0 1 0 0 0

0 0 0 0 1 0 0 0

0 0 0 0 1 0 0 0

1 0 0 0 0 1 2 0

0 0 0 0 0 0 0 1

2 0 0 0 0 0 0 0

0 0 0 0 1 0 0 0

1 1 5 6 8

2 3 5 7 1

4 5 7 1 2

8 5 1 2 5

1 1

1 2

1 2

14

(1,3) in the GLCM has the value 0 because there are no instances of two

horizontally adjacent pixels with the values 1 and 3. graycomatrix continues

processing the input image, scanning the image for other pixel pairs (i,j) and

recording the sums in the corresponding elements of the GLCM.

Let us now consider the concept of rotational invariance,

The (Δx, Δy) parameterization makes the co-occurrence matrix sensitive to

rotation. We choose one offset vector, so a rotation of the image not equal to 180

degrees will result in a different co-occurrence distribution for the same (rotated)

image. This is rarely desirable in the applications co-occurrence matrices are used

in, so the co-occurrence matrix is often formed using a set of offsets sweeping

through 180 degrees (i.e. 0, 45, 90, and 135 degrees) at the same distance to

achieve a degree of rotational invariance.

15

8 APPLICATIONS OF CBIR:

1. Image search engines.

2. Security

o Thumb recognition

o Iris recognition

Fuzzy logic is used most of the times it divides the

image and uses co-occurrence matrix.

3. Investigations

o Compare images

o Collect relevant images

4. Analysis

o Geographical and enterprise related

5. Medical diagnosis

o Cancer detection.

6. Military and remote sensing Systems

16

9 ADVANTAGES OF CBIR OVER TBIR:

Among other image retrieval methods, content-based image retrieval4

(CBIR) is an approach that exclusively relies on the visual features, such as

color histogram, texture, shape, and so forth, of the images.

One of the obvious advantages of CBIR over other methods, e.g., text-based

image retrieval, is that CBIR can be done in a fully automatic process since

the visual features are automatically extracted.

It is based on Artificial intelligence.

10 SOME OF THE ISSUES RELATED TO CBIR:

Which features should be derived to describe the images better within

database.

Which data structure should be used to store the feature vectors

Which learning algorithms should be used in order to make the CBIR wiser

How to participate the user’s feedback in order to improve the searching

result.

11 DRAWBACKS OF CBIR:

More Resources are needed like CPU time, memory, disk space,etc.

Determining Semantic meaning of Objects is not possible.Results may contain

False Hits.

Problems on Processing Different aspects of Images.

A drawback associated with CBIR is the increased computational cost arising

from tasks such as image processing, feature extraction.

17

12 CONCLUSION:

CBIR at present is still very much a research topic. The technology is

exciting but immature, and few operational image archives have yet shown any

serious interest in adoption .CBIR technology replacing more traditional methods

of indexing and searching. Use of text and image features might well yield better

performance than either type of retrieval cue on its own.

12 REFERENCES:

1. [http://en.wikipedia.org/wiki/Content_based_image_retrieval].

Introduction and concepts

2. [http http://www.eecs.berkeley.edu/~cchang/docs/Spie01.pdf] content

based image retrieval by Bhakthvatsalam (ppt) – technique used,

reference feedback.

3. content-based Image RetrievalJohn Eakins Margaret Graham

University of Northumbria at Newcastle [Technique used]

4. Image retrieval based on region shape similarity by Cheng Chang , Liu

Wenyin and Hongjiang Zhang.[22-25]

5. Feature extraction and image retrieval by Mark s Nixon and alberto 1st

edition[99-157].

6. http://homepages.inf.ed.ac.uk/rbf/HIPR2/gryimage.html [gray scale

images].

7. Issues [http://imageprocessing.wordpress.com/2007/12/09/content-based-

image-retrieval-cbir/]

8. http://matlab.izmiran.ru/help/toolbox/images/enhanc15.html for glcm.

18