CS292 Computational Vision and Language

Visual Features - Colour and Texture

Visual cues Visual cues for recovering 3D Information

• There are number of cues available in the visual stimulus– Motion– Binocular stereopsis– Texture– Shading– Contour

"Colour is the visual effect that is caused by the spectral composition of the light emitted, transmitted, or reflected by objects.“

Colour

Arrangement of colours is connected to psychological effects.

Studies by psychologist and artists have demonstrated that the presence and distribution of colours induce sensations and convey meanings in the observer, according to specific rules - colour images can also be retrieved according to the meaning they convey or to sensations they provoke.

What we have learnt to measure colour content?

• Colour histogram

• Colour intensity value itself

Retrieval based on colour Retrieval based on colour

• Retrieval by colour similarity requires that models of colour stimuli are used, such that distances in the colour space correspond to human perceptual distances between colours.

• Colour patterns must be represented in such a way that salient chromatic properties are captured.

Colour Histogram• Application example - colour histogram

retrieval: This method is to retrieve images from the database that have perceptually similar colour to the input image or input description from the user.

• The basic idea is to quantize each of the RGB values into m intervals resulting in a total number of m3 colour combinations (or bins)

• A colour histogram H(I) is then constructed. This colour histogram is a vector {h1, h2, …, hm3} where element hx represents the number of pixels in image I falling within bin x.

Colour Histogram

• The colour histogram becomes the index of this image• To retrieve image from the database, the user supplies

either a sample image or a specification for the system to construct a colour histogram H(Q).

• A distance metric is used to measure the similarity between H(Q) and H(I). Where I represents each of the images in the database. And example distance metric is shown as follows:

x=m3

D(Q, I ) = |qx-ix| x=1

Where qx and ix are the numbers of pixels in the image Q and I, respectively, falling within bin x.

Colour Histogram

• may fail in recognizing images with perceptually similar colours but no common colours. This may be due to a shift in colour values, noise or change in illumination. – measure the similarity

• Not enough for complicated images where spatial position is more important information. – May consider dividing each image into multiple

regions and compare the colour histograms of the same regions in the images.

• May combine with other methods such as shape and/or texture based retrieval to improve the accuracy of the retrieval.

Texture

• What is texture?

• How to detect texture information?

• How to measure it?

• How to use it?

Shape and texture

TextureTexture• Texture is a phenomenon that is widespread, easy to recognise and hard to define.

TextureTexture

• Together with colour, texture is a powerful discriminating feature, present almost everywhere in nature.

• Like colours, textures are connected with psychological effects. In particular, they emphasize orientations and spatial depth between overlapping object.

Texture

• There are three standard problems to do with texture:– Texture segmentation is the problem of breaking an

image into components within which the texture is constant. Texture segmentation involves both representing a texture, and determining the basis on which segment boundaries are to be determined.

– texture synthesis seeks to construct large regions of texture from small example images.

– Shape from texture involves recovering surface orientation or surface shape from image texture.

recover shape from texture

RecentRecent Technical Definition of texture:- Texture is a broad term used in pattern recognition to identify image patches (of any size) that are characterized by differences in brightness.

•Techniques to extract meaningful texture descriptors from image are many, based on different models and assumptions.

•An effective representation of textures can be based on statistical and structural properties of brightness patterns.

Texture Content Measurement

• Textures may be described according to their spatial, frequency or perceptual properties.

• Periodicity, coarseness, preferred direction, degree of complexity are some of the most perceptually salient attributes of a texture.

• Feature spaces based on these attributes are particularly interesting for image retrieval by texture similarity.

Space – based models

• Auto-correlation function A texture can be represented taking into account the spatial size of grey-level primitives. Fine textures have a small size of their grey-level primitives. Coarse textures a large size.  

• Co – occurrence matrix A different way of measuring textures is by taking into account the spatial arrangement of grey-level primitives.

Calculate Texture

energy

entropy

contrast

Image features