texture tutorial1CHK may08 · In computer graphics, texture refers to two notions, distinguished by context: the tactile, 3D ... geometry can act as an alternative approach to the

Jorge Marquez - UNAM 2008 Texture Tutorial I 1/79

Texture Characterization and Analysis Tutorial I - version 2.0

Jorge Márquez Flores, PhD - [email protected] Images and Vision Laboratory, CCADET-UNAM Copyright ©2005,2007

Dept. Traitement de Signal et Images - Ecole Nationale Supérieure des Télécommunications, Paris, Copyright ©2000,2007 Texture Analysis Workshop - Technische Universität Hamburg, Copyright ©1996,2004

Computer Science Department, The University of Auckland, Copyright © 2008. This tutorial introduces several concepts and ways to define and to analyze textural features in complex images from various fields of scientific research. It is designed as a preliminary lecture, before attending the short course/workshop on texture characterization and analysis, where further material will be explained, accompanied with many graphic examples. Note it is copyrighted material, at present belonging to the CCADET-UNAM in Mexico City, Mexico. It is only provided to course attendants; please do not reproduce without permission. Introduction: texture definitions We begin by recognizing that: there is no universally accepted mathematical definition of texture. In the modern framework of fractal analysis (which we only will see very briefly), according to Benoit Mandelbrot,


“Texture is an elusive notion which mathematicians and scientists tend to avoid because they cannot grasp it and much of fractal geometry could pass as an implicit study of texture [Mandelbrot 1983, The Fractal Geometry of Nature].”

A vague but intuitive definition from the Wikipedia:

Texture refers to the properties held and sensations caused by the external surface of objects received through the sense of touch. The term texture is used to describe the feel of non-tactile sensations. Texture can also be termed as a pattern that has been scaled down (especially in case of two dimensional non-tactile textures) where the individual elements that go on to make the pattern are not distinguishable.

Textures are often defined as a class of “patterns” (but note the scaling down condition. above), but the later is a notion in itself, where a number of elements are identified and more or less isolated, and their relationships studied in “pattern analysis”. There are also “sound patterns” and “sound textures”, implying complex-varying details; all human senses (and all kind of information) prompt the existence of domain-textural qualities, even in psychological behavior, literature and abstract concepts. A simpler but still limited definition in image analysis is that of gray-level variations, following some kind of periodic or quasi-periodic pattern (but there are also textures with no periodicities at all) Here “pattern” stands for a set of rules or an approximated configuration or structure. Also, a degree of randomness (against determinism) has to be included.


In computer graphics, texture refers to two notions, distinguished by context: the tactile, 3D variations of surfaces, and the color maps or texture mapping (at a pixel level, any bitmap image) wrapped on computer meshes of objects, to make their rendering easier. Most models of textures began by treating them as “repeating” patterns and some old definitions abuse of this simplification. Some statistical repetitiveness may indeed be present in many textures, specially at some scales where features average out. Complex and even just irregular textures were for a long time ignored, since no mathematical or computer tools could grasp all their properties with success. The ability to characterize, model and even synthesize a particular pattern or texture has been accompanied by an understanding of physical, biochemical or other process giving rise to them. Modern definitions rely on descriptive approaches, mainly statistical and structural. Repetitiveness may or may not be present (or only in some statistical, or filtered sense), and textures may have textural components with both extremes. Randomness is another element present in natural or “real” textures. Literature on texture analysis reports up to date about 2000 textural parameters!, but most of them are strongly correlated and may be difficult to interpret and to compare. We rather start by a higher level of description, based on a few, simple, intuitive notions based on human perception.


1. Human-Perceived Measures of Texture It is one objective of computer vision that an unsupervised, automatic means of extracting texture features that agrees with human sensory perception can be found. Tamura et al., in [Tamura, H., Mori, S., and Yamawaki, T. Textural features corresponding to visual perception. IEEE Transactions on Systems, Man, and Cybernetics, 8(6):460, 1978] defined six mathematical measures which are considered necessary to classify a given texture. These measures are as follows: Coarseness, coarse versus fine. Coarseness is the most fundamental texture measure and

has been investigated since early studies in the 1970s by Hayes [IEEE Transac. On Systems, Man and Cybern. 4:467, 1974]. When two patterns differ only in scale, the magnified one is coarser. For patterns containing different structure, the bigger the element size, or the less the elements are repeated, the coarser it is felt to be. Some authors use busy or close for fine. The elements are also known as “the grain of the texture”. They may include the texels in a structural approach (see below); the bigger the texels, the larger the coarseness. When two texture regions are joinmtly scaled in such a way that they have “the same” coarseness or size of grain (features), we may thing of other measured features as being invariant to scale, to some degree.

Contrast, high versus low. The simplest method of varying image contrast is the

stretching or shrinking of the grey scale [Rosenfeld and Kak, Digital Picture Processing, 1982]. By changing (globally) the contrast level of an image we alter the image quality,


not the image structure. When two patterns differ only in the grey-level distribution, the difference in their contrast can be measured. However, more factors are supposed to influence the contrast differences between two texture patterns with different structures. The following four factors are considered for a definition of contrast: • Dynamic range of grey scale, • Polarisation (bias, skewness) of the distribution of black and white, on the grey-level histogram, or the distribution of the ratio of black and white areas. • Sharpness of edges (against background) and shape sharpness (corners). • Period of repeating patterns, frequency characterisation of them. High global contrast is associated with weakness or strenght of a texture, but it may always be normalized to, for example, full dynamic range. This is why most texture efatures may be though as invariant to contrast.

Directionality, directional versus non-directional. This is an average property over the given region. Directionality involves both element shape and placement rules. Bajcsy [7] divides directionality into two groups, mono-directional and bidirectional, but Tamura measures only the total degree of directionality. In such a scheme the orientation of the texture pattern does not matter. That is patterns which differ only in orientation should have the same degrees of directionality. A subtle difference exists between directionality and coherence; the latter is a measure of a kind of spatial correlation where local directionality exists. Two directions may co-exist an even be separable. Another tool for studying both, orientation and directionality is the local tensor of


inertia, whose eigenvalues –magnitude and angle- give the orientation and an “intensity” or degree of directionality/coherence. Directionality may be also called “anisotropy”, and no dominant directionality is the characteristic of an isotropic texture.

Since the dominant orientation of a texture region can be obtained, a rotation in the

principal axes is always possible and this constitutes a normalization with respect to orientation: it is best to compare textures once they are aligned (same region). In this sense, the other texture features may be though as invariant under rotations.

Line-likeness, line-like versus blob-like. This concept is concerned only with the shape

of a texture element. This aspect of texture may supplement the major ones previously mentioned, especially when two patterns cannot be distinguished by directionality. The blob-like nature corresponds to the “mosaic index” used in metallurgy and porosity analysis: it describes the existence of domains or large isotropic grains.

Regularity, regular versus irregular. This can be thought of as a variation in a

placement rule. For texture elements, however, it can be supposed that variation between elements, especially in the case of natural textures, reduces the overall regularity. Additionally, a fine structure tends to be perceived as regular. Irregularity is not exactly synonim of randomness or noise.

Roughness, rough versus smooth. This description was originally intended for tactile

textures, not for visual textures. However, when we observe natural textures, such as


Hessian cloth, we are able to compare them in terms of rough and smooth. Tamura asks is this subjective judgment due to the total energy of changes in grey level or due to our imaginary tactile sense when we touch it physically? It is worth noting that roughness is different from contrast, which deals only with gray level variations (“vertical” in a z=I(x,y) representation), while roughness inlcudes local shape variations.

Tamura gives mathematical definitions of all these measures plus results of experiments to find the correlation between these measures and human perception. A final property of textures concerns the degree to which they are regular or random. The vast majority of natural textures are random in nature, which allows any of the above parameters to vary. Formulae for all these techniques are presented in the following paragraphs. A structure which is also possible is the use of a pyramid to hold an image at multiple different resolutions, each a factor of 4 smaller than the previous level. This provides an intuitive way of detecting different textures at different scale levels. Other properties are sometimes taken into account, such as the presence of spatial gradients (a texture for which one or more features change in one or many directions), local, regional and scaling properties. Also a degree of sharpeness or smoothness, due to blurring by filtering (digital or from the PSF of the acquisition system) may be though or treated as textural properties. Again, some degree of invariance with respect to sharpness, may be obtained by properly correcting (by deconvolution) or regularizing (smoothing, to obtain the “same focus”) two textures, before comparison.


Color variations are similar to contrast variation, when two textures change only in hue, saturation on RGB intensities. Color normalization may be difficult, in order to enhance color changes due to textural differences, but a global equalization may be useful. In relation with the scaling properties in the texture itself, we show in Tutorial 2, that fractal geometry can act as an alternative approach to the problem and provide texture classification through fractal descriptions. It should be noted, however, that these approaches are not mutually exclusive. For example, some of the recent papers on morphology also include fractal ideas [96]. Ohanian and Dubes [107], compare the performance of four classes of textural features and come to the conclusion that the results show that co-occurrence features perform best followed by fractal features but however, there is no universally best subset of features. The feature selection task has to be performed for each specific problem to decide which feature of which type one should use. Other qualities have been identified with textures, including those mentioned, it can be coarse, fine, smooth, granulated, rippled, regular, irregular or linear.


2. Summary:

• Textural attributes are difficult to define and characterize. • There is certain proximity to the nature of noise (blurred distinction). • Textural details are often also close to resolution limits and to the aperture functions

(in particular PSF -Point Spread Functions) characterizing the observing system or tools of analysis.

• When details frequencies fall below the Nyquist (sampling) frequency, observed textures are sub-sampled textures and may exhibit superposed “emergent” textural patterns, such as Moïré and multiplicative noise (speckle).

• Set of many local characteristics: need of analyzing spatial relations, etc. among two or more pixels – co-occurrence, correlation, mutual information…

• Language adds to confusion with ill-defined terms: roughness, grain, blobby-ness, busyness, energy…

• Several textures could be defined like this:

kk

texture textures=∑ (1)

where the texture k may be simpler different textures, and Σ may be: averaging, morphological blending, set union, intersection, overlapping, modulation or any superposition in general. A texture may be called “separable” if the above decomposition


is possible. Most often, it is only an approximation (a superposition model, with coefficients weighting components from a “textural” basis). There are other kinds of decomposition, for example, at different resolutions and the time-frequency paradigm is the used (wavelets).

Note: a linear decomposition or linear projection into a (orthonormal) base E={ek} would imply a “E-domain transform TE” obtaining coefficients wk = TE[texture] such that:

k kk

texture w=∑ e (2)

Which can be interpreted as a weighted average (blending) or superposition (the set W={wk} being the spectrum in E-domain) and the set E may also be a set of texture primitives in all possible configurations (positioning rules).

An important observation is that a texture may combine (or even be the result of): more than two simpler textures (as superposition or other form of combination), gradients (either illumination, material changes in darkness or by projection, such as persepective from 3D orientation), noise (of several kinds) and distorions, artefacts and structure (not being properly textures), blurring (from the system acquisition PSF, or from explicit filtering), aliasing effects from several sources (mostly sub-sampling by other textures rather than discretization or resolution). Coloring or decoloration can be though as a specific kind of noise, gradient or artefact (a splotch).


3. Transitions between structure and texture:

Images are often though as comprising shape and texture. Shape is also called structure.

The boundary may be difficult to establish and is not necessary implied by sampling limits (Nyquist), but rather by the PSF of the sensing/perceiving system or any other aperture size characterizing a particular task.

A more general and realistic decomposition includes also background. Texture may or may not include noise elements and shape also may include artifacts.

Many analysis aproaches decompose also a signal f (t) in at least 3 components:

ftrend (t) + finterest(t) + fdetails(t) (details, texture or “noise”) (3) .

The first, ftrend (t), is also known as “background” and may be characterized by low frequencies (a relative notion), the second, finterest(t), is also known as the “foreground” and may include middle and, optionally, high frequencies, it also comprises structural or shape components of the signal/image (it may also be considered that ftrend (t) may comprise some structure, too) and the third component fdetails(t) may constitute some texture, noise and is characterized by high frequencies. An approximated way to separate each component is by using a bank of 3 filters: a low pass, a band pass and a high pass. In a real application the “information of interest” may possess high intensity or spatial frequencies (for example, the sharp corners of a triangle) and the noise may have a broad frequency spectrum, thus a clear


separation is more difficult. Models or knowledge of the nature of noise (its spectrum or statistics), the background as well as the foreground, are then used to better accomplish such decomposition.

Another question is where does each “component” begins and where does it ends. In one problem a texture may be taken as “noise” (unwanted information affecting the analysis of other features) and in another as the very subject of analysis. Artifacts are defined as objects that are confounded with foreground, and other criteria must be used to their elimination. The transitions also depend on maximum resolution, and the PSF of the acquisition/processing system.


Texture Classification Approaches

Statistical Structural

Periodic • Surface analysis • Moments • ACF • Transforms • Edge-ness • Co-occurrence matrices • Texture transforms • PCA • Random field models • Variograms • HVS approach: features from banks of filters (Laws, Gabor, Hermite) and pyramids / scalespace

• Spectral Analysis

• Hibrid approaches • Mosaic models -Voronoi • Grammatical models •Autoregressive models • Fractal dimensions • Stereological (sampling) • Porous material properties • Mathematical Morphology

Other approaches

Quasi-periodic Random

• Fuzzy-spectral • Wavelet Analysis

• Edge density • Extreme density • Run lenghts • PCA • Variograms • Codewords, LBP

Complex features:

• Flows • Gradients • Color textures • 3D and time varying textures

Primitives (texels):

• Gray levels and color • Shape • Homogeneity

Placement rules:

• Period(s) • Adjacency • Closest distances • Orientation • Scaling


Blobbi-ness (mosaic index)

Line-like

Coarseness Grain distribution

Bussi-ness

High/Low Frequencies

Near/Far of transition to structure/shape Wavy-ness

Randomness Noise (gaussian, white, etc.)

Repeating patternsRegularity

Directionality orientation

Coherence

Roughness

Local Correlations Co-occurrence

Contrast Intensity variation

Self-simmilarityScale relations

hierarchies

microstructure

The “Many” Dimensions of Textural Features

COLOR (Hue Sat Lum)

Anisotropy

Rose of Directions

Elemental grain / TEXEL


In the above figure, axes do not exactly reflect opposed features, many are somehow considered by some authors as synonims or special cases of other features, and labels in black refer to those human vision clearly distinguish, after psychophysical experiments. 4. Two (or three) Main Approaches. Structural Approach – When we can identify (or there exist) one or more texture

primitives (also called tonal primitives), called texels for texture elements, (or also texons, or textons, for textural atoms, and also structural elements), which consist of any (small) pattern which repeats according to some rules:

Texture = texel(s) + positioning rules (from deterministic to random). The latter includes: translation, rotation, scaling, superposition, blending or other texel-combination methods. Positioning may be periodic, quasi-periodic, anisotropic, or change in complex ways. As discrete sub-images, the texels may be though as any small set (array) of pixel values (binary, gray-level intensities or color values), such as the symbol “4”, a logo, or colored dots. Note: any pattern, such as a line, a branch, a blob, a fading Point Spread Function, etc. are potential texels. In old texts the texels are said to “repeat periodically”, which is only a small subset of possible positioning rules.

Human written language can be viewed as structural textures. Chinese, arabic, roman, cyrilic, jerogliphics, mathematical or any other symbol-based texts, are constituted by texels called characters or symbols, following precise, organized


combination rules, called syntax of the specific language. Some quasi-periocities may appear, and are more evident in poetry or tables describing properties of a list of items.

Spectral approach – texel definition may be extended to include basis functions (sin, cos, etc.) and texture is then characterized by frequency-domain properties. Note that this is equivalent to non-finite space-support texels and the particular positioning rule of linear superposition. Many authors consider that, by its importance and special properties, the Spectral Approach is the third-one, to be mentioned, since most of the analysis requires to obtain frequency domain features.

Statistical Approach – Analysis of distributions and statistical properties. No order or

deterministic structure is assumed. Textures are treated as instantiations (or “realizations” and sometimes also “snapshots”) of random fields. Local stationarity is defined by local constancy of statistical properties. We review Markov Random Fields applied to texture analysis in Tutorial 2.

Hybrid approach: ½ structural + ½ statistical (or other proportions): when texels may have a random nature or random distribution properties. If texels strictly follow deterministic placement rules, the produced texture is called a strong texture. For pure randomly placed texels, the associated texture is then called a weak texture. An example of strong, deterministic texture is an image (specially a high resolution, computer


rendering) of a text, in any human language, since placement rules are complex (syntax, orthography, logic), and there is a fixed number of texels called characters or symbols. A less strong texture may be hand-written language, which includes random and stylistic variations of spacing and texel-shape. Even errors, loose syntax and bad orthography add noise. In fingerprints a structural primitive is difficult to locate, as well as a positioning rule, but there are clearly curved lines which rarely cross or bifurcate, some coherence, and average separations. Some specialist consider fingerprints rather as “patterns” (identified components spatially related), and texture descriptors may miss information that shape analysis and morphometrics better capture. So we may also account for a morphometrical approach to texture analysis, in which a texture is treated as a pattern, provided its components (not exactly texels) can be identified/extracted. Statistics are also employed in another level (distribution of shape features, or configuration properties). The pattern or morphometrical approach is described in Tutorial 2. Fingerprints and other textures (or patterns) introduce a seventh important feature, which can be regarded as “flow”: a changing directionality in an anisotropic texture.

Discrete texels: Better described as texel patterns (or texels represented by pixel patterns), they are the array of pixel intensities (or colors) that make up a texture primitive in practical computer images (it is pattern, since individual pixels are identified). Simple examples are fonts of text characters (for vectorized graphics, there are rules to build continuous patterns, as splines from a low number of control points; these are derived texels).


Sub-sampled structural textures, are those where the primitive(s) are highly “noised” blurred, or sub-sampled, and may or may not have random components.

Note that the statistical approach corresponds more to a decision (to take that very approach) rather than to the nature of the texture: both are models. The use of statistics reflects our lack of information of underlying structures or laws, its excessive complexity or a very high cardinality of components. The use of structural modeling may be empirical, but eventually requires deterministic information (how the texture has arisen from physical, mathematical or other causes). If the statistical approach is applied to a texture where texels and positioning rules exist, both texels and rules are secondary or ignored in such analysis.

Most textures turn to fall between the structural and the statistical descriptions.

In the modern called stereological approach (Tutorial 2), the analysis is about sampling of the texture by stochastic geometry probes (point, line processes, given texels, or any other sampling schema). The analysis may be based on studying the resulting interactions of a texture and the stochastic geometry probes. Such kind of analysis may also be defined as “Monte Carlo” methods of analysis (the statistical part being part of the measurement tools).

Another approach is the surface-intensity or 3D-approach, where the image gray levels I(x,y) are interpreted as a third dimension (height) z = I(x,y), and we analyze the 3D surface defined by all (x, y, z) coordinates. In this approach, shape and intensity are “married” and the local normal vectors are sensible descriptors of local roughness and other textural features. In


Section 8 a parameter known as RED (Ratio of Extrema Density) is described and further properties are found in Tutorial 2. Interactions (superpositions) with the acquisition/display system: • PSF (all apertures), discrete array of sensors and pixelization. Introduces a

superposed structured texture, giving rise to phenomena similar to sub-sampled textures: aliasing and Moïrè patterns.

Interactions with the Human Visual System (HVS): • Interaction with the “sensor texture” of cone and rods, and resolution limits. • Tendency to group, to perceive clusters, domains or cells, lines, flux lines, networks and

Gestalt configurations (to complete lines, extrapolate figures, subitize items, etc.). • Integrated Fourier analysis and scale-space perception also extracts high-level features.

Note: subitize (Spanish: “conteo súbito” or “subitemizar”) is an implicit “counting” task of a cognitive system, in which a pattern of N objects is recognized (and the number N associated), without explicitly identifying and counting each object. The input to the recognition system is a pattern, the output is N, or a reaction or behavior related to N in some way. Usually N < 7 in normal individuals, since the number M of possible configurations of simple items (identified as “dots”) in a planar perceptual space is about M ~ O(N!). Subliminal subitizing occurs when texture details is well over the Nyquist sampling limits of the HVS. This may lead to the intuitive notion of what is not a texture −precisely when subitizing occurs!


Pointillistic paintings (Saurat, 1880?) – have a local texture of colored dots. Question: where lies the transition to perceived patterns and structure? (faces, trees, landscape). The interaction with the HVS at different distances varies the PSF and the bank of neural pass-band filters at the Lateral Geniculus brain structures, before the visual cortex. In simple words: there is a perceived transition: “these are dots,… now they become texture 1, … texture 2, …, this is now a colored region,… wait! - now it is a figure, … and this is context,… this is art!” “to see the trees and not seeing the wood” “to see the wood and not seeing the wood”

- these opposed points of view refer to perceptual properties of textures or figures or context, and describe the same goal-oriented decomposition of the section Transitions Between Structure and Texture.


5. Texture-related tasks: • Parameter extraction and characterization or modeling. • Texture segmentation: segmenting regions by texture, besides intensity) • Texture classification: given N texture classes, decide where to classify a new texture. • Identifying texture boundaries assisted by models of regions. • Texel modeling/identification: adjust a structural-based model. • Texture synthesis: to mimic a given texture, to produce a texture from parameters or

texture samples, or to “grow” a textured region into non-textured region. • Certain (artificial) textures may relate to encrypted or coded information, as text. • Separate a given textural component (texture filtering). A particular texture may

have a physical cause, but it may also be composed of several textures corresponding to many physical causes (thus explaining out a textural component). Physical phenomena may happen at different scales (v.g. erosion by rain, by wind, by dust, by glaciers, by human or animal action, by microorganisms and by chemicals).

Computer vision tasks and paradigms exist known as “shape from texture”, “depth or 3D-orientation (perspective) from texture” and other. In this tasks, an assumption is made about textural properties of an object/scene (shift-invariance, for example, and no texture gradients), and then a deviation of this behavior is interpreted as the effect of some transformation that is then found (v.g., varying illumination, and orientation).


6. Textured Images as Noised Bi-dimensional Signals.

An image I, or any sub-region A ⊆ I, considered as a bi-dimensional signal, may be described as: (a) deterministic, exhibiting periodic or quasi-periodic (a few dominant frequencies

concentrate most energy, in Fourier space; there are shift-invariant properties), (b) non-deterministic (random), but stationary (statistical properties are more or less

constant), (c) random, not stationary (statistical properties are space-varying), (d) deterministic or non-deterministic with a transient structure (gradients,

discontinuities, with wavelet-analysis being a more useful framework than Fourier spatial-frequency analysis) or,

(e) chaotic, a special, complex, deterministic behavior which may be modeled by dynamical-systems attractors in a phase space, and by fractal analysis. Periodicities may exist locally or with complex frequency modulation.



The proper and most used frameworks to characterize, filter or synthesize each type of noise are:

• For deterministic, periodic signals/images: (Spatial) frequency-domain techniques, mainly Fourier analysis, power spectrums and periodograms.

• For deterministic, quasi-periodic signals/images: Wavelets or other frequency-spatial domain techniques, such as the short-time Fourier Transform.

• For transients: Wavelets or other frequency-spatial domain techniques; impulse/step response analysis and non-linear apporaches.

• For stationary stochastic signals/images: Statistical analysis, Auto-Regressive Moving average (ARMA) models, stochastic-process frameworks and Principal Component Analysis.

• For non-stationary stochastic signals/images: Independent Component Analysis, Markov-Random field analysis and stochastic-process frameworks.

• For chaotic signals/images: Dynamical complex system analysis (linear and non-linear), analysis of attractors in phase-space and fractal analysis, since these patterns have scale-invariance. An example of fractal feature that meaningful in a fractal signal is the correlation dimension.

A highly complex texture may be a mixture of all six kinds of noise, besides texels! 7. Color Textures. In a simplistic approach, a color image may be treated as three different B&W images, corresponding to RGB components (red, green. blue color channels), or HSL (Hue-Saturation-Intensity representation), or other color space. Thus, a colored-texture may be analyzed channel-per-channel (of any color space), as three textures. That is, different


textures may exist in different color-space channels, but the very nature of color perception makes it a far more rich field of study and inter-channel coupling may require ad-hoc tools to understand and characterize some color textures, specially from point of view of human perception. Some textural parameters for color textures are described in Tutorial 2. 8. Simple Texture Parameters (Statistical Approach) After defining a suite of edge detection and grouping algorithms at our disposal, the problem of texture segmentation turns into a problem of texture classification. Given a method that can classify all the elements in an image as a certain type of texture, we can apply a general edge detector to divide different texture classified areas into the required segments. In Section 3 we presented in outline a series of six different characteristics to define a texture. We will shortly explore the first five of these in a mathematical sense and then investigate in Tutorial 2 the use of fractal dimensions as a texture classifier. The sixth classifier, roughness, is more a psychological test and has been mathematically defined as the sum of the coarseness and contrast. We have shown above how grey scale images can use their intensity values as a simple form of texture measure. If each pixel in an image can be classified with reference to its neighbors and given a numeric value, this can be substituted as the image's grey scale value at that point and all the above techniques can be applied. We now present a short aside to show some simple statistical measures that can be used.


Rank Operators A rank operator works on ranked values of intensity in a small region. Once ranked, we may extract parameters calculated form the maximum, minimum and median values, as well as percentile intervals (the first quartile (of four) is from the minimum to 25% of the maximum values, and the interquartile starts at 25% and ends at 75% of the maximum. One of the simplest texture operators to detect different textures, like those present in microscope images of histological samples, is the range or difference between minimum and maximum brightness values in the neighborhood of each pixel. This produces a range image, where gray levels are the range values. A suitable threshold allows to produce a mask to separate textures in the original image. The size of such neighborhood must be large enough to include small uniform details (note the similarity to the sampling Nyquist criteria). For a uniform, soft region, the range is small or 0. A surface with larger roughness gives larger values of the range. Another operator producing similar results on the same images, is the statistical variance, calculated on a moving neighborhood. It is described in the following paragraph.


Statistical Moments The first statistical classifiers that can create this measure are simple moments. Given an image I we can calculate its local mean value over a region or window W, centered in the image position (i,j) as

1i, j i-k, j-li,j

k,l WNμ

∈

= = ∑I I (4)

Alternatively the mode or median value can be used instead of the mean. The size of the averaging region W allows to define what details are not texture (since they are larger, they will not be smoothed out by this low-pass filtering). Thus, the textural details will be the residual information, either after subtracting μi,j or considering the variance, below). Robust weighted averages may be also defined, including outlier rejection or outlier penalization. The local variance, σ2 (with a window neighborhood centered at pixel i,j), is a measure that characterizes how the values deviate from the mean value,

( )212i, j i-k, j-l i, j

k,l WNσ μ

∈

= −∑ I (5)


The image σ2W may already provide a “map” of textural regions, if they have different local

variance. It is in fact, the simplest textural feature, and depends on the size of window W. A whole family of local moments can be calculated and used:

( )1

n i, j i-k, j-l i, jnn

μ μ= −I (6)

The standard deviation, σ2, is equivalent to the second moment. Two other popular statistics are skewness,

( )3

31 1

i, j i-k, j-l i, jk,l Wi, j

N-1s

σμ

∈

= −∑ I (7)

and the kurtosis,

42i, j

i, ji, j

μ

σα =

(8)

Skewness (“biais” in French, “Schiefe” in German and “sesgo” in Spanish) is a measure of the shape bias (asymmetry) of the distribution of values {Ii-h, j-l}. The fourth moment 4 i, jμ is also called flatness or relative monotony. The derived feature called Kurtosis (ec. 8) measures the concentration –sharpness- of the shape (peak) of the distribution: a large value correspond to ecto-peak shapes (above the Gaussian), a medium value (about the std. dev.


value) correspond to meso-peak shapes (similar to the Gaussian) and a small value (sharp, under the Gaussian) are called endo-peak shapes. Higher-order moments bear information without a simple interpretation relating to the shape of the distribution. Note that once we define a local-average image μi,j which constitutes a “bakground” we may define a second image by background removing (pixel by pixel): Idetails i,j = Ii,j − αμi,j , with α ≤ 1 a parameter with typical values 1.0 or 0.5 (degree of background removing). Idetails enhances textural features under half the size of the averaging window.

A related measure calculated from the image derivatives is the total variance, TV. This consists of the sum of gradient values throughout the window W. Any of the different gradient operator, G, can be used, with Rudin and Osher presenting a preference for G’:

( )i, j i-k, j-lk,l W

TV u∈

= ∑ G, (9)

where x x y y⋅ + ⋅G = D D D D is the image gradient magnitude and Dx,y are the horizontal and vertical derivatives. Sobel and Kirsch operators for enhancement of oriented edges In its continuous version, G constitutes the Sobel operator and is also written as:


22

x x y y x y⎛ ⎞∂ ∂⎛ ⎞⋅ + ⋅ = + ⎜ ⎟⎜ ⎟∂ ∂⎝ ⎠ ⎝ ⎠

G = D D D DI I

(10) In their discrete version, they correspond to directional convolution kernels [-1,0], [-1,0]T, [1,0], [1,0]T or [0,+1], [0,+1]T, [0,-1], [0,-1]T, or [-1,0,1], etc., since (first example, finite backward difference) Dx = ∂I(x,y) / ∂x ≈ Δ I(x,y)/ Δx = I(x,y)- I(x-1, y), since at pixel level, we set Δx=Δy=1. We also may define the forward finite difference ∂I(x,y) / ∂x ≈ I(x+1,y)- I(x, y), and the average, as well as diagonal mixtures, giving all rise to the known Prewitt and Roberts operators for edge detection. A direction value, for each pixel can be also calculated, in adition to the magnitude of the Sobel operator:

Direction angle arctan yx

⎛ ⎞∂ ∂= ⎜ ⎟∂ ∂⎝ ⎠

II (11)

A combined image of magnitude and direction information, from the Sobel gradient operator, can be obtained by coding direction as Hue, and magnitude as gray level Intensity. A common approximation of the discrete Sobel gradient operator is exemplified by the left-to-right and the top-down 3×3 edge-detection kernels, where the boxed element indicates the origin location:


1 0 1 1 2 12 0 2 and 0 0 01 0 1 1 2 1

− − − −⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥−⎢ ⎥ ⎢ ⎥−⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦

(12)

The Kirsch operator is another practical way to avoid the square root of G, by applying each of the eight orientations of the derivative kernel and keeping the maximum. Both, the Sobel and the Kirsch operator allow to extract orientation information and to form orientation components of a texture. The Spatial Autocorrelation Function (ACF) In the hybrid approach (half structural, half statistical), the spatial size of the texels can be approximated by the width of the ACF in space domain defined as:

r(k, l) = m2(k, l) /m2(0, 0), where (13)

[ ]( , )

1( , ) = ( , )m n WW

iim k l m k n l

N ∈

− −∑ ∑ I

with i=1,2,…, and NW is the number of pixels in the moving window W.


When texels are more or less contrasted against the background, they become like grains of the texture and the coarseness of the texture may be expected to be proportional to the width of the spatial ACF. The latter can be represented by distances x0, y0 such that the ACF is r(x0, 0) = r(0, y0) = ½ . Different measures of the spread of the ACF can be obtained from the moment-generating function:

1 2( , ) ( ) ( ) ( , )k l

m n

M k l m n r m nμ μ− −∑∑ (14) 9. Statistical Moments from the Grey-Level Histograms. A very similar statistical approach to first-order statistics of grey-levels directly from an image, is the use of statistical moments from the grey-level histogram of a image or region. The histogram refers to (1) the graphic plot of grey-level intensities against the event incidence of them (frequency of occurrence of any particular intensity) and (2) an approximation of the Probabilistic Density Function (probabilistic distribution), or, in practice, the discrete probabilities p(ui) of grey levels ui , with i =0,…, L-1 (usually L=256), in an image or region. We rewrite the above definitions, referring to the histogram p(ui):

1

0( ) ( ) ( )

Ln

n i ii

u u u p uμ−

=

= − < >∑ (15)

where uM is the statistical mean value of u (the statistical average gray level or expected value):


1

0( )

L

i ii

u u p u−

=

< >=∑ (16) Note the weighting probability (histogram); the mean is not over pixels, as in a sample average, but over all possible intensities according to their frequency p(ui). Since histograms lost any domain information, when such domain is location of the attribute (instensity values), a global histogram may be exaclty the same, regardless of shape and texture, provided the distributions are identical. In binary images where there is the same proportions of black and pixels (say half and half), the histogram consist of two peaks. The figure shows examples of images with the same graylevel distributions.

Block pattern Checkerboard Diagonal stripped pattern A shape

Figure. Four different textures with the same global distribution (histogram)

of black and white.


As defined, we note that moments of order 0 and 1 are μ 0 = 1 and μ 1 = 0. The second-order moment is the variance σ 2(u) = μ 2(u):

12

20

( ) ( ) ( )L

i ii

u u u p uμ−

=

= − < >∑ (17)

and is specially important as a textural descriptor, since it is a measure of gray level contrast that can be used to establish descriptors of relative smoothness. Histogram features are extracted from the histogram of an image or a region of an image. In the following it is required that Σk p(uk)=1, otherwise a normalization constant must be introduced (e.g., total number of pixels in the ROI).

Histogram features (histogram {p(uk)}) ( ) Prob[ ]uk kp u u = and uk ∈{u0 , u1 , …, uL-1}

max

max

med

max{ ( )},

argmax{ ( )}

median{ ( )}

k

k

k

k

k

p u

p u

p u

pu

u

=

=

=

Maximum probability,

Mode (most frequent intensity)

Median (sort p, chose middle)

(1)

1

10

[ ] ( )u uL

k kk

m E u p u−

=

= == ∑ First moment (centroid) or statistical average gray level (2)


1

0[ ] ( )u k

Ln

n kk

nm E u p u−

=

= = ∑ Moments of order n Order 2: Contrast (3)

1

0

ˆ [ | | ] | | ( )uL

n nn k

km E u p u

−

=

= = ∑ Absolute Moments of order n (4)

( )1

10

[[ ] ] ( ) ( )u uLn n

n k kk

E E u m p uμ−

=

= = −− ∑ Central moments of order n μ 0 = m0 =1 and μ 1 = 0. (5)

12 2

2 10( ) ( )

L

k kk

u m p uσ μ−

=

= = −∑ Variance = Central moment of order 2 (6)

1

10

[ˆ [ ] ] ( )u uLn n

n k kk

E E u m p uμ−

=

= = −− ∑ Absolute central moments (7)

1

0

( )[ ] , 0uL

knn kn

k k

p um E uu

−−

−=

= = ≠∑ Inverse moment of order n (8)

12

0( ) [ ] ( )u

L

kk

E p p u−

=

= = ∑E Uniformity or Energy (9)

1

2 2[ log ] ( ) log ( )L

k kk

H E p p u p u−

= − = −∑ Entropy (in bits) (10)

Note that in the latter, the domain does not appear; equations refer to histograms of a set of samples, a signal, an image or any nD domain with discrete attribute uk. Most common histogram features have a name:


Summary of most common histogram features

1

24

1

med

max

22

2

3

ˆ

muu

m

μ

μ σ

σ μ

μκ

=

=

Mean or statistical average gray level Median Mode Dispersion Variance Contrast or Mean Square Value or Average Energy Skewness Kurtosis

10. Texture Measures Based on Human Perceived Features We now return to the six basic features of textures, based on psychophysical studies of the HVS, and mathematically define the main five texture classifiers from Tamura that were mentioned in Section1. 1. Roughness It corresponds to one of the simplest and most employed measure of local variation. It was also introduced as a statistical moment, with a different notation. The equivalent to surface-physics RMS-roughness models image intensities as heights in the surface-intensity representation S= { ( I, hi,j =I(I) ) }. We first obtain a local measure of average height (1st


moment of the image intensity distribution) and then obtain the 2nd moment (standard deviation):

1i, j i,j i-k, j-lW

k,l Wh h h

card W ∈

= = ∑ (18)

the cardinality of window W is the number of pixels. Note all correspondences with equations 1 and 2 of Section 8, and how interpretation and notation change. Typical sizes of W should be at least as twice the size of details to be considered as texture.

( )212W,RMS i, j i-k, j-l i, j W

k,l Wh h

card Wρ σ

∈

= = −∑ (19) Local measures can be made “robust” by discarding from the window those pixels/values that do not pertain to the texture (noise, artifacts, edges), and by weighting heights (intensities) by other image properties (see below). Note that ρ RMS (i,j) is an image where black indicates un-textured areas and white very busy, noisy or rough areas. An elemental “texture segmentation” can be employed in textures where roughness is the predominant feature. Several windows may be tested and obtain a pyramidal or scale-space set of features. The Ration of Extrema Density is a normalized local measure of roughness in 1D (profiles), introduced by Haralick et al. In RED, shape properties (width of the peaks) are combined with intensity: the local maxima in a very small window W are obtained for a larger ROI:


{ } { }max( ) min( )

arg max arg min

1REDi-k, j-l i-k, j-l

k,l Wk,l W

i-k, j-l i-k, j-lk,l W k,l W

ROI,Wk,l ROI

h h

h hcard ROI∈∈

∈ ∈∈

−

−= ∑

(20)

Since it only measures one-dimensional profiles of 2D images, it is an example of stereological parameters (see Tutorial 2), as well as an example of the surface-approach, where profiles of relieves are studied. Our group [Corkidi etal, 1998] introduced a slight variation in RED, by weighting/penalizing the width terms (numerators), and sampling extrema by 1D profiles, under a sterological-approach justification. The new textural parameter is called MRWH, the mean ration of width (ξ

⊥, ζ ⊥) and height ((ξ | |, ζ | |, or depth):

i i

|| ||

i i

ξ ζ

2 (ξ ζ )

( )1MRWH iROI,W

i rows ROI

w

card ROI

⊥ ⊥

∈ ⊂

+

+= ∑ (21)

with

min( , )max( , )1 ( ) 2

0

i i

i ii

ξ ζξ ζ i i

if and ξ ζw

otherwise

⊥ ⊥⊥ ⊥

⊥ ⊥

⎧> γ + > η⎪= ⎨

⎪⎩


where γ and η are parameters to be adjusted, according to the kind of peak to be accepted; we used γ=0.6, independent of scale, and η depends on the average gap between peaks of interest. In the following paragraphs, the choice of feature parameters correspond to physiological perception experiments. 2. Coarseness, coarse versus fine Coarseness is a measure of scale in micro texture within the image. A simple procedure was presented by Rosenfeld that detects the largest size at which repetitive patterns are present. • Take averages at every point in the image at different scales, k, which are a power of 2.

This defines the arrays

1 1

11 11( ) ( )

k k

kk2k

ki x-2 i y-2

y+2x+2A x,y g i,j 2

− −

−−

= =

−−

= ∑ ∑ (22)

• Calculate the differences between neighboring averages which are non-overlapping. This is calculated in both horizontal and vertical directions.

1 1( ) ( , ) ( , )k k

h,k k kE x,y A x-2 y A x+2 y− −= − (23)


1 1( ) ( ) ( )k kv,k k kE x,y A x, y-2 A x, y+2− −= − (24)

• Then pick the largest size which gives the maximum value,

{ }1 1 2 2 k kk( ) 2 where max , , , ,..., , ,...n

h, v, h, v, h, v,nS x, y = , E E E E E E E= (25)

• Finally take an average as the coarseness measure, ( )crsF = S x, y . Note that S gives also the scale at which variation attains a maximum.

Coarseness, as well as other textural features can be quantified (mathematically characterised) by other parameter definitions. 3. Contrast, high versus low Two of the four factors, presented previously, have been proposed to influence the contrast within an image. 1. Dynamic range of grey levels, specified most commonly as the (local/regional/global) standard deviation σ2.


2. Polarisation of the distribution of the black and white on the grey level histogram. The kurtosis, α4=μ4/σ2, is used where μ4 is the fourth moment about the mean and σ2 is the variance. Skewness is also defined in terms of μ3 . Combining these two factors, we have a measure for contrast, Fcon = σ/(α4)n. A value of n = 1/4, has been recommended when compared with physiological comparisons. 4. Directionality, directional versus non-directional The values defined above for θ(G), can be quantized into a histogram, Hd. The position and number of peaks can be used to extract the type of directionality. To quantify this feature, the sharpness, or variance, of, np peaks, positioned at angles φp can be calculated. By defining wp as the range of angles between two valleys, we have without normalization,

2

( )( ) ( )

p

p

n

p p di, j w

dirF = n Hφ

φ φ φ∈

−∑ ∑ (26)

Other measure of local directionality, orientation and coherence is the local tensor of inertia

I(x,y) (a feature not corresponding to physiological perception models) defined on a window W as an image whose attributes is a tensor:

yy

xx xy

yx

ℑ ℑℑ ℑ

−⎡ ⎤= ⎢ ⎥−⎣ ⎦ℑ

, (27)


where the 2nd normalized moments are defined as:

,

,

,

2

,

2

,

,

( , )

( , )

( , )

( ) ( , )

( ) ( , )

( )( ) ( , )

x y W

x y W

x y W

x y W

x y Wyy

x y W

C

xx

C

C C

xy yx

x y

x y

x y

y y x y

x x x y

x x y y x y

ℑ

ℑ

ℑ

∈

∈

∈

∈

∈

∈

−=

−=

− −= ℑ =

∑

∑

∑

∑

∑

∑

I

I

I

I

I

I (28)

where local centroids are obtained from: , ,

, ,

( , ) ( , ),

( , ) ( , )x y W x y W

x y W x y W

C C

x x y y x yx y

x y x y∈ ∈

∈ ∈

= =∑ ∑∑ ∑

I I

I I


Note: some authors do not normalize the moments but we cannot asume that ,

( , ) 1x y W

x y∈

=∑ I

for any window W. A second normalization from dimensional analysis may be needed for a correct physical interpretation.

Since the inertia matrix ℑ is diagonal when computed with respect to the principal axes (in the eigenvalue system of coordinates), the centroid and the principal axes completely describe the local orientation of an arbitrary neighborhood W of point (x,y). If we extract the eigenvalues λa, λb, ordered by length (λa ≥ λb), and the corresponding eigenvectors av , bv we may define the:

Local directionality (eccentricity): Y = λa / λb , where λb >0 , (29) and the

Local anisotropy (local orientation ): x

y

proy

proy =argtan a

a

θ⎛ ⎞⎜ ⎟⎜ ⎟⎝ ⎠

vv (30)

In stead of calculating ℑ(x,y) on a window of an image I(x,y), several authors obtain it from the gradient ∇I(x,y). 5. Line-likeness, line-like versus blob-like An adaptive tensor of inertia may provide a line-likeness measure: the eigenvalues for a circular neighborhood or window are used to build a new elliptic oreinted window, where a


second tensor of inertia is obtained, and eccentricity may increase, making evident a line-likeness structure. Other methods allow for direct line-like or blob-like assessment. Having calculated G and O(G), for all locations, a definition of line-likeness considers how probably the direction, at a specific point, is similar to one at a certain distance away. A matrix, Pd(i, j), is defined as the relative frequency with which two points on an edge, separated by a distance d, have direction codes i and j. A measure of line-likeness follows from:

( ) cos(( )2 / )

( )

n ndi j

n ndi j

lin

P i,j i j nF =

P i,j

π−∑ ∑∑ ∑ (31)

6. Regularity: regular versus irregular Four different features have been described so far, and regularity can be thought of as how much these features change over an image. The standard deviation of these four features (excluding roughness, which was not considered in physiological models of texture perception) can be used, so

= 1 ( + + + ) reg crs con dir linF σ σ σ σ− (32)


In general, outside the context of physiological perception studies, similar measures has to be normalized in some way and the normalization depends also on excluding or including other variation terms:

feature1

1 = 1N

reg nnN

F σ=

− ∑ (33) There may be other regularity measures based on the coefficient of variation of a given parameter β , as

CVβ = σβ /μβ. (34) Entropy

A commonly used indicator to describe the information content of any set of data is Shannon's formula for entropy. It is worth pointing out that Shannon's entropy was originally defined for all information streams and it also serves as a mesure of disorder, since pure random noise should not contain any information, by being completely disordered. We will consider it here in terms of image data.

1

1 20

1H logN

n nn p

p−

=

=−∑ (35) This gives order-1 entropy1 where pn is the probability of pixel value n, in the range 0...

N−1, of occurring. This formula results in a single number that gives the minimum average


code length if every pixel is encoded independently of the other pixels. Entropy is in effect the information content of the image.

Order-m entropy can also be calculated by extending the formula. By defining Pm( x1, x2,…, xm) as the probability of seeing the sequence of m pixels x1, x2,…, xm we have

( )( )

1

20

1H logm 1 2 m1 2 m

N

m 1 2 mx ,x ,...,x

m P x ,x ,...,xP x ,x ,...,x

−

=

=− ∑ (36)

And defining <⋅> as the expected value, the full entropy of data is given by

( )21 1H lim log

m 1 2 m

m m P x ,x ,...,xm→∞

=− (37)

This limit always exists if the data stream is stationary and ergodic. These numbers can be calculated on a local window of values and used as a measure. _______________________________________________________________________________ 1There is some disagreement in terminology and although it is called first order entropy it is often also written as order-0 entropy.


11. Textural Descriptors from Gray Level Co-occurrence Matrices. A very important and widely used set of textural descriptors, in the statistical approach, are obtained from information regarding the relative position of pixels with respect to each other. This information is not carried by individual histograms and is considered as second order statistics, since two joint random variables are considered: the intensities of a reference pixel and a second one, in a different position (an offset Δx, Δy). Such information is present in bi-dimensional histograms of the simultaneous occurrence (hence, co-occurrence, or concurrence) of two pixel intensity bins i, j. Note that i,j do not refer to pixel coordinates. The size of the bidimensional histogram bins (classes) is also very large; a 0-255 range of gray levels is tipically quantized to only four gray-level classes (say, 0-63, 64-127, 128-191, 192-255). They are thus reduced to 4×4 arrays of probabilities pi,j, hence, the term of concurrence or co-occurrence matrices, and where introduced first by Robert Haralick. Concurrence of pixels is also known as spatial dependence. The co-occurrence of two pixels requires the specification of a simple spatial relationship or placement rule (hence, its relationship with the structural approach). These spatial relationships define each one a particular co-occurrence matrix and mainly include: adjacency or short separation, at a given orientation. As for gray levels, the possible ranges of such separations and angle orientations are reduced to few discrete values (1 to 3 pixels


separations and angles of 0o, 45o, 90o, 135o, 180o, 225o, 270o and 315o, but often reduced for symmetry to the first 4 directions).

Fig. Simple pair relationships can be coded by signed offsets (Δx, Δy) from the central pixel

at (x,y), for example the upper left corner has offsets (-4,4).

The co-ocurrence matrix entries hij, for each offset combination (Δx, Δy), are obtained from accumulating a count of pixels (see also eq. 40) satisfying:

( ) ( )1 1

0 0

1 ( , ) AND ( , )0

N Mi j

ijx y

if I x y u I x x y y uhotherwise

− −

= =

⎧ = + Δ + Δ =⎪= ⎨⎪⎩

∑∑ (38)

where image intensities are still in the range [0,255], but the entries i, j correspond to bins according to quantization (usually in four classes, thus, i, j ∈ [0,3] and for example ui ∈


[i×64, 63+i×64]. Note that the “AND” condition expresses the joint occurrence of intensities i, j.

Fig. A co-ocurrence 4×4 matrix is a histogram of 4×4 bins.

Remember: For each pair of pixels tested, a 4×4 co-occurrence matrice is a bidimensional histogram with 4×4 classes (quantized intensities) for each pixel, one the reference or independent site (position and angle 0), the second at the dependent site (position L and angle θ, also quantized). There is ONE matrix GLCM per pixel configuration and each entry i,j is the count of pairs having the first pixel the intensity i and the second pixel the intensity j. Bins or classes partition all gray levels in N×N array (4×4 if 4 bins are chosen). An image

12 - 10 - 8 - 6 - 4 - 2 - 0

⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦

2 4 5 48 6 2 124 3 10 43 2 2 1

172

Normalize 1i ji j

p =∑∑

hi j

j

i

i j

i ji j

i jh

hp =

∑∑


with gray levels in [0,255] has four clases described as intervals: [0,63], [64,127], [128,191], and [192,255]. Given one configuration (a pixel and its diagonal neighbor, for example) the first pixel has a gray level of 68 and the second a gray level of 180, it is in entry bin 2,3 of the matrix for that configuration, and the count of that entry is incremented.

10 01 11 11

i j i i i

j j j

p p p pu u u u u

u u u

−

Most authors present the matrices with a gray level normalization of I(x,y) in equation (38), replacing intensities ui and uj by their respective indices i,j which are directly the entry indices in the GLCMs (which, by the way seems clearer):

p10 p01 p11 p-11 i j i i i j j j

The difference is of course a look-up table from indices i,j to (quantized) intensities ui,uj. We combine both notations in the following example:


1 1 0 01 1 0 0 4 0 2 4 0 2 2 0 2 2 1 10 0 2 2 2 2 0 2 2 0 2 1 1 1 1 00 0 2 2 0 0 2 0 0 2 0 0 1 1 0 1

Image :

j j j j

i

H V D D

0 1 2 0 1 2 0 1 2 0 1 2

0 0 0 01 1 1 12 2 2 2

I GLCMs M M M M

u u u u

u

−

⎧⎪⎨⎪⎩

Fig. (Top) Notice that notation as “offsets” to the right (zero or positive 1), or left (negative -1) and down (1). (Middle) Four different co-occurrence matrices for a gray-level image (levels 0,1,2). (Bottom) The

gray level pattern, before quantization in three level bins.

The normalization constants are, for each matrix: 1/12, 1/12 and 1/9. Thus, for example the energy of matrix MD is: E= 1/81 (4+0+4+4+1+1+0+0+1) = 15/81 = 0.1852. In general, GLMCs matrices can be obtained not only from gray-level values of pixels, but also for any local feature or code assigned to one pixel (and corresponding perhaps to local


information), or from unstructured data. An example are run-lengths, which are codes obtained from the length of pixels along a row, which are connected if their gray-level lies within an interval. A new run-length is registered if the chain is “broken” (a pixel lies outside the tolerance interval). We summarize some dimensions, ranges and facts that often seem confusing, when they coincide:

• Image dimensions are N×M, and intensity ranges over [0,umax]. • Co-ocurrence histograms = co-ocurrence matrices (GLCMs) are L×L (always square),

where L is the number of gray levels in which [0,umax] is reduced (quantized), usually to just 4 gray levels , thus GLCMs are often 4×4.

• Pixel pair combinations: It is common to use 4 per offset pair, with offsets (Δx, Δy) starting at 1, 2,..., Offmax: horizontal, vertical, diagonal (45°), and anti-diagonal (135°). So, there are 4 GLCMs for each offset (|Δx|, |Δy|) ≥ (1,1). Offmax is determined by the size of the largest texture features, that is, when a texture disappears if a Gaussian blur of sigma Offmax is applied.

• The GLMC entry (i,j) ∈ [0, L-1]×[0, L-1] has the count of pairs of pixels in which the central (reference) pixel has intensity i (note the normalization, with 4 classes, there are 4 intensities 0,1,2,3) and the second pixel, at the selected offset/direction (for that matrix) has intensity j.

Finally, the original gray levels do not enter into any calculation, just the classes i, j, in some co-ocurrence features.


Fig. (Left) Gray level quantization from 256 levels to 4, corresponding to intervals [0,63], [64,127], [128,191], [192,255]. The gray shades chosen to represent each bin are values 0, 85, 170 and 255. (Right) Compare the latter with quantization from 256 to 16 gray levels in intervals [16i,(16i-1)],

with i = 0,..,16. A GLCM using this quantization would have 16×16 bins (entries).


Some co-occurrence matrix features (bidimensional histogram {pi j })

( , ) Prob[ , ]u ui j i j a i b jp p u u u u = = = i, j ∈ {0,…, L-1}

( ),

max i ji jp Maximum probability (1)

( )ki j

i ji j p−∑∑

Element difference moment of order k Order 2: Contrast (2)

( ),k

i j

i j i j

pi j

−≠∑ ∑ Inverse element difference moment of order k (3)

2i j

i jp∑ ∑ Uniformity or Energy (4)

2logi j i ji j

p p−∑∑

Joint Entropy 1 1

20 0

( , ) ( , ) log ( , )L L

i j i ji j

a bH u u p u u p u u− −

= =

= −∑∑ (5)

i j

i jp i j−∑ ∑ Dissimilarity (6)


1i j

i j

p

i j+ −∑ ∑

Homogeneity (7)

( )( )i j i ji j

i j

i j pμ μσ σ− −∑ ∑

Correlation (1st order) (8)

1 1 1 1

2 2 20 0 0 0

1 1

20 0

( , ) ( ) ( ) ( , )

log log log

log

L L L L

i i j j i j i ji j i j

L Li j

i ji j i j

a a ab b bMI u u H u H u H u u

p p p p p p

pp

p p

− − − −

= = = =

− −

= =

= + −

= − − +

=

∑ ∑ ∑∑

∑∑

Mutual information (9)

Marginal means and variances are also used.

Statistic Description

Contrast Measures the largest local variations in the gray-level co-occurrence matrix

Correlation Measures the joint probability occurrence of the specified pixel pairs


Energy Provides the sum of squared elements in the GLCM. Also known as uniformity or the angular second moment.

Homogeneity Measures the closeness of the distribution of elements in the GLCM to the GLCM diagonal. If only diagonal values are different of zero, then there is no texture (an homogeneous region), if no diagonal dominance exist at all, the texture is rather noise (or very irregular).

Mutual information

When pixel pairs share some property, this feature measures the strength of that common property. This measure is better suited for comparing two textures, and rather as a measure of similarity.

Equation (38) is impractical for actual computation of a GLMC; indirection is used by scanning both images pixel by pixel (x, y) and incrementing the count at the matrix entry indicated by image attributes I(x, y) and I(x+ Δx, y+Δy), after quantization to L levels (and probabilities normalization must be performed later):

1( , ) ( , ) ( , ) ( , )

0

( , )

( , )with

: 1

0,

t tI x y I x x y y I x y I x x y y

i j

x y

i j

h h

h

++Δ +Δ +Δ +Δ= +

=

∀

∀

(40)

A common practice is to increment only Δx, from 1 to Offsetmax, and then rotate the image a desired angular increment Δθ, from 0° to 180°. Two methods are used to speed-up the entry-filling of GLMCs:


• Sampling the textural region of interest, in stead of scan all pixels. Look-up-tables of offsets is effective when many offset/directions are to be explored.

• Using sliding windows (also known as roller buffers) in which a local GLMC (one per offset/direction) is updated in an incremental way. It is more effective for simultaneously considering several, large offsets.

Co-occurrence matrices are typically large (several bins), at least four per offset and sparse, and besides the co-occurrence matrix features mentioned (also called Haralick features), some sparse-matrix techniques may be further applied, to reduce some computations. 12. Binary Texture Analysis, Codewords and Local Binary Patterns Gray-level or color textures may be studied separating intensity bins as binary images, each one representing a particular bin (which in turns quantizes the full dynamical range into a few number of classes, typically from 4 to 20, which become as “channels”). Thus, binary texture analysis is a special case where many approaches exist, most of them based in information codification of local bit-by-bit patterns. In this approach, for example a 3×3 neighborhood in 8-connectivity is coded into a 8-bit word and a texture dictionary is then built, and statistics obtained from possible configurations. Moreover, GLCMs matrices can be obtained from the image of 8-bit codes extracted from a texture. The following figure illustrates (right) how a binary codeword (with decimal value of 180) is obtained for a textural pattern (left), reading bits clockwise:


0 0 1 codeword for pixel1 00 1 1

starting bit 0

bit 7 bit 0

:10110100 180

↓

=↑ ↑

xx

The central bit (in red at left) is ignored, making the codeword robust to pixel noise. To make an anisotropic codeword, gray-level intensities (before separation into plane-bits or channels) are interpolated as illustrated by the following pixel configurations, which also extends the radius R of the neighborhood to larger values:

Square neighborhood Circular neighborhood

(g3, g1, g7, g5 are interpolated)


A sequence of different radii R-neighborhoods is equivalent to sampled multirresolution. Gaussian filters may also be applied to obtain a pyramid of decreasing-resolution images. In a gray-level analysis similar to co-occurrence (only two pixels), the joint distribution of all eight pixeles is obtained as well as its 8th-order statistics. Texture descriptors in this approach are rotation-invariant. In the Local Binary Pattern (LBP) approach, in stead of gray levels, the differences with respect to the central value are kept, turning the new textural descriptors contrast-invariant. If only the sign is taken into account, it can be coded as a bit 0 for negative and 1 as positive, and again a binary code, and an 8-bit codword is obtained. A number of LBP features can then be obtained. Codewords, on the other hand, lead to vector-quantization and vector-support machine approaches as well as Self-Organizaing Maps (SOM)-based classification.

13. The Variogram Given a separation h, and an orientation θ, the variogram γ (h,θ) is the distribution of the average quadratic difference (variance) in a property (image intensity) at pixels separated by h (lag) at orientation θ. In cartesian coordinates, we define vector (hx,,hy) = (h cosθ, h sin θ ) and define mathematically:


( ) 2

2

1( , ) [ ( , ) ( , )]21 [ ( , ) ( , ) ( , ) ]21( ) [ ( ) ( ) ]2

x y x y

x y

h h Var x y x h y h

E x y x y h h

γ

γ

= − + +

= − +

= < − + >h x x h

I I

I I

I I (40)

A variogram is independent of location (x,y) and allows to detect features that change with orientation θ and separation h. Given a fixed orientation θ, a variogram is also the plot of γ

(h,θ) against lag h (similar to an histogram definition). In practice it is estimated by variance in intensity at N samples of the pairs of pixels:

2* 1( ) ( ( ) ( ))2 ( )

i j

i jNγ

− =

= −∑hx x

h x xh

I I (41)


where ( )N h is the number of vector-separations (or number of samples of pixel pairs separated at h at orientation θ) to be considered (as the number of classes in a histogram). Samples are usually taken as multiples h,2h, 3h,...,Nh.

Variogram for two displacements lags h1, h2

( )2+x hI2h

1h

( )1+x hI

( )xI



Variogram map: An image integrating variograms at several angular bins (in polar coordinates) or made by “pixels” γ (hx, hy):


Isocontour plot (from a more detailed version of last color plot)


Variogram features Sill and Range in terms of correlated pixels

14. Principal Component Analysis (PCA) of Discrete Sample Texels.

A difficult feature of any texture is the quantification of the answer to this question:

“to what extend a texture has structural-approach properties”.


A related problem is, given a (structural) texture, to identify (or to obtain the “best fitting” of) texture primitives (texels), which may be extended to small building regions that repeat under some positioning rules (to be found, too). A qualitative answer to the first question is the complexity of the texel or texels and the complexity (length of description) of the positioning rules… if they are known! If texels exist, they eventually repeat, even if only with statistical variation. If repetition includes repeated spatial separations, then the Fourier Analysis and other unitary transform-based analysis are suitable tools to establish positioning rules and maybe to identify texels. If repetition is rather random, and positioning rules include texel rotation, scaling and gradients in spacing, it is more difficult to “identify” such rules, as well as the texels. A few observations may lead to a new approach in a structural-based analysis of a texture. As first defined, a texture image generated by texel plus position rules, complies with shift-invariant properties. If a correspondence can be established, then the ACF-based analysis, can be used to characterize texels. However, under complex positioning rules, several degrees of freedom increase complexity of correspondence finding. Such degrees of freedom un positioning include: scaling, rotations, cuasi-periodic translations, gradients and non-linear deformations, such as shearing and perspective distorsion, to mention the simplest. Under simple, known variations (rotation and translation (even if random), for example) a texel can be isolated by identifying a group of pixels exhibiting certain invariant properties, which somehow repeat in a given image. In a first approach the texel may be defined by its


gray level, shape or homogeneity of some local property, such as size, orientation, or second-order histogram, as defined by the co-occurrence matrices. Texels may be re-defined in a discrete image as any sub-image region(s) that either forms a texture by tiling the plane somehow, or spatially correlates with other regions, regardless of what lies between. Such regions will be also called “sample texels”, and may be K×K rectangles, where K is smaller than image dimensions N,M, say, by at least an order of magnitude. The simplest case, when the regions are single pixels, has already been formulated as “pixel co-occurrence analysis”, and all extracted parameters from the co-occurrences histograms (matrices). Co-occurrence of larger regions (sample texels) becomes more complicated, with exponential combinatory. A way to extract correlation information between small regions (say, sub-images A ⊆ I of K×K = 4×4 pixels), is by abstracting the sample texel as a sample vector (in fact a random-vector variable) of gray-level intensities in a K2 dimension space. The order of vector components (coding) is irrelevant, as long as the same order rules are used. We may use the following lexicographic order:


A = coded as the 16D vector:

(a11, a12, a13, a14, a21, a22, a23, a24, a31, a32, a33, a34, a41, a42, a43, a44),

a11 a12 a13 a14

a21 a22 a23 a24

a31 a32 a33 a34

a41 a42 a43 a44

and re-indexed as column vector

X = [x1 x2 … x16] T

Any other ordering choice is equally useful, but should be the same for all samples. If a vector is returned to image space, it has to use the inverse ordering rule. Suppose now that we obtain L sample texels An with n=1,2,…,L, and code them into the random-variable vectors X n with n=1,2,…,L, then we define the L×L covariance matrix:

( )( ) n n= − < > − < >X X X XΣ T for all vectors n = 1,2,…,L. (45)

Where < X > is the expected value (or vector average) of vectors { X n }, with n,m = 1,2,…,L. Note that < X > is a vector, not a scalar (it is not an “average gray level”). Note than in one dimension (X n

T = (x1) n), Σ becomes the scalar variance σ2 = 1/L Σ(<x> −x1 )2. In the nD case,

the (n,m) element of Σ is the usual covariance between random variables xn and xm. (there are


n2 such pairs). In literature on statistics there are two conflicting notations for Σ: var(X) and cov(X ) (or auto-covariance), and the two random vector extension: but also there is the “cross-covariance”, cov(X, Y) (for which cov(X, X) coincides with var(X). We now perform a Principal Component Analysis (PCA), where the diagonalized Σ will show in its diagonal the de-correlated principal modes of variation of the texel samples. So, a texel pattern will correspond to the strongest few principal eigenvectors. If no dominant eigenvectors exists (the eigenvalue “spectrum” is spread over the vector space), then no evident texel pattern exists. The later analysis is very powerful and we have only sketched the starting steps. The structure properties of covariance matrix Σ allow to quantify, by sampling many K×K regions of the image, the degree of structural-ness (existence of texel primitives represented as K×K arrays of pixels) of a textured image. It allows to count and identify such texels, and several properties of the eigenvector spectrum would modulate their importance and repetitiveness. PCA may be as well applied to vectors of textural features, to know and quantify the dominant features, by defining a feature space, where similar textures are defined by short Euclidean distance separations (or other metrics) and different textures are defined by large distance separations in that space. Neural networks may also work on feature spaces, to extract vector descriptors for classification tasks, where the texture class has a complex shape in that feature space.


Other methods and algorithms (see Tutorial 2):

• Porous-material properties - percolation, mean length of branches • Mathematical Morphology filtering (granulometries). • Stereological approach (also considering stochastic geometry) • Spectrograms and spectral power features • Spatial information + co-ocurrence GLMC of length-code features • Mosaic models (as Voronoi Zones Of Influence). • Law bank of filters, Gabor and Hermite-transform descriptions • Local Binary Patterns II • Markovian random field descriptions and methods • Dynamic systems (attractors) and Fractal analysis • Surface-intensity or 3D-approach - facets • Inter-texture and multi-texture analysis • Color-Science image processing and 3D textures.

Documents

texture tutorial1CHK may08 · In computer graphics, texture refers to two notions, distinguished by context: the tactile, 3D ... geometry can act as an alternative approach to the