A Q-Ising model application for linear-time image segmentation

Cent. Eur. J. Phys. • 8(5) • 2010 • 689-698DOI: 10.2478/s11534-009-0165-y

Central European Journal of Physics

A Q-Ising model application for linear-time imagesegmentation

Research Article

Frank W. Bentrem∗

1 Marine Geosciences Division, Naval Research Laboratory, Stennis Space Center, Mississippi 39529 (present address)

2 Center for Bio/Molecular Science and Engineering, Naval Research Laboratory, Washington, DC 20375

3 Department of Physics and Engineering Physics, Tulane University, New Orleans, Louisiana 70118

Received 9 November 2009; accepted 30 November 2009

Abstract: A computational method is presented which efficiently segments digital grayscale images by directly apply-ing the Q-state Ising (or Potts) model. Since the Potts model was first proposed in 1952, physicists havestudied lattice models to gain deep insights into magnetism and other disordered systems. For some time,researchers have realized that digital images may be modeled in much the same way as these physicalsystems (i.e., as a square lattice of numerical values). A major drawback in using Potts model methods forimage segmentation is that, with conventional methods, it processes in exponential time. Advances havebeen made via certain approximations to reduce the segmentation process to power-law time. However, inmany applications (such as for sonar imagery), real-time processing requires much greater efficiency. Thisarticle contains a description of an energy minimization technique that applies four Potts (Q-Ising) mod-els directly to the image and processes in linear time. The result is analogous to partitioning the systeminto regions of four classes of magnetism. This direct Potts segmentation technique is demonstrated onphotographic, medical, and acoustic images.

PACS (2008): 75.10.Hk, 89.20.Ff

Keywords: image segmentation • Potts model© Versita Sp. z o.o.

1. Introduction

Segmenting digital images into regions of distinct typeshas applications in a great many fields, e.g. medicalimaging [1–4], surveillance by synthetic aperture radar [6]and satellite, and underwater acoustic imaging (e.g. seeRef. [7]), to name a few. In general, image segmentation is∗E-mail: [email protected]

performed by classifying image regions by color, intensity,and texture. Of course, only the latter two are consideredin grayscale segmentation, which is the subject we con-sider in this article. While classification by intensity isa straightforward assessment of the brightness/darknessof an image pixel or group of pixels (as with histogramsegmentation methods [1]), texture classification is muchmore complex [1, 8]. Although it is difficult to define im-age texture precisely, we can say that it is the spatialrelationship of the intensities (i.e. “graininess”) of an im-age region. Clearly, there are indeed a large number of689

A Q-Ising model application for linear-time image segmentation

possible texture types in digital images.A grayscale digital image may be represented as a matrixof numerical values, as in Fig. 1, which indicate the inten-sity or brightness (gray level) of the corresponding imagepixel. Techniques for image segmentation (such as thresh-olding and histogram methods) that focus solely on inten-sity are the most computationally efficient since they gen-erally require just one or two passes through the intensitymatrix. Identifying regions of different texture, however,tend to be relatively expensive in terms of computationtime [8]. Here I describe four types of magnetism usingthe Ising model [9] and show how its generalization, theQ-Ising (or Potts) model [10], can be used analogously tosegment grayscale images into four categories [11]. Thecommon approach (e.g. see [3, 6, 12, 13]) to segmenta-tion using the Potts model applies the Potts model to thecategorizing labels, i.e., the segmented image. (A sim-ilar approach is also used for more general data clus-tering [14–16].) The optimal segmentation is often foundusing a Monte Carlo approach, which processes in expo-nential time (time for computer processing increases expo-nentially with the number of pixels contained in the dig-ital image [12]). Use of the Bethe approximation [12, 17]allows an optimal segmentation to be obtained in power-law time. However, the direct Potts segmentation methodpresented in this article, which utilizes both intensityand texture, processes in linear time–among the fastestmethods currently used. (Other unrelated linear-time ornearly linear-time segmentation methods are describedin [18–20], though, these contain undesirable feature formany applications. For example, graph partitioning meth-ods [19, 20] cannot relate disjoint sets with equivalenttextures and require training sets.) This computationalspeedup arises from applying the Potts model directly tothe original image itself rather than to the segmentedrepresentation. The speedup is important for high-data-volume/high-resolution applications where time is con-strained.2. Ising model

In 1925, Ernst Ising proposed the model [9, 21, 22], whichnow bears his name, as a means to study phase transitionsin magnetism. Ising solved the original Ising model (onlytwo states–namely 1, or spin up, and -1, or spin down)in one dimension. However, the one-dimensional Isingmodel did not exhibit a phase transition, and the two-dimensional Ising model was not solved until 1944. At thattime Onsager [23] demonstrated that the two-dimensionalIsing model did, in fact, exhibit an order/disorder phasetransition. The Ising model and related models have been

Figure 1. On left, a grayscale image varies from black to white ingoing from top to bottom. The image is represented as amatrix of intensity values as illustrated on the right.

widely used to study magnetism and other order/disorderphenomena in physical systems. The two-dimensionalmodel is described here, and it will be used to definefour classes of magnetism. The image texture analogs willbe presented in a subsequent section.In the Ising model, given a two-dimensional lattice of mag-netic spins oriented either up or down as in Fig. 2 (numer-ically represented as a matrix of 1’s and −1’s), the totalenergy E associated with the lattice isE = −µH∑

isi − J

∑ij, nn sisj , (1)

where i represents a magnetic element, si is the valueof the magnetic spin of i (1 or −1), µ is the magneticmoment of the element, H is the external magnetic field,and J is the interaction coupling constant. The sum overall nearest-neighbors pairs i,j is represented by ∑ij, nn.Nearest neighbors are considered to be the elements di-rectly above, below, left, and right of a given element. If nocoupling interaction is present (J = 0) then µ > 0 createsan energetically favorable condition for the spins to alignin the direction of the external field H, whereas µ < 0leads to spins aligned in the direction opposite of H (seeFig. 3).

3. Four classes of magnetismIn common language, magnetism usually refers to the be-havior of magnets that attract iron (known as ferromag-nets). Other materials, however, reveal different magneticbehavior. These behaviors may be classified into the fol-lowing four groups:

1. Ferromagnetism is exhibited by materials for which,even in the absence of an external magnetic field, anet magnetization is present with north and southpoles. A ferromagnet will both attract and repelanother ferromagnet depending on the alignmentof their poles.690

Frank W. Bentrem

Figure 2. A spin matrix is shown on the left with arrows indicatingelements whose magnetic states are either spin-up andspin-down. The matrix on the right represents the samematrix with the arrows exchanged for 1’s and (-1)’s. In theabsence of an external magnetic field (H = 0), a ferro-magnet’s spins will spontaneously align as depicted in thetop half of the matrices, while the spins for antiferromag-nets will oppositely align (alternate in opposite directions)as depicted in the bottom half of the matrices.

Figure 3. Magnetic spins are aligned with (left, top) and against (left,bottom) the magnetic field H. These regions are associ-ated with paramagnetism and diamagnetism, respectively.

2. Antiferromagnetism is displayed by materials whichdo not form magnetic poles even with a modest ex-ternal magnetic field present. An antiferromagnetwill neither attract nor repel another antiferromag-net. Likewise an antiferromagnet will have no mag-netic interaction with a ferromagnet.3. Paramagnetism shows no spin ordering in the ab-sence of an external field, although the paramag-net’s spins align in the same direction of an exter-nal field when present. A paramagnet will neitherattract nor repel another paramagnet, but will beattracted to a ferromagnet.4. Diamagnetism also shows no ordering in the ab-sence of an external field; however, the diamagnet’sspins will align in the opposite direction of any ex-ternal field. A diamagnet will neither attract norrepel another diamagnet and will be repelled by aferromagnet.

4. Q-Ising modelThis section describes the Q-Ising model, also known asthe Potts model, which will be used to analyze grayscaleimages where the pixel intensities are considered in placeof the magnetic “spin” values. The Potts model generalizesthe (2-state) Ising model into a Q-state model by replac-ing the 1’s and -1’s with the numbers 0, 1, 2, 3, . . . , Q − 1(the pixel intensities in a grayscale image). According tothis model, the total energy associated with the system is

E = −µH∑i

(si − st)− J ∑ij, nn δ(si, sj ), (2)

where st is the spin threshold (separating light from darkwith 0 < st < Q − 1) and the kronecker delta functionδ (1 if the elements are equal, 0 otherwise) replaces thesummand for the second summation in Eq. (1). Now letssuppose that a magnetic spin matrix represents the inten-sities in a grayscale image and that a uniform externalmagnetic field, H = 1, is applied to the system. We wishto find a set of pairs (µα , Jα ) for Eq. (2) which character-izes the regions α in the spin matrix. Then each spin inthe matrix will be associated with a particular parameterpair. The “optimal characterization” is taken to be thepartitioning of the spin matrix in such a way that the to-tal energy of the matrix is minimized. Minimizing Eq. (2)requires the set of four parameter pairs (µ > 0, J > 0),(µ > 0, J < 0), (µ < 0, J > 0), and (µ < 0, J < 0). For con-venience, and without loss of generality, we simply use(µ1 = 1, J1 = 1), (µ2 = 1, J2 = −1), (µ3 = −1, J3 = 1),and (µ4 = −1, J4 = −1) for what we will call regionsα = 1, 2, 3, 4 respectively. The energies for the regionscan be written as

E1 = −∑i∈A1(si − st)−

∑i∈A1,j∈nn δ(si, sj ), (3)

E2 = −∑i∈A2(si − st) + ∑

i∈A2,j∈nn δ(si, sj ), (4)E3 = ∑

i∈A3(si − st)−∑

i∈A3,j∈nn δ(si, sj ), (5)E4 = ∑

i∈A4(si − st) + ∑i∈A4,j∈nn δ(si, sj ), (6)

where Aα is the set of all spins i labeled with categoryα . The j’s in the interaction terms are nearest-neighborspins that may or may not belong to Aα . Then the totalenergy E is simply the sum over all regions,

E = ∑αEα , (7)

691


where α = 1, 2, 3, 4 are the region segments assignedto categories 1, 2, 3, and 4, respectively. These ener-gies correspond to regions which are 1) paramagnetic andferromagnetic, 2) paramagnetic and antiferromagnetic, 3)diamagnetic and ferromagnetic, and 4) diamagnetic andantiferromagnetic.

(a) (b)

(c) (d)Figure 4. Sample grayscale image (a) with regions of visibly distinct intensity and texture. Upper right (b) is the segmented image using the

direct Potts method with Q = 16 and st = 7.5. At lower left (c), the segmented image from (a) is filtered with a despeckle algorithmto reveal four distinct texture categories. The categories are bright and smooth (white), bright and grainy (light gray), dark and grainy(dark gray), and dark and smooth (black). Lower right (d) is the despeckled segmentation result produced using the four-level histogramapproach. The histogram method does not properly distinguish regions of different textures, which demonstrates the problem in usingthe histogram approach for texture analysis.

692

Frank W. Bentrem

Minimizing the total energy (Eq. (7)) effectively partitionsthe spin matrix into regions corresponding to these fourclasses of magnetism. Reverting back to the grayscale im-age, these categories correspond to image regions whichare 1) bright and smooth, 2) bright and “grainy”, 3) darkand smooth, and 4) dark and “grainy”. The direct Pottssegmentation process requires the matrix to be separatedinto regions of these four types in such a way as to mini-mize the total energy. An example image with four distinctintensity and texture regions is depicted in Fig. 4a andthe result from the direct Potts segmentation is shown inFig. 4b. The parameters used in this example are Q = 16and st = 7.5, and the segmented image is shown with fourshades of gray representing the four texture categories.The segmented image is filtered using the ImageMag-ick1 despeckle algorithm and is shown in Fig. 4c. Forcomparison, a segmented image using a simple four-levelhistogram method2 is shown in Fig. 4d. The four-levelhistogram method performs poorly in distinguishing theimage texture regions.5. Algorithm

To segment a digital grayscale image, we first transformthe image into a matrix of intensity values si,j as illus-trated in Fig. 1. Since it is common for 8-bit digitalgrayscale images to contain 256 gray levels (0 ≤ si,j ≤255), reducing the gray levels to 16 ≤ Q ≤ 64 usuallyhelps in identifying visual textures. Next we must specifyan intensity threshold st , which separates bright pixelsfrom dark pixels. For each of the examples in this article,the threshold is simply st = (Q−1)/2, or halfway betweenmaximum and minimum intensity. (If texture identificationat a certain scale is desired, one may rescale the image.)Then to perform the energy minimization described above,we follow these two steps for each of the image pixels(i, j):1. If si,j > st then (i, j) is assigned either texture cat-egory 1 or category 2. Otherwise, (i, j) is assignedtexture category 3 or 4.

1 ImageMagick Studio LLC, Landenberg, PA, USA2 The four-level histogram method used for comparisonin this article is really a quantization into four equal in-tensity ranges. In practice, a heuristic is often used toset the intensity ranges. The Potts method under con-sideration could similarly use adjusted intensity ranges.In either case, the intensity ranges would then be imagedependent, which may be an undesirable feature.

2. Check the nearest-neighbor values (i.e., the pixelsdirectly above, below, left, and right) of pixel (i, j).If one or more nearest-neighbor values are equalto si,j , then pixel (i, j) is assigned texture category1 or 3. Otherwise, (i, j) is assigned either texturecategory 2 or category 4).This algorithm guarentees that the total energy definedby Eq. (7) is minimized and processes in linear time sinceeach pixel requires only a fixed number of computations.Only a single comparison and assignment required in Step1, and a maximum of four comparisons (nearest neighbors)and assignment required in Step 2. Note that since theenergy associated with each pixel is independent of thelabeling of its neighbors, choosing an energy minimum forevery pixel guarantees a total energy minimum (Eq. (7))for the entire image. Finally, the segmented image issmoothed by applying the ImageMagick despeckle algo-rithm. This serves to eliminate much of the statisticalnoise in the image.6. Results

Three examples using the direct Potts segmentation willnow be discussed using a photographic image, a medi-cal image, and an acoustic seafloor image. Each seg-mentation result is compared to a histogram segmenta-tion technique in which the image is partitioned into fouruniformly distributed intensity ranges. The four-level his-togram method is a very efficient, linear-time process butdoes not distinguish grainy regions from smooth regions.The photographic image in Fig. 5a3 contains depictionsof water, sky, vegetation, and rocks. Using step 1 of thedirect Potts segmentation method as described in the pre-vious section (with Q = 16 and st = 7.5), sky and rock aredistinguished from water and vegetation according to pixelintensities. The sky and rock are represented as lightershades, while water and vegetation are depicted as darkershades of gray in Figs. 5b and 5c. To further separate theshades of gray, step 2 in the previous section effectivelyidentifies regions of different texture (smooth or grainy).The full (steps one and two) direct Potts segmentationresult is shown in Fig. 5b. After filtering by using the de-speckle algorithm, we see in Fig. 5c that the sky (white,bright/smooth texture) is distinguished from the rock re-gions (light gray, bright/grainy), and water (dark gray,3 From the Berkeley Segmentation Data Set(http://www.eecs.berkeley.edu/Research/Projects/CS/ vi-sion/grouping/segbench/). Used with permission.

693


dark/smooth) is distinguished from the vegetation (black,dark/grainy). Segmenting the image by intensity usingthe four-level histogram produces the image in Fig. 5d.The histogram approach does not distinguish sky from rock(both are light gray) and does comparatively less well in

distinguishing vegetation (lower left of image) from thebackground. A comparison of Figs. 5c and 5d underscoresthe importance of considering both intensity and texturein image segmentation.

(a) (b)

(c) (d)Figure 5. Grayscale photo (a) showing water, sky, vegetation, and rocks. Segmentation of image using the direct Potts method (b) withQ = 16 and

st = 7.5. Image is from the Berkeley Segmentation Data Set (see [24]). The image at lower left (c) is a 4-level direct Potts segmentationof (a) that has been despeckled. Here the sky, rock, vegetation, and water are represented as white, light gray, dark gray, and blackregions, respectively. The image at lower right (d) is the segmented and despeckled image using the four-level histogram methodapplied to (a).

694

Frank W. Bentrem

(a) (b)

(c) (d)Figure 6. MRI of a tumorous brain at upper left (a). A tumor is seen as the bright spot in the upper left region of the MRI. This image is from the

Iowa Neuroradiology Library. Image at upper right (b) is the direct Potts segmentation with Q = 64, st = 31.5. Lower left (c) is the directPotts segmentation that has been despeckled. The clearly visible white segment identifies the tumor. A tumor is clearly visible as thebright spot in the upper left region. The periphery of the tumor is shown in light gray, while cerebrum is represented as dark gray. Otherbrain matter is shown in black. For comparison, lower right (d) is the despeckled four-level histogram segmentation, which identifies thetumor region, but cannot distinguish cerebrum from other brain matter.

695


(a) (b)

(c)Figure 7. Despeckled segmentation images of Fig. 6a using the direct Potts method are shown for different values for Q. Results are shown for

Q = 16 (a), Q = 32 (b), and Q = 128. In each case, the intensity threshold is st = (Q − 1)/2. For higher values of Q, more of the imageis categorized as textured (light and dark gray).

696

Frank W. Bentrem

Figure 8. Acoustic image of the seafloor formed using a sidescansonar system, which creates an image going out on ei-ther side via an acoustic time series. The direct Potts seg-mentation with Q = 16 and st = 7.5 separates the regioncontaining sand ripples (in green) from the rest of the im-age. The ripples are identified as a textured region andis labeled as sand since neither mud nor rock are likely toexhibit ripples.

Fig. 6a4 shows an MRI image of a brain with a tumorin the upper left quadrant. The segmented image usingthe direct Potts segmentation method with Q = 64 andst = 31.5 is shown in Fig. 6b. After filtering, we obtainthe the image in Fig. 6c. Here, the white regions may beinterpreted as possible tumorous tissue, light gray corre-sponds to matter at the periphery of the tumor, dark grayrepresents cerebrum, and black is other brain matter. Au-tomated screening for tumors may be performed by findingwhite regions of significant size. Although, the four-levelhistogram method (Fig. 6d) also identifies a similar tumorregion, the cerebrum is not distinguished from other brainmatter. The affect of the chosen value for Q is demon-strated in Fig. 7. Larger values of Q result in more ofthe image being categorized as textured (represented aslight and dark gray). Finally, acoustic seafloor imagesmay be analyzed in real-time with the direct Potts seg-mentation method as demonstrated in Fig. 8. Here sandripples are identified (in green) as a grainy texture and isseparated from the remainder of the image. Rapid seafloorcharacterization enables effective mine hunting/avoidance4 From the Iowa Neuroradiology Library(http://www.uiowa.edu/c064s01/index.html). Used withpermission.

operations [7, 25, 26], by indicating where seafloor minesare likely to be buried (mud), partially buried (sand), orunburied (rock).7. ConclusionsWe have examined the direct application of the Pottsmodel to grayscale image segmentation. The Potts modelis applied to the grayscale image itself rather than itsmore common application to the segmented image. Theresult is a segmentation method which processes in lineartime instead of the exponential or power-law time requiredwith conventional Potts segmentation methods. This in-crease in computational efficiency is vital for real-time,high data volume, and high resolution applications. Themethod is analogous to minimizing the total energy in atwo-dimensional magnetic spin glass where the energy iscalculated according to four Potts models. Each of the fourPotts models corresponds to a distinct type of magnetism,so the direct Potts segmentation method effectively distin-guishes regions in the spin glass that most closely adhereto the properties of four classes of magnetism, namelyferromagnetism, antiferromagnetism, paramagnetism, anddiamagnetism. In digital image segmentation, pixel inten-sities are substituted for the magnetic spins in the spinglass. The astute reader will note that no mention hasbeen made of the physical basis for the coupling constantsJα for the regions in the segmented image. The origins ofthe image pixels’ couplings may, in fact, be complicatedor unknown. Branching of plants give a textured appear-ance in Fig. 5a, while the distribution of blood vesselsand other brain material likely gives the textured appear-ance to Fig. 6a. Underwater sand ripples may arise fromthe fluid dynamics and saltation [27, 28] at the seafloor.The direct Potts segmentation has the advantages of be-ing unsupervised (no training sets needed), very efficient(linear-time, only a few operation per pixel), and effectivefor texture identification. For many image segmentationapplications in which texture is important, this direct Pottsmethod may be expected to produce better results thanother linear-time methods.AcknowledgementsThis work was supported by the Naval Research Labora-tory’s Advanced Graduate Research Program. The authorthanks the Tulane Center for Polymer Reaction Monitoringand Characterization (PolyRMC) for resources provided. Aspecial thanks goes to Dr. Luca Celardo for helpful dis-cussions. The views expressed in this article are those of

697


the author and do not represent opinion or policy of theUS Navy or Department of Defense.References

[1] D. L. Pham, C. Xu, J. L. Prince, Annu. Rev. Biomed.Eng. 2, 315 (2000)[2] Y. H. Yang, M. J. Buckley, S. Dudoit, T. P. Speed, J.Comput. Graph. Stat. 11, 108 (2002)[3] S. Peng, B. Urbanc, L. Cruz, B. T. Hyman, H. E. Stan-ley, P. Natl. Acad. Sci. USA 100, 3847 (2003)[4] V. Grau, A. U. J. Mewes, M. Alcañiz, IEEE T. Med.Imaging 23, 447 (2004)[5] S. Hadjidemetriou, C. Studholme, S. Mueller, M.Weiner, N. Schuff, Med. Image Anal. 13, 36–48 (2009)[6] X. Descombes, M. Moctezuma, H. Maître, J.-P.Rudant, Signal Process. 55, 123–132 (1996)[7] F. W. Bentrem, W. E. Avera, J. Sample, Sea Technol.47, 37 (2006)[8] T. Asano, D. Z. Chen, N. Katoh, T. Tokuyama, Int. J.Comput. Geom. Ap. 11, 145 (2001)[9] E. Ising, Z. Phys. 31, 253 (1925)[10] R. B. Potts, P. Camb. Philos. Soc. 48, 106 (1952)[11] F. W. Bentrem, Provisional Patent Application, NavyCase No. 99, 755 (2009)[12] K. Tanaka, J. Phys. A-Math. Gen. 35, R81 (2002)[13] J. P. Neirotti, S. M. Kurcbart, N. Caticha, Phys. Rev.E 68, 031911 (2003)[14] M. Blatt, S. Wiseman, E. Domany, Phys. Rev. Lett.76, 3251 (1996)[15] M. Blatt, S. Wiseman, E. Domany, Neural Computa-tion 9, 1805 (1997)[16] S. Wiseman, M. Blatt, E. Domany, Phys. Rev. E 57,3767 (1998)[17] K. Tanaka, H. Shouno, M. Okadak, D. M. Titterington,J. Phys. A-Math. Gen. 37, 8675 (2004)[18] E. Sharon, A. Brandt, R. Basriy, Proceedings of theIEEE Conference on Computer Vision and PatternRecognition (IEEE, Hilton Head Island, 2000) 1, 70[19] P. F. Felzenszwalb, D. P. Huttenlocher, Int. J. Comput.Vision 59, 167 (2004)[20] A. X. Falcão, P. A. V. Miranda, A. Rocha, Lect. NotesComp. Sci. 4179/2006, 138 (2006)[21] D. Chandler, Introduction to Modern Statistical Me-chanics (Oxford University Press, New York, 1987)[22] J. C. Lee Thermal Physics: Entropy and Free Ener-gies (World Scientific Publishing Company, Singa-pore, 2002)[23] L. Onsager, Phys. Rev. 65, 117 (1944)[24] D. Martin, C. Fowlkes, D. Tal, J. Malik, Proceedings ofthe 8th International Conference on Computer Vision

2, 416 (2001)[25] F. W. Bentrem, J. Sample, M. T. Kalcic, M. E. Duncan,Proceedings of Oceans 2002 (MTS/IEEE, Biloxi) 1, 7(2002)[26] F. W. Bentrem, J. T. Sample, M. M. Harris, ScientificComputing 25, 30 (2008)[27] R. A. Bagnold, The Physics of Blown Sand and DesertDunes (Methuen, London, 1941)[28] L. Kang, L. Guo, Phys. Lett. A 330, 198 (2004)

698

Documents

A Q-Ising model application for linear-time image segmentation