Research Article A Novel Framework for Interactive ...downloads.hindawi.com/journals/jece/2016/2635124.pdfimage analysis that focus on interactive raw data exploration. We combine

Research ArticleA Novel Framework for Interactive Visualization and Analysis ofHyperspectral Image Data

Johannes Jordan Elli Angelopoulou and Andreas Maier

Pattern Recognition Lab University of Erlangen-Nuremberg Erlangen Germany

Correspondence should be addressed to Johannes Jordan johannesjordancsfaude

Received 24 May 2016 Revised 23 August 2016 Accepted 8 September 2016

Academic Editor Sos Agaian

Copyright copy 2016 Johannes Jordan et alThis is an open access article distributed under theCreativeCommonsAttributionLicensewhich permits unrestricted use distribution and reproduction in any medium provided the original work is properly cited

Multispectral and hyperspectral images are well established in various fields of application like remote sensing astronomy andmicroscopic spectroscopy In recent years the availability of new sensor designs more powerful processors and high-capacitystorage further opened this imaging modality to a wider array of applications like medical diagnosis agriculture and culturalheritage This necessitates new tools that allow general analysis of the image data and are intuitive to users who are new tohyperspectral imaging We introduce a novel framework that bundles new interactive visualization techniques with powerfulalgorithms and is accessible through an efficient and intuitive graphical user interface We visualize the spectral distribution of animage via parallel coordinates with a strong link to traditional visualization techniques enabling new paradigms in hyperspectralimage analysis that focus on interactive raw data exploration We combine novel methods for supervised segmentation globalclustering and nonlinear false-color coding to assist in the visual inspection Our framework coined Gerbil is open source andhighly modular building on established methods and being easily extensible for application-specific needs It satisfies the need fora general consistent software framework that tightly integrates analysis algorithms with an intuitive modern interface to the rawimage data and algorithmic results Gerbil finds its worldwide use in academia and industry alike with several thousand downloadsoriginating from 45 countries

1 Introduction

Multispectral imaging allows capturing of rich reflectanceinformation that is not available with traditional RGB cam-eras It has played a key role in the field of remote sensing fordecades ever since aircrafts and satellites became equippedwith hyperspectral sensors systems One such notable exam-ple is the Airborne VisibleInfrared Imaging Spectrometer(AVIRIS) [1] Hyperspectral sensors are less widely used ldquoonthe groundrdquo though they are becomingmore popular in areaslike astronomy cultural heritage agriculture and medicalimaging

In amultispectral image each pixel is a vector of intensityvalues where each value corresponds to the scene radianceover a small range of wavelengths The resulting vectorstypically are of lengths ranging between 31 and 200 Thehigh-dimensional nature of the data and its strong interbandcorrelations pose challenges to computer vision algorithms

Adaptations are needed to expose and exploit the informationcontained in the data A similar problem arises when one triesto manually process such high-dimensional data There is alack of intuition which demands a presentation that guidesthe user during data exploration Effectively one needs twocomponents simultaneous visualization of different aspectsof the information and strong user interaction

As the popularity of multispectral sensors has beenincreasing so has the need for computer-aided interactiveanalysis For example one of the first popular analysis frame-works for providing a graphical interface for inspection wasthe Spectral Image Processing System which was introducedin 1992 [2] However almost all available multispectral anal-ysis software is tailored to specific applications As expectedmost of the available software focuses on remote sensing andis designed to foster a workflow specific to this domain

In recent years the availability of cheaper easier-to-use multispectral cameras further opened this modality to

Hindawi Publishing CorporationJournal of Electrical and Computer EngineeringVolume 2016 Article ID 2635124 17 pageshttpdxdoiorg10115520162635124

2 Journal of Electrical and Computer Engineering

(a)

(b)

(c)

(d)

(e)

Figure 1 Gerbil user interface (a) Spectral distribution view (b)spectral gradient distribution view (c) spatial view of a single imageband with segment overlays (d) label manager and (e) false-colordisplay

new applications Various devices based on electronicallytunable filters [3] or imaging spectrometers [4] are com-mercially available from several vendors An example ofsensor targeting emerging applications is MuSIS [5] which iscurrently in use in several museums and libraries Howeverthe users of these sensors lack a software solution that enablesthem to explore the data in an interactive and intuitivefashion

To address this limitation we developed the Gerbilsoftware platform which incorporates established interactivevisualization concepts to provide a new presentation of spec-tral images as well as a new workflow for inspecting themFigure 1 depicts a screenshot of Gerbil in use for inspecting amultispectral image Gerbil is developed by an open-sourcesoftware project in the tradition of free software frame-works in the signal processing research community such asOpenCV [8] orWeka [9] In this paper we provide a scientificreference for Gerbil and describe the core methodologies itencompasses We also provide a detailed presentation of thefeasibility and performance of the algorithms in Gerbil andshow how our novel approach is a significant improvementcompared to established practice

The paper bundles and expands on the following contri-butions

(i) Interactive visualization of spectral distributionsbased on efficient parallel coordinates [6]

(ii) Supervised segmentation of hyperspectral data [10](iii) Fast global clustering with superpixel support [11](iv) Fast nonlinear false-color visualization [12]

These methods were derived by adapting establishedalgorithms for hyperspectral data and interactive time con-straints They allow us to introduce new paradigms in hyper-spectral image analysis that focus on interactive raw dataexploration built on the tight incorporation of the afore-mentioned techniques into a comprehensive open-sourcesoftware framework The software is publicly available athttpgerbilvisorg under a free software license

2 Previous Work

One of the first widely recognized software packages formultispectral analysis LARSYS became available in the1960s It was operated via a text terminal Several frameworksfor graphical interactive multispectral or hyperspectral dataanalysis that are still of broad influence today date backto the early 1990s Earlier frameworks focus on a specificapplication most prominently in the field of remote sensingBoardman et al [13] provided an overview of the history ofestablished software systems for remote sensing hyperspec-tral data Biehl [14] created an updated list on the web in2007 In this section we give first a brief overview of domain-specific software packages before reviewing more widelyapplicable software We focus on the interactive inspection ofimages

21 Remote Sensing The Spectral Image Processing System(SIPS) was presented by Kruse et al [2] in 1993 It introducedthe Spectral Angle Mapper (SAM) a spectrum-comparisontool still popular in the field of spectral matching [15] Datawas presented either as single bands or in a false-coloredcomposite of three user-selectable bandsThe user could alsoview a selected pixelrsquos spectrum Color-coded stacked spectrawere provided for a selected slice (a vertical or horizontalline scan or an arbitrary path in the image) This set ofvisualization forms are still typical of popular software forhyperspectral capture or analysis

The MultiSpec freeware package by Biehl and Landgrebe[16] can analyze multispectral images from various sourcesusing the powerful GDAL library [17] for data IO Thefocus of this software is classification The target audience isthe general Earth science community Besides providing allcommon visualizations already mentioned it can generatebiplots that relate selected regions in a pair of bands and astatistics image display which depicts the correlation betweenthese bands

A widely used commercial software is ENVI [18] initiallydeveloped by Boardman et al [13] Several innovations inhyperspectral analysis were introduced in ENVI includingfor example the Pixel Purity Index (PPI) [19] PPI finds themost spectrally pure pixels in an imageThese pixels typicallycorrespond to endmembers (constituent spectra of an image)A notable innovation in visualization is the 119899-DimensionalVisualizer which uses PPI as input It is an interactive datavisualization method that allows real-time rotation of scat-terplots in 119899-dimensions The presentation of 119899-dimensionalscatterplots in 2D can be somewhat unintuitive Yet the toolis valuable to experts for identification of endmembers basedon the depicted point clouds

Two other notable pieces of software for hyperspectralremote sensing data analysis areHyperCube [23] andOpticks[24] HyperCube is released by the US Army Corps ofEngineers and contains functions to filter warp (registertwo images) calibrate and undistort photometrically projectand arithmetically manipulate the data Opticks also includesmany common tools like GIS annotations false coloring andhistograms Additionally third parties can provide function-ality through an external module adding further capabilities

Journal of Electrical and Computer Engineering 3

to the software For example the Spectral Processing Exten-sion [25] includes typical tools for hyperspectral data analysis

Within the remote sensing context many works exist onvisualizing a multispectral image by employing dimensional-ity reduction The goal is to present the image in false-colorRGB by reducing dimensionality from spectral vectors oflength119863 to length 3 which are then used as r g and b valuesBased on the resulting pixel colors the user should be ableto discriminate regions of the image according to his specificinterests Methods to achieve this include variants of theprincipal component analysis (PCA) independent compo-nent analysis (ICA) and nonlinear methods Some variantsalso incorporate classification results A recent approach byCui et al [26] focuses on the interactive adaptation of suchvisualization However it still uses the common pixel maprepresentation of the image

22 Astronomy A field with a long history of hyperspectralimaging is astronomy The most notable software frameworkin this area is ds9 [27] available from the SmithsonianAstrophysical Observatory This software is very powerful inthe spatial representation of astronomical imagery Howeverit provides a limited 3D visualization of the data cube that israrely viable for nonastronomical data

Recently Li et al [28] explicitly tackled the question ofhow to present multiband data They draw image bands in3D either as an image-stack or as a volume-rendered modelfor example horseshoe Their volume rendering handlesthe obvious problem of clutter by applying transparency toindividual data points based on their intensity or on a user-adjusted mask However one cannot generally assume thatlarge image regions can be faded out to still unobstructedlydisplay more relevant data

23 Other Application Domains Multispectral imaging hasbecome increasingly popular in the preservation and analysisof artwork as well as historical documents Colantoni et al[29] analyzed multispectral images of paintings from theperspective of human color perception From the image dataa representation in the CIE119883119884119885 color space [30] is computedunder controlled virtual illumination Several tools can thenbe applied for visualization of trichromatic data The originalspectra are not considered in the analysis

In 2010 Kim et al [31] presented a solution for interac-tive visualization of historical documents They provided anicely designed self-contained analysis tool and incorporatedinnovations in how the data is presented Some are specific todocument analysis for example dealing with aging-relatedartifacts while others are more general The spectral lensfeature for example presents data from two spectral bandsin a single display Similarity maps can be computed basedon the 1198712 norm between themean spectral value of a selectedregion and all image pixels However similaritymeasures likeSAM [15] would be better suited for spectra comparison A3D histogram plot is used to compare two spectra

The National ICT Australia (NICTA) offers the Scyvensoftware free of chargewhich features the reflectance recoveryand material classification pipeline developed at NICTA [732] Its visualization includes false coloring and an adaptation

of the parallel coordinates visualization introduced in Gerbil(see Section 32)

Labitzke et al [33] introduced an interactive frameworkfor linear spectral unmixing of multispectral datasets Spec-tral unmixing is especially popular in remote sensing wherethe spatial resolution is often low Many algorithms existfor finding endmembers and performing spectral unmixing[34] Labitzke et al introduced an incremental methodthat can semiautomatically find endmembers Then visualfeedback is provided by their complementary visualizationthat reflects the quality of the characterizing set This set canbe interactively modified in order to improve the unmixingThe authors explicitly differentiated their approach fromGerbil [33] Spectral unmixing is integral to their interactivevisualization approach while in ourmethods we visualize theraw data The algorithm and workflow proposed by Labitzkeet al are not in opposition to our approach but instead theynicely complement the visualization capabilities of Gerbil

3 Visual Inspection

In general existing approaches share the same basic set ofdata representations and user interactions while individualextensions typically follow a specific application scenarioHowever the wider range of applications of multispectralimaging necessitates an interactive visualization frameworkthat is both sufficiently general for a broader range ofapplications and more versatile than existing basic repre-sentations In our framework Gerbil we follow a novelconcept that enables an entirely new workflow in exploringa multispectral image It revolves around presentation andexploration that make the image data more apparent to theuser effectively allowing a more direct interpretation of thedata The user does not rely on but is merely guided byautomatic processing

31 Main Concept Figure 2 gives an overview of the keyfeatures that constitute Gerbil In order to have a frameworkthat is suitable for use in a large variety of applications weincorporate several well-understood hyperspectral analysistools into Gerbil while adding concepts from the datavisualization and computer vision communities The valueof our approach lies in employing these features in a fash-ion that enriches the user experience while investigating amultispectral image One such example is the incorporationof parallel coordinates a technique for display of high-dimensional data Parallel coordinates are used in visualizingspectral distributions The visualization is highly dynamicand interacts with the topological representations of theimage (eg a grayscale depiction of a single image band)Another contributing factor is the emphasis on the spectralgradient descriptor [35] Its distribution is displayed next tothe spectral distributionThe spectral gradient is amultispec-tral descriptor that focuses on data aspects which are directlyrelated to material and reflectance properties

Figure 1 depicts the three main components of thegraphical user interface the visualization of original spectra(Figure 1(a)) and the spectral gradient spectra (Figure 1(b))both via parallel coordinates and the spatial image view


Spectraldistributionvisualization

Supervisedsegmentation

Globalclustering

Labelingtools

Extensibledesign

Illuminantcontrol

False-colordisplay

GerbilSection 41

Section 42

Section 43 Section 62

Section 51

Sections 32 33

Sections 52 53

Figure 2 Overview of the Gerbil feature set and correspondingarticle sections

(Figure 1(c)) The interaction between these elements as isexplained in detail in Section 33 is augmented by toolsthat give further guidance to the user One such example isdimensionality reduction via PCA or nonlinear methods asdiscussed in Section 5 Another tool is feature-space cluster-ing of data described in Section 42 Especially importantin our workflow is supervised segmentation where the userprovides input depending on the goal of their investigationor on prior knowledge In Section 41 we describe how weemploy a graph-based segmentation algorithm for this task

32 Spectral Distribution View Existing approaches ondepicting a multispectral image in its entirety are limited bythe spatial layout of the image The image data is viewedas a cube with the 119911-axis corresponding to spectral bandswhich are stacked on top of each otherUse ofmodern volumerendering techniques can make this representation useful insome scenarios [28]However inmost cases where there is nosparsity in pixels of interest a very cluttered view is obtainedthat reveals the shortcomings of a simple 3D arrangementOther methods such as scatter plots or false coloring rely ondimensionality reduction While the visualization of reduceddata is helpful in many applications it is hard to preservesubtle details In contrast a good visualization of the originaldata helps the observer evaluate how well a dimensionalityreduction method fits a specific application

One way of addressing this issue is to defer from thetopological relations in the image for the time-being andconcentrate instead on a graspable representation of thespectral distribution To do this efficiently we employ theparallel coordinates [36] method as explained below This isa well established technique for visualizing high-dimensionalgeometry and analyzing multivariate data It has been widelyused for example in financial applications and geographicinformation systems One can see the traditional spectralvisualization as a specific instantiation of parallel coordinatesvisualization By building on the more general concept wecan incorporate tools from the visualization community forhigh-dimensional data presentation

According to the parallel coordinates concept a 119863-dimensional feature space (resulting from 119863 spectral bands)is projected onto a two-dimensional view as follows 119863parallel vertical lines denote the119863 axes that is the119863 spectralbands The 119910-coordinate on the 119894th axis corresponds to aspectrumrsquos value at band 119894 To display the spectral vector ofa pixel a polyline is drawn with its vertices lying on thecorresponding vertical axes The resulting display follows thelayout of a plot where the 119909-axis would denote wavelengthand the 119910-axis denotes intensity321 Optimized Parallel Coordinates Drawing a polyline foreach single multispectral vector has several drawbacks it istime consuming and the display may easily get cluttered inwhich case single polylines may not separate well from therest of the data A solution to both clutter and speed concernsis to draw fewer polylines where a polyline can represent a setof pixels With this distinction polylines that represent morepixels appear stronger

We realize this representation by introducing a histogramin the feature space with 119861 evenly distributed bins in eachdimension 119861 is user adjustable between 2 and the dynamicrange of the captured data (typically 28 to 216) For examplethe histogram for a 31-band multispectral image with 119861 =256 would hold 119861119863 = (28)31 = 2248 bins Building adense histogram of this size is not feasible however a sparsehistogram can be created by using an ordinary hashingalgorithm The key idea here is that the amount of occupiedbins is bound by the spatial resolution of the input imageFor example a spatial resolution of 512 times 512 leaves only 218possible distinct values effectively giving an upper bound tothe amount of bins filled in a sparse histogram or the amountof hash keys needed

For each populated bin a polyline is drawn in the parallelcoordinates visualization The strength of the polyline ismanipulated by assigning it an opacity 120572 120572 is determined bythe relationship between the number of pixels represented bythe bin 119899bin and the total number of processed pixels 119899total

120572 = 119888120572119891 (119899bin + 1)119891 (119899total) (1)

where 119888120572 is a user-adjustable factor and 119891(sdot) is a logarithmicfunction The logarithm emphasises bins with fewer pixelsThe idea is that even a single pixel should be perceptible Thelogarithm also ensures that the resulting dynamic range ofalpha values can be represented reasonably well with an 8-bitalpha channel Nowadays Gerbil overcomes this limitationby drawing onto a floating-point framebuffer and the user canchoose 119891(sdot) to be linear322 Drawing Refinements The newer design of Gerbilincludes several measures to further enhance the visualquality and accuracy of the spectral distribution display Bydividing the feature space into a fixed number of equallyspaced bins the histogram applies a nonadaptive quantiza-tion of a spectral vector x A possible strategy to reduce theintroduced quantization error is to employ a binning that isadapted to the observed data A straight-forwardmethod is to


(a) True-color display

245

184

123

62

0

700

670

640

610

580

550

520

490

460

430

400

(b) Previous work [6] 119861 = 20

245

184

123

62

0

700

670

640

610

580

550

520

490

460

430

400

(c) Proposed method 119861 = 20

245

184

123

62

0

700

670

640

610

580

550

520

490

460

430

400

(d) Proposed method 119861 = 256

Figure 3 Spectral distribution view of cropped feathers image with different bin parameters Spectra in true color

perform a separate histogram equalization in each dimension119894 which enforces a uniform PDF of the mapped vectorvalues 119909119894 [37] While it works well for big clusters of similarpixels spectra that are sparsely represented in the image willsuffer from such an operation It may increase the averageaccuracy at a great expense in the precision of single pixelrepresentations which is not desirable

We employ an alternative strategy by adjusting how abin is drawn Our method achieves an improved generalaccuracywithout a significant expense in the accuracy for anysingle pixel When drawing a bin we draw the mean vectorof its contents instead of its midrange values This can becomputed on the fly while adding new vectors with a finalnormalization based on the already stored number of entriesFigure 3 shows an example of the visual quality gain A broaddescription of the spectral distribution is already possiblewith a low 119861

Another hindrance in visual quality is the mutualobstruction of pixel representations Inmanyuse-cases pixelsare color coded (see Section 33) This involves effectivelydrawing several distributions on top of each other In highlypopulated intensity ranges this can lead to extensive occlu-sions There exists work on clutter reduction in parallelcoordinate plots that tackles this problem [38] The key ideais to only render a randomly sampled subset of the datapoints Instead we significantly reduce clutter by drawingdata in a randomly shuffled order without dismissing anyinformation

323 Evaluation The error introduced by the histogram-based quantization can be measured by the average rootmean squared error (RMSE) as well as themaximumabsolute

deviation (MAD) between original vectors x119894 and theirquantized counterparts x119894 This gives us the measure

RMSE = 119873sum119894=1

radicsum119863119895=1 (119894119895 minus 119909119894119895)2 119863119873 (2)

for the average RMSE and

MAD = max (max (10038161003816100381610038161003816119894119895 minus 11990911989411989510038161003816100381610038161003816 119895 le 119863) 119894 le 119873) (3)

where max(sdot) is the sample maximumThe number of bins per dimension 119861 is a crucial parame-

ter It lets the user choose between drawing speed viewingquality and accuracy Even a considerably low 119861 shouldprovide acceptable accuracy and the speed-up by lowering119861 should be effective We evaluate RMSE and MAD for boththe previously published [6] and the newly proposed draw-ing methods on several datasets from different applicationdomains Table 1 provides statistics on the datasetsTheCAVEdataset consists of objects in a laboratory settings whileFoster captured natural (outdoor) scenes Indian Pines andDC Mall are widely used remote sensing images

We use a desktop machine equipped with a quad-coreIntel Core i7 CPU running at 280GHz and a GeForce GTX550 Ti consumer graphics card for testing the computationalperformanceWemeasure the time needed for drawing oper-ations via GL TIMESTAMP and draw in WUXGA resolution

In Figure 4 we plot running times against accuracy forvarying 119861 Bin Center describes the originally publisheddrawing method while Mean denotes our refined drawingmethod We can observe that the average RMSE becomes


Table 1 Datasets used for evaluation

Name Size Resolution Bands 120582 (nm) SourceCAVE 32 512 times 512 31 400ndash700 [20]Foster 8 1340 times 1020 33 400ndash720 [21]Indian Pines 1 145 times 145 220 400ndash2500 [22]DC Mall 1 307 times 1280 191 400ndash2475 [22]

253

240

227

214

201

188

175

162

149

136

123

110

97

84

71

58

45

32

19

Number of bins per dimension

Dev

iatio

n

Mean MADMean RMSEBin Center MADBin Center RMSE

BinningLoadingDrawing

4

3

2

1

0

001

01

1

0001

Seco

nds

Figure 4 Experimental results on a scene from the Foster datasetQuantization errors are plotted for both the old (Bin Center) and therefined (Mean) drawing method

negligible with higher 119861 for both methods However ourmethod achieves low RMSE values for considerably lowersettings of119861 Due to outliers present in some of the histogrambins the MAD for the refined method is somewhat higherthan the original MAD

The time needed to build the histogram is denoted asbinning and is not determined by 119861 The time needed forpreparing the geometry and loading it on the GPU (Loading)slowly grows with 119861 In contrast 119861 plays an important rolefor the time needed for the drawing operation (Drawing)For higher 119861 values the time needed for drawing growsto multiples of the time needed for preparation Hencethe histogramming plays an important role in achievinginteractivity

In Table 2 the running times and accuracy measuresfor the refined drawing method are listed on all datasetsexcept Indian Pines where due to the very low resolutionof the image running times are negligible The times shownfor preparation are the combined histogram building andgeometry loading times As in Figure 4 it is observed that alow 119861 can already achieve a small quantization error Setting119861 = 64 is a reasonable compromise between speed andaccuracy on the tested datasets It provides an effective speed-up in comparison to a high histogram resolution without aperceivable loss in drawing accuracy

It can be seen in Table 2 that on our test machine evenwith a moderate setting drawing the spectra from a large

Table 2 Average drawing times and accuracy

Dataset bins Prep (s) Draw (s) RMSE MAD

CAVE3231 0089 0161 0002 00276431 0115 0240 0001 001425631 0177 0390 0000 0003

Foster3233 0274 0146 0005 00316433 0358 0554 0002 001425633 1085 2955 0000 0004

DC Mall32191 0585 1789 0001 002664191 0881 3171 0000 0013256191 1020 3554 0000 0003

imagemay still take several seconds Typically it is impracticalto work on a high-resolution image without a region of inter-est Yet we alleviate longer drawing times by incrementallydrawing the data (disabled in the benchmarking) in orderto provide direct visual feedback in the form of a roughapproximation of the full data (as pixels are drawn in arandom order)

33 Interactive Inspection An important aspect of todayrsquosvisualization approaches for multivariate data is interactivemanipulation of the presentation A single view most oftencannot provide the full understanding that may be gained bya series of user-controlled depictions User input is vital toparallel coordinates in particular We provide several mech-anisms for both transient (cursor highlights) and persistent(color labels) interactive viewing

In the parallel coordinate plots the user can dynamicallyhighlight specific data points that is spectral vectors thatrepresent pixels These are displayed in yellow and with fullopacity as an overlay over nonhighlighted spectra We realizethis in OpenGL with layered frame buffers Updates to thehighlight only need to redraw highlighted spectra Whilethe highlight constantly follows the keyboard andor mouseinput the corresponding pixels are instantly highlighted inthe spatial view While scrolling through the spectra thetopological view always reveals which pixels contribute to thehighlighted spectra

Twomodes of operation exist for highlighting in the spec-tral distribution single-band limited and multiband limitedThe single-band limited highlight is formed by all spectralvectors falling into bins that share a specific intensity rangein one band (see Figure 5(a))The coarseness of this selectionis therefore directly related to the binning parameterTheuserselects the band and intensity range via mouse click or cursorkeys The multiband limited highlight provides a higher levelof control in exchange for more detailed user input Here ineach band separate lower and upper intensity bounds can beset (see Figure 5(b))

Another method of highlighting exists in the topologicalview Here the user can direct a cursor over individualpixels in the spatial view In the corresponding viewports thespectrum and spectral gradient data of the pixel under thecursor are presented as a yellow overlay


(a) Single-band limited (b) Multiband limited

Figure 5 Highlighting modes used in the spectral gradient distribution view of a scene with two objects In (a) one objects is highlightedIn (b) further discrimination is performed by adding intensity bounds in two further bands

While highlights give instant feedback they constantlychange as the user investigates the data It is often desired tokeep a selection of pixels distinguished from the rest of thedata for example for comparison purposesWe call this pixelset a label each pixel can be part of at most one label A labelcan be seen as a permanent highlight For each label a distinctsparse histogram is created as described in Section 321 It isthen drawn in the label color When a histogram bin is partof the current transient highlight the color is significantlyshifted towards yellow

When the user has selected pixels within the transienthighlight they may add this set of pixels to a label or deletetheir label association By doing so they can iteratively refinethe labeling of the data to concentrate on specific detailsAnother way to alter the labeling is to use a ldquolabelrdquo brushin the topological view or to use automated segmentationmethods as discussed in Section 4 Labelings can also bestored for later use

Labels are important because they serve as a memoryin the connection between different representations of themultispectral image such as between a single band and thespectral distribution A selection or temporary highlight inone representation is instantly propagated to the others Bylabeling this highlight it becomes permanent The user canthen continue his investigation within another representa-tion for example the spectral gradient view that may revealnew insights within this labeling For example a user maystart by hand-labeling parts of a scene in the topological viewThen they may restrict the spectral distribution view to thislabel The parallel coordinates visualization could reveal acertain variance within the labeled pixels By using selectiontools inside this viewport the user could separate parts ofthe pixels and assign them to a second label The topologicalview instantly reveals which parts of the object contribute towhich portion of the distinguished spectra Next the usermay initialize a multiband limited highlight based on thissecond labelThis reveals other regions in the scene that sharespectral characteristics with the labeled pixels

As a result we facilitate a workflowof inspecting an imagethat is not possible with a traditional hyperspectral analysisframework It is based on concurrent concerted work withboth topological and spectral views and allows a smooth andinstantaneous switch in attention between them Such a step-by-step exploration enables the user to quickly discover andgrasp underlying information In the visualization domain

this procedure is considered a valuable tool for understandingcomplex data [39] A narrated video demonstration of thisworkflow can be accessed at httpvideogerbilvisorg

4 Segmentation and Labeling

An interactive interface to the multispectral data is noreplacement for automatic processing In fact the twoapproaches together form a powerful combination Withinour framework it is easy to interpret and assess the results ofalgorithms used in automated analysis These results can be agood starting point for further interactive analysis Gerbil isequipped with two powerful methods that segment the dataeither according to spectral characteristics on a global level orbased on topological relation and local similarity In the lattercase we bring supervised segmentation to the multispectraldomain especially for the purpose of interactive inspection

41 User-Guided Segmentation Supervised segmentationincorporates user-provided prior knowledge A user specifiesa set of background and foreground pixels All other pixels arethen determined as belonging to either the background or theforegroundWemake this concept a powerful tool within ourinteractive workflow

The early version of Gerbil [6] included a rudimentaryversion of supervised segmentation We have now adaptedan existing algorithm family specifically to the multispectraldomain [10]

411 Graph-Based Supervised Segmentation In recent yearsgraph-based algorithms have dominated supervised segmen-tation on both grayscale and RGB images In 2011 Couprieet al introduced a framework for supervised segmentationthat incorporates several key methods based on graph theory[40] Their power watersheds integrate graph cuts randomwalker and watersheds algorithms in a single mathematicalframework

For this algorithm family the input consists of two sets ofpixels the foreground seeds F and background seeds B aswell as the pixel values x119894 1 le 119894 le 119873 which are strictly usedin a differential manner The topological relation of pixels isreflected in a graph structure Each pixel 119894 corresponds toa vertex V119894 A neighboring pixel 119895 with corresponding V119895 isconnected to V119894 via an edge 119890119894119895 The edge weight 119908119894119895 of anedge 119890119894119895 is a function of the similarity between x119894 and x119895 We


compute an 119873-element vector y where 119910119894 isin [0 1] indicatesthat a pixel 119894 belongs to foreground or background via

ylowast = argminy

sum119890119894119895isin119864

119908119901119894119895 10038161003816100381610038161003816119910119894 minus 11991011989510038161003816100381610038161003816119902 +sumV119894119908119901119865119894 10038161003816100381610038161199101198941003816100381610038161003816119902

+sumV119894119908119901119861119894 1003816100381610038161003816119910119894 minus 11003816100381610038161003816119902

st forall119894 isin F 119910119894 = 1 forall119894 isin B 119910119894 = 0(4)

where 119908119865119894 and 119908119861119894 denote unary weights penalizing fore-ground and background affinity Simple thresholding leadsto a binary segmentation 119904 with 119904119894 = 1 (foreground pixel) if119910119894 ge 12 and 0 (background pixel) otherwise Based on theselection of parameters 119901 and 119902 this minimization matchesthe graph cuts random walker or shortest paths algorithmsWith 119901 = infin 119902 ge 1 the power watershed algorithm isobtained See Couprie et al [40] for details

Couprie et al define the edge weights for grayscale andRGB images

119908119894119895 = exp (minus120573 (119889 (x119894 x119895))2) (5)

where 120573 is a constant and 119889(sdot) is the 1198712 norm or the 119871infin norm[40] For hyperspectral data these choices are reasonable ifa single band or the PCA of the image is used as inputHowever when operating on the full spectra of the image amore appropriate distance function should be employed

412 Adaptation to Hyperspectral Data In our previouswork we evaluated a range of distance functions on multi-spectral data including established similarity measures forspectral mapping [41 42] and a data-driven measure [10]In spectral mapping captured spectra are compared to adatabase of known spectra to determine a material Hencespectral mapping similarity measures are well suited forhyperspectral distance functions

The best performing measurement in our experimentsis the spectral angle from the Spectral Angle Mapper [15]The spectral angle has the property that it disregards pureintensity changes It is defined for two spectra u and k as

SA (u k) = cosminus1 ( ⟨u k⟩u2 sdot k2) (6)

where ⟨sdot sdot⟩ denotes the inner product and sdot 2 the 1198712 norm

413 Example Use-Case We demonstrate the use of super-vised segmentation on the superballs image It shows plasticballs of various color that highly reflect on each other InFigure 6 the user investigates a red ball specifically Withan RGB camera we could only spot the reflection from theyellow ball (see Figure 6(a)) Calculating the PCA on thespectral gradient (as discussed in Section 52) also revealsthe reflection from the green ball (see Figure 6(b)) Howeverthe blue balls also reflect on the red ball which is notrevealed by these depictions The first labeling Figure 6(c)is obtained by setting one foreground seed in the middle of

the ball and drawing a circle around the ball as backgroundseeds When looking at the spectral gradient distribution ofthat segment (see Figure 6(d)) we see the deviations fromthe distribution caused by these interreflections Browsingthrough these parts of the spectral gradient distribution plotwith the interactive highlight reveals their spatial locationsBy setting corresponding seed points again we can find threeadditional segments within the ball (see Figure 6(e)) Thespectral gradient distribution plot (see Figure 6(f)) revealshow the pixels from these three segments contribute to thedeviations in the distribution To further understand themthe user might continue by also segmenting the surroundingballs effectively adding them to the plot

Our user-guided segmentation method is especially use-ful in interactive analysis Segmentation results are helpfulquick to obtain and easy to refine within the provided userinterface

42 Unsupervised Segmentation While graph-based seg-mentation is valuable within an interactive session it isobject-driven in a sense that spatially connected pixels aregrouped However often it is desired to explore the imageas a whole which means connecting pixels that share nospatial relation in the image for example to form a segmentconsisting of all scene objects that share the same albedoNote that while the multiband thresholding detailed inSection 33 and illustrated in Figure 5(b) often allows findingsuch spatially disconnected material clusters in the spectralgradient manually it can be a tedious task especially whenseveral material clusters in the scene are to be revealedA global clustering method method however can reveal allmaterial clusters in the scene and be used as a starting pointfor further analysis

For this task we incorporate the mean shift algorithminto Gerbil Mean shift has been used for global clustering ofmultispectral data for several tasks [43 44] Mean shift is agood choice due to its general purpose design For examplewhile the well-studied and fast k-means algorithmworks beston clusters with circular shape and the number of clustersneeds to be defined a priori mean shift is a density gradientestimator and it neither requires prior knowledge of thenumber of clusters nor constrains the shape of the clusters[45] which is desired property for analysis of arbitrary imagedata A drawback of the original mean shift is computationalcomplexity The Fast Adaptive Mean Shift (FAMS) algorithmby Georgescu et al [46] provides an important improvementto computation time by introducing an approximate nearest-neighbor search However the method still takes minutesto hours to segment a hyperspectral image To facilitate theuse of mean shift in interactive inspection we introduceda new variant that combines mean shift with a superpixelpresegmentation [11]

421 Superpixel Mean Shift Clustering The mean shift algo-rithm works in feature space alone to obtain a global clus-tering rather than spatially constrained segments It thusdisregards any topological context We reintroduce the useof spatial information by incorporating superpixel detection


(a) True color (b) Spectral gradient PCA

(c) First labeling

16

095

03

minus1

685

665

645

625

605

585

565

545

525

505

485

465

445

425

405

minus035

(d) Spectral gradient for (c)

(e) Second labeling

16

095

03

minus1

685

665

645

625

605

585

565

545

525

505

485

465

445

425

405

minus035

(f) Spectral gradient for (e)

Figure 6 Inspection of cropped superballs image Labelings are shown as overlay on the spectral gradient at 490ndash500 nm

Both spatial context and superpixels gained attention recentlyas means to improve the performance of hyperspectral imageprocessing tasks [47 48]

In our algorithm we first obtain a superpixel segmen-tation by adapting the method of Felzenszwalb and Hut-tenlocher [49] to hyperspectral data We obtain a set ofsmall homogeneous regionsThen FAMS is performed usingthese regions as data points This significantly reduces thecomplexity of the mean shift operation Overall computationtimes are in the range of five to ten seconds for images fromthe CAVE database [20] The reliance on superpixels impactsthe spatial resolution of the segmentation in a way thatoften improves the spatial coherence of segments A detailedpresentation of this work can be found in [11]

422 Clustering Results andRefinement In Figure 7 we showan example result on an image from the CAVE databaseClusters are assigned random colors with an equal spacing inthe hue range Our superpixel mean shift algorithm denotedas MSSP uses an established spectral matching measure[41] to form superpixels in the original feature space andthen performs mean shift for each superpixel in the spectralgradient feature space It is compared to FAMS operating inboth the original feature space as well as the spectral gradient

feature space (denoted as SG-FAMS) The results reveal thatthe spectral gradient distribution is more helpful for materialclustering as compared to the original spectral distributionaffected by geometry and that we can obtain a very goodapproximation of the SG-FAMS result with MSSP

In Figure 8 we show another example result on an imagefrom the Foster database which other than the previousimage captures a natural scene Due to its high spatialresolution clustering is a time-consuming task on this imageWe compare our approach to the method by Huynh andRobles-Kelly [7] denoted HRK Due to memory constraintsHRK was applied on a down-sampled (by a factor of two)version of the image and took 6 minutes Note that for HRKthe number of desired clusters needs to be set in advanceOur method took 154 seconds on the original image and 15seconds on the down-sampled version

Within Gerbil it is easy to further refine clusteringresults The spectral distribution visualization reveals thecompactness of a cluster It helps the user to spot under-segmentation where a single cluster contains several spectralprofiles (as seen in Figure 6(d)) Oversegmentation as is thecase in Figure 8(c) can also be easily spotted by comparingthe spectral distribution of two clusters Gerbil providesan intuitive interface to merge such segments To obtain


(a) True color (b) FAMS (41) (c) SG-FAMS (24) (d) MSSP (30)

Figure 7 Clustering results on fake and real food image The number of clusters is shown in parentheses in the respective caption

(a) True color (b) HRK [7] (20)

(c) MSSP (27) (d) MSSP refined (20)

Figure 8 Clustering results on Ribeira image The number of clusters is shown in parentheses in the respective caption

the result in Figure 8(d) the user performed four simplemerging operations The user may also run a supervisedsegmentation to refine the clustering especially in the caseof an undersegmented area

43 Label Inspection and Manipulation The segmentationmethods in Gerbil provide beneficial input in a typical usagescenario However a good presentation of the results isequally important Depending on the underlying data meanshift clustering may produce a high number of meaning-ful segments In the supervised scenario often follow-upsegmentations are used to locally refine obtained segmentsSeveral measures aid in discerning and manipulating seg-mentations

The first measure is the automatic determination ofpractical label colors A preset of label colors consists of theprimary and secondary palette excluding yellow which isreserved for temporary highlights For more than five labelscolors are determined by an equal spacing in the hue rangeexcluding yellow Figures 1 6 8 and 10 show examples ofautomatically labeled segmentations In cases where a tem-porary highlight falls onto labeled pixels the correspondingpixels are drawn in their label colors significantly shiftedtowards yellow

The secondmeasure is a specificmode of operationwhereexisting single labels can be selected through mouse hoverin the topological view In this mode the user can betterexamine which pixels and spectra are part of a specific label


(a) Three-band composite (b) True color (c) PCA false color (d) SOM false color

Figure 9 Color visualizations of Indian Pines image SOM visualization is more feature-rich when compared to PCA

Several labels can be selected at once and then be eitherdeleted or merged using the label manager (Figure 1(e))

Finally a label can be used to initialize a multibandlimited highlight The limits on each band are set to includeall pixels contained in the label This helps both in findingsimilar spectra that are not yet included in the label and inseparating several clusters within the label by adjusting thelimits The remaining selection can then be added to anotherlabel or form a new label

Gerbil is a very versatile tool for the creation of labelingsfor further use A common use-case is the creation ofground truth data for evaluation of algorithms For thistedious task the user can start from a high-quality clusteringor segmentation and quickly incorporate the informationcontained in all bands

5 Color Coding and Color Display

A popular and useful method of visualizing multispectraland hyperspectral data is the generation of a color imagethat preserves the spatial relations in the image While agrayscale image can present a single piece of information perpixel a color image displays three data values within the rg b color triplet There are four common categories of colordisplays for hyperspectral image data (a) Band compositethree image bands are selected and then mapped to r g bcolor channels (b) Data fusion a subset of bands are fusedaccording to a specific criterionMost often this is the humancolor perception modeled by the CIE 119883119884119885 color-matchingfunctions [30] (c) Dimensionality reduction the entire spec-tral information is reduced to three dimensions which aremapped to r g b (d) Feature extraction a combination ofapplication-specific indicators (eg contributions of threeuser-selected endmembers to each spectrum) forms a false-color image For example in [50] false-color images candepict the occurrence of forest types to assess forest structure

In line with other analysis frameworks Gerbil pro-vides color displays based on human color perception (truecolor) and on linear dimensionality reduction via principalcomponent analysis (PCA) Additionally a fast nonlineardimensionality reduction method based on a self-organizingmap (SOM) was developed Figure 9 depicts the results of thethree methods on an example image

51 Human Color Perception For images whose spectrumlies within the range of visible light an intuitive naturalrepresentation can be formed by mimicking human colorperception This representation is often referred to as truecolor The CIE 1931 colorimetric system defines three color-matching functions 119909(120582) 119910(120582) 119911(120582) [30] These functionsdescribe the spectral sensitivity of a human observer calledthe CIE 1931 standard colorimetric observer The tristimulusvalues 119883 119884 and 119885 are then obtained by integrating theproduct of observed intensities and the respective color-matching function over the wavelength 120582 In practice theintegrations are replaced by summations over bands of equalwidth Δ120582 centered at wavelength 120582 [30] Popular filter-basedmultispectral sensors capture the incoming light with Δ120582 =10 The 119883 119884 and 119885 values are directly used in the CIE119883119884119885 color space They can be transformed to RGB given areference white and a gamma correction value We displaytrue-color images in sRGB space as it comes with defaultvalues for both and is specifically designed for computerdisplays [51]

Scene illumination plays an important role in true-colorvisualization As hyperspectral images provide a completespectral response of each pixel and homogeneous lightingconditions are present in most data capture scenarios illu-mination is easy to manipulate Gerbil offers removal andexchange of illuminants [6] and includes a set of referenceilluminants modeled as black-body radiators [30] Apartfrom a true-color image with applied reillumination theeffect of the illuminant is also illustrated within the spectraldistribution plot

52 Principal Component Analysis Objects of varied reflec-tance properties can yield the same color sensation renderingtrue-color representations and their variants unsuitable forseveral applications As an alternative a false-color imagecan be created that puts emphasis on different characteristicsof the data A popular approach uses principal componentanalysis (PCA) [52] where the three components with max-imum variance become the r g b values of a color imageDue to the nature of PCA the per-component variancediffers significantly Therefore an automatic white balancingis performed for display purposes [52]


200

150

50

0

400 430 460 490 520 550 580 610 640 670 700

(a) Spectral distribution (b) Labeling

(c) True color (d) PCA false color

Figure 10 Displays of cropped fake and real peppers image The spectral distribution (a) is label colored (b) shows the respective labeling asan overlay on the image band at 530 nm

Using three bases that explain most of the variancein the data is not always the most helpful representationfor discerning relevant spectra Figure 10 shows a counterexample The contrast of the green peppers against thebackground in Figure 10(d) is very low It is a direct effectof the low object reflectance when compared to the red andyellow peppers in the image as can be seen in Figure 10(a)This is unintuitive to human observers (Figure 10(c)) whohave a high sensitivity for green We would expect the false-color display to better distinguish the green pepper from thebackground

The simultaneous availability of the parallel coordinatesrepresentation and the PCA-based false-color visualizationallows one to easily judge the quality of the dimensionalityreduction in comparison to the original image data

53 Self-Organizing Map A self-organizing map (SOM)converts the nonlinear statistical relationship between high-dimensional data into simpler geometric relationships [53]In other words a SOM provides a topological representationof the spectral vector distribution of a multispectral imageA typical SOM consists of a 1D array or 2D grid of modelvectors (also referred to as neurons) that represent vectors inthe original data space The 3D SOM variant extends a 4-connected 2D lattice to a 6-connected 3D lattice [54] Thethird dimension often enables the SOM to learn a moreaccurate topology on hyperspectral data We exploit thistopology for false coloring by mapping the 3D coordinatesto the RGB cube [12 55 56]

531 Training and Traditional Approach A SOM consists of119899 model vectors m119895 isin R119863 with 119863 corresponding to thenumber of bands in the input imageThe 3D SOM is a cube ofside length 1198991015840 = 3radic119899 For any input vector x (a pixelrsquos captured

spectrum)we can find the best-matching unit (BMU)m119888withindex

119888(x) = argmin119895

119889 (xm119895) (7)

where 119889(sdot) is the Euclidean distance In our experimentswe did not observe any advantage by changing the distancefunction for example to the spectral angleThe location r(119895) isinZ3 of a neuron m119895 describes its position in the 3D latticetopology and is a bijective mapping of 119895

Our SOM training is unsupervised and uses 119904 unlabeledpixels from the input imageModel vectors are initializedwithrandom values In each iteration 1 le 119905 le 119904 we randomlydraw a spectrum x We first determine the BMUm119888 of inputxThen allmodel vectors are updated asmlowast119895 = ℎ119888119895(119905)sdot(xminusm119895)where ℎ119888119895(119905) defines the influence of x on the model vectorsm119895 based on the SOM topology As suggested by Kohonen[53]

ℎ119888119895 (119905) = 120572 (119905) sdot exp(minus10038171003817100381710038171003817r(119888) minus r(119895)10038171003817100381710038171003817221205902 (119905) ) (8)

The learning-rate factor 120572(119905) is a user-adjustable parame-ter that is monotonically decreasing The kernel width 120590(119905)describes the sphere of influence of a sample vector in theSOM topology and is also monotonically decreasing Duringthe early training phase a SOM seeks a rough global orderingIn later iterations local regions are smoothed out We trainthe SOM on the input image itself on the fly and do notdepend on any prior knowledge

The trained SOM is then used as input for false coloringIn previous work [55] a false-color image has been createdby finding BMU 119888(x) for each image pixel x (7) Then anr g b triplet is created by scaling r(119888) by 2551198991015840 However


this method leads to suboptimal visual quality and achieveseven lower entropy values than PCA false coloring Thereason is the limited amount of model vectors resulting in astrongly quantized output In the configuration of [55] only64 different color values could be produced While we useSOMs of size 119899 = 103 we are still far from the capabilities ofan 8-bit color display Training a SOM of larger sizes how-ever would become infeasible both due to longer learningtimes and due to the limited amount of available trainingsamples

532 Improved Look-Up for False Coloring To avoid theaforementioned quantized output we introduce a novel look-up method [12] Firstly instead of using a single best-matching unit we develop a set of best-matching units(BMUs) Furthermore we order the BMUs according to the1198712 distance and assign a set of predetermined weights tothe ordered set These rank-based weights are crucial Whilea simple unweighed combination would only smooth theresult using the 1198712 distances directly as weights is not reliable[57] In our high-dimensional space distances would appearvery close to each other and the weights would not discernwell We define rank weights that ensure both a majoritycontribution by the first BMU and significant contributionsby the additional BMUs

Consider as in [58] a vector of BMU indices

c(x) = argminj

119862sum119896=1

119889 (xm119895119896) (9)

where 119862 is the number of desired BMUs For each pixel valuex we calculate a representative location r1015840 as

r1015840 = 119862sum119896=1

119908119896 sdot r(119888(x)119896 ) (10)

given weights 119908119896 with the properties

119908119896 = 2119908119896+1119862sum119896=1

119908119896 = 1forall119908119896 119896 lt 119862

119889 (xm119888119896) lt 119889 (xm119888119896+1) forallm119888119896 119896 lt 119862

(11)

which expresses that the BMUs are sorted according todistance to the query vector and weighed by their rankwhere the weight for rank 119896 is always twice as high as theweight for subsequent rank 119896 + 1 The representative location

r1015840 = (11990310158401 11990310158402 11990310158403)describes the position in the learned topologythat best represents x We finally obtain

r = 119903101584011198991015840 g = 119903101584021198991015840 b = 119903101584031198991015840

(12)

Our method gives a high-quality nonlinear dimension-ality reductionThe advantage over previous false-color workbased on SOMs is that we do not suffer from quantizationeffects The advantage to other nonlinear methods for exam-ple ISOMAP [59] is speed While ISOMAP takes minutes tohours to compute we train a SOMwith 119899 = 103 in less than 20seconds on a typical hyperspectral remote sensing image Asmost computation is spent in training execution time is notsignificantly affected by image size Rather it mostly dependson the SOM size and the dimensionality of the data

The SOM itself can be visualized within Gerbil during aswell as after its training The topological views correspondto the traditional SOM-based data visualization and revealthe SOM topology The parallel coordinate plots depict thedistribution of the model vectors and make it visuallycomparable to the distribution of the training data Thesevisualizations help in both assessing the training and in fine-tuning its parameters Figure 11 depicts a remote sensingimage and its spectral distribution view as compared to thespectral distribution of a SOM trained on the image as wellas the ten principal components computed by PCA

The true-color and false-color display capabilities ofGerbil provide and extend the state of the art in hyperspec-tral visualization They are especially helpful for selectingregions of interest (ROIs) that should be examined Inlarge images due to clutter and computational expenseit is beneficial to select a ROI before performing furthervisualization tasks Additionally the false-color displays cangive a good first impression of the data for example tospot the same reflectance in several regions of the image orto find inhomogeneities within a depicted object The useris directed to specific regions for starting an investigationAn illustrating example is the fake and real peppers imageas depicted in Figure 12 In the SOM false-color displaypixels with the same material or reflectance properties areconsistently colored One such example are the specularhighlights Another instance is the stems of the plastic andreal peppers as annotated in Figure 12

6 Software Framework

Gerbil is built around the idea of being able to form a base forother research projects We ensure this by working with codelibraries that are already popular in image processing and bya modular and easily expandable design

61 Software Foundation Gerbil is written in C++ It relieson OpenCV [8] for all image processing tasks and for the


(a) Image region

2448

2354

2255

1530

1322

1160

9965

4848

4396

4013

327e + 04

245e + 04

164e + 04

82e + 03

0

5415

616

7155

844

1671

1803

2042

2152

(b) Original distribution

1

0

2448

2354

2255

2152

2042

1803

1671

1530

1322

1160

9965

844

7155

616

5415

4848

4396

4013

(c) SOM distribution

162e + 05

102e + 05

43e + 04

minus17e + 04

minus76e + 04

10987654321

(d) Image PCA distribution

Figure 11 Center part of the DC Mall image and parallel coordinates visualizations computed from the data Pixels and spectral vectors arecolored using the SOM visualized in (c)

Plastic green pepper stem

Organic green pepperstem

Figure 12 Figure 1(e) spectral gradient SOM display The occur-rences of two materials in their respective color coding (blue andlight gray) are annotated with circles

internal representation of multispectral data Due to thetight integration of Gerbil and OpenCV it is very easy toapplyOpenCVrsquos extensive functionality onmultispectral datawithin Gerbil Furthermore when available the softwareutilizes CUDA through OpenCV for some of its common

calculations For image IO we incorporate the GeospatialData Abstraction Library (GDAL) [17] It reads a large varietyof image formats from various sources including FITS LANGIS and the ENVIhdr header formats

For all operations related to the graphical user interfacethe Qt framework [60] is used Together with a CMake-basedbuild systemwemaintain a truly cross-platform software thatruns on a variety of operating systems including all recentversions of Windows MacOS X and GNULinux

62 Internal Design Our software framework consists ofboth a powerful core and a modular system of extensions Inits core Gerbil provides a flexible easy to use programminginterface to the image data It is built on an internal represen-tation of multispectral images that combines both per-bandand per-pixel data structures [6]

621 Modularity Gerbil is organized in modules that workindependently of each other and of the user interface Amodule can (optionally) depend on external libraries andother modules Both the graphical user interface of Gerbiland a sophisticated command-line interface are stand-aloneunits that expose the functionality of the othermodules to theuserThe command-line interface enables batch processing ofimage analysis tasks


Cmd runner

Dist view

BackgroundView Presenter Model

Viewport

ViewportViewport

Graph cut

Graph cutIllumination

Illumination

Normalization

Clustering

Clustering

Window

Band view

Labeling

Labeling

RGB view

ROI selector

Dist viewDist view Dist viewDist viewDist viewDist view

Main

Docks

Image

False color

Task queue

Cmd runner

Figure 13 GUI software design Presenter components control bothview and model components

622 Parallelization and Backgrounding Performance andresponsiveness are both crucial for interactive analysisOpenCVutilizes vector instruction sets inCPU computationIn order to employ all CPU cores we parallelize computa-tionally demanding tasks via Intel Thread Building Blocks[61] We also heavily recycle previously computed data Forexample image representations and distribution histogramsare incrementally updated when the current region of interestis changed

Computations need to run in the background to keep theinterface responsive to the user and enable the user to cancela running task We introduce a custom shared datatype withlocking facilities and an atomic swap operation Backgroundtasks calculate new data on their own shadow copy and lockthe data for a single swapwhen finished By using a sequentialbackground queue and defined data swaps at the end of eachtask we guarantee that at most two copies of the data areheld in memory at once An exception to this are tasks thatcan run for a longer time on their own and are not part ofthe regular user interaction Examples are global clusteringor computation of false-color representationsThese tasks runindependently on their own data copy

623 GUI Software Design A significant level of sophistica-tion is required in creating Gerbilrsquos GUI in order to exposethe frameworkrsquos functionality to the user while efficientlyreacting to user input We model the GUI module after themodelndashviewndashpresenter [62] design pattern Figure 13 gives anoverview of the GUI components View components displaydata and receive user input Image data as well as otherrepresentations for example distribution histograms is han-dled by the respectivemodel componentsThese componentshave access to both the background queue to enqueue tasksand may spawn command runners which can run in thebackground The presenter components handle user requestsand trigger calculationsThey are in control of both view andmodel components which do not see each other For datasynchronization Qtrsquos signalslot messaging [60] is employed

With this design we ensure a strict separation betweenmodels and views and reduce unnecessary overhead in datasynchronization code where signals can be passed through

624 Extensibility With our software design it is easy toextend the functionality of Gerbil New methods can beintegrated as modules in the framework They benefit froma defined interface for parameter setting and commandexecution through the command-line interface They mayalso (optionally) depend on other modules which provide astandardized method call To add a new method in the GUIa new dock widget can be created as a view component whileinterfacing data calculation with the corresponding modulein a model component The high level of abstraction onseveral levels keeps interfaces simple and prevents side effectsof changes in other components

7 Conclusions

Gerbil contains several key components that are novel andessential to interactive visual analysis of multispectral andhyperspectral image data We introduce interactive visual-ization via parallel coordinates supervised segmentation viaa graph-cut algorithm family fast global clustering usinga combination of superpixel segmentation and mean shiftdensity estimation and fast nonlinear false coloring basedon a 3D self-organizing map In Gerbil we tightly integratethese algorithms into a consistent framework that allows theuser to explore the data in a novel way Existing research andcommercial software are often comprised of a rich toolbox ofuseful algorithms for specific applications However the goalof a general software framework that also provides intuitiveaccess to the raw image data is unmet Gerbil fills this gapwithin a modern architecture and GUI

In recent years multispectral capture has gained wideradoption across a range of applications in particular incultural heritage Domain experts who did not work withmultispectral data before need assistance in understandingand utilizing the modality The new intuition in workingwith multispectral images that Gerbil strives to provide maytherefore broaden the scope of multispectral analysis andfoster adoption

Gerbil is an active open-source project and acceptscontributions on Github [63] As of May 2016 it includes25 902 physical source lines of code (SLOC) in 326 sourcecode files with a total of 37 273 lines In the past 12 months3622 downloads of theGerbil software package were countedoriginating from 45 countries

For future work more methods that are established inremote sensing need to be incorporated and adapted to inter-active analysis as done in [33] Drawing techniques for thespectral distribution views should be further investigated forexample for clutter reduction Gerbil is an evolving projectthat constantly seeks more input from diverse applicationdomains

Competing Interests

The authors declare that they have no competing interests


Acknowledgments

The authors gratefully acknowledge funding of this work bythe German Research Foundation (GRK 1773 HeterogeneousImage Systems andOpenAccess Publishing Programme) andfunding of the Gerbil open-source project by the EuropeanSpace Agency (ESA SOCIS 2012ndash2016)

References

[1] R O Green M L Eastwood C M Sarture et al ldquoImagingspectroscopy and the Airborne VisibleInfrared Imaging Spec-trometer (AVIRIS)rdquo Remote Sensing of Environment vol 65 no3 pp 227ndash248 1998

[2] F A Kruse A B Lefkoff J W Boardman et al ldquoThe SpectralImage Processing System (SIPS)mdashinteractive visualization andanalysis of imaging spectrometer datardquo Remote Sensing ofEnvironment vol 44 no 2-3 pp 145ndash163 1993

[3] N Gat ldquoImaging spectroscopy using tunable filters a reviewrdquoin Proceedings of the Society of Photo-Optical InstrumentationEngineers (SPIE) Conference Series vol 4056 of SPIE Societyof Photo-Optical Instrumentation Engineers pp 50ndash64 Belling-ham Wash USA 2000

[4] J Fisher M M Baumback J H Bowles J M Grossmannand J A Antoniades ldquoComparison of low-cost hyperspectralsensorsrdquo in Proceedings of the SPIErsquos International Symposiumon Optical Science Engineering and Instrumentation vol 3438pp 23ndash30 San Diego Calif USA July 1998

[5] Forth Photonics ldquoMuSISrdquo July 2012 httparchiveismusisforth-photonicscom

[6] J Jordan and E Angelopoulou ldquoGerbilmdasha novel softwareframework for visualization and analysis in the multispectraldomainrdquo in Vision Modeling and Visualization pp 259ndash2662010

[7] C P Huynh and A Robles-Kelly ldquoA probabilistic approachto spectral unmixingrdquo in Structural Syntactic and StatisticalPattern Recognition pp 344ndash353 Springer Berlin Germany2010

[8] G Bradski ldquoOpen Source Computer Vision Libraryrdquo Dr DobbrsquosJournal of Software Tools 2016 httpopencvorg

[9] M Hall E Frank G Holmes B Pfahringer P Reutemann andI H Witten ldquoThe WEKA data mining software an updaterdquoACM SIGKDD Explorations Newsletter vol 11 no 1 pp 10ndash182009

[10] J Jordan andEAngelopoulou ldquoSupervisedmultispectral imagesegmentation with power watershedsrdquo in Proceedings of the 19thIEEE International Conference on Image Processing (ICIP rsquo12)pp 1585ndash1588 IEEE Orlando Fla USA September-October2012

[11] J Jordan and E Angelopoulou ldquoMean-shift clustering forinteractive multispectral image analysisrdquo in Proceedings of the20th IEEE International Conference on Image Processing (ICIPrsquo13) pp 3790ndash3794 IEEE Melbourne Australia September2013

[12] J Jordan and E Angelopoulou ldquoHyperspectral image visu-alization with a 3-D self-organizing maprdquo in Proceedings ofthe Workshop on Hyperspectral Image and Signal ProcessingEvolution in Remote Sensing (WHISPERS rsquo13) pp 1ndash4 IEEE2013

[13] JW Boardman L L Biehl R N Clark et al ldquoDevelopment andimplementation of software systems for imaging spectroscopyrdquo

in Proceedings of the IEEE International Geoscience and RemoteSensing Symposium (IGARSS rsquo06) pp 1969ndash1973 IEEE July-August 2006

[14] L Biehl ldquoEval-ware hyperspectral imaging [best of the web]rdquoIEEE Signal Processing Magazine vol 24 no 4 pp 125ndash1262007

[15] P E Dennison K Q Halligan and D A Roberts ldquoA compari-son of error metrics and constraints for multiple endmemberspectral mixture analysis and spectral angle mapperrdquo RemoteSensing of Environment vol 93 no 3 pp 359ndash367 2004

[16] L Biehl and D Landgrebe ldquoMultiSpecmdasha tool for multispect-ral-hyperspectral image data analysisrdquo Computers amp Geo-sciences vol 28 no 10 pp 1153ndash1159 2002

[17] F Warmerdam ldquoThe geospatial data abstraction libraryrdquo inOpen Source Approaches in Spatial Data Handling G B HallM G Leahy S Balram and S Dragicevic Eds vol 2 pp 87ndash104 Springer Berlin Germany 2008

[18] Harris Geospatial Solutions ldquoENVIrdquo 2016 httpwwwhar-risgeospatialcomProductsandSolutionsGeospatialProductsENVIaspx

[19] J Boardman F Kruse andRGreen ldquoMapping target signaturesvia partial unmixing of AVIRIS datardquo in Proceedings of theSummaries 5th JPL Airborne Earth ScienceWorkshop vol 1 pp23ndash26 1995

[20] F Yasuma T Mitsunaga D Iso and S Nayar ldquoGeneral-ized assorted pixel camera post-capture control of resolutiondynamic range and spectrumrdquo Tech Rep ColumbiaUniversity2008

[21] D H Foster K Amano S M C Nascimento and M J FosterldquoFrequency of metamerism in natural scenesrdquo Journal of theOptical Society of AmericaA vol 23 no 10 pp 2359ndash2372 2006

[22] Purdue Research Foundation ldquoHyperspectral imagesrdquo May2016 httpsengineeringpurdueedu biehlMultiSpechyper-spectralhtml

[23] United States Army Corpse of Engineers ldquoHypercuberdquo May2016 httpwwwerdcusacearmymilMediaFactSheetsFact-SheetArticleViewtabid9254Article610433hypercubeaspx

[24] Ball Aerospace amp Technologies Corp ldquoOpticksrdquo May 2016httpsopticksorg

[25] ldquoOpticks-Spectral Processing Extensionrdquo July 2012 httpsopticksorgdisplayopticksExt

[26] M Cui A Razdan J Hu and P Wonka ldquoInteractive hyper-spectral image visualization using convex optimizationrdquo IEEETransactions on Geoscience and Remote Sensing vol 47 no 6pp 1673ndash1684 2009

[27] W A Joye and E Mandel ldquoNew features of SAOImage DS9rdquo inAstronomical Data Analysis Software and Systems XII vol 295pp 489ndash492 2003

[28] H Li C-W Fu and A J Hanson ldquoVisualizingmultiwavelengthastrophysical datardquo IEEE Transactions on Visualization andComputer Graphics vol 14 no 6 pp 1555ndash1562 2008

[29] P Colantoni R Pillay C Lahanier and D Pitzalis ldquoAnalysisof multispectral images of paintingsrdquo in Proceedings of theEuropean Signal Processing Conference Florence Italy 2006

[30] G Wyszecki and W S Stiles Color Science Concepts andMethods Quantitative Data and Formulae Wiley-InterscienceNew York NY USA 2nd edition 2000

[31] S J Kim S Zhuo F Deng C-W Fu andM Brown ldquoInteractivevisualization of hyperspectral images of historical documentsrdquoIEEE Transactions on Visualization and Computer Graphics vol16 no 6 pp 1441ndash1448 2010


[32] National ICT Australia Limited ldquoScyvenrdquo May 2016 httpwwwscyvencom

[33] B Labitzke S Bayraktar and A Kolb ldquoGeneric visual analysisfor multi- and hyperspectral image datardquo Data Mining andKnowledge Discovery vol 27 no 1 pp 117ndash145 2013

[34] N Keshava and J F Mustard ldquoSpectral unmixingrdquo IEEE SignalProcessing Magazine vol 19 no 1 pp 44ndash57 2002

[35] E Angelopoulou S W Lee and R Bajcsy ldquoSpectral gradient amaterial descriptor invariant to geometry and incident illumi-nationrdquo in Proceedings of the 7th IEEE International Conferenceon Computer Vision (ICCV rsquo99) vol 2 pp 861ndash867 KerkyraGreece September 1999

[36] A Inselberg and B Dimsdale ldquoParallel coordinates a tool forvisualizingmulti-dimensional geometryrdquo in in IEEE Conferenceon Visualization pp 361ndash378 1990

[37] R Gonzalez RWoods and S EddinsDigital Image ProcessingPrentice Hall Press New York NY USA 3rd edition 2008

[38] G Ellis and A Dix ldquoEnabling automatic clutter reduction inparallel coordinate plotsrdquo IEEE Transactions on Visualizationand Computer Graphics vol 12 no 5 pp 717ndash724 2006

[39] R Fuchs and H Hauser ldquoVisualization of multi-variate scien-tific datardquo Computer Graphics Forum vol 28 no 6 pp 1670ndash1690 2009

[40] C Couprie L Grady L Najman and H Talbot ldquoPowerwatershed a unifying graph-based optimization frameworkrdquoIEEE Transactions on Pattern Analysis andMachine Intelligencevol 33 no 7 pp 1384ndash1399 2011

[41] Y Du C-I Chang H Ren C-C Chang J O Jensen and FM DrsquoAmico ldquoNew hyperspectral discrimination measure forspectral characterizationrdquoOptical Engineering vol 43 no 8 pp1777ndash1786 2004

[42] A E Gutierrez-Rodrıguez M A Medina-Perez J F Martınez-Trinidad J A Carrasco-Ochoa and M Garcıa-Borroto ldquodis-similarity measures for ultraviolet spectra identificationrdquo inAdvances in Pattern Recognition SecondMexican Conference onPattern RecognitionMCPR 2010 PueblaMexico September 27ndash29 2010 Proceedings vol 6256 of Lecture Notes in ComputerScience pp 220ndash229 Springer New York NY USA 2010

[43] S Bo L Ding H Li F Di and C Zhu ldquoMean shift-basedclustering analysis of multispectral remote sensing imageryrdquoInternational Journal of Remote Sensing vol 30 no 4 pp 817ndash827 2009

[44] X Huang and L Zhang ldquoAn adaptive mean-shift analysisapproach for object extraction and classification from urbanhyperspectral imageryrdquo IEEE Transactions on Geoscience andRemote Sensing vol 46 no 12 pp 4173ndash4185 2008

[45] D Comaniciu and P Meer ldquoMean shift a robust approachtoward feature space analysisrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 24 no 5 pp 603ndash6192002

[46] B Georgescu I Shimshoni and P Meer ldquoMean shift basedclustering in high dimensions a texture classification examplerdquoin Proceedings of the 9th IEEE International Conference onComputer Vision pp 456ndash463 October 2003

[47] M Fauvel J A Benediktsson J Chanussot and J R SveinssonldquoSpectral and spatial classification of hyperspectral data usingSVMs and morphological profilesrdquo IEEE Transactions on Geo-science and Remote Sensing vol 46 no 11 pp 3804ndash3814 2008

[48] D R Thompson L Mandrake M S Gilmore and R CastanoldquoSuperpixel endmember detectionrdquo IEEE Transactions on Geo-science and Remote Sensing vol 48 no 11 pp 4023ndash4033 2010

[49] P F Felzenszwalb andD PHuttenlocher ldquoEfficient graph-basedimage segmentationrdquo International Journal of Computer Visionvol 59 no 2 pp 167ndash181 2004

[50] S L Ustin and A Trabucco ldquoUsing hyperspectral data to assessforest structurerdquo Journal of Forestry vol 98 no 6 pp 47ndash492000

[51] M Stokes M Anderson S Chandrasekar and R Motta ldquoAstandard default color space for the internet-sRGBrdquo Tech RepMicrosoft and Hewlett-Packard 1996 httpwwwcolororgsRGBxalter

[52] N P Jacobson and M R Gupta ldquoDesign goals and solutionsfor display of hyperspectral imagesrdquo IEEE Transactions onGeoscience and Remote Sensing vol 43 no 11 pp 2684ndash26922005

[53] T Kohonen Self-Organizing Maps vol 30 of Springer Series inInformation Sciences Springer Berline Germany 3rd edition2001

[54] J Jordan and E Angelopoulou ldquoEdge detection in multispec-tral images using the n-dimensional self-organizing maprdquo inProceedings of the 18th IEEE International Conference on ImageProcessing (ICIP rsquo11) pp 3181ndash3184 IEEE Brussels BelgiumSeptember 2011

[55] J Gorricha and V Lobo ldquoImprovements on the visualizationof clusters in geo-referenced data using self-organizing mapsrdquoComputers amp Geosciences vol 43 pp 177ndash186 2012

[56] J M Fonville C L Carter L Pizarro et al ldquoHyperspectralvisualization of mass spectrometry imaging datardquo AnalyticalChemistry vol 85 no 3 pp 1415ndash1423 2013

[57] K Beyer J Goldstein R Ramakrishnan and U Shaft ldquoWhen islsquonearest neighborrsquo meaningfulrdquo inDatabaseTheorymdashICDTrsquo99C Beeri and P Buneman Eds vol 1540 of Lecture Notes inComputer Science pp 217ndash235 1999

[58] M Sjoberg and J Laaksonen ldquoOptimal combination of SOMsearch in best-matching units and map neighborhoodrdquo inAdvances in Self-Organizing Maps vol 5629 pp 281ndash289Springer Berlin Germany 2009

[59] C M Bachmann T L Ainsworth and R A Fusina ldquoImprovedmanifold coordinate representations of large-scale hyperspec-tral scenesrdquo IEEE Transactions on Geoscience and RemoteSensing vol 44 no 10 pp 2786ndash2803 2006

[60] The Qt Company ldquoQt-cross-platform application and UIframeworkrdquo 2016 httpqt-projectorg

[61] J Reinders Intel Threading Building Blocks Outfitting C++ forMulti-Core Processor Parallelism OrsquoReilly Media 2010

[62] M PotelMVPModel-View-Presenter the Taligent ProgrammingModel for c++ and Java Taligent 1996

[63] J Jordan Gerbil Hyperspectral Visualization and AnalysisFramework 2016 httpgithubcomgerbilvisgerbil

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014


Active and Passive Electronic Components

Control Scienceand Engineering

Journal of



RotatingMachinery


Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design



Shock and Vibration


Civil EngineeringAdvances in

Acoustics and VibrationAdvances in



Electrical and Computer Engineering

Journal of

Advances inOptoElectronics


Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of


Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014


Chemical EngineeringInternational Journal of Antennas and

Propagation




Navigation and Observation



DistributedSensor Networks



(a)

(b)

(c)

(d)

(e)

Figure 1 Gerbil user interface (a) Spectral distribution view (b)spectral gradient distribution view (c) spatial view of a single imageband with segment overlays (d) label manager and (e) false-colordisplay

new applications Various devices based on electronicallytunable filters [3] or imaging spectrometers [4] are com-mercially available from several vendors An example ofsensor targeting emerging applications is MuSIS [5] which iscurrently in use in several museums and libraries Howeverthe users of these sensors lack a software solution that enablesthem to explore the data in an interactive and intuitivefashion

To address this limitation we developed the Gerbilsoftware platform which incorporates established interactivevisualization concepts to provide a new presentation of spec-tral images as well as a new workflow for inspecting themFigure 1 depicts a screenshot of Gerbil in use for inspecting amultispectral image Gerbil is developed by an open-sourcesoftware project in the tradition of free software frame-works in the signal processing research community such asOpenCV [8] orWeka [9] In this paper we provide a scientificreference for Gerbil and describe the core methodologies itencompasses We also provide a detailed presentation of thefeasibility and performance of the algorithms in Gerbil andshow how our novel approach is a significant improvementcompared to established practice

The paper bundles and expands on the following contri-butions

(i) Interactive visualization of spectral distributionsbased on efficient parallel coordinates [6]

(ii) Supervised segmentation of hyperspectral data [10](iii) Fast global clustering with superpixel support [11](iv) Fast nonlinear false-color visualization [12]

These methods were derived by adapting establishedalgorithms for hyperspectral data and interactive time con-straints They allow us to introduce new paradigms in hyper-spectral image analysis that focus on interactive raw dataexploration built on the tight incorporation of the afore-mentioned techniques into a comprehensive open-sourcesoftware framework The software is publicly available athttpgerbilvisorg under a free software license

2 Previous Work

One of the first widely recognized software packages formultispectral analysis LARSYS became available in the1960s It was operated via a text terminal Several frameworksfor graphical interactive multispectral or hyperspectral dataanalysis that are still of broad influence today date backto the early 1990s Earlier frameworks focus on a specificapplication most prominently in the field of remote sensingBoardman et al [13] provided an overview of the history ofestablished software systems for remote sensing hyperspec-tral data Biehl [14] created an updated list on the web in2007 In this section we give first a brief overview of domain-specific software packages before reviewing more widelyapplicable software We focus on the interactive inspection ofimages

21 Remote Sensing The Spectral Image Processing System(SIPS) was presented by Kruse et al [2] in 1993 It introducedthe Spectral Angle Mapper (SAM) a spectrum-comparisontool still popular in the field of spectral matching [15] Datawas presented either as single bands or in a false-coloredcomposite of three user-selectable bandsThe user could alsoview a selected pixelrsquos spectrum Color-coded stacked spectrawere provided for a selected slice (a vertical or horizontalline scan or an arbitrary path in the image) This set ofvisualization forms are still typical of popular software forhyperspectral capture or analysis

The MultiSpec freeware package by Biehl and Landgrebe[16] can analyze multispectral images from various sourcesusing the powerful GDAL library [17] for data IO Thefocus of this software is classification The target audience isthe general Earth science community Besides providing allcommon visualizations already mentioned it can generatebiplots that relate selected regions in a pair of bands and astatistics image display which depicts the correlation betweenthese bands

A widely used commercial software is ENVI [18] initiallydeveloped by Boardman et al [13] Several innovations inhyperspectral analysis were introduced in ENVI includingfor example the Pixel Purity Index (PPI) [19] PPI finds themost spectrally pure pixels in an imageThese pixels typicallycorrespond to endmembers (constituent spectra of an image)A notable innovation in visualization is the 119899-DimensionalVisualizer which uses PPI as input It is an interactive datavisualization method that allows real-time rotation of scat-terplots in 119899-dimensions The presentation of 119899-dimensionalscatterplots in 2D can be somewhat unintuitive Yet the toolis valuable to experts for identification of endmembers basedon the depicted point clouds

Two other notable pieces of software for hyperspectralremote sensing data analysis areHyperCube [23] andOpticks[24] HyperCube is released by the US Army Corps ofEngineers and contains functions to filter warp (registertwo images) calibrate and undistort photometrically projectand arithmetically manipulate the data Opticks also includesmany common tools like GIS annotations false coloring andhistograms Additionally third parties can provide function-ality through an external module adding further capabilities











3 Visual Inspection







Globalclustering

Labelingtools

Extensibledesign

Illuminantcontrol

False-colordisplay

GerbilSection 41

Section 42


Section 51

Sections 32 33

Sections 52 53








120572 = 119888120572119891 (119899bin + 1)119891 (119899total) (1)




245

184

123

62

0

700

670

640

610

580

550

520

490

460

430

400

(b) Previous work [6] 119861 = 20

245

184

123

62

0

700

670

640

610

580

550

520

490

460

430

400


245

184

123

62

0

700

670

640

610

580

550

520

490

460

430

400








RMSE = 119873sum119894=1

radicsum119863119895=1 (119894119895 minus 119909119894119895)2 119863119873 (2)










253

240

227

214

201

188

175

162

149

136

123

110

97

84

71

58

45

32

19


Dev

iatio

n



4

3

2

1

0

001

01

1

0001

Seco

nds








CAVE3231 0089 0161 0002 00276431 0115 0240 0001 001425631 0177 0390 0000 0003

Foster3233 0274 0146 0005 00316433 0358 0554 0002 001425633 1085 2955 0000 0004

DC Mall32191 0585 1789 0001 002664191 0881 3171 0000 0013256191 1020 3554 0000 0003






















ylowast = argminy

sum119890119894119895isin119864

119908119901119894119895 10038161003816100381610038161003816119910119894 minus 11991011989510038161003816100381610038161003816119902 +sumV119894119908119901119865119894 10038161003816100381610038161199101198941003816100381610038161003816119902

+sumV119894119908119901119861119894 1003816100381610038161003816119910119894 minus 11003816100381610038161003816119902




119908119894119895 = exp (minus120573 (119889 (x119894 x119895))2) (5)














(c) First labeling

16

095

03

minus1

685

665

645

625

605

585

565

545

525

505

485

465

445

425

405

minus035


(e) Second labeling

16

095

03

minus1

685

665

645

625

605

585

565

545

525

505

485

465

445

425

405

minus035
































200

150

50

0

400 430 460 490 520 550 580 610 640 670 700









119888(x) = argmin119895

119889 (xm119895) (7)










c(x) = argminj

119862sum119896=1

119889 (xm119895119896) (9)


r1015840 = 119862sum119896=1

119908119896 sdot r(119888(x)119896 ) (10)


119908119896 = 2119908119896+1119862sum119896=1

119908119896 = 1forall119908119896 119896 lt 119862


(11)



r = 119903101584011198991015840 g = 119903101584021198991015840 b = 119903101584031198991015840

(12)








(a) Image region

2448

2354

2255

1530

1322

1160

9965

4848

4396

4013

327e + 04

245e + 04

164e + 04

82e + 03

0

5415

616

7155

844

1671

1803

2042

2152


1

0

2448

2354

2255

2152

2042

1803

1671

1530

1322

1160

9965

844

7155

616

5415

4848

4396

4013


162e + 05

102e + 05

43e + 04

minus17e + 04

minus76e + 04

10987654321












Cmd runner

Dist view


Viewport

ViewportViewport

Graph cut


Illumination

Normalization

Clustering

Clustering

Window

Band view

Labeling

Labeling

RGB view

ROI selector


Main

Docks

Image

False color

Task queue

Cmd runner







7 Conclusions





Competing Interests



Acknowledgments


References




































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation



















3 Visual Inspection







Globalclustering

Labelingtools

Extensibledesign

Illuminantcontrol

False-colordisplay

GerbilSection 41

Section 42


Section 51

Sections 32 33

Sections 52 53








120572 = 119888120572119891 (119899bin + 1)119891 (119899total) (1)




245

184

123

62

0

700

670

640

610

580

550

520

490

460

430

400

(b) Previous work [6] 119861 = 20

245

184

123

62

0

700

670

640

610

580

550

520

490

460

430

400


245

184

123

62

0

700

670

640

610

580

550

520

490

460

430

400








RMSE = 119873sum119894=1

radicsum119863119895=1 (119894119895 minus 119909119894119895)2 119863119873 (2)










253

240

227

214

201

188

175

162

149

136

123

110

97

84

71

58

45

32

19


Dev

iatio

n



4

3

2

1

0

001

01

1

0001

Seco

nds








CAVE3231 0089 0161 0002 00276431 0115 0240 0001 001425631 0177 0390 0000 0003

Foster3233 0274 0146 0005 00316433 0358 0554 0002 001425633 1085 2955 0000 0004

DC Mall32191 0585 1789 0001 002664191 0881 3171 0000 0013256191 1020 3554 0000 0003






















ylowast = argminy

sum119890119894119895isin119864

119908119901119894119895 10038161003816100381610038161003816119910119894 minus 11991011989510038161003816100381610038161003816119902 +sumV119894119908119901119865119894 10038161003816100381610038161199101198941003816100381610038161003816119902

+sumV119894119908119901119861119894 1003816100381610038161003816119910119894 minus 11003816100381610038161003816119902




119908119894119895 = exp (minus120573 (119889 (x119894 x119895))2) (5)














(c) First labeling

16

095

03

minus1

685

665

645

625

605

585

565

545

525

505

485

465

445

425

405

minus035


(e) Second labeling

16

095

03

minus1

685

665

645

625

605

585

565

545

525

505

485

465

445

425

405

minus035
































200

150

50

0

400 430 460 490 520 550 580 610 640 670 700









119888(x) = argmin119895

119889 (xm119895) (7)










c(x) = argminj

119862sum119896=1

119889 (xm119895119896) (9)


r1015840 = 119862sum119896=1

119908119896 sdot r(119888(x)119896 ) (10)


119908119896 = 2119908119896+1119862sum119896=1

119908119896 = 1forall119908119896 119896 lt 119862


(11)



r = 119903101584011198991015840 g = 119903101584021198991015840 b = 119903101584031198991015840

(12)








(a) Image region

2448

2354

2255

1530

1322

1160

9965

4848

4396

4013

327e + 04

245e + 04

164e + 04

82e + 03

0

5415

616

7155

844

1671

1803

2042

2152


1

0

2448

2354

2255

2152

2042

1803

1671

1530

1322

1160

9965

844

7155

616

5415

4848

4396

4013


162e + 05

102e + 05

43e + 04

minus17e + 04

minus76e + 04

10987654321












Cmd runner

Dist view


Viewport

ViewportViewport

Graph cut


Illumination

Normalization

Clustering

Clustering

Window

Band view

Labeling

Labeling

RGB view

ROI selector


Main

Docks

Image

False color

Task queue

Cmd runner







7 Conclusions





Competing Interests



Acknowledgments


References




































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation












Globalclustering

Labelingtools

Extensibledesign

Illuminantcontrol

False-colordisplay

GerbilSection 41

Section 42


Section 51

Sections 32 33

Sections 52 53








120572 = 119888120572119891 (119899bin + 1)119891 (119899total) (1)




245

184

123

62

0

700

670

640

610

580

550

520

490

460

430

400

(b) Previous work [6] 119861 = 20

245

184

123

62

0

700

670

640

610

580

550

520

490

460

430

400


245

184

123

62

0

700

670

640

610

580

550

520

490

460

430

400








RMSE = 119873sum119894=1

radicsum119863119895=1 (119894119895 minus 119909119894119895)2 119863119873 (2)










253

240

227

214

201

188

175

162

149

136

123

110

97

84

71

58

45

32

19


Dev

iatio

n



4

3

2

1

0

001

01

1

0001

Seco

nds








CAVE3231 0089 0161 0002 00276431 0115 0240 0001 001425631 0177 0390 0000 0003

Foster3233 0274 0146 0005 00316433 0358 0554 0002 001425633 1085 2955 0000 0004

DC Mall32191 0585 1789 0001 002664191 0881 3171 0000 0013256191 1020 3554 0000 0003






















ylowast = argminy

sum119890119894119895isin119864

119908119901119894119895 10038161003816100381610038161003816119910119894 minus 11991011989510038161003816100381610038161003816119902 +sumV119894119908119901119865119894 10038161003816100381610038161199101198941003816100381610038161003816119902

+sumV119894119908119901119861119894 1003816100381610038161003816119910119894 minus 11003816100381610038161003816119902




119908119894119895 = exp (minus120573 (119889 (x119894 x119895))2) (5)














(c) First labeling

16

095

03

minus1

685

665

645

625

605

585

565

545

525

505

485

465

445

425

405

minus035


(e) Second labeling

16

095

03

minus1

685

665

645

625

605

585

565

545

525

505

485

465

445

425

405

minus035
































200

150

50

0

400 430 460 490 520 550 580 610 640 670 700









119888(x) = argmin119895

119889 (xm119895) (7)










c(x) = argminj

119862sum119896=1

119889 (xm119895119896) (9)


r1015840 = 119862sum119896=1

119908119896 sdot r(119888(x)119896 ) (10)


119908119896 = 2119908119896+1119862sum119896=1

119908119896 = 1forall119908119896 119896 lt 119862


(11)



r = 119903101584011198991015840 g = 119903101584021198991015840 b = 119903101584031198991015840

(12)








(a) Image region

2448

2354

2255

1530

1322

1160

9965

4848

4396

4013

327e + 04

245e + 04

164e + 04

82e + 03

0

5415

616

7155

844

1671

1803

2042

2152


1

0

2448

2354

2255

2152

2042

1803

1671

1530

1322

1160

9965

844

7155

616

5415

4848

4396

4013


162e + 05

102e + 05

43e + 04

minus17e + 04

minus76e + 04

10987654321












Cmd runner

Dist view


Viewport

ViewportViewport

Graph cut


Illumination

Normalization

Clustering

Clustering

Window

Band view

Labeling

Labeling

RGB view

ROI selector


Main

Docks

Image

False color

Task queue

Cmd runner







7 Conclusions





Competing Interests



Acknowledgments


References




































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation











245

184

123

62

0

700

670

640

610

580

550

520

490

460

430

400

(b) Previous work [6] 119861 = 20

245

184

123

62

0

700

670

640

610

580

550

520

490

460

430

400


245

184

123

62

0

700

670

640

610

580

550

520

490

460

430

400








RMSE = 119873sum119894=1

radicsum119863119895=1 (119894119895 minus 119909119894119895)2 119863119873 (2)










253

240

227

214

201

188

175

162

149

136

123

110

97

84

71

58

45

32

19


Dev

iatio

n



4

3

2

1

0

001

01

1

0001

Seco

nds








CAVE3231 0089 0161 0002 00276431 0115 0240 0001 001425631 0177 0390 0000 0003

Foster3233 0274 0146 0005 00316433 0358 0554 0002 001425633 1085 2955 0000 0004

DC Mall32191 0585 1789 0001 002664191 0881 3171 0000 0013256191 1020 3554 0000 0003






















ylowast = argminy

sum119890119894119895isin119864

119908119901119894119895 10038161003816100381610038161003816119910119894 minus 11991011989510038161003816100381610038161003816119902 +sumV119894119908119901119865119894 10038161003816100381610038161199101198941003816100381610038161003816119902

+sumV119894119908119901119861119894 1003816100381610038161003816119910119894 minus 11003816100381610038161003816119902




119908119894119895 = exp (minus120573 (119889 (x119894 x119895))2) (5)














(c) First labeling

16

095

03

minus1

685

665

645

625

605

585

565

545

525

505

485

465

445

425

405

minus035


(e) Second labeling

16

095

03

minus1

685

665

645

625

605

585

565

545

525

505

485

465

445

425

405

minus035
































200

150

50

0

400 430 460 490 520 550 580 610 640 670 700









119888(x) = argmin119895

119889 (xm119895) (7)










c(x) = argminj

119862sum119896=1

119889 (xm119895119896) (9)


r1015840 = 119862sum119896=1

119908119896 sdot r(119888(x)119896 ) (10)


119908119896 = 2119908119896+1119862sum119896=1

119908119896 = 1forall119908119896 119896 lt 119862


(11)



r = 119903101584011198991015840 g = 119903101584021198991015840 b = 119903101584031198991015840

(12)








(a) Image region

2448

2354

2255

1530

1322

1160

9965

4848

4396

4013

327e + 04

245e + 04

164e + 04

82e + 03

0

5415

616

7155

844

1671

1803

2042

2152


1

0

2448

2354

2255

2152

2042

1803

1671

1530

1322

1160

9965

844

7155

616

5415

4848

4396

4013


162e + 05

102e + 05

43e + 04

minus17e + 04

minus76e + 04

10987654321












Cmd runner

Dist view


Viewport

ViewportViewport

Graph cut


Illumination

Normalization

Clustering

Clustering

Window

Band view

Labeling

Labeling

RGB view

ROI selector


Main

Docks

Image

False color

Task queue

Cmd runner







7 Conclusions





Competing Interests



Acknowledgments


References




































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation












253

240

227

214

201

188

175

162

149

136

123

110

97

84

71

58

45

32

19


Dev

iatio

n



4

3

2

1

0

001

01

1

0001

Seco

nds








CAVE3231 0089 0161 0002 00276431 0115 0240 0001 001425631 0177 0390 0000 0003

Foster3233 0274 0146 0005 00316433 0358 0554 0002 001425633 1085 2955 0000 0004

DC Mall32191 0585 1789 0001 002664191 0881 3171 0000 0013256191 1020 3554 0000 0003






















ylowast = argminy

sum119890119894119895isin119864

119908119901119894119895 10038161003816100381610038161003816119910119894 minus 11991011989510038161003816100381610038161003816119902 +sumV119894119908119901119865119894 10038161003816100381610038161199101198941003816100381610038161003816119902

+sumV119894119908119901119861119894 1003816100381610038161003816119910119894 minus 11003816100381610038161003816119902




119908119894119895 = exp (minus120573 (119889 (x119894 x119895))2) (5)














(c) First labeling

16

095

03

minus1

685

665

645

625

605

585

565

545

525

505

485

465

445

425

405

minus035


(e) Second labeling

16

095

03

minus1

685

665

645

625

605

585

565

545

525

505

485

465

445

425

405

minus035
































200

150

50

0

400 430 460 490 520 550 580 610 640 670 700









119888(x) = argmin119895

119889 (xm119895) (7)










c(x) = argminj

119862sum119896=1

119889 (xm119895119896) (9)


r1015840 = 119862sum119896=1

119908119896 sdot r(119888(x)119896 ) (10)


119908119896 = 2119908119896+1119862sum119896=1

119908119896 = 1forall119908119896 119896 lt 119862


(11)



r = 119903101584011198991015840 g = 119903101584021198991015840 b = 119903101584031198991015840

(12)








(a) Image region

2448

2354

2255

1530

1322

1160

9965

4848

4396

4013

327e + 04

245e + 04

164e + 04

82e + 03

0

5415

616

7155

844

1671

1803

2042

2152


1

0

2448

2354

2255

2152

2042

1803

1671

1530

1322

1160

9965

844

7155

616

5415

4848

4396

4013


162e + 05

102e + 05

43e + 04

minus17e + 04

minus76e + 04

10987654321












Cmd runner

Dist view


Viewport

ViewportViewport

Graph cut


Illumination

Normalization

Clustering

Clustering

Window

Band view

Labeling

Labeling

RGB view

ROI selector


Main

Docks

Image

False color

Task queue

Cmd runner







7 Conclusions





Competing Interests



Acknowledgments


References




































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation

























ylowast = argminy

sum119890119894119895isin119864

119908119901119894119895 10038161003816100381610038161003816119910119894 minus 11991011989510038161003816100381610038161003816119902 +sumV119894119908119901119865119894 10038161003816100381610038161199101198941003816100381610038161003816119902

+sumV119894119908119901119861119894 1003816100381610038161003816119910119894 minus 11003816100381610038161003816119902




119908119894119895 = exp (minus120573 (119889 (x119894 x119895))2) (5)














(c) First labeling

16

095

03

minus1

685

665

645

625

605

585

565

545

525

505

485

465

445

425

405

minus035


(e) Second labeling

16

095

03

minus1

685

665

645

625

605

585

565

545

525

505

485

465

445

425

405

minus035
































200

150

50

0

400 430 460 490 520 550 580 610 640 670 700









119888(x) = argmin119895

119889 (xm119895) (7)










c(x) = argminj

119862sum119896=1

119889 (xm119895119896) (9)


r1015840 = 119862sum119896=1

119908119896 sdot r(119888(x)119896 ) (10)


119908119896 = 2119908119896+1119862sum119896=1

119908119896 = 1forall119908119896 119896 lt 119862


(11)



r = 119903101584011198991015840 g = 119903101584021198991015840 b = 119903101584031198991015840

(12)








(a) Image region

2448

2354

2255

1530

1322

1160

9965

4848

4396

4013

327e + 04

245e + 04

164e + 04

82e + 03

0

5415

616

7155

844

1671

1803

2042

2152


1

0

2448

2354

2255

2152

2042

1803

1671

1530

1322

1160

9965

844

7155

616

5415

4848

4396

4013


162e + 05

102e + 05

43e + 04

minus17e + 04

minus76e + 04

10987654321












Cmd runner

Dist view


Viewport

ViewportViewport

Graph cut


Illumination

Normalization

Clustering

Clustering

Window

Band view

Labeling

Labeling

RGB view

ROI selector


Main

Docks

Image

False color

Task queue

Cmd runner







7 Conclusions





Competing Interests



Acknowledgments


References




































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation











ylowast = argminy

sum119890119894119895isin119864

119908119901119894119895 10038161003816100381610038161003816119910119894 minus 11991011989510038161003816100381610038161003816119902 +sumV119894119908119901119865119894 10038161003816100381610038161199101198941003816100381610038161003816119902

+sumV119894119908119901119861119894 1003816100381610038161003816119910119894 minus 11003816100381610038161003816119902




119908119894119895 = exp (minus120573 (119889 (x119894 x119895))2) (5)














(c) First labeling

16

095

03

minus1

685

665

645

625

605

585

565

545

525

505

485

465

445

425

405

minus035


(e) Second labeling

16

095

03

minus1

685

665

645

625

605

585

565

545

525

505

485

465

445

425

405

minus035
































200

150

50

0

400 430 460 490 520 550 580 610 640 670 700









119888(x) = argmin119895

119889 (xm119895) (7)










c(x) = argminj

119862sum119896=1

119889 (xm119895119896) (9)


r1015840 = 119862sum119896=1

119908119896 sdot r(119888(x)119896 ) (10)


119908119896 = 2119908119896+1119862sum119896=1

119908119896 = 1forall119908119896 119896 lt 119862


(11)



r = 119903101584011198991015840 g = 119903101584021198991015840 b = 119903101584031198991015840

(12)








(a) Image region

2448

2354

2255

1530

1322

1160

9965

4848

4396

4013

327e + 04

245e + 04

164e + 04

82e + 03

0

5415

616

7155

844

1671

1803

2042

2152


1

0

2448

2354

2255

2152

2042

1803

1671

1530

1322

1160

9965

844

7155

616

5415

4848

4396

4013


162e + 05

102e + 05

43e + 04

minus17e + 04

minus76e + 04

10987654321












Cmd runner

Dist view


Viewport

ViewportViewport

Graph cut


Illumination

Normalization

Clustering

Clustering

Window

Band view

Labeling

Labeling

RGB view

ROI selector


Main

Docks

Image

False color

Task queue

Cmd runner







7 Conclusions





Competing Interests



Acknowledgments


References




































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation











(c) First labeling

16

095

03

minus1

685

665

645

625

605

585

565

545

525

505

485

465

445

425

405

minus035


(e) Second labeling

16

095

03

minus1

685

665

645

625

605

585

565

545

525

505

485

465

445

425

405

minus035
































200

150

50

0

400 430 460 490 520 550 580 610 640 670 700









119888(x) = argmin119895

119889 (xm119895) (7)










c(x) = argminj

119862sum119896=1

119889 (xm119895119896) (9)


r1015840 = 119862sum119896=1

119908119896 sdot r(119888(x)119896 ) (10)


119908119896 = 2119908119896+1119862sum119896=1

119908119896 = 1forall119908119896 119896 lt 119862


(11)



r = 119903101584011198991015840 g = 119903101584021198991015840 b = 119903101584031198991015840

(12)








(a) Image region

2448

2354

2255

1530

1322

1160

9965

4848

4396

4013

327e + 04

245e + 04

164e + 04

82e + 03

0

5415

616

7155

844

1671

1803

2042

2152


1

0

2448

2354

2255

2152

2042

1803

1671

1530

1322

1160

9965

844

7155

616

5415

4848

4396

4013


162e + 05

102e + 05

43e + 04

minus17e + 04

minus76e + 04

10987654321












Cmd runner

Dist view


Viewport

ViewportViewport

Graph cut


Illumination

Normalization

Clustering

Clustering

Window

Band view

Labeling

Labeling

RGB view

ROI selector


Main

Docks

Image

False color

Task queue

Cmd runner







7 Conclusions





Competing Interests



Acknowledgments


References




































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation
































200

150

50

0

400 430 460 490 520 550 580 610 640 670 700









119888(x) = argmin119895

119889 (xm119895) (7)










c(x) = argminj

119862sum119896=1

119889 (xm119895119896) (9)


r1015840 = 119862sum119896=1

119908119896 sdot r(119888(x)119896 ) (10)


119908119896 = 2119908119896+1119862sum119896=1

119908119896 = 1forall119908119896 119896 lt 119862


(11)



r = 119903101584011198991015840 g = 119903101584021198991015840 b = 119903101584031198991015840

(12)








(a) Image region

2448

2354

2255

1530

1322

1160

9965

4848

4396

4013

327e + 04

245e + 04

164e + 04

82e + 03

0

5415

616

7155

844

1671

1803

2042

2152


1

0

2448

2354

2255

2152

2042

1803

1671

1530

1322

1160

9965

844

7155

616

5415

4848

4396

4013


162e + 05

102e + 05

43e + 04

minus17e + 04

minus76e + 04

10987654321












Cmd runner

Dist view


Viewport

ViewportViewport

Graph cut


Illumination

Normalization

Clustering

Clustering

Window

Band view

Labeling

Labeling

RGB view

ROI selector


Main

Docks

Image

False color

Task queue

Cmd runner







7 Conclusions





Competing Interests



Acknowledgments


References




































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation






















200

150

50

0

400 430 460 490 520 550 580 610 640 670 700









119888(x) = argmin119895

119889 (xm119895) (7)










c(x) = argminj

119862sum119896=1

119889 (xm119895119896) (9)


r1015840 = 119862sum119896=1

119908119896 sdot r(119888(x)119896 ) (10)


119908119896 = 2119908119896+1119862sum119896=1

119908119896 = 1forall119908119896 119896 lt 119862


(11)



r = 119903101584011198991015840 g = 119903101584021198991015840 b = 119903101584031198991015840

(12)








(a) Image region

2448

2354

2255

1530

1322

1160

9965

4848

4396

4013

327e + 04

245e + 04

164e + 04

82e + 03

0

5415

616

7155

844

1671

1803

2042

2152


1

0

2448

2354

2255

2152

2042

1803

1671

1530

1322

1160

9965

844

7155

616

5415

4848

4396

4013


162e + 05

102e + 05

43e + 04

minus17e + 04

minus76e + 04

10987654321












Cmd runner

Dist view


Viewport

ViewportViewport

Graph cut


Illumination

Normalization

Clustering

Clustering

Window

Band view

Labeling

Labeling

RGB view

ROI selector


Main

Docks

Image

False color

Task queue

Cmd runner







7 Conclusions





Competing Interests



Acknowledgments


References




































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation










200

150

50

0

400 430 460 490 520 550 580 610 640 670 700









119888(x) = argmin119895

119889 (xm119895) (7)










c(x) = argminj

119862sum119896=1

119889 (xm119895119896) (9)


r1015840 = 119862sum119896=1

119908119896 sdot r(119888(x)119896 ) (10)


119908119896 = 2119908119896+1119862sum119896=1

119908119896 = 1forall119908119896 119896 lt 119862


(11)



r = 119903101584011198991015840 g = 119903101584021198991015840 b = 119903101584031198991015840

(12)








(a) Image region

2448

2354

2255

1530

1322

1160

9965

4848

4396

4013

327e + 04

245e + 04

164e + 04

82e + 03

0

5415

616

7155

844

1671

1803

2042

2152


1

0

2448

2354

2255

2152

2042

1803

1671

1530

1322

1160

9965

844

7155

616

5415

4848

4396

4013


162e + 05

102e + 05

43e + 04

minus17e + 04

minus76e + 04

10987654321












Cmd runner

Dist view


Viewport

ViewportViewport

Graph cut


Illumination

Normalization

Clustering

Clustering

Window

Band view

Labeling

Labeling

RGB view

ROI selector


Main

Docks

Image

False color

Task queue

Cmd runner







7 Conclusions





Competing Interests



Acknowledgments


References




































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation













c(x) = argminj

119862sum119896=1

119889 (xm119895119896) (9)


r1015840 = 119862sum119896=1

119908119896 sdot r(119888(x)119896 ) (10)


119908119896 = 2119908119896+1119862sum119896=1

119908119896 = 1forall119908119896 119896 lt 119862


(11)



r = 119903101584011198991015840 g = 119903101584021198991015840 b = 119903101584031198991015840

(12)








(a) Image region

2448

2354

2255

1530

1322

1160

9965

4848

4396

4013

327e + 04

245e + 04

164e + 04

82e + 03

0

5415

616

7155

844

1671

1803

2042

2152


1

0

2448

2354

2255

2152

2042

1803

1671

1530

1322

1160

9965

844

7155

616

5415

4848

4396

4013


162e + 05

102e + 05

43e + 04

minus17e + 04

minus76e + 04

10987654321












Cmd runner

Dist view


Viewport

ViewportViewport

Graph cut


Illumination

Normalization

Clustering

Clustering

Window

Band view

Labeling

Labeling

RGB view

ROI selector


Main

Docks

Image

False color

Task queue

Cmd runner







7 Conclusions





Competing Interests



Acknowledgments


References




































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation










(a) Image region

2448

2354

2255

1530

1322

1160

9965

4848

4396

4013

327e + 04

245e + 04

164e + 04

82e + 03

0

5415

616

7155

844

1671

1803

2042

2152


1

0

2448

2354

2255

2152

2042

1803

1671

1530

1322

1160

9965

844

7155

616

5415

4848

4396

4013


162e + 05

102e + 05

43e + 04

minus17e + 04

minus76e + 04

10987654321












Cmd runner

Dist view


Viewport

ViewportViewport

Graph cut


Illumination

Normalization

Clustering

Clustering

Window

Band view

Labeling

Labeling

RGB view

ROI selector


Main

Docks

Image

False color

Task queue

Cmd runner







7 Conclusions





Competing Interests



Acknowledgments


References




































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation










Cmd runner

Dist view


Viewport

ViewportViewport

Graph cut


Illumination

Normalization

Clustering

Clustering

Window

Band view

Labeling

Labeling

RGB view

ROI selector


Main

Docks

Image

False color

Task queue

Cmd runner







7 Conclusions





Competing Interests



Acknowledgments


References




































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation










Acknowledgments


References




































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation












































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation











RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation









Documents

Research Article A Novel Framework for Interactive ...downloads.hindawi.com/journals/jece/2016/2635124.pdfimage analysis that focus on interactive raw data exploration. We combine