Detection of Buried Mines and Explosive Objects Using Dual-band

7/30/2019 Detection of Buried Mines and Explosive Objects Using Dual-band

1/12

Detection of buried mines and explosive objects using dual-band

thermal imageryJason J. Lepley

*and Michael T. Averill

SELEX Galileo, Sigma House, Christopher Martin Road, Basildon, Essex, SS14 3EL, UK

ABSTRACTWe demonstrate the development and use of novel image processing methods to combine dual-band (MWIR and LWIR)images from SELEX GALILEO's Condor II camera to extract characteristics of observed scenes comprising buried

mines and explosive objects. We discuss the development of a statistical processing technique to extract the different

characteristics of the two bands. We further present a statistical classifier used to detect targets on independently trained

images with a high detection probability and low false negative rates and discuss methods to mitigate the impact of falsepositives through the selective processing of image regions and the contextual interpretation of the scene content.

Keywords: Mine detection, thermal imagery, image processing, statistical classifier, IED, dual-band, object detection

1. INTRODUCTIONThere is an acknowledged need to detect buried objects with little or no clearly visible signature on the surface. This has

direct application in addressing the needs of service personnel in current theatres of operation by assisting them in

detecting concealed explosive devices. It may also be applied in humanitarian applications that need a detection

capability in de-mining operations.

Thermal imagery has an established ability to provide contrast of the surface signatures above buried objects such thatthey can be distinguished from surrounding undisturbed earth. This contrast is particularly clear where the surface soil

has been recently disturbed resulting in a change in the soil porosity or moisture content. Single band thermal imagers,

however, provide only a one-dimensional picture of this information that comprises an inseparable mixture of the surfacematerials emissive and thermal properties. In this work we explore the use of a dual-band imager and combine the

information from the two different spectral bands. Each band responds differently to the physical information the scene

provides so, by simultaneously (or near-simultaneously) recording the two band information and processing the data to

contrast the differences, we are able to more readily separate the information relating to an emplaced object from theundisturbed ground.

This work has been enabled by the development, under the UK MOD ALBION programme and subsequently further bySELEX Galileo, of a dual-band thermal imaging camera (Condor). The Condor camera contains a detector able to

record in either the medium (3-5m) or long (8-12m) thermal bands simply by switching the bias voltage on thedetector. By co-locating the pixels of the two bands, the camera is able to record images in each of the two bands

without suffering mis-registration issues. The use of electrical switching between the bands also permits fast (of theorder of millisecond) switching between the bands, which enables the near-simultaneous recording of both bands at the

full 25 fps frame rate of the camera. This greatly reduces the mis-registration that will occur when imaging on the move,

opening the possibility of a mobile implementation of the detection system.

In the early phases of this work (running from 2008 to 2009), the project explored ratiometric techniques to establish the

physical radiometric information (temperature and emissivity) from the scene. The thesis behind which was that the

changes in emissivity and temperature incurred above an emplaced object may provide information on the altered surfacematerial and sub-surface thermal lag resulting from the emplaced device. Whilst some success was achieved with this

approach, the work concluded that the separated radiometric information was too non-linear and insensitive to provide

adequate discrimination of the target material and further that the grey-body assumption resulted in large errors whenapplied to real-world materials. The approach therefore taken was altered to employ statistical processing of the image

information and the results of this study are summarized in this paper.

*[email protected]; phone +44 (0)1268 883470; selexgalileo.com

Detection and Sensing of Mines, Explosive Objects, and Obscured Targets XVI,edited by Russell S. Harmon, John H. Holloway Jr., J. Thomas Broach, Proc. of SPIE

Vol. 8017, 80171V 2011 SPIE CCC code: 0277-786X/11/$18 doi: 10.1117/12.883375

Proc. of SPIE Vol. 8017 80171V-1

Downloaded from SPIE Digital Library on 21 Sep 2011 to 147.96.22.163. Terms of Use: http://spiedl.org/terms


2/12

2. EXPERIMENTAL RESULTS2.1 Trials DataAs a result of the progress made within this project and associated activities, SELEX were invited to participate in trialsin Arizona and fielded the dual band Condor II camera along with two other thermal cameras. Participation in this trial

was invaluable, since it provided alternative environmental conditions. Figure 1 shows a series of MW and LW images

captured by the Condor camera at the indicated times, over a period of 24 hours.

Figure 1: MW images (left set) and LW images (right set) over 24 hours

2.2 Analysis of Data SetThe imagery contained significant periodic noise from the cooling engine in the imager, which was visible on all thetrials data. To prevent this affecting subsequent statistical analysis a temporal noise removal process was employed. Of

the methods investigated, frame averaging was rejected due to it being sensitive to the outlying nature of the periodic

noise; instead a 10-frame median filter was applied to each pixel to provide a robust estimate of the true signal. The

imagery was taken from a tripod-mounted camera and can thus be considered to be spatially registered frame to frame.

It is envisaged that this noise would not be present in a production-standard imager and thus the requirement for sets ofregistered image frames would not in reality arise. Figure 2 shows an example of the noise removed.

Figure 3: Noise removed from a LW image

A number of basic observations were made by viewing trials imagery over the diurnal cycle. During the day the MW

signal is dominated by reflected solar radiation, yet the LW signal clearly shows signatures relating to buried objects.

During the evening and night the LW signal weakens, often becoming doughnut shaped representing a faint haloaround the burial region, yet the MW signal clearly shows the buried objects as cooler regions. It is clear from this that

the optimum image band to detect the disturbed earth varies over the course of the day. A trained end-user might realise

this and select the optimum image band depending on the period of the day. However, a more optimal solution would be

to intelligently fuse the two image bands in a manner that provides the greatest visibility of the targets of interest.Principle Component Analysis (PCA) allows us to select the projection of the MW/LW signal pair that has maximum




3/12

variance and is thus a candidate technique for selecting the band that provides the best contrast on the targets. Initial

experiments with this resulted in a projection that was almost always dominated by the MW signal due to the largevariance that results from the spatial non uniformities in the medium wave imagery that are apparent as vignetting. This

non uniformity is largely a result of temperature variations and re-radiation within the pre-production lens optics and

should not be present in a production standard imager with built in NUC. To prevent this upsetting subsequent analysis,the non uniformity, which is very low frequency, was removed using a wavelet filter. The approximation coefficients at

the seventh level of decomposition were set to zero and the signal then reconstructed. An example is shown in Figure 3,the left hand image is uncorrected, the right hand image corrected. Both are scaled from 5% to 95% of the cumulative

greyscale distribution black to white. Clearly in the right hand signal the remaining variation is the more useful part of

the signal.

Figure 3: Example of wavelet uniformity correction (MW, 2-2-2009, 21:15)

Although spatial non uniformity was predominantly a problem with the MW imagery it is also present to a degree in the

LW. Both sets of imagery were therefore corrected prior to carrying out the PCA calculation.

During the Arizona trials, data was gathered in blocks where each block represents a period of time (usually 24 hours)

over which the cameras had a fixed observation of a static scene comprising several targets. Figure 4 shows the angle ofthe principle component against time of day for imagery from block 2 (x), block 3 (+), block 4 (), block 5 (), and

block 6 (). Where the block covered multiple days, colour is varied as indicated in the legend.

Figure 4: Angle of the principal component

An angle of 0 or represents a principal component lying purely in the MW signal; conversely an angle of /2

represents a principal component lying purely in the LW signal. For data blocks 2 to 5 a clear day/night switching isobserved, with the solar-dominated MW selected predominantly during the day. This is opposite to that required given




4/12

the initial observations, although this is not a problem as the 2nd principle component will be orthogonal and will thus

directly correlate with the observed data. Block 6 (imagery of the adjacent trial line to blocks 2 5) behaves somewhatdifferently; here the MW signal predominates all the time and some fluctuations are noticeable on the final early

morning. The differences in the scenery within block 6 are small; marker posts absent, and a few surface rocks

different. Another potential difference with block 6 is that the weather conditions have altered, with block 6 occurring inthe presence of mostly overcast conditions, thus the surface heating effects of the sun would be somewhat reduced. This

suggests that this PCA technique is sensitive to image content or lighting conditions and would therefore not be a goodselector of the best signal for buried object detection.

To further investigate the properties of the imagery the dual band [MW,LW] signal was normalised using the technique

of sphereing. For each pixel, i, in an image comprising n pixels, we define the signal vector:

i

i

i

LW

MWx [1]

The sample mean and covariance of this are:

i in xx 1 [2]

T

i

i

i

n)()(

1

1xxxxS

[3]

The sphered data can be written as:

)(21

xxZ

i

T

i[4]

with and being the diagonal matrix of eigenvalues and the column matrix of eigenvectors resulting from the spectraldecomposition of the covariance:

TS [5]

This results in a signal with standardised variance in each of the principle axes and allows for easier observation of the

data. Scatter plots of the sphered data have been analysed for manually selected regions of buried object, backgroundsoil, and surface object (rock). Distinct clusters are visible for the different signal types. For consistency in presentation,

the sphered data has been rotated back to the original MW/LW axis. The data points are also range-limited to , to keep

the display sensible. Buried object is red, background soil is green, and surface object (rock) is blue. Figure 6 shows thescatter plot after target emplacement and Figure 5 shows the same image regions before emplacement recorded at the

same time on the previous day. The circular structure visible in blue in Figure 6 is range-limited data, indicating those

points that would otherwise be off the scale. The separation of buried object from background surface is clearly visible.

The surface object is not as clearly separated but appears to be differently distributed. This object only had a small

number of pixels and so it is less clear cut.




5/12

Figure 5: Scatter before target emplacement (block 4). The colours represent background soil (Green and Red) and a surfacerock (Blue)

Figure 6: Scatter after target emplacement (block 5).

The separation of regions is maintained throughout the diurnal cycle. The following examples (Figure 7) are from block6, showing a 24 hour period with a larger surface rock. The separation of buried items and background surface is

maintained. The surface rock has a more variable signature than those of the surface signatures. Within these scatters,

more unusual objects tend to be represented farther from the centre. Materials with different properties tend to cluster

in different directions. We have attempted to capture these properties in a fused colour display that highlights both theunusualness of objects and the separation of material types. Direction, being represented here as a periodic function,

needs care in being mapped to an image display to avoid discontinuity at the boundary. To this end we have used thecircularly continuous colour palette provide by the HSI colour scheme, mapping direction of each pixel within the scatter

axis to the H component (Hue) and the Mahalanobis distance to the I component (intensity), the S component

(saturation) is fixed at 1. The resulting images, Figure 8, clearly partition the scene throughout the 24hr cycle (note that

this is the same image set as shown in Figure 7). For comparison, the four MW/LW pairs are shown in Figure 9.




6/12

Figure 7: Scatter plots over a diurnal cycle (block 6, times indicated in the top left of the plot)

19:00 01:00

07:00 13:00

Figure 8: Fused colour image at 4 times thoughout the day {19:00, 01:00, 07:00, 13:00} (block 6)




7/12

19:00 01:00

07:00 13:00

Figure 9: MW/LW pair, block 6, at four times {19:00, 01:00, 07:00, 13:00)

3. CLASSIFIER DEVELOPMENT3.1 Gaussian Mixture ModelsClusters such as those shown in Figure 6 and Figure 7 are not well modelled by simple Gaussian distributions however

they can be modelled as a weighted mixture of multiple Gaussian distributions, known as a Gaussian Mixture Model

(GMM). The common technique for estimating the parameters of a GMM distribution is the Expectation-Maximisation

algorithm (EM). This is an iterative algorithm that produces a maximum likelihood estimate of the parameters given asample of data. Prior to using the EM algorithm it is necessary to decide how many distributions are included in the

mixture and to make an initial estimate of the parameters. To address this we have implemented a recursive EM

algorithm (RUEM) [1] that simultaneously estimates the parameters of the mixture and the number of components. Thisis followed by the conventional EM algorithm using the full data set using the selected number of components and

parameters as the initial state.

To facilitate a valid experiment using the available data, the two groups have been selected from the Arizona trials data

to be used for training and evaluation respectively. The left hand trial lane is used for training and the right hand trial for

evaluation.

Three applications of GMM to buried object detection have been considered; classification of segmented regions,classification of each image pixel based on its local neighbourhood, and anomaly detection. In the first two of these

cases GMMs are fitted to training data corresponding to target and non target regions and the processed images are

labelled as target/non target. This clearly needs imagery representative of the target and background classes prior tooperation and may need multiple target and non target classes to cope with a wide range of material and environmental

conditions. The third approach considered makes no prior assumptions about the nature of the target or background but

models what is normal by analysis of scene regions. These three methods are discussed in the following three sections.

3.2

Classification of segmented regionsInitial segmentation of target sized objects from the scene is performed using a watershed by reconstructionsegmentation algorithm [4]. The segmentation is used to produce the region boundaries for the truth data described

above.

The watershed by reconstruction segmentation algorithm has been used in a number of previous Selex

detection/recognition studies and has been shown to produce accurate boundaries. In order to control which regions are

segmented we first need to generate appropriate markers. Morphological top-hat by reconstruction operators allow shape

selective extraction of appropriately sized dark or bright regions of the scene. In this case an upper size limit of 100 x 20




8/12

pixels has been set and a lower size limit of 7 x 3 pixels to mitigate detection of surface texture. The approach to

segment the scene is based on the contrast in the bands. With two wavebands of imagery and two modes of regiondetection (dark and bright) there are potentially four sets of markers to be evaluated. As expected, based on the

observations of the dataset two of these four combinations perform considerably better; long wave bright markers during

the day, and medium wave dark markers at night. Figure 10 and Figure 11 show the performance of initial regionsegmentation using these marker functions on training data as a function of time of day. Note that this is an initial

detection process prior to classification; for this purpose a high probability of detection is more important than a lowfalse alarm rate.

Figure 10: Probability of detection for initial segmentation using two different markers

Figure 11: False alarm rate for initial segmentation using two different markers

A clear switch point between the two modes is observable. As a consequence we have selected to use long wave brightmarkers between 9:30 and 17:30 and medium wave dark markers for the remainder of the day. An example of the initialsegmentation is shown in Figure 12.




9/12

Figure 12: Initial region segmentation

A set of 48 (target and non-target, 1 each for each hour of the day) GMMs were trained using the training data truthregions. An upper limit of 120000 was placed on the number of samples used for each GMM so as to keep runtimes

sensible. A maximum likelihood classifier was then used to classify the segmented regions from the evaluation data set.

The performance in terms of correctly identifying target and non-target regions within the segmented region set is shownin the Figure 13 as a function of time of day.

Figure 13: Evaluation using segmented regions (trained on truthed regions)

Although the performance for identifying non-target regions is moderate, the target Pd is poor. A possible explanation

for this is that the segmented target regions extend beyond the declared truth regions thus contaminating the sample. Toinvestigate the effect of this contamination, the experiment was re-run whilst restricting the target samples to the truthed

regions. The results (in Figure 14) show a marked improvement in the correct identification of the target regions.




10/12

Figure 14: Evaluated using truthed subset of segmented regions (trained on truthed regions)

This suggests that either the training or segmentation needs to be adjusted so that the match is better. To address this, theGMMs were retrained using samples generated by the segmentation process, labelled according to whether they

overlapped the declared truth. The results, shown in Figure 15, are better than the original results but can still be

improved upon.

Figure 15: Evaluated using segmented regions (trained on segmented regions)

3.3 Classification of local neighbourhoodsEach pixel within the image was classified using samples from a square 9 x 9 neighbourhood centred on the pixel.

Boundary pixels, where the neighbourhood would be incomplete, were ignored. A maximum likelihood classifier was

used and the GMM were trained using samples from the truth regions as before.

The performance, evaluated on a per pixel basis, is shown in Figure 16 (left) as a function of time of day. These results

are encouraging. The mean probability of detection over the 24 hour cycle is 84% for target pixels and 86% for non-target pixels. Also shown in Figure 16 (right) is an example of a classified image with true positives (correct declaration

of target) in white, true negatives (correct declaration of non-target) in black, false positives (incorrect declaration of

target) in cyan, and false negatives (incorrect declaration of non-target) in red. It is clear from this that all the regions offalse negatives are connected to regions of true positive and as such all targets have been detected. Also a significant

percentage of the false positives are located in the vicinity of the true positive regions. We may have been overly harsh




11/12

in our definition of the truth. If we consider areas in the image rather than individual pixels we see all the targets have

been detected and there are approximately 3 areas of false positive (cyan areas not contiguous with a white area of abovea reasonable size).

Figure 16: Left, performance of local neighbourhood based classification, and Right, example classified image (truepositives in white, true negatives in black, false positives in cyan and false negatives in red)

3.4 Anomaly detectionA limitation of the first two methods, particularly given the variable, nature of buried objects and emplacement, is the

requirement to have training data sets. It is therefore desirable to be able to detect regions within a scene that are

anomalous without a prior knowledge of the target.

Common multispectral anomaly detectors such as RX [2] find pixels that are locally extreme by measuring theMahalanobis distance with respect to the local spectral distribution. This may be sufficient for locating a small piece of

foreign material, but is unlikely to detect buried objects, as the surface materials will be on the whole similar to those in

the surroundings. The most likely change will be a change in the mixture of materials as the soil and rock from

subsurface will have different grain sizes and slightly different makeup due to the lack of weathering.

A mixture model, such as a GMM, is an ideal way of modelling this mixture of surface materials. By looking for local

changes in probability density function we should be able to detect regions of disturbance. The challenge of thisapproach comes in estimating the parameters of the GMM for small local neighbourhoods. The large number of

parameters would require a very large neighbourhood to provide sufficient samples for a reliable estimate. The number

of parameters increases linearly with the number of mixture components and quadratically with the number of bands. Forexample a 2 band model with 3 components requires 18 parameters to be estimated.

A solution to this local estimation problem is the borrowed strength methodology presented [3]. The key assumption in

this method is that the component distributions are present throughout the larger data set and that the local variation is

primarily in the mixing coefficients. A GMM is estimated for the whole data set. When examining local regions of the

data the component parameters (mean and covariance) are borrowed from the global GMM and just the local mixing

coefficients estimated using EM. In the case of the example in the previous paragraph this reduces the number ofparameters estimated locally from 18 to 3. Local anomalies can then be detected by inspecting the integrated square error

between the global density function and the local density function.

To demonstrate the strength of this technique, an example is given using a synthetically generated single band image.Each pixel in the image is selected at random from one of two Gaussian distributions (=-2, = 1) and (= 2, = 9/16).

For the majority of the image the mixing weights are 0.4 and 0.6 however the central 25x25 pixels the weights are 0.2

and 0.8. Figure 17 shows the input image and Figure 26 the anomaly map based on the ISE.

The algorithm has been implemented to process dual band images and to use a region map from a scene segmentation

routine to define the higher level regions within which to search. By restricting the search to particular scene regions theGMM produced will be more appropriate to each region being searched and should result in better sensitivity.




12/12

The current truthed data set, gathered at Arizona, is not ideal for testing this algorithm since there is an unusually high

density of buried objects. In this case it is unlikely that the target regions would be truly anomalous within the context ofthe scene. It is planned to gather more appropriate test data for this algorithm in the trials planned later under this project.

Figure 17: Single band random scene with anomaly (left) and Anomaly map (right)

4. CONCLUSIONSWe present results from experimental trials and subsequent analysis to evaluate the benefits of a Dual-Band thermal IR

camera for the detection of buried objects. We show how, through the use of statistical analysis of the image data, we

are able to detect buried objects with a high probability of detection using a GMM based classification technique trainedusing an independent data set. We further demonstrate the development of a scene anomaly detector for the detection of

potential threats in un-trained data sets.

5. ACKNOWLEDGEMENTSThis work has been funded by the UK MoD through the EMRS DTC.

REFERENCES

[1] Zivkovic, Z, Recursive Unsupervised Learning of finite mixture models, IEEE Trans. on PAMI, Vol. 26(5),(2004).

[2] Hylta, P. C, et al, Anomaly detection in hyperspectral imagery: comparison of methods using diurnal and

seasonal data, J. Applied Remote sensing, Vol. 3, 1-29, (2009).[3] Priebe, C. E., Nonhomogeneity analysis using borrowed strength, J. American Statistical Association, Vol.

91(436), (1996).

[4] Doughert, E. R. (Ed.), Mathematical morphology in image processing, Marcel Decker, Chapter 12 (Beucher

and Meyer), (1993).

P f SPIE V l 8017 80171V 12

Documents

Detection of Buried Mines and Explosive Objects Using Dual-band