Research ArticleOptimization of the Oil Drilling Monitoring System Based on theMultisensor Image Fusion Algorithm

Dong Wang,1,2 Yongming Li ,1 Haojie Yu,2 Ruizhi Tang,3 and Ningping Yan4

1Petroleum Engineering School, Southwest Petroleum University, Sichuan, Chengdu 610500, China2Petrochina Changqing Oilfield Company, Shaanxi, Xi’an 710018, China3South Sulige Operating Company of Changqing Oilfield Company, PetroChina, Shaanxi, Xi’an 710018, China4The Second Gas Production Plant of PetroChina Changqing Oilfield Company, Shaanxi, Xi’an 710018, China

Correspondence should be addressed to Yongming Li; 201711000114@stu.swpu.edu.cn

Received 6 July 2021; Revised 2 August 2021; Accepted 11 August 2021; Published 6 September 2021

Academic Editor: Haibin Lv

Copyright © 2021 Dong Wang et al. This is an open access article distributed under the Creative Commons Attribution License,which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

The working environment of the oil drilling platform is harsh, with many uncertain factors and high operating risks. During thedrilling process, due to sudden formation factors or improper process operations, it is extremely easy to cause well wall instability,sticking, lost circulation, well kick, and blowout. In addition, other complicated situations and accidents have brought majorchallenges to drilling safety. In order to improve the technical level of oil and gas exploration and development and achieve thegoal of reducing costs and increasing efficiency, it is necessary to strengthen the optimization of traditional oil drillingmonitoring systems. This article summarizes the advantages and disadvantages of the existing image multiscale analysisalgorithms, from wavelet transform, stationary wavelet transforms to contourlet transform, and nondownsampled contoursbased on the characteristics of the images collected by different sensors in the oil drilling monitoring system and the needs ofpractical applications. Wave transforms detailed comparison of the fusion performance of these image analysis algorithmsunder the same fusion rules. Aiming at the shortcoming of the large amount of calculation of nonsubsampled contourlettransform, a fast implementation algorithm (IFNSCT) is proposed. The multichannel filter bank structure is used to replacethe original tree filter bank structure, which reduces the time-consuming to the original without affecting the analysisperformance of the algorithm. One-half of the oil drilling monitoring efficiency has been improved.

1. Introduction

With the year-on-year increase in the amount of drilling engi-neering operations, the increasing renewal of technology, andthe development of the information industry, offshore oil hasentered the era of network information [1, 2]. According to thecharacteristics of offshore drilling technology, through specialresearch, test analysis, evaluation optimization, and engineer-ing practice, a complete set of the intelligent auxiliarydecision-making monitoring system for offshore oil drillinghas been formed [3, 4]. It ensures operation safety, improvesoperation timeliness, saves drilling costs, and provides solidtechnical support for the exploration and development of off-shore oil and gas resources.

The most important part of the optimization of the oildrilling monitoring system is the information fusion of the

acquired images of monitoring equipment. Informationfusion technology is such a processing process, and it canmake full use of advanced computer technology to compre-hensively analyze and process the information obtainedthrough different (multisource) sensors and then obtain acomprehensive and accurate information about a certaintarget or scene. Image fusion belongs to the category ofinformation fusion [5–7]. As an important part of it, thistechnology integrates many hot subjects, such as imageand signal processing, computer technology, and sensortechnology and has huge application and development pros-pects. The so-called image fusion is not simply superimpos-ing images of different natures after obtaining them throughmultisource sensors. However, it is committed to adoptingappropriate fusion algorithms, but to make full use of theredundancy of these images through appropriate fusion

HindawiJournal of SensorsVolume 2021, Article ID 5229073, 12 pageshttps://doi.org/10.1155/2021/5229073

algorithms [8, 9]. Based on the remaining and complemen-tary information, they are fused, so that the final generatedimage has richer details and fuller content [10]. The fusionimage has many advantages that cannot be compared withthe image obtained by a single sensor, which greatly over-comes the latter are limitations in some aspects such as spec-trum, spatial geometry, and resolution.

According to the different images processed, imagefusion methods can be divided into two categories: one isthe most widely studied grayscale image fusion; the otheris the increasingly emerging color image fusion method.Commonly used fusion algorithms are all trying to makethe fusion image with higher quality, which belongs to thecategory of pixel-level fusion [11, 12]. Literature [13] carriedout the simplest image fusion experiment; he carried out asimple fusion processing of Landsat image and radar imageand successfully applied the fused image in the interpreta-tion of terrain and landform. Later in the literature [14],the MSS image and Landsat-RBV image were fused, and pre-liminary results were obtained. Image fusion technology hasgradually attracted attention and has begun to be applied togeneral images such as visible light images. Image fusiontechnology has begun to become one of the researchhotspots in the field of remote sensing image processingand analysis. In the first type of the fusion method, pixel-based or region-based fusion is more widely used andsimpler. This type of method runs fast, but often has unde-sirable effects such as washout. Principal component analysis(PCA) is a commonly used algorithm for matrix dimensionreduction and data relevance removal. It is similar to the KLtransformation in the image compression fields [15–17]. Alinear transformation processes the grayscale image into atwo-dimensional matrix, solves the eigenvalues and corre-sponding eigenvectors of the matrix, and finally fuses theextracted principal components to obtain the fusion result[18]. The above algorithms are all performed on a singlescale when processing the image, and there is no differencein the processing of pixels, which will cause the loss of imagedetail information. The idea of multiresolution analysissolves the abovementioned problems to a certain extent.Literature [19] first proposed the tower decompositionmethod, namely, the Gaussian pyramid and the Laplace pyr-amid. On this basis, Fedele and Merenda [20] proposed alow pass pyramid layered fusion method based on the con-trast pyramid. Feng et al. proposed the gradient pyramidalgorithm [21], which makes the poor directionality of thepyramid structure get to improve. In general, in the fusionmethod based on the tower decomposition, the Laplace pyr-amid does not express the directional information, the con-trast and gradient pyramid algorithms will increase theamount of data in the processing process, and their stabilityneeds to be enhanced. Wavelet transform technology wasproposed in the 1990s. This method has many excellentproperties such as variable time and frequency domain, gooddirectionality, and multiscale and has been widely studiedand applied [22–25]. The directionality of wavelet transformis not flexible enough, and it is not very ideal in expressingthe curve. In response to this defect, scholars have proposedanalysis tools such as ridge let and Curvelet transform. The

literature [26] proposed the Curvelet transform. To obtainthe decomposed subband, the source image needs to befiltered many times. Due to the multiple convolution of theimage, it will inevitably affect the computational efficiencyof the digital image.

In this paper, we optimize the monitoring system of oildrilling based on multisensor image fusion. First, analyze thestructure of the oil drilling monitoring system and design areasonable structure of the multisensor image fusion algo-rithm. Secondly, the relevant theories of themultisensor imagefusion algorithm are analyzed and researched. Aiming at theslower image information fusion of the traditional nonsub-sampled contourlet transforms multiscale transform method,an improved fast nonsubsampled contourlet transform algo-rithm is proposed. Finally, simulation experiments verify theeffectiveness of the algorithm and improve the monitoringefficiency of oil drilling. The second part of the article is anintroduction to the overall framework of the article and relatedtheories; the third part is the algorithm structure and specificimplementation; the fourth part is the simulation experimentverification; the fifth part is the full text summary. The maincontributions are follows: (1) simultaneous interpreting thecharacteristics and practical applications of different sensorscollected from the oil drilling monitoring system, a nonsam-pling contour is proposed. (2) A fast implementation algo-rithm (IFNSCT) is proposed to solve the problem of largeamount of computation of nosubsampled contour wave trans-form. (3) The multi-channel filter bank structure is used toreplace the original tree filter bank structure, which reducesthe time-consuming of the original filter bank structure with-out affecting the analysis performance of the algorithm.

2. Related Theories and Technologies

2.1. Oil Drilling Monitoring System Design. The overall struc-ture of the offshore oil drilling intelligent auxiliary decision-making monitoring system can be divided into 4 parts, asshown in Figure 1, which are, respectively, the preview layer,the monitoring layer, the decision-making layer, and theoptimization layer. The preview layer is the simulation eval-uation layer of the drilling design.

The drilling design is simulated before drilling, the keyparameter changes are analyzed, and the possible complica-tions are predicted. The monitoring layer is the tracking eval-uation layer of the land center, which simultaneouslyanalyzes the drilling data transmitted on site, intelligentlyevaluates the drilling conditions, and provides real-timeguidance for the site. The decision-making layer is the planformulation layer of the head office. It integrates drillingand geological conditions and provides an auxiliary role fortechnical experts in decision-making in a three-dimensionaldynamic visualization method. The optimized layer is themonitoring image analysis layer after drilling, which per-forms overall monitoring and analysis of the drilled singlewell or regional multiwells, and provides reference materialsfor subsequent construction or adjacent well design.

Each area of the drilling well site is equipped with avideo security monitoring system to monitor its operationstatus in real time to ensure that the system runs more

2 Journal of Sensors

continuously and reliably. Adopt professional methods toconduct real-time intelligent analysis of the video data ofthe well site, such safety risks are avoided and prevented,and the zero-accident construction behavior is counted tofacilitate the orderly progress of safe construction workand make this work more standardized. The multisensorimage self-adaptive fusion frame structure proposed in thispaper is used for the optimization of the drilling monitoringsystem to facilitate the intelligent analysis of monitoringvideo information. The framework consists of a multiscaleanalysis module, a coefficient fusion module, a multiscalereconstruction and image evaluation module, and a parame-ter optimization module, as shown in Figure 2.

Image fusion is a preprocessing operation for subsequenttasks such as detection, recognition, segmentation, and clas-sification. Different subsequent tasks often require observa-tion or processing of different features in the same image.Therefore, unlike most fusion frameworks based on multi-scale analysis, the image fusion framework proposed in thischapter introduces the evaluation of the fusion image qualityinto the fusion process, and the result of the image qualityevaluation is used as feedback information to optimize theparameters in the coefficient fusion module, thereby getbetter fusion results adaptively. As shown in Figure 2, first,the input source image is subjected to multiscale transforma-tion to obtain high-frequency coefficients and low-frequencycoefficients. Different fusion rules are used to fuse the corre-sponding coefficients to obtain fusion coefficients, and thefusion coefficients are multiscale reconstruction to obtainthe fused image. Then, the quality of the fusion image isevaluated, and the evaluation result is fed back to the optimi-zation algorithm to optimize the parameters in the coeffi-cient fusion module, so that the final fusion image will be abetter fusion result for the selected evaluation index.

2.2. Analysis of Classical Multiscale Image Technology.Multi-scale analysis is a fast and effective image signal processingalgorithm. By decomposing the image at different scales, itcan effectively extract the characteristic information of theimage signal at each scale. In view of the different nature ofthe statistical characteristics of the information at differentscales, a targeted image fusion rule is designed, which caneffectively retain the important information of the sourceimage and improve the overall quality and usability of thefused image. The current academic circles have proposedmore than ten different multiscale image analysis algorithms,among which the classic multiscale analysis algorithmsmainly include wavelet transform and contourlet transform.

Wavelet transform (WT) is one of the first multiscaleimage analysis algorithms introduced into the field of imagefusion research. It has good time-frequency analysis capabil-ities [27–29]. Because the image under the computer plat-form is stored in the form of a pixel matrix, the wavelettransform used for image processing and fusion is mainlytwo-dimensional discrete wavelet transform.

In discrete wavelet transform (DWT), the family of func-tions ζa,bðtÞ can be expressed in the following form:

ζa,b tð Þ = 1ffiffiffiffiffiffiffi∣a ∣

p ζt − bt + a

� �, ð1Þ

where a and b are scale and translation coefficients, a, b ∈ R,a is a positive value, a ≠ 0, and the function ζ satisfies

cζ =ð+∞0

ζ ωð Þj j2∣ω∣+1 dω <∞, ð2Þ

�e internet

Partial relation coefficientOffshore oil drilling

Rehearsal evaluation Smart evaluation Remote decision making Analysis and optimization

Design preview

Remote decision making

Multi-drilling data

Real-time monitoring

Figure 1: Overall structure diagram of the oil drilling monitoring system.

3Journal of Sensors

where w is the angular frequency. Discretize a and b withpower exponents: a = aj0 and b = kaj0b0. Among them, j ∈ R,the expansion step a0 is fixed values, and a0 ≠ 1.The DWTcoefficient is defined as follows:

cj,k =ð+∞−∞

f tð Þζj,k tð Þdt = <f , ζj,k tð Þ > : ð3Þ

In image fusion, permanent a0 = 2, b0 = 1, the waveletbasis function is simplified to

ζj,k tð Þ = 2−j/2ζ t − 2jk2j + 1

� �: ð4Þ

After the source image is wavelet transformed, each levelof wavelet decomposition will get four subband images,including LL (low frequency sub band), LH (horizontal subband), HL (vertical sub band), and HH (diagonal sub band).The length and width of each subband image are 1/2 of thesource image, and the data volume is 1/4 of the sourceimage. The second-level wavelet decomposition is to decom-pose the LL subband iteratively and so on.

In the one-dimensional signal representation, DWT isundoubtedly an effective algorithm, which to some extentprovides the best representation of the one-dimensionalsignal. However, because the image is not directly stackedby one-dimensional signals, the poor direction extractionability of DWT is only suitable for capturing point singular-ities, and it is not enough to encounter two-dimensionalimages containing line singularities and surface singularities.The multiscale decomposition is completed by Laplace filterbank, and a directional filter that satisfies the tree expansionrule achieves the multidirectional analysis. The device group

is complete. The structure of the filter bank is shown inFigure 3. The yellow part is the range of the image frequencyband filtered by the Laplace filter bank.

Laplace decomposition can generate a low-resolutionimage and a difference image (that is, the differencebetween the upper level decomposition image and the cur-rent level decomposition prediction image) at each level,and then iteratively decompose the low-resolution image,and finally obtain a one lowest level low-resolution imageand several differential images at various levels. The recon-struction process is a process in which low pass and low-resolution images are predicted to the upper level andsuperimposed on the upper level difference image, andfinally, the original image is obtained. Figure 3(b) describesthe Laplace decomposition and reconstruction process. Hand G are Laplacian decomposition filter and Laplacianreconstruction filter, respectively.

After improving the traditional Laplacian filter, the direc-tional filter structure in the contourlet transform (CT) isshown in Figure 3(c). Among them, H0 and G0 are sector fil-ters, H1 and G1 are quadrant filters, and Q is plum blossom.DFB can generate 2k wedge-shaped frequency segmentationthrough a binary tree decomposition of k layers. In order toobtain the desired frequency segmentation, it is necessary touse a plum blossom sampling filter bank that satisfies the treeexpansion law for image and filter adjustment.

2.3. Evaluation Index of Image Fusion Quality. Due to thecomplexity of the image itself, it is a relatively difficult taskto evaluate the effectiveness of the fusion algorithm and thequality of the fusion image. Therefore, the evaluation of fusionalgorithms and fusion quality requires a number of indicatorswith different focuses. At present, the evaluation indicators ofimage fusion are mainly divided into two categories: subjective

Visible light image

Infrared image

Fusion image

Multi-scale analysis

Coefficient fusion

00.2Refactoring and

parameter optimization0.4 0.6

Recall

Prec

ision

0.8 1.0

0.2

0.4

0.6

0.8

1.0 A

C

B

Figure 2: Multisensor image adaptive fusion framework structure.

4 Journal of Sensors

evaluation indicators and objective evaluation indicators, andgenerally both types of indicators are used to comprehensivelyevaluate image fusion algorithms. The structure diagram ofthe evaluation index is shown in Figure 4.

For a visual image, the most intuitive way to evaluate thefusion effect is human observation. Human beings have devel-oped a complete set of image evaluation standards related tolife experience in daily life and study. For an image, humanscan easily qualitatively judge the clarity, intelligibility, rational-ity, and information content of the image and other intuitiveevaluations. However, subjective evaluation also has inevitableerrors, and its evaluation conclusions are often different. Com-pared with the single and imprecise subjective evaluation stan-dard, the objective evaluation standard provides a quantitativeevaluation of a certain attribute of the fusion result. The exist-ing objective evaluation indicators can be roughly divided intotwo categories, based on evaluation methods for the statisticalcharacteristics of a single image: information entropy, averagegradient, spatial frequency, image mean, standard deviation,etc. In addition, evaluation methods based on the amount ofinformation transfer between multiple images: mutual infor-mation (MI), Q, QE, QAB/F, structural similarity (SSIM),visual information (VIF), etc. Because the evaluation methodbased on the information transfer of the source image com-bines the information of the source image and the fusionimage, it has better reliability. The mathematical definitionsof several evaluation indicators used in this article are givenbelow:

(1) MI: this parameter reflects the amount of informa-tion transferred from the original image to the fusionimage and is defined as shown in formula (5). The

y0 y1

y3

y2

y0y1

y3

y2

w1

w2

2H = 1 −10 1

2A

B

A

B

(b) Laplace transform and inverse transform filter

(a) Contourlet transform filter organization structure

Q

Q

(c) Two-dimensional directional filter bank

Evolution

l = 2, DFB frequency division

LF

IF

HFOil drilling

H1 = 1 −10 1

H0 = 1 −10 1

G1 = 1 −10 1

G0 = 1 −10 1

G = 1 10 1

G = 1 10 1

Figure 3: Multisensor image adaptive fusion framework structure.

larger the value, the more information the fusionimage inherits from the original image and the betterthe fusion quality

MIABF = IFA f , að Þ + IFB f , bð Þ =〠f ,apFA f , að Þ log pFA f , að Þ

pF fð ÞpA að Þ

+〠f ,bpFB f , bð Þ log pFA f , bð Þ

pF fð ÞpB bð Þ ,

ð5Þ

where IFAð f , aÞ and IFBð f , bÞ mean the intensitylevel in image A and image B. pFAð f , aÞ, pFAð f , bÞ,pFð f Þ, pAðaÞ, and pBðbÞ mean probability massfunction.

(2) VIF: by modeling the human visual system, naturalscenes, and image distortion models, comprehensivequantitative evaluations including additive noise,blur, and global or local contrast distortion can becarried out. The larger the value, the better. 1 meansthere is no distortion, and its definition is shown informula (6) [30]:

VIF = 〠j=subbands

I C!N ,j

; F!N ,j����sN ,j

� �〠

j=subbandsI C

!N ,j; E!N ,j����sN ,j

� �

= 12〠

N

i=1〠M

k=1log2 1 + g2i s

2i λk

σ2v + σ2n

� �

ð6Þ

5Journal of Sensors

(3) SSIM: the quality of the fusion image is evaluated bythe structural similarity evaluation of the sourceimage and the fusion image. The larger the value 1means no distortion, and its definition is given by for-mula (7) [30]:

SSIM x, yð Þ =2μxμy − C1

� �2σxy − C2�

μ2x + μ2y − C1� �

σ2xσ2y − C2

� � : ð7Þ

3. Research on Multisensor Image FusionAlgorithm Based on Improved Fast NSCT

Two kinds of monitoring equipment that often appear inpetroleum mines are ordinary visible light monitoring andinfrared monitoring, and in the application field of imagefusion, infrared and visible light fusion is an importantapplication direction. Due to the different characteristicsand points of interest between infrared images and visiblelight images, these two images contain a large amount ofcomplementary useful information, such as clear textureinformation in visible light images and hidden targets ininfrared images. The fusion of these two images, and thenthe effective use of the complementary information, is ofgreat significance to facilitate human observation and latercomputer image processing.

3.1. Nonsubsampled Contourlet Transform (NSCT). In theimage multiscale and multidirectional analysis algorithm,NSCT is widely used because of its resolution ability in anydirection and any scale, translation invariance provided bynondown sampling, and excellent performance withoutspectrum aliasing and Gibb’s phenomenon [22–25]. Atten-tion. It holds the segment-based structure provides a

progressive optimal, sparse image approximate way, betterthan the vast majority of image analysis algorithms ondecomposition and reconstruction performance.

NSCT is a nonsubsampling version developed based onCT. It can be divided into two parts: nonsubsampled pyra-mid (NSP) decomposition and nonsubsampled directionalfilter bank (NSDFB) decomposition. The former guaranteesthe multiscale characteristics of NSCT. The latter providesNSCT with powerful multidirectional decomposition perfor-mance. Figure 5 shows the breakdown structure of NSCT.

The nonsubsampled pyramid (NSP) transform isformed by cascading a two-channel filter bank. At eachdecomposition level, one high-frequency subband imageand one low-frequency subband image can be obtained,and then each the hierarchical low-frequency sub-bandimage is iteratively filtered to complete the multilayer NSPdecomposition. In K-level NSP filtering, a low-frequencysubband image and K high-frequency sub-band images aregenerated, and the size of all subband images is the sameas the source image. Figure 5 shows a schematic diagramof three-layer NSP decomposition, where H0ðz2

kIÞ repre-sents the low pass filter bank, H1ðz2

kIÞ represents the highpass filter bank, k represents the decomposition level, andthe grey area represents the filter passband of each NSPdecomposition step.

The nondownsampling direction filter bank transform isa tree-shaped nondownsampling filter bank composed of aseries of fan-shaped filter banks and quadrant filter banksaccording to the tree expansion principle, which can providerich directional detailed information. In the 1-level NSDFBdecomposition, 21 directional subbands of the same size asthe source image can be obtained. In Figure 5, a 2-levelNSDFB structure and its frequency splitting diagram aregiven, where U0ðzÞ and U1ðzÞ are sector filters, and U0ðzQÞand U1ðzQÞ are quadrant filters.

Distortion evaluation

Image fusion quality

Multi-image fusion evaluation index

Subjective indicators

Objective indicators

Information entropy

Average gradient

Spatial frequency

Image mean

Based on a single image

Mutualinformation

Structural similarity

Visualinformation Q information

Based on multiple images

Amount of information

Figure 4: The structure diagram of the evaluation index.

6 Journal of Sensors

3.2. Fast Implementation of NSCT. Although NSCT hasexcellent multiscale and multidirectional analysis capabili-ties, its application range is limited by its huge computa-tional overhead. As described in the previous section, thetraditional NSCT is divided into two parts. The sourceimage to be decomposed needs to be decomposed by NSPand NSDFB in turn. Therefore, to obtain a decomposedsub band, the source image needs to be filtered multipletimes. Due to the large amount of digital image data, calcu-lating image convolution multiple times will inevitablyaffect the efficiency of the algorithm. On the contrary, ifthey obtained decomposed subbands that can be com-pressed in the primary filtering, the algorithm efficiency willbe greatly improved. Based on this idea, we redesigned thefilter structure, and its implementation on a single channelis shown in Figure 6.

As shown in Figure 6, H0ðzÞ and H1ðz2Þ represent lowpass and high pass filters in NSP, and U0ðzÞ and U0ðzQÞ rep-resent sector and quadrant filters in NSDFB. To obtain afinal directional decomposition subband, the source imageneeds to be filtered by these four filters in sequence, sincethe size of the digital filter is much smaller than the size ofthe digital image. The amount of calculation to combinethe filters is negligible compared to the image convolution.Therefore, the filter banks of the same channel in NSCT(H0ðzÞ, H1ðz2Þ, U0ðzÞ, and U0ðzQÞ) are combined into a sin-gle filter in fast NSCT FðzÞ. Furthermore, the filter signalpasses through a two-dimensional convolution network;finally, the signal sends to the NSCT algorithm. In thisway, the fast implementation of NSCT is realized.

3.3. Multisensor Image Fusion Rule Design. Pixel-level imagefusion is to directly integrate each pixel in the source imageinto a fusion pixel, and the calculation of the fusion weight is

based on the importance of the original pixel in the informa-tion. Therefore, regardless of the fusion rule, it is a process ofweighted summation of the original pixels, and these weightsreflect the importance of the original pixels. The most essen-tial requirement of image fusion is to retain as much impor-tant information in the source image as possible. Therefore,pixels with more information should be assigned with higherweights to retain more information, while pixels with lowinformation should be assigned higher weights to retainmore information in contrast. Based on this, a new fusionrule based on information theory pixel information estima-tion (PIE) is proposed to measure the information containedin the pixel and to determine the pixel weight.

In NSCT, the low-frequency subband image is severelyblurred in the NSP decomposition, which means that asmall neighborhood contains no more information than asingle pixel, and too much time is spent calculating theneighborhood information of the pixel. It is not necessary.In addition, the current popular regional energy fusion ruleignores the dark information, and the regional variance ruleonly strengthens the pixels in the edge area. Therefore, inorder to solve the above problems, the low frequency partof the PIE method is proposed. It only calculates the back-ground brightness of the overall image once instead of cal-culating the neighborhood information for each pixelsuccessively and then determines the fusion weight basedon the difference between the pixel grayscale and the over-all background brightness.

In NSCT, the high-frequency subband reflects the edgeand texture information distribution of the image, and itscoefficient is small in the smooth area, but increases sharplyin the edge part. Different coefficient values represent differ-ent object characteristics, and different rules are needed tofuse them. Suppose that cvisðm, nÞ and cinf ðm, nÞ are the

Two-level NSDFB structure

Low frequency subband

Lower low frequency subband

Lowest low frequency subband

Original image

…NSDFB

NSP�ree-level NSP structure

Figure 5: NSCT decomposition and its partial details.

7Journal of Sensors

visible light image and infrared image in m rows and n col-umns, respectively, for high-frequency coefficient values, ifone of them belongs to the texture or edge part (that is,

has a larger coefficient value), the value with the larger abso-lute value is selected as the fusion coefficient value to ensurethat important information is preserved. If both belong to

Frequency domain range of filtered image0

0

Primary filter

Secondary filtering

�ree-stage filtering

Four-stage filtering

Two-dimensional convolution

Fast filter with IFNSCT

k

100 200 300 400 500 600

100

200

300

Figure 6: The fast implementation structure of NSCT.

(a) Visible light picture (b) Infrared image (c) CT fusion image

(d) NSCT fusion image (e) NSCT-PCNN fusion image (f) IFNSCT fusion image

Figure 7: Subjective evaluation of multisensor image fusion.

8 Journal of Sensors

the smooth part (that is, both have smaller coefficientvalues), then they are weighted and averaged to ensure thatas much source image information as possible is inherited.Therefore, it is necessary to calculate a threshold to distin-

guish the smooth region coefficient from the texture edgecoefficient and apply different fusion rules.

4. Simulation Results andPerformance Analysis

In this part, three different sets of infrared and visible lightdata are used to test the fusion performance of the proposedalgorithm. For each set of data, the algorithm in this chapterwill be compared with other existing algorithms, includingCT algorithm, NSCT algorithm, and NSCT-PCNN algo-rithm. These algorithms have outstanding performances in

0 5 101.6

1.8

2

2.2

2.4

2.6

2.8

3

3.2

3.4

3.6

Figure

Val

ue

CTNSCT

NSCT-PCNNIFNSCT

(a) Evaluation index MI

0 5 10

0.4

0.5

0.6

0.7

0.8

0.9

1

Figure

Val

ue

(b) Evaluation index VIF

0 5 100.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Figure

Val

ue

(c) Evaluation index SSIM

Figure 8: “UN Camp image” collection objective evaluation.

Table 1: EFQI and WFQI of infrared and visible image fusionresults.

Mallat CT NSCT NSCT-PCNN FNSCT

EFQI 0.766 0.723 0.792 0.793 0.811

WFQI 0.768 0.782 0.795 0.791 0.813

9Journal of Sensors

the field of nonlinear fusion, image decomposition and rep-resentation, and field information weighting, respectively[31–34]. In the experiment, the image decomposition levelis set to level three, and all parameter settings refer to thesettings in the reference. In addition, the algorithm runningenvironment is Core i7 2630, 4G RAM. The adopted “UNCamp” data set images cover 256 grey levels, all of which areinfrared and visible light images, and are used as test imagesin a large number of documents. For more precise fusionresults, the watermark in the Octet image collection is cut.

4.1. Simulation and Results of Subjective Evaluation of Multi-Sensor Image Fusion Effect. The comparison of the fusionresults of the “UN-Camp” image set is shown in Figure 7.In these images, Figures 7(a) and 7(b) represent the originalvisible light image and infrared image, respectively, andFigures 7(c)–7(f) represent the fusion results of six fusionalgorithms, including the IFNSCT algorithm proposed inthis chapter.

Obviously, the visible light image depicts the backgroundinformation of the environment well, while the infraredimage highlights the target information of the charactershidden in the bushes. It can be seen that all the fusion algo-rithms retain the main information of the original image inthe fusion image. However, there are still significant differ-ences in details. The contrast of Figure 7(c) is the worst,because the low-frequency average fusion rule compressesthe grey-scale range of the image. The overall grey scale ofFigure 7(d) is too bright, so that more dark part informationis lost. Figure 7(e) has a good visual effect and sense of hier-archy, but there is serious blur in some areas. This is becausethe PCNN algorithm enhances visual contrast but ignorestexture details. In contrast, Figure 7(f) has the best perfor-

mance in terms of contrast, image sharpness, target defini-tion, and edge details, which reflects that the fusion rule isapplicable to infrared and visible light fusion.

4.2. Simulation and Results of Objective Evaluation ofMultisensor Image Fusion Effect. Subjective evaluation indi-cators can often provide humans with an intuitive compari-son, but due to differences between individuals, determiningthe best results may be difficult and even controversial.Therefore, an effective objective evaluation index can giveus a quantitative analysis of the fusion quality. In this part,three objective evaluation indicators including MI, VIF,and SSIM are used to objectively evaluate the quality ofthe above fusion results.

Based on the statistical data in Figure 8 and the aboveanalysis, it is obvious that the IFNSCT algorithm proposedin this chapter is very effective in retaining useful informa-tion, reducing image distortion and maintaining reasonableimage contrast, and it is better than existing subjective andobjective evaluations. The image fusion algorithm is better.The detailed results are shown in Table 1.

Table 1 is the performance comparison data of theresults of the five fusion methods. It can be seen from theexperimental data in Table 1 that compared with the imagefusion algorithm based on wavelet transform and NSCTtransform, the fusion algorithm based on fast NSCT achieveshigher EFQI and WFQI. In particular, the fusion algorithmproposed in this paper has the highest EFQI and WFQI,which shows that the algorithm proposed in this paper canbetter extract the edge information of the image and is morein line with human visual characteristics. Higher perfor-mance fusion images are obtained.

(a) Visible light picture (b) Infarred image

50

50100150x

y

200250

Der

ivat

ive

50100

150

100150200250

(c) IFNSCT Feature image with a

50

50100150x

y

200250

Der

ivat

ive

50100

150

100

150

0

100150x

200250

100150

(d) IFNSCT Feature image with b

Figure 9: Oil drilling test results based on the IFNSCT multiimage fusion algorithm.

10 Journal of Sensors

4.3. Feature Extraction Experiment and Result of Oil DrillingMonitoring Video Image. Based on the above analysis andsimulation experiments, the effectiveness and superiority ofthe proposed IFNSCT multisensor image fusion algorithmare verified. The proposed algorithm is applied to the oildrilling monitoring system to optimize the image processingcapability of the monitoring system. The video securitymonitoring system of the drilling well site is mainly usedto supplement manual duty. If the monitoring result is thesame as the algorithm setting rules, it will automaticallyprompt the monitoring system and give specific processingmethods to achieve linkage alarm and manual intervention.Add a video intelligent analysis module to the intermediatemedia processing layer platform and make it the core ofthe system. At the same time, focus on analyzing illegalintrusions, high-altitude operations, hot work, smokingbehavior, etc. and implement key monitoring on them toachieve active early warning. The algorithm in this paperprovides a theoretical basis for these functions, and the testresults are shown in Figure 9.

Obviously, it can be seen from Figure 9 that the pro-posed IFNSCT algorithm can process the frame images inthe surveillance video very well. For different types ofmonitoring equipment, visible light monitoring equipment,and infrared monitoring equipment, although the extractedgradient features will be different, the overall trend is almostthe same, and the fusion of their features can better reflectthe actual situation of the oil drilling site. Furthermore,Figures 9(c) and 9(d) mainly show the image features ofthe original image and infrared image processed based onthe IFNSCT algorithm. It can be seen from the figure thatalthough the infrared image accelerates the processing speedof the algorithm to a certain extent, the accuracy is obviouslya little lower than that of the original image.

5. Conclusion

The research on the oil drilling intelligent auxiliary decision-making monitoring system has effectively improved theintelligent monitoring level of offshore drilling, greatlyreduced the difficulty of drilling in complex formations,reduced the complexity, effectively avoided the occurrenceof engineering accidents, and made a contribution to therealization of safe and efficient drilling operations. Startingfrom the multisensor image fusion algorithm, this paper firststudies the composition of the monitoring system of oil dril-ling, then analyzes the basic framework of the multisensorfusion algorithm, and studies the fusion of multi-imageinformation based on the multiscale analysis algorithm ofthe improved fast nonsubsampled contourlet transform.The feature information of the whole image is directly calcu-lated, and the fusion weight is directly calculated through thedifference between the pixel and the overall information ofthe image, which greatly improves the efficiency of the algo-rithm on the basis of improving the fusion performance.Finally, the image information of the visible light monitoringequipment and the infrared monitoring equipment in the oildrilling monitoring system is fused to improve the recogni-tion accuracy of the monitoring system.

Data Availability

The data used to support the findings of this study are avail-able from the corresponding author upon request.

Conflicts of Interest

The authors declare that they have no known competingfinancial interests or personal relationships that could haveappeared to influence the work reported in this paper.

Acknowledgments

This work was supported by the Southwest PetroleumUniversity.

References

[1] Y. Pan, R. Hong, J. Chen, J. Singh, and X. Jia, “Performancedegradation assessment of a wind turbine gearbox based onmulti- sensor data fusion,” Mechanism and Machine Theory,vol. 137, pp. 509–526, 2019.

[2] S. Budhiraja, R. Sharma, S. Agrawal, and B. S. Sohi, “Infraredand visible image fusion using modified spatial frequency-based clustered dictionary,” Pattern Analysis and Applications,vol. 24, no. 2, pp. 575–589, 2021.

[3] A. Dameshghi andM. H. Refan, “Wind turbine gearbox condi-tion monitoring and fault diagnosis based on multi-sensorinformation fusion of SCADA and DSER-PSO-WRVMmethod,” International Journal of Modelling and Simulation,vol. 39, no. 1, pp. 48–72, 2019.

[4] S. Liu, D. Zheng, and R. Li, “Compensation method for pipe-line centerline measurement of in-line inspection duringodometer slips based on multi-sensor fusion and LSTM net-work,” Sensors, vol. 19, no. 17, p. 3740, 2019.

[5] L. Zou, Z. Wang, J. Hu, and Q. L. Han, “Moving horizon esti-mation meets multi-sensor information fusion: development,opportunities and challenges,” Information Fusion, vol. 60,pp. 1–10, 2020.

[6] H. Kaur, D. Koundal, and V. Kadyan, “Image fusion tech-niques: a survey,” Archives of Computational Methods in Engi-neering, 2021.

[7] L. Guan, X. Cong, Q. Zhang et al., “A comprehensivereview of micro-inertial measurement unit based intelligentPIG multi-sensor fusion technologies for small-diameterpipeline surveying,” Micromachines, vol. 11, no. 9, p. 840,2020.

[8] M. C. Chen, S. Q. Lu, and Q. L. Liu, “Uniqueness of weak solu-tions to a Keller-Segel-Navier-Stokes system,” Applied Mathe-matics Letters, vol. 121, article 107417, 2021.

[9] X. Shen, J. Yuan, and Y. Shan, “A novel plume trackingmethod in partial 3D diffusive environments using multi-sensor fusion,” Expert Systems with Applications, vol. 178, arti-cle 114993, 2021.

[10] I. Kyere, T. Astor, R. Graß, and M. Wachendorf, “Agriculturalcrop discrimination in a heterogeneous low-mountain rangeregion based on multi-temporal and multi-sensor satellitedata,” Computers and Electronics in Agriculture, vol. 179, arti-cle 105864, 2020.

[11] S. Xie, Y. Chen, S. Dong, and G. Zhang, “Risk assessment of anoil depot using the improved multi-sensor fusion approachbased on the cloud model and the belief Jensen-Shannon

11Journal of Sensors

divergence,” Journal of Loss Prevention in the Process Indus-tries, vol. 67, article 104214, 2020.

[12] X. Zhu, Z. Jiang, X. Zhao, M. Zhang, and X. Chen, “A novelfuzzy fusion algorithm of multi-sensor data and its applicationin coalmine gas monitoring,” Instrumentation Mesure Métro-logie, vol. 18, no. 6, pp. 577–582, 2019.

[13] Y. Xu, N. M. M. Nascimento, P. H. F. de Sousa et al., “Multi-sensor edge computing architecture for identification of fail-ures short- circuits in wind turbine generators,” Applied SoftComputing, vol. 101, article 107053, 2021.

[14] J. H. Aheto, X. Huang, X. Tian et al., “Multi-sensor integrationapproach based on hyperspectral imaging and electronic nosefor quantitation of fat and peroxide value of pork meat,” Ana-lytical and Bioanalytical Chemistry, vol. 412, no. 5, pp. 1169–1179, 2020.

[15] M. Gao and G. Y. Shi, “Ship spatiotemporal key feature pointonline extraction based on AIS multi-sensor data using animproved sliding window algorithm,” Sensors, vol. 19, no. 12,p. 2706, 2019.

[16] J. Shi, J. Yi, Y. Ren et al., “Fault diagnosis in a hydraulic direc-tional valve using a two-stage multi- sensor informationfusion,” Measurement, vol. 179, article 109460, 2021.

[17] M. Huang, Z. Liu, and Y. Tao, “Mechanical fault diagnosis andprediction in IoT based on multi-source sensing data fusion,”Simulation Modelling Practice and Theory, vol. 102, article101981, 2020.

[18] T. M. Talal, G. Attiya, M. R. Metwalli, F. E. Abd el-Samie, andM. I. Dessouky, “Satellite image fusion based on modified cen-tral force optimization,” Multimedia Tools and Applications,vol. 79, no. 29-30, pp. 21129–21154, 2020.

[19] G. Chen, Y. Wu, L. Fu, and N. Bai, “Fault diagnosis of full-hydraulic drilling rig based on RS–SVM data fusion method,”Journal of the Brazilian Society of Mechanical Sciences andEngineering, vol. 40, no. 3, 2018.

[20] R. Fedele and M. Merenda, “An IoT system for social distanc-ing and emergency management in smart cities using multi-sensor data,” Algorithms, vol. 13, no. 10, p. 254, 2020.

[21] Y. Feng, J. Hu, R. Duan, and Z. Chen, “Credibility assessmentmethod of sensor data based on multi-source heterogeneousinformation fusion,” Sensors, vol. 21, no. 7, article 2542, 2021.

[22] L. Song, J. Huang, X. Liang, S. X. Yang, W. Hu, and D. Tang,“An intelligent multi-sensor variable spray system with chaoticoptimization and adaptive fuzzy control,” Sensors, vol. 20,no. 10, article 2954, 2020.

[23] K. Jafarian, M. Mobin, R. Jafari-Marandi, and E. Rabiei, “Mis-fire and valve clearance faults detection in the combustionengines based on a multi-sensor vibration signal monitoring,”Measurement, vol. 128, pp. 527–536, 2018.

[24] Y. Xie, Q. Wang, L. Yao, X. Meng, and Y. Yang, “Integratedmulti-sensor real time pile positioning model and its applica-tion for sea piling,” Remote Sensing, vol. 12, no. 19, article3227, 2020.

[25] J. Liu, Q. Feng, J. Gong, J. Zhou, J. Liang, and Y. Li, “Winterwheat mapping using a random forest classifier combined withmulti-temporal and multi-sensor data,” International Journalof Digital Earth, vol. 11, no. 8, pp. 783–802, 2018.

[26] R. Yang, M. Xu, T. Liu, Z. Wang, and Z. Guan, “Enhancingquality for HEVC compressed videos,” IEEE Transactions onCircuits and Systems for Video Technology, vol. 29, no. 7,pp. 2039–2054, 2019.

[27] M. Xu, C. Li, Z. Chen, Z. Wang, and Z. Guan, “Assessing visualquality of omnidirectional videos,” IEEE Transactions on Cir-cuits and Systems for Video Technology, vol. 29, no. 12,pp. 3516–3530, 2019.

[28] Q. Jiang, F. Shao, W. Lin, K. Gu, G. Jiang, and H. Sun, “Opti-mizing multistage discriminative dictionaries for blind imagequality assessment,” IEEE Transactions on Multimedia,vol. 20, no. 8, pp. 2035–2048, 2018.

[29] Y. Yang, L. Tao, H. Yang et al., “Stress sensitivity of fractured andvuggy carbonate: an X-Ray computed tomography analysis,”Journal of Geophysical Research: Solid Earth, vol. 125, no. 3, 2020.

[30] J. Yang, J. Zhang, and H. Wang, “Urban traffic control in soft-ware defined internet of things via a multi-agent deep rein-forcement learning approach,” IEEE Transactions onIntelligent Transportation Systems, vol. 22, no. 6, pp. 3742–3754, 2021.

[31] W.Wei, Q. Ke, D. Polap, andM.Wozniak, “Spline interpolationand deep neural networks as feature extractors for signature ver-ification purposes,” IEEE Internet of Things Journal, 2021.

[32] Y. Li, Y. Sun, X. Huang, G. Qi, M. Zheng, and Z. Zhu, “Animage fusion method based on sparse representation andsum modified-Laplacian in NSCT domain,” Entropy, vol. 20,no. 7, p. 522, 2018.

[33] S. Singh and R. S. Anand, “Multimodal neurological imagefusion based on adaptive biological inspired neural model innonsubsampled Shearlet domain,” International Journal ofImaging Systems & Technology, vol. 29, no. 1, pp. 50–64, 2019.

[34] Z. Zhu, M. Zheng, G. Qi, D. Wang, and Y. Xiang, “A phasecongruency and local Laplacian energy based multi-modalitymedical image fusion method in NSCT domain,” IEEE Access,vol. 7, pp. 20811–20824, 2019.

12 Journal of Sensors