Application of Multiple Regression and Artificial Neural ...74.3.176.63/publications/recorder/2004/09sep/09sep-multiple-regression.pdf · networks are in seismic inversion, log analysis

Abstract

Estimation of shear wave velocity (Vs) using log data is animportant approach in the seismic exploration and character-ization of a hydrocarbon reservoir. So far all the availableempirical models for Vs prediction are mathematical modelsthat incorporate only one or two petrophysical parametersand they lack the generalization capability.

This study has concentrated on the “multiple regression”method and the “neural network” technique to predict Vsfrom wireline log data. Neural networks can be trained fasterto converge the network quickly and without getting stuck inlocal minima. Neural networks can correlate a stable modelfor Vs prediction and this is known as “dynamic regression”when comparing with multiple regression.

In this study, after evaluation of the effective petrophysicalproperties on shear wave velocity, a statistical method wasused to establish a correlation among effective petrophysicalproperties and shear wave velocities. Then a fast trainingneural net was used to predict Vs from effective parameters.The model is not a ‘black box’-type approach, because weused data from results of multiple regression but it can beeasily modified by incorporating adequate Vs data. The estab-lished method can predict shear wave velocity from petro-physical parameters and any two of compressional wavevelocity, porosity and density, in carbonate rocks with corre-lation coefficients of about 0.94 for multiple regression and0.96 for the generalization stage of neural net.

Introduction

The natural complexities of petroleum reservoir systemscontinue to provide a challenge to geoscientists. The absenceof reliable data leads to an inadequate understanding ofreservoir behavior and consequently to poor predictions. Inpast decades, classical data processing tools and physicalmodels were adequate for the solution of relatively simplegeological problems. We are increasingly being faced withmore complex problems, and reliance on current technologiesbased on conventional methodologies is becoming less satis-factory (Wong and Nikravesh, 2001).

There are many applications for shear wave velocities inpetrophysical, seismic and geomechanical studies. In manydeveloped oil fields, only compressional wave velocities maybe available through conventional sonic logs or seismicvelocity check shots. For practical purposes such as in seismicmodeling, amplitude variation with offset (AVO) analysis,and engineering applications, shear wave velocities ormoduli are needed. In these applications, it is important toextract, either empirically or theoretically, the needed shearwave velocities or moduli from available compre s s i o n a lvelocities or moduli (Wang, 2000).

In rock physics and its applications, three methods are usednormally to study the elastic properties of rocks: theoreticaland model studies, laboratory measurements and investiga-tions, and statistical and empirical correlations (Wang, 2000).Multiple regression is an extension of the regression analysisthat incorporates additional independent variables in thepredictive equation (Balan et al., 1995).

Artificial neural networks are adaptive and parallel informa-tion-processing systems that have the ability to develop func-tional relationships between data and provide a powerfultoolbox for nonlinear, multidimensional interpolations. Thisfeature of neural nets makes it possible to capture the existingnonlinear relationships that are most of the time not wellunderstood between input and output parameters(Silpngarmlers et al., 2002). Major applications of neuralnetworks are in seismic inversion, log analysis and 3D reser-voir modeling. The applications include determination oflithology, porosity, permeability and fluid saturation fromwireline logs and the generation of synthetic wireline logs(missing and unconventional) from other (conventional) logs(Wong and Nikravesh, 2001).

During the past years, many studies have been done onelastic wave velocities focused on related petrophysical prop-erties of rocks. Unfortunately, most of these studies are aboutsandstones. In Iran most reservoirs are carbonate rocks andthus more study on the petrophysical parameters for carbon-ates are needed.

In the studied area which is a carbonate oil field in ZagrosBasin, south Iran, there isn’t any well with shear wave

40 CSEG RECORDER September 2004

Continued on Page 42

Application of Multiple Regression andArtificial Neural Network Techniques toPredict Shear Wave Velocity from WirelineLog Data for a Carbonate Reservoir,South-West Iran

Eskandari, H.,1 Rezaee, M.R.,2 and Mohammadnia, M.,3

1.Amirkabir University of Technology, Tehran, Iran2. Tehran University, Faculty of Science, Geology Dept., Tehran, Iran3. Research Institute of Petroleum Industry, Tehran, Iran


Article Cont’d


velocity (Vs) data, thus prediction of Vs from other logs wasnecessary. Even when an S-wave log has been run, comparisonwith its prediction from other logs can be a useful qualitycontrol.

In this study, a statistical method was utilized to create a corre-lation among effective petrophysical properties and shear wavevelocities in carbonate rocks. The introduced method can esti-mate Vs with correlation coefficients of about 0.94 for multipleregression and 0.96 for the generalization stage of neural net.

Data Sources

A data set of both compressional and shear velocities in 35carbonate core samples was used, of which 23 are limestones andremaining are dolomites. The velocities are measured at both dryand water saturated conditions. These data were gathered at anultrasonic frequency of 0.5-1 MHz.

X-ray diffraction (XRD), thin sections and scanning electronicmicroscopy (SEM) are used to determine mineralogy, volume ofindividual minerals and other microscopic sedimentologicalfeatures. The petrophysical properties of these core samplescover a wide range for exploration interest, with porosity from0.2% to 29%, permeability from 0.02 to 228.2 mD, clay contentfrom 0% to 10%, calcite content from 47% to 98% and dolomitefrom 0% to 49%.

Petrophysical wireline log data were gathered from four wellsand after deletion of bad hole data, all data are environmentallycorrected and of course with comparison of core porosity theselogs were depth matched.

Shear Wave Velocity Predictor

In the development of the Vs p re d i c t o r, 35 samples of Vs w e recollected. These data were used for multiple re g ression to estab-lish a model for Vs p rediction and because we did not haveenough data to train the network, some data sets that werec reated from multiple re g ression were used during the trainingstage, while the other sets were preserved to test the pre d i c t i o nability of the model.

Input Parameters

Because of the difficulty in finding Vs values along with all rock

and fluid properties, only the most commonly reported andeasily measurable rock and fluid properties that can be obtainedfrom wireline log, were selected as the main input parameters tothe model. These selected parameters must also have significantinfluence on the Vs.

In order to find effective parameters on Vs, we can study empir-ical equations that incorporate various petrophysical parametersto predict Vs. There are several empirical equations (for example,Han et al., (1986) and Castagna et al., (1993)) to predict Vs fromother logs. Due to high validation of Vp-Vs relationships incarbonate rocks, we apply only the Castagna equation and otherparameters and their influence will be distinguished from cross-plots for these parameters and Vs.

In general, these empirical relationships give good results onlyin similar formations and their reliability for other rocks shouldbe considered suspect until a calibration is established. It is there-fore useful to have a physical model that provides some under-standing of shear wave behavior (Wang, 2000).

Although the prediction should be the same, if all measurementsare error free, comparison of predictions with laboratory andlogging measurements show that predictions using compres-sional wave velocities are the most reliable, especially forcarbonate rocks. Figure 1 shows a good relation between Vp andVs in well No. 3 for samples measured under water-saturatedcondition at 31-33Mpa effective stress. Figure 2 shows the plots ofpredicted Vs, using the equation proposed by Castagna et al.,(1993), versus measured data for water saturated samples. Noticethat the Vp data for this equation are derived from sonic logs. TheCastagna et al., equations for limestone and dolomite are:

Vs (km/s) = - 0.05509Vp2 + 1.0168Vp - 1.0305 (1)

Vs (km/s) = 0.583Vp - 0.07776 (2)

where, Vp is in km/s and derived from sonic logs.

In order to deliver an equation with a better correlation coeffi-cient (Castagna et al., equation has a correlation coefficient ofabout 0.70), we used a statistical method to approach a statisticalcorrelation that calculates Vs in this field. At first, we used onlyVp from sonic logs as input to the Castagna equation for thesecarbonate samples. The obtained equation is:

Vs (km/s) = - 0.1236Vp2 + 1.6126Vp - 2.3057 (3)

w h e re, Vp and Vs a re in km/s. Figure 3 shows the plots of pre d i c t e dVs using the equation 3. This equation has one input parameter andc o r relation coefficient for this equation is approximately 0.80.

Application of Multiple Regression and Artificial Neural Network Techniques...”Continued from Page 40

F i g u re 1. A good correlation between Vp and Vs in well#3 at 30MPa effectives t ress and water-saturated condition.

F i g u re 2. Plots of Vs calculated from Castagna et al., (1993) equation versus meas-u red Vs under 30MPa effective stress and water saturated condition.


Article Cont’d


F i g u re 3. Predicted Vs, using linear or single re g ression (eq’n 3), versus measured Vs under 30MPa effective stress and water-saturated condition.

F i g u re 4. The effect of porosity on Vs under water saturated condition.

F i g u re 5. The effect of clay content on Vs under saturated condition. F i g u re 6. The effect of bulk density on Vs under saturated condition.

F i g u re 7. The effect of deep resistivity on Vs under water saturated condition. Figure 8. Plots of predicted Vs using multiple regression equation (eq’n 5) versusmeasured Vs under 30MPa effective stress and water-saturated condition.



Article Cont’d


Other parameters including Neutron Porosity (NPHI), BulkDensity (RHOB), Gamma Ray (GR) and deep laterolog (LLD)were considered for inclusion to the presented equation in orderto increase the accuracy of predicted Vs using multiple variableregression. Figures 4 through 7 show the effect of porosity, claycontent, bulk density and deep resistivity on Vs. Clay content isa significant factor in the study of acoustic velocities. Due to alow amount of clay content of the studied samples (mainly lessthan 6%), the results suggest that clay content had a negligibleeffect on velocity. Vs decreases with increasing porosity and deepresistivity and increases with increasing bulk density.

Multiple Regression

Multiple re g ression is an extension of the re g ression analysis thatincorporates additional independent variables in the pre d i c t i v eequation (Balan et al., 1995). However, the previous empirical studiesp rovide the guidelines for selecting the dependent variables whicha re to be used in the predictor development. A d i ff e rent pre d i c t i v eequation must be established for each new area or new field.

Now, we can use five parameters that were mentioned before(Vp, neutron porosity, bulk density, deep resistivity and Gammaray) as input to multiple regression.

A multivariate model of the data solves for unknown coefficientsa0, a1, a2,…, a5 of a multivariate equation such as equation 4:

Vs = a0 + a1 Vp+ a2 NPHI+ a3 RHOB+ a4 GR+ a5 LLD (4)

The weight of the input variables to predict Vs is given by theirdegree of contribution to the Vs , which is determined by themultiple regression. Contribution factors (3.28, 0.4380, -1.3820, -1.0544, 0.0037 and -0.0011 respectively for a0, Vp, NPHI, RHOB,LLD and GR) indicate that the most important variables in thisregression are the NPHI, RHOB and Vp. The weakest variablesare the GR and LLD. This means that they may be taken out ofthe model as in this case, R2 was nearly 91% when all parameterswere used. We omitted these two factors, GR and LLD andadded the power of other two effective factors and then R2

increased about three percent and reached to 0.94. The new equa-tion is as follows:

Vs = -17.0885+0.4068*Vp-2.1907*NPHI2-1.1794*NPHI-3.2747*RHOB 2+15.3587*RHOB (5)

Estimated Vs using equation 5 shows a good match with meas-u red Vs (Fig. 8) with R2 of about 0.94. Figure 9 presents the

computed Vs f rom multiple re g ression and Vs f rom core versusdepth for well 3. The results were considered to be slightly betterthan those obtained from linear analysis. Multiple re g re s s i o np resented a robust correlation to predict Vs f rom wireline log data.Multiple re g ression is an extension of the re g ression analysis thatincorporates additional independent variables into the pre d i c t i v ee q u a t i o n .

Both methods, empirical and multiple regression were applied tolog data to predict Vs for the studied carbonate reservoir. Theresult shows that statistical methods perform better than empir-ical models.

Network Design and Development

Artificial neural networks are parallel distributed informationprocessing models that can recognize highly complex patternswithin available data (Mohaghegh and Ameri, 1995). An artificialneural network is an information processing system that hascertain performance characteristics in common with biologicalneural networks and therefore, each network is a collection ofneurons that are arranged in specific formations. The basicelements of neural network comprise neurons and their connec-tion strengths (weights). Neurons are grouped into layers. In amulti-layer network there are usually an input layer, one or morehidden layers and an output layer (Mohaghegh, 2001).

The layer that receives the inputs is called the input layer. It typi-cally performs no function on the input signal. The networkoutputs are generated from the output layer. Any other layers arecalled hidden layers because they are internal to the network andhave no direct contact with the external environment. Sometimesthey are likened to a “black box” within the network system.However, just because they are not immediately visible does notmean that one cannot examine the function of those layers.

The ‘topology’ or structure of a network defines how the neuronsin different layers are connected. In this case we have threeparameters (neutron porosity, bulk density and transit time-DT)with significant influence on Vs. Variables such as Gamma Ray,deep laterolog, X and Y coordinates can provide valuable infor-mation to the network, thus were used as input parameters(Figure 10).

A mathematical function then combines the input with some‘ p r i o r’ connection weights (initial weights); it applies anupdating algorithm, and produces the final weights after a


F i g u re 9. Core and computed shear wave velocity using multiple re g ression methodfor well#3.

F i g u re 10. Neural network arc h i t e c t u re used in this study.


Article Cont’d


number of iterations, when a performance criterion is achieved.This process is often referred to as “learning”.

Learning can be performed by “supervised” or “unsupervised”algorithms. The performer requires a set of known input-outputdata patterns (or training patterns), while the latter requires onlythe input patterns (Wong and Nikravesh, 2001).

In a typical neural data processing procedure, the data base isdivided into two separate portions called training and test sets.The training set is used to develop the desired network. In thisprocess, the desired output in the training set is used to help thenetwork adjust the weights between its neurons (supervisedtraining). Once the network has the learned information in thetraining set and has ‘converged,’ the test set is applied to thenetwork for verification. It is important to note that, although theuser has the desired output of the test set, it has not been seen bythe network. This ensures the integrity and robustness of thetrained network (Mohaghegh and Ameri, 1995).

In this study, DTs predictors were developed using a back prop-agation network (BPN), which is one type of feed-forward andsupervised neural network using the generalized delta rule, apowerful learning rule. Once the network weights and biaseshave been initialized, the network is ready for training. Theability of a network to generalize can be developed by training itwith different examples and in this study the network can betrained for function approximation (nonlinear regression thathas been described in the last section). As was mentioned, duringtraining the weights and biases of the network are iterativelyadjusted to minimize the network performance function net(Figure 11). The default performance function for a feed-forwardnetwork is average (mean) square error between the networkoutputs and target outputs that is below a certain tolerance value(in this study, MSE is set to 0.001 it is shown in Figure 12). Oncethe error becomes smaller than the specified criterion, thenetwork is considered to be trained, the learning process isstopped, and connection weights are fixed. The network is nowready for the testing stage.

Three-layer (input, hidden, and output layers) BPNs were devel-oped as a DTs predictor. A tangent sigmoid function producingoutputs in the range of [-1,1] is used as a transfer characteristicfor each neuron in the hidden and output layers. The tangentsigmoid is used because inputs are normalized within this range(Figure 13).

BPNs are probably the most well known and widely usednetworks among the current types of neural network systemsavailable. Even though the back-propagation algorithm ispowerful and simple to implement, BPNs have some drawbacks,such as slow convergence and the possibility that the networkconverges to a local minimum. Some improvement can be madeto overcome these drawbacks. The rate of convergence can beaffected by a learning rate that determines how fast a networkwill learn relationships between input and output patterns. Thelearning rate has a value between 0 to 1. The smaller the value ofthe learning rate, the slower the learning process will be.Adjusting the learning rate during the course of training canaccelerate the convergence. One approach is varying the learningrate during the training stage (Silpngarmles et. al., 2002) and inorder to apply a varying learning rate, we can choose a faster

F i g u re 11. Flow diagram of Variable Learning Rate (VLR) algorithm (max perf-inc,Lr-dec and Lr-Inc are 1.04, 0.7 and 1.05 re s p e c t i v e l y ) .

F i g u re 13. Logistic (tangent hyperbolic) activation function used for all hidden andoutput neurons (Mohaghegh, 2001).

F i g u re 12. Number of iterations (epochs) versus mean square error (MSE) fortraining stage with variable learning rate algorithm.


Initialize primary values of weightand bias matrices.

Do these operations forN times

Do calculation of go path and calculatenew values of weight and bias

Set calculated values to weightsand bias matrices

Article Cont’d



training such as variable learning rate (such as ‘traingda’ or‘traingdx’ in Matlab software).

These faster algorithms fall into two main categories:

1. Heuristic techniques that developed from an analysis of theperformance of the standard steepest descent algorithm suchas variable learning rate BPNs.

2. Standard numerical optimization techniques such as conjugategradient and Leverberg Marquardt algorithms.

In standard BPNs the learning rate is held constant thro u g h o u ttraining and so is very sensitive to the proper setting of thelearning rate. Furthermore, this algorithm is often too slow forpractical problems. In faster training, we have algorithms that canc o n v e rge faster than standard algorithms. In the variable (adap-tive) learning rate algorithm, the learning rate is decreased (typi-cally by multiplying by lr-dec equal to 0.7) if the new error is lessthan the old error (global error increases) and learning ratei n c reased (typically by multiplying by lr-inc equal to 1.05). Whenthe learning rate is too high to guarantee a decrease in erro r, itgets decreased until stable learning resumes. Figure 11 shows aflow diagram of this algorithm. Another approach to prevent thenetwork getting stuck in a local minimum is to incorporate amomentum term that tends to accelerate coverage when theweight vector is moving in a consistent direction (momentum hasa constant value between 0 and 1 and we choose 0.7 for it). Thistraining continues until the average error decreases below 0.001.

When the training is completed, the network is tested for itslearning and generalization capabilities. The test for its learningability is conducted by testing its capability to produce outputsfor the set of inputs that was used in the training. For thispurpose about 10% of inputs from the training data have beenselected and it is observed that outputs with the desired outputsof this data (have a correlation coefficient of about 0.972).

The test for its generalization ability is carried out by investi-gating its capability to predict the output sets that were notincluded in the training process. In this study, for test networkgeneralization, we plot output sets and desired outputs of datathat have been chosen for this stage of network development.This plot has a correlation coefficient of about 0.96 betweenoutput sets and desired outputs (Figure 14). Figure 15 shows Vsfrom core and Vs from neural network prediction.

Results and Discussion

In this study, 35 carbonate core samples, from four wells, wereused. Vp, Vs, porosity and permeability have been measured forall samples. At first, parameters that have significant influenceon Vs were determined and all samples were used to develop themultiple regression. Variables used for the ultimate equationwere Vp, bulk density and neutron porosity (equation 5). Theneural network developed used multiple regression equations.The neural network was developed using a back propagationneural network with 10 hidden neurons in the middle layer, andlogistic activation function (tangent sigmoid) in all hidden andoutput neurons.

The multiple regression method gave good results during theapplication phase, but where it is applied to new wells (the data

from wells that have been put aside) it usually faces problems.Such problems can be avoided with intelligent solution tech-niques. Neural networks have abilities to adapt data that hasbeen presented to them in the form of input-output patterns.This characteristic of neural networks has earned them the titleof “dynamic regression” when compared to rigid regressionmethods.

Figure 16 shows Vs from core, multiple regression and neuralnetwork methods. As can be seen a good agreement existsbetween these methods. We did not have a black box (the math-ematical relationship between input-output data was introducedto the network) in this study, but the excellent results indicatethat training the network was done successfully, especially withthe use of faster training methods.

It seems that neural networks, with their ability to discoverinput-output relationships will increasingly be used in engi-neering applications, especially those in petroleum engineeringusually associated with a high inherent complexity.


F i g u re 14. Vs p redictions from multiple re g ression (as input to network) and neuralnetwork techniques.

F i g u re 15. A c ross plot between measured Vs and predicted Vs f rom neuraln e t w o r k .

Conclusion

This petrophysical study has investigated the use of laboratorymeasurements of acoustic properties on core samples for predic-tion of shear wave velocities using sonic logs. In this study allthree methods, empirical (only one robust model), multipleregression, and neural network, were applied to log data topredict Vs. The results show that the last two techniques performbetter than an empirical model, which can be used to obtain anorder of magnitude for Vs. The intelligent technique seems to bean ideal tool, if used properly, and enough data is available fortraining (for Vs prediction). We observed that the most importantvariable to this regression are the NPHI, RHOB and Vp that playsignificant roles in the statistical model. The introduced equationcan predict Vs with R2 of about 0.94 and 0.96 for an establishednetwork in generalization stage. For this network we used onefast training algorithm (variable learning rate) and did notrequire setting the sensitive parameter such as learning rate. Itseems that the faster training rate causes the network to convergeas early as possible, and the network will not get stuck in localminima.

From this study we make the following recommendations andbelieve that future work should:

1. Apply a conventional model to recognize parameters thataffect the object parameter.

2. Incorporate both conceptual geological models and expert ro l e s .

3. Develop hybrid intelligent models (neural-statis-tics) in order to minimize the technique’s individualweaknesses.

4. Optimize the model parameters of the intelligenttechniques (e.g. number of neurons in neuralnetworks) using advanced numerical techniquessuch as evolutionary computing and fast training toeasily and quickly converge the network whileavoiding local minima.

Acknowledgements

The authors are grateful for support by the NIOCResearch Institute of Petroleum Industry (RIPI),Tehran University and Amirkabir University ofTe c h n o l o g y. The authors acknowledge NIOCResearch Institute of Petroleum Industry (RIPI) fortheir permission to publish this paper. R

ReferencesAl-Qahtani F. A., 2000. Porosity prediction using artificial neuraln e t w o r k: M.Sc. thesis, Morgantown Vi rginia University, We s tVirginia.

Balan, B., Mohaghegh, S., Ameri, S., 1995, State-of-art in permeabilitydetermination from well log data: Part 1- a comprehensive study,model development, SPE 30978.

Braunschweig, B., and Day, R., 1996, Artificial intelligence in thepetroleum industry: Edition Technip – Paris.

Castagna, J. P., Batzle, M. L. and Eastwood, R.L., 1985, Relationshipbetween compressional and shear wave velocities in silicate ro c k s:Geophysics, 50, 571-581.

Castagna, J. P., and Batzle, M. L., and Kan, T. K., 1993, Rock physics- The link between rock properties and AVO response, in Castagna, J. P.,

and Backus, M. M., Eds., Offset-dependent reflectivity-Theory and practice of AVOanalysis: Soc. Expl. Geophys., 124-157.

Greenberg, M. L., and Castagna, J. P., 1992, Shear- wave velocity estimation in porousro c k s: Theoretical formulation, preliminary verification and applications:Geophysical Prospecting, 40, 195-209.

Han, D., 1989, Empirical relationships among seismic velocity, effective pressure, porosity,and clay content in sandstone : Geophysics, 54, 82-89.

Han, D. and Nur, A., and Morgan, D., 1986, Effects of porosity and clay content on wavevelocities in sandstones: Geophysics, 51, 2093-2107.

Koesoemadinata, A. P., and McMechan, G. A., 2001, Empirical estimation ofviscoelastic seismic parameters from petrophysical properties of sandstone: Geophysics, 66,1457-1470.

Mohaghegh, S., Balan, B., Ameri, S., 1995, State-of-the-art in permeability determinationfrom well log data: part 2- verifiable, accurate permeability predictions, the touch-stone of all models, SPE 30979.

Mohaghegh, S., 2001, Virtual intelligence and its applications in petroleum engineering, 1-Artificial neural network: Distiguished Author Series, June 2001.

Silpngarmles, N., Guler, B., Ertekin, T., and Grader, A., S., 2002, Development andtesting of two-phase relative permeability prediction using artificial neural networks: SPE,79547, 299-308.

User,s guide of neural network toolbox of MATLAB software (in pdf)

Wang, Z., 2000, Velocity relationships in granular rocks, in Wang, Z. and Nur, A., Eds.,Seismic and acoustic velocities in reservoir rocks, vol. 3: Recent development,published by Society of Exploration Geophysicists.

Wang, Z., 2001, Fundamentals of seismic rock physics: Geophysics, 66, 398-412.

Wong, P. M. and Nikravesh M., 2001, Field applications of intelligent computing tech-niques: Journal of petroleum geology, 24(4), 381-387.

Wyllie M. R. J., Gregory A. R. and Gardner L. W., 1956. Elastic wave velocities inheterogeneous and porous media: Geophysics, 21, 41-70.


Article Cont’d

F i g u re 16. A comparison between Vs c o re, Vs f rom multiple re g ression and Vs f ro mneural network techniques in an interval of well#3.


Documents

Application of Multiple Regression and Artificial Neural ...74.3.176.63/publications/recorder/2004/09sep/09sep-multiple-regression.pdf · networks are in seismic inversion, log analysis