1-s2.0-S0304380097001804-main.pdf

Ecological Modelling 105 (1998) 337–345

Effect of uncertainty in input and parameter values on modelprediction error

D. Wallach a,*, M. Genard b

a Unite d’Agronomie, Institut National de la Recherche Agronomique (INRA), BP 27, 31326 Castanet Tolosan Cedex, Franceb Unite de Recherche en Ecophysiologie et Horticulture, Institut National de la Recherche Agronomique (INRA),

Domaine Saint-Paul, Site Agroparc, 84914 A6ignon Cedex 9, France

Accepted 26 September 1997

Abstract

Uncertainty in input or parameter values affects the quality of model predictions. Uncertainty analysis attempts toquantify these effects. This is important, first of all as part of the overall investigation into model predictive qualityand secondly in order to know if additional or more precise measurements are worthwhile. Here, two particularaspects of uncertainty analysis are studied. The first is the relationship of uncertainty analysis to the mean squarederror of prediction (MSEP) of a model. It is shown that uncertainty affects the model bias contribution to MSEP, butthis effect is only due to non linearities in the model. The direct effect of variability is on the model variancecontribution to MSEP. It is shown that uncertainty in the input variables always increases model variance. Similarly,model variance is always larger when one averages over a range of parameter values, as compared with using themean parameter values. However, in practice, one is usually interested in the model with specific parameter values.In this case, one cannot draw general conclusions in the absence of detailed assumptions about the correctness of themodel. In particular, certain particular parameter values could give a smaller model variance than that given by themean parameter values. The second aspect of uncertainty analysis that is studied is the effect on MSEP of having bothliterature-based parameters and parameters adjusted to data in the model. It is shown that the presence of adjustedparameters in general, decreases the effect of uncertainty in the literature parameters. To illustrate the theory derivedhere, we apply it to a model of sugar accumulation in fruit. © 1998 Elsevier Science B.V.

Keywords: Model evaluation; Uncertainty analysis; Sensitivity analysis; Prediction error; Parameter adjustment

1. Introduction

Mathematical modeling is increasingly used asa tool for the study of agricultural and ecologicalsystems. Typically, there is uncertainty in the val-

* Corresponding author. Tel.: +33 561285033; fax: +33561735537; e-mail: [email protected]

0304-3800/98/$19.00 © 1998 Elsevier Science B.V. All rights reserved.

PII S 0 304 -3800 (97 )00180 -4

D. Wallach, M. Genard / Ecological Modelling 105 (1998) 337–345338

ues of the input variables and the parameters usedin these models. It is important to quantify theeffects of such uncertainty on the quality of modelpredictions, for two reasons. First of all, in orderto better understand model behavior, it is useful toseparate different contributions to the errors inmodel predictions, and the uncertainty in inputsand parameters is one of these contributions.Secondly, this uncertainty could be reduced bydoing additional or more accurate measurements,and it is important to know how this additionaleffort might improve model predictions.

Sensitivity analysis is one approach to analyzingthe effects of varying input or parameter values onmodel output. It consists of varying the input orparameter values over some range and observingthe effect on some output. This is the approachproposed in de Wit and Goudriaan (1978), andapplied, for example, in Friend (1995). One canexamine the inputs or parameters one at a time, orexplore interactions (Salam et al., 1994). Sensitivityanalysis can be used to identify the parameters towhich the system is most sensitive, with a viewtoward changing the true values of those parame-ters in order to modify system behavior (Silberbushand Barber, 1983; Thornley and Johnson, 1990;Teo et al., 1995). Sensitivity analysis is also used asan exploratory tool to aid in understanding modelbehavior, by indicating which parameters have thelargest effect on the model outputs. However,sensitivity analysis does not indicate whether addi-tional measurements are worthwhile or not. Firstof all, results of sensitivity analysis do not dependon the true uncertainty in the inputs and parame-ters. Also, sensitivity analysis is not explicitlyrelated to the quality of model predictions.

Uncertainty analysis is similar to sensitivityanalysis, but takes into account explicitly the un-certainty in input and in parameter values onoutput. The idea is that, assuming that the distri-butions of the inputs and parameters are known,one can sample from those distributions and gener-ate resulting output variable distributions (e.g.Rossing et al., 1994a,b; Aggarwal, 1995). It is stillnot clear, however, exactly how this result isrelated to model predictive ability.

The purpose of this paper is to examine twoaspects of uncertainty analysis that have not pre-

viously been investigated. The first is the relationbetween uncertainty analysis and the meansquared error of prediction (MSEP) of a model.MSEP is a natural, simple measure of the predic-tive quality of a model. It is of interest then toshow explicitly how uncertainties in input andparameter values affect this measure. The secondgoal here is to show the effect of adjusting certainparameters on the effect of uncertainty in theremaining parameters. Often, one wishes to im-prove the agreement between the model and databy adjusting parameters (Jansen and Heuberger,1995), but the total number of model parametersis too large to envision adjusting them all. Onepossible approach is to settle for adjusting only aselection of parameters. We will show how thispractice changes the significance of uncertainty inthe unadjusted parameters.

In the following section, we define the differentsources of variability that will be considered. Wethen present the definition of MSEP, and showhow variability contributes to MSEP. The effectof the parameter adjustment is investigated usinga Taylor series approximation for the contribu-tion of various sources of variability to MSEP. InSection 3, a model of fruit growth is analyzed asan example. The goal here is to illustrate how thetheory of Section 2 can be applied, and how tointerpret the results. The final section contains asummary and conclusions.

2. Theory

2.1. Definition of 6ariabilities

We begin with the model input variables,defined as the variables in the model that areimposed rather than calculated. This includes ini-tial conditions, climate variables and site andmanagement characteristics. We will refer to thefull collection of input variables as U, so that U isa vector. We consider the situation where we donot have access to U itself but rather to estimatedvalues U.

U. =U+oU

D. Wallach, M. Genard / Ecological Modelling 105 (1998) 337–345 339

where oU is the random error, assumed to havezero expectation. For example, long-term averagemeteorological values used in forecasts could bethe values of U. that are used in the model, inplace of the true but unknown values U. Anotherexample of U. values could be meteorological vari-ables measured at some distance from the sitebeing studied. U then are the meteorological vari-ables at the site. The extent of the variability in U.is indicated by the variance–covariance matrixSU=var(oU).

The parameters in the model may be of twofundamentally different types, depending onwhether they are obtained by some process whichdoes not involve the model, or by adjusting themodel to data. We refer to the first type asliterature parameters since often, though not nec-essarily, the values of these parameters are takenfrom the literature. The vector of true values forthese parameters is denoted P, and the vector ofestimators of these parameters is denoted P. =P+oP. We assume that E(oP)=0. This assump-tion is in fact not very restrictive. It essentiallyrequires that we have some precise definition ofwhat each parameter should represent, and anunbiased estimator of each parameter. The vari-ance-covariance matrix of P. is denoted SP.

The second type of parameters will be referredto as adjusted parameters. The vector of truevalues of these parameters is denoted Q(P. ), andthe vector of estimators is denoted Q. (P. ), withQ. (P. )=Q(P. )+oQ(P. ). The variance–covariancematrix of Q. (P. ) is denoted SQ(P. ). In general,parameter adjustment will be done by non-linearregression, and SQ(P. ) will often be furnished bythe fitting algorithm.

It is important to distinguish between, U. , P. andQ. (P. ). For different randomly chosen situations,the errors in the input variables, oU, are indepen-dent. On the other hand, once the literatureparameter estimates are chosen, the same esti-mates are used for all predictions of the model.That is, for given literature parameter values, theerrors oP are the same for all predictions. Thespecificity of the estimated adjusted parameters isthat they depend on P. , as indicated by the nota-tion Q. (P. ).

2.2. Definition of MSEP

We now introduce the mean squared error ofprediction (MSEP), which is a natural measure ofmodel quality when a major goal of the modellingeffort is prediction of some particular quantitysuch as total harvest weight, or value of a fruitcrop, etc. The MSEP criterion is discussed indetail by Wallach and Goffinet (1987, 1989) andby Colson et al. (1995). A general definition ofMSEP is

MSEP=E{[Y*− f(U. *, P. , Q. (P. )]2}.

Here Y* is the value of the output of interest foran individual chosen at random from the popula-tion of interest and f(U. *, P. , Q. (P. )) is the corre-sponding model prediction. Thus the quantity inbraces is simply the squared difference betweenthe true output and the predicted output, for arandomly chosen individual with some specificvalue of oU and for the model with some specificvalues of oP and oQ(P. ). The expectation is over allthe random variables, that is over the individualsin the population (that is over Y*) as well as overU. , P. and Q. (P. ).

MSEP is averaged over the distribution ofparameter values. The model that one uses, on theother hand, has some particular fixed choice ofparameter values. We will thus also be interestedin examining the effect of uncertainties in theinput values, for fixed parameter values. Thisleads us to consider

MSEP(P. , Q. (P. ))

=E{[Y*− f(U. *, P. , Q. (P. )]2 �P. , Q. (P. )}.

The notation �P. , Q. (P. ) means that the values ofthe random variables P. and Q. (P. ) are fixed. Thusthe expectation in the above expression is nolonger over these variables. If there are noparameters adjusted to data, then the above ex-pression reduces to

MSEP(P. )=E{[Y*− f(U. *, P. ]2�P. }


2.3. Effect of uncertainty on MSEP, no adjustableparameters

It is easy to show that MSEP can be decom-posed into three terms as

MSEP=L+D+G (1)

where,

L=E{[Y*−E(Y* �U*)]2}

population variance

D=E{[E(Y* �U*)−E( f(U. *, P. �U*)]2}

model bias

G=E{[E( f(U. *, P. �U*)− f(U. *, P. )]2}

model variance.

(See Bunke and Droge, 1984 and Wallach andGoffinet, 1987 for similar decompositions). Thepopulation variance term measures how variableY* is for fixed values of the inputs U*. Thisterm is small if the input variables in the modelexplain most of the variability in Y. Thus, thevalue of this term can be used to judge thechoice of input variables (see Wallach andGoffinet, 1987, 1989), but it will not concern ushere since it is independent of variability in theinput variables or parameters. The model biasterm measures the average squared differencebetween the average Y* for a given U*, and thecorresponding model prediction averaged overU. * and P. . It is a measure of how well themodel equations represent Y as a function of U.If U and P enter linearly in the model, thenE(f(U. *, P. �U*)= f(U*, P), and so the bias term,like the population variance term, is indepen-dant of the variability in U. * and in P. . Fornonlinear models however, the model bias termwill not be independent of variability. Themodel variance term G represents the direct ef-fect of uncertainty in the input variables orparameters. For each value of U*, one calcu-lates the variance of the model predictions, andthen one averages over the input values for thepopulation of interest.

It is of interest to compare the effect of un-certainty on MSEP, with the calculations of un-

certainty analysis. First of all, there is an effectof uncertainty on the model bias term, whichhas no equivalent in uncertainty analysis. How-ever, this effect is only due to the nonlinearityof the model, and may often be secondary. Wewill ignore this effect of variability in the rest ofthe discussion. The direct effect of uncertaintyon MSEP is through the model variance term.This term is similar to what is calculated in un-certainty analysis, but there are important differ-ences. First of all, in its most general form,uncertainty analysis calculates the full distribu-tion of the model output variables, and not onlythe model variance. In our case, there is a clearjustification for simplifying and considering onlythe model variance. This is because MSEP onlydepends on this aspect of the distribution. Sec-ondly, the term here involves an expectationover the population of interest. This choice ofinput variables is dictated by the fact that weare looking at the contribution to MSEP. Un-certainty analysis on the other hand, does notprescribe a particular set of input conditions tobe studied. Since the model variance may bevery different for different values of the inputvariables U*, the specification of these variablesis important.

To better understand what G represents, sup-pose for the moment that there is no randomerror in the parameters, and consider just theeffect of variability in the estimated input val-ues. If there were no error in the input variableswe would have SU=0 and G would be zero.Otherwise, G is necessarily positive, since it is anaverage of a variance. That is, if there is a dis-tribution of estimated input values around thetrue values, the result is always an increase inMSEP. The same is true if there is a distribu-tion of parameter values around the true values.MSEP is larger, when averaged over the esti-mated values, than for the true values.

There is however a major difference betweenthe input variables and the parameters. If there isvariability in the input variables, then we cannotavoid the effect of that variability, because foreach individual in the population we will be usinga value drawn at random from the distribution ofthe input variables. This, as we have shown, nec-


essarily leads to an increase in MSEP, comparedwith the case of no variability. With respect to theparameter values, on the other hand, the modelthat is used involves some particular choice ofparameter values. It is of theoretical interest tonote that averaging over possible parameter val-ues increases MSEP, compared with using P.However, for practical purposes, we are interestednot in MSEP averaged over possible parametervalues, but in MSEP(P. ), the value of MSEP forour specific choice of parameter values. Based onthe above arguments, we cannot affirm thatMSEP(P. ) is necessarily larger than MSEP(P).This lack of a clear result is a consequence of thefact that we have made no assumptions about thecorrectness of the model. If, for example, themodel equations are incorrect, then it is possiblethat the model predictions could be improved byusing incorrect parameter values, rather than thecorrect values.

2.4. Effect of parameters adjusted to data

We use a truncated Taylor series expansion toshow how the presence of adjusted parametersaffects G, the contribution of model variance toMSEP. The expansion in the absence of adjustedparameters gives

f(U. *, P. ): f(U*, P)+�(f(u, p)(u

�U*,PnT

(U. *−U*)

+�(f(u, p)(p

�U*,PnT

(P. *−P*)

where partial derivatives with respect to vectorsare column vectors. This leads to

G:GU+GP,

with

GU=!�(f(u, p)

(u�U*, P

nT

SU

(f(u, p)(u

�U*,P"

and

GP=E��(f(u, p)

(p�U*, P

�T

SP

(f(u, p)(p

�U*,P)n

.

If there are adjusted parameters, on the otherhand, the Taylor series expansion gives

f(U. *, P. , Q. (P. ))

: f [U*, P, Q(P)]

+�(f(u, p, q)

(u�U*,P,Q. (P)

n(U. *−U*)

+�(f(u, p, q)

(p�U*,P,Q. (P)

+(Q(p)(p

�P(f(u, p, q)(q

�U*,P,Q. (P)nT

(P. −P)

+�(f(u, p, q)

(q�U,P,Q(P)

nT

[Q. (P)−Q(P)].

Substituting into the expression for G now gives

G:GU+GPA+GQ,

where

GPA=E!�(f(u, p, q)

(p�U*,P,Q. (P)

+(Q(p)(p

(f(u, p, q)(q

�U*,P,Q(P)nT

SP�(f(u, p, q)

(p�U*,P,Q. (P)

+(Q(p)(p

(f(u, p, q)(q

�U*,P,Q. (P)n"

and

GQ=E!�(f(u, p, q)

(q�U*,P,Q(P)

nT

SQ (P)(f(u, p, q)(q

�U*,P,Q(P)"

.

There are two important differences betweenthe two different expressions for G. First of all, inthe case of adjustable parameters there is a termGQ which represents the effect of variability in theadjusted parameters. This term is non negative, sothat any uncertainty in the adjusted parametersincreases MSEP. The second difference is that inthe case of adjusted parameters there is the termGPA, while the corresponding term in the absenceof adjusted parameters is GP. The term GPA

reflects two ways in which the variability in P.affects G. First of all, there is a direct effect.Variations in the values of the literature parame-ters change the model predictions, and this affects


G. Secondly, changing the values of the literatureparameters changes the values of the adjustedparameters, and this also changes the model pre-dictions and therefore G. Often, the two contribu-tions to GPA have opposite signs, and thus tend tocancel one another. The result is that GPA willoften be smaller than GP. The basic reason is thatthe adjustable parameters tend to compensatechanges in the values of the literature parameters.If the values of literature parameters change insuch a way as to increase predicted values, thenthe adjustable parameters will tend to change insuch a way as to decrease these values, in order tokeep the predictions close to the data used foradjustment. Thus, the effect of variability in theliterature parameters on MSEP will usually besmaller in a model with adjustable parameters,than in the same model without parameter adjust-ment. The example in the next section illustratesthis.

3. A model of sugar accumulation in fruit as anexample

3.1. The model

We will apply the presented theory to the modelfor sugar accumulation in peaches of Genard andSouty (1996). We consider a simplified version ofthis model which only includes the sucrose com-ponent which is the major sugar at maturity.

The model first calculates the amount of carbonin sucrose in a fruit. The rate of change equationis

dCsu(t)dt

=k1

dCph(t)dt

−exp[−k2DD(t)]Csu(t),

where t is time after the start of the calculations,Csu(t) is the amount of carbon in sucrose in afruit, Cph(t) is carbon in the phloem, k1 and k2 areparameters and DD(t) is the number of degreedays, calculated using a lower temperature thresh-hold of T0 degrees, after full bloom. The first termon the right represents input of carbon from thephloem, and the second losses of carbon due totransformations to other sugars. Overall, carbonfrom the phloem goes either to increase the total

fruit carbon content or to the respiration losses.Thus

dCph(t)dt

=ct

dDW(t)dt

+gdDW(t)

dt

+mDW(t)[T(t)−T0],

where DW(t) is the dry weight at time t, ct is aparameter that represents the ratio of total fruitcarbon weight to fruit dry weight, g is the growthrespiration coefficient, m is the maintenance respi-ration coefficient and T(t) is temperature at timet. Dry weight is assumed to follow a logistic curveas a function of number of days after full bloomDAB :

DW(t)

=a1+a2

1+exp{−a3[DAB(t)−DAB(0)−a4]/a2},

where, a1, a2, a3 and a4 are parameters, andDAB(0) is the number of days after full bloom atthe time the model calculations begin. The weightof sucrose per fruit S(t) is related to the weight ofcarbon in sucrose per fruit by

S(t)=Csu(t)/0.421.

3.2. Variability of parameters

We consider MSEP for the prediction of finalweight of sucrose per fruit, and for one particularset of conditions studied by Genard and Souty(1996). The treatment considered is peaches ontrees with a leaf-to-fruit ratio of 30, growing atAvignon, France in the summer of 1993. We havechosen to examine the effect on MSEP of varia-tions in just two of the literature parameters,namely ct and g, and in the adjusted parameter k2.The estimated values and variances of the modelparameters are presented in Table 1. The vari-ances for all the other parameters, and of theinput variables (the 1993 Avignon temperatures)have been set to zero.

The estimate ct of ct is the average of 42 mea-surements carried out at the same time as theexperiments of Genard and Souty (1996). Let s2

c

represent the sample variance calculated from


these 42 measurements. Then the estimated vari-ance of ct is s2

c /42.The estimate g of g is the result for peaches

from DeJong and Goudrian (1989). The variancerequired here is the variance between differentdeterminations of g, but this cannot be estimatedfrom a single measurement. We therefore use thevariance calculated from 21 literature values of gfor different species (DeJong and Goudrian, 1989;Penning de Vries et al., 1989), which gives s2

g=0.00784. This is equivalent to assuming that thevariability between different measurements forpeaches would be the same as between measure-ments for different species (which no doubt leadsto an overestimation of the variability), and thatwe have access to just one of these measurements(which is the case).

Since values for the two parameters come fromindependent experiments, it is reasonable to as-sume that the errors in the two parameter valuesare independent. Thus the estimated variance–co-variance matrix S. P is diagonal, with only two nonzero elements equal to the estimated variances ofct and g.

The only parameter used here that Genard andSouty (1996) estimated by fitting the model to datawas k2, and so the vector Q here contains just thatone element. The estimated variance of thisparameter, SQ(P. ), was given by the program thatfurnished the maximum likelihood estimate of k2.

Table 2Contributions to G

Parameter Contribution to G (g2)

0.0001act

0.0049ag0.0039bk2

a The sum of these two terms is GPA.b This is the value of GQ.

3.3. Effect of 6ariability on model 6ariance

We used the Taylor series approximation of theprevious section to obtain an estimate of G and itscomponents. The partial derivatives were esti-mated numerically. The alternative approach is touse a Monte Carlo calculation, which involvesrepeated sampling from the distributions of theparameters. Iman and Helton (1988) compare theMonte Carlo calculation with the Taylor seriesapproximation in uncertainty analysis, and con-clude that the Monte Carlo technique has the bestoverall performance. Haness et al. (1991) alsocompare these methods, and conclude that thedifferences are minor in the particular case thatthey examine. In general, the Taylor series ap-proximation should be adequate if the variancesare relatively small.

Consider first the effect of uncertainty in theliterature parameters on G. From Table 2, theestimated contributions of uncertainty in the liter-ature parameters ct and g are, respectively, 0.0001g2 and 0.0049 g2, for a total of G. PA=0.0050 g2.The two contributions are additive because of theassumptions inherent in the Taylor series approxi-mation that we have used. We could reduce theuncertainty in these parameters to zero by carry-ing out a very large number of additional mea-surements for each. With perfect estimations ofthe two parameters GPA would be zero, and so theestimated average reduction in MSEP would be0.0050 g2. In the present case this reduction issmall (the root mean square error is G. PA=0.07g, compared with the model prediction of 8 gsucrose per fruit), and so additional measure-ments do not seem warranted.

Table 1Estimated values and variances of parameters

TypebParametera Estimated value Estimated variance

la1 (g) 2.48 0a2 (g) 026.1l

la3 (g/day) 2.15 0a4 (day) l 35.3 0

5×10−5m (1/DD) 0l7l 0T0 (°C)

k1 l 0.54 0ct 0.445l 3.4×10−6

g l 0.084 7.8×10−3

ak2 (1/DD) 2.5×10−90.00308

a Units in parentheses; DD, degree days; threshold temperatureT0.b l, Literature parameter; a, parameter estimated by adjustingmodel to data.


We have discussed the effect of reducing thevariances to zero. We can also easily consider theeffect of doing any fixed number of additionalexperiments. Suppose, for example, that one in-creased the number of measurements of g from 1to 4. The contribution to G. PA is proportional tothe variance of g, and the variance is inverselyproportional to the number of measurements.Thus, multiplying the number of measurementsby 4 would reduce the contribution to G. PA by afactor of 4, from the original value of 0.0049 to0.0012 g2.

The same discussion as above can be applied tothe adjusted parameter. The estimated root meansquare error due to the uncertainty in the adjustedparameter k2 is G. Q=0.06 g. Here also, in thespecific case of the example, extra measurementsdo not seem justified.

If there are no adjusted parameters, then thecontribution that uncertainty in the literatureparameters makes to model variance is given byGP. In the present case, if k2 is fixed at its esti-mated value of 0.00308, then the value of G. P dueto uncertainty in the two literature parameters ct

and g is G. P=0.16 g2. This value was calculatedusing the Taylor series approximation. It is 32times as large as G. PA. Thus adjusting k2 ratherthan using a fixed value reduces considerably theeffect of uncertainty in the literature parameterson MSEP, as expected.

4. Summary and conclusions

We have examined the effect of uncertainties ininputs, in parameter values from the literature orin parameter values obtained by adjustment of themodel to data, on the MSEP of a model. Noassumptions are made concerning the correctnessof the model equations. The uncertainty affectstwo different components of MSEP, the modelbias component and the model variance compo-nent. The model bias is affected only because ofnon linearities in the model, while uncertaintyalways contributes to the model variance term.

We have shown that uncertainty in model in-puts always increases the model variance contri-bution to MSEP. Averaging over the distribution

of parameter values also invariably increases themodel variance contribution to MSEP, comparedwith using the true parameter values. However,for any specific parameter value estimates, wecannot know whether MSEP(P. , Q. (P. )), the corre-sponding MSEP, is larger or smaller than for themodel which uses the expectation of P. .

More precise measurements of the input vari-ables are worthwhile if GU, the contribution ofinput variability to model variance, is large com-pared with the prediction error that one is willingto accept. The usefulness of additional measure-ments for the literature parameters or adjustedparameters depends on MSEP. If the contribu-tions of literature and adjusted parameter uncer-tainty to MSEP are small compared withacceptable prediction error, then additional mea-surements are not worthwhile. If these values arelarge, then one should examine MSEP(P. , Q. (P. )).If this is acceptably small, once again additionalmeasurements are not worthwhile. It is only ifboth the contributions of parameter uncertainyand MSEP(P. , Q. (P. )) are large that additionalmeasurements could be useful. It must be reem-phasized, however, that there is no guarantee thatin any particular case additional measurementswill improve the MSEP.

We have also studied the effect of adjustableparameters on the effect of uncertainty in theliterature parameters. It is sometimes argued thata truly mechanistic model should not have anyadjustable parameters. However, it is often thecase that such models contain a fairly large num-ber of parameters, and while the meaning of eachparameter may be well defined, there will nor-mally be some uncertainty in the values. Thecumulative result of these uncertainties may bequite large (Metselaar and Jansen, 1995). Thismay lead one to introducing some adjustableparameters (Jansen and Heuberger, 1995). Wehave explicitly shown here the advantage of thisapproach. The estimated contribution of literatureparameter uncertainty to MSEP depends onwhether or not the model has adjustable parame-ters. In general, the presence of adjustableparameters is expected to decrease the effect ofuncertainty in the literature parameters on MSEP.


References

Aggarwal, P.K., 1995. Uncertainties in crop, soil and weatherinputs used in growth models: implications for simulatedoutputs and their applications. Agric. Syst. 48, 361–384.

Bunke, O., Droge, B., 1984. Estimators of the mean squarederror of prediction in linear regression. Technometrics 26,145–155.

Colson, J., Wallach, D., Denis, J.B., Jones, J.W., Bouniols, A.,1995. The mean squared error of yield prediction by SOY-GRO. Agron. J. 87, 397–402.

DeJong, T.M., Goudrian, J., 1989. Modeling peach fruitgrowth and carbohydrate requirements: reevaluation of thedouble sigmoid growth pattern. J. Am. Soc. Hort. Sci. 114,800–804.

Friend, A.D., 1995. PGEN: an integrated model of leaf photo-synthesis, transpiration and conductance. Ecol. Model. 77,233–255.

Genard, M.M., Souty, M., 1996. Modeling the peach sugarcontents in relation to fruit growth. J. Am. Soc. Hort. Sci.12, 1122–1131.

Haness, S.J., Roberts, L.A., Warwick, J.J., Cale, W.G., 1991.Testing the utility of first order uncertainty analysis. Ecol.Model. 58, 1–23.

Iman, R.L., Helton, J.C., 1988. An investigation of uncer-tainty and sensitivity analysis techniques for computermodels. Risk Anal. 8, 71–90.

Janssen, P.H.M., Heuberger, P.S.C., 1995. Calibration of pro-cess-oriented models. Ecol. Model. 83, 55–66.

Metselaar, K., Jansen, M.J.W., 1995. Evaluating parameteruncertainty in crop growth models. IMACS/IFAC 1st IntSymp on Mathematical Modelling and Simulation in Agri-culture and Bio-industries, Brussels, May 1995.

Penning de Vries, F.W.T., Jansen, D.M., ten Berge, H.F.M.,Bakema, A., 1989. Simulation of ecophysiological pro-cesses of growth in several annual crops. Pudoc, Wagenin-gen, 271 pp.

Rossing, W.A.H., Daamen, R.A., Jansen, M.J.W., 1994a.Uncertainty analysis applied to supervised control ofaphids and brown rust in winter wheat. 1. Quantificationof uncertainty in cost-benefit calculations. Agric. Syst. 44,419–448.

Rossing, W.A.H., Daamen, R.A., Jansen, M.J.W., 1994b.Uncertainty analysis applied to supervised control ofaphids and brown rust in winter wheat. 2. Relative impor-tance of different components of uncertainty. Agric. Syst.44, 449–460.

Salam, M.N., Street, P.R., Jones, J.G.W., 1994. Potentialproduction of Boro rice in the Haor region of Bangladesh.1. The simulation model, validation and sensitivity analy-sis. Agric. Syst. 46, 257–278.

Silberbush, M., Barber, S.A., 1983. Sensitivity analysis ofparameters used in simulating K uptake with a mechanisticmathematical model. Agron. J. 75, 851–854.

Teo, Y.H., Beyrouty, C.A., Gbur, E.E., 1995. Evaluation of amodel to predict nutrient uptake by field-grown rice.Agron. J. 87, 7–12.

Thornley, J.H.M., Johnson, I.R., 1990. Plant and Crop Mod-elling. Clarendon Press, Oxford, pp. 669.

Wallach, D., Goffinet, B., 1987. Mean squared error of predic-tion in models for studying ecological and agronomicsystems. Biometrics 43, 561–573.

Wallach, D., Goffinet, B., 1989. Mean squared error of predic-tion as a criterion for evaluating and comparing systemmodels. Ecol. Model. 44, 299–306.

de Wit, C.T., Goudriaan, J., 1978. Simulation of EcologicalProcesses. Pudoc, Wageningen, pp. 175.

.

Documents

1-s2.0-S0304380097001804-main.pdf