Catch-at-age assessment in the face of time-varying ... · Statistical catch-at-age analysis (SCAA) is a common method of ﬁsheries stock assessment. Age-structured catch data from

Catch-at-age assessment in the face of time-varying selectivity

Brian C. Linton 1* and James R. Bence 2

1NOAA Southeast Fisheries Science Center, 75 Virginia Beach Drive, Miami, FL 33149, USA2Quantitative Fisheries Center, Department of Fisheries and Wildlife, Michigan State University, 13 Natural Resources Building, East Lansing,MI 48824, USA

*Corresponding Author: tel: +1 305 361 4592; fax: +1 305 361 4219; e-mail: [email protected].

Linton, B. C., and Bence, J. R. 2011. Catch-at-age assessment in the face of time-varying selectivity. – ICES Journal of Marine Science, 68:611–625.

Received 27 January 2010; accepted 12 October 2010; advance access publication 24 December 2010.

Age-based fishery selectivity represents the relative vulnerability of specific ages of fish to a fishery, whereby age classes that are highlyselected tend to be overrepresented in the catch compared with their relative abundance in the population. Statistical catch-at-ageanalysis (SCAA) results can be sensitive to misspecification of selectivity, which can occur when changes in selectivity over time are notaccounted for properly in the assessment model. Four approaches for modelling time-varying selectivity were evaluated within SCAAusing Monte Carlo simulations: double-logistic functions with one, two, and all four of the function parameters varying over time, andage-specific selectivity parameters that all varied over time. None of these estimation approaches outperformed the others always. Inaddition, methods of model selection were compared to identify good estimation models, i.e. those accurately matching the true fishpopulation. The degree of retrospectivity, the best selection method, was based on a retrospective analysis of errors in model estimatesas the data time-series for estimation is sequentially shortened. This selection method performed about as well as knowing the correctselectivity model and led to substantial benefits over misspecifying the selectivity model.

Keywords: age-structured stock assessment, model selection, time-varying selectivity.

IntroductionStatistical catch-at-age analysis (SCAA) is a common method offisheries stock assessment. Age-structured catch data from afishery are used to estimate quantities of interest, such as popu-lation abundance and mortality rates, employing likelihoodmethods (Fournier and Archibald, 1982). Auxiliary data thatprovide an index of abundance either directly or indirectly, suchas survey catch per unit effort or fishery effort, are essential forreliable estimation (Deriso et al., 1985; Methot, 1990). Estimatedpopulation quantities from the final year of the analysis are typi-cally used as a starting point for short-term projections that arethe basis for recommending harvest limits or targets.

In many SCAA models, fishing mortality is assumed to beseparable into year and age effects, with their product being therate of fishing mortality for a given year and age (Doubleday,1976). Here, the year effect is referred to as fishing intensity, andthe age effect is referred to as fishery selectivity. Therefore, selectiv-ity is the relative vulnerability of a specific age of fish to a fishery, sothat age classes that are highly selected tend to be overrepresentedin the catch compared with their relative abundance in the popu-lation. Selectivity is influenced by fishing gear characteristics, aswell as by fishing and fish behaviour.

Selectivity often is modelled either as a function of age or isallowed to vary freely among ages. The parameters of the selectivityfunction or the selectivity values for each age are estimated withinSCAA along with other model parameters. Logistic (Millar, 1995;Punt et al., 2001), double logistic (Methot, 1990; Ebener et al.,2005), exponential logistic (Thompson, 1994), normal (Millar,

1995), lognormal (Millar, 1995), gamma (Deriso et al., 1985;Millar, 1995), and polynomials (Fournier, 1983) are some of thefunctions used to model selectivity. Regardless of how selectivityis modelled, a restriction often must be applied to ensure uniqueparametrization of age and year effects (Doubleday, 1976).Selectivity functions generally are constrained by normalizingthe function to a reference age or to the age of maximum estimatedselectivity. When selectivity is allowed to vary freely with age, selec-tivity commonly is constrained by setting selectivity at some refer-ence age(s) equal to 1.

The separability assumption can be relaxed within SCAA,allowing selectivity to change over time, when there is evidenceto suggest that selectivity is not constant, e.g. the gear character-istics or fish behaviour have changed. Separate selectivity valuescan be estimated for different blocks of time within SCAA(Radomski et al., 2005). Some of the selectivity function’s par-ameters can vary over time independently from year to year(Bence and Hightower, 1993), according to a polynomial in time(Ebener et al., 2005) or random-walk process (Gudmundsson,1994; Ianelli, 1996). Non-additive models have been used toallow selectivity to vary with changes in fishing mortality (Myersand Quinn, 2002; Radomski et al., 2005). Allowing for such tem-poral change in selectivity within SCAA provides a middle groundbetween a strict assumption of separability and assuming knowncatch-at-age, as in virtual population analysis.

SCAA is sensitive to the choice of how selectivity is modelled.Incorrect assumptions about selectivity generate errors in theSCAA estimates of biomass (Kimura, 1990), spawning biomass

# 2010 International Council for the Exploration of the Sea. Published by Oxford Journals. All rights reserved.For Permissions, please email: [email protected]

ICES Journal of Marine Science (2011), 68(3), 611–625. doi:10.1093/icesjms/fsq173

at Michigan State U

niversity on September 13, 2013

http://icesjms.oxfordjournals.org/

Dow

nloaded from

mailto:[email protected]




(Punt et al., 2002; Radomski et al., 2005), exploitation rate(Radomski et al., 2005), and the ratio of stock biomass in thefirst year to stock biomass in the final year of analysis (Yin andSampson, 2004). Radomski et al. (2005) looked specifically athow specification of time-varying selectivity influenced SCAA.They compared three specifications for estimating selectivity, con-stant, time-blocked, and non-additive, and found that none of thespecifications performed best in all situations. Rather, time-varying selectivity SCAA models performed as well as constantselectivity SCAA models when selectivity was constant, and out-performed constant selectivity SCAA models when selectivityvaried with time. They also speculated that allowing selectivityto vary according to a random walk might improve the estimationof time-varying selectivity. Radomski et al. (2005) also rec-ommended research to determine the extent to which correct oradequate selectivity models could be identified.

The objective of this study was to compare the performance ofdifferent time-varying selectivity functions within SCAA. Inaddition, model-selection methods that could allow analysts toselect among alternative time-varying selectivity functions withina specific SCAA were identified. This contrasts with an objectiveof determining a single “best” time-varying selectivity estimationapproach, which works well in most situations. Of course, onepossible outcome of this work could have been that an omnibusprocedure for modelling selectivity works better than selectingamong alternatives. These objectives were addressed throughMonte Carlo simulations, in which different methods of bothmodelling time-varying selectivity within a stock assessment andselecting among the methods were evaluated.

Material and methodsMonte Carlo simulations were used to compare time-varyingselectivity estimation approaches and model-selection techniques.These simulations included two scenarios based on two differentmethods for generating time-varying selectivity. A data-generatingmodel was used to simulate 500 datasets from a hypothetical fishpopulation for each scenario, for a total of 1000 simulated datasets.Each of four estimation models, representing different approachesto modelling time-varying selectivity, was fitted to each simulateddataset, and three model-selection techniques were applied to theset of four estimation model results for each. The data-generatingmodel and four estimation models were all built using AD ModelBuilder software (Otter Research Limited, 2004).

Two approaches were used to simulate time-varying selectivityin the data-generating model: (i) a double-logistic function, inwhich the first inflection point varied according to a first-orderautoregressive process; and (ii) selectivity for each age variedaccording to independent (among ages) first-order autoregressiveprocesses. These two alternatives were chosen to provide contrastin the simulations in how freely selectivity varied over time. Thedouble-logistic function allowed selectivity of younger fish (rela-tive to older fish) to change over time, but constrained thechanges so that adjacent ages changed in a similar way. The useof independent autoregressive processes for each age allowed selec-tivity to vary more freely. The use of autoregressive processesmeans that temporal changes in selectivity developed over time,so adjacent years tended to have similar selectivity patterns.

Four estimation models were fitted to the simulated datasets,differing only in how time-varying selectivity was modelled. Theselectivity estimation models consisted of (i) a double-logisticfunction in which the first inflection point varied according to a

random walk, (ii) a double-logistic function in which the firstand the second inflection points varied according to randomwalks, (iii) a double-logistic function in which all four parametersvaried according to random walks, and (iv) selectivity for each agevaried according to a random walk, with a smoothing functionacross ages. These approaches were selected because they rep-resented the two general methods for modelling selectivity inSCAA, namely as a function of age vs. age-specific selectivity par-ameters. In addition, the four approaches form a continuum ofincreasing flexibility in how selectivity is allowed to vary overtime. Finally, the approaches included estimation models thatwere similar to each of the data-generating models in how selectiv-ity was modelled, differing only in the use of random walks ratherthan autoregressive processes for the time-varying quantities (thereason for including this difference is described below).

Model-selection methods were used to select a “best” assess-ment among those fitted to the same simulated data, based on stat-istics measuring model fit. Three model-selection techniques wereconsidered: (i) root mean square error (RMSE), (ii) degree of ret-rospectivity (DR), and (iii) deviance information criterion (DIC).The performance of methods that chose among selectivity modelswas compared against one another and against methods thatassumed the same selectivity model for all datasets.

Descriptions of all symbols used in the model equations areprovided in Table 1, and most of the equations describing themodels are given in Tables 2 and 3. Equations are referenced asEquation (x.y), where Equation (y) is found in Table x, e.g.Equation (3.1) is the first equation in Table 3.

Data-generating modelThe data-generating model was based on lake whitefish (Coregonusclupeaformis) stocks in the upper Great Lakes of the United States.The model-simulated population dynamics, in terms ofabundance-at-age and age-specific mortality rates. A gillnetfishery operating on the simulated population produced observedtotal annual catch, age composition, and fishing effort data. Eachsimulated dataset included 20 years of data for fish aged 1–8+years, where 8+ is a plus group containing all fish aged 8 andolder.

The data-generating model used a conventional deterministicapproach to model the dynamics of abundance for ages olderthan age 1 and after the first year by applying mortality using anegative exponential population function [Equation (2.1)].Abundance-at-age in the first year was generated allowing for vari-ation in recruitment (initial abundance at age 1) among the rep-resented cohorts, but assuming that each cohort was exposed tothe same age-specific mortality [Equation (2.2)]. The recruitmentvalues for each cohort represented in the first year’s age compo-sition were drawn from a lognormal distribution [Equation(2.2)], with parameter values listed in Table 4. The mean of thislognormal distribution was set equal to equilibrium recruitmentat the assumed mortality rates for these initial cohorts, and thesame stock–recruitment function was used to model subsequentrecruitment. Numbers for the first age in each year after the first(recruitment) were calculated with a stochastic Ricker stock–recruitment function [Equation (2.3); Table 4]. The number offemale spawners was set equal to one-half the number of fishaged 3 and older, thereby assuming knife-edge maturity and a1:1 sex ratio.

Total mortality was partitioned into natural and fishing mor-tality, following the standard approach for modelling

612 B. C. Linton and J. R. Bence

at Michigan State U



Dow

nloaded from


Table 1. Symbols and descriptions of variables used in data-generating and estimation models.

Symbol Description Application Symbol Description Application

Cy,a Number of fish caught by year and age Both hy Process error in inflection points ofdouble-logistic function by year

Estimation

Cy Observed number of fish caught by year Both ly Error in fishing intensity by year BothEy Fishery effort by year Both mN Mean number of age 1 fish for abundance in

first yearGeneration

Fy,a Instantaneous fishing mortality by year andage

Both u Set of all model parameters Estimation

M Instantaneous natural mortality Both ql Prior standard deviation of log-scale fishingintensity standard deviation

Estimation

Ny,a Abundance by year and age Both qh Prior standard deviation of log-scale inflectionpoints standard deviation

Estimation

N0 Mean abundance in first year Estimation qn Prior standard deviation of log-scale total catchstandard deviation

Estimation

NE Number of fish used to calculate agecomposition each year

Both qt Prior standard deviation of log-scale slopesstandard deviation

Estimation

Py,a Proportion of catch by year and age Both r1 First correlation parameter for first-orderautoregressive process

Generation

Py,a Observed proportion of catch by year andage

Both r2 Second correlation parameter for first-orderautoregressive process

Generation

R0 Mean recruitment Estimation sN Standard deviation of number of age 1 fish forabundance in first year

Generation

Sy Number of female spawners by year Generation sd Standard deviation of log-scale first inflectionpoint

Generation

Zy,a Instantaneous total mortality by year andage

Both s′d Generating mean of log-scale first inflection

point standard deviationGeneration

Z0,a Instantaneous total mortality forabundance in first year by age

Generation se Standard deviation of log-scale recruitment Generation

b1,y First inflection point of double-logisticselectivity function by year

Both sg Standard deviation of log-scale selectivity Generation

b2 First slope of double-logistic selectivityfunction

Both s′g Generating mean of log-scale selectivity standard

deviationGeneration

b3 Second inflection point of double-logisticselectivity function

Both sh Standard deviation of log-scale inflection points Estimation

b4 Second slope of double-logistic selectivityfunction

Both s′h Prior mean of log-scale inflection points standard

deviationEstimation

b′1 Mean of first inflection point ofdouble-logistic selectivity function

Estimation sw Age-specific standard deviation of log-scaleselectivity

Estimation

fy Fishing intensity by year Both sl Standard deviation of log-scale fishing intensity Bothm Total number of ages Both s′

l Generating and prior mean of log-scalefishing-intensity standard deviation

Both

n Total number of years Both st Standard deviation of log-scale slopes Estimationp u x|( ) Posterior probability density of parameters

conditional on dataEstimation s′

t Prior mean of log-scale slopes standard deviation Estimation

p x u|( ) Probability density of data conditional onparameters

Estimation sn Standard deviation of log-scale total catch Both

p u( ) Prior probability density of parameters Estimation s′n Generating and prior mean of log-scale total

catch standard deviationBoth

q Fishery catchability Both s4 Year-specific standard deviation of log-scaleselectivity

Estimation

sy,a Fishery selectivity by year and age Both ty Process error in slopes of double-logistic functionby year

Estimation

s′a Mean fishery selectivity by age Generation y y Observation error in number of fish caught byyear

Both

wa Mean fish weight by age Both 4y,a Process error in selectivity by year and age Estimationa Productivity parameter of Ricker

recruitment functionGeneration vy Process error in recruitment by year Estimation

b Density-dependent parameter of Rickerrecruitment function

Generation ca Process error for abundance in first year by age Estimation

xi,y Process error in selectivity parameter i byyear

Estimation zd Generating standard deviation of log-scalefirst-inflection-point standard deviation

Generation

dy Process error in first inflection point ofdouble-logistic function by year

Generation zg Generating standard deviation of log-scaleselectivity standard deviation

Generation

Continued

Catch-at-age assessment in the face of time-varying selectivity 613

at Michigan State U



Dow

nloaded from


simultaneous fishing and natural mortality [Equation (2.4)].Natural mortality was constant for all years and ages (M inTable 4). Fishing mortality was modelled by relaxing the assump-tion of full separability [Equation (2.5)]. Year- and age-specificselectivity were generated using two methods to create adome-shaped selectivity curve, which is typical of gillnet fisheries.Fishing intensity was defined as roughly proportional to fishingeffort, deviating from perfect proportionality because of a multi-plicative process error [Equation (2.6)]. In general, fishing inten-sity deviates from being directly proportional to fishingmortality through a combination of process error attributable toannual variation in catchability and observation error innominal fishing effort. Variation in catchability was assumed tooutweigh the observation error in fishing effort. The value forthe standard deviation of log fishing intensity sl was generated

randomly from a lognormal distribution for each simulationused to produce a dataset (s′

l and zl in Table 4). The intenthere is to produce different standard deviations for each simu-lation that will need to be estimated. Fishing effort was specifiedso that effort increased to peak in the middle of the time-seriesand then decreased towards its end (Ey in Table 4). This patternof fishing effort simulated a growing fishery during the first halfof the time-series that was regulated by effort limits during thesecond half. Given that effort time-series can affect results, simu-lations were run with three effort-series, where (i) effort increasedlinearly, (ii) effort increased to an asymptote in the middle of theseries, and (iii) constant effort. Results were qualitatively similarfor all effort time-series, so results from these alternative runswere not presented. The total mortality used to produceabundance-at-age in the first year Z0 [Equation (2.2)] was

Table 1. Continued

Symbol Description Application Symbol Description Application

ey Process error in recruitment by year Generation zl Generating standard deviation of log-scalefishing-intensity standard deviation

Generation

f Subset of time-varying selectivityparameters

Estimation zn Generating standard deviation of log-scaletotal-catch standard deviation

Generation

gy,a Process error in selectivity by year and age Generation

Table 2. Data-generating and estimation model equations.

Equation number Equation Application

2.1 Ny,a = Ny−1,a−1 exp(−Zy−1,a−1), for a , m

Ny,a = Ny−1,a−1 exp(−Zy−1,a−1) + Ny−1,a exp(−Zy−1,a), for a = m

Both

2.2N1,a = N2−a,1 exp −

∑a−1

j=1Z0,j

( ),N2−a,1 � LN(mN,s

2N)

Generation

2.3 Ny,1 = aSy−1 exp(−bSy−1 + 1y), 1y � N(0,s21) Generation

2.4 Zy,a = M + Fy,a Both2.5 Fy,a = sy,afy Both2.6 fy = qEy exp(ly), ly � N(0,s2

l) Both2.7

Cy,a = Fy,a

Zy,aNy,a[1 − exp(−Zy,a)]

Both

2.8 Cy =∑

aCy,a exp(yy), yy � N(0,s2

y) Both

2.9Py,a = Cy,a

Cy

Both

Table 3. Posterior probability density equations for estimation models.

Equation number Equation

3.1 p(u x| )/ p(x u| )p(u)3.2a − ln[ p(u)]/− ln[ p(x|u)] − ln[ p(u)]3.2b − ln[ p(u x| )]/− ln[ p(x u| )] − ln[ p(u)] + g(sy,a;s

2w)

3.3 u = {N0, [ca]ma=1, R0, [vy]n

y=2, q,f,sv,sl}3.4a f = {b1,1, [hy]n−1

y=1 ,sh, b2, b3, b4}3.4b f = {b1,1, [h1,y]n−1

y=1 , b2, b3,1, [h3,y]n−1y=1 , b4,sh}

3.4c f = {b1,1, [h1,y]n−1y=1 , b2,1, [t2,y]n−1

y=1 , b3,1, [h3,y]n−1y=1 , b4,1, [t4,y]n−1

y=1 ,sh,st}3.4d f = {[s1,a]m

a=1, [[4y,a]n−1y=1 ]m

a=1,s4,sw}3.5 ln[ p(x u| )] = ln[ p(C u| )] + ln[ p(P u| )]3.6

ln[ p(C u| )] = − 1

2s2v

∑n

y=1

[(ln Cy − ln Cy)2] − n lnsv

3.7ln[ p(P u| )] =

∑ny=1

NE∑ma=1

(Py,a ln Py,a)


at Michigan State U



Dow

nloaded from


generated using Equations (2.4)–(2.6), with the assumption thatfishing effort in the years before the first year of the analysis wasequal to fishing effort in the first year of the analysis, and selectivityin the years before the first year of the analysis was constant at theinitial values.

The two methods chosen to generate time-varying selectivityprovide contrast in how selectivity changes over time. For thefirst, selectivity was generated using a double-logistic function ofage (Methot, 1990):

sy,a =1/ 1+exp[−b2(a − b1,y)]( )

1−1/ 1 + exp[−b4(a − b3)]( )[ ]

MAXa(numy,a).

(1)

Here, MAXa(numy,a) indicates the maximum of the numeratorterm for the equation, evaluated over all ages for that year.This normalizes age-specific selectivity in a given year, so thatthe most selected age has selectivity of 1.0. The function allowsfor flexible dome-shaped selectivity patterns. The first inflectionpoint varied over time from an initial value, according to

a first-order autoregressive process (Table 4):

loge b1,y+1 = loge b′1 + r1(loge b1,y − loge b′1) + dy,

dy � N(0,s2d). (2)

The initial value of the first inflection point was drawn randomlyfrom a lognormal distribution with mean b′1 and standard devi-ation sd. The standard deviation of the log first inflection pointsd was generated randomly from a lognormal distribution foreach simulated dataset (s′

d and zd in Table 4). This, like the stan-dard deviation for the fishing-intensity error, produced differentvalues for each simulation that needed to be estimated. By allowingthe first inflection point to vary over time, a scenario was simu-lated in which the relative vulnerability of young fish to thefishery was changing over time. The second method used to gen-erate time-varying selectivity allows for variation in selectivity forall ages that does not follow a smooth functional form, based on amethod used by Butterworth et al. (2003). In this approach, age-specific selectivity varied over time from initial values accordingto a first-order autoregressive process (Table 4):

sy+1,a =exp[loge s′a + r2(loge sy,a − loge s′a + gy,a)]

MAXa(numy,a),

gy,a � N(0,s2g). (3)

The same correlation and standard deviation parameters wereused for all ages. The initial values for selectivity at each agewere drawn randomly from lognormal distributions with meanss′a and standard deviation sg. The value for the standard deviationof log selectivity sg was generated randomly from a lognormal dis-tribution for each simulated dataset (s′

g and zg in Table 4).Age-specific selectivity in a given year was normalized using themaximum generated age-specific selectivity value for that year,allowing the vulnerability of each age class of fish to the fisheryto change independently of the others over time. First-order auto-regressive processes were used in the data-generating models tokeep selectivity patterns from becoming unrealistic as theychanged over time. The correlation parameters (r1 and r2 inTable 4) keep the time-varying selectivity parameters from strayingtoo far from their initial values. Examples of the selectivity pat-terns produced by the data-generating model are presented inFigure 1. Corresponding examples illustrating the dynamics ofstock biomass are shown in Figure 2.

Observed data from a gillnet fishery were generated from simu-lated abundance-at-age and mortality rates, while allowing for theobservation error. Actual catch-at-age was calculated usingBaranov’s catch equation [Equation (2.7)]. Observed totalannual catch was calculated by summing actual catch-at-ageacross ages for each year and incorporating the observation error[Equation (2.8); Table 4]. The value for the standard deviationof log total catch sn was generated randomly from a lognormaldistribution for each simulated dataset (s′

v and zn in Table 4).Observed fishery age composition data were generated bydrawing a random sample from a multinomial distribution witha sample size of 400, with population proportions equal to the pro-portions in the actual catch-at-age in the fishery [Equation (2.9)].

Estimation modelsThe estimation models used the same equations as the data-generating model, except for abundance-at-age in the first year,recruitment, and selectivity. Annual recruitment was modelled

Table 4. Values of quantities used in the data-generating model tocreate simulated datasets, and in the estimation models for priordensities, with values assumed known at their correct values forestimation shown emboldened.

Quantity Generating model valueEstimation

model prior

n 20 –m 8 –mN 355 000 –sN 0.4 –a 10.1 –b 5.10E–06 –se 0.4 –[wa]m

a=1 0.20, 0.48, 0.73, 0.91, 1.32, 1.52, 1.76,2.15

–

M 0.24 –[Ey]n

y=1 0.8, 1.6, 2.4, 3.2, 4.0, 4.8, 5.6, 6.4, 7.2,8.0, 8.0, 7.2, 6.4, 5.6, 4.8, 4.0, 3.2,2.4, 1.6, 0.8

–

q 0.15 –s′l 0.4 0.4

zl 0.12 –ql – 0.3b′1 4.01 –[bi]4

i=2 1.40, 3.49, 0.50 –[s′a]m

a=1 0.04, 0.15, 0.43, 0.85, 1.00, 0.82, 0.57,0.37

–

r1 0.9 –s′d 0.2 –

zd 0.12 –r2 0.9 –s′g 0.15 –

zg 0.1 –s′n 0.05 0.05

zn 0.1 –qn – 0.1NE 400 –s′h – 0.2

qh – 0.5s′t – 0.2

qt – 0.5


at Michigan State U



Dow

nloaded from


by an estimated mean recruitment parameter and an estimatedvector of annual recruitment deviation parameters, i.e. a vectorof deviations that must sum to zero. This is algebraically equivalentto estimating recruitment for each year, but is numerically morestable. Similarly, abundance-at-age in the first year was modelledby a mean abundance parameter and a vector of abundance-deviation parameters, i.e. a vector of deviations that must sumto zero. Thus, the estimation models do not presume that theanalyst has any prior knowledge of an underlying stock–recruit-ment relationship, or of the equilibrium conditions that gaverise to the initial age compositions. Abundance-at-age [Equation(2.1)], total mortality [Equation (2.4)], fishing mortality[Equation (2.5)], fishing intensity [Equation (2.6)], catch-at-age[Equation (2.7)], total catch [Equation (2.8)], and proportion ofcatch-at-age [Equation (2.9)] were calculated using the equationsdescribed for the data-generating model. Natural mortality and

observed effort were known without error in these simulations,so the same values from the generating model were known con-stants in the estimation models. True parameter values producedby the data-generating model were used as starting values for par-ameters in the estimation models to expedite numerical conver-gence during simulations. Linton and Bence (2008) determinedthat this approach for selecting parameter starting values reliablyproduced optimal parameter estimates.

The estimation models differed from each other in how theestimated time-varying selectivity for the fishery was modelled.The four approaches chosen for the estimation models representincreasing flexibility in how estimated selectivity could vary overtime. The cost associated with increased flexibility is an increasein the number of parameters that must be estimated. In all fourtime-varying selectivity estimation approaches, age-specific selec-tivity was normalized so that the maximum estimated age-specific

Figure 1. Examples of time-varying selectivity patterns produced by two data-generating models. The data-generating models include adouble-logistic function with one time-varying parameter (a–d) and time-varying, age-specific selectivity parameters (e–h). Selectivitypatterns are labelled by year (1–20), with five years displayed in each panel.


at Michigan State U



Dow

nloaded from


selectivity value for each year was 1.0. In the first approach, the firstinflection point of the double-logistic function was allowed to varyover time according to a random walk:

sy,a =1/ 1+exp −b2(a − b1,y)

[ ]( )1−1/ 1+exp[−b4(a − b3)]

( ){ }MAXa(numy,a)

,

(4)loge b1,y+1 = loge b1,y + hy, hyN(0,s2

h).

This approach is the least flexible of those examined, because itonly changes selectivity for the younger ages at which selectivityincreases most rapidly over time. It is similar to one of thedata-generating models, differing only in that the time-varyingselectivity parameter is modelled as a random walk, rather thanan autoregressive process. In the second approach, the first andthe second inflection points of the double-logistic function wereallowed to vary over time according to random walks:

sy,a =1/ 1+exp[−b2(a−b1,y)]( )

1−1/ 1+exp[−b4(a − b3,y)]( ){ }

MAXa(numy,a),

(5)loge bi,y+1 = loge bi,y + hi,y, hi,y � N(0,s2

h),

where i indexes the inflection points of the double-logistic func-tion (i.e. b1,y and b3,y). The simplifying assumption was madethat the standard deviations of the two log-scale inflectionpoints were equal. This approach of varying the two inflectionpoints allows the ascending and the descending limbs of the selec-tivity curve to expand and contract over time. In the third

approach, the two inflection points and the two slopes of thedouble-logistic function were allowed to vary over time accordingto random walks:

sy,a =1/ 1+exp[−b2,y(a−b1,y)]( )

1−1/ 1+exp[−b4,y(a−b3,y)]( ){ }

MAXa(numy,a),

(6)loge bi,y+1 = loge bi,y +hi,y,hi,y �N(0,s2

h),

loge bj,y+1= loge bj,y +tj,y,tj,y �N(0,s2t),

where i indexes the inflection points and j indexes the slopes (i.e.b2,y and b4,y) of the double-logistic function. Again, as with theinfection points, the standard deviations of the two log-scaleslopes were assumed to be equal. This approach of allowing allthe double-logistic-function parameters to vary over time providesmaximum flexibility in the estimation of time-varying selectivityfor this functional form. In the fourth approach, age-specific selec-tivity values were allowed to vary over time according to randomwalks (Butterworth et al., 2003):

sy+1,a=exp(loge sy,a+4y,a)

MAXa(numy,a), (7)

4y,a �N(0,s24).

Age-specific selectivity was constrained with a curvature penaltyusing squared third-differences to ensure some measures ofsmoothness in selectivity across age classes (Butterworth et al.,2003):

g(sy,a;s2w)

=∑n

y=1

∑m−3

a=1

(loge sy,a+3−3 loge sy,a+2+3 loge sy,a+1− loge sy,a)2

2s2w

.

(8)

This curvature penalty term was added to the negative log pos-terior density. The use of such a smoothness penalty is reasonablewhen dome-shaped selectivity is expected (Butterworth et al.,2003), as is true for the gillnet fishery for lake whitefish uponwhich this study is based. This last approach to estimating selectiv-ity is similar to one of the data-generating models, differing in theuse of a random walk rather than an autoregressive process for thelog-scale selectivities.

Random-walk processes were used to vary selectivity par-ameters over time in all four estimation models. The randomwalk was chosen over the first-order autoregressive process usedin the data-generating models because the correlation parameterof the first-order autoregressive process is often difficult to esti-mate. Therefore, no estimation model was correctly specified, i.e.exactly matching the data-generating models.

Statistical inference was made on the posterior density of theparameters conditional on the observed data [Equation (3.1)],which was derived using a Markov-chain Monte Carlo (MCMC)method. More specifically, MCMC with the Metropolis–Hastings algorithm, as implemented in AD Model Builder(Otter Research Limited, 2004), was employed. Highest posterior-density parameter estimates (similar to maximum likelihoodexcept that the sum of log-likelihood and log-prior components

Figure 2. Examples of population biomass time-series produced bytwo data-generating models. The data-generating models include (a)a double-logistic function with one time-varying parameter, and (b)time-varying, age-specific selectivity parameters.


at Michigan State U



Dow

nloaded from


is maximized) were used as starting values for each MCMC chain.If an estimation model failed to converge to a highest posterior-density solution, the estimation model was rerun using differentsets of random-parameter starting values until either convergenceor ten additional sets of random-parameter starting values hadbeen tried. If there was still no convergence after trying the tenadditional sets of starting values, then that simulation run wasnot included in the analysis of that estimation approach. TheMCMC chain for each model was run for 1 000 000 cycles,saving parameter values every tenth cycle. The first 40 000 cycleswere dropped from the chain of saved MCMC values as aburn-in period, which reduced the effect of chain starting valueson final MCMC estimates (Gelman et al., 2004). Model runswith poor convergence properties were dropped from the analysis.The MCMC chain convergence was judged to be poor if the effec-tive sample size for the negative log sampling density of the dataconditional on the parameters, which takes the form of the nega-tive log-likelihood function, was ,300. The sampling density wasselected because it provides an overall measure of how the MCMCchains mix. Effective sample sizes were calculated from MCMCchains using the method described by Thiebauz and Zwiers(1984), with lags out to 100 for autocorrelation calculations. Thenegative log posterior density [Equation (3.2a)] was minimizedfor ease of computation. For the fourth estimation approach, inwhich age-specific selectivity values varied over time, the curvaturepenalty term [Equation (8)] was added to the negative log pos-terior density [Equation (3.2b)]. The parameters estimated inthe model [Equation (3.3)] included the subset of parameterscommon to all estimation models and the subset of time-varyingselectivity parameters f specific to each estimation model.

The subset of estimated parameters used to model time-varyingselectivity differed among the estimation approaches used forselectivity. For the approach in which the first inflection point ofthe double-logistic function varied with time, the selectivity par-ameters included the first inflection point in the first year,annual deviations in the first inflection point, standard deviationof the log-scale first inflection point, and the other three par-ameters of the double-logistic function [Equation (3.4a)]. Forthe approach in which both inflection points of the double-logisticfunction varied with time, the second inflection point was replacedby a second inflection point in the first year and annual deviationsin the second inflection point [Equation (3.4b)]. For the approachin which all four parameters of the double-logistic function variedwith time, both slopes were also replaced by corresponding slopesin the first year, annual deviations for each of these parameters,and a standard deviation of the log-scale slope deviations[Equation (3.4c)]. For the approach in which the age-specificselectivity values varied with time, the selectivity parametersincluded the age-specific selectivity values in the first year,annual deviations for each age-specific selectivity value, and stan-dard deviations for the year- and age-specific log selectivity values[Equation (3.4d)].

The sampling density of the data conditional on the parameterswas separated into two components for total catch and proportionof catch-at-age [Equation (3.5)]. Total annual catch was assumedto follow a lognormal distribution, with the log density (ignoringsome additive constants) given by Equation (3.6). Proportion ofcatch-at-age was assumed to follow a distribution that wouldarise if NE fish were observed, with numbers observed at eachage following a multinomial distribution, and log density (ignor-ing some additive constants) expressed by Equation (3.7).

For all time-varying selectivity estimation approaches, the priorprobability densities of the random-walk deviations for selectivityparameters were assumed to follow lognormal distributions, withthe log-prior densities (ignoring some additive constants)expressed as

ln[ p(xi)] = − 1

2s2i

∑n−1

y=1

[x2i,y] − (n − 1) lnsi, (9)

where i indexes the time-varying selectivity parameters, e.g. thefirst inflection point of the double-logistic function. The priorprobability densities of the log total catch, log catchability, andlog selectivity standard deviations were assumed to follow lognor-mal distributions, with the log-prior densities (ignoring someadditive constants) expressed as

ln[ p(si)] = − 1

2q2i

(lns′

i − lnsi)2 − lnqi, (10)

where i indexes the error sources, e.g. observation errors in thetotal catch. A strong informative prior density identical to the gen-erating distribution from the data-generating model was assignedto the log total catch standard deviation (Table 4), so the analystwas assumed to have good prior information on how observationerrors in total catch were distributed, a reasonable assumption fora well-monitored commercial fishery. The analyst was alsoassumed not to have such strong prior information for the otherstandard deviations, so more weakly informative prior densitieswere assigned, allowing the remaining standard deviations tovary over a realistic range of values (Table 4). The time-varying,age-specific selectivity-parameter-estimation model failed to con-verge to a solution when weakly informative prior densities wereassigned to the year- and age-specific log selectivity standard devi-ations, s4 and sf, respectively. Hence, the values for the year- andage-specific log selectivity standard deviations were fixed at 0.15and 0.08, respectively, for all simulations. This solution followedthe common practice of assuming variances to be known whenthey cannot be estimated in the estimation model. Weakly infor-mative uniform prior densities were assigned to the logs of allother model parameters, so prior densities for each log-scalemodel parameter, excluding the selectivity random-walk devi-ations and their associated variances, were constants for all par-ameter values.

The performance of the four estimation models was comparedby calculating the relative error (RE) of population biomass andexploitation rate in the final year of analysis for each simulateddataset:

RE = X − X

X× 100%, (11)

where X is the point estimate of the quantity of interest from theestimation model, and X is the true value of the quantity of interestfrom the data-generating model. The median of the marginal pos-terior distribution was used as a point estimate. Preliminary simu-lation runs suggested that the median posterior distributionestimates of several important parameters, e.g. observation- andprocess-error variances, were less biased than the highestposterior-density estimates, which are widely used and frequentlyreferred to as maximum likelihood estimates. Estimated biomass


at Michigan State U



Dow

nloaded from


and exploitation rate in the last year often play an important rolewhen stock assessment results are used to inform managementactions. In addition, the median of the REs (MRE) was used toexamine whether there was systematic bias in estimates from theestimation models. The median absolute RE (MARE), which cap-tures elements of bias and precision, was used to compare therange of REs made when using the estimation models.

Model selectionThe performance of three model-selection techniques was evalu-ated to determine which technique(s), if any, could identify con-sistently the “best” time-varying selectivity estimation approach.The three model-selection techniques used to identify the besttime-varying selectivity estimation approach were RMSE, DR,and DIC. The best approach was considered to be the one whichmost closely predicted the true fish population produced by thedata-generating model. More specifically, the best approach for agiven simulation run was the estimation model producing thelowest final population biomass or exploitation rate RE. TheMRE and the MARE values were used to examine bias and pre-cision in estimates from the estimation models chosen by eachselection method. Comparison of the model-selection methodswas made using the full set of 1000 simulation runs, because atleast two estimation models converged to solutions for each simu-lated dataset.

The first model-selection procedure focused on the fit betweenobserved and predicted proportions of catch-at-age, with theselected model minimizing the RMSE for these residuals, whichwere calculated by subtracting the posterior medians of the pre-dicted proportions of catch-at-age from the observed proportionsof catch-at-age. This was chosen as one possible method because itwas thought that generally poor prediction of the proportionalcatch-at-age might transpire for estimation models that modelledselectivity patterns incorrectly.

The second model-selection method was based on retrospectiveanalysis, which involves comparison of successive estimates ofmodel-output quantities as additional years of data are added tothe stock assessment (Parma, 1993; Mohn, 1999). For this selec-tion method, the model that minimized the absolute value ofMohn’s (1999) DR statistic was selected:

DR =∑n

y=n−10

X(1:y),y − X(1:n),yX(1:y),y

, (12)

where X(1:y),y is the predicted value of quantity X in year y esti-mated from the dataset spanning year 1 to year y, and X(1:n),y isthe predicted value of quantity X in year y estimated from thedataset spanning year 1 to the last year of the full dataset n.Examples of retrospective stock assessment estimates of biomassthat would be used in calculating the DR are given in Figure 3.A retrospective analysis for each estimation-model-simulateddataset fit was conducted by dropping a year of data from thesimulated dataset and refitting the estimation model, repeatingthe process until the final 10 years of data had been removedsequentially from the analysis. Retrospective patterns in modelquantities can be systematic when time-varying processes aremodelled as constant over time (Mohn, 1999). Although all theestimation models allowed selectivity to change over time, retro-spective patterns might be seen when an estimation model had dif-ficulty tracking changes in selectivity. To make this approach

practical, highest posterior-density estimates of the parameterswere used, with the variance parameters fixed at their point esti-mates from the analysis of the full dataset.

The final selection method was to select the model that mini-mized the DIC (Spiegelhalter et al., 2002). Information-theoretic,model-selection criteria generally work by balancing modelgoodness-of-fit against model complexity, i.e. the number of par-ameters in the model. The effective number of parameters incomplex models, such as these SCAA models, is often fewer thanthe actual number of parameters because of various constraintsplaced upon those parameters. The DIC was chosen rather thanthe more commonly used Akaike’s information criterion(Akaike, 1973) and Bayesian information criterion (Schwartz,1978) because the DIC provides a means of estimating the effectivenumber of parameters. Wilberg and Bence (2008) demonstrated,in a different SCAA application, that selection by DIC couldresult in estimates with lower mean square errors than whenalways using any particular single model.

The DIC consists of two components (Spiegelhalter et al.,2002):

DIC = D′ + pD, (13)

where D′ is the average deviance and pD the effective number ofparameters. The average deviance was calculated (Spiegelhalteret al., 2002) as

D′ = 1

C

∑C

c=1

−2 loge p(x uc| ), (14)

where C is the number of MCMC cycles saved minus the burn-in,and P(x|uc) is the probability of data x conditional on parameters ufrom MCMC cycle c. The effective number of parameters was esti-mated (Wilberg and Bence, 2008) as

pD = D′ − D(uHPD), (15)

where D(uHPD) is the deviance evaluated at the highest posterior-density parameter estimates, and the other variables are describedabove. For each model-selection method, the distribution of REsfor the final population biomass and exploitation rate estimateswas examined.

The performance using estimation models selected by DR, cal-culated from a retrospective analysis conducted on each simulateddataset, was compared with the performance of always using thesame estimation model for each estimation model. The objectivewas to determine whether this model-selection technique outper-formed the omnibus approach of always using the same estimationmodel. Model selection by DR was used in this evaluation becauseof the good performance of this method (see below). Comparisonswere made using MRE and MARE values for the final populationbiomass and exploitation rate.

ResultsModel runs exhibiting poor convergence characteristics for a par-ticular estimation approach were dropped from the analysis ofresults for that approach. The following results are based onsample sizes of 410–498 model runs per data-generatingmodel–estimation model combination (Table 5). All droppedmodel runs failed to converge to highest posterior-density


at Michigan State U



Dow

nloaded from


solutions, so MCMC simulations could not be run. With sufficienteffort, which would be warranted for a real assessment, it is sus-pected that an analyst could have made adjustments, e.g. changeparameter starting values, bounds, and phases of estimation, inmany of these cases to achieve convergence. This was not practicalin the context of this simulation study. Note that the subsets ofsimulation runs demonstrating poor convergence characteristicsgenerally were different for each of the estimation models, i.e.that incidents of poor convergence were not attributable solelyor primarily to characteristics of particular simulated datasets.

Simulation results based on the exploitation rate in the finalyear were qualitatively similar to results based on the populationbiomass in the final year (Table 5). Therefore, only results basedon the final population biomass are presented.

Estimation modelsThe four estimation models produced less-biased estimates of thefinal population biomass when applied to data from thedouble-logistic-generating scenario than data from the age-

specific, selectivity-parameter-generating scenario. There was lessdifference in the biases of the four estimation models’ estimatesof population biomass in the final year of analysis for thedouble-logistic-generating scenario than for the age-specific,selectivity-generating scenario (Figure 4). The values of MRE forthe final population biomass ranged from 0.07 to 3.9% forthe double-logistic, function-generating scenario (Table 5). Incontrast, MRE values for the final population biomass rangedfrom 226.3 to 40.6% for the age-specific, selectivity-parameter-generating scenario. Bias increased among the three double-logistic, function-estimation models as the number oftime-varying parameters increased in both data-generatingscenarios.

The four estimation models produced more precise estimates ofpopulation biomass in the final year of analysis when the esti-mation models more accurately represented the true underlyingpopulation, i.e. when selectivity estimation and data-generatingmodels were similar (Figure 4). MARE for the final populationbiomass varied from 20.5 to 24.0% for the three double-logistic,

Figure 3. Example of true population biomass produced by the data-generating model and retrospective patterns of population biomassproduced by two estimation models from a simulation run in which the double-logistic function with two time-varying parameters (DL2) wasthe best estimation model. (a) The DR selected the best estimation model, whereas (b) the DIC and RMSE selected the same less-than-bestestimation model, the double-logistic function with one time-varying parameter (DL1).


at Michigan State U



Dow

nloaded from


function-estimation models in the double-logistic, function-generating scenario (Table 5). On the other hand, the age-specific,selectivity-parameter-estimation model had a final populationbiomass MARE value of 58.9% for the double-logistic, function-generating scenario. The age-specific, selectivity-parameter-estimation model had a final population biomass MARE valueof 37.0% for the age-specific, selectivity-parameter-generatingfunction. In contrast, the three double-logistic, function-estimation models had final population biomass MARE valuesranging from 47.5 to 59.9% for the age-specific, selectivity-parameter-generating scenario.

As summarized above, all estimators were biased, to someextent, for biomass in the final year. This is true although someestimation models were similar to generating models (differingonly in that time-varying selectivity parameters followed randomwalks in the estimation models rather than the autoregressive pro-cesses in the generating models). The existence of such biaswithout substantial model misspecification is a basic property ofnon-linear models with non-normal error. Bence et al. (1993)discuss this issue for catch-at-age models in more detail.

Results for the estimation of biomass in the final year indicate acost in terms of precision and bias either to allow for temporalvariation in selectivity at older ages when it does not exist or toignore or model it in an oversimplified way when it does. Giventhe general difficulty in estimating the descending limb of a selec-tivity function, it would not be surprising if poor estimates of thedescending limb might result in either case and be at least partlythe cause for these results. Examination of the simulation studyresults lends support to this hypothesis. Estimates of selectivityin the final year produced by the four estimation models, alongwith the true selectivity produced by the data-generating modelfrom an arbitrary selection of simulation runs, are presented inFigure 5. One or more of the estimation models has difficulty inestimating the descending limb of the selectivity function in themajority of these runs (Figure 5a–e).

Model selectionSelecting estimation models using DR produced estimates ofpopulation biomass in the final year of analysis with equal orbetter characteristics (lower or equal bias, higher or equal pre-cision) than those obtained by selecting estimation models usingDIC and RMSE (Figure 6). In particular, DR selected estimationmodels that produced final population biomass estimates thatwere less biased and more precise than those selected by DICand RMSE in the age-specific, selectivity-parameter-generatingscenario (Table 5). An example of actual and estimated biomassesfrom a retrospective analysis is shown in Figure 3, where DRselected the “correct” (most similar to actual) model and DICand RMSE did not, illustrating the type of retrospective patternsbeing used when selecting models by DR.

The DR performed as well as or better than the individual esti-mation models in estimating the final population biomass. It pro-duced a final population biomass MRE of 0.5% and MARE of24.2% in the double-logistic-generating scenario, comparablewith the estimation performances of the three time-varying,double-logistic functions in the same scenario (Table 5). The DRproduced a final population biomass MRE of 28.7% andMARE of 39.7% in the age-specific, selectivity-parameter-generating scenario, comparable with the least biased, i.e. double-logistic with one time-varying parameter, and most precise, i.e.

Table 5. MREs, MAREs, and number of replicates (n) for theestimates of final population biomass and the exploitation rateproduced by the time-varying selectivity estimation models:double-logistic functions with one (DL1), two (DL2), and four (DL4)time-varying parameters, and time-varying, age-specific selectivityparameters (ASP), and chosen by the model-selection methods:RMSE, DIC, and DR.

Estimation model

DL1 ASP

MRE MARE n MRE MARE n

Population biomassDL1 0.7 20.5 415 6.7 49.0 459DL2 2.6 22.1 434 12.1 47.5 492DL4 3.9 24.0 487 40.6 59.9 498ASP 2.6 58.9 410 226.3 37.0 410RMSE 1.8 22.0 500 36.3 64.9 500DIC 2.1 22.3 500 28.4 61.4 500DR 0.5 24.2 500 28.7 39.7 500

Exploitation rateDL1 21.7 20.3 415 27.4 53.0 459DL2 23.5 21.3 434 210.6 49.6 492DL4 24.3 22.5 487 230.2 58.2 498ASP 21.3 63.0 410 34.4 45.4 410RMSE 21.3 20.9 500 227.5 68.1 500DIC 21.2 21.4 500 223.8 66.1 500DR 1.3 23.8 500 10.3 45.1 500

Estimation models were fitted to observed data from the data-generatingmodels: double-logistic functions with one time-varying parameter (DL1),and time-varying, ASP.

Figure 4. Box plots representing RE distributions for estimates ofpopulation biomass in the last year of analysis for twodata-generating models and four estimation models. Thedata-generating and estimation models include double-logisticfunctions with one (DL1), two (DL2), and four (DL4) time-varyingparameters, and time-varying, age-specific selectivity parameters(ASP). Generating models are distinguished by location on the x-axis,and estimation models by shading. The bars represent MREs. Theboxes and whiskers represent 25th and 75th, and 10th and 90thpercentiles of the distributions, respectively.


at Michigan State U



Dow

nloaded from


age-specific selectivity parameters, of the individual estimationapproaches (Table 5).

DiscussionThere was no single time-varying, selectivity estimation model thatoutperformed the others in all situations examined. Rather, theestimation model(s) that produced the estimates most tightly dis-tributed about the true population biomass in the final year ofanalysis was the one that most closely represented the true under-lying population. The three estimation models that used variantsof the double-logistic function to model time-varying selectivityproduced better estimates of final population biomass than theage-specific, selectivity-parameter-estimation model when theselectivity of the true population was generated with a double-logistic function. Likewise, the age-specific, selectivity-parameter-estimation model produced better estimates of finalpopulation biomass than the three double-logistic,function-estimation models when the selectivity of the true popu-lation was generated with age-specific selectivity parameters. Thistype of result is common to simulation studies where there aresimilarities between data-generating and estimation models(Radomski et al., 2005; Wilberg and Bence, 2006).

A subset of estimation-model fits produced large positive errorsin estimates of final population biomass. These large errorsoccurred in cases where the estimation model differed from the

data-generating model in how time-varying selectivity was mod-elled. The likelihood surface must have been relatively flat inthese runs to make such extreme biomass values seem plausibleto the estimation model. It probably was a combination ofmodel misspecification along with a flat likelihood surface thatproduced the large errors. In a real stock assessment, an analystlikely would notice such unrealistic estimates of populationbiomass and make adjustments to the assessment model. Theresults of this study suggest that some such errors might beavoided by explicitly considering a range of selectivity models,and selecting among them. This range should include bothsmooth functions and functions that estimate selectivity for eachage group. When substantial temporal change in selectivityseems likely, this can be incorporated with selectivity model var-iants, where one or many parameters are allowed to vary, andwe found the use of random walks an effective way to incorporatesuch variations into assessment models. Time-blocks, withselectivity constant within a time-block, but differing amongthem, are another model worth considering, although it was notused in this study. Radomski et al. (2005) found time-blocks towork well for a range of temporal variation in selectivity.

If an analyst knows the underlying form that selectivity takes ina fish population, then he or she can model time-varying selectiv-ity reasonably well. This begs the question, how well can one knowthe true selectivity patterns and how they vary over time? Many

Figure 5. Examples of true selectivity patterns produced by two data-generating models and the selectivity patterns produced by fourestimation models for the final year of the assessment. The data-generating models include a double-logistic function with (a–c) onetime-varying parameter and (d–f) time-varying, age-specific selectivity parameters (ASP). The estimation models include double-logisticfunctions with one (DL1), two (DL2), and four (DL4) time-varying parameters, and time-varying ASP.


at Michigan State U



Dow

nloaded from


fishing gears such as gillnets and trapnets are size-selective. If theassumption is made that the size and the age of fish are correlatedand, by extension, that selectivity and age of fish are correlated,then modelling selectivity as some function of age, producing asmooth selectivity curve, is reasonable. Such approaches areeasily adapted to incorporate time-varying selectivity by allowingone or more parameters to change over time. Although suchsmooth functions are likely, Kimura (1990) demonstrated (forconstant selectivity) that estimating selectivity independently foreach age can outperform the use of a selectivity function whenthe function was specified incorrectly. The approach ofButterworth et al. (2003) used in this study is a modification ofthe approach used by Kimura (1990) and enforces some smooth-ness to the selectivity function while allowing selectivity at each ageto be influenced by independent stochastic errors. Further study isneeded to determine the relative performance of estimationmodels that use time-varying, age-specific selectivity parametersand estimation models that use a time-varying selectivity function,when that function is misspecified in various ways.

Model complexity is another issue that needs to be addressedwhen evaluating different time-varying selectivity models.Increased model complexity means that more parameters mustbe estimated, which can lead to overparametrization of themodel. An overparametrized model can produce poor parameterestimates with high variances (Burnham and Anderson, 2002).Here, the issue of model complexity was most clearly demon-strated in the performance of the double-logistic functionwith four time-varying parameters and two associated variances.The four-time-varying parameter, selectivity-function estimationapproach was expected to outperform the other double-logistic-

function approaches when the observed data were generatedusing age-specific selectivity parameters, because of the increasedflexibility granted by allowing all four double-logistic-functionparameters to vary over time. Instead, the four time-varying par-ameter, double-logistic-function approach produced less-accurateand less-precise estimates of the final population biomass than theother double-logistic, function-estimation approaches. One of thetwo variances associated with the slopes and inflection points ofthe four time-varying parameter, double-logistic-functionapproach was estimated to be nearly zero, i.e. making it effectivelythe same as the two time-varying parameter, double-logistic-function approach in many of the simulation runs, suggestingthat the observed data were not informative enough to estimateall the selectivity parameters.

In hindsight, perhaps this result is not that surprising.Estimation of the declining limb of a domed-shaped selectivitycurve can be particularly challenging, and choices can have sub-stantial influence on assessment results. Here, the challenge isincreased by attempting to estimate temporal changes, when thenature of those temporal changes has been misspecified. The rela-tively better performance of the estimation model that allowed forindependent age-specific changes, when that was the case, suggeststhat addressing such changes in selectivity is possible. However,there appear to be substantial costs to allow for some types of tem-poral changes in selectivity that are not present. This contrasts witha previous work on allowing for temporal change in catchability,where there was little cost to allowing for potential changes thatwere not occurring (Wilberg and Bence, 2006). It also contrasts,to some extent, with the results of Radomski et al. (2005), whofound little cost in estimating separate selectivity by 5-year time-blocks when selectivity was constant. It is clear that one shouldnot simply use a very flexible omnibus approach for estimationof time-varying selectivity.

We recommend that DR be used to select between estimationmodels using different models for time-varying selectivity. Aswell as outperforming other selection methods here, selecting byDR worked as well as or better than choosing any single estimationmodel, even the correct one. Nothing was lost in estimation per-formance by using the DR rather than a correctly specified selec-tivity model, and much was gained over assuming an incorrectselectivity model.

Considering and selecting among a range of selectivity modelsbased on DR is logistically feasible in many assessment situations.Calculating DR only requires that each assessment model variantbe refitted to produce point estimates for each year of dataomitted. This contrasts strikingly with model selection by DIC,which requires MCMC runs for each model variant. Althoughthis study was done in an explicitly Bayesian context, the DRcan be calculated for models fitted using any definition ofgoodness-of-fit, and we suspect that the usefulness of DR in select-ing among models would extend to a broader array of model-fitting procedures. Even when a formal model-selection approachis not being used, an implication of the results here is that DR, andretrospective analysis in general, can provide useful informationon whether an assessment appropriately models how selectivitychanges over time.

It was surprising to see how well DR performed as a model-selection method compared with DIC and RMSE, especially withthe former. The DIC has generally performed well in model selec-tion, including applications to fishery assessment models(Cardoso and Tempelman, 2003; Berg et al., 2004; Ward, 2008;

Figure 6. Box plots representing RE distributions for estimates ofpopulation biomass in the final year of analysis chosen bymodel-selection methods for two different data-generating models.The data-generating models include double-logistic functions withone time-varying parameter (DL1), and time-varying, age-specificselectivity parameters (ASP). The model-selection methods includeRMSE, DIC, and DR. The bars represent MREs. The boxes andwhiskers represent 25th and 75th, and 10th and 90th percentiles ofthe distributions, respectively.


at Michigan State U



Dow

nloaded from


Wilberg and Bence, 2008). It is basically an overall assessment ofmodel fit and of whether the data support models with more orfewer parameters. The better performance of model selection byDR implies that one should favour models that produce less-severesystematic retrospective patterns even when this leads to overallpoorer model fits to data and assessment models with apparentlybetter predictive ability. The robustness of the DR for selectingamong time-varying selectivity estimation models probably isbecause it can detect consistent patterns in model estimates overtime, i.e. as new years of data are added to the model. As expected,these consistent or retrospective patterns appear to be indicative ofan estimation model that is having difficulty in appropriately esti-mating time-varying selectivity. The DIC and the RMSE lack thisability to detect the retrospective patterns, because they merelyevaluate the model fit to the complete time-series of observeddata. Parma (1993) developed an alternative metric for identifyingthe retrospective patterns using the square root of the mean squareerror between the retrospective estimate of a model quantity and acorresponding reference estimate on the log scale. Mohn (1999)points out that this mean square error metric is unable to differ-entiate between retrospective and random patterns because ituses a mean square, rather than signed sum, in its calculation.Although Parma’s (1993) metric was not used here, it is suspectedthat it would perform similar to the RMSE method.

Here, we looked only at the performance of model-selectionmethods on an individual basis. The ability to select the best esti-mation model might be improved by using combinations of differ-ent selection techniques. For example, estimation models can beranked based on their DR. If multiple estimation models haveequal or nearly equal values of the DR, then DIC or RMSEvalues could be used to select between them. Wilberg and Bence(2008) found that model selection by DIC performed well forselecting models for time-varying fishery catchability. Furtherstudy using such multiple model-selection methods, particularlywhen there are alternative submodels for several aspects of theassessment model, would be informative.

AcknowledgementsWe thank D. Hayes, M. Jones, and G. Rosa of Michigan StateUniversity, C. Porch and A. Chester of the Southeast FisheriesScience Center, and the editor and two anonymous reviewers fortheir valued comments. The manuscript is publication 2010-09of the Quantitative Fisheries Center at Michigan StateUniversity. The project was funded through Studies 230713 and236102 of the Michigan Department of Natural Resources, withfunding from US Fish and Wildlife Service sportfish restorationprogram project F-80-R. The research was completed in partialfulfilment by BCL of the requirements for the degree of Doctorof Philosophy at Michigan State University.

ReferencesAkaike, H. 1973. Information theory as an extension of the maximum

likelihood principle. In Second International Symposium onInformation Theory, pp. 267–281. Ed. by B. N. Petrov, and F.Csaki. Akademiai Kiado, Budapest.

Bence, J. R., Gordoa, A., and Hightower, J. E. 1993. The influence ofage-selective surveys on the reliability of stock synthesis assess-ments. Canadian Journal of Fisheries and Aquatic Sciences, 50:827–840.

Bence, J. R., and Hightower, J. E. 1993. Status of Bocaccio in theConception/Monterey/Eureka INPFC areas in 1992 and rec-ommendations for management in 1993. In Status of the Pacific

Coast Groundfish Fishery Through 1992 and RecommendedAcceptable Biological Catches for 1993, Appendix B. PacificFishery Management Council, Portland, OR.

Berg, A., Meyer, R., and Yu, J. 2004. Deviance information criterion forcomparing stochastic volatility models. Journal of Business andEconomic Statistics, 22: 107–120.

Burnham, K. P., and Anderson, D. R. 2002. Model Selection andMultimodel Inference: a Practical Information-TheoreticApproach. Springer, New York. 496 pp.

Butterworth, D. S., Ianelli, J. N., and Hilborn, R. 2003. A statisticalmodel for stock assessment of southern bluefin tuna with temporalchanges in selectivity. African Journal of Marine Science, 25:331–361.

Cardoso, F. F., and Tempelman, R. J. 2003. Bayesian inference ongenetic merit under uncertain paternity. Genetics SelectionEvolution, 35: 469–487.

Deriso, R. B., Quinn, T. J., and Neal, P. R. 1985. Catch-at-age analysiswith auxiliary information. Canadian Journal of Fisheries andAquatic Sciences, 42: 815–824.

Doubleday, W. G. 1976. A least squares approach to analyzing catch atage data. Research Bulletin International Commission for theNorthwest Atlantic Fisheries, 12: 69–81.

Ebener, M. P., Bence, J. R., Newman, K., and Schneeberger, P. 2005.Application of statistical catch-at-age models to assess lake white-fish stocks in the 1836 treaty-ceded waters of the upper GreatLakes. In Proceedings of a Workshop on the Dynamics of LakeWhitefish (Coregonus clupeaformis) and the Amphipod Diporeiaspp. in the Great Lakes, pp. 271–309. Ed. by L. C. Mohr, andT. F. Nalepa. Great Lakes Fishery Commission Technical Report, 66.

Fournier, D. 1983. An analysis of the Hecate Strait Pacific cod fisheryusing an age-structured model incorporating density-dependenteffects. Canadian Journal of Fisheries and Aquatic Sciences, 40:1233–1243.

Fournier, D., and Archibald, C. P. 1982. A general theory for analyzingcatch at age data. Canadian Journal of Fisheries and AquaticSciences, 39: 1195–1207.

Gelman, A., Carlin, J. B., Stern, H. S., and Rubin, D. B. 2004. BayesianData Analysis, 2nd edn. Chapman and Hall, Boca Raton, FL.695 pp.

Gudmundsson, G. 1994. Time series analysis of catch-at-age obser-vations. Applied Statistics, 43: 117–126.

Ianelli, J. 1996. An alternative stock assessment model of the easternBering Sea pollock fishery. In Stock Assessment and FisheryEvaluation Report for the Groundfish Resources of the BeringSea/Aleutian Islands Regions, November 1996, pp. 81–104.North Pacific Fisheries Management Council, Anchorage, AL.

Kimura, D. K. 1990. Approaches to age-structured separable sequen-tial population analysis. Canadian Journal of Fisheries andAquatic Sciences, 47: 2364–2374.

Linton, B. C., and Bence, J. R. 2008. Evaluating methods for estimatingprocess and observation error variances in statistical catch-at-ageanalysis. Fisheries Research, 94: 26–35.

Methot, R. D. 1990. Synthesis model: an adaptable framework foranalysis of diverse stock assessment data. In Proceedings of theSymposium on Applications of Stock Assessment Techniques toGadids, pp. 259–277. Ed. by L. Low. Bulletin of the InternationalNorth Pacific Fisheries Commission, 50.

Millar, R. B. 1995. The functional form of hook and gillnet selectioncurves cannot be determined from comparative catch data alone.Canadian Journal of Fisheries and Aquatic Sciences, 52: 883–891.

Mohn, R. 1999. The retrospective problem in sequential populationanalysis: an investigation using cod fishery and simulated data.ICES Journal of Marine Science, 56: 473–488.

Myers, R. A., and Quinn, T. J. 2002. Estimating and testing non-additivity in fishing mortality: implications for detecting a fisheriescollapse. Canadian Journal of Fisheries and Aquatic Sciences, 59:597–601.


at Michigan State U



Dow

nloaded from


Otter Research Limited. 2004. An introduction to AD Model Builderversion 7.1.1: for use in nonlinear modeling and statistics. OtterResearch Limited, Sidney, BC, Canada.

Parma, A. N. 1993. Retrospective catch-at-age analysis of pacifichalibut: implications on assessment of harvesting policies. InProceedings of the International Symposium on ManagementStrategies for Exploited Fish Populations, pp. 247–265. Ed. byG. Kruse, D. M. Eggers, C. Pautzke, R. J. Marasco, and T. J. Quinn.Alaska Sea Grant College Program.

Punt, A. E., Smith, A. D. M., and Cui, G. 2002. Evaluation of manage-ment tools for Australia’s South East Fishery. 2. How well can man-agement quantities be estimated? Marine and Freshwater Research,53: 631–644.

Punt, A. E., Smith, D. C., Thomson, R. B., Haddon, M., He, X., andLyle, J. M. 2001. Stock assessment of the blue grenadierMacruronus novaezelandiae resource off south-eastern Australia.Marine and Freshwater Research, 52: 701–717.

Radomski, P., Bence, J. R., and Quinn, T. J. 2005. Comparison ofvirtual population analysis and statistical kill-at-age analysis for arecreational kill-dominated fishery. Canadian Journal of Fisheriesand Aquatic Sciences, 62: 436–452.

Schwartz, G. 1978. Estimating the dimension of a model. Annals ofStatistics, 6: 461–464.

Spiegelhalter, D. J., Best, N. G., Carlin, B. P., and van der Linde, A.2002. Bayesian measures of model complexity and fit. Journal ofthe Royal Statistical Society, Series B, 63: 583–639.

Thiebauz, H. J., and Zwiers, F. W. 1984. The interpretation and esti-mation of effective sample size. Journal of Climate and AppliedMeteorology, 23: 800–811.

Thompson, G. G. 1994. Confounding of gear selectivity and thenatural mortality rate in cases where the former is a nonmonotonefunction of age. Canadian Journal of Fisheries and AquaticSciences, 51: 2654–2664.

Ward, E. J. 2008. A review and comparison of four commonly usedBayesian and maximum likelihood model selection tools.Ecological Modelling, 211: 1–10.

Wilberg, M. J., and Bence, J. R. 2006. Performance of time-varyingcatchability estimators in statistical catch-at-age analysis.Canadian Journal of Fisheries and Aquatic Sciences, 63:2275–2285.

Wilberg, M. J., and Bence, J. R. 2008. Performance of deviance infor-mation criterion model selection in statistical catch-at-age analysis.Fisheries Research, 93: 212–221.

Yin, Y., and Sampson, D. B. 2004. Bias and precision of estimates froman age-structured stock assessment program in relation to stockand data characteristics. North American Journal of FisheriesManagement, 24: 865–879.


at Michigan State U



Dow

nloaded from


Documents

Catch-at-age assessment in the face of time-varying ... · Statistical catch-at-age analysis (SCAA) is a common method of ﬁsheries stock assessment. Age-structured catch data from