Template for modules of the revised handbookec.europa.eu/eurostat/cros/system/files/20.02 Quality... · Web viewFrom the study by Hulliger and Münnich (2006) variance estimators

Theme Variance estimation methods

0 General information

01 Module code

Theme- variance estimation methods

02 Version history

Version Date Description of changes Author Institute10 28-02-2012 First version Andrzej Młodak GUS (PL)20 13-06-2012 Second version Andrzej Młodak GUS (PL)30 13ndash08ndash2012 Third version Andrzej Młodak GUS (PL)

03 Template version and print date

Template version used 10 p 3 dd 28-6-2011

Print date 18-5-2023 1522

Contents

General description ndash Theme 3

1 Summary 3

2 General description 4

21 Sources of variability of an estimator 4

22 General methods for variance estimation 11

23 Variance estimation in the case of unit nonndashresponse or item nonndashresponse 19

24 Domains and classes 23

25 Surveys over time 24

3 Design issues 26

4 Available software tools 27

5 Decision tree of methods 28

6 Glossary 29

7 Literature 29

Specific description ndash Theme 34

Interconnections with other modules 34

Related themes described in other modules 34

Methods explicitly referred to in this module 34

Mathematical techniques explicitly referred to in this module 34

GSBPM phases explicitly referred to in this module 34

Tools explicitly referred to in this module 34

Process steps explicitly referred to in this module 34

1 Summary 3

23 Variance estimation in the case of imputation 17

3 Design issues 22

6 Glossary 24

7 Literature 25

General description ndash Theme1

1 I would like to express my great gratitute to Mr Paolo Righi (ISTAT Italy) and Mr Ioannis Nikolaidis (ElndashStat Greece) for very valuable comments and suggestions

1 Summary

In this module we describe the main methods of variance estimation which are necessary to properly assess the quality of obtained results (including estimates of total values for the population) Of course the choice of methods of variance estimation should depend on the complexity of designs and diversity and distribution of statistics to be collected Correct procedures for computing sampling er-rors and variance estimation have to meet the following basic requirements (UNSD(1993))

the variance estimation procedure must take into account the currently used structure of the design which can be complex

the procedures should be as general as possible and simultaneously flexible that is they should be theoretically applicable to various survey designs

the the procedure should be convenient and have optimized costs especially for large scale applications ie to produce results for diverse variables type of statistics and subclasses in large complex surveys

some basic assumptions about the nature of the sample design required for the procedure of computation should not be too restrictive

the method should be economical in terms of the effort and cost involved including technical as well as computer resources

the procedure should have desirable statistical properties such as small variance of the gener-ated variance estimates small bias andor mean square error and accuracy in the probability levels with which the estimated confidence levels actually cover the population parameters of interest

suitable and efficient computer software should be available This may be both individual specially constructed for a given survey or institution and generally available with a possibil-ity of adjusting it to individual needs It should have the necessary capacity to process large data sets and complex (including iterative) algorithms in reasonable time

The complexity of a variance estimation method results from two aspects ndash the complexity of the statistic under study (linear statistics non-linear but smooth statistics and nonndashsmooth statistics) and the complexity of the sampling design Variance estimation methods can be classified according to their fitness to deal with complex statistics and complex sampling designs as follows2

methods that can be used for complex statistics eg Taylor linearisation methods that can be used for complex designs eg jackknife method methods that can be used both for complex statistics and complex sampling designs eg jack-

knife method

2 According to our knowledgeAs far as we know this division was not formally expressed but is is clearly considerableevident when analysing the scientifics literature (cf eg Deville (1999) and Lee (1973)) and was proposed by Mr I Nikolaidis

The importance of variance estimation was also stressed in EU regulations (cf Eurostat (2009)) Tra-ditionally and formally variance is defined as the expected value of the squared deviation of the value of a given statistics from its expectations (Var ( X )=E ( XminusEX )2) If statistics X is an estimator of the

parameter θ then its total variance is usually assessed usually using the Mean Square Error (()()( )() ie it is athe sum of the variance of the statistics and its squared bias) MSE reflects the complexity of the problem of variance and its significance The larger is the variance of the estimator the worse itsis ist quality because the expected error of estimation is larger In classical theory the op-timum estimator should be unbiased (ie EX=θ) and should have minimum MSE This fact follows from the key role of variance as a basic statistics describing the diversification of estimation results of estimation and its function of both the form of the estimator statistics and the nature of the sampling design These properties are visible when the probabilistic background opf the problem will be is repaklaced by estimators of MSE Var or bias constructed using the observations for sampled units Wolter (2007) notes correctly that an estimator of variance must take account of both the estimator and the sampling design and presents various forms of variance estimators for various types of estimators of population parameters His discussion is concentratedfocuses on nonndashstandard situations ie omit-tingwithout the assumption onf unbiasedness of estimators or resignation from without the minimiza-tion of MSE and instead taking into account the sufficient flexibility to accommodate most features of a complex survey

Now let us concentrate on factors influencing variance estimation and their impact on the final results of such assessment (eg type of sampling design type of estimation various errors etc) and analyze variance estimation methods for all sampling designs and all types of statistics (eg analytical or repli-cation methods models for systematic and double sampling imputation variance etc) We will also consider variance estimation for various types of domains and classes and for surveys over time (in the context of annual averages and indices)

2 General description

21 Sources of variability of an estimator

The main factors affecting the level of variance estimation are sampling design estimator type type of nonndashresponse corrections (for unit nonndashresponse and item nonndashresponse cases) frame errors as well as measurement processing and substitution errors Of course effects of particular factors may vary ndash some can be stronger others weaker Therefore one should observe how this influence is realized and to which extent it affects the quality of statistical research

Sampling design

The first source of variability of an estimator comes from the procedure used in selecting the sample (which is commonly called sampling design) The variability caused by observing a sample instead of the whole population is called sampling error The role of the sampling design in this context is rather complex Salganik (2006) points out that one common method of measuring estimation precision is to determine a confidence interval that provides a range within which the researcher expects to find the true population value with some level of certainty In the case of simple random sampling procedures to generate confidence intervals are well developed One of the main statistics used to obtain them is the estimated variance (cf Thompson 2002) A problem occurs when other designs are used eg complex sample designs (probably with strata levels etc) where not all units have the same proba-bility of selection Of course one can ignore the fact that their data were collected with another sample design and construct confidence intervals as if they had a simple random sample Salganik (2006) calls this approach the naive method it may produce confidence intervals that are too small and in other situations ndash too large Therefore final inference from such a survey can be flawed or even completely wrong The impact of design effects on estimation quality is usually measured by a ratio of variations of estimation obtained using a comparable method The commonly used basis of such a comparison is simple random sampling (SRS) That is we define

deff ≝ Var (M θ )Var (SRS θ )

where M is a method to be assessed and θ is the estimate of a given population parameter θ The greater the effects are the worse the analyzed method is The term lsquodesign effectrsquo is also alternatively defined using relevant standard errors and in this case it is denoted by iquest For example Salganik (2006) considers an original respondent-driven method (based on the idea of beginning the sampling process with the selection of a set of respondents called seeds after participating in the study these seeds are provided with a fixed number of unique recruitment documents which are used to recruit other re-spondents they know to participate in this survey) supported by a relevant bootstrap procedure (by dividing sample members into two sets based on how they are were recruited drawing the seed ndash us -ing the uniform distribution ndash and drawing new sampled units on the basis of the membership of the seed this sample is called bootstrap sample sampling is repeated until the bootstrap sample is the same size as the original sample) for which deff =2 and the variation can be greater than in eg SRS but this method is very useful in special cases of surveys where more classical attempts cannot be applied (eg in studies of the non-observed economy or other data which economic entities are unwill-ing to reveal) However in business statistics if we are interested in a good spatial representation of the sample (eg if we are going to analyze the concentration of a given type of economic activity in various areas and its impact on the region where these enterprises operate) it would be better to use some stratification depending on location size and the prevailing type of activity Obtaining informa-tion about sub-groups or strata In his detailed review and discussion on stratification (among others in business statistics) Hayward (2010) notes that this type of sampling design is often a convenient way to obtain information about subgroups as well as the overall population of interest ensures the representation of subgroups within a sample (ie that there is a similar representation of groups in a population within the sample) exploits some administrative convenience (eg proximity to branches or respondents that results in benefits from a stratification by the likes of geographic area to minimize the overall cost of conducting a survey) and improves the accuracy of the overall estimates (by con-structing homogenous sub-populations to minimize variance within groups) So stratification can extend our knowledge onf structures of a given phenomenaon Of course the strata should be of suffi-ciently large size to be relevantly represented in the sample Otherwise other sample designs have to be used which can provide less comprehensive data sometimes even of worse quality (especially at lower levels of aggregations)

To assess the efficiency of a given methodology of sample survey we have to evaluate the estimation quality which is mainly deciding on affects such efficiency As we have noted in the summary of this module the basic measure of estimation quality are bias and variance The former allows one to assess the expected deviation of an estimate of a given parameter from its true value The latter is used to analysisze whether the estimate is as much adjusted to the structure of values of the relevant variable for the analyzed units The smaller is the variance the better is the quality of thea given estimator Therse two aspects are in some sense combined in the coefficient of variation which is athe ratio of standard deviation to the mean value of the estimator Sometimes the collection of methods of estima-tion quality assessment also involvescludes also the consistency of the analyzed estimator ie verifi-cation whether it converges in probability to the true value of the parameter to be estimated

Let us now indicate some advantages and drawbacks of other sampling schemes from the point of view of estimation quality Systematic sampling (hyperlink) (see the chapter devoted to sample selec-tion) is a very comfortable method which minimizes the effort involved in conducting the survey and gives a better precision of results by exploiting hidden stratification of the frame However it provides no unbiased estimator of the variance of analyzed population statistics So we have to resort to some biased estimators On the other hand if the distribution of a given phenomenon in a given frame is cyclic then this method is inappropriate So the effect of this type of sampling is often ignored and formulas for SRS are used instead One can also make pseudondashstratification of the sample (sometimes the strata are in practice not identified as they actually were actually established in the sampling plan and hence a modifications in defining strata for variance estimation may be necessary to make the sampling plan actually used fit into one of the other allowable ndash eg in used the software used ndash sam-pling plan options such actiona solution is called pseudondashstratification) ie the systematic sample is here regarded as stratified with two-element strata) or use bootstrap techniques and then the primary error component resulting from SRS turns out to be negligible If systematic sampling within strata is more precise than simple random sampling within strata then this method is much more efficient than SRS The authors of the handbook published by Eurostat (2002 a) argue that since no unbiased vari-ance estimation exists for this design the simplifying assumption of simple random (or stratified in this case) sampling may be adopted as long as the ordering of the sampling units before the systematic selection has been performed in such a way so as to lead to heterogeneous samples (as is usually the case) This restriction is imposed in order to prevent an underestimation of the variance However a more close approximation of the underlying sampling design can be achieved under the conceptual construction of a stratified two-stage clustered sampling In this case the variance of a total can be estimated via the Jackknife linearisation method (Holmes and Skinner 2000)rdquo To obtain a precise assessment of the sampling design effect one can also use multistage stratification or clustering of units with weighting adjustment (eg by jackknife methods) Probability proportionalndashtondashsize sample designs are rather complicated procedures The secondndashorder inclusion probabilities are sophisticated and therefore variance estimation is also not simple This problem is often solved by relevant approxi-mations derived from corresponding simplifications in the sampling schemes (Saringrndal et al 1992) so that one does not need to estimate second order probabilities However when using these methods we should also consider the increase in bias and bias components in variance estimation

Besides the sampling scheme the usefulness of these and other sampling designs depends on several various factors connected with the technical realization of sampling They are connected with the fol-lowing aspects (cf Eurostat (2002 a))

The greater the number of stages of the sampling the greater the variability of final estimates In one-stage sample designs the quality of variance estimation depends in general only on the sampling scheme used and ndash if applicable ndash on stratification or clustering In the case of more complex sampling designs (ie with several stages) there are many sources of variation In each stage the sampling of units induces an additional specific component of variability It can be assessed either by computing variance at each stage or by estimating the variability among primary sampling units because this is most often the dominant component of total variance For example if we implement a survey where sampling is conducted in two stages ie at the first level relevant spatial units are sampled next eco-nomic entities are drawn from sampled areas total variance can be the effect of variation at the first and second stage of the procedure So we can decompose the total variance into these two compo-nents The final level of variation also depends of course on the sampling scheme used at each stage of the survey

The use of stratification of sampling units leads to the reduction of total variance (in this case the weighted variance of each stratum) since it improves sample representativeness It is possible be-cause strata usually define homogenous subpopulations However the sampling design within each stratum could be different from other strata Eurostat (2002 a) observed that this independence among-between samples in different strata implies that any estimator as well as its corresponding variance estimator is simply the sum of the corresponding estimators within each stratum So the problem of finding the most appropriate variance estimator for a singlendashstage stratified sampling reduces toboils down to the problem of the most appropriate variance estimator for the sampling designs deemployed in each stratum ICES (2010) notes that stratification often increases survey precision for a given sam-pling effort and also ensures that precise estimates can be obtained for selected subpopulations The stratification designed before data collections enables one to control the number of samples in particu-lar domains

According to this paper a stratified sample is obtained by taking samples of a predetermined size from each stratum or subndashgroup of the population Frequently samples are allocated to strata in proportion to some stratum attributes of the strata Hence the sampling design may be the same in each strataum and in such a situation this allocation is the main source of variance One approach consists in an allo-cation of ng samples to strata in proportion to strataum sizes If the allocation is properly established (ie a heterogeneous population is divided into subpopulations each of which is internally homoge-nous) stratified sampling may produce a gain in precision for estimates ThereforeIn general a total estimate is the sum of estimates obtained within particular strata So to optimize variance estimation (ie to obtain sampling allocation minimizing the variance) it is sufficient and necessary to optimize estimators in each stratum The optimization problem is important becausesince the main goal of each statistician is to create a survey such methodology of survey which enables them to maximize the quality of estimation (expressed here by the minimization of the variance of the used estimator used) and minimize its cost

The use of clustering of sampling units plays an important role in variance estimation Clustering is usually conducted to reduce the cost of survey sampling The optimum clustering usually gives results in a division of the set of units into internally homogeneous and mutually heterogeneous disjoint non-empty subsets and therefore each cluster can be considered a sampling unit (instead of smaller units which it contains) The cost of such a survey is essentially lower but its variance usually increases In business statistics clusters may be generated on the basis of a set of several variables describing the similarity of physical location type of activity or production structure of employees structure of KAULKAU ( Kind of Activity Unit Local Kind of Activity Unit) units ( if an enterprise consists of many units spread across a large area ndash eg the country or even EU) etc Clustering can be useful in several stage samplings where units at a higher (eg secondary) stage can be grouped to simplify the sampling procedure The internal homogeneity of clusters usually leads to an increase in total variance (because units within the same cluster are usually highly correlated with one another) in comparison with simple random sampling As the authors of the handbook by Eurostat (2002 a) point out in clus-tered samples variance consists of two components variance within clusters (which depends on the intra-correlation between elements) and variance among clusters Therefore total variation depends on both these factors One can estimate these two components either by analytical methods (especially for simple sampling) or by applying relevant re-sampling techniques

Type of estimator used

The estimator type also has an impact on variance For example calibration estimators are known to be generally more accurate than lsquouncalibratedrsquo estimators A lot of arguments for calibration are pro -vided by Market Torrent (2012) One of the most important is that very few individuals (peoples units etc) are naturally calibrated estimators That is the declared uncertainty about a given piece of data (resulting from various causes eg from sampling or systematic errors) can differ from its true value Many studies showed that almost every possible respondent tends to be biased either towards ldquooverconfidencerdquo or ldquounderconfidencerdquo about the estimates Hence the calibration of weights im-proves significantly the quality of estimates by reducing variance For details see the module devoted to weight calibration in sample surveys (chapter ldquoWeighting and estimationrdquo)

As regards the impact of the estimator type on the variance of estimates Eurostat (2002 b) argues that using one type of estimator (most often HorvitzndashThompson) to calculate total estimates and another one (like GREG) for variance estimation is inappropriate In this situation GREG can yield lower variance and therefore the assessment of estimation precision can be distorted Consequently if coef-ficients of variation of GREG total estimators are calculated then total estimates have to correspond to the same GREG total estimator For more information about properties of these estimators see chapter ldquoWeighting and estimationrdquo

Of course in most surveys the choice of estimator to be used is made arbitrarily in advance ie be -fore starting the processing of individual data Riviegravere and Depoutot (2010) observe that even if in a business survey a completely enumerated (also called take-all) stratum occurs where all units belong-ing to such a stratum take part in the survey (eg a survey of large enterprises) then the estimation of variance is very sensitive to the chosen method of estimation and missing data They suggest comput -ing not the variance of the total but the variance of the total error This solution may be desirable eg when outliers can be detected various additional robustification methods are used to improve the pre-cision of estimation etc This method can be motivated by special treatment and processing of nonndashre-spondents changing criteria for deciding where individual data can be regarded as outliers adaptation of processing outliers postndashstratification and many other arguments

Another example of the problem of choosing the estimator from the point of view of variance is the use of totals (or rather data expressed in absolute values eg revenues in EUR profit in EUR num-ber of employees etc) which may strongly distort variance estimation That is among units of inter-est there can be a few such that ndash for natural reasons (size rank on the market etc) are greater than others For them totals will be much greater than for others If these units are included in the sample variance can be underestimated otherwise ndash it can be overestimated Therefore it is better to use rela-tive values at least as the basis for sampling (eg wages and salaries per employee change of finan-cial result average revenue per employee share in production of given goods etc) From the point of view of the construction of statistics the mean is very sensitive to outliers and should therefore be used carefully especially if the size and other basic statistical features of the population are unknown If the number of outliers is not extremely small and they are recognized one can adjust the sampling design to obtain a desired level of representativeness in the sample On the other hand the median reduces the unfavorable impact of outliers and can be described as lsquothe voice of majorityrsquo of observa -tions Then it seems to be better for large surveys Moreover the order equivalent of standard devia-tion is the median absolute deviation which is the median of the absolute deviations of values of a given variable from its median )

Nonndashresponse unit

Problems with variance can also result from the occurrence of nonndashresponse units ie units which were selected to the sample but have returned no data (for more details see module ldquoResponse bur-denrdquo) More formally let us recall that variability comes from the fact that we have a subset of re-spondents selected as a subset from the sample with the conditional probability The variance of the estimator increases because the size of the subset of respondents is smaller compared to the size of the original sample

Nonndashresponse units are treated by weighting adjustment Sampling weights (the inverse of inclusion probabilities) are corrected to account for the unit nonndashresponse The most frequent compensation method used to assess the negative effects of unit nonndashresponse is weighting adjustment where re-sponding units have their weights increased to account for the loss of sample units due to non-re-sponse But to make such adjustments we should have some information on the nonndashresponse unit which can approach its importance in the sample For this purpose we can use specially collected basic variables Their data can be found in business registers or other administrative sources The use of auxiliary variables can be realized in two possible ways The first involves calibration and after its final step adjusting weights by special coefficients established using external variables (eg multiply-ing the weights for nonndashresponse units by their share of employment) Another possibility is a unique calibration step including nonndashresponse correction Eurostat (2002 a) indicated that to produce high-quality estimates for business statistics the use of ratio combined ratio or regression is recommended for which ndash even if they are biased ndash the bias is usually very small The authors of this document argue that the use of these calibration estimators may cause difficulties in statistics production as business surveys are multipurpose and multivariate and as a result modelndashbased estimators may be suitable for some statistics but not for others The final quality of estimation usually depends on the type of calibration For example since sampling weights lead to unbiased estimators in multistage sampling designs of weights should be respectively adjusted at each stage of sampling to account for the sam-pling of higherndashlevel units Final estimates should be asymptotically unbiased

Item nonndashresponse

Another practical problem is item nonndashresponse that is failure to collect information on certain items only Item nonndashresponse is usually treated by imputation In this situation imputation seems to be an efficient solution because the units with nonndashresponse items have provided some information that may be used to guide imputation and thus reduce bias (see Kalton (1983 and 1986)) One common source of error in variance calculations is to the tendency to treat imputed values as exact values

In the case of nonndashresponse items sampling variance can increase (due to the reduction of sample size in relation to the planned one) and ndash if some outliers occur ndash the estimator can be seriously biased Imputation can reduce these inconveniences A review of possible imputation techniques is provided in the relevant chapter of this handbook Of course imputation can also produce some errors The problem is how to minimize their impact on final results That is we have to consider two components of total error ie the ordinary sampling error and the imputation error In fact as mentioned in Kovar and Whitridge (1995) even nonndashresponse as low as 5 can lead to an underestimation of variance of the order of 2ndash10 while nonndashresponse rate of 30 may lead to 10ndash50 underestimation So taking these factors into account improves the estimation of total variance and exploits the properties of ap-plied imputation procedures Of course imputation methods used and their structure as well as signifi -cant sampling fractions should be taken into considerations

Coverage (frame) errors

Frame imperfections are another potential source of variability in estimates Over-ndashcoverage gener-ally increases variance because it results in a reduced sample (elements which do not belong to the target population are wasted) compared to what would have been obtained under no overndash-coverage Missclassification may be caused e by the initiala wrong Neyman allocation (a stratification which minimizes the sampling variance of the stratified sample when the sample size is fixed ndash see eg Cochran (1977) or Najmusseharl and Ahsan (2005)) and as a result of whichthe variance increases

It is obvious that tThere are two main types of deviations between the frame population and the target population (cf Bergdahl et al (2001))

under-coverage units belonging to the target population but not included in the frame popula-tion

over-coverage units included in the frame population but not belonging to the target popula-tion

These differences can occur at the level of the whole population or within particular subdomains One consequence of under-coverage is that observations about part of the target population are not col-lected This may cause a bias in resulting statistics and distorted approximations of variance The greater the number of non-covered units the lower the quality of estimation If we are not able to re-duce underndashcoverage by simple methods (eg direct contact with respondents or deduction) we should use imputation techniques taking into account possible errors they generate But this is not the only advanced solution to this problem Saumlrndal and Lundstroumlm (2005) also propose weighting through calibration as a new and powerful technique for surveys with nonndashresponse items They also try to combine weighting and imputation and discuss the use of imputation as a complement to weight-ing by calibration Over-coverage means inclusion of irrelevant units The estimation of variance could be inadequate (over-coverage can lead to more data gaps or too flat distributions of some vari-ables and then to bias and ndash in the latter case ndash also too low variance)

Measurement errors

Another important factor areisare the measurement errors Usually nonndashsampling errors of this type are dealt with under the heading of quality but we should remember that they have an important im-pact on variance and hence they cannot be omitted here (Grovens (2004) even thinks that ldquoThe the to-tal survey error approach attempts to acknowledge all sources of errors simultaneouslyrdquo including measurement errors) All these sources contribute to another component of variance resulting from deviations of observed values from true ones Four types of such errors are distinguished (cf Bergdahl et al (2001) or Groves (2004))

major occasional errors for continuous variables (eg reporting data per employee instead of per 100 employees reporting values of sold production in domestic currency instead of EUR etc) These errors are easily identifiable and can be easily corrected so they have little impact on variance

misreporting of zeros for continuous variables (reporting zero whereas the true value is nonzero eg wrong recording of revenue can lead to incorrect zero value in one item and non-zero in another) Such errors may lsquovanishrsquo when data are aggregated but at lower levels of information variance can be seriously distorted

other errors for continuous variables (guessing values and errors due to minor differences in reference periods) They are not large but can be modeled as deviations between reported and true values drawn from a continuous probability distribution Therefore the bias and variance can be modeled and estimated using such formula

misclassification for categorical variables ndash that is the wrong classification of categorty i in category jcan be measured by a misclassification matrix with elements the probability of classifying category as category The matrix can be easily used to estimate Groves shows a method to check the expected disturbances in variance in presence of these errors

In general the variance inflating impact of measurement error is likely to be most important for the largest businesses in completely enumerated strata Such businesses do not contribute at all to sam-pling variance but random errors in their reported values may have a significant impact on total vari -ance of survey estimates

When Ddescribing of the impact of the measurement error on the total variance of estimation we can-not omit the problem of its assessment which can be a key component of a variance estimate Bergdahl et al (2001)) proposes provide a formula enabling us to assess the variance impact of measurement error it In their approach the component of variance derived from measurement errors is given as

σ eh2 nh and in terms of expected value minusσ eh

2 Nh (Nh is the number of units in hndashth stratum nh -

number of sample units in this stratum) and σ eh2 is the variance of measurement error (according to the

model where the reported value Y is determined from the true value y by Y= y+e where e is the measurement error) If measurement errors are not independent the problem seems to be much more sophisticated (cf Henderson et al (2000) or Tsiatis and Davidian (2004))

Processing errors

As regards processing errors most of them are of the same nature as measurement errors and generate similar problems However the case of coding is especially interesting in this context Bergdahl et al (2001) observed that in manual coding and computer assisted coding different coders may allocate different codes to the same description and argued that ldquoIn particular each individual coder may un-consciously over-allocate businesses to some codes and under-allocate them to others This is known as correlated coder error The errors in the codes allocated by a particular coder may lead to bias in the estimate of the proportion of businesses in a given industry group for industries coded by that coder However since for many surveys coding is shared over a number of coders if the errors made by coders are different the impact of these individual biases on the final survey estimates may cancel out In this case although the final survey estimates may not be biased the variance of the estimates will be increased The overall bias is reduced as the number of different coders increases so in some surveys the code list is provided with or as part of the questionnaire so that each respondent codes their own answer This minimizes correlated coder error at the expense of a potential increase in mea-surement errorrdquo

Substitution errors

Substitution errors are also of the same nature as measurement errors Substitution errors are caused by substituting a unit in a sample with another unit considered asto be a good proxy for the original unit We should remember however that such a unit can fail to respond or can respond only to some items In the first case we have very restricted possibilities for improvement although the use of administra-tive data can slightly help in this task That is we can draw another unit from the same class indicated by these classification variables (eg the number of employees) Of course variance will be biased by a deviation of the value reported by the substituted unit from the unknown relevant value for the origi-nal unit A well prepared classification of units can reduce this problem

If we are only faced with the case of nonndashresponse item we can use some type of Gibbs sampling (Liu (1994)) Namely the new unit which is to replace the given one is sampled from the conditional distri-

bution P (x jorxminus j ) where x j is the variable for which data of a given unit are missing and xminus j refers to

all variables other than x j (assuming that data relating to the unit to be replaced are available) This technique makes it possible to retain sampling variance and sometimes even reduces it Processing errors

As regards processing errors most of them are of the same nature as measurement errors and generate similar problems However the case of coding is especially interesting in this context Bergdahl et al (2001) observed that in manual coding and computer assisted coding different coders may allocate different codes to the same description and argued that ldquoIn particular each individual coder may un-consciously over-allocate businesses to some codes and under-allocate them to others This is known as correlated coder error The errors in the codes allocated by a particular coder may lead to bias in the estimate of the proportion of businesses in a given industry group for industries coded by that coder However since for many surveys coding is shared over a number of coders if the errors made by coders are different the impact of these individual biases on the final survey estimates may cancel out In this case although the final survey estimates may not be biased the variance of the estimates will be increased The overall bias is reduced as the number of different coders increases so in some surveys the code list is provided with or as part of the questionnaire so that each respondent codes their own answer This minimizes correlated coder error at the expense of a potential increase in measurement errorrdquo

Substitution errors

Substitution errors are also of the same nature as measurement errors Substitution errors are caused by substituting a unit in a sample with another unit considered as a good proxy for the original unit We should remember however that such a unit can fail to respond or can respond only to some items In the first case we have very restricted possibilities for improvement although the use of administrative data can slightly help in this task That is we can draw another unit from the same class indicated by these classification variables (eg the number of employees) Of course variance will be biased by a deviation of the value reported by the substituted unit from the unknown relevant value for the original unit A well prepared classification of units can reduce this problem

If we are only faced with the case of nonndashresponse item we can use some type of Gibbs sampling (Liu (1994)) Namely the new unit which is to replace the given one is sampled from the conditional distri-bution where is the variable for which data of a given unit are missing and refers to all variables other than (assuming that data relating to the unit to be replaced are available) This technique makes it possible to retain sampling variance and sometimes even reduces it

22 General methods for variance estimation

Variance estimation can be performed using one of many different methods Presented below are their most important categories At the end we concentrate on the problem of variance estimation taking nonndashsampling errors and their consequences into account

Analytical methods

Analytic methods provide direct variance estimators which aim at reflecting the main features of the sampling design (unequal weighting clustering stratification etc)

Unequal weighting occurs when inclusion probabilities of units in the sample vary There is a wide family of methods of sampling with unequal probabilities without replacement One example of such a method is unequal probability systematic sampling It is very efficient and applicable to any sample size Wolter (2007) discusses difficulties of this method from the point of view of variance estimation Most of them result from joint inclusion probabilities If they have zero values for certain pairs of units then they generate a bias On the other hand if they are unknown variance estimation is diffi-cult Therefore he proposes several types of variance estimators based on a special approximation of the joint inclusion probability Another estimator of variance is obtained by treating the sample as if it were a sample drawn using probability proportional to size sampling with replacement A third estima-tor is obtained by treating the sample as if units were selected from within each of equalndashsized strata Wolter (2007) gives the formula for variance estimation in this case and and its correction to increase the number of ldquodegrees of freedomrdquo which ndash as opposed to the former one ndash utilizes overlap-ping differences Other proposals are obtained by applying the random group principle (where the systematic sample is divided into systematic subsamples each of size where m n are integers or re-gression estimators Another universal proposal for various sampling designs (based on the calibration of weights and the calibrated GREG version for variance estimations) is given by Deville and Saumlrndal (1992)

Taylor linearization

The most popular analytical methods are aimed at finding a formula for an (at least approximately) unbiased estimator of sampling variance These formulas can be exact or approximate Exact formulas are connected mainly with linear estimators In nonndashlinear cases some more advanced methods are necessary It is good if such estimates can be linearized (otherwise the problem can be much more complicated ie the estimates ndash especially regression ndash will often have to be often approximated by iterative approximations using such methods as eg NewtonndashRaphson procedure such algoruithmsn are timendashconsuming and the obtained variance estimates could be much more inadequate due to grow-ing bias it is good to avoid these problems) Most of these approximate formulas are derived by means of the Taylor series linearization This method is a wellndashestablished to obtain variance estima-tors for nonndashlinear and differentiable statistics

There are two options of applying the Taylor series in variance estimation The first one is based on the classical Taylor series for a function of population parameters (cf Wolter ( 2007) Namely we consider a given finite population and let be a -dimensional vector of population parame-ters and let denote a corresponding vector of estimators based on sample of size Sup-pose that we want to estimate the population parameter by where is a fuction possessing contin-uous derivatives of order 2 in an open sphere containing and then the bias is given as a Taylor series

( ) ()

and the MSE

( ) ( () )(sum

())sum

Wolter (2007) also presents multivariate generalizations of this approach where instead of one function g we have many functions ie for q-dimensional apareter of our interest

()(( )( )( )) He discusses problems concerning the use of Taylor series and observes that even when convergence of the Taylor series is guaranteed for all possible samples the series may con-verge slowly for a substantial number of samples and first-order approximations discussed here may not be adequate It may be necessary to include additional terms in the Taylor series when approximat-ing the mean square error A special case of this method (1) is that if we assume that the parameter to be estimated is of the form

for some Theis version assumption (2) is especially useful if we would like to estimate the ratio pa-rameters such as eg number of employers per 1000 adult population average wage and salary per employee etc

The second technique option is applied eg for various types of ratio estimators and is based on pre-senting sample means as products of population means and a factor 1+e where e is a random term with zero mean and variance equal to a function of population variance of the relevant variable defined in such a way that it doesnrsquot exceed one An estimator presented in this form is expanded in the Taylor series Terms greater than two are neglected Hence we obtain an approximate linear form of the orig-inal estimator More details and analysis of special cases can be found eg in papers by Olufadi (2010) Singh et al (2008) Perri (2007) For example if we have a simple ratio estimator of the form t= y ( X x ) then defining y=Y (1+eY ) and x=X (1+eX ) (x and ydenote sample means for sam-ple of size n and X and Y population means for the population of size N respectively) where

E (e X )=E (eY )=0 E (eX2 )=( (1n )minus(1N ) ) (1(Nminus1))sum

(x iminusX )2

E (eY2 )=( (1n )minus(1N ) ) (1(Nminus1))sum

( y iminusY )2 and the covariance is given as

E (e X eY )=( (1n )minus(1N ) )radic(1 (Nminus1))sumi=1

(x iminusX )2X radic (1(Nminus1))sumi=1

( y iminusY )2Y we can transpose

the estimator to the form t=Y (1+eY )(1+eX)minus1 Then expanding the term (1+e X)

minus1 using the Tay-

lor series (and neglecting terms raised to the power greater than two) we can easily present this for-mula in linear form

Replication methods

A special group of estimation methods (including variance estimation) are re-sampling (or replication) methods Replication methods are based on repeatedly drawing sub-samples from a sample in order to build a sampling distribution of the statistic of interest and to estimate variance from the variability of estimates from subndashsamples They are aimedintended to measure the quality of the estimation and hence to support inits They are aimed at improving improvement the quality of estimationit On the other hand they are very easy from the computational point of view which is the main reason for their popularity We will briefly describe shortly their most important types but more information on these algorithms can be found in Shao and Tu (1995) and Wolter (2007) Recently some of these methods are also presented by Haziza (2010) who discusses their usefulness in the case of imputed data

Jackknife algorithmmethod It consists in omitting some groups of units from the sample The for-mal idea of the jackknife approach was presented by Wolter (2007) What follows is a brief overview Let be independent identically distributed random variables and be the population param-eter to be estimated in a sample survey using estimator Assume that the sample is divided into groups of observations each where and are all integers Hence Let be the estimator deter-

mined from the reduced sample of size obtained by omitting the -th group and define The most popular jackknife estimation tool baeses on the jackknife approach is the Quenouillersquos estimator which is the mean of the lsquotrimmedrsquo versions of the primary estimator of the parameter of interest ie

This estimator reduces the bias in comparison to (by removing some terms in rele-

vant expressions ndash see Wolter (2007)) In addition the Quenouillersquos estimator removes the bias for estimators that are quadratic functionals The jackknife estimator of variance is then of the form

This estimator also has very important asymptotic properties (cf Wolter (2007)) That is let μ be a point on real line (usually it is the common theoretical mean of the analyzed variables and the esti -mated parameter θ is given as θ=g (μ)) Iif is a function defined on the real line that has bounded

second derivatives in the neighborhood of then the statistics radic () converges in its distribution to a

normal random variable with mean zero and variance () Moreover its variance converges in

probability to ()These can be generalized in various ways Using the Quenouille formula one can construct unbiased estimators of the parameter Wolter (2007) observed that the jackknife method does work for the sample median if m is large enough As regards the choice of the number of groups that guarantee the satisfactory precision of estimators the commonly preferred choice is

In general the jackknife algorithm for eg simple random sampling with replacement consists of the following steps Firstly we remove the unit j=1from the sample next adjust design weights to obtain so-called jackknife weights which are usually equal to N (nminus1) for units other than j and 0 for the first unit compute the estimator using the adjusted weights instead of the design weights insert back unit i=1 which was previously deleted The algorithm is then repeated for i=23 hellip n A jackknife variance estimator of the mean of Y is then given by

V=( nminus1n )sum

( y(i)minus y )2()

where y(i ) is a jackknife estimator with jackknife weights when i-th units is removed from the sample Jackknife methods (with Taylor linearization) are often used in business statistics to estimate popula-tion covariance Full and Lewis (2011) observe that the jackknife method has the advantage of being more flexible in that it is relatively straightforward to adapt the formula to take into account imputa-tion etc and hence it is very useful in business surveys to estimate variance

Bootstrap method As we know a bootstrap sample (or bootstrap replicate) is a simple random sam-ple with replacement of size selected from the main sample (which can also be a superpopulation for this survey) Denoting bootstrap observations as

the estimator of variance is given as

( ) ie it is equal to conditional variance given the main sample Wolter (2007) de-scribes a three-step procedure to determine the variance of the bootstrap estimator if its exact formula is unknown

(i) draw a large number say of independent bootstrap replicates from the main sample and label the corresponding observations as

(ii) for each bootstrap replicate compute the corresponding estimator of the parameter of interest

(iii) calculate the variance between the values as

() sum

() where

Of course () if Wolter (2007) also discusses practical applications of these formulas and

algorithms to various types of random sampling

There are many types of the bootstrap method (Efron 1987) One of them is the RaondashWu rescaling bootstrap (Rao and Wu 1985) proposed estimating the variance of the estimator of means That is we draw bootstrap samples of size n0 (which may be different from n) with replacement from the rescaled sample s According to the description given by Haziza (2010) the rescaling factor denoted by C is chosen so that the variance under re-sampling matches the usual variance estimator of the population mean This approach may be described using following steps Firstly let nrsquobe the bootstrap sample

size z i= y+radicC ( y iminus y ) with C=n (1minus1048576(n N ))(nminus1) Next we draw a simple random sample

z iiquest i=1

n with replacement from z i i=1n and determine the arithmetic mean of its values This sampling is

repeated many (assume that m) times and each time such a mean is computed Finally variance is estimated as

V= 1mminus1sumi=1

( ziquest(i )minusziquest)2

where ziquest(i ) is the aritmetitc mean for units in the i-th bootstrap sample and ziquest is the total bootstrap

mean ie ziquest=sumi=1

ziquest(i) m Practical problems connected with applying these models in business statis-

tics are discussed by R Seljak (2006)

Some bBootstrap methods are more accurate when applied to business statistics than classical methods (cf Hesterberg et al (2003)) They are very useful eg in the situation when the distribution of a given feature in the population and in the sample is strongly skewed which affects the sampling distri-bution of the sample mean Thus only taking many samples and analyzing the bootstrap distribution can enable relatively efficient estimation of the sample distribution and indirectly ndash also the population distribution Many examples of the use of bootstrapping in business statistics (eg in surveys of telecommunication repair times or real estate sale prices) are given by Hesterberg et al (2003)

The balanced repeated replication (BRR) method requires that the full sample be drawn by using a stratified sample design with two primary sampling units (PSUs) per stratum Its exact description is given in the book by Wolter (2007) and what follows is a brief overview starting with the formal background of this approach Suppose that we have to estimate a population mean To do it we use a stratified design with two units selected per stratum where the selected units in each stratum comprise a simple random sample with replacement Let denote the number of strata be the num-

ber of units within the -th stratum and sum

be the size of the entire population Let and

denote values of the target variable for units selected from ndashth stratum The unbiased estimator of

is given as sum

with and ( ) Its variance can be estimated as

() Taking into account that in this case we have two independent random

groups () and () we have the random group estimation of variance in the form

() ( ) where sum

The BRR method is aimed at finding a way of esti-

mating variance with both the computational simplicity of () and the stability of () To meet this postulate subsamples (also called halfndashsamples) consisting of one unit sampled from each of the strata are investigated Keeping the assumption that for each stratum only two elements can be sam-pled we allow different halfndashsamples to contain some common units (and some different units) in a systematic manner Because of the overlapping units the halfndashsamples will be correlated with one another and thus this approach is sometimes called ldquopseudoreplicationrdquo Therefore the estimator of

the population mean from ndashth halfndashsample is given as sum

() where if unit hi is

selected to the ndashth halfndashsample and otherwise Wolter (2007) proved that the statistics

( ) is an unbiased estimator of variance of and that for better quality of estimation one can (without any loss of information) take the mean of such statistics over all halfndashsamples However owing to possible computational problems (we have totally halfndashsamples) the mean can be re-

stricted to simple random sample without replacement of halfndashsamples and ()sum

An algorithm for practical computation of variance for multistage generalization of the BRR using some concepts expressed by cf Lohr (1999) can be found in the

SAS 92 Userrsquos Guide3 formulates the following algorithm for multistage generalization of this method Let the number of replicates which are the smallest multiple of 4 that is greater than We determine the mtimes m Hadamard matrix4 of the sampling design If this isnrsquot possible the number of replicates is increased until the Hadamard matrix can be created Each replicate is obtained by deleting one PSU per stratum according to the corresponding Hadamard matrix and adjusting the original weights for the remaining PSUs The new weights are called replicate weights Replicates are con-structed by using the first hiquest columns of the mtimes m Hadamard matrix The rndashth (r=12 hellip m) repli-cate is drawn from the full sample according to the rndashth row of the Hadamard matrix as follows

If the r hiquestndashth element of the Hadamard matrix is 1 then the first PSU of stratum hiquest is included in the rndashth replicate and the second PSU of stratum is excluded

If the r hiquestndashth element of the Hadamard matrix is 1 then the second PSU of stratum hiquest is included in the rndashth replicate and the first PSU of stratum is excluded

The replicate weights of the remaining PSUs in each half sample are then doubled to their original weights (cf Lohr (1999)) Suppose that θ is a population parameter of interest and θ is the estimate

from the full sample for θ Let θr be the estimate from the rndashth replicate subsample by using replicate

weights Variance is estimated by

with degrees of freedom An additional problem in the use of replication methods are possible strong perturbations of weights used (which occur especially when estimating the ratio of small domain totals or in the case of poststratification or unit nonndashresponse adjustment involving ratio weighting within small cells) To overcome these inconveniences Rao and Shao (1999) analyze a modified balanced repeated replication method using a gentler perturbation of weights and smooth or nonndashsmooth esti-mators The method is extended to the case of imputation for item nonndashresponse A discussion on ap-plication of the BRR method in business statistics (in can be applied in designs where the size of the clusters varies) can be found in the paper by Bergdahl et al (1999)

The method of random groups method This attempt consists in drawing several samples from the population (using consistently the same sampling design) constructing separate estimates of the target population parameter of interest using data from each sample and an additional estimate on the basis of data obtained from the combination of all selected samples and computing the sample variance among the several estimates More formally (cf Wolter (2007)) one can describe this process as fol-lows a sample is drawn from the finite population according to a fixed sampling design without any restrictions Next the first sample is replaced into the population and a second sample is drawn according to the same sampling design This process is repeated until samples are drawn Each of them is selected according to the common sampling design and each is replaced before the selection of the next sample These samples are called random groups

3 SASSTAT(R) 92 Users Guide Second Edition SAS Institute Inc available in the Internat at the web-site httpsupportsascomdocumentationcdlenstatug63033HTMLdefaultviewerhtmstatug_surveymeans_a0000000225htm

4 A Hadamard matrix is an mtimes m matrix H with entries plusmn 1 such that H HT=n I

Denote by estimates of the target parameter for samples and ndash sum

its estimate for the whole combination The idea of variance estimation in this case is based on the fact that (cf Wolter (2007)) are treated as uncorrelated random variables with the same ex-

pected value and is unbiased estimator of Hence an unbiased estimator of variance of is given by

Wolter (2007) shows several ways of applying this formula for stratified and clustered samples and general results for sampling without replacement He also analyses a special case when a single pri -mary sampling unit (PSU) per stratum is selected as a result of which an unbiased estimator of vari -ance is not available The only rational solution (albeit with slightly overestimated variance) is the col-lapsed stratum estimator constructed under the assumption that one PSU is selected independently from each of the given strata and that any subsampling is independent from one primary to the next The estimator of the population mean is based on a combination of the strata into a number of groups of at least two strata belonging to each of them Wolter (2007) also discusses the problem of estimati-ing the coefficient of variation for variance estimation Assuming that are iid random variables this coefficient can be presented as

(())radic ()

( ) ( )

and hence efficient practical approximations for empirical data are possible

The problem may occur when the assumption of nonndashcorrelation of group estimates of the target pa-rameter is false Such a situation may be due to the presence of measurement errors different sets of actual interviewers in the successive subsamples use of different processing facilities and some post-stratification adjustments for various subsamples In this case estimation based on order statistics is recommended (see Wolter (2007)) West Kratzke and Robertson (1994) present an application of this type of sampling and variance estimation in comparison with other designs They have used employ-ment data from the Bureau of Labor Statistics Universe Data Base (UDB) in USA They constructed a micro data file using information obtained from quarterly UI reports (the Unemployment Insurance) which each employer is required to submit The authors obtained data from Michigan for the follow-ing industries (2-digit SIC code is in parenthesis) Agricultural Services (07) Lumber and Wood Prod-ucts (24) Transportation Equipment (37) Trucking and Warehousing (42) Transportation Services (47) General Merchandise Stores (53) Apparel and Accessory Stores (56) Miscellaneous Retail (59) Nondepository Credit Institutions (61) Miscellaneous Repair Services (76) Membership Organiza-tions (86) and Private Households (88) They estimated variances by standard jackknife and random groups method showing that random group methods is very efficient especially if used for data with imputed values (eg by mean imputation) This statement leads us to the problem of estimating vari -ance in the case of imputation which will be discussed in the next subsection

Treatment of nonndashsampling errors in variance estimation

The problem of how to treatment of nonndashsampling errors belongs tois one of the most important ques-tions from the point of view of variance estimation The proper treatment of non-sampling errors re-duces the variance of the estimates and in particular the bias Therefore they deserve to be presented separately

In general the model with nonndashsampling error terms contributesd additive components to the total mean square error of the estimate It is worth noting that in suchthis situation the MSE decreases in-versely proportionally to the number of interviewers coders analystticians etc rather than to the sample size Hence there is a serious threat of variance underestimation of variance when using classi-cal methods seriously exists

Therefore the variance estimation tools often contain terms connected with various types of nonndashsam-pling error The best presentation of such elements can be found in the paper by Hartley and Biemer (1978) They construct the following error models to enable an application of rdquoadaptive errors modelsrdquo where particular types of nonndashsampling errors are associated with relevant error terms That is we have

= error variable contributed by i-th interviewer common to all units t interviewed by i-th inter-viewer

= error variable contributed by c-th coder common to all units t coded by c-th coder

and = elementary interviewer coder and respondent errors afflicting the content item of unit t (respondent t)

It is assumed that the are random samples from infinite populations with zero mean and variances

and respectively and δ r t is a (non-observed) error with zero mean

sampled from the finite population of respondents by the implemented survey design

Hartley and Biemer (1978) investigate a threendashstage sampling design under the conditions that the sample contains at least two primaries per stratum two secondary units per primary two tertiary units per secondary etc in each primary there are at least two secondary units entirely interviewed by the same interviewer and coded by the same coder in at least one primary there are at least two secondary units entirely interviewed by different interviewers but coded by the same coder and finally that in at least one primary there are at least two secondary units entirely coded by different coders

Assuming that tertiary units within a particular secondary unit are handled by the same interviewer and the same coder (hence the elements indexed on t are neglected) they construct the model

where ηp ∙ ∙ is the average true values of target variable Y for primary unit counted by its averages by

secondary unit on their averages by relevant tertiary components δ ps ( denotes the average

true value for sndashth secondary unit belonging to pndashthe primary unit) and e ps ∙ is the sondashcalled average

pooled term ie e ps ∙=iquestsum

() ( pst denotes the tndashthe tertiary unit

of the ndashth secondary unit of the pndashth primary unit and ndashthe number of tertiary unit in sndashth sec-ondary unit of the pndashth primary unit) In a synthetic multivariate form this model can be written as

sum ()

where y η b c and δ p are the vectors of the terms in (4) and X Ub U c and W p are relevant design matrices

Hartley and Biemer (1978) implement the iterative procedure of determination of ing particular com-ponents It is based on updating of the component matrices starting from their following forms

) () ()

) () and using covariance matrices for the pooled terms for their suc-

cessive updating They also give also an adjustment of this algorithm to the design adopted for inter-viewers and coder allocation (where interviewers are associated tio primary units within a given stra-tum and coders are associated to tertiary units within secondary units) as well as provide formulas for estimated variances of the target parameter estimator

Skinner C (1999) discusses the problems and properties of calibration estimation in the presence of both nonndashresponse and measurement errors These ideas are illustrated with a simple example con-cerning the estimation of the number of sight empirical tests He proposed a modification of classical ration estimation under nonndashresponse the approximate expectations with respect to the response and sampling mechanisms and an analysis ofze its variance

23 Variance estimation in the case of imputationunit nonndashresponse ort item nonndashresponse

Estimation of variance can also be used to assess the quality of estimation of population statistics ob -tained by imputation in the case when nonndashresponse units (understood as units for which all data are unknown) or item nonndashresponse (data gaps in some items of the questionnaire) occur However usu-ally we have at our disposal some data on (sometimes restricted) number of auxiliary variables which can help us to efficiently reduce inconveniences resulteding from existing of data gaps We will present how the nonndashresponse units affect the variance estimation and how to assess its quality Let us start from the classical methods Deville and Saumlrndal (1992) remindedrecall that the quality of the HorvitzndashThompson estimator of total can be improved by choosing the weights to be that are as close as possible to the original weights in terms of some metric subject to the constraint that the weighted auxiliary variables match the marginal valuess One can use here the classical Euclidean formula or squared error metric or other distance measures suggested in this elaborationbook Its Aauthors observed that the calibration of weights also be used to allow for unit nonndashresponse Item nonndashre-sponse where the covariate is known but the target variable is missing requires an application of a different approach Usually it is the imputation based on available data and auxiliary information (see chapter ldquoImputationrdquo) Hampel et al (1986) show the utilitysefulness of use athe generalized linear model based on the vectors of auxiliary variables and its close connection here with M-estimation commonly used in robust statistics If the response is dindash or polychotomous it is natural to use logistic regression as the imputation model There existare also some more original proposals eg a more robust imputation model based on the Huberrsquos proposal (Huber (1981)) Many of these models are discussed in the chapter ldquoImputationrdquo of this handbook

Davison and Sardy (2007) discuss variance estimation in the case of missing data by resampling deriv-ing a formula for linearization in the case of calibrated estimation with deterministic regression impu-tation and compare the resulting variance estimates with balanced repeated replication with and with-out grouping the bootstrap the block jackknife and multiple imputation They also investigate the number of replications needed for reliable variance estimation under resampling in this context They conclude that most optimistic results (smaller bias and Mean Square Error) giveare obtained by apply-ing linearization the bootstrap and multiple imputation They remindpoint out that in such a situation classical variance formulae for sample survey estimators are derived using approximations based on Taylor series expansion of ethe estimators

As we have indicated in the relevant chapter of this handbook the main aim of imputation is to esti -mate descriptive aggregated (most often population) statistics Hence the variance estimation for im-putation has to be based on a squared deviation of an estimate forfrom its true value It can be esti-mated by analysis ofzing results of a successive replication of the imputation trial or by using available auxiliary data and records for which the information on the target variable is available Let θ be the parameter to be estimated and θ ndash its estimate obtained using the relevant estimator and imputed data

Let θk denote the estimate obtained using θ in the kndashth trial in the simulation study using imputed

data k=12 hellipq The basic measure of precision of estimation θ is the betweenndashtrial Mean Square Error (MSE) of θ defined as

MSE (θ )= 1q2sum

(θkminusθ )2 ()

It can be supplied with relative bias (RB)

RB ( θ )=(sumk=1

θk q)minusθ

Both formulas were used eg by Kim (2000) and Chauvet et al (2011) Arcaro and Yung (2001) also propose the empirical relative bias (ERB) which in some sense combines these two approaches (5) and (6)

ERB (θ )=sumk=1

q ∙ MSE (θ )minus1()

In all these cases the smaller the relevant index the better the quality of the estimator An estimator which minimizes all these three indices will be regarded as optimal

The most frequently estimated population parameters are mean and variance The well-known and new solution of the former case was presented in the modules devoted to relevant types of imputation Now let us analyze the latter case and present some estimators of variances

D Rubin (1978) demonstrates that the variance of θ=sumk=1

θk q can be estimated as

V=V W+q+1

qV B ()

where V W=sumk=1

(Var (θk)) q is the average withinndashimputation variance for particular trials and

V B=sumk=1

( θkminusθ )2 (qminus1) denotes the betweenndashimputation variance

Let θA be an estimator of θ computed using all sample data about the target variable Variance estima-

tion is strictly connected with θ In general Saringrndal (1992) showed that the total variance can be de-composed into sampling imputation and mixed effect components

V=V SAM+V IMP+2 V MIX ( )

where V SAM=E ( θAminusθ )2 V IMP=E ( θminusθ A )2 V MIX=E (( θAminusθ ) ( θminusθ A )) are the aforementioned com-

ponents respectively Of course in a simulation study these expected values can be approximated by arithmetic means of relevant deviations obtained by a consecutive replication of sampling Kim et al (2006) analyze the problem of variance estimation in the complex sample design when imputation is repeated q times and prove that the difference between the expected value of multiple imputation vari-

ance estimation and the variance of estimator θ can be expressed asymptotically as minus double the

expected value of conditional covariance of imputation effect and estimation parameter given avail -able data in the current sample They have investigated such models for subpopulations (also called also domains) and linear regression models Arcaro and Yung (2001) propose approximately unbiased

statistics for V SAM V IMP and V MIX using weighted mean and weighted ratio imputation with weights derived from traditional generalized regression estimator (GREG) for the mean based on relevant aux-iliary data Alternatively they have analyzed the variance estimator for weighted ratio imputation with

specially adjusted jackknife GREG weights as V=sumj=1

n jminus1 ( θ jminusθ )2 where θ j is the imputation esti-

mator of θ corrected using jackknife weights in the jndashth stratum n j is the number of units belonging to this stratum j=12 hellip h and hisinN is the number of strata in the sampling design Of course strata could be replaced by repetitions of simple random sampling in the simulation study Assuming that θ is unbiased Kim (2000) proposes unbiased unweighted variance estimators for regression and ratio imputation models Fuller and Kim (2005) prove the formula for a fully efficient fractionally imputed variance estimator based on the squared deviation of mean estimator and response probabili-ties in particular imputation cells and subpopulations Similar research for balanced random imputa-tion has been conducted by Chauvet et al (2011)

In the case of the ex post quality control we can use decomposition (96) That is assuming that the units were sampled independently variance will be approximated by ~V=~V SAM+

~V IMP+2~V MIX where the relevant components are estimated using the following statistics

a) sampling effects~V SAM=

iquest Aoriquest2sumiisin A

(~y iminusθ A)2iquest

b) imputation effects~V IMP=

1iquest Aoriquest2sum

iisin A( y i

iquestminus~y i )2iquest

c) mixed effects~V MIX=

( y iiquestminus~y i ) (~y iminusθA ) iquest

where y iiquest=p y i+(1minusp) y i and ~yi=p y i+(1minus p)iquest θA=sum

iisin Ay iiquestiquest Aoriquestiquest with p=1 if the value of

Y for i-the unit on Y is available and p=0 otherwiseθR=sumiisin R

y iiquest Roriquestiquest and iquest ∙oriquest denotes the car-

dinality of a given set Thus we can perform detailed diagnostics of our imputation method

It is also worthwhile to consider variance for repeated imputation methods The idea comes from Ru-bin (1987) According to this approach we can use the classical model of a full sample estimator for

the mean (ie t=sumiisins

wi yi sumiisins

wi) to fill in the missing values of the target variable in nonndashsampled

units and create a completed sample This process can be repeated any number of times Kott (1995) considers a paradox which occurs when the methodology is applied to weighted survey data although imputations themselves are often based on models of variable behavior variance estimates derived from repeatedly imputed data sets are not conditioned on realized survey respondents as is typical in model-based sampling theory In this case variance estimation relies rather on the assumption of a quasi-random response mechanism An interesting review of variance estimation in re-sampling meth-ods of imputation (such as the RaondashShao jackknife and the ShaondashSitter bootstrap) can be found in the paper by Haziza (2010) He also provides a proposal of bootstrap procedure for estimating total vari-ance based on the bootstrap approach by RaondashWu and using repeated mean imputation

An extensive review and comparison (both theoretically and empirically via simulation) of the above methods is conducted Lee et al (1999) A comparative study with emphasis on multiple imputation is provided in Luzi and Seeber (2000) In general the choice of the variance estimation procedure that takes into account imputation requires knowledge of the following information (Lee et al (1999))

indication of the responsenon response status imputation method used and information about the auxiliary variables (if applicable) information about the donor (in the case of donor imputation) imputation classes (if applicable)

In general of the last three conditions at least one should be satisfied

Let us now analyse analyze the efficiency of variance estimation in various sampling designs As regards systematic sampling there is no unbiased estimator of variance in this case There are how-

ever several types of biased variance estimators such as V=( N 2(1minus(nN ))n ) S y2 where Sy

2 is the

sample variance for the drawn sample (cf Goga (2008)) For systematic sampling with implicit strati-fication (ie stratification by ordering the sampling units in the frame by certain criteria which are correlated with the key estimates to be obtained from the survey data) Megill et al (1987) propose special formulas of variance estimators using weighted statistics for sample segments created accord-ing to the assumed rule of implicit stratification

The case of double sampling is an interesting situation This design is applied when we would like to confirm uncertain inferences from the first sample The second sample is usually drawn when the number of defectives lies between the number of acceptance and the number of rejections of the first sample Judkins and Hidiroglou (2004) construct a special estimator of variance for multindashstage sam-pling based on the Yates-Grundy-Sen type of approach using Horvitz and Thompsonrsquos original for-mula to rewrite the conditional variance of a set of sampled units They show next that for a two-stage design with stratified sampling without replacement at each stage and fixed sample sizes at the first stage if first-stage sampling is independent across first-stage strata and the second-stage sampling is independent across second-stage strata and all pairs of first-stage units and all pairs of second-stage units also have non-zero joint probabilities of selection then they propose a statistic in the form of an unbiased estimator of variance of the double expansion estimator of population totals It is worth un-derlining that there is no requirement for fixed sample sizes for the second-stage strata and there is no requirement for second-stage sampling to be independent across PSUs nor even across first-stage strata This makes ndash according to the authorsrsquo opinion - the estimator ideal for use in double-sampling applications The next question is how to treat outliers occurring during imputation in the estimation process Bergdahl et al (2001) argues that ldquooutliers often arise because industry and size characteris-tics used to define strata are out of date and so a stratum ends up containing units whose lsquocurrentrsquo characteristics (and resulting economic performance) are quite unrelated to that of the majority of units in the stratumrdquo A good estimator should be robust ie unaffected by outliers To this end one can apply special robustness weights per observation which indicate the degree of robustification and help to determine tuning constants which robustness of an estimator depends on From the study by Hulliger and Muumlnnich (2006) variance estimators for robust estimators are derived from their estimat-ing equation and can be expressed as

V (θ)asymp V (sumiisinSw iui e i)

where w i is a classical sampling weight ui is a robustness weight and e i is residual y iminusθ (let us re-

member that θ denotes the estimator of parameter θ of the population) considered to be fixed An-other option is more intensive use of order statistics (such as medians median absolute deviations quantiles etc) there are also relevant approximation theorems for them (cf Serfling (1980))

Another problem occurs when the sample size per stratum is small (extremely small if there is only one unit per stratum) In such cases some surveys conducted in Great Britain by the Office for Na -tional Statistics use combined ratio estimation based on the assumption of a constant ratio (or regres -sion slope) over the size stratastratum size This attempt is similar to the synthetic estimation method (cf Bergdahl et at (2001))

24 Domains and classes

It is worth noting that estimates from sample surveys are often produced for different subgroups classes or subpopulations which the population can be divided into Such a division can be based on various criteria eg spatial units type of activity type of produced goods etc These subgroups are called study domains or subdomains Researchers may be interested in a particular domain in which units have certain characteristics In this case point estimates can be produced from the study domain but a full sample is required for variance estimation This is called domain analysis (cf AHRQ (2011))

If the analyzed group is only a subset of the study domain then there will be no problem in producing point estimates of classical descriptive statistics such as mean percentage or total However if the subset does not adequately reflect the full sample design or the share of the domain in the full popula-tion variance cannot be estimated correctly For example oif the simple random sampling with re-placement is used then sampling errors in sampling in one strataum resulting in (hidden or consider-able) dependency distorts the final results On the other hand if the share of sampled units belonging to a given strataum in the whole sample in terms ofcontext of a given feature amounts to eg 10 whereas in the same stratum for the whole population it amounts in the same strata to 50 the result-ing estimates will be strongly affect theof poor quality of estimates Variances or standard errors of these point estimates may then be distorted Of course this doesnrsquot apply to the situation when the sample is selected separately and independently of each domain and the size of domains in the popula-tion are known and the weighting adjustment or calibration is made independently within each do-main However if we cannot make this assumption sample size becomes random in repeated sam-pling AHRQ (2011) notes also that ldquoif the population total of the domain is not known and not computed benchmarked at the domain level then the variance of the estimate of a total of a vari-able (say total expenseexpenditures) not only depends on the variance of the mean expense expen-diture per unitperson but also the variance of the estimate of the total number of persons units in the domain Hence tThe full file with all domains is required to compute the variance of the total or the proportion that belong to the domain The Aauthors of this document note that in the latter case tThe estimate of the mean in this case becomes the case of an estimate of a ratio because both the numerator and the denominator of the mean are estimates and the variance of the mean needs to be correctly estimated by treating it as a ratio For means this complication can be avoided by assuming that the sample size for the domain is fixed over repeated draws of the sample of the same overall size The problem is more complex for computing the variance of an estimate of totalrdquo This document proposes the following estimation of variance

V c=N s2

where s2=sumi=1

( y iiquestminus y i

iquest)2 (nminus1) and y iiquest= y i if I belongs to the domain c and y i

iquest=0 otherwise

y iiquest=sum

y iiquest n

Its authors also point out that ldquowhen a complex cluster sample design is used the variance of a survey estimate is often computed based on variance strata and clusters (PSUs) using the Taylor series ap-proximation This approach needs at least two PSUs within each variance stratum to compute the variance by accounting for the variance contributions from all strata If the domain of interest is small or clustered in certain areas so that some PSUs do not include any case from the domain then some variance strata appear to have no PSU or only one PSU when the file is subset to the domain The situation of one PSU within a stratum is known as the singleton PSU problem In this case it is not possible for variance computation software to correctly compute the variance from that stratum unless the full file is provided and a domain analysis is requestedrdquo They analyze efficiency of various soft-ware in this context (eg SAS procedures SUDAAN etc) Of course the aforementioned document concerns first of all the problems of health surveys but these problems are directly related to business statistics and are here better described here asrather than anyelsewhere

Of course some consideration onto such issues can be found alsoalso is also given inby Wolter (2007)

The problem of variance estimation for small domains is spread across the above consideration Incon-veniences may result from stratification missing data domain and unit clustering etc It can be noted however that classes can cut across data and the cluster used in the selection There are several treat-ment options one consists in treating common parts of clusters (domains) and strata as separate do-mains and estimating relevant data for them and then performing the required aggregation This at-tempt may result however in improper estimation of variance if some of these domains contain few units So it seems better to establish the share of each domain in each stratum estimate required data for such strata and next ndash using this share and the ratio estimator ndash estimate the value for a particular cross section of the domains and these strata and finally make aggregates

254 Surveys over time

Variance estimation perceived from the point of view of surveys repeated over time also depends on time series factors Namely there may be gaps in data for some periods and these should be filled by coldndashdeck imputation or time series modeling (see relevant chapters of this handbook) In the first case variance can be large because of errors in the optimization of donor and time series data which have to be imputed to fill missing items In the second case the problem lies in the quality of adjust-ingtment of the trend function and the resulting standard error of estimating missing values So both factors are involved Of course smoothing or seasonal adjustment can improve the quality of such models (eg it is usually observed that firms at the end of the year pay their employees higher wages and salaries owing to the division of annual profit or dividend in some companies conducting a sea-sonal activity ndash eg in construction ndash economic results in winter are usually considerably lower that than in spring or summer) In such situations the use of relative errors can be a useful tool for assess -ing data quality They enable an easy interpretation of specific results especially in terms of modeling the regularities of variation However they are more sensitive to variation among subclasses when estimating proportions Hence in this case absolute values or counts are recommended On the other hand a positive correlation between overlapping samples from smaller periods (eg quarters) can oc-cur Reducing data collection frequency (eg from quarterly to annual averages) can help to decrease the bias of estimators of totals

The question of variances for seasonally adjusted series can be perceived in a number of contexts In the case of model-based seasonal adjustment there are various solutions n for theirto the problem of estimation That is if we use for adjustment the Xndash11 or Xndash12 methods linear approximations to the X-11 estimator isare applied The fundamental formulas for them can be found in Wolter and Monseur (1981) and some their extensions in Pfeffermann (1992) His considerations are based on the assump-tion that X-11 produces an unbiased estimate of the true seasonal adjustment His method should be used with settings specified for autocorrelations identified in the sampling error of the original series Cleveland (2002) performsed a study using simulated series and comparisons of alternative seasonal adjustment results for a few economic series to assess the accuracy of variance estimates He com-paresd estimates of Pffefermanrsquos method with model-based estimates computed directly with formulas and estimates obtained from the TRAMO-SEATS programs The starting point of his analysis is the two component model

where is the observed series indexed by time periods t is the seasonal component with

where is the trend or trendndashcycle component and is a noise The variance of the seasonal com-ponent estimate of the seasonally adjusted series is given as

and the estimator for the seasonal component has the following form

The variance of the residual component is another interesting but sophisticated problem discussed by Liitiaumlinen et al (2007)

The nextA specific problem is the variance estimation of indices It is analyzed by Seljak (2006) who assumes that ldquoGenerally index is defined as a ratio of two or more values all measured with the same unit Index can compare values in time or space but for the purposes of this study we will only con-sider time indices hence we define index as a ratio of two or more values measured with the same unit in two different time pointsrdquo Considering the turnover of economic entities he suggests the fol-lowing formula for estimating variance

Var (T t 0 )=Var ( T t

T0)= (1+T t 0 )Var ( T t )+(T t 0+T t 0

2 )Var (T 0 )minusT t 0Var (T t+T 0 )T 0

T t 0=sumi=1

wiquestT iquest

sumi=1

wi 0T i 0

T iquest is the turnover of indashth unit w iquest ndash its weight and nt ndash sample size in period t To estimate the vari-

ance Var (T t+T 0 ) he proposes to construct the common weight of the sum as

wsumiquest=(w iquestiquest iquestT iquest+w i0 T i 0)(T iquest+T i 0)iquestiquest He also proposes a special option of variance estimation designed for strat-

ified sampling and based on the sampling covariance

(()) (( )))()

As regards sampling errors of measures of the net change in the analyzed rates from one time period (cross-section) to another we can observe that a large proportion of individuals are common in differ-ent panels Cross-sectional samples are not independent resulting inproducing correlations between measures from different waves Apart from correlations at the individual level we also have to deal with additional correlation that arises from the same structure (stratification and clustering) of panel waves Such correlation would exist for instance in samples coming from the same clusters even if there was no overlap in terms of individual units Some researchers suggest using the Jackknife Re-peated Replication (JRR) approach extended to enable the use of the common sample structure of cross-sections in a panel and for each replication the required measure is constructed for each cross-section involved next these replication-specific cross-sectional measures are differentiatedaggre-gated to obtain the required net-change and average measures for the replication5

3 Design issues

Elements of designing within this topic concern mainly the problem of properly choosing the sampling design To do itso we should be aware of errors which may occur and the sources of possible of vari-ability of estimators used The problem of the optimal choice of the sampling design is discussed in the chapter ldquoSample selectionrdquo It is also necessary to assess the quality of the sampling frame used For instance we should identify units which most often give no or incomplete responses and conduct a trial to overcome this problem (see chapter ldquoResponserdquo) It would be useful to rely on experience from previous rounds of a given survey (if it is repeated over time) As regards the estimator type weight calibration is recommended This reduces the variance caused by sampling errors (see also chapter ldquoWeighting and estimationrdquo) Software capacity in this respect should also be verified

5 This approach is also used in European Social Surveys see httpwwweuropeansocialsurveyorgin-dexphpoption=com_docmanamptask=doc_downloadampgid=360ampItemid=80

Of course in most surveys the choice of an estimator to be used is made arbitrarily in advance ie before starting the processing of individual data Riviegravere and Depoutot (2010) observe that even if in a business survey a completely enumerated (also called takendashall) stratum occurs in a business survey where all units belonging to such a stratum take part in the survey (eg a survey of large enterprises) then the estimation of variance is very sensitive to the chosen method of estimation and missing data They suggest computing not the variance of the total but the variance of the total error This solution may be desirable eg when outliers can be detected various additional robustification methods are used to improve the estimation precision of estimation etc This method can be motivated by special treatment and processing of nonndashrespondents changing criteria for deciding where individual data can be regarded as outliers adaptation of processing outliers postndashstratification and many other argu-ments

Another example of the problem of choosing the estimator from the point of view of variance is the use of totals (or rather data expressed in absolute values eg revenues in EUR profit in EUR num-ber of employees etc) which may strongly distort variance estimation That is among units of inter-est there can be a few such that ndash for natural reasons (size rank on the market etc) are greater than others For them totals will be much greater than for others If these units are included in the sample variance can be underestimated otherwise ndash it can be overestimated Therefore it is better to use rela-tive values at least as the basis for sampling (eg wages and salaries per employee change of finan-cial result average revenue per employee share in production of given goods etc) From the point of view of the construction of statistics the mean is very sensitive to outliers and should therefore be used carefully especially if the size and other basic statistical features of the population are unknown If the number of outliers is not extremely small and they are recognized one can adjust the sampling design to obtain a desired level of representativeness in the sample On the other hand the median reduces the unfavorable impact of outliers and can be described as lsquothe voice of majorityrsquo of observa -tions Then it seems to be better for large surveys Moreover the order equivalent of standard devia-tion is the median absolute deviation which is the median of the absolute deviations of values of a given variable from its median mad (X )=iquest||

4 Available software tools

There are many efficient software packages which can also be used for variance estimation in sample surveys Eurostat (2002 a) provides a rich collection of computer support programs

Generalized estimation system (GES) developed by Statistics Canada is a SASndashbased appli-cation with a Windows-type interface It can be used for stratified random sampling designs and for computing variance estimators for totals means proportions or ratios (for the whole population or domains) It also enables the user to apply methods of variance estimation in-cluding Taylor linearisation and jackknife techniques

Generalized software for sampling errors (GSSE) is a generalized software in the SAS envi-ronment developed within ISTAT mainly devoted to the calculation of statistics and corre-sponding standard errors of data from sample surveys (for the whole population or domains) In the case of multindashstage sampling like in other applications estimated variance is based solely on PSU variance Weights for nonndashresponse adjustment complete or incomplete post-stratification can be incorporated via the GSSW companion software Standard errors are cal -culated using the Taylor linearisation method for variance estimation It is worth menytion-ing however that Ssince 2005 Istat has been usesing the Genesees software and now it is introducing the ReGenesees software (based on the Rndashcode)

Imputation and variance estimation software (IVEware) is also a SASndashbased application which accounts for stratified random sampling designs elements or cluster designs Variance estimators can be obtained for means proportions and linear combinations of these using the Taylor linearisation procedure as well as for the parameters of linear logistic Poisson and polytomous regression (using the jackknife technique)

PC CARP (Iowa State University) is a standndashalone package used to analyze survey data Us-ing Taylor linearisation method it can enable to compute variances of totals means quantiles ratios difference of ratios taking into account the sampling design A postndashstratification and weighting is also possible

Poulpe (Programme optimal et universel pour la livraison de la precision des enquecirctes) is based on SAS has been developed by INSEE and can incorporate sampling features such as stratification clustering or multistagendashsampling It takes GREG weights provided by the latter in order to estimate variance of totals ratios etc based on the Taylor linearisation technique

SAS procedures the SAS statistical package (from version 7 and onwards) provides possibili-ties to estimate standard errors of simple descriptive statistics as well as linear regression mod-els (Surveymeans and Surveyreg SAS procedures) Surveymeans procedure esti-mates descriptive statistics and their corresponding standard errors taking into account the sampling design (stratification or clustering) and possible domain estimation while Sur-veyreg algorithm performs a linear regression analysis providing variance estimates for regression coefficients (taking into account the specified sampling design and weights)

STATA is a complete statistical software package and survey commands are part of it This software can correctly (ie accounting for sampling design stratified clustered or multi-stage) estimate the variance of measures such as totals means proportions ratios (either for the whole population or for different subpopulations) using the Taylor linearisation method The software also includes commands for jackknife and bootstrap variance estimation

Sudaan is a statistical software package for the analysis of data from sample surveys (simple or complex) It uses the SASndashlanguage and has a similar interface it is a standndashalone package It can estimate the variance of simple quantities (such as totals means ratios in the whole population or within domains) as well as more sophisticated techniques The available vari-ance estimation techniques include the Taylor linearisation jackknife and balanced repeated replication

WesVar is a package primarily aiming at the estimation of basic statistics (as well as specific models) and corresponding standard errors from complex sample surveys (under various sam-pling designs) utilizing the method of replications (balanced repeated replication jackknife and bootstrap) Moreover it can calculate (and take into account in variance estimation) weights of nonndashresponse adjustments complete or incomplete postndashstratification

5 Decision tree of methods

Not applicableThere doesnrsquot existis no any efficient decision tree of methods The choice will depend on the type and purpose of the survey sampling design expected contribution of nonndashsampling errors capacity of hardware and software etc Nevertheless firstly we should first verify results of the previ-ous rounds in a given survey (if theyre were any) or related ones and on the ir basis of them assess what we can expect For instance if the nonndashresponse is always large then we must use respectively adjusted estimators or calibrated weights

6 Glossary

Term Definition Source of definition (link)

Synonyms (optional)

Balanced repeated repli-cation (BRR)

A sampling method consisting in thatwhere the full sample is drawn by a stratified sample design with two primary sampling units (PSUs) per stratum

Wolter (2007)

Bootstrap A sampling based on bootstrap samples (or boot-strap replicates) which is a simple random sample with replacement of a fixed size selected from the main sample (which can also be a superpopulation for this survey)

Wolter (2007)

deff The impact of design effects on the quality of esti-mation usually measured by a ratio of variations of estimation obtained using different methods

M J Salganik (2006)

ERB Empirical Relative Bias (ERB) which combines ER and MSE as their ratio

C Arcaro and W Yung (2001)

Jackknife algorithm

A sampling scheme based on omitting of some groups of units from the sample Assuming that the sample is divided into a number of groups of obser-vations each an estimator is determined from the reduced sample of smaller size obtained by omit-ting the some of the determined groups of re-spondents

Wolter (2007)

Method of random groups

This attempt consists in drawing several samples from the population (using consistently the same sampling design) constructing separate estimates of the target population parameter of interest using data from each sample and an additional estimate on the basis of data obtained from the combination

Wolter (2007)

Synonyms (optional)

of all selected samples and computing the sample variance among the several estimates

MSE Mean Square Error ndash expected square deviation of an estimator from the estimated population param-eter estimated by relevant sums of squared dis-tances of estimates

Rubin D B (1987)

RB Empirical bias ndash relative deviation of an average estimate (obtained in many trials) from the target parameter value

Rubin D B (1987)

Sampling error The variability caused by observing a sample in-stead of the whole population

Wolter (2007)

Taylor lin-earization

The most popular analytical method which is aimed at finding an approximate linear formula for an (at least approximately) unbiased estimator of sampling variance This method is well established to obtain variance estimators for nonndashlinear and differentiable statistics

Wolter (2007)

7 Literature

AHRQ (2011) Variance Estimation from MEPS Event Files Methodology Report 26 Agency for Healthcare Research and Quality US Department of Health amp Human Services Rockville MD USA httpmepss-3commepswebdata_filespublicationsmr26mr26pdf

Arcaro C Yung W (2001) Variance estimation in the presence of imputation SSC Annual Meeting Proceedings of the Survey Method Section pp 75 ndash 80

Bergdahl M Black O Bowater R Chambers R Davies P Draper D Elvers E Full S Holmes D Lundqvist P Lundstroumlm S Nordberg L Perry J Pont M Prestwood M Richardson I Skinner Ch Smith P Underwood C Williams M (2001) Model Quality Report in Business Statistics General Editors P Davies P Smith httpuserssoeucscedu~draperbergdahl-etal-1999-v1pdf

Chauvet G Deville JndashC Haziza D (2011) On Balanced Random Imputation in Surveys Biometrika vol 98 pp 459 ndash 471

Cleveland W P (2002) Estimated Variance of Seasonally Adjusted Series Series Finance and economics discussion series Federal Reserve Board Washington DC USA available at httpwwwfederalreservegovpubsfeds2002200215200215pappdf

Cochran WG (1977) Sampling Techniques John Wiley amp Sons New York U S A

Deville J-C (1999) Variance estimation for complex statistics and estimators linearization and residual techniques Survey Methodology 25193204

Davison A C Sardy S (2007) Resampling Variance Estimation in Surveys with Missing Data Journal of Official Statistics Vol 23 pp 371ndash386

Deville JndashC Saumlrndal C E (1992) Calibration Estimators in Survey Sampling Journal of the American Statistical Association vol 87 pp 376ndash382

Eurostat (2009) ESS Handbook for Quality Reports 2009 edition Series Eurostat Methodologies and Working papers Office for Official Publications of the European Communities Luxembourg httpunstatsunorgunsddnssdocs-nqafEurostat-EHQR_FINALpdf

Eurostat (2002 a) Monographs of Official Statistics Variance estimation methods in European Union Series Research in Official Statistics Theme 1 General Statistics Office for Official Publications of European Communities Luxembourg available on the webpage of the Eurostat at httpeppeuro-stateceuropaeuportalpageportalqualitydocumentsMOS20VARIANCE20ESTIMATION202002pdf

Eurostat (2002 b) Implementation of the SBS Quality Regulation in Candidate Countries Meeting of The Working Group ldquoStructural Business Statisticsrdquo Quality Issues 27 th September 2002 DocEuro-statD2SBSSept02Q04EN Statistical Office of European Communities LuxembourgndashKirchberg Luxembourg

Full S Lewis D (2011) Estimating Sampling Errors for Movements in Business Surveys Office for National Statistics United Kingdom document available at httpwwwscbseGruppProdukter_TjansterKurserTidigare_kurserq2001Session_11pdf

Fuller W A Kim J K (2005) Hot Deck Imputation for the Response Model Survey Methodology vol 31 pp 139ndash149

Goga C (2008) Variance estimators in survey sampling Universiteacute de Bourgogne Dijon France httpmathu-bourgognefrIMBgogaChapVar1_coursBesanpdf

Groves R M (2004) Survey Errors and Survey Costs Wiley Series in Survey Methodology John Wiley ampSons Hoboken New Jersey U S A

Hampel F R Ronchetti E M Rousseeuw P J and Stahel W A (1986) Robust Statistics The Ap-proach Based on Influence Functions John Wiley amp Sons New York USA

Hartley H O and Biemer P P (1978) The estimation of non-sampling variances in current surveys Proceedings section on survey research American Statistical Association Washington D C U S A pp 257 ndash 262

Hayward M C (2010) A comparative study of optimal stratification in business and agricultural surveys A thesis submitted in partial fulfillment of the requirements for the Degree of Master of Sci-ence in Statistics at the University of Canterbury University of Canterbury UK httpircanter-buryacnzbitstream1009256971thesis_fulltextpdf

Haziza D (2010) Resampling methods for variance estimation in the presence of missing survey data Proceedings of the conference of the annual conference of the Italian Statistical Society

Hesterberg T Monaghan S Moore D S Clipson A Epstein R (2003) Bootstrap Methods And Per-mutation Tests Companion Chapter 18 to the Practice of Business Statistics W H Freeman and Company New York USA

Henderson R Diggle P Dobson A (2000) Joint modeling of longitudinal measurements and event time data Biostatistics vol 4 465ndash480

Holmes D G Skinner C J (2000) Variance estimation for labour force survey estimates of level and change GSS Methodology Series No 21

Huber P J (1981) Robust Statistics John Wiley amp Sons New York

Hulliger B Muumlnnich R (2006) Variance Estimation for Complex Surveys in the Presence of Out-liers Proceedings of the Survey Research Methods Section of the American Statistical Association pp 3153 ndash 3161

ICES (2010) ICES Wkprecise Report 2009 Report of the Workshop on methods to evaluate and estimate the precision of fisheries data used for assessment (WKPRECISE) 8-11 September 2009 ICES Advisory Committee ICES CM 2009ACOM40 REF PGCCDBS International Council for the Exploration of the Sea Copenhagen Denmark document available at httpwwwicesdkreportsACOM2009WKPRECISEWKPRECISE20Report202009pdf

Judkins D R Hidiroglou M A (2004) Double Sampling in a MultindashStage Design Proceedings of the Section on Survey Research Methods of the American Statistical Association pp 3743 ndash 3749

Kalton G (1983) Compensating for Missing Survey Data Research Report Series Ann Arbor Michigan Institute for Social Research University of Michigan

Kalton G (1986) Handling wave non-response in panel surveys Journal of Official Statistics vol 2 No 3 pp 303-314

Kim K (2000) Variance estimation under regression imputation model Proceedings of the Survey Research Methods Section American Statistical Association

Kim J K Brick M Fuller W A Kalton G (2006) On the bias of the multiple-imputation variance estimator in survey sampling Journal of the Royal Statistical Society Series B (Statistical Methodology) vol 68 pp 509ndash521

Kott P S (1995) A paradox of multiple imputation Proceedings of the Survey Research Methods Section of the American Statistical Association pp 380 ndash 383

Kovar J G Whitridge P J (1995) Imputation of business survey data Business Survey Methods eds Cox B G Binder D A Chinnappa B N Christianson A Colledge M J and Kott P S pp 403ndash423 New York John Wiley amp Sons

Liitiaumlinen E Lendasse A and Corona F (2007) Non-parametric Residual Variance Estimation in Supervised Learning [in] Sandoval F Prieto A G Cabestary J Grantildea M (eds) IWANN 2007 LNCS vol 4507 Springer Heidelberg pp 63 ndash 71

Lee H Rancourt E Saumlrndal C E (1999) Variance estimation from survey data under single imputation paper presented at the International Conference on Survey Non-response to be published in a monograph

Lee H Rancourt E Saumlrndal C E (1999) Variance estimation from survey data under single impu-tation paper presented at the International Conference on Survey Non-response to be published in a monographLee K H (1973) Variance Estimation in Stratified Sampling Journal of the American Statistical Association vol 66 pp 336ndash342

Liu J S (1994) The collapsed Gibbs sampler in Bayesian computations with applications to a gene regulation problem Journal of the American Statistical Association vol 89 pp 958ndash966

Lohr S L (1999) Sampling Design and Analysis Duxbury Press Pacific Grove CA USA

Luzi O Seeber A C (2000) Multiple imputation mdash An experimental application prepared for Working Group on the Assessment of Quality on Statistics Eurostat

Market Torrent (2012) Finding the Value of Intangibles in Business paper available at httpmarket-torrentcomcommunityviewtopicphpf=10ampt=39235

Megill D J Gomez E E Balmaceda A Castillo M (1987) Measuring the Efficiency Of Implicit Stratification In Multistage Sample Surveys Proceedings of the Survey Research Methods Section American Statistical Association pp 166 ndash 171

Najmusseharl K M G M Ahsan M J (2005) Optimum Stratification for Exponential Study Variable under Neyman Allocation Journal of Indian Society of Agricultural Statistics vol 59 pp 146ndash150

Olufadi Y (2010) On the estimation of ratio-cum-product estimator using twondashstage sampling Statistics in Transition ndash new series vol 11 pp 253ndash265

Perri P F (2007) Improved Ratio-Cum-Product Type Estimators Statistics in Transition-New Series vol 8 No 1 pp 51mdash69

Pfeffermann D (1994) A general method for estimating the variance of X-11 seasonally adjusted estimators Journal of Time Series Analysis 15 pp 85ndash116

Rao JNK and Shao J (1999) Modified balanced repeated replication for complex survey data Biometrika vol 86 pp 403ndash415

Riviegravere P Depoutot R (2010) Estimating variance from a completely enumerated stratum and other aspects of variance calculation in business surveys INSEE France amp University of Southampton UK httpwwwscbseGruppProdukter_TjansterKurserTidigare_kurserq2001Session_5pdf

Rubin D B (1987) Multiple Imputation for Nonresponse in Surveys John Wiley amp Sons New York

Salganik M J (2006) Variance Estimation Design Effects and Sample Size Calculations for Respondent-Driven Sampling Journal of Urban Health Bulletin of the New York Academy of Medicine Vol 83 No 7 pp 98 ndash 112

Saumlrndal C E Lundstroumlm S (2005) Estimation in Surveys with Nonresponse John Wiley amp Sons New York

Saringrndal C E Swensson B and Wretman J (1992) Model assisted survey sampling Springer-Verlag New York Inc

Seljak R (2006) Estimation of Standard Error of Indices in the Sampling Business Surveys Proceedings of Q2006 European Conference on Quality in Survey Statistics httpeppeurostateceuropaeuportalpageportalqualitydocumentsESTIMATION20OF20STANDARD20ERROR20OF20INDICES20IN20THE20SAMPLINGpdf

Serfling R J (1980) Approximation Theorems of Mathematical Statistics John Wiley amp Sons

Singh R Kumar M Smarandache F (2008) Almost Unbiased Estimator for Estimating Population Mean Using Known Value of Some Population Parameter(s) Pakistan Journal of Statistics and Operation Research vol 4 pp 53 ndash 76

Skinner C (1999) Calibration Weighting and Non-Sampling Errors Research in Official Statistics 2 pp 33 ndash 43

Thompson S K (2002) Sampling John Wiley ampSons New York 2002

Tsiatis A A Davidian M (2004) Joint Modeling of Longitudinal and Time-To-Event Data An Overview Statistica Sinica vol 14 pp 809ndash834

UNSD (1993) Sampling Errors in Household Surveys National Household Survey Capability Programme No UNFPAUNINTndash92ndashP80ndash15E United Nations Department For Economic and Social Information and Policy Analysis Statistical Division New York USA Available at httpunstatsunorgunsdpublicationunintUNFPA_UN_INT_92_P80_15Epdf

West S A Kratzke D ndash T Robertson K W (1994) Variance Estimators for Variables that have Both Observed and Imputed Values Proceedings of the Survey Research Methods Section of The American Statistical Association document available in the Internet at the website httpwwwamstatorgsectionssrmsproceedingspapers1994_064pdf

Wolter K (2007) Introduction to Variance Estimation Second Edition New York Springer

Wolter K M and Monsour N J (1981) On the problem of variance estimation for a deseasonalized series [in] Current Topics in Survey Sampling An International Symposium (eds D Krewski R Platek and J N K Rao) New York Academic Press pp 367ndash 403

Specific description ndash Theme

Interconnections with other modules

Related themes described in other modules

1 User needs

2 Design of Data Collection Methodology

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

Methods explicitly referred to in this module

1 Use of administrative data

2 Sample selection

3 Data imputation

4 Estimation methods

Mathematical techniques explicitly referred to in this module

1 Estimators of variance

2 Effective sample selection algorithms

3 Imputation algorithms

4 Weighting algorithms

GSBPM phases explicitly referred to in this module

1 GSBPM Phases 41 52 ndash 56

Tools explicitly referred to in this module

1 Reporting portals e-questionnaires

2 Software for statistical analysis (eg R SAS)

Process steps explicitly referred to in this module

General description ndash Theme

1 Summary







3 Design issues



6 Glossary

7 Literature









Contents

1 Summary 3

23 Variance estimation in the case of unit nonndashresponse or item nonndashresponse 19

3 Design issues 26

6 Glossary 29

7 Literature 29

1 Summary 3

23 Variance estimation in the case of imputation 17

3 Design issues 22

6 Glossary 24

7 Literature 25

1 Summary

knife method

Sampling design

Measurement errors

Processing errors

Substitution errors

Analytical methods

( ) ()

and the MSE

( ) ( () )(sum

())sum

(x iminusX )2

Replication methods

V=( nminus1n )sum

( y(i)minus y )2()

() sum

() where

z iiquest i=1

V= 1mminus1sumi=1

is given as sum

() ( ) where sum

(())radic ()

( ) ( )

sum ()

) () ()

MSE (θ )= 1q2sum

(θkminusθ )2 ()

RB ( θ )=(sumk=1

θk q)minusθ

ERB (θ )=sumk=1

V=V W+q+1

qV B ()

where V W=sumk=1

V B=sumk=1

iisin A( y i

wi yi sumiisins

2 is the

V c=N s2

where s2=sumi=1

iquest=0 otherwise

y iiquest=sum

y iiquest n

T0)= (1+T t 0 )Var ( T t )+(T t 0+T t 0

T t 0=sumi=1

wiquestT iquest

sumi=1

wi 0T i 0

(()) (( )))()

3 Design issues

6 Glossary

Synonyms (optional)

Wolter (2007)

M J Salganik (2006)

Jackknife algorithm

Wolter (2007)

Synonyms (optional)

Rubin D B (1987)

Wolter (2007)

7 Literature

1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









6 Glossary 24

7 Literature 25

1 Summary

knife method

Sampling design

Measurement errors

Processing errors

Substitution errors

Analytical methods

( ) ()

and the MSE

( ) ( () )(sum

())sum

(x iminusX )2

Replication methods

V=( nminus1n )sum

( y(i)minus y )2()

() sum

() where

z iiquest i=1

V= 1mminus1sumi=1

is given as sum

() ( ) where sum

(())radic ()

( ) ( )

sum ()

) () ()

MSE (θ )= 1q2sum

(θkminusθ )2 ()

RB ( θ )=(sumk=1

θk q)minusθ

ERB (θ )=sumk=1

V=V W+q+1

qV B ()

where V W=sumk=1

V B=sumk=1

iisin A( y i

wi yi sumiisins

2 is the

V c=N s2

where s2=sumi=1

iquest=0 otherwise

y iiquest=sum

y iiquest n

T0)= (1+T t 0 )Var ( T t )+(T t 0+T t 0

T t 0=sumi=1

wiquestT iquest

sumi=1

wi 0T i 0

(()) (( )))()

3 Design issues

6 Glossary

Synonyms (optional)

Wolter (2007)

M J Salganik (2006)

Jackknife algorithm

Wolter (2007)

Synonyms (optional)

Rubin D B (1987)

Wolter (2007)

7 Literature

1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









1 Summary

knife method

Sampling design

Measurement errors

Processing errors

Substitution errors

Analytical methods

( ) ()

and the MSE

( ) ( () )(sum

())sum

(x iminusX )2

Replication methods

V=( nminus1n )sum

( y(i)minus y )2()

() sum

() where

z iiquest i=1

V= 1mminus1sumi=1

is given as sum

() ( ) where sum

(())radic ()

( ) ( )

sum ()

) () ()

MSE (θ )= 1q2sum

(θkminusθ )2 ()

RB ( θ )=(sumk=1

θk q)minusθ

ERB (θ )=sumk=1

V=V W+q+1

qV B ()

where V W=sumk=1

V B=sumk=1

iisin A( y i

wi yi sumiisins

2 is the

V c=N s2

where s2=sumi=1

iquest=0 otherwise

y iiquest=sum

y iiquest n

T0)= (1+T t 0 )Var ( T t )+(T t 0+T t 0

T t 0=sumi=1

wiquestT iquest

sumi=1

wi 0T i 0

(()) (( )))()

3 Design issues

6 Glossary

Synonyms (optional)

Wolter (2007)

M J Salganik (2006)

Jackknife algorithm

Wolter (2007)

Synonyms (optional)

Rubin D B (1987)

Wolter (2007)

7 Literature

1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









1 Summary

knife method

Sampling design

Measurement errors

Processing errors

Substitution errors

Analytical methods

( ) ()

and the MSE

( ) ( () )(sum

())sum

(x iminusX )2

Replication methods

V=( nminus1n )sum

( y(i)minus y )2()

() sum

() where

z iiquest i=1

V= 1mminus1sumi=1

is given as sum

() ( ) where sum

(())radic ()

( ) ( )

sum ()

) () ()

MSE (θ )= 1q2sum

(θkminusθ )2 ()

RB ( θ )=(sumk=1

θk q)minusθ

ERB (θ )=sumk=1

V=V W+q+1

qV B ()

where V W=sumk=1

V B=sumk=1

iisin A( y i

wi yi sumiisins

2 is the

V c=N s2

where s2=sumi=1

iquest=0 otherwise

y iiquest=sum

y iiquest n

T0)= (1+T t 0 )Var ( T t )+(T t 0+T t 0

T t 0=sumi=1

wiquestT iquest

sumi=1

wi 0T i 0

(()) (( )))()

3 Design issues

6 Glossary

Synonyms (optional)

Wolter (2007)

M J Salganik (2006)

Jackknife algorithm

Wolter (2007)

Synonyms (optional)

Rubin D B (1987)

Wolter (2007)

7 Literature

1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









Sampling design

Measurement errors

Processing errors

Substitution errors

Analytical methods

( ) ()

and the MSE

( ) ( () )(sum

())sum

(x iminusX )2

Replication methods

V=( nminus1n )sum

( y(i)minus y )2()

() sum

() where

z iiquest i=1

V= 1mminus1sumi=1

is given as sum

() ( ) where sum

(())radic ()

( ) ( )

sum ()

) () ()

MSE (θ )= 1q2sum

(θkminusθ )2 ()

RB ( θ )=(sumk=1

θk q)minusθ

ERB (θ )=sumk=1

V=V W+q+1

qV B ()

where V W=sumk=1

V B=sumk=1

iisin A( y i

wi yi sumiisins

2 is the

V c=N s2

where s2=sumi=1

iquest=0 otherwise

y iiquest=sum

y iiquest n

T0)= (1+T t 0 )Var ( T t )+(T t 0+T t 0

T t 0=sumi=1

wiquestT iquest

sumi=1

wi 0T i 0

(()) (( )))()

3 Design issues

6 Glossary

Synonyms (optional)

Wolter (2007)

M J Salganik (2006)

Jackknife algorithm

Wolter (2007)

Synonyms (optional)

Rubin D B (1987)

Wolter (2007)

7 Literature

1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









Measurement errors

Processing errors

Substitution errors

Analytical methods

( ) ()

and the MSE

( ) ( () )(sum

())sum

(x iminusX )2

Replication methods

V=( nminus1n )sum

( y(i)minus y )2()

() sum

() where

z iiquest i=1

V= 1mminus1sumi=1

is given as sum

() ( ) where sum

(())radic ()

( ) ( )

sum ()

) () ()

MSE (θ )= 1q2sum

(θkminusθ )2 ()

RB ( θ )=(sumk=1

θk q)minusθ

ERB (θ )=sumk=1

V=V W+q+1

qV B ()

where V W=sumk=1

V B=sumk=1

iisin A( y i

wi yi sumiisins

2 is the

V c=N s2

where s2=sumi=1

iquest=0 otherwise

y iiquest=sum

y iiquest n

T0)= (1+T t 0 )Var ( T t )+(T t 0+T t 0

T t 0=sumi=1

wiquestT iquest

sumi=1

wi 0T i 0

(()) (( )))()

3 Design issues

6 Glossary

Synonyms (optional)

Wolter (2007)

M J Salganik (2006)

Jackknife algorithm

Wolter (2007)

Synonyms (optional)

Rubin D B (1987)

Wolter (2007)

7 Literature

1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









Measurement errors

Processing errors

Substitution errors

Analytical methods

( ) ()

and the MSE

( ) ( () )(sum

())sum

(x iminusX )2

Replication methods

V=( nminus1n )sum

( y(i)minus y )2()

() sum

() where

z iiquest i=1

V= 1mminus1sumi=1

is given as sum

() ( ) where sum

(())radic ()

( ) ( )

sum ()

) () ()

MSE (θ )= 1q2sum

(θkminusθ )2 ()

RB ( θ )=(sumk=1

θk q)minusθ

ERB (θ )=sumk=1

V=V W+q+1

qV B ()

where V W=sumk=1

V B=sumk=1

iisin A( y i

wi yi sumiisins

2 is the

V c=N s2

where s2=sumi=1

iquest=0 otherwise

y iiquest=sum

y iiquest n

T0)= (1+T t 0 )Var ( T t )+(T t 0+T t 0

T t 0=sumi=1

wiquestT iquest

sumi=1

wi 0T i 0

(()) (( )))()

3 Design issues

6 Glossary

Synonyms (optional)

Wolter (2007)

M J Salganik (2006)

Jackknife algorithm

Wolter (2007)

Synonyms (optional)

Rubin D B (1987)

Wolter (2007)

7 Literature

1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









Measurement errors

Processing errors

Substitution errors

Analytical methods

( ) ()

and the MSE

( ) ( () )(sum

())sum

(x iminusX )2

Replication methods

V=( nminus1n )sum

( y(i)minus y )2()

() sum

() where

z iiquest i=1

V= 1mminus1sumi=1

is given as sum

() ( ) where sum

(())radic ()

( ) ( )

sum ()

) () ()

MSE (θ )= 1q2sum

(θkminusθ )2 ()

RB ( θ )=(sumk=1

θk q)minusθ

ERB (θ )=sumk=1

V=V W+q+1

qV B ()

where V W=sumk=1

V B=sumk=1

iisin A( y i

wi yi sumiisins

2 is the

V c=N s2

where s2=sumi=1

iquest=0 otherwise

y iiquest=sum

y iiquest n

T0)= (1+T t 0 )Var ( T t )+(T t 0+T t 0

T t 0=sumi=1

wiquestT iquest

sumi=1

wi 0T i 0

(()) (( )))()

3 Design issues

6 Glossary

Synonyms (optional)

Wolter (2007)

M J Salganik (2006)

Jackknife algorithm

Wolter (2007)

Synonyms (optional)

Rubin D B (1987)

Wolter (2007)

7 Literature

1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









Measurement errors

Processing errors

Substitution errors

Analytical methods

( ) ()

and the MSE

( ) ( () )(sum

())sum

(x iminusX )2

Replication methods

V=( nminus1n )sum

( y(i)minus y )2()

() sum

() where

z iiquest i=1

V= 1mminus1sumi=1

is given as sum

() ( ) where sum

(())radic ()

( ) ( )

sum ()

) () ()

MSE (θ )= 1q2sum

(θkminusθ )2 ()

RB ( θ )=(sumk=1

θk q)minusθ

ERB (θ )=sumk=1

V=V W+q+1

qV B ()

where V W=sumk=1

V B=sumk=1

iisin A( y i

wi yi sumiisins

2 is the

V c=N s2

where s2=sumi=1

iquest=0 otherwise

y iiquest=sum

y iiquest n

T0)= (1+T t 0 )Var ( T t )+(T t 0+T t 0

T t 0=sumi=1

wiquestT iquest

sumi=1

wi 0T i 0

(()) (( )))()

3 Design issues

6 Glossary

Synonyms (optional)

Wolter (2007)

M J Salganik (2006)

Jackknife algorithm

Wolter (2007)

Synonyms (optional)

Rubin D B (1987)

Wolter (2007)

7 Literature

1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









Measurement errors

Processing errors

Substitution errors

Analytical methods

( ) ()

and the MSE

( ) ( () )(sum

())sum

(x iminusX )2

Replication methods

V=( nminus1n )sum

( y(i)minus y )2()

() sum

() where

z iiquest i=1

V= 1mminus1sumi=1

is given as sum

() ( ) where sum

(())radic ()

( ) ( )

sum ()

) () ()

MSE (θ )= 1q2sum

(θkminusθ )2 ()

RB ( θ )=(sumk=1

θk q)minusθ

ERB (θ )=sumk=1

V=V W+q+1

qV B ()

where V W=sumk=1

V B=sumk=1

iisin A( y i

wi yi sumiisins

2 is the

V c=N s2

where s2=sumi=1

iquest=0 otherwise

y iiquest=sum

y iiquest n

T0)= (1+T t 0 )Var ( T t )+(T t 0+T t 0

T t 0=sumi=1

wiquestT iquest

sumi=1

wi 0T i 0

(()) (( )))()

3 Design issues

6 Glossary

Synonyms (optional)

Wolter (2007)

M J Salganik (2006)

Jackknife algorithm

Wolter (2007)

Synonyms (optional)

Rubin D B (1987)

Wolter (2007)

7 Literature

1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









Measurement errors

Processing errors

Substitution errors

Analytical methods

( ) ()

and the MSE

( ) ( () )(sum

())sum

(x iminusX )2

Replication methods

V=( nminus1n )sum

( y(i)minus y )2()

() sum

() where

z iiquest i=1

V= 1mminus1sumi=1

is given as sum

() ( ) where sum

(())radic ()

( ) ( )

sum ()

) () ()

MSE (θ )= 1q2sum

(θkminusθ )2 ()

RB ( θ )=(sumk=1

θk q)minusθ

ERB (θ )=sumk=1

V=V W+q+1

qV B ()

where V W=sumk=1

V B=sumk=1

iisin A( y i

wi yi sumiisins

2 is the

V c=N s2

where s2=sumi=1

iquest=0 otherwise

y iiquest=sum

y iiquest n

T0)= (1+T t 0 )Var ( T t )+(T t 0+T t 0

T t 0=sumi=1

wiquestT iquest

sumi=1

wi 0T i 0

(()) (( )))()

3 Design issues

6 Glossary

Synonyms (optional)

Wolter (2007)

M J Salganik (2006)

Jackknife algorithm

Wolter (2007)

Synonyms (optional)

Rubin D B (1987)

Wolter (2007)

7 Literature

1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









Measurement errors

Processing errors

Substitution errors

Analytical methods

( ) ()

and the MSE

( ) ( () )(sum

())sum

(x iminusX )2

Replication methods

V=( nminus1n )sum

( y(i)minus y )2()

() sum

() where

z iiquest i=1

V= 1mminus1sumi=1

is given as sum

() ( ) where sum

(())radic ()

( ) ( )

sum ()

) () ()

MSE (θ )= 1q2sum

(θkminusθ )2 ()

RB ( θ )=(sumk=1

θk q)minusθ

ERB (θ )=sumk=1

V=V W+q+1

qV B ()

where V W=sumk=1

V B=sumk=1

iisin A( y i

wi yi sumiisins

2 is the

V c=N s2

where s2=sumi=1

iquest=0 otherwise

y iiquest=sum

y iiquest n

T0)= (1+T t 0 )Var ( T t )+(T t 0+T t 0

T t 0=sumi=1

wiquestT iquest

sumi=1

wi 0T i 0

(()) (( )))()

3 Design issues

6 Glossary

Synonyms (optional)

Wolter (2007)

M J Salganik (2006)

Jackknife algorithm

Wolter (2007)

Synonyms (optional)

Rubin D B (1987)

Wolter (2007)

7 Literature

1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









Measurement errors

Processing errors

Substitution errors

Analytical methods

( ) ()

and the MSE

( ) ( () )(sum

())sum

(x iminusX )2

Replication methods

V=( nminus1n )sum

( y(i)minus y )2()

() sum

() where

z iiquest i=1

V= 1mminus1sumi=1

is given as sum

() ( ) where sum

(())radic ()

( ) ( )

sum ()

) () ()

MSE (θ )= 1q2sum

(θkminusθ )2 ()

RB ( θ )=(sumk=1

θk q)minusθ

ERB (θ )=sumk=1

V=V W+q+1

qV B ()

where V W=sumk=1

V B=sumk=1

iisin A( y i

wi yi sumiisins

2 is the

V c=N s2

where s2=sumi=1

iquest=0 otherwise

y iiquest=sum

y iiquest n

T0)= (1+T t 0 )Var ( T t )+(T t 0+T t 0

T t 0=sumi=1

wiquestT iquest

sumi=1

wi 0T i 0

(()) (( )))()

3 Design issues

6 Glossary

Synonyms (optional)

Wolter (2007)

M J Salganik (2006)

Jackknife algorithm

Wolter (2007)

Synonyms (optional)

Rubin D B (1987)

Wolter (2007)

7 Literature

1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









Processing errors

Substitution errors

Analytical methods

( ) ()

and the MSE

( ) ( () )(sum

())sum

(x iminusX )2

Replication methods

V=( nminus1n )sum

( y(i)minus y )2()

() sum

() where

z iiquest i=1

V= 1mminus1sumi=1

is given as sum

() ( ) where sum

(())radic ()

( ) ( )

sum ()

) () ()

MSE (θ )= 1q2sum

(θkminusθ )2 ()

RB ( θ )=(sumk=1

θk q)minusθ

ERB (θ )=sumk=1

V=V W+q+1

qV B ()

where V W=sumk=1

V B=sumk=1

iisin A( y i

wi yi sumiisins

2 is the

V c=N s2

where s2=sumi=1

iquest=0 otherwise

y iiquest=sum

y iiquest n

T0)= (1+T t 0 )Var ( T t )+(T t 0+T t 0

T t 0=sumi=1

wiquestT iquest

sumi=1

wi 0T i 0

(()) (( )))()

3 Design issues

6 Glossary

Synonyms (optional)

Wolter (2007)

M J Salganik (2006)

Jackknife algorithm

Wolter (2007)

Synonyms (optional)

Rubin D B (1987)

Wolter (2007)

7 Literature

1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









Substitution errors

Analytical methods

( ) ()

and the MSE

( ) ( () )(sum

())sum

(x iminusX )2

Replication methods

V=( nminus1n )sum

( y(i)minus y )2()

() sum

() where

z iiquest i=1

V= 1mminus1sumi=1

is given as sum

() ( ) where sum

(())radic ()

( ) ( )

sum ()

) () ()

MSE (θ )= 1q2sum

(θkminusθ )2 ()

RB ( θ )=(sumk=1

θk q)minusθ

ERB (θ )=sumk=1

V=V W+q+1

qV B ()

where V W=sumk=1

V B=sumk=1

iisin A( y i

wi yi sumiisins

2 is the

V c=N s2

where s2=sumi=1

iquest=0 otherwise

y iiquest=sum

y iiquest n

T0)= (1+T t 0 )Var ( T t )+(T t 0+T t 0

T t 0=sumi=1

wiquestT iquest

sumi=1

wi 0T i 0

(()) (( )))()

3 Design issues

6 Glossary

Synonyms (optional)

Wolter (2007)

M J Salganik (2006)

Jackknife algorithm

Wolter (2007)

Synonyms (optional)

Rubin D B (1987)

Wolter (2007)

7 Literature

1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









Analytical methods

( ) ()

and the MSE

( ) ( () )(sum

())sum

(x iminusX )2

Replication methods

V=( nminus1n )sum

( y(i)minus y )2()

() sum

() where

z iiquest i=1

V= 1mminus1sumi=1

is given as sum

() ( ) where sum

(())radic ()

( ) ( )

sum ()

) () ()

MSE (θ )= 1q2sum

(θkminusθ )2 ()

RB ( θ )=(sumk=1

θk q)minusθ

ERB (θ )=sumk=1

V=V W+q+1

qV B ()

where V W=sumk=1

V B=sumk=1

iisin A( y i

wi yi sumiisins

2 is the

V c=N s2

where s2=sumi=1

iquest=0 otherwise

y iiquest=sum

y iiquest n

T0)= (1+T t 0 )Var ( T t )+(T t 0+T t 0

T t 0=sumi=1

wiquestT iquest

sumi=1

wi 0T i 0

(()) (( )))()

3 Design issues

6 Glossary

Synonyms (optional)

Wolter (2007)

M J Salganik (2006)

Jackknife algorithm

Wolter (2007)

Synonyms (optional)

Rubin D B (1987)

Wolter (2007)

7 Literature

1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









( ) ()

and the MSE

( ) ( () )(sum

())sum

(x iminusX )2

Replication methods

V=( nminus1n )sum

( y(i)minus y )2()

() sum

() where

z iiquest i=1

V= 1mminus1sumi=1

is given as sum

() ( ) where sum

(())radic ()

( ) ( )

sum ()

) () ()

MSE (θ )= 1q2sum

(θkminusθ )2 ()

RB ( θ )=(sumk=1

θk q)minusθ

ERB (θ )=sumk=1

V=V W+q+1

qV B ()

where V W=sumk=1

V B=sumk=1

iisin A( y i

wi yi sumiisins

2 is the

V c=N s2

where s2=sumi=1

iquest=0 otherwise

y iiquest=sum

y iiquest n

T0)= (1+T t 0 )Var ( T t )+(T t 0+T t 0

T t 0=sumi=1

wiquestT iquest

sumi=1

wi 0T i 0

(()) (( )))()

3 Design issues

6 Glossary

Synonyms (optional)

Wolter (2007)

M J Salganik (2006)

Jackknife algorithm

Wolter (2007)

Synonyms (optional)

Rubin D B (1987)

Wolter (2007)

7 Literature

1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









(x iminusX )2

Replication methods

V=( nminus1n )sum

( y(i)minus y )2()

() sum

() where

z iiquest i=1

V= 1mminus1sumi=1

is given as sum

() ( ) where sum

(())radic ()

( ) ( )

sum ()

) () ()

MSE (θ )= 1q2sum

(θkminusθ )2 ()

RB ( θ )=(sumk=1

θk q)minusθ

ERB (θ )=sumk=1

V=V W+q+1

qV B ()

where V W=sumk=1

V B=sumk=1

iisin A( y i

wi yi sumiisins

2 is the

V c=N s2

where s2=sumi=1

iquest=0 otherwise

y iiquest=sum

y iiquest n

T0)= (1+T t 0 )Var ( T t )+(T t 0+T t 0

T t 0=sumi=1

wiquestT iquest

sumi=1

wi 0T i 0

(()) (( )))()

3 Design issues

6 Glossary

Synonyms (optional)

Wolter (2007)

M J Salganik (2006)

Jackknife algorithm

Wolter (2007)

Synonyms (optional)

Rubin D B (1987)

Wolter (2007)

7 Literature

1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









V=( nminus1n )sum

( y(i)minus y )2()

() sum

() where

z iiquest i=1

V= 1mminus1sumi=1

is given as sum

() ( ) where sum

(())radic ()

( ) ( )

sum ()

) () ()

MSE (θ )= 1q2sum

(θkminusθ )2 ()

RB ( θ )=(sumk=1

θk q)minusθ

ERB (θ )=sumk=1

V=V W+q+1

qV B ()

where V W=sumk=1

V B=sumk=1

iisin A( y i

wi yi sumiisins

2 is the

V c=N s2

where s2=sumi=1

iquest=0 otherwise

y iiquest=sum

y iiquest n

T0)= (1+T t 0 )Var ( T t )+(T t 0+T t 0

T t 0=sumi=1

wiquestT iquest

sumi=1

wi 0T i 0

(()) (( )))()

3 Design issues

6 Glossary

Synonyms (optional)

Wolter (2007)

M J Salganik (2006)

Jackknife algorithm

Wolter (2007)

Synonyms (optional)

Rubin D B (1987)

Wolter (2007)

7 Literature

1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









z iiquest i=1

V= 1mminus1sumi=1

is given as sum

() ( ) where sum

(())radic ()

( ) ( )

sum ()

) () ()

MSE (θ )= 1q2sum

(θkminusθ )2 ()

RB ( θ )=(sumk=1

θk q)minusθ

ERB (θ )=sumk=1

V=V W+q+1

qV B ()

where V W=sumk=1

V B=sumk=1

iisin A( y i

wi yi sumiisins

2 is the

V c=N s2

where s2=sumi=1

iquest=0 otherwise

y iiquest=sum

y iiquest n

T0)= (1+T t 0 )Var ( T t )+(T t 0+T t 0

T t 0=sumi=1

wiquestT iquest

sumi=1

wi 0T i 0

(()) (( )))()

3 Design issues

6 Glossary

Synonyms (optional)

Wolter (2007)

M J Salganik (2006)

Jackknife algorithm

Wolter (2007)

Synonyms (optional)

Rubin D B (1987)

Wolter (2007)

7 Literature

1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









is given as sum

() ( ) where sum

(())radic ()

( ) ( )

sum ()

) () ()

MSE (θ )= 1q2sum

(θkminusθ )2 ()

RB ( θ )=(sumk=1

θk q)minusθ

ERB (θ )=sumk=1

V=V W+q+1

qV B ()

where V W=sumk=1

V B=sumk=1

iisin A( y i

wi yi sumiisins

2 is the

V c=N s2

where s2=sumi=1

iquest=0 otherwise

y iiquest=sum

y iiquest n

T0)= (1+T t 0 )Var ( T t )+(T t 0+T t 0

T t 0=sumi=1

wiquestT iquest

sumi=1

wi 0T i 0

(()) (( )))()

3 Design issues

6 Glossary

Synonyms (optional)

Wolter (2007)

M J Salganik (2006)

Jackknife algorithm

Wolter (2007)

Synonyms (optional)

Rubin D B (1987)

Wolter (2007)

7 Literature

1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









(())radic ()

( ) ( )

sum ()

) () ()

MSE (θ )= 1q2sum

(θkminusθ )2 ()

RB ( θ )=(sumk=1

θk q)minusθ

ERB (θ )=sumk=1

V=V W+q+1

qV B ()

where V W=sumk=1

V B=sumk=1

iisin A( y i

wi yi sumiisins

2 is the

V c=N s2

where s2=sumi=1

iquest=0 otherwise

y iiquest=sum

y iiquest n

T0)= (1+T t 0 )Var ( T t )+(T t 0+T t 0

T t 0=sumi=1

wiquestT iquest

sumi=1

wi 0T i 0

(()) (( )))()

3 Design issues

6 Glossary

Synonyms (optional)

Wolter (2007)

M J Salganik (2006)

Jackknife algorithm

Wolter (2007)

Synonyms (optional)

Rubin D B (1987)

Wolter (2007)

7 Literature

1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









(())radic ()

( ) ( )

sum ()

) () ()

MSE (θ )= 1q2sum

(θkminusθ )2 ()

RB ( θ )=(sumk=1

θk q)minusθ

ERB (θ )=sumk=1

V=V W+q+1

qV B ()

where V W=sumk=1

V B=sumk=1

iisin A( y i

wi yi sumiisins

2 is the

V c=N s2

where s2=sumi=1

iquest=0 otherwise

y iiquest=sum

y iiquest n

T0)= (1+T t 0 )Var ( T t )+(T t 0+T t 0

T t 0=sumi=1

wiquestT iquest

sumi=1

wi 0T i 0

(()) (( )))()

3 Design issues

6 Glossary

Synonyms (optional)

Wolter (2007)

M J Salganik (2006)

Jackknife algorithm

Wolter (2007)

Synonyms (optional)

Rubin D B (1987)

Wolter (2007)

7 Literature

1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









sum ()

) () ()

MSE (θ )= 1q2sum

(θkminusθ )2 ()

RB ( θ )=(sumk=1

θk q)minusθ

ERB (θ )=sumk=1

V=V W+q+1

qV B ()

where V W=sumk=1

V B=sumk=1

iisin A( y i

wi yi sumiisins

2 is the

V c=N s2

where s2=sumi=1

iquest=0 otherwise

y iiquest=sum

y iiquest n

T0)= (1+T t 0 )Var ( T t )+(T t 0+T t 0

T t 0=sumi=1

wiquestT iquest

sumi=1

wi 0T i 0

(()) (( )))()

3 Design issues

6 Glossary

Synonyms (optional)

Wolter (2007)

M J Salganik (2006)

Jackknife algorithm

Wolter (2007)

Synonyms (optional)

Rubin D B (1987)

Wolter (2007)

7 Literature

1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









sum ()

) () ()

MSE (θ )= 1q2sum

(θkminusθ )2 ()

RB ( θ )=(sumk=1

θk q)minusθ

ERB (θ )=sumk=1

V=V W+q+1

qV B ()

where V W=sumk=1

V B=sumk=1

iisin A( y i

wi yi sumiisins

2 is the

V c=N s2

where s2=sumi=1

iquest=0 otherwise

y iiquest=sum

y iiquest n

T0)= (1+T t 0 )Var ( T t )+(T t 0+T t 0

T t 0=sumi=1

wiquestT iquest

sumi=1

wi 0T i 0

(()) (( )))()

3 Design issues

6 Glossary

Synonyms (optional)

Wolter (2007)

M J Salganik (2006)

Jackknife algorithm

Wolter (2007)

Synonyms (optional)

Rubin D B (1987)

Wolter (2007)

7 Literature

1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









MSE (θ )= 1q2sum

(θkminusθ )2 ()

RB ( θ )=(sumk=1

θk q)minusθ

ERB (θ )=sumk=1

V=V W+q+1

qV B ()

where V W=sumk=1

V B=sumk=1

iisin A( y i

wi yi sumiisins

2 is the

V c=N s2

where s2=sumi=1

iquest=0 otherwise

y iiquest=sum

y iiquest n

T0)= (1+T t 0 )Var ( T t )+(T t 0+T t 0

T t 0=sumi=1

wiquestT iquest

sumi=1

wi 0T i 0

(()) (( )))()

3 Design issues

6 Glossary

Synonyms (optional)

Wolter (2007)

M J Salganik (2006)

Jackknife algorithm

Wolter (2007)

Synonyms (optional)

Rubin D B (1987)

Wolter (2007)

7 Literature

1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









RB ( θ )=(sumk=1

θk q)minusθ

ERB (θ )=sumk=1

V=V W+q+1

qV B ()

where V W=sumk=1

V B=sumk=1

iisin A( y i

wi yi sumiisins

2 is the

V c=N s2

where s2=sumi=1

iquest=0 otherwise

y iiquest=sum

y iiquest n

T0)= (1+T t 0 )Var ( T t )+(T t 0+T t 0

T t 0=sumi=1

wiquestT iquest

sumi=1

wi 0T i 0

(()) (( )))()

3 Design issues

6 Glossary

Synonyms (optional)

Wolter (2007)

M J Salganik (2006)

Jackknife algorithm

Wolter (2007)

Synonyms (optional)

Rubin D B (1987)

Wolter (2007)

7 Literature

1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









iisin A( y i

wi yi sumiisins

2 is the

V c=N s2

where s2=sumi=1

iquest=0 otherwise

y iiquest=sum

y iiquest n

T0)= (1+T t 0 )Var ( T t )+(T t 0+T t 0

T t 0=sumi=1

wiquestT iquest

sumi=1

wi 0T i 0

(()) (( )))()

3 Design issues

6 Glossary

Synonyms (optional)

Wolter (2007)

M J Salganik (2006)

Jackknife algorithm

Wolter (2007)

Synonyms (optional)

Rubin D B (1987)

Wolter (2007)

7 Literature

1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









wi yi sumiisins

2 is the

V c=N s2

where s2=sumi=1

iquest=0 otherwise

y iiquest=sum

y iiquest n

T0)= (1+T t 0 )Var ( T t )+(T t 0+T t 0

T t 0=sumi=1

wiquestT iquest

sumi=1

wi 0T i 0

(()) (( )))()

3 Design issues

6 Glossary

Synonyms (optional)

Wolter (2007)

M J Salganik (2006)

Jackknife algorithm

Wolter (2007)

Synonyms (optional)

Rubin D B (1987)

Wolter (2007)

7 Literature

1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









V c=N s2

where s2=sumi=1

iquest=0 otherwise

y iiquest=sum

y iiquest n

T0)= (1+T t 0 )Var ( T t )+(T t 0+T t 0

T t 0=sumi=1

wiquestT iquest

sumi=1

wi 0T i 0

(()) (( )))()

3 Design issues

6 Glossary

Synonyms (optional)

Wolter (2007)

M J Salganik (2006)

Jackknife algorithm

Wolter (2007)

Synonyms (optional)

Rubin D B (1987)

Wolter (2007)

7 Literature

1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









V c=N s2

where s2=sumi=1

iquest=0 otherwise

y iiquest=sum

y iiquest n

T0)= (1+T t 0 )Var ( T t )+(T t 0+T t 0

T t 0=sumi=1

wiquestT iquest

sumi=1

wi 0T i 0

(()) (( )))()

3 Design issues

6 Glossary

Synonyms (optional)

Wolter (2007)

M J Salganik (2006)

Jackknife algorithm

Wolter (2007)

Synonyms (optional)

Rubin D B (1987)

Wolter (2007)

7 Literature

1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









T0)= (1+T t 0 )Var ( T t )+(T t 0+T t 0

T t 0=sumi=1

wiquestT iquest

sumi=1

wi 0T i 0

(()) (( )))()

3 Design issues

6 Glossary

Synonyms (optional)

Wolter (2007)

M J Salganik (2006)

Jackknife algorithm

Wolter (2007)

Synonyms (optional)

Rubin D B (1987)

Wolter (2007)

7 Literature

1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









T0)= (1+T t 0 )Var ( T t )+(T t 0+T t 0

T t 0=sumi=1

wiquestT iquest

sumi=1

wi 0T i 0

(()) (( )))()

3 Design issues

6 Glossary

Synonyms (optional)

Wolter (2007)

M J Salganik (2006)

Jackknife algorithm

Wolter (2007)

Synonyms (optional)

Rubin D B (1987)

Wolter (2007)

7 Literature

1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









(()) (( )))()

3 Design issues

6 Glossary

Synonyms (optional)

Wolter (2007)

M J Salganik (2006)

Jackknife algorithm

Wolter (2007)

Synonyms (optional)

Rubin D B (1987)

Wolter (2007)

7 Literature

1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









6 Glossary

Synonyms (optional)

Wolter (2007)

M J Salganik (2006)

Jackknife algorithm

Wolter (2007)

Synonyms (optional)

Rubin D B (1987)

Wolter (2007)

7 Literature

1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









6 Glossary

Synonyms (optional)

Wolter (2007)

M J Salganik (2006)

Jackknife algorithm

Wolter (2007)

Synonyms (optional)

Rubin D B (1987)

Wolter (2007)

7 Literature

1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









6 Glossary

Synonyms (optional)

Wolter (2007)

M J Salganik (2006)

Jackknife algorithm

Wolter (2007)

Synonyms (optional)

Rubin D B (1987)

Wolter (2007)

7 Literature

1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









Synonyms (optional)

Rubin D B (1987)

Wolter (2007)

7 Literature

1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









1 User needs

3 Sample selection

4 Imputation

5 Weighting

6 Estimation

7 Response

2 Sample selection

3 Data imputation

1 Summary







3 Design issues



6 Glossary

7 Literature









Documents

Template for modules of the revised handbookec.europa.eu/eurostat/cros/system/files/20.02 Quality... · Web viewFrom the study by Hulliger and Münnich (2006) variance estimators