3 February 2012 Glossary 1 - United Nationsunstats.un.org/unsd/dnss/docs-nqaf/NQAF GLOSSARY.pdf · Source: • Methodological Documents - Glossary, Assessment of quality in statistics,

3 February 2012

Glossary1 (Compiled by the Expert Group on National Quality Assurance Frameworks)

ACCESSIBILITY ............................................................................................................................................................................. 1

ACCURACY ..................................................................................................................................................................................... 1

ADMINISTRATIVE DATA ............................................................................................................................................................ 1

BENCHMARKING .......................................................................................................................................................................... 1

BIAS................................................................................................................................................................................................... 1

CERTIFICATION............................................................................................................................................................................ 2

CLARITY.......................................................................................................................................................................................... 2

CODING............................................................................................................................................................................................ 2

COHERENCE................................................................................................................................................................................... 2

COMPARABILITY.......................................................................................................................................................................... 2

COMPLETENESS............................................................................................................................................................................ 3

CONFIDENTIALITY ...................................................................................................................................................................... 3

CONSISTENCY................................................................................................................................................................................ 3

CONVERSION RATE ..................................................................................................................................................................... 3

COST ................................................................................................................................................................................................. 4

COST-EFFECTIVENESS ............................................................................................................................................................... 4

COST-BENEFIT ANALYSIS.......................................................................................................................................................... 4

CREDIBILITY.................................................................................................................................................................................. 4

DATA ANONYMIZATION............................................................................................................................................................. 4

DATA CAPTURE............................................................................................................................................................................. 4

DATA CHECKING.......................................................................................................................................................................... 5

DATA EDITING............................................................................................................................................................................... 5

DATA REVISION ............................................................................................................................................................................ 5

DATA VALIDATION ...................................................................................................................................................................... 5

DEVELOPMENT OF A SELF-ASSESSMENT PROGRAMME (DESAP) ............................................................................... 6

DISSEMINATION............................................................................................................................................................................ 6

DISSEMINATION STANDARD..................................................................................................................................................... 6

DOCUMENTATION........................................................................................................................................................................ 6

ESTIMATE ....................................................................................................................................................................................... 6

ETHICAL STANDARDS................................................................................................................................................................. 7

EUROPEAN STATISTICS CODE OF PRACTICE (COP)......................................................................................................... 7

EUROSTAT STATISTICAL PROCESSES ASSESSMENT CHECKLIST (ESPAC) .............................................................. 7

FRAME.............................................................................................................................................................................................. 7

FRAME ERROR .............................................................................................................................................................................. 7

GENERIC STATISTICAL BUSINESS PROCESS MODEL (GSBPM)..................................................................................... 8

GOOD PRACTICES ........................................................................................................................................................................ 8

GOVERNANCE STRUCTURE ...................................................................................................................................................... 8

GUIDELINES ................................................................................................................................................................................... 8

IMPARTIALITY .............................................................................................................................................................................. 8

IMPUTATION .................................................................................................................................................................................. 8

INSTITUTIONAL ENVIRONMENT............................................................................................................................................. 8

INTEGRITY...................................................................................................................................................................................... 9

INTERMEDIATE RESULTS.......................................................................................................................................................... 9

INTERNATIONAL STATISTICAL PROCESSES ASSESSMENT CHECKLIST (ISPAC) ................................................... 9

LABELLING..................................................................................................................................................................................... 9

METADATA ................................................................................................................................................................................... 10

METHODOLOGICAL SOUNDNESS.......................................................................................................................................... 10

MICRODATA................................................................................................................................................................................. 10

NATIONAL STATISTICAL SYSTEM (NSS) ............................................................................................................................. 10

NON-RESPONSE ........................................................................................................................................................................... 10

NON-RESPONSE ERROR............................................................................................................................................................ 11

NON-RESPONSE RATE ............................................................................................................................................................... 11

NON-SAMPLING ERROR ........................................................................................................................................................... 11

OBJECTIVITY............................................................................................................................................................................... 11

OBSERVATION STATUS ............................................................................................................................................................ 12

OUTPUT (PRODUCT) QUALITY............................................................................................................................................... 12

PARADATA.................................................................................................................................................................................... 12

PEER REVIEW .............................................................................................................................................................................. 12

PRE-RELEASE ACCESS.............................................................................................................................................................. 12

PREREQUISITES OF QUALITY ................................................................................................................................................ 12

PROCESS DESCRIPTION ........................................................................................................................................................... 13

PROCESS METADATA................................................................................................................................................................ 13

PROCESS QUALITY .................................................................................................................................................................... 13

PROCESS VARIABLE .................................................................................................................................................................. 13

PROCESSING ERROR ................................................................................................................................................................. 13

PROFESSIONAL INDEPENDENCE........................................................................................................................................... 13

PROFESSIONALISM.................................................................................................................................................................... 14

PROVIDER LOAD......................................................................................................................................................................... 14

PUNCTUALITY ............................................................................................................................................................................. 14

QUALITY........................................................................................................................................................................................ 14

QUALITY ASSESSMENT ............................................................................................................................................................ 15

QUALITY ASSURANCE .............................................................................................................................................................. 15

QUALITY ASSURANCE FRAMEWORK.................................................................................................................................. 15

QUALITY AUDIT.......................................................................................................................................................................... 15

QUALITY COMPONENTS .......................................................................................................................................................... 15

QUALITY CONTROL................................................................................................................................................................... 15

QUALITY CULTURE ................................................................................................................................................................... 16

QUALITY DIMENSIONS ............................................................................................................................................................. 16

QUALITY FRAMEWORK ........................................................................................................................................................... 16

QUALITY INDICATORS ............................................................................................................................................................. 16

QUALITY MANAGEMENT......................................................................................................................................................... 16

QUALITY MANAGEMENT SYSTEM ....................................................................................................................................... 17

QUALITY POLICY ....................................................................................................................................................................... 17

QUALITY REPORT ...................................................................................................................................................................... 17

QUALITY TARGETS.................................................................................................................................................................... 17

RELEVANCE ................................................................................................................................................................................. 17

RECORD LINKAGE ..................................................................................................................................................................... 18

REFERENCE PERIOD ................................................................................................................................................................. 18

REFUSAL RATE............................................................................................................................................................................ 18

REGISTER...................................................................................................................................................................................... 18

RELIABILITY................................................................................................................................................................................ 19

RESPONDENT BURDEN ............................................................................................................................................................. 19

REVISION POLICY ...................................................................................................................................................................... 19

ROLLING REVIEW ...................................................................................................................................................................... 19

SAMPLING ERROR...................................................................................................................................................................... 19

SECURITY (DATA SECURITY) ................................................................................................................................................. 20

SELF-ASSESSMENT..................................................................................................................................................................... 20

SERVICEABILITY........................................................................................................................................................................ 20

STAKEHOLDERS ......................................................................................................................................................................... 20

STANDARDS .................................................................................................................................................................................. 20

STATISTICAL (PRODUCTION/BUSINESS) PROCESS ......................................................................................................... 20

STATISTICAL DATA ................................................................................................................................................................... 21

STATISTICAL DATA AND METADATA EXCHANGE (SDMX) .......................................................................................... 21

STATISTICAL OUTPUT .............................................................................................................................................................. 21

STATISTICAL STANDARD......................................................................................................................................................... 21

STATISTICAL UNIT..................................................................................................................................................................... 21

SUPPORTED SELF-ASSESSMENT............................................................................................................................................ 22

TARGET POPULATION .............................................................................................................................................................. 22

TIMELINESS.................................................................................................................................................................................. 22

TOTAL QUALITY MANAGEMENT (TQM)............................................................................................................................. 22

TRANSPARENCY ......................................................................................................................................................................... 23

USER................................................................................................................................................................................................ 23

USER NEEDS ................................................................................................................................................................................. 23

USER SATISFACTION................................................................................................................................................................. 23

USER SATISFACTION SURVEY................................................................................................................................................ 23

1Note: The quality-related terms included in this Glossary consist mainly of those that appear in the Template for a Generic National Quality Assurance Framework and its accompanying Guidelines, especially those that are mentioned in section 3 of the Template or are directly related to its coded lines. Therefore, the Glossary’s coverage of terms is not exhaustive, nor is it intended to be. The main source for the current entries is the SDMX’s Metadata Common Vocabulary that was developed by a partnership of international organizations. Since the SDMX Metadata Common Vocabulary is an agreed global standard, preference is given to the definitions it presents in cases where several definitions may be available for the same terms. The Glossary will continue to be updated by UNSD when additional new and relevant information is provided or identified.

1

Accessibility

Definition: The ease and conditions under which statistical information can be obtained. Context: Accessibility refers to the availability of statistical information to the user. It includes the ease with

which the existence of information can be ascertained, as well as the suitability of the form or medium through which the information can be accessed. The cost of the information may also be an aspect of accessibility for some users. Accessibility refers to the physical conditions in which users can obtain data: where to go, how to order, delivery time, clear pricing policy, convenient marketing conditions (copyright, etc.), availability of micro or macro data, various formats (paper, files, CD-ROM, Internet), etc.

Source: • SDMX (2009) Hyperlinks: • http://www.sdmx.org/

Accuracy

Definition: Closeness of computations or estimates to the exact or true values that the statistics were intended to measure.

Context: The accuracy of statistical information is the degree to which the information correctly describes the phenomena. It is usually characterized in terms of error in statistical estimates and is often decomposed into bias (systematic error) and variance (random error) components. Accuracy can contain either measures of accuracy (numerical results of the methods for assessing the accuracy of data) or qualitative assessment indicators. It may also be described in terms of the major sources of error that potentially cause inaccuracy (e.g., coverage, sampling, non response, response error). Accuracy is associated with the "reliability" of the data, which is defined as the closeness of the initial estimated value to the subsequent estimated value. This concept can be broken down into: Accuracy - overall (summary assessment); Accuracy - non-sampling error; Accuracy - sampling error.


Administrative data

Definition: Information primarily collected for the purpose of record-keeping, which is subsequently used to produce statistics.

Context: Some examples include data from registrars, hospital morbidity data, housing assistance data and child protection data.

Source: • Data Quality Online - Glossary of Terms Hyperlinks: • http://www.nss.gov.au/dataquality/glossary.jsp

Benchmarking

Definition: Comparing data, metadata or processes against a recognised standard. Context: Benchmarking may refer, for instance, to the case where there are two sources of data for the same

target variable with different frequencies, e.g. quarterly and annual estimates of value-added from different sources. Benchmarking is generally done retrospectively, as annual benchmark data are available some time after quarterly data. Benchmarking does have a forward-looking element however, in that the relationship between benchmark and indicator data is extrapolated forward to improve quarterly estimates for the most recent periods for which benchmark data are not yet available (Maitland-Smith, F, "Use of Benchmark Data to Align or Derive Quarterly/Monthly Estimates", paper presented at the June 2002 meeting of the OECD Short-term Economic Statistics Working Party, Paris).


Bias

Definition: An effect which deprives a statistical result of representativeness by systematically distorting it, as distinct from a random error which may distort on any one occasion but balances out on the average.

Context: The bias of an estimator is the difference between its mathematical expectation and the true value it estimates. In the case it is zero, the estimator is said to be unbiased.

Source: • The International Statistical Institute, "The Oxford Dictionary of Statistical Terms", edited by Yadolah Dodge, Oxford University Press, 2003

Hyperlinks: • Above taken from SDMX (2009) http://www.sdmx.org/; hyperlink to original source(s) unavailable.

2

Certification

Definition: Activity which assesses whether a particular product, service, process or system (e.g. quality management system) complies with requirements defined by a standard or other document containing criteria.

Context: It is conducted by an external independent certification body. The result of the successful certification is the certificate awarded to the organisation by the certification body. Applies when a suitably qualified external independent body accepts that the particular product, service, process or systems meets a pre-defined standard.

Source: • Based on ISO definition, ISO 9000/2005: Quality Management and Quality Assurance Vocabulary Hyperlinks: • Above taken from ESS Quality Glossary 2010,

http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/ESS%20Quality%20Glossary%202010.pdf ; hyperlink to original source(s) unavailable.

Clarity

Definition: The extent to which easily comprehensible metadata are available, where these metadata are necessary to give a full understanding of statistical data.

Context: Clarity is sometimes referred to as "interpretability". It refers to the data information environment: whether data are accompanied by appropriate metadata, including information on their quality, and the extent to which additional assistance is provided to users by data providers. In the European Statistics Code of Practice, clarity is strictly associated to accessibility to form one single quality criteria: "accessibility and clarity".


Coding

Definition: The process of converting verbal or textual information into codes representing classes within a classification scheme, to facilitate data processing, storage or dissemination.

Context: Source: • SDMX (2009) Hyperlinks: • http://www.sdmx.org/

Coherence

Definition: Adequacy of statistics to be combined in different ways and for various uses. Context: When originating from different sources, and in particular from statistical surveys using different

methodology, statistics are often not completely identical, but show differences in results due to different approaches, classifications and methodological standards. There are several areas where the assessment of coherence is regularly conducted: between provisional and final statistics, between annual and short-term statistics, between statistics from the same socio-economic domain, and between survey statistics and national accounts. The concept of coherence is closely related to the concept of comparability between statistical domains. Both coherence and comparability refer to a data set with respect to another. The difference between the two is that comparability refers to comparisons between statistics based on usually unrelated statistical populations and coherence refers to comparisons between statistics for the same or largely similar populations. Coherence can be generally broken down into "Coherence - cross domain" and "Coherence - internal". Users should be aware that, in the Data Quality Assessment Framework of the International Monetary Fund, the term "consistency" is used for indicating "logical and numerical coherence". In that framework, "internal consistency" and "intersectoral and cross-domain consistency" can be mapped to "internal coherence" and "cross-domain coherence" respectively.


Comparability

Definition: The extent to which differences between statistics can be attributed to differences between the true values of the statistical characteristics.

Context: Comparability aims at measuring the impact of differences in applied statistical concepts and definitions on the comparison of statistics between geographical areas, non-geographical dimensions, or over time. Comparability of statistics, i.e. their usefulness in drawing comparisons and contrast among different populations, is a complex concept, difficult to assess in precise or absolute terms. In general terms, it means that statistics for different populations can be legitimately aggregated, compared and interpreted in relation to each other or against some common standard. Metadata must

3

convey such information that will help any interested party in evaluating comparability of the data, which is the result of a multitude of factors. In some quality frameworks, for instance in the European Statistical Code of Practice, comparability is strictly associated with the coherence of statistics. The concept can be further broken down into: (a) Comparability - geographical, referring to the degree of comparability between statistics measuring the same phenomenon for different geographical areas. (b) Comparability over time, referring to the degree of comparability between two or more instances of data on the same phenomenon measured at different points in time. (c) Comparability between domains, referring to the comparability between different survey results which target similar characteristics in different statistical domains.


Completeness

Definition: The extent to which all statistics that are needed are available. Context: Completeness is usually described as a measure of the amount of available data from a statistical

system compared to the amount that was expected to be obtained. Source: • Methodological Documents - Glossary, Assessment of quality in statistics, Luxembourg, 2003 Hyperlinks: • http://unstats.un.org/unsd/dnss/docs-nqaf/Eurostat-GLOSSARY_1.pdf

Confidentiality

Definition: A property of data indicating the extent to which their unauthorised disclosure could be prejudicial or harmful to the interest of the source or other relevant parties.

Context: Confidentiality refers to a property of data with respect to whether, for example, they are public or their disclosure is subject to restrictions. For instance, data allowing the identification of a physical or legal person, either directly or indirectly, may be characterised as confidential according to the relevant national or international legislation. Unauthorised disclosure of data that are restricted or confidential is not permitted and even legislative measures or other formal provisions may be used to prevent disclosure. Often, there are procedures in place to prevent disclosure of restricted or confidential data, including rules applying to staff, aggregation rules when disseminating data, provision of unit records, etc. Sensitivity (of information) is sometimes used as a synonym to confidentiality. This concept can be broken down into: Confidentiality - policy; Confidentiality - status; Confidentiality - data treatment.


Consistency

Definition: Logical and numerical coherence. Context: An estimator is called consistent if it converges in probability to its estimand as sample increases (The

International Statistical Institute, "The Oxford Dictionary of Statistical Terms", edited by Yadolah Dodge, Oxford University Press, 2003). Consistency over time, within datasets, and across datasets (often referred to as inter-sectoral consistency) are major aspects of consistency. In each, consistency in a looser sense carries the notion of "at least reconcilable". For example, if two series purporting to cover the same phenomena differ, the differences in time of recording, valuation, and coverage should be identified so that the series can be reconciled. Inconsistency over time refers to changes that lead to breaks in series stemming from, for example, changes in concepts, definitions, and methodology. Inconsistency within datasets may exist, for example, when two sides of an implied balancing statement - assets and liabilities or inflows and outflows - do not balance. Inconsistency across datasets may exist when, for example, exports and imports in the national accounts do not reconcile with exports and imports within the balance of payments. Within the IMF definition of quality, "consistency" is one of the elements of "serviceability".


Conversion rate Definition: The conversion rate is defined as the number of successful refusal conversions divided by the total

successful and unsuccessful attempts. Context: Refusal conversions are the procedures that survey researchers use to gain cooperation from a

sampled respondent who has refused an initial survey request. Refusal conversion may include different versions of the survey introductions and other written scripts or materials (e.g. cover letters),

4

study contact rules, incentives, and interviewer characteristics and training. This is a common procedure for many surveys, but it requires careful consideration of the details of the refusal conversion efforts and the potential costs versus the potential benefits of the effort. The goal of converting initial refusals is to raise the survey response rate, under the assumption that this may lower the potential for refusal-related unit non-response error.

Source: • Proceedings of the Annual Meeting of the American Statistical Association, August 5-9, 2001 - How Long Should You Wait Before Attempting to Convert a Telephone Refusal?

Hyperlinks: • http://www.amstat.org/sections/srms/proceedings/y2001/Proceed/00288.pdf

Cost

Definition: Cost associated with the collection and production of a statistical product.

Context: The cost is associated with a statistical product and can be financial, human or time-related. It may consist of staff costs, data collection costs and other costs related to reporting obligations.


Cost-effectiveness

Definition: Characteristic of a process where the costs of producing the statistics are in proportion to the importance of the results and the benefits sought, the resources are optimally used and the response burden minimised.

Context: Where possible, the information requested is readily extractable from available records or sources. All statistical authorities operate in a context where the demand for statistics or services exceeds the capacity of the office to deliver them. Likewise, reporting load on respondents should always be minimized consistent with collecting only the data necessary for the statistical compilation.

Source: • Regulation (EC) No 223/2009 of the European Parliament and of the Council of 11 March 2009 on European statistics

Hyperlinks: • http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2009:087:0164:0173:EN:PDF

Cost-benefit analysis

Definition: A technique for deciding whether to make a change. As its name suggests, it compares the values of all benefits from the action under consideration and the costs associated with it.

Context: Source: • United Nations, European Commission, International Monetary Fund, Organisation for Economic

Co-operation and Development, World Bank , 2005, Handbook of National Accounting: Integrated Environmental and Economic Accounting 2003, Studies in Methods, Series F, No.61, Rev.1, Glossary, United Nations, New York, para. 1.87

Hyperlinks: • http://stats.oecd.org/glossary/detail.asp?ID=6377

Credibility

Definition: Confidence that users place in statistical products based simply on their image of the data producer, the statistical authority i.e., the brand image.

Context: Credibility depends upon the extent to which data are perceived to be produced professionally in accordance with appropriate statistical standards with transparent policies and practices. Where lacking, doubts may arise in the minds of the users about the quality of the statistics being produced by the authority.

Source: • OECD, "Quality Framework for OECD Statistics" (2003) Hyperlinks: • http://ec.europa.eu/eurostat/ramon/coded_files/Quality_Frame_OECD_Stat_Act.pdf

Data anonymization

Definition: The process of removing or modifying the identifying variables contained in the dataset. Context: Identifying variables include: direct identifiers such as names, addresses, or registration numbers; and

Indirect identifiers, which are characteristics that may be shared by several respondents, and whose combination could lead to the re-identification of one of them.

Source: • Based on International Household Survey Network Tools and Guidelines on Data Anonymization Hyperlinks: • http://www.internationalsurveynetwork.org/HOME/index.php?q=tools/anonymization

Data capture

Definition: The process by which collected data are put into a machine-readable form. Context: Elementary edit checks are often performed in sub-modules of the software that does data capture.

5

Source: • Commission for Europe of the United Nations (UNECE), "Glossary of Terms on Statistical Data Editing", Conference of European Statisticians Methodological material, Geneva, 2000

Hyperlinks: • http://www.unece.org/stats/publications/editingglossary.pdf

Data checking

Definition: Activity through which the correctness conditions of the data are verified. Context: It also includes the specification of the type of error or condition not met, and the qualification of the

data and its division into the "error free" and "erroneous data". Data checking may be aimed at detecting error-free data or at detecting erroneous data.

Source: • Economic Commission for Europe of the United Nations (UNECE), "Glossary of Terms on Statistical Data Editing", Conference of European Statisticians Methodological material, Geneva, 2000

Hyperlinks: • http://www1.unece.org/stat/platform/download/attachments/15007777/SDE+glossary+2000.pdf?version=1

Data editing

Definition: Activity aimed at detecting and correcting errors, logical inconsistencies and suspicious data.

Context: Editing techniques refer to a range of procedures and processes used for detecting and handling errors in data, also aiming at avoiding their future repetition. An "edit" is the correction of an error in data. An "editing rule" is the specification of the conditions under which edits are made. Examples of different techniques include the different approaches to editing such as micro-editing/ macro-editing, input/output editing, or to the various tools available for editing such as graphical editing, interactive editing, etc. Edit types refer to the actual nature of edits applied to data during input or output processing. Examples include: - logical edits, to ensure that two or more data items do not have contradictory values; - consistency edits, to ensure that precise and correct arithmetic relationships exists between two or more data items; - range edits, identifying whether or not a data item value falls inside a determined acceptable range; - variance edits, looking for suspiciously high variances at the output edit stage. Micro-editing and macro-editing may be distinguished in order to calculate rate of edits. Edit types may also refer to whether these edits are fatal or query type, i.e. whether they detect errors with certainty or point to suspicious data items.


Hyperlinks: • http://www1.unece.org/stat/platform/download/attachments/15007777/SDE+glossary+2000.pdf?version=1

Data revision

Definition: Any change in a value of a statistic released to the public by an official statistical agency. Context: Preliminary data are revised when more and better source data become available. "Data revision"

describes the policy and practice for identifying the revision status of the data, as well as the availability of revision studies and analyses. This concept can be broken down into: "Data revision - policy"; "Data revision - practice"; "Data revision - studies". "Revision policy" refers to the policy aimed at ensuring the transparency of disseminated data. The general guidelines for handling data revisions applied by a data providing agency should be described. "Revision practice" refers to documentation regarding the source data used and the way they are adjusted. It also describes the revision status of available data. Data may also be subject to regular or ad hoc revisions as a result of the introduction of new classifications, compilation frameworks and methodologies which result in the compilation of historical data that replaces previously released data. "Revision studies" refers to the information about available studies and analyses on data revision. These studies can contain mean revision and revision variance in estimates or other quantitative measures of the effects of revisions.


Data validation

Definition: Process of monitoring the results of data compilation and ensuring the quality of the statistical results. Context: Data validation describes methods and processes for assessing statistical data, and how the results of

the assessments are monitored and made available to improve statistical processes. All the controls made in terms of quality of the data to be published or already published are included in the validation process. Validation also takes into account the results of studies and analysis of revisions and how

6

they are used to improve statistical processes. In this process, two dimensions can be distinguished: (i) validation before publication of the figures and (ii) validation after publication. This concept can be further broken down into "Data validation: intermediate", "Data validation: output" and "Data validation: source".


Development of a Self-Assessment Programme (DESAP)

Definition: A generic checklist for a systematic quality assessment of surveys in the European Statistical System. Context: The DESAP checklilst is a tool for survey managers, to support them in assessing the quality of their

statistics and considering improvement measures. Source: • DESAP - The European Self Assessment Checklist for Survey Managers Hyperlinks: • http://unstats.un.org/unsd/dnss/docs-nqaf/Eurostat-desap%20G0-LEG-20031010-EN.pdf

Dissemination

Definition: Distribution or transmission of statistical data and metadata to users. Context: Dissemination covers all activities by statistical producers aiming at making data and metadata

accessible to users. For data dissemination, various release media are possible, such as electronic format including the internet, CD-ROM, paper publications, files available to authorised users or for public use; fax response to a special request, public speeches, press releases.

Source: • Expert Group on NQAF based on SDMX(2009) • Economic Commission for Europe of the United Nations (UNECE), "Terminology on Statistical

Metadata", Conference of European Statisticians Statistical Standards and Studies, No. 53, Geneva, 2000

Hyperlinks: • http://unstats.un.org/unsd/dnss/QualityNQAF/nqaf.aspx • http://www1.unece.org/stat/platform/download/attachments/9110092/Metadata+terminology+2000.pd

f?version=1

Dissemination standard

Definition: Guideline or legally-binding prescription developed by international organisations for guiding the dissemination of statistics to the public.

Context: The formulation of data and metadata dissemination standards entails a consultation process between international organisations and with member countries. An example of a dissemination standard is the Special Data Dissemination Standard (SDDS) guiding members of the International Monetary Fund in the dissemination to the public of comprehensive, timely, accessible, and reliable economic and financial statistics.


Documentation

Definition: Descriptive text used to define or describe an object, design, specification, instructions or procedure. Context: Documentation is essential in providing instructions to data processors and to ensure a historical record is

maintained so that reproducibility is possible. At least some of the documentation may also be useful to users to inform them about attributes of the statistics and the processes used to compile them.

Source: • Economic Commission for Europe of the United Nations (UNECE), "Terminology on Statistical Metadata", Conference of European Statisticians Statistical Standards and Studies, No. 53, Geneva, 2000

Hyperlinks: • http://www1.unece.org/stat/platform/download/attachments/9110092/Metadata+terminology+2000.pdf?version=1

Estimate

Definition: The particular value yielded by an estimator in a given set of circumstances. Context: The expression is widely used to denote the rule by which such particular values are calculated. It

seems preferable to use the word "estimator" for the rule of procedure, and "estimate" for the values to which it leads in particular cases.



7

Ethical standards

Definition: Principles or rules describing practices or behaviour which is morally acceptable to society at a given time.

Context: Element 1.3 of IMF's DQAF. Under "Assurances of integrity", it is stated that policies and practices are guided by ethical standards. Ethical standards refer to guidelines for staff behaviour that are in place and are well known to the staff, which are an important aspect in assuring integrity and trust. There is no one consistent set of standards that all institutions follow, but each institution has the right to develop the standards that are meaningful for their organization. Ethical standards are not always easily enforceable, as they are frequently vaguely defined and somewhat open to interpretation. They are never universal and are always changing.

Source: • BusinessDictionary.com Hyperlinks: • http://www.businessdictionary.com/definition/ethical-standards.html

European Statistics Code of Practice (CoP)

Definition: European Statistical System's (ESS) quality framework, providing a structure for supporting improvements of quality for the ESS.

Context: The CoP is based on 15 Principles covering the institutional environment, the statistical processes and the output of statistics.

Source: • European Statistics Code of Practice Hyperlinks: • http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/code_practice.pdf

Eurostat Statistical Processes Assessment Checklist (ESPAC)

Definition: The European version of ISPAC (see also ISPAC). Context: A version of ISPAC tailored to Eurostat's specific needs.

Source: • Eurostat Hyperlinks: • http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/ESPAC%2011034_18616_200

9_EN_2.pdf

Frame

Definition: A list, map or other specification of the units which define a population to be completely enumerated or sampled.

Context: The frame consists of previously available descriptions of the objects or material related to the physical field in the form of maps, lists, directories, etc., from which sampling units may be constructed and a set of sampling units selected (Eurostat, "Assessment of Quality in Statistics: Glossary", Working Group, Luxembourg, October 2003). The frame may or may not contain information about the size or other supplementary information about the units, but should have enough details so that a unit, if included in the sample, may be located and taken up for inquiry. The nature of the frame exerts a considerable influence over the structure of a sample survey. It is rarely perfect, and may be inaccurate, incomplete, inadequately described, out of date or subject to some degree of duplication. Reasonable reliability in the frame is a desirable condition for the reliability of a sample survey based on it.



Frame error

Definition: Error caused by imperfections in the frame (business register, population register, area sample, etc.) from which units are selected for inclusion in surveys or censuses.

Context: Frame errors cover: - coverage errors - erroneous inclusions, omissions and duplications; - classification errors - units not classified, or misclassified by industry, geography or size; - contact errors - units with incomplete or incorrect contact data. A frame error consists of a difference in the information presented in the frame and the information as it should be. There are various reasons for the differences between this image and the real world. The sources of information used to maintain and update the frame will generally contain irregularities of some sort. The frame may be subject to certain lags in the recording of real world events, or it may have gaps due to the lack of adequate sources for certain types of information. If these distortions of the real world are considered to be acceptable by users of the frame, they should not be considered to be errors. If they are not acceptable, procedures or sources need to be changed or improved.

Source: • Expert Group on NQAF Hyperlinks: • http://unstats.un.org/unsd/dnss/QualityNQAF/nqaf.aspx

8

Generic Statistical Business Process Model (GSBPM)

Definition: A flexible tool to describe and define the set of business processes needed to produce official statistics. Context: Useful in deciding on arrangements for the conduct of particular statistical activities and to assess

where quality control processes can built in. Source: • Generic Statistical Business Process Model, Version 4.0 – April 2009, paper for the Joint

UNECE/Eurostat/OECD Work Session on Statistical Metadata (METIS), page 28 Hyperlinks: • www.unece.org/stats/gsbpm

Good practices

Definition: Processes or methodologies that have been proven in practice to be effective ways of achieving a specific objective.

Context: Good practices are known or believed to have a positive influence on the quality of the result, and are therefore recommended as a model. In official documents, the "good practices" are viewed as something more general than guidelines.


Governance structure

Definition: The overall legal and administrative arrangements under which accountabilities and responsibilities are established for the conduct of a statistical activity.

Context: Referred to in section 5 (Quality and Performance Management and Improvement) of the proposed template for a generic national quality assurance framework.


Guidelines

Definition: Directions or principles used in the development, maintenance and application of rules. Context: Guidelines may or may not be necessarily mandatory, but are provided as an aid to interpretation and

use of rules. Source: • SDMX (2009) Hyperlinks: • http://www.sdmx.org/

Impartiality

Definition: An attribute confirming that statistical authorities develop, produce and disseminate statistics in a neutral manner, and that all users must be given equal treatment.

Context: A critical pre-requisite for production of high quality statistics. The requirement for impartiality in statistical authorities is stated in the European Statistical Law and in the European Statistics Code of Practice.



Imputation

Definition: Procedure for entering a value for a specific data item where the response is missing or unusable. Context: Imputation is the process used to determine and assign replacement values for missing, invalid or

inconsistent data. This can be done by changing some of the responses or assigning values when they are missing on the record being edited to ensure that estimates are of high quality and that a plausible, internally consistent record is created. (Statistics Canada, "Statistics Canada Quality Guidelines", 4th edition, October 2003, page 41, available at: http://www.statcan.ca:8096/bsolc/english/bsolc?catno=12-539-X&CHROPG=1)


Hyperlinks: • http://www.unece.org/fileadmin/DAM/stats/publications/editingglossary.pdf

Institutional environment

Definition: Set of rules and the organisational structures that are used as the basis for producing statistics. Context: Quality is the degree to which a set of inherent characteristics fulfils requirements.

According to the European Statistics Code of Practice ("Code"), quality is determined by three major

9

factors: the institutional environment, the statistical processes and the statistical output. The Code distinguishes between six quality components for the institutional environment: - professional independence - mandate for data collection - adequacy of resources - quality commitment - statistical confidentiality - impartiality and objectivity

Source: • ISO 9000/2005: Quality Management and Quality Assurance Vocabulary

Hyperlinks: • Above taken from ESS Quality Glossary 2010, http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/ESS%20Quality%20Glossary%202010.pdf ; hyperlink to original source(s) unavailable.

Integrity

Definition: Values and related practices that maintain confidence in the eyes of users in the agency producing statistics and ultimately in the statistical product.

Context: Under the SDDS framework, "integrity" is the third of four dimensions of the standard (i.e., data, access, integrity, and quality) for which evidence of a subscribing member's observance of the standard can be obtained. Integrity refers to the description of the policy on the availability of the terms and conditions under which statistics are collected, processed, and disseminated. It also describes the policy of providing advanced notice of major changes in methodology, source data, and statistical techniques; the policy on internal governmental access to statistics prior to their release; the policy on statistical products’ identification. One important aspect, in integrity, is the trust in the objectivity of statistics. It implies that professionalism should guide policies and practices and it is supported by ethical standards and by transparency of policies and practices.

Source: • International Monetary Fund, "Data Quality Assessment Framework - DQAF - Glossary", unpublished


Intermediate results

Definition: A quantity or value derived from a statistical operation performed in the course of a statistical program which is used as an input to further operations.

Context: Source: • McGraw-Hill Dictionary of Scientific & Technical Terms, 6E, Copyright © 2003 by The McGraw-Hill

Companies, Inc. Hyperlinks:

International Statistical Processes Assessment Checklist (ISPAC)

Definition: A generic checklist for the systematic quality assessment of statistics which was compiled by international and supranational organisations building on the European Self Assessment Checklist for Survey Managers (DESAP) which was developed for assessing the quality of data provided by national statistical systems.

Context: ISPAC focuses on the processes of transformation applied by the international or supranational organisations in order to meet current and future information needs. It has been developed within the Committee for Coordination of Statistical Activities (CCSA) project on the use and convergence of international quality assurance frameworks, coordinated by Eurostat. A version tailored to Eurostat's specific needs (ESPAC) is maintained. The checklist's goal is to foster improvements of the quality of the produced statistics. It can be used to promote and assess the compliance with the Principles Governing International Statistical Activities, and covers principles 1, 4, 5, 6 and 7.

Source: • Revised International Statistical Processes Assessment Checklist, 2009, CCSA and Eurostat • Checklist for Quality Assessment of Statistics compiled by International and Supranational

Organisations, 2007, CCSA and Eurostat Hyperlinks: • http://unstats.un.org/unsd/accsub/2009docs-14th/SA-2009-12-Add3-CondensedChecklist.pdf

• http://unstats.un.org/unsd/accsub/2007docs-10th/SA-2007-14-Add3-Checklist.pdf

Labelling

Definition: A procedure to identify that compliance with defined (quality) standards has been verified. Context: The labelling method means that a label is attached to some statistics. The label has a message about

these statistics, a message that is related to quality and quality assessment. The label is a form of

10

communication with the end user. For this communication to be effective the label must be meaningful and the meaning of the label must be consistent across the range of products that carry that label (Eurostat (2007), "Handbook on Data Quality Assessment Methods and Tools (DatQAM)" Eurostat granted project led by FSO Germany with project members: Statistics Sweden, Hungarian Central Statistical Office, State Statistical Institute Berlin-Brandenburg, National Statistical Institute of Portugal, Statistics Norway, available at http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/HANDBOOK%20ON%20DATA%20QUALITY%20ASSESSMENT%20METHODS%20AND%20TOOLS%20%20I.pdf)

Source: • Expert group on NQAF Hyperlinks: • http://unstats.un.org/unsd/dnss/QualityNQAF/nqaf.aspx

Metadata

Definition: Data that defines and describes other data. Context: For the ISO standard, metadata is defined as data that defines and describes other data and

processes. This means that metadata are data that describe other data, and data become metadata when they are used in this way. This happens under particular circumstances and for particular purposes, as no data are always metadata. The set of circumstances and purposes (or perspective) for which some data are used as metadata is called the context. So, metadata are data about data in some context.

Source: • ISO/IEC FDIS 11179-1 "Information technology - Metadata registries - Part 1: Framework", March 2004


Methodological soundness

Definition: The extent to which the methodology used to compile statistics complies with the relevant international standards, including the professional standards enshrined in the Fundamental Principles for Official Statistics.

Context: A critical pre-requisite for the production of high quality statistics.


Microdata

Definition: Non-aggregated observations, or measurements of characteristics of individual units. Context: Microdata set is the result of a survey instance or other data collection instance after unit-level editing

and imputation and possible matching with other unit-level data. It organizes unit level data so that relationships between individual units and their characteristics can be identified, so as to allow all forms of aggregation. This is sometimes called an observation register (see, for instance, B. Sundgren: Guidelines for the modelling of statistical data and metadata, UN Geneva and New York, 1995). Anonymised microdata are individual statistical records which have been modified in order to minimise, in accordance with current best practice, the risk of identification of the statistical units to which they relate (European Commission Regulation (EC) No 831/2002 of 17 May 2002 implementing Council Regulation (EC) No 322/97 on Community Statistics, concerning access to confidential data for scientific purposes). Longitudinal microdata sets combine unit-level data from succeeding data collection instances over multiple time periods. Related terms Longitudinal data.


National statistical system (NSS)

Definition: The national statistical system (NSS) is the ensemble of statistical organisations and units within a country that jointly collect, process and disseminate official statistics on behalf of the national government.

Context: Source: • Measuring the Non-Observed Economy: A Handbook, OECD, IMF, ILO, Interstate Statistical

Committee of the Commonwealth of Independent States, 2002, Annex 2, Glossary Hyperlinks: • http://www.oecd.org/dataoecd/9/20/1963116.pdf

Non-response

Definition: A form of observation present in most surveys, which means failure to obtain a measurement on one or more study variables for one or more elements selected for the survey.

11

Context: The term encompasses a wide variety of reasons for non observation, such as "impossible to contact", "not at home", "unable to answer", "incapacity","hard core refusal", "inaccessible" or "unreturned questionnaire". In the first two cases, a contact with the selected element is never established. There are two broad types of non-response: first, a sampled unit that is contacted may fail to respond: this represents "unit non-response"; second, the unit may respond to the questionnaire incompletely: this is referred to as "item non-response". Non-response leads to an increase in variance as a result of a reduction in the actual size of the sample and the recourse to imputation. This produces a bias if the non-respondents have characteristics of interest that are different from those of the respondents. Furthermore, there is a risk of significantly underestimating the sampling error, if imputed data are treated as though they were observed data. (Statistics Canada, "Statistics Canada Quality Guidelines", 4th edition, October 2003, page 59).

Source: • Sarndal C.E., Swensson B., Wretman J., "Model assisted survey sampling", Springer - Verlag, New York, 1992


Non-response error

Definition: Error that occurs when the survey fails to get a response to one, or possibly all, of the questions. Context: Non-response errors result from a failure to collect complete information on all units in the selected

sample. These are known as "unit non- response" and "item non-response". Non-response errors affect survey results in two ways. First, the decrease in sample size or in the amount of information collected in response to a particular question results in larger standard errors. Second, and perhaps more important, a bias is introduced to the extent that non-respondents differ from respondents within a selected sample. Non-response errors are determined by collecting any or all of the following: unit response rate, weighted unit response rate, item response rate, item coverage rate, refusal rate, distribution of reason for non response, comparison of data across contacts, link to administrative data for non-respondents, estimate of non-response bias (Statistical Policy Working Paper 15: Quality in Establishment Surveys, Office of Management and Budget, Washington D.C., July 1988, page 68).

Source: • Statistics Canada, "Statistics Canada Quality Guidelines", 3rd edition, October 1998. Hyperlinks: • http://www.statcan.ca/english/freepub/12-539-XIE/12-539-XIE.pdf

Non-response rate

Definition: The proportion of non-response in a sample. Context: Source: • The International Statistical Institute, "The Oxford Dictionary of Statistical Terms", edited by

Yadolah Dodge, Oxford University Press, 2003 Hyperlinks: • Above taken from SDMX (2009) http://www.sdmx.org/; hyperlink to original source(s) unavailable.

Non-sampling error

Definition: Error in sample estimates which cannot be attributed to sampling fluctuations. Context: Non-sampling error may arise from many different sources such as defects in the sampling frame,

faulty demarcation of sample units, defects in the selection of sample units, mistakes in the collection of data due to personal variations, misunderstanding, bias, negligence or dishonesty on the part of the investigator or of the interviewer, mistakes at the stage of the processing of the data, etc. Non- sampling errors may be categorised as: - Coverage errors (or frame errors) due to divergences between the target population and the frame population ; - Measurement errors occurring during data collection. - Non-response errors caused by no data collected for a population unit or for some survey variables. - Processing errors due to errors introduced during data entry, data editing, sometimes coding and imputation. - Model assumption errors.


Objectivity

Definition: An attribute confirming that statistical authorities develop, produce and disseminate statistics in a systematic, reliable and unbiased manner.

Context: It implies the use of professional and ethical standards, and that the policies and practices followed are transparent to users and survey respondents.

12



Observation status

Definition: Information on the quality of a value or an unusual or missing value. Context: This item is normally coded and uses codes providing information about the status of a value, with

respect to events such as "break", "estimated value", "forecast", "missing value", or "provisional value". In some cases, there is more than one event that may have influenced the value (e.g. a break in methodology may be accompanied with the fact that an observation is an estimate). A textual comment providing more detailed information on important events related to an observation can be added via the attribute "Comment".


Output (product) quality

Definition: Degree to which a set of inherent characteristics fulfils output requirements. Context: Source: • ISO 9000/2005: Quality Management and Quality Assurance Vocabulary

• European Statistics Code of Practice Hyperlinks: • Above taken from ESS Quality Glossary 2010

(http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/ESS%20Quality%20Glossary%202010.pdf); hyperlink to original source(s) unavailable.

Paradata

Definition: Data about the processes by which statistics are compiled. Context: Paradata are a type of metadata, sometimes referred to as “process metadata”. They provide

information about how data are collected and processed, for example the mode of collection, edit rules, validation checks and disclosure control techniques applied to the data. Paradata can refer to micro-data or aggregate data, and can be used to manage process quality.


Peer review

Definition: Evaluation of the performance, or the quality of work, of a member of a peer group by the experts drawn from that group.

Context: The peer review is a special kind of external audit, carried out e.g. by a National Statistical Institute (NSI) for another NSI (=peers). In general, it is less formal than an audit. It aims rather at assessing the general quality than at controlling the conformity with an external quality standard.

Source: • BusinessDictionary.com Hyperlinks: • http://www.businessdictionary.com/definition/peer-review.html

Pre-release access

Definition: The practice of giving certain individuals or organisations access to data under embargo before those data are released to the public.

Context: This entails the transparent recording of persons or officials holding designated positions within the government, but outside the statistical system producing the data, who have pre-release access to the data and the reporting of the schedule according to which they receive access. Also called "internal access" in the Data Quality Assessment Framework of the IMF.


Prerequisites of quality

Definition: Institutional conditions for the pursuit of data quality. Context: The Data Quality Assessment Framework (DQAF) groups the indicators of this kind into three

elements: legal and institutional environment, resources, and quality awareness. These elements and indicators are identified to reinforce the idea that data users, who often cannot replicate or otherwise verify data, must place their trust in the institutions that produce statistics and the people who staff them. Typically, these pointers refer to the larger institution (called the "umbrella institution" in the DQAF) of which the compiling unit, such as a national accounts division or a balance

13

of payments department, is a part. Further, these prerequisites typically influence more than one of the five dimensions in the DQAF.

Source: • International Monetary Fund, "Data Quality Assessment Framework - DQAF - Glossary", unpublished Hyperlinks: • Above taken from SDMX (2009) http://www.sdmx.org/; hyperlink to original source(s) unavailable.

Process description

Definition: Documentation describing the features and parameters of a procedure used in statistical production. Context: A process description should contain:

- the name and the aim of the process - who is the process owner and operators - inputs (and the process they come from); outputs (and the process they go to) - sub-processes (activities) that transform inputs into outputs - regulatives (internal, external) that characterise the regulated environment - resources that are used in the transformation - how the process is managed and improved (performance and quality indicators with target values; the way of monitoring, measurement, analysis; improvements; records stating results achieved or providing evidence of activities performed).


• Context above taken from ESS Quality Glossary 2010 (http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/ESS%20Quality%20Glossary%202010.pdf); hyperlink to original source(s) unavailable.

Process metadata

See Paradata.

Process quality

Definition: Degree to which a set of inherent process characteristics fulfils process requirements. Context: Source: • ISO 9000/2005: Quality Management and Quality Assurance Vocabulary

• European Statistics Code of Practice Hyperlinks: • Above taken from ESS Quality Glossary 2010


Process variable

Definition: Variable that gives an indication of the quality of the process. Context: Key process variables are those factors that can vary with each repetition of the process and have the

largest effect on critical product characteristics, i.e. those characteristics that best indicate the quality of the product.

Source: • Eurostat (2004), “Handbook on improving quality by analysis of process variables" produced by ONS-UK, INE Portugal, NSS of Greece and Statistics Sweden

Hyperlinks: • http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/HANDBOOK%20ON%20IMPROVING%20QUALITY.pdf

Processing error

Definition: The error in final survey results arising from the faulty implementation of correctly planned implementation methods.

Context: Sources of processing errors include all post-collection operations, as well as the printing of questionnaires. Most processing errors occur in data for individual units, although errors can also be introduced in the implementation of systems and estimates. In survey data, for example, processing errors may include transcription errors, coding errors, data entry errors and errors of arithmetic in tabulation.

Source: • United States Federal Committee on Statistical Methodology, "Statistical Policy Working Paper 15: Quality in Establishment Surveys", Washington D.C., July 1988, page 79

Hyperlinks: • http://www.fcsm.gov/working-papers/wp15.html

Professional independence

Definition: Characteristic of a statistical institute or authority to develop, produce and disseminate statistics in an independent manner, particularly as regards the selection of techniques, definitions, methodologies and sources to be used, and the timing and content of all forms of dissemination, free from any

14

pressures from political or interest groups or from Community or national authorities, without prejudice to institutional settings, such as Community or national institutional or budgetary provisions or definitions of statistical needs.

Context: Professional independence is defined in the European Statistical Law and in the European Statistics Code of Practice.


• European Statistics Code of Practice Hyperlinks: • http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2009:087:0164:0173:EN:PDF

• http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/code_practice.pdf

Professionalism

Definition: (When referring to statistical agencies and their staff) - The standard, skill and ability suitable for producing statistics of good quality.

Context: To retain trust in official statistics, the statistical agencies need to decide according to strictly professional considerations, including scientific principles and professional ethics, on the methods and procedures for the collection, processing, storage and presentation of statistical data (Fundamental Principles of Official Statistics, principle 2). "Professionalism" describes the elements providing assurances that: statistics are produced on an impartial basis; elements providing assurances that the choices of sources and statistical techniques as well as decisions about dissemination are informed solely by statistical considerations; elements providing assurances that the recruitment and promotion of staff are based on relevant aptitude; elements providing assurances that the statistical entity is entitled to comment on erroneous interpretation and misuse of statistics, guidelines for staff behaviour and procedures used to make these guidelines known to staff; other practices that provide assurances of the independence, integrity, and accountability of the statistical agency. This concept can be further broken down into: Professionalism - code of conduct; Professionalism - impartiality; Professionalism - methodology; Professionalism - statistical commentary.


Provider load

Definition: The effort, in terms of time and cost, required for respondents to provide satisfactory answers to a survey.

Context: This burden can lead to providers experiencing annoyance, anger, frustration, etc., at being requested to participate, with escalation of these feelings generated by the complexity, length and/or frequency of surveys. The terms "provider load", “respondent load” and "respondent burden" are also used to describe provider burden. If perceived to be too onerous, non response rates may increase, or partial or erroneous data may be provided.

Source: • Australian Bureau of Statistics, Service Industries Statistics, "Glossary of Terms"; unpublished on paper

Hyperlinks: • http://www.abs.gov.au/CA25670D007E9EA1/0/DB35F160E9383A1FCA256B650006C3D0?Open&Highlight=0,Glossary

Punctuality

Definition: Time lag between the actual delivery of the data and the target date when it should have been delivered.

Context: This concept is linked to that of "timeliness" (see below). A statistical release can be punctual, i.e. released on the scheduled release date, but still be untimely if the release date is so far away from the reference period as to degrade their value of decision-making.


Quality

Definition: The degree to which a set of inherent characteristics fulfils requirements. Context: Quality is a multi-faceted concept. The dimensions of quality that are considered most important depend

on user perspectives, needs and priorities, which vary across groups of users. Several statistical organisations have developed lists of quality dimensions, which, for international organisations, are being harmonised under the leadership of the Committee for the Coordination of Statistical Activities (CCSA). The European Statistical Law defines statistical quality in terms of the following criteria: relevance,

15

accuracy, timeliness, punctuality, accessibility, clarity, comparability and coherence. Source: • ISO 9000/2005: Quality Management and Quality Assurance Vocabulary


Quality assessment

Definition: Quality assessment is a part of quality assurance that focuses on assessing the extent to which quality requirements have been fulfilled.

Context: "Quality assessment" contains the overall assessment of data quality, based on standard quality criteria. This may include the result of a scoring or grading process for quality. Scoring may be quantitative or qualitative.

Source: • Based on ISO definitions: ISO 9000/2005: Quality Management and Quality Assurance Vocabulary Hyperlinks: • Above taken from ESS Quality Glossary 2010


Quality assurance

Definition: All the planned and systematic activities implemented that can be demonstrated to provide confidence that the processes will fulfil the requirements for the statistical output.

Context: This includes the design of programmes for quality management, the description of planning process, scheduling of work, frequency of plan updates, and other organisational arrangements to support and maintain planning function.


Quality assurance framework

Definition: The structure for implementing quality assurance activities within an organisation. Context: The quality assurance framework is based on the core quality standards defined in the organisation's

quality framework. Source: • Based on The Eurostat Quality Assurance Framework Hyperlinks: • http://unstats.un.org/unsd/dnss/docs-nqaf/Eurostat_QAF%20leaflet.pdf

Quality audit

Definition: Periodic onsite-verification (by a certification authority) to ascertain whether or not a documented quality system is being effectively implemented.

Context: A quality audit is a systematic, independent and documented process for obtaining quality audit evidence (records, statements of fact or other information, which are relevant to the quality audit criteria and verifiable) and evaluating it objectively to determine the extent to which the quality audit criteria (set of policies, procedures or requirements) are fulfilled.

Source: • Eurostat (2007), "Handbook on Data Quality Assessment Methods and Tools (DatQAM)" Eurostat granted project led by FSO Germany with project members: Statistics Sweden, Hungarian Central Statistical Office, State Statistical Institute Berlin-Brandenburg, National Statistical Institute of Portugal, Statistics Norway

• BusinessDictionary.com Hyperlinks: • http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/HANDBOOK%20ON%20DAT

A%20QUALITY%20ASSESSMENT%20METHODS%20AND%20TOOLS%20%20I.pdf • http://www.businessdictionary.com/definition/audit.html

Quality components

See Quality dimensions.

Quality control

Definition: Subset of quality assurance (QA) process, it comprises of activities employed in detection and measurement of the variability in the characteristics of output attributable to the production system, and includes corrective responses.

Context: Quality Control of the data collection process assures that the underlying statistical assumptions of a survey are not violated, i.e. the meaning of the principal statistical measures and the assumptions which condition their use is maintained. Quality Control in data review process measures the impact of data adjustment on the data.

Source: • Economic Commission for Europe of the United Nations (UNECE), The Knowledge Base on Statistical Data Editing, Online glossary developed by the UNECE Data Editing Group, 2000

16

• ISO 9000/2005: Quality Management and Quality Assurance Vocabulary • BusinessDictionary.com

Hyperlinks: • http://www1.unece.org/stat/platform/display/kbase/Glossary • http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=42180 • http://www.businessdictionary.com/definition/quality-control-QC.html

Quality culture

Definition: Individual and collective commitment to quality maintenance and improvement. Context: Quality culture includes both measures geared to improve quality and individual and collective

commitment to maintaining and improving quality. Source: • Expert Group on NQAF Hyperlinks: • http://unstats.un.org/unsd/dnss/QualityNQAF/nqaf.aspx

Quality dimensions

Definition: Concepts used to describe some part or facet of the overall concept of quality, when applied to statistical outputs.

Context: Quality has many aspects that need to be reported separately. Terms which are often used interchangeably for dimensions include components, aspects, attributes, characteristics, etc. Some examples of quality dimensions: ● ABS's seven dimensions of quality: institutional environment; relevance; timeliness; accuracy; coherence; interpretability; accessibility ● Canada's (Statistics Canada) six dimensions of information quality: relevance; accuracy; timeliness; accessibility; interpretability; coherence ● Eurostat's six components / dimensions of output quality: relevance; accuracy; timeliness and punctuality; accessibility and clarity; comparability; coherence ● OECD's seven dimensions of quality: relevance; accuracy; timeliness; accessibility; interpretability; coherence; credibility ● South Africa's eight dimensions of quality: relevance; accuracy; timeliness; accessibility; interpretability; comparability and coherence; methodological soundness; integrity


Quality framework

Definition: A management system to direct and control an organisation with regard to quality – ranging from generally applicable, basic quality management systems and advanced forms referred to as excellence models, to systems or models developed for the concrete areas (e.g. for statistical production and dissemination).

Context: The distinguishing characteristics of a quality framework seem to be that: it provides an umbrella for quality practices; it refers to a range of surveys or an entire survey programme rather than a single survey; it covers a range of aspects of survey data output or operations, not just a single aspect; it involves a template that can be used for quality assessment.

Source: • Based on International Organization for Standardization (ISO), ISO Standard 9000/2005: "Quality management systems - Fundamentals and vocabulary", Geneva, 2005

Hyperlinks: • Above taken from ESS Quality Glossary 2010, http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/ESS%20Quality%20Glossary%202010.pdf ; hyperlink to original source(s) unavailable.

Quality indicators

Definition: Quantitative or qualitative measures that give an indication of process or output quality. Context: Examples are estimated standard errors and response rates, which relate specifically to the accuracy of

the output. Quality indicators differ from process variables, which give an indication of the quality of the process.

Source: • Expert Group on NQAF based on the Handbook on improving quality by analysis of process variables, produced by ONS-UK, INE Portugal, NSS of Greece and Statistics Sweden

Hyperlinks: • http://unstats.un.org/unsd/dnss/QualityNQAF/nqaf.aspx • http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/HANDBOOK%20ON%20IMPR

OVING%20QUALITY.pdf

Quality management

Definition: Systems and frameworks in place within an organisation to manage the quality of statistical products and processes.

17

Context: "Quality management" refers to the application of a formalised system that documents the structure, responsibilities and procedures put in place for satisfying users, while continuing to improve the data production and dissemination process. It also includes how well the resources meet the requirement. This concept can be broken down into: "Quality management - quality assurance"; "Quality management - assessment"; "Quality management - documentation". "Quality assurance" refers to all the planned and systematic activities implemented that can be demonstrated to provide confidence that the processes will fulfil the requirements for the statistical output. This includes the design of programmes for quality management, the description of planning process, scheduling of work, frequency of plan updates, and other organisational arrangements to support and maintain planning function. "Quality assessment" contains the overall assessment of data quality, based on standard quality criteria. This may include the result of a scoring or grading process for quality. Scoring may be quantitative or qualitative. "Quality documentation" contains documentation on methods and standards for assessing data quality, based on standard quality criteria such as relevance, accuracy and reliability, timeliness and punctuality, accessibility and clarity, comparability, and coherence.


Quality management system

Definition: Collective policies, plans, practices, and the supporting infrastructure by which an organization aims to reduce and eventually eliminate non-conformance to specifications, standards, and customer expectations in the most cost effective and efficient manner.

Context: Quality management system is a management system to direct and control an organisation with regard to quality.

Source: • Based on the BusinessDictionary.com

Hyperlinks: • http://www.businessdictionary.com/definition/quality-management-system-QMS.html

Quality policy

Definition: Top management's expression of its intentions, direction, and aims regarding quality of its products and processes.

Context: Such a document sets the overall context within which data quality is to be addressed in the statistical authority and sets expectations to guide agency staff and to inform other stakeholders.

Source: • BusinessDictionary.com Hyperlinks: • http://www.businessdictionary.com/definition/quality-policy.html

Quality report

Definition: Report providing a factual account of the quality of a statistical process and its outputs, together with recommendations for quality improvements and justification for their implementation.

Context: Can take various forms but should be regarded as an essential part of the overall documentation about the statistical product or survey. Under European Statistical Law, EU Member States are obliged to provide reports on the quality of the data they transmit to Eurostat. Eurostat uses these reports to compile reports on the quality of European statistics. Eurostat has produced standards and guidelines on the compilation of quality reports. See also Documentation and Quality Profile.

Source: • Based on the ESS Standard for Quality Reports (2009) Hyperlinks: • http://unstats.un.org/unsd/dnss/docs-nqaf/Eurostat-ESQR_FINAL.pdf

Quality targets

Definition: Agreed levels of achievement, performance or conformance of a statistical output for any given quality dimension.

Context: Quality targets, once met, will ensure that the deliverable meets the user needs. Source: • Expert Group on NQAF Hyperlinks: • http://unstats.un.org/unsd/dnss/QualityNQAF/nqaf.aspx

Relevance

Definition: The degree to which statistics meet current and potential users' needs. Context: Relevance is concerned with whether the available information sheds light on the issues that are

important to users. Assessing relevance is subjective and depends upon the varying needs of users. The Agency's challenge is to weight and balance the conflicting needs of current and potential users to

18

produce statistics that satisfy the most important needs within given resource constraints. In assessing relevance, one approach is to gauge relevance directly, by polling users about the data. Indirect evidence of relevance may be found by ascertaining where there are processes in place to determine the uses of data and the views of their users or to use the data in-house for research and other analysis. Relevance refers to the processes for monitoring the relevance and practical usefulness of existing statistics in meeting users" needs and how these processes impact the development of statistical programmes. This concept can be broken down into: "Relevance - completeness"; "Relevance - user needs"; "Relevance - user satisfaction". "Completeness" refers to the extent to which all statistics that are needed are available. The measurement of the availability of the necessary statistics normally refers to data sets and compares the required data set to the available one. "User Needs" refers to the description of users and their respective needs with respect to the statistical data. The main users (e.g. official authorities, the public or others) and user needs should be stated, e.g. official authorities with the needs for policy indicators, national users, etc. "User Satisfaction" refers to the measure to determine user satisfaction. This concerns how well the disseminated statistics meet the expressed user needs. If user satisfaction surveys have been conducted, the domain manager should mention them. Otherwise, any other indication or measure to determine user satisfaction might be used.


Record linkage

Definition: The methodology of bringing together corresponding records from two or more files or finding duplicates within files.

Context:

Source: • The State of Record Linkage and Current Research Problems, William E. Winkler, U. S. Bureau of the Census

Hyperlinks: • http://www.census.gov/srd/papers/pdf/rr99-04.pdf

Reference period

Definition: The period of time or point in time to which the measured observation is intended to refer. Context: In many cases, the reference period and time period will be identical, but there are also cases where

they are different. This can happen if data are not available for the target reference period, but are available for a time period which is judged to be sufficiently close. For example, the reference period may be a calendar year, whereas data may only be available for a fiscal year. In such cases, “reference period” should refer to the target reference period rather than the actual time period of the data. The difference between target and actual reference period can be highlighted in a free text note.

Source: • SDMX (2009)

Hyperlinks: • http://www.sdmx.org/

Refusal rate

Definition: The proportion of observation units for which the reporting unit has been successfully contacted, but has refused to give the information sought.

Context: The proportion is usually and preferably calculated by dividing the number of refusals by the total number of the sample which was originally desired to achieve. Observation and reporting units are often (but not always) the same entity.



Register

Definition: A written and complete record containing regular entries of items and details on particular set of objects. Context: A statistical register is a regularly updated list of units and their characteristics to be used for statistical

purposes. In some cases statistical registers may hold characteristics that are deemed by the appropriate legislation not to be subject to statistical confidentiality. In such cases, there may be non-statistical uses of data relating to these characteristics.

Source: • Economic Commission for Europe of the United Nations (UNECE), "Terminology on Statistical Metadata", Conference of European Statisticians Statistical Standards and Studies, No. 53, Geneva, 2000

Hyperlinks: • http://www.unece.org/stats/publications/53metadaterminology.pdf

19

Reliability

Definition: Closeness of the initial estimated value to the subsequent estimated value. Context: Studies should always be undertaken to make this assessment. If the initial estimate shows consistent

higher or lower estimates than the subsequent estimates, then there may be evidence of bias in the initial estimates. If the differences are random, but large, then consideration should be given to reassessing the timeliness/reliability trade-off that has been adopted.


Respondent burden

Definition: (Referred to by the source as provider load) - The effort, in terms of time and cost, required for respondents to provide satisfactory answers to a survey.

Context: This burden can lead to providers experiencing annoyance, anger, frustration, etc., at being requested to participate, with escalation of these feelings generated by the complexity, length and/or frequency of surveys. The terms "provider load", “respondent load” and "respondent burden" are also used to describe provider burden. If perceived to be too onerous, non response rates may increase, or partial or erroneous data may be provided. The terms "respondent load", "provider load", and "reporting burden" are also used to describe this concept.

Source: • Australian Bureau of Statistics, Service Industries Statistics, "Glossary of Terms"; unpublished on paper

Hyperlinks: • ·http://www.abs.gov.au/CA25670D007E9EA1/0/DB35F160E9383A1FCA256B650006C3D0?Open&Highlight=0,Glossary

Revision policy

Definition: A policy or set of policies, aimed at ensuring the transparency of disseminated data whereby preliminary data are compiled that are later revised when more and better source data become available.

Context: In SDMX, "Revision Policy and Practice" describes the data revision policy, the policy and practice for identifying the revision status of available data, as well as the availability of revision studies and analyses. Providing users with documentation regarding the source data used and the way they are adjusted gives compilers the possibility to incorporate new and more accurate information into estimates, thus improving their accuracy without introducing breaks in the time series. Data may also be subject to ad hoc revisions as a result of the introduction of new classifications, compilation frameworks and methodologies which result in the compilation of historical data that replaces previously released data. Whether or not such changes constitute an actual "revision" or the compilation of a "new" series is a matter of judgment on the part of the statistical agency. Under the requirements of the Special Data Dissemination Standard (SDDS), an organisation's revision policy for specific statistics is disseminated on the Internet on the IMF`s Dissemination Standards Bulletin Board (DSBB).

Source: • OECD Glossary of Statistical terms

Hyperlinks: • http://stats.oecd.org/glossary/detail.asp?ID=6146

Rolling review

Definition: An in-depth assessment of a statistical process, carried out by an external expert, including a user survey and a partner survey.

Context: Generally undertaken over time and across statistical programmes to assess their continuing relevance and other quality attributes. Rolling reviews provide a broader assessment of a statistical process compared with self-assessments.

Source: • Based on the Eurostat Quality Assurance Framework

Hyperlinks: • http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/QAF%20leaflet.pdf

Sampling error

Definition: That part of the difference between a population value and an estimate thereof, derived from a random sample, which is due to the fact that only a subset of the population is enumerated.

Context: Sampling errors are distinct from errors due to imperfect selection, bias in response or estimation, errors of observation and recording, etc. For probability sampling, the random variation due to sampling can be calculated. For non-probability sampling, random errors cannot be calculated without reference to some kind of model. The totality of

20

sampling errors in all possible samples of the same size generates the sampling distribution of the statistic which is being used to estimate the parent value.



Security (Data security)

Definition: The measures taken to prevent unauthorized access or use of data. Context: Source: • Economic Commission for Europe of the United Nations (UNECE), "Terminology on Statistical

Metadata", Conference of European Statisticians Statistical Standards and Studies, No. 53, Geneva, 2000.

Hyperlinks: http://www.unece.org/stats/publications/53metadaterminology.pdf

Self-assessment

Definition: Comprehensive, systematic and regular review of an organisation's activities and results referenced against a model/framework, carried out by the organisation itself.

Context: Source: • Eurostat (2007), "Handbook on Data Quality Assessment Methods and Tools (DatQAM)" Eurostat

granted project led by FSO Germany with project members: Statistics Sweden, Hungarian Central Statistical Office, State Statistical Institute Berlin-Brandenburg, National Statistical Institute of Portugal, Statistics Norway

Hyperlinks: • http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/HANDBOOK%20ON%20DATA%20QUALITY%20ASSESSMENT%20METHODS%20AND%20TOOLS%20%20I.pdf

Serviceability

Definition: Set of practical aspects describing how well the available data meet users' needs. Context: Serviceability is a term that captures the practical aspects of usability of data. The emphasis on "use"

thus assumes that data are available. Thus, key aspects of usability include relevance, timeliness and frequency, consistency, revision policy and practices and availability of metadata.

Source: • International Monetary Fund, "Data Quality Assessment Framework - DQAF - Glossary", unpublished


Stakeholders

Definition: People or organisations with an interest in the output and various other aspects of an agency.

Context: Stakeholder expectations are varied and must be managed at all stages in the statistical production process.


Standards

Definition: Documented agreements containing technical specifications or other precise criteria to be used consistently as rules, guidelines, or definitions of characteristics, to ensure that materials, products, processes and services are fit for their purpose.

Context: Documents, established by consensus and approved by a recognized body, that provide, for common and repeated use, rules, guidelines or characteristics for activities or their results, aimed at the achievement of the optimum degree of order in a given context - Note - Standards should be based on the consolidated results of science, technology and experience, and aimed at the promotion of optimum community benefits.

Source: • ISO Concept Database ISO/CDB and the ISO/IEC (guide 2: 2004) Hyperlinks: • http://www.iso.org/iso/concept_database_cdb.htm

Statistical (production/business) process

Definition: The complete set of sub-processes needed to support statistical production. Context: The statistical business process is described in Eurostat’s Cycle de Vie des Données (CVD) model,

and the Generic Statistical Business Process Model, being developed by the METIS. Source: • SDMX (2009)

21


Statistical data

Definition: Characteristics or information that have been collected for statistical purposes, or processed from non-statistical sources, to contribute to the production of official statistics.

Context: The ultimate objective of any statistical activity is to generate statistical data.


Statistical Data and Metadata Exchange (SDMX)

Definition: Set of technical standards and content-oriented guidelines, together with an IT architecture and tools, to be used for the efficient exchange and sharing of statistical data and metadata.

Context: Seven organisations (Bank of International Settlements, European Central Bank, Eurostat, International Monetary Fund, Organisation for Economic Co-operation and Development, United Nations Statistical Division and World Bank) act as sponsors of SDMX. The first version of the SDMX technical standard (1.0, finalised in 2004) has been approved as an ISO standard (ISO/TS 17369:2005). SDMX V2.0 was also approved by the sponsors in 2005. Public consultations on the content-oriented guidelines were conducted in 2006 and in 2008, with the aim of compiling guidelines on cross-domain concepts and code-lists, statistical subject-matter domains and the metadata common vocabulary. The first set of content-oriented guidelines was finalised at the end of 2008 to be published in January 2009.


Statistical output

Definition: Results from a statistical process to be accessed by the final users. Context: Can take the form of aggregate statistics, analysis, and microdata releases and can include different

forms of media (e.g. the Internet and paper reports). Source: • ESS Handbook for Quality Reports (2009 edition), Methodologies and Working Papers, Eurostat,

Luxembourg. Hyperlinks: • http://unstats.un.org/unsd/dnss/docs-nqaf/Eurostat-EHQR_FINAL.pdf

Statistical standard

Definition: An agreed rule or guideline on how one or more parts of the statistical business process should be carried out, conforming with requirements for professionalism.

Context: Components of a standard include: - definition(s) - statistical units - classification(s) - coding process(es) - questionnaire module(s) - output categories The use of statistical standards permits the repeated collection of statistics on a consistent basis. They also enable the integration of data over time and across different data sources, allowing the use of data beyond the immediate purpose for which it was produced. Standards also reduce the resource requirements associated with many aspects of survey development and maintenance (Statistics New Zealand," Classifications and Standards"; unpublished on paper).


Statistical unit

Definition: Entity for which information is sought and for which statistics are ultimately compiled. Context: The statistical unit is the object of a statistical survey and the bearer of statistical characteristics. These

units can, in turn, be divided into observation units and analytical units. Statistical units for economic statistics are defined on the basis of three criteria: 1) Legal, accounting or organisational criteria; 2) Geographical criteria; 3) Activity criteria. Statistical units comprise the enterprise, enterprise group, kind-of-activity unit (KAU), local unit, establishment, homogeneous unit of production, persons, households, geographical areas, events etc. Statistical units can be categorised into basic statistical units, i.e. those for which data is collected, and derived statistical units, i.e. those which are constructed during the statistical production process. A

22

basic statistical unit is the most detailed level to which the obtained characteristics can be attached. In other statistical domains, statistical units can include persons, households, geographical areas, events etc. Statistical units can be categorised into basic statistical units, i.e. those for which data is collected, and derived statistical units, i.e. those which are constructed during the statistical production process.


Supported self-assessment

Definition: A special type of self assessment, conducted by the production unit, with the active participation of the Quality unit.

Context: Supported self-assessments involve an evaluation of the strengths and weaknesses of an activity, resulting in an action plan for improvements. A useful approach to assessing data quality. As compared, though, to the next higher levels of assessments, i.e. peer reviews and rolling reviews, the results of supported self-assessments may be seen as being less objective in the absence of participation by an independent/neutral outside expert.

Source: • The Eurostat Quality Assurance Framework Hyperlinks: • http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/QAF%20leaflet.pdf

Target population

Definition: The set of elements about which information is wanted and estimates are required. Context: Practical considerations can dictate that a survey population be defined which excludes some units in

the target population or which is comprised of differently defined units through which the target population can be accessed.

Source: • Statistics Canada, "Statistics Canada Quality Guidelines", 5th edition, October 2009, page 19

Hyperlinks: • http://www.statcan.gc.ca/pub/12-539-x/12-539-x2009001-eng.pdf

Timeliness

Definition: Length of time between data availability and the event or phenomenon they describe. Context: Timeliness refers to the speed of data availability, whether for dissemination or for further processing,

and it is measured with respect to the time lag between the end of the reference period and the release of data. Timeliness is a crucial element of data quality: adequate timeliness corresponds to a situation where policy-makers can take informed decisions in time for achieving the targeted results. In quality assessment, timeliness is often associated with punctuality, which refers to the time lag between the release date of data and the target date announced in some official release calendar. It is typically involved in a trade-off against accuracy. The timeliness of information will influence its relevance. Timeliness can be further broken down into "Timeliness - output" and "Timeliness - source data". "Timeliness - output" refers to the lapse of time between the end of a reference period (or a reference date) and the release of a version of the data: provisional, preliminary, or final results. This reflects many factors, including some that are related to institutional arrangements, such as the preparation of accompanying commentary and printing. Usually, data are not released immediately at the end of the period they refer to, since data collection, data processing and data dissemination work needs to be performed. "Timeliness - source data" refers to the time between the end of a reference period (or a reference date) and actual receipt of the data by the compiling agency. Compared to the parent concept - timeliness - this concept only covers the time period between the end of the reference period and the receipt of the data by the data compiling agency. This time period is determined by factors such as delays accommodating the institutional arrangements for data transmission.


Total Quality Management (TQM)

Definition: Management philosophy that is driven by customer needs and expectations. TQM aims to create a Quality Culture, and is based on a number of core values such as: customer orientation; leadership; participation of all staff; process orientation; teamwork; staff development; and continuous improvement.

Context: At the core of the TQM approach is the concept of continuous improvement, often illustrated by the Plan-Do-Check-Act cycle made popular by Deming.

Source: Eurostat (2004), “Handbook on improving quality by analysis of process variables" produced by ONS-UK, INE Portugal, NSS of Greece and Statistics Sweden

23

Hyperlinks: http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/HANDBOOK%20ON%20IMPROVING%20QUALITY.pdf

Transparency

Definition: The condition in which all policies and practices surrounding a statistical activity are made known to the stakeholders - particularly the end users.

Context: In Regulation (EC) No 223/2009 on European statistics, it states that the development, production and dissemination of European statistics shall be governed by specific statistical principles, including "objectivity", where objectivity is defined as: "meaning that statistics must be developed, produced and disseminated in a systematic, reliable and unbiased manner; it implies the use of professional and ethical standards, and that the policies and practices followed are transparent to users and survey respondents".


User

Definition: Recipient of statistical information, who transforms it into knowledge needed for decision making or research.

Context: The ultimate client of the statistical authority, who will make the judgment as to whether the data or services are fit for purpose.


User needs

Definition: Data or metadata requirements of persons or organisations to meet a particular use or set of uses. Context: User needs refers to the description of users and their respective needs with respect to the statistical

data. The main users (e.g. official authorities, the public or others) and user needs should be stated, e.g. official authorities with the needs for policy indicators, national users, etc. User needs may be specified in terms of the quality dimensions promulgated by international organisations or national agencies.


User satisfaction

Definition: Measure to determine how well the disseminated statistics meet the expressed user needs. Context: If user satisfaction surveys have been conducted, the domain manager should mention them.

Otherwise, any other indication or measure to determine user satisfaction might be used. Source: • SDMX (2009) Hyperlinks: • http://www.sdmx.org/

User satisfaction survey

Definition: Survey which aims at assessing the satisfaction or the perception of the users, normally as a basis for improvement actions.

Context: User satisfaction surveys can take different forms e.g. using standardised questionnaires or through qualitative interviews. Care needs to be taken with interpreting the results of such surveys as some users are inherently more important to the statistical authority.

Source: • Eurostat (2007), "Handbook on Data Quality Assessment Methods and Tools (DatQAM)" Eurostat granted project led by FSO Germany with project members: Statistics Sweden, Hungarian Central Statistical Office, State Statistical Institute Berlin-Brandenburg, National Statistical Institute of Portugal, Statistics Norway

Hyperlinks: • http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/HANDBOOK%20ON%20DATA%20QUALITY%20ASSESSMENT%20METHODS%20AND%20TOOLS%20%20I.pdf