Guidelines on Statistical Business Registers Draft Chapter 8: Quality of SBR Caterina Viviano, Monica Consalvi ISTAT Meeting of the Group of Experts on

Guidelines on Statistical Business Registers Draft Chapter 8: Quality of SBR

Caterina Viviano, Monica ConsalviISTAT

Meeting of the Group of Experts on Business Registers,2-4 September 2013, Geneva

1. Provide guidance on how to evaluate quality for SBR

2. Describe quality criteria with an eye to the complexity of SBR

3. Provide measurement tools for quality evaluation

4. Suggest a conceptual framework to build up quality indicators, some practical examples

5. Strategies for improving quality

Purpose

1

Draft chapter 8: quality of SBR, Caterina Viviano, Monica Consalvi– Geneva, 2-4 September 2013

1. Quality components

2. Specificities of SBR in comparison with a survey (extensive use of administrative data; heterogeneity and variability of inputs, users, relevant units; relevance of technological aspects; the continuous updating; etc..)

3. Frame errors: an overall description of the main errors and consequences in statistical outputs

4. Tools for quality evaluation and metadata

5. The conceptual framework for quality indicators

Overview

2


5. Framework for quality indicators (1)

3


The three dimensions for defining a Quality Indicator:

1. The BR’s phases (Input, Process, Output)

2. The quality components (relevance, accuracy, timeliness, punctuality, accessibility and clarity, comparability, coherence)

3.The key factors (time, scope, sub-population, variable, criterion)

5. Framework for quality indicators (2)

4


Example: to evaluate the completeness in 2-digit Nace code when using a given input source to update economic activity in a given sub-population (i.e. large ent.) of the SBR

I(t)% = % 2-digit Nace missing codes (out of the total number of units) in large enterprises at time t in input source Tax register

I(t, t-1) = Percentage variation Var[I(t)%, I(t-1)%]

1. phase: Input (source=tax register)2. component: Relevance-completeness3. factors: Variable=economic activity; scope=enterprises; sub-

pop=ent. 100+ employees; criterion=temporal consistency

1. To better address quality components to the SBR peculiarities

2. to develop indicators to measure each single quality dimension

3. Add quality reports as tool for quality evaluation

4. Due to the huge use of administrative data, is it necessary to develop more in-depth the assessment of the quality of administrative sources? To which extent quality of administrative sources should be described in Ch. 6 or Ch.8 ?

Issues for discussion (further in-depth analysis)

5


Good practices from countries (1)

1. Quality of Input Component – 1.1 Completeness 1.1.1 ) Address, s=CCIAA: Number of records ( % weight) with missing information

INDICATOR COMPUTATION It=2005 VI=It=2005 - It=2004 Records with missing address (cciaa)

% weight (abs.number of records)

0.49 (37,408)

-0.03

2. Quality of process Component – 2.1 Coverage 1) Number of records, s=CCIAA, not matched with the base MEF

INDICATOR COMPUTATION It=2005 VI=It=2005 - It=2004 Not matched Records (cciaa)

% weight (abs. number of records)

5.25 (338,304)

0.03

3. Quality of output Component – 3.2 Timeliness 3.2 Lag, in days, between dissemination time of BR and reference year of data

INDICATOR COMPUTATION It=2005 VI=It=2005 - It=2004 Timeliness of BR dissemination

Days of delay between the dissemination time and the reference year of data

492 +24

SBR Quality Indicators - Italy (some examples taken from the quality declaration)

6

Good practices from countries (2)SBR Quality Indicators – Colombia (some examples of proposed indicators)

• Indicator 1• Name: Level of update• Objective: to know the rate of update for each of the economic sectors in the

frame.• Type of Indicator: Quality of the process• Variables used in the calculations are:• A_j: Total records update for sector j• B_j: Total records expected for update in sector j• The formula used for the calculation is:

• Calculation Frequency: Annually• Tolerance ranges: Critical <= 70; 70> Fair <= 90; Satisfactory> 90.

Any other suggestions / good practices?7

Titolo intervento, nome cognome relatore – Luogo, data

Contacts

8

Caterina Viviano, [email protected]

Monica Consalvi, [email protected]

mailto:[email protected]

mailto:[email protected]

Documents

Guidelines on Statistical Business Registers Draft Chapter 8: Quality of SBR Caterina Viviano, Monica Consalvi ISTAT Meeting of the Group of Experts on