Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
Backstage of Population Census in Italy:
fieldwork and paradata to manage data
quality collection
The role of Istat in Tuscany
Bianca Maria Martelli, Linda Porciani, Alessandro Valentini
5th Conference of European Survey Research Association
Ljubljana, Slovenia, July 15-19, 2013
Summary
� 15th Population and Housing Census in Italy
� Territorial organization
� Managing data quality in the case of Tuscany
�Theoretical approach
� Application� Application
• Field work
• Paradata indicators
• Post census ad hoc survey
� Discussion
Backstage of Population Census in Italy: fieldwork and paradata to manage data quality collection. The role of Istat in Tuscany
B. M. Martelli, L. Porciani, A. Valentini – Ljubljana, Slovenia, July 15-19, 2013
Main innovations of Census
� From a totally door-to-door to assisted census (by LAU2
Population Register)
� Use of other administrative registers to reduce under-coverage
(fiscal registers, ‘permits-to-stay’ , etc…)
� Mail-out of census forms� Mail-out of census forms
� Multi-way modalities to answer the questionnaires (paper and
web based)
� Sample strategies in order to reduce response burden (short
and long forms)
� Supervision of the process via on line system SGR (Survey
Recording System) SGR
Backstage of Population Census in Italy: fieldwork and paradata to manage data quality collection. The role of Istat in Tuscany
B. M. Martelli, L. Porciani, A. Valentini – Ljubljana, Slovenia, July 15-19, 2013
1
NUTS0 - Italy NUTS2 - Region
- ISTAT
- Statistical Office
of the Italian
Ministry of
Interior
Regional Offices
of Census (URCs)
Territorial organization
Responsibility to manage the data collection process
in its own territory
Responsibility to manage the data collection process
in its own territory
NUTS3 - Provinces LAU2 - Municipalities
Statistical Offices
of Prefectures
(UPCs)
Municipal Offices
of census (UCCs)
Backstage of Population Census in Italy: fieldwork and paradata to manage data quality collection. The role of Istat in Tuscany
B. M. Martelli, L. Porciani, A. Valentini – Ljubljana, Slovenia, July 15-19, 2013
1
Territorial organization: a focus on Tuscany
Tuscany:
Regional Office of
Istat URC
10 Provinces UPCs 287 Municipalities
(212 UCCs)
Backstage of Population Census in Italy: fieldwork and paradata to manage data quality collection. The role of Istat in Tuscany
B. M. Martelli, L. Porciani, A. Valentini – Ljubljana, Slovenia, July 15-19, 2013
2
Managing Data Quality (DQ) in the case of Tuscany
Top-down approach
Researcher selects
DQ attributes
According Wang and Strong (1996), the methodological
framework adopted is the following
Field workIntuitive
Top-down approach
How data quality
weakens during
data collection process
Bottom up approach
Investigation on
actors’ opinions
about DQ attributes
Paradata tool
Ad hoc survey
on census
operators
Theoretical
Empirical
Backstage of Population Census in Italy: fieldwork and paradata to manage data quality collection. The role of Istat in Tuscany
B. M. Martelli, L. Porciani, A. Valentini – Ljubljana, Slovenia, July 15-19, 2013
3
Definition of a matrix quality measures/actions developed by URC
Intuitive approach: field work
Quality measures
Backstage of Population Census in Italy: fieldwork and paradata to manage data quality collection. The role of Istat in Tuscany
B. M. Martelli, L. Porciani, A. Valentini – Ljubljana, Slovenia, July 15-19, 2013
4
Field-work actions
Training
Scheduled: between July and October 2011[Census day: October, 9th 2011]
Subjects: UPCs, UCCs, Enumerators
(around 5,000 persons)
SupportingIn-distance: by phone (91%) and by e-mail (9%)
Backstage of Population Census in Italy: fieldwork and paradata to manage data quality collection. The role of Istat in Tuscany
B. M. Martelli, L. Porciani, A. Valentini – Ljubljana, Slovenia, July 15-19, 2013
5
SupportingIn-distance: by phone (91%) and by e-mail (9%)
In-presence: additional training sessions
MonitoringInspections: checks to UCCs by URC staff
Interventions: on critical cases
Response rate by nationalities
Field-work actions: an intervention
The “elusive population” of foreigners in Prato[15% of the total population in 2011]
1. Official actions promoted by local
institutions (meeting URC, UCC, head of local community);
2. mass media communication (press conference, internet pages by chineselanguage)
1. Official actions promoted by local
institutions (meeting URC, UCC, head of local community);
2. mass media communication (press conference, internet pages by chineselanguage)
1. Informal actions (direct mailing from UCC)
1. Informal actions (direct mailing from UCC)
Backstage of Population Census in Italy: fieldwork and paradata to manage data quality collection. The role of Istat in Tuscany
B. M. Martelli, L. Porciani, A. Valentini – Ljubljana, Slovenia, July 15-19, 2013
6
Theoretical approach: paradata tool
1. Dashboard indicators
Real time indicators -> to realize an
immediate set of interventions to
solve general or local criticisms
Backstage of Population Census in Italy: fieldwork and paradata to manage data quality collection. The role of Istat in Tuscany
B. M. Martelli, L. Porciani, A. Valentini – Ljubljana, Slovenia, July 15-19, 2013
7
2. Roadmap indicators
Ex post indicators -> to improve DQ
in incoming surveys
(reengineering of the process)
Paradata tool 1.Dashboard indicators
Indicator Description Target
value (%)
QUES.LAC Percentage ratio between number of questionnaires filled in and number of questionnaires in the population registry lists (LAC) at 31.12.2010
≥ 97 ≤ 103
QUES.CA Percentage ratio between number of questionnaires processed and number of questionnaires enabled to the comparison (CA) between census and population registers.
100
Dashboard indicators consist of 6 indicators derived from SGR data
population registers.
DIARIO.LIFA Percentage ratio between processed rows and total rows of potential undercoverage derived from administrative archives (LIFA)
100
DIARIO.NETTO Percentage ratio between rows processed and total number of rows excluding (NETTO) rows of potential undercover derived from LIFA
100
CFR01.EDI Percentage ratio between number of surveyed building (EDI) and number of surveyed buildings in 2001 census
≥ 100 ≤ 150
CFR01.ABI Percentage ratio between number of surveyed unoccupied dwellings (ABI) and number of surveyed unoccupied dwellings in 2001 census
≥ 85 ≤ 115
Backstage of Population Census in Italy: fieldwork and paradata to manage data quality collection. The role of Istat in Tuscany
B. M. Martelli, L. Porciani, A. Valentini – Ljubljana, Slovenia, July 15-19, 2013
8
Indicator DIARIO.LIFA
(tested undercoverage)
An application to LAU2
0 – 20,000 inhabitants
Paradata tool 1.Dashboard indicators
Problem
Indicator CFR01.ABI (unoccupied dwellings)(tested undercoverage)
Problem
Localized criticism
Action
Targeted intervention on specific LAU2
Problem
Widespread criticism due to lacking information
Action
Additional training phase to all network census actors
Backstage of Population Census in Italy: fieldwork and paradata to manage data quality collection. The role of Istat in Tuscany
B. M. Martelli, L. Porciani, A. Valentini – Ljubljana, Slovenia, July 15-19, 2013
9
Paradata tool 2. Roadmap indicators
97.9
94.9
97.6
95.095
96
97
98
99
100
%
Italy
Tuscany
33.431.7
22.624.5
31.7 31.4
20
25
30
35
40
%
Italy
Tuscany
Standardization process and rate of postal
delivery (%). Italy and TuscanyQuestionnaires by channel of answer (%).
Italy and Tuscany
90
91
92
93
94
95
Standardization
process
Rate of postal
delivery
%
12.3 12.4
0
5
10
15
20
Web Municipal data
collection
centers
Post offices Enumerators
NO need of specific territorial intervention Need of deeper analysis
Backstage of Population Census in Italy: fieldwork and paradata to manage data quality collection. The role of Istat in Tuscany
B. M. Martelli, L. Porciani, A. Valentini – Ljubljana, Slovenia, July 15-19, 2013
10
∑=
+=4
1 i
o (WR)logit ii xββ
Paradata tool 2. Roadmap indicators
Where:
WR = Quota of web respondent (by Municipality)
X1 = Number of members per households
Backstage of Population Census in Italy: fieldwork and paradata to manage data quality collection. The role of Istat in Tuscany
B. M. Martelli, L. Porciani, A. Valentini – Ljubljana, Slovenia, July 15-19, 2013
11
X1 = Number of members per households
[effect of class >2.29; ref. category ≤2.29]
X2 = Quota of foreigners
[effect of class >0.0745; ref. category ≤0.0745]
X3 = Ageing index
[effect of class >200.3; ref. category ≤200.3]
X4 = Demographic dimension of Municipality
[effect of class > 5,573 ; ref. category ≤5,573]
Paradata tool 2. Roadmap indicators
+- 0
Demographic
dimension
Quota of
foreignersAgeing indexHousehold
dimension
Results:
Main insights:
1. Use of web is linked to demographic characteristics of population;
2. To increase the quota of web users it is necessary to support specific
subsets of population: ‘traditional families’ and ‘old population’;
3. A particular care has to be dedicated to small LAU2, with more traditional
habits
dimensionforeignersdimension
Backstage of Population Census in Italy: fieldwork and paradata to manage data quality collection. The role of Istat in Tuscany
B. M. Martelli, L. Porciani, A. Valentini – Ljubljana, Slovenia, July 15-19, 2013
12
Empirical approach: ad hoc survey on operators
At the end of Census operations an ad hoc survey (Ivalcens, ‘inquiry to evaluate the census’) was submitted to qualified census operators
The survey was realized via a web technique. The rate of response was 94.6% for national level (100% in Tuscany)
Main results of Ivalcens
Backstage of Population Census in Italy: fieldwork and paradata to manage data quality collection. The role of Istat in Tuscany
B. M. Martelli, L. Porciani, A. Valentini – Ljubljana, Slovenia, July 15-19, 2013
13
Main results of Ivalcens
� Scarce interest for the internet channel (one of the lowest evaluation in Italy)� Criticisms about data collection of unoccupied dwellings (problem already emerged during the field work and paradata analysis);� Low appreciation of mail-out (despite the good quality of road-map indicators)� Limited appreciation of the (institutional) context
Discussion
The key factor for DQ should be represented by a complete integration between the central organism of planning and the peripheral offices devoted to supervise the application of rules (URCs in our case).
From our experience, the set of instruments introduced is a From our experience, the set of instruments introduced is a valid tool to:� easily point out the criticalities through simple indicators� prevent bad practices in data collection and help best ones
� monitor the field work and, finally� assess data and process quality
Backstage of Population Census in Italy: fieldwork and paradata to manage data quality collection. The role of Istat in Tuscany
B. M. Martelli, L. Porciani, A. Valentini – Ljubljana, Slovenia, July 15-19, 2013
14
Thanks for your attention!
Bianca Maria Martelli, [email protected]
Linda Porciani, [email protected]
Alessandro Valentini, [email protected]
Backstage of Population Census in Italy: fieldwork and paradata to manage data quality collection. The role of Istat in Tuscany
B. M. Martelli, L. Porciani, A. Valentini – Ljubljana, Slovenia, July 15-19, 2013
15