European Conference on Quality in Official Statistics Session 26: Census 2011 « Helsinki, 6 May 2010 « Census quality control with BSC: the Portuguese

European Conference on Quality in Official Statistics

Session 26: Census 2011«

Helsinki, 6 May 2010

«

Census quality control with BSC: the Portuguese experience

«

1

Carlos Dias - Statistics PortugalÁlvaro Rosa and Paula Vicente – ISCTE

2 «Census quality control with BSC: the Portuguese experience

«Summary

1.Pilot Survey;

2.Strategic mapping of 2011 Census operations;

3.Human resources process;

4.Delivery and collection processes;

5.Conclusions.

«

3 «

«1. Pilot Survey

The final stage of preparation of the 2011 Census.

In this test, respondents had two response modes available:

«


Paper questionnaire

Online form (e-Census)

4 «

«1. Pilot Survey

«


e-Censos

e-Censos + Paper

Paper

2 weeks 1 week 2 weeks 2 week

__________________ Delivery

_______________________________________________________ Collection

Census day

5 «

«2. Strategic mapping of 2011 Census operation

To produce high quality data from the Census, the integrated process of data collection and processing must undergo a stringent system of quality management.

To achieve the designated objective Statistics Portugal will adopt the Balanced Scorecard (BSC) as its quality management framework.

«


6 «

«2. Strategic mapping of 2011 Census operations

Mission

The mission of Statistics Portugal is to produce and disseminate, in an effective, efficient and independent manner, high-quality official statistical information relevant to the whole society. The 2011 Census shall be carried out under high quality standards and accountability.

Vision

The Vision is the guideline for the development of an organization, establishing the goals to be achieved: "The 2011 Census will be acknowledged at an internal and external level as an official statistical information of great quality."

«


7 «


Values of 2011 Census operations

The 2011 Census operations staff, in accordance with the European Statistics Code of Practice, is subject to the following Values:

• Professionalism, ethics and observance of confidentiality;

• Respect for primary data providers;

• Technical independence, objectivity and impartiality.

«


8 «


Strategic framework

The 2011 Census programme and products will be guided by the following strategic vectors:

• Increasing confidence in the final results;

• Improving the final product quality by minimizing errors;

• Facilitating the response by maximizing the overall response rate;

• Increasing resource efficiency: increase the quickly availability of information and cost savings.

«


9 «


For the Pilot Survey operation we have identified two key processes that are associated with:

• Human resources;

• Delivery and collection.

«


10 «

«3. Human resources process

The completion of a census requires the mobilization of a large number of human resources. In the last Census (2001), 26,500 people have been involved and submitted to training sessions (including effective enumerators and substitutes).

The quality and effectiveness of census operations are strongly determined by the capabilities of fieldwork staff and the training provided.

The key factors for assessment and intervention in the processes associated with human resources are the recruitment, selection and training. «


11 «


In the processes associated with human resources we have established the following strategic goals:

1 – To enhance the process of recruiting and selecting staff to ensure the recruitment of candidates with the required profile;

2 – To minimize the turnover rate (withdrawal) of the census;

3 – To increase / improve training;

4 – To increase the satisfaction of human resources involved;

5 – To identify the critical points of the process.

«


12 «


«


Indicators Targets/Tolerance

1.1. - percentage of candidates selected to be enumerators with complete secondary school, by civil parish

1.2. - percentage of candidates selected to be enumerator with computer knowledge, by civil parish

1.3. - percentage of candidates with a positive note in the selection test, by civil parish

2.1. - percentage of enumerators who remain until the end of the operation, by civil parish

3.1. - percentage of participants who received training in accordance with the programmed plan

3.2. - percentage of trainees with 80% and more, at the end of training assessment

4.1. - fieldwork staff satisfaction level

4.2. - fieldwork staff motivating index

5.1. - total number of candidates / number of needed enumerators, by civil parish

100% / 30%

70% / 20%

100% / 10%

80% / 10%

80%/ 5%

100% / 10%

70% / 5%

70% / 5%

140% / 10%

13 «


Matrix Initiatives/Strategic goals

«


Strategic goals

Initiatives 1 2 3 4 5

A - Identify the required profiles X

B – Test for selection of candidates X

C - Training programme X X X

D - Technical evaluation of the trainees X

E – Human resources monitoring X X X

F - Quality Assurance Questionnaire for fieldwork staff

X

14 «


F - Quality Assurance Questionnaire for fieldwork staff

When completing the Pilot Survey, a questionnaire of quality control will be administered to all fieldwork staff including Municipal Delegates, Civil parish Coordinator and the enumerators.

The questionnaire is intended to evaluate the logistics and the procedural aspects of the Pilot Survey operation and will evaluate the level of fieldwork staff motivation and their perception of operational and organizational problems. «


15 «

«4. Delivery and collection processes

The distribution and collection of questionnaires in a properly and thoroughly manner, are key tasks of the enumerator.

These activities are of great importance to ensure the quality of a census.

The focus on the fieldwork supervision is to monitor and assess the proper implementation of procedures carried out by the enumerators.

In order to closely monitor the fieldwork, two mechanisms were developed: Individual fieldwork Control procedure and the Alert Indicator System.

«


16 «


In the processes associated with delivery and collection we establish the following strategic goals:

1 – To meet the deadlines for delivery of support material for enumerators;

2 – To meet the deadlines for delivery and collection of questionnaires to the population;

3 – To minimize the error rate in completing the questionnaires;

4 – To implement procedures for fieldwork control;

5 – To detect and correct in real time the possible deviations,

6 – To identify the critical points of the process.

«


17 «

«4. Delivery

and collection

processes

«


Indicators Targets /Tolerance

1.1. - percentage of civil parish with all the supporting material delivered until February 28

2.1. - percentage of questionnaires delivered to the public until April 4 (1st progress report by the civil parish coordinators)

2.2. - percentage of questionnaires delivered to the public until April 11

2.3. - percentage of questionnaires collected in paper and Internet versions until April 25 (Progress report)

2.4. - percentage of questionnaires collected in paper until May 16

3.1. - percentage of housing properly addressed to the enumerators via SMS

4.1. - percentage of questionnaires checked by the coordinator/subcoordinator (paper + Internet) in a sample of 5% of housing (Fieldwork control report)

4.2. - percentage of re-interviewing housing (paper + internet) provided by the civil parish coordinator/subcoordinator

4.3. - percentage of dismiss enumerators in reason of infringement procedures

5.1. - percentage of civil parishes with positive deviations not justified relatively to expected values in housings (Variances analysis report)

5.2. - percentage of civil parishes with negative deviations not justified relatively to expected values in housings (Variances analysis report)

5.3 - percentage of civil parishes with positive deviations not justified relatively to expected values in persons (Variances analysis report)

5.4. - percentage of civil parishes with negative deviations not justified relatively to expected values in persons (Variances analysis report)

6.1. - percentage of civil parishes in need of reassessment of the enumerator's work resulting from the analysis of deviations

100% / 1%

50% / 10%

00 % / 5%

50% / 10%

100% / 5%

98% / 1%

100% / 5%

100% / 5%

0%/5%

0% / 10%

0% / 10%

0% / 10%

0% / 10%

5% / 5%

18 «


«


Matrix Initiatives/Strategic goals Strategic goals

Initiatives 1 2 3 4 5 6

A – Assessment of the situation X X

B – Fieldwork control X X

C – Failure of procedures X X

D – Alert indicators system (SIA) X X

E – Variances analysis report X X X

F – Analysis and justification for deviations X

19 «


D – Alert Indicators System (SIA);

SIA will enable the monitoring of the quality of information during the collection process and will allow correcting deficiencies prior to completion.

It is intended to monitor the results of the statistical units of dwelling and persons by comparing to expected values.

When deviations are identified according to established parameters, it is triggered an additional checking procedure of the fieldwork.

«


20 «

«5. Conclusions

Monitoring and evaluating the quality of the census involves complex challenges that need adjusted solutions.

The success of its use in the Pilot Survey of Census 2011 encourages its adoption in the final Census operation.

For the Pilot Survey, we have identified two key processes to monitor: human resources; delivery and collection. For each of these processes, the objectives, indicators, targets and tolerances, and actions /initiatives are established. «


21 «

«5. Conclusions

In this presentation, we intend to share with other experts in quality management in this forum, the control and monitoring procedures that Statistics Portugal is preparing to adopt for Census 2011.

This is to ensure the implementation of a monitoring and quality assessment process which can detect and correct in real-time any rising deficiency during the various stages of the operation and to ensure that the final results are doubtless as to its quality.

«


22 «

«

Thank you for your attention

e-mail:[email protected] ;

[email protected]; [email protected].

«


Documents

European Conference on Quality in Official Statistics Session 26: Census 2011 « Helsinki, 6 May 2010 « Census quality control with BSC: the Portuguese