United Nations Workshop on Evaluation and Analysis of Census Data, 1-12 December 2014, Nay Pyi Taw, Myanmar DATA VALIDATION-I Evaluation of editing and

United Nations Workshop on Evaluation and Analysis of Census Data, 1-12 December 2014, Nay Pyi Taw , Myanmar

DATA VALIDATION-IEvaluation of editing and imputation


Census processing overview

Steps of data processing depend on the technology used in general, the process covers the following steps:


Validation

It is a process of checking consistency in data after editing/imputation phase of the census:

Editing rules may be insufficient to identify all types of errors

Editing/imputation may introduce new errors in data because of incorrect application

Some unexpected patterns may not be identified with editing/consistency rules


Validation

In general, two methods for data validation

Evaluation of performance of editing/imputation to ensure correct application or imputation

Analysing key aggregated data to check consistency among variables and with expected values/distribution to identify the unusual values/pattern


Basic definitions

Editing: List of rules to determine invalid and inconsistent data

Imputation : The process of resolving problems concerning invalid or inconsistent data – and missing values- identified during editing

All records must respect a set of editing rules formulated to correct errors and finally disseminate reliable data


Some examples for invalid data-Myanmar pilot census questionnaire

Age Equal to 99

Instruction – if it is greater or equal to 98, write 98 If age is written in one digit, such as

How to correct? Place of birth, place of usual residence and place of

previous residence If code given by enumerators is not consistent with the code

list or code written in one or two digitsHow to correct?

1

5


Some examples for inconsistent data-Myanmar pilot census questionnaire

Age and marital status If age of married person is below the minimum age at first

marriage Children ever born alive, living and dead children

If number of children ever-born is not equal to the sum of number of living children and number of dead children

Last live birth and household deaths There is an infant birth who is not alive, but no infant death

registered in the household deaths

What will be decision?


Some examples for inconsistent data-Myanmar pilot census questionnaire

Sex, age and relationship to the head of household If sex of the head of household and spouse is same If age difference between the head of household and

son/daughter is less than 13 or 14

Age, the highest completed level of education and occupation Age is 9, completed level is primary school and the person is

secondary school teacher

What will be decision?


Assessing the performance of imputation

After implementation of editing/imputation: Data should be classified as follows :

Observed (consistent) data: the values which meet with all editing rules

Non-response or unknown : no value Inconsistent data : the values which failed at least one

editing rule Imputed data for inconsistency –and non-response

For this analysis, all procedures performed in the database should be identifiable



1. Compare the distribution of the observed values with the distribution of the imputed values if non-response and inconsistent data are distributed

randomly, no difference is expected between the distribution of

the observed and the imputed values

If there are differences between the people who responded and those who did not or not give accurate data The imputed data should not follow the same

distribution as the observed data


Assessing the performance of the imputation

2. Compare the distribution of the observed values with the distribution of all values including the imputed values

In general, imputed values should have a minimal effect on the distribution of the complete data

Unless the non-response rate is particularly high or the bias for certain characteristics


Source: England and Wales, Office for National Statistics, 2011 Census:Item Edit and Imputation: Evaluation Report, June 2012


Table 2: Distribution of bedrooms

Observed responses

Imputed responses

Difference Total Change

Number of bedrooms

(Imputed-Observed) Including imputed

(total-observed)

N % N % % N % % (1) (2) (3) (4) (5)=(4)-(2) (6)=(1)+(3) (7) (8)=(7)-(2)

0 62 0.3 5 0.8 0.5 67 0.3 0.0141 2,378 10.7 124 19.2 8.5 2,502 10.9 0.2402 6,097 27.4 192 29.8 2.3 6,289 27.5 0.0663 9,375 42.2 228 35.3 -6.8 9,603 42.0 -0.1924 3,279 14.7 70 10.9 -3.9 3,349 14.6 -0.1105 809 3.6 19 2.9 -0.7 828 3.6 -0.0206 166 0.7 5 0.8 0.0 171 0.7 0.0017 39 0.2 1 0.2 0.0 40 0.2 -0.001

8 or more 27 0.1 1 0.2 0.0 28 0.1 0.001

Total 22,232 100 645 100 0.0 22,877 100 0.000




Maximum change








Understanding data editing and potential errors


Boundary of school age

Boundary of working age



Summary indexes at the variable level

Maximum absolute percent change Maximum absolute percent change across all categories for

each variable

Dissimilarity Index Degree of change of two distributions (observed and total

including imputed values) at the variable level

Imputation rate Share of the imputed records in the total records




Maximum absolute percent change between the observed and final (imputed) distributions across all categories within each of the questions


Maximum absolute percent change between the observed and final (imputed) distributions across all categories within each of the questions




Index of dissimilarity

To assess the degree of change induced by imputation on the initial distribution of variables

K

kyy kk

ffID1

*

2

1

Where;k : categories of the variablef : percentage distribution of the variable before imputationf * : percentage distribution of the variable after imputation



K

kyy kk

ffID1

*

2

1

It assumes a 0 value when the two distributions before and after imputation are equal

It is greater than 0 when they are different and reaches its maximum value of 100 when there is maximum dissimilarity between the two distributions when both are concentrated in one category which is

different from each other

0 ≤ ID ≤ 100






Source: Albania, Quality Dimensions of 2011 Population and Housing Census, May 2014



Source: Albania, Quality Dimensions of 2011 Population and Housing Census, May 2014


Hands-on exercises

England and Wales – 2011 Census

A. Marital and civil partnership B. Distribution of highest level attended

Documents

United Nations Workshop on Evaluation and Analysis of Census Data, 1-12 December 2014, Nay Pyi Taw, Myanmar DATA VALIDATION-I Evaluation of editing and