12015

1990 Module 10: Module 10:

Data Sources and MetadataData Sources and Metadata

Tools for Civil Society to Understand Tools for Civil Society to Understand and Use Development Data: and Use Development Data:

Improving MDG Policymaking and Improving MDG Policymaking and MonitoringMonitoring

22015

1990

What you will be able to do byWhat you will be able to do by the end of this module the end of this module

• Identify and use effectively the different types of Identify and use effectively the different types of data used to produce indicatorsdata used to produce indicators

• Understand the concept of metadata and the role Understand the concept of metadata and the role it plays in using indicatorsit plays in using indicators

32015

1990

Sources of DataSources of Data

• Administrative or routine dataAdministrative or routine data

• Census data Census data

• Survey dataSurvey data

• Surveillance dataSurveillance data

42015

1990

Administrative (or Routine) DataAdministrative (or Routine) Data

• Primary purpose is management of processesPrimary purpose is management of processes

• Summary and/or dissemination occurs later Summary and/or dissemination occurs later (but usually within one or two years)(but usually within one or two years)

Example

1.1. Registration of birthRegistration of birth– Record birth a child, to be used later for

purposes of child health, education, etc.

2.2. Immunization against measlesImmunization against measles– Purpose is to manage the national

immunization programme

52015

1990

Administrative SourcesAdministrative Sources

• Vital registration (births, deaths, etc)Vital registration (births, deaths, etc)

MoldovaMoldova:: the civil registration offices (birth, death, marriages the civil registration offices (birth, death, marriages and divorces). Information on migration is received from the and divorces). Information on migration is received from the Ministry of Information Development and the National Ministry of Information Development and the National Bureau for MigrationBureau for Migration

Belarus:Belarus: the data on vital events comes from the acts of civil the data on vital events comes from the acts of civil status registered by the Offices of Civilian Registration status registered by the Offices of Civilian Registration (ZAGS), and data on migration of population is collected (ZAGS), and data on migration of population is collected from the Ministry of Internal Affairsfrom the Ministry of Internal Affairs

• Health systems (immunization rates, mortality Health systems (immunization rates, mortality rates, maternal health data, etc.)rates, maternal health data, etc.)

62015

1990

Administrative Sources and Statistical Administrative Sources and Statistical authoritiesauthorities

• Education (Enrollment and completion data)Education (Enrollment and completion data)

BelarusBelarus: : NSCNSC collects annual information on pre-school, higher, collects annual information on pre-school, higher, and special secondary education and the and special secondary education and the Ministry of Ministry of EducationEducation –general secondary education and out-of-school –general secondary education and out-of-school education, on educational establishments for children with education, on educational establishments for children with psychophysical traits, vocational technical education, psychophysical traits, vocational technical education, children's homes, and tutelage authorities. children's homes, and tutelage authorities. student-teacher student-teacher ratio, etc.)ratio, etc.)

MoldovaMoldova: For primary and secondary education, NBS collects : For primary and secondary education, NBS collects information from private schools only. Information about other information from private schools only. Information about other schools is received from the Ministry of Education; data are schools is received from the Ministry of Education; data are processed by the NBS. For other types of education, the data processed by the NBS. For other types of education, the data are collected by NBS itself, from colleges, professional are collected by NBS itself, from colleges, professional schools and universitiesschools and universities

72015

1990

Administrative Sources and Statistical Administrative Sources and Statistical authoritiesauthorities

• Employment (numbers employed, industry, level of Employment (numbers employed, industry, level of participation)participation)

Moldova:Moldova: NBS of Moldova is responsible for Labour Force NBS of Moldova is responsible for Labour Force SurveySurvey

Belarus:Belarus: There is no Labour Force Survey There is no Labour Force Survey

• Business (Industry, sector, size)Business (Industry, sector, size)

Moldova:Moldova: NBS of Moldova is responsible for business register NBS of Moldova is responsible for business register

Belarus:Belarus: NSC is responsible for NSC is responsible for keeping of a business keeping of a business register. Information comes from the registration authorities register. Information comes from the registration authorities responsible for state registration/ cessationresponsible for state registration/ cessation

82015

1990

CensusesCensuses

• Collect data from every unit in the populationCollect data from every unit in the population

• 100% coverage (in theory)100% coverage (in theory)

• ExpensiveExpensive

• Time consumingTime consuming

92015

1990

Population CensusPopulation Census

• Identify each member of the populationIdentify each member of the population

• Collect certain basic data about themCollect certain basic data about them

– age, gender, location, etc. (Moldova 2004, age, gender, location, etc. (Moldova 2004, Belarus 2009)Belarus 2009)

• Modules to collect data on specific topics may be Modules to collect data on specific topics may be addedadded

• Normally about every 10 yearsNormally about every 10 years

• Modeling methods used to generate population Modeling methods used to generate population estimates between censusesestimates between censuses

• Good for small-area poverty mappingGood for small-area poverty mapping

102015

1990

Population Census DataPopulation Census Data

• AdvantageAdvantage– Excellent coverageExcellent coverage– Creates sampling frame for household surveysCreates sampling frame for household surveys

• DisadvantageDisadvantage– Potential for some bias – for example, could miss Potential for some bias – for example, could miss

nomadic groups or homelessnomadic groups or homeless– May be inaccurate due to infrequencyMay be inaccurate due to infrequency– Limited data collectedLimited data collected– Lag before data producedLag before data produced

112015

1990

Establishments CensusesEstablishments Censuses

Censuses of businesses, hospitals, other Censuses of businesses, hospitals, other organizationsorganizations

• Provide a frame for later surveysProvide a frame for later surveys

• Collect basic data, as for population censusCollect basic data, as for population census

• Problems:Problems:

– Smaller or informal establishments often Smaller or informal establishments often excludedexcluded

– Establishments may change more frequently Establishments may change more frequently than householdsthan households

122015

1990

Sample SurveysSample Surveys

• Vehicle for collecting data from a subset of Vehicle for collecting data from a subset of the populationthe population

• Advantages of focusing on a subset:Advantages of focusing on a subset:– Save money and resourcesSave money and resources– Reduce time to collect dataReduce time to collect data– Reduce time to analyse dataReduce time to analyse data

132015

1990

Characteristics of Sample SurveysCharacteristics of Sample Surveys

• Primary purpose is to get dataPrimary purpose is to get data

• Some units will be omittedSome units will be omitted

• Well designed survey should be representative of Well designed survey should be representative of the populationthe population

• There needs to be reliable sampling frame (e.g. There needs to be reliable sampling frame (e.g. generated from census data)generated from census data)

142015

1990

Sample Survey DataSample Survey Data

• AdvantageAdvantage– Cheaper and quicker than censusCheaper and quicker than census

– More frequent than census, though usually only More frequent than census, though usually only every every 11 to to 33 years years

– Can collect wider range of data than census and Can collect wider range of data than census and administrative systemsadministrative systems

– Reduced potential for bias than in census and Reduced potential for bias than in census and administrative dataadministrative data

• DisadvantageDisadvantage– Sampling error since coverage < 100%Sampling error since coverage < 100%

– Requires more sophisticated design to ensure Requires more sophisticated design to ensure consistency and accuracyconsistency and accuracy

152015

1990

Household SurveysHousehold Surveys

• Reporting usually takes about 1 year after Reporting usually takes about 1 year after completion of data collectioncompletion of data collection

• Focus is on socio-economic and health issuesFocus is on socio-economic and health issues

• Governments should develop inter-census survey Governments should develop inter-census survey programsprograms

162015

1990

Household SurveysHousehold Surveys (2) (2)

• Multi Indicator Cluster Surveys (MICS)Multi Indicator Cluster Surveys (MICS)

• Demographic and Health Surveys (DHS)Demographic and Health Surveys (DHS)

• Living Standards Measurement Study Surveys Living Standards Measurement Study Surveys (LSMS)(LSMS)

• World Health Surveys (WHS)World Health Surveys (WHS)

• Core Welfare Indicators Questionnaires (CWIQ)Core Welfare Indicators Questionnaires (CWIQ)

172015

1990

Household Surveys (Household Surveys (33))

• Household budget surveysHousehold budget surveysMoldova:Moldova: NBSNBS conducts Household Budget Survey (HBS),conducts Household Budget Survey (HBS),

the sample size is 9.000 households, half of which remain the sample size is 9.000 households, half of which remain in the sample for 4 years, while the other half is replaced in the sample for 4 years, while the other half is replaced after two years.after two years. Response rate is about 70% Response rate is about 70%

• Household income and expenditure surveysHousehold income and expenditure surveysBelarus:Belarus: Minstat conducts the Household Income and Minstat conducts the Household Income and

Expenditure Survey (HIES), which is held annually. The Expenditure Survey (HIES), which is held annually. The sample consists of 6,000 households. The households sample consists of 6,000 households. The households participated at the survey are rotated every year.participated at the survey are rotated every year. Response Response rate is 91%rate is 91%

182015

1990

Demographic Surveillance Demographic Surveillance Systems (DSS)Systems (DSS)

• Longitudinal monitoring of sentinel populations Longitudinal monitoring of sentinel populations (60,000 to 100,000)(60,000 to 100,000)

– Follow same people every year through life of Follow same people every year through life of surveysurvey

• 100% event registration100% event registration

• 40 countries40 countries

192015

1990

Demographic Surveillance Demographic Surveillance Systems (DSS) (Systems (DSS) (22))

• AdvantagesAdvantages– Coverage of sentinel pop = 100%Coverage of sentinel pop = 100%– Rapid data availabilityRapid data availability– Facilitates targeting and short term monitoringFacilitates targeting and short term monitoring

• DisadvantagesDisadvantages– Few large clusters – potential for serious biasFew large clusters – potential for serious bias– Expensive to include many clustersExpensive to include many clusters

http://indepth-network.htmhttp://indepth-network.htm

202015

1990

Data Sources ComparedData Sources Compared

CharacteristicCharacteristic AdminAdmin SurveySurvey DSSDSS CensusCensus

Inclusion Inclusion criterioncriterion

All ‘noticed’ All ‘noticed’ eventsevents

Designated Designated unitsunits

All events in All events in clustersclusters

All unitsAll units

CoverageCoverage

Variable, Variable, depending depending upon upon systemsystem

% coverage % coverage specifiedspecified

Coverage of Coverage of clusters onlyclusters only

~100% ~100% coveragecoverage

BiasBias

May be May be biasedbiased

Designed to Designed to minimize minimize biasbias

Urban/rural Urban/rural included, but included, but may not be may not be sufficiently sufficiently representativerepresentative

Lack of Lack of coverage coverage may lead to may lead to some biassome bias

212015

1990

Data Sources Compared (Data Sources Compared (22))

CharacteristicCharacteristic AdminAdmin SurveySurvey DSSDSS CensusCensus

CostCost CheapCheap MediumMedium MediumMedium ExpensiveExpensive

TimeTime

Ongoing, Ongoing, +1-2 years +1-2 years for for reportingreporting

3-5 years + 3-5 years + 1 year for 1 year for reportingreporting

Ongoing, Ongoing, report in < 1 report in < 1 yearyear

10 years + 2 10 years + 2 years for years for reportingreporting

Potential forPotential for

Policy/advocacyPolicy/advocacy

targetingtargeting

V good, but V good, but limited info, limited info, and and problem if problem if coverage coverage poorpoor

Good, but Good, but only for only for medium to medium to long termlong term

V good, but V good, but only for only for clusters and clusters and similar similar populationspopulations

Good for Good for long term long term and as input and as input with other with other datadata

222015

1990

Synergy Across Data SourcesSynergy Across Data Sources

• Potential to use different types of data together toPotential to use different types of data together to– Build a wider pictureBuild a wider picture– Provide a means for cross-checkingProvide a means for cross-checking

• Population estimates from censuses needed to Population estimates from censuses needed to supplement other sourcessupplement other sources

• Need to ensure compatibility ofNeed to ensure compatibility of– DefinitionsDefinitions– Time framesTime frames

232015

1990

MetadataMetadata

• Clearly vital to know, when using dataClearly vital to know, when using data– How they were collectedHow they were collected– When they were collectedWhen they were collected– By whomBy whom– Potential for biasPotential for bias– How data items are definedHow data items are defined– MethodologyMethodology

• The whole collection of this type of data is The whole collection of this type of data is called the called the metadata metadata

242015

1990

Major Purposes of MetadataMajor Purposes of Metadata

1.1. Confirm definitionsConfirm definitions

2.2. Facilitate decisions about compatibility of data Facilitate decisions about compatibility of data from different sourcesfrom different sources

3.3. Guides and validates the interpretation of data Guides and validates the interpretation of data and their indicatorsand their indicators

252015

1990

SummarySummary

In this module we have discussed In this module we have discussed

• The main sources of data used in the production The main sources of data used in the production of indicatorsof indicators

• The definition and importance of metadataThe definition and importance of metadata

262015

1990

• Which of the surveys discussed in this module are Which of the surveys discussed in this module are

carried out in your country? How frequently?carried out in your country? How frequently?

• Summarize the progress towards Goal 3, using the Summarize the progress towards Goal 3, using the

MDG report. Include in your assessment a MDG report. Include in your assessment a

discussion of the limitations of the data which discussion of the limitations of the data which

contribute to the relevant indicators. Use the contribute to the relevant indicators. Use the

metadata to inform this discussionmetadata to inform this discussion

Practical 10Practical 10