Workshop on Improving Gender Statistics in Rwanda Session 5 Statistics’ Quality and Comparability:...
If you can't read please download the document
Workshop on Improving Gender Statistics in Rwanda Session 5 Statistics’ Quality and Comparability: Metadata and International Comparisons Serena Lake Kivu
Workshop on Improving Gender Statistics in Rwanda Session 5
Statistics Quality and Comparability: Metadata and International
Comparisons Serena Lake Kivu Hotel, Rubavu District March 25-27,
2014
Slide 2
Learning Objectives At the completion of this module,
participants should be familiar with: What are metadata and why are
they important? Examples of metadata that can be used for gender
indicators Suggestions for producing the GSF metadata Why and How
to compare gender statistics internationally Current Problems with
International Data Comparability International mechanisms for
keeping informed about developments on gender statistics Sources:
Gardner, Jessica. Importance of metadata. Workshop on Writing
Metadata for Development Indicators Lusaka, Zambia, 30 July 1
August 2012, UNECA and African Union Oakley, Graeme, Australian
Bureau of Statistics.
www.unescap.org/stat/apex/2/APEX2_S.4_conference_Statistical%20Metadata%20Standards.pdfwww.unescap.org/stat/apex/2/APEX2_S.4_conference_Statistical%20Metadata%20Standards.pdf;
OECD, Management of Statistical Metadata at the OECD, V/ 2.0,
6/9/2006. http://www.oecd.org/std/33869551.pdf
http://www.oecd.org/std/33869551.pdf UN Statistics Division,
Department of Economic and Social Affairs. Millennium Development
Goals Indicators. Series Metadata,
http://mdgs.un.org/unsd/mdg/metadata.aspx World Bank. 2
Slide 3
Part 1. Metadata for gender Statistics 3
Slide 4
What are Metadata? Provide information that defines or
describes the data or statistics Describe the data collection,
production, processing, computation and analysis process as well as
the content and source of the data Also discuss the limitations and
quality of the data Created and used throughout the data production
process Respond to and inform national standards and systems 4
Metadata: the range of information, generally textual, that fosters
understanding of the context in which statistical data have been
collected, processed and analyzed with the objective of creating
statistical information African Charter for Statistics (2009)
MetadataMetadata provide information on data and about processes of
producing and using data. Metadata describe statistical data and -
to some extent - processes and tools involved in the production and
usage of statistical data. UNECE, "Guidelines for the Modeling of
Statistical Data and Metadata (1995).
Slide 5
Metadata include information not only about the ingredients
(components) but also about how the data were produced, i.e., the
process
Slide 6
Why Metadata are important for reporting on Gender and
Development Indicators Inform users about the source, definition,
collection process and limitations of the data Provide users with
knowledge and understanding of statistics availability and use
Clarify indicators from multiple data sources with different
definitions, data collection process, dates, etc. Explain
discrepancies in indicators estimates--e.g., for the MDGs Guide new
data collection and statistics and indicators production 6
Slide 7
Example 1. Metadata for Rwanda GSF Indicators: What to look for
There is no international consensus about what appropriate metadata
should contain -- different producers include different information
items Example 1: OECD - List of common metadata items: 7
ItemContent 1.Source from which data was submitted or extracted,
including original data source (administrative data, household
survey, enterprise/ establishment survey,); periodicity; date when
data was last received 2.Characteristics and collection unit of
measure, power code, variables collected, sampling, periodicity,
reference period, base period, date last updated, other
3.Statistical population and scope of data geographic, sector,
institutional, item, population and product coverage 4.Statistical
concepts and classifications used key statistical concepts and
classifications 5.Manipulation & dissemination aggregation and
consolidation; estimation, imputation, transformations, weights,
seasonal adjustment, other manipulation and adjustments; OECD
dissemination format, related publishing 6.Other
aspects:recommended uses and limitations, quality comments, other
comments Source: OECD, Management of Statistical Metadata at the
OECD, V/ 2.0, 6/9/2006. http://www.oecd.org/std/33869551.pdf
Slide 8
Example 2: World Banks World Development Indicators Metadata
The metadata for the World Banks World Development Indicators
includes the following: 1.Code 2.Development relevance
3.Statistical concept and methodology 4.Indicator Name 5.Long
definition 6.Source 7.Topic 8.Periodicity 9.Aggregation method
10.Limitations and exceptions 11.General comments Handout 5.1 has
an example of the WDI metadata for Under-5 mortality rate This same
metadata is being developed for the Banks Gender Statistics
database. Source: World Bank, World DataBank. World Development
Indicators.
http://databank.worldbank.org/data/views/variableselection/selectvariables.aspx?source=world-
development-indicators 8
Slide 9
Example 3. Millennium Development Goals (MDGs) Indicators
Metadata 9 Source: UN Statistics Division, Department of Economic
and Social Affairs. Millennium Development Goals Indicators. Series
Metadata, http://mdgs.un.org/unsd/mdg/metadata.aspx The metadata
for the MDGs includes the following items: 1.Contact point in the
international agency that produces the data 2.Definition 3.Method
of computation 4.Comments and limitations 5.Sources of
discrepancies between global and national figures 6.Process of
obtaining data 7.Treatment of missing values 8.Data availability
9.Regional and Global estimates 10.Expected time of release
Slide 10
Example 5: Metadata for the UN Minimum Set of Gender Indicators
(MSGI) UNSD has compiled the metadata for the set based on
information from data collection agencies: WHO, UNESCO, ILO, etc.
The data and metadata are provided by international agencies based
on national data or information reported to them Some data are old
or are estimates or projections produced by the international
agencies Rwanda has more recent data that is not included in the
MSGI Metadata are available for 46 indicators; all but 2 are Tier I
indicators (they meet the 3 criteria discussed yesterday) The
information is the result of consultation and agreement among the
data collection agencies. The metadata highlight the problems and
limitations of the data for international comparability; Some
problems or limitations highlighted for the international level may
not be relevant at the national data because there may be more
detailed information about the data. 10
Slide 11
Metadata for the UN Minimum Set of Gender Indicators (2) The
metadata contain information on 10 areassimilar to MDGs metadata:
1.Indicator Name: 2.Contact point in international agency
3.International agreed definition 4.Method of computation
5.Importance of the indicator in addressing gender issues and its
limitation 6.Sources of discrepancies between global and national
figures 7.Process of obtaining data 8.Treatment of missing values
9.Data availability and assessment of countries capacity
10.Expected time of release For Rwanda, the UN MSGI has data for 34
of the 52 indicators, although there are no metadata for 2 of them
see Handout 5.2 UN MSGI metadata can serve as a source or example
for producing GSF metadata: They can help improve the collection
and quality of the gender indicators and statistics, But, they may
not be suitable for the GSF because they were produced for the
international and not the national level they provide information
that is not relevant at the national level 11
Slide 12
Rwanda GSF Indicators: examples Without Metadata Gender parity
index for secondary gross enrolment (females to males): 1.00
(Rwanda Education Statistics, EMIS, 2010) 1.04 (Rwanda Education
Statistics, EMIS, 2011) Literacy rate among population aged 15-24,
by sex (Youth Literacy rate): Female: 85% (DHS 2010) Male: 83% (DHS
2010) Life expectancy at birth (years): Female: 54.8 (Rwanda
Population Projection, 2009) Male: 50.8 (Rwanda Population
Projection, 2009) Maternal mortality ratio, MMR (maternal deaths
per 100,000 live births: 476 (DHS 2010) 12
Slide 13
Metadata for Rwanda GSF Indicators: Gender parity index for
secondary gross enrolment (females to males) With Metadata Gender
parity index for secondary gross enrolment: Definition: Index is
the ratio of female to male gross enrollment ratios in secondary
education. Gross enrolment ratio, GER: total enrollment in
secondary education, regardless of age, expressed as the percentage
of the population of official secondary education age (World Bank
Metadata). Can be calculated separately for females and males.
Numerator: population enrolled in secondary school Denominator:
total population of official secondary school age GER can exceed
100% due to the inclusion of over-aged and under-aged students
because of early or late school entrance and grade repetition
(World Bank WDI Metadata). Unit: ratio Computation: Numerator: GER
Females (enrolled females as a % of all females of secondary school
age) Denominator: GER males (enrolled males as a % of all males of
secondary school age) Measurement/Estimation: GER females/GER
males. Source: Administrative dataRwanda Education Statistics,
January 2011 When and how were the data actually collected? Lead
Agency/producer: Ministry of Education? 13
Slide 14
Metadata for Rwanda GSF Indicators: Literacy rate among
population aged 15-24, by sex (Youth Literacy rate): With Metadata
Definition: The percentage of the population aged 1524 years who
can both read and write with understanding a short simple statement
on everyday life. (Source: UIS) Unit: % Computation: Literacy rates
are computed by dividing the number of persons [females or males]
aged 15-24 years who are literate by the total [female or male]
population in the same age group. The result is then multiplied by
100 to yield the literacy rate in per cent. (Source: UIS)
Importance of the indicator in addressing gender issues and its
limitation: The Youth Literacy Rate reflects the outcome of primary
education over the previous decade. As a measure of the
effectiveness of the education system, it is often seen as a proxy
measure of social progress and economic achievement. Reasons for
failing to achieve the literacy standard may include
non-attendance, low quality of schooling or dropping out before
completion of primary education. Differences in literacy levels
between young women and men will often reflect recent inequalities
in access to formal education and persisting inequalities in adult
life and the world of work. (Source: UIS) Source: DHS 2010 Lead
Agency/producer: Measure DHS? 14
Slide 15
Metadata for Rwanda GSF Indicators: Life expectancy at birth by
sex With Metadata Life expectancy at birth: Definition: Estimate of
the average number of years a newborn is expected to live based on
current age-specific mortality rates. Life expectancy at birth by
sex gives a statistical summary of current differences in male and
female mortality across all ages. In areas with high infant and
child mortality, the indicator is strongly influenced by trends and
differentials in infant and child mortality (Source: UN DESA,
Population Division, 2011). Unit: Number of years Year/Date:
Calendar year when data were collected -- When and how were the
data actually collected? Source: Rwanda Population Projection 2009
Based on which data sources: census, survey, administrative
records, several sources? Assumption: Current age specific death
rates/mortality patterns will remain constant in the future Lead
Agency/producer: NISR? 15
Slide 16
Metadata for the Rwanda Maternal Mortality Ratio, MMR
Definition: Annual number of female deaths from any cause related
to or aggravated by pregnancy or its management (excluding
accidental or incidental causes) during pregnancy and childbirth or
within 42 days of termination of pregnancy, irrespective of the
duration and site of the pregnancy, per 100,000 live births, for a
specified year (WHO). Unit: Ratio Computation: Numerator: For
Rwanda: Any death that occurred during pregnancy, childbirth, or
within two months after the birth or termination of a pregnancy.
Includes all deaths occurring during the specified period even if
due to causes that are not pregnancy related (DHS 2010).
Denominator: International convention--Number of live births (in
100,000s), based on either a written record or the mother's recall.
MMR for Rwanda-DHS 2010: Expressed per 100,000 live births;
calculated as the maternal mortality rate divided by the
age-adjusted general fertility rate, which is the average number of
live births per 1,000 women of reproductive age (age 15-44) (DHS
2010). Year/date: Fieldwork was conducted September 2010-March 2011
(DHS 2010). Measurement/estimation: For Rwanda: Women respondents
reported the number of their sisters who died, and the number who
died of maternity-related causes. No definitive procedure for
establishing completeness or accuracy of retrospective data on
sibling survivorship (DHS 2010). Source: DHS 2010. Lead Agency:
MEASURE DHS (previously Macro International) Limitations of the
indicator: Based on recall of deaths and live births by key
informant (mothers sister or mother). DHS uses a different
denominator to overcome the limitations of the recall. 16
Slide 17
Suggested steps for producing metadata for the GSF indicators:
Not a prescriptive or exhaustive list : 1.Decide who to partner
with on this work e.g., NISR metadata initiative Possible pilot
with one data collection exercise-e.g., survey 2.Decide which
template to use Adopt features from different examples and
customize them for Rwandas needs May involve consultation with data
producers and users Explore experiences from other countries in
Africa or developing countries 3.Decide which indicators to focus
on first: main ones, by sector, or other criteria GSF has hundreds
of indicators so it will take a lot of time and work to produce
metadata for all Consider piloting in one sector or one data
collection initiative 4.Go to the data sources for Rwanda to find
out information about data collection process, definitions used,
limitations and problems, coverage and response rate, collection
dates, etc. May involve reviewing data collection manuals,
templates, questionnaires and consulting with staff involved in the
process 17
Slide 18
Suggestions for producing metadata for the GSF indicators
5.Complement with information from international sources when
appropriate e.g., internationally agreed definitions, importance
from a gender perspective, limitations, comparability 6.Consult and
negotiate with data producers and compilers about feasibility of
using the framework or template in censuses, surveys,
administrative data collection NISR, line ministries, civil
registration personnel, international agencies This may involve an
iterative and continuous process to adjust and improve the template
7.Agree on and harmonize the metadata with all the data producers
8.Publish and circulate the metadata to all producers and users
9.Train data collectors on how to produce and report the metadata
Training may need to be provided to data collectors, compilers and
reporters every time a new data collection is started 18
Slide 19
Disseminated data should always be accompanied by metadata
Disseminated data should be accompanied by metadata to help users
understand the data. Can be included after the data (each section),
as an annex, or as links to the indicators in electronic format
Metadata should include, but is not limited to, information on:
Concepts, definitions and classifications used Basic features of
the data sources Data collection methodology: censuses, surveys,
administrative records Guidelines on use of the data Data quality
(e.g. sampling and non-sampling error, non-response rates, data
comparability). In some circumstances, it can be useful to release
particular types of metadata in a dedicated publication. For
example, in Vietnam, the GSO published a Gender Statistical
Handbook in 2011. 19 Important!
Slide 20
Part 2. Comparing Gender Statistics Internationally 20
Slide 21
Benefits of International Comparability of Gender Statistics
Similarities and differences in gender issues between individual
countries and between regions can be studied and relative progress
on gender- related goals can be assessed by undertaking data
comparisons across countries or regions. The overall quality of a
countrys statistics can be enhanced because producing comparable
statistics involves adoption of international standards and best
practice in methodology. Gender issues and developments can be
analysed in an international context by combining statistics across
countries to produce regional and global aggregates. 21
Slide 22
Bringing gender statistics from different countries together:
The World Bank Gender Statistics electronic database
http://datatopics.worldbank.org/gender/
http://datatopics.worldbank.org/gender Allows users to compare
statistics on gender for regions and countries in 6 areas: economic
structures and access to resources; education; health and related
services; public life and decision-making; and human rights of
women and girl children. The data come from the World Development
Indicators and additional sources.World Development Indicators
Users can create their own tables and download the data into Excel
The UNSD publication The Worlds Women 2010, Trends and Statistics
highlights the differences between the status of women and men in
various areas of contemporary life. covers 196 countries across the
world. presents and analyses data at global, regional, and
individual country levels.
http://unstats.un.org/unsd/demographic/products/Worldswomen/Executive%2
0summary.htm
http://unstats.un.org/unsd/demographic/products/Worldswomen/Executive%2
0summary.htm The following charts from that publication illustrate
how country data can be brought together to inform gender issues in
a wider context. 22
Slide 23
Global sex distribution Source: UNSD The Worlds Women 2010:
Trends and Statistics 23
Slide 24
Regional sex distribution Source: UNSD The Worlds Women 2010:
Trends and Statistics 24
Slide 25
Time spent in domestic work per day, 1999-2008 Source: UNSD The
Worlds Women 2010: Trends and Statistics 25
Slide 26
Employed persons in vulnerable employment by region and sex,
2004-2007 Source: UNSD The Worlds Women 2010: Trends and Statistics
26
Slide 27
Current Problems with International Data Comparability Gaps in
the availability of gender statistics in many countries For
example, in 12 of the 22 topic areas specified in the 2012 UNSD
Global Review of Gender Statistics Programmes, more than a quarter
of 126 countries reviewed were not producing any gender statistics.
Lack of comparability in many of the gender statistics that are
available for individual countries International standards for
producing comparable gender statistics are not available or
incomplete for some topics Existing international standards are not
always fully implemented by countries For example, a country may
consider that a particular standard classification is not useful,
impractical or inappropriate in its circumstances and therefore not
adopt the standard. 27
Slide 28
International mechanisms for keeping informed about
developments on gender statistics United Nations Statistical
Commission (UNSC)annual meetings Reports on gender statistics
produced and available for all meetings in the last few years. UNSD
Global Forum on Gender Statistics biennial meetings A range of
documents, presentations and reports are publicly available. The
last Global Forum was held in Jordan in March 2012 and focused on
womens empowerment; Next Global Forum is planned for 2014 and
expected to focus on gender analysis and use of gender data and
indicators. UNSD regional meetings and training workshops UNECA
meetings 28
Slide 29
Exercise 5.1: Group Activity 1.Provide several examples that
illustrate the value of producing gender statistics for Rwanda that
are comparable with those of other countries. 2.What are the main
obstacles to producing internationally comparable data on gender
statistics? 3.What actions would need to be taken, and by whom, to
ensure gender statistics in Rwanda are internationally comparable?
29