Data sources and collection methods

Ken Mease

Cairo, June 2009

What types of Data?

A thorough assessment may well include:• Archival and secondary data• Survey data • Quantitative and qualitative approaches and

data• It also will likely include de jure and de facto

information

Qualitative and Quantitative data

• There are basically two types of data: qualitative and quantitative

• Qualitative data are usually text or words and quantitative data - numbers

• Qualitative approaches, if conducted in a rigorous manner, require more skill than most quantitative approaches

Qualitative Data and Approaches

• Qualitative approaches provide text data, but increasingly audio, video and images

• They are more time consuming to analyze

• Text management software, such as NVIVO, AtlasTI and AnSWR (available free at http://www.cdc.gov)

• Coding is a very subjective process and open to various problems, such as investigator bias or a lack of inter-coder reliability

Quantitative Data and Approaches

• Quantitative approaches provide increased rigor by investigating relationships at known levels of probability

• They are easier to analyze because researchers use standard, replicable techniques

• Common software includes SPSS, Stata and SAS

 

Sources and Types of Governance Data

• Country reports/desk studies • Cross-country (region)comparative surveys• Expert assessments• Government data• Household surveys• Mass opinion surveys• Key Stakeholder Surveys• Media Indicators• Business surveys

Combining different sources and methods

• Combining methodologies and types of data often provides the most useful results.

• Combining archival information and administrative data with original qualitative and quantitative survey data allows for triangulation

• This approach can increase the level of professionalism, credibility and legitimacy.

Administrative, archival and secondary

These data, both qualitative and quantitative, objective, reported events, perception and proxies come from a variety of sources:

•Narrative reports, administrative data and other information routinely collected by government ministries

•The constitution, laws, policies and legislation

•Statistics and data gathered by NGOs, international organizations and academics.

Data Mapping

• It is the best way to identify existing data - accessibility, quality and gaps

• One tool available to assist with this process is the IMF’s Data Quality Assessment Framework

• A senior academic conducted the data mapping exercise in Zambia and was very valuable

Survey Data

• Qualitative and Quantitative

• Sample sizes can range from 20 to 20,000 or more

• Costs can range accordingly

• There are a range of options for who collects the data – independent surveys firms, academics, NSOs

• Perception and reported events data

Different Types of Interviewing

• Structured interviews use an identical instrument for each respondent,

• Interviewers are trained and have explicit instructions.

• This technique uses primarily structured questions with fixed response sets and very few open-ended questions

Types of Interviewing

• Semi-structured interviews use a written list of questions that need to be covered in a particular order

• The questions are often developed from informal discussions and focus groups.

• They can include open-ended and/or more structured questions

• Ideal when working with elites, managers, bureaucrats and other people who have limited time

Types of surveys and data collection methods

• Face-to-face data collection is likely the best option in most developing countries

• It is also the most expensive and time-consuming

• It requires professional management of trained interviewers, sampling and other aspects of the study

Mail surveys

• Work well only if the postal system is reliable

• The questionnaire must be carefully designed for self-administration, and there should not be too many language issues

• The cost is usually quite reasonable

• It misses the homeless and other vulnerable groups who may not have a valid mailing address

Internet surveys

• Require special planning and design

• Very good for certain populations

• These surveys can suffer from low response rates.

• Developing and implementing internet surveys has become very reasonable in terms of cost

• Very reasonably priced internet services are available - Survey Gizmo at www.surveygizmo.com

Telephone surveys

• May have coverage issues in most countries.

• In many countries, people do not have a phone in their home, and if they do have a phone, it is usually a mobile or cell phone

• Most cell phone providers do not make telephone numbers available for use in random-digit-dial surveys.

Focus groups

• Can generate information about the background conditions surrounding governance issues.

• Focus groups are usually efficient in terms of time and money.

• They are highly participatory and have the potential of generating solutions to the problems identified by the group members

• It is very demanding and requires highly skilled coordinators.

Focus groups

• Make individual ratings insignificant

• Accuracy can suffer, as some individuals may not feel comfortable to speak up in public, while others are hard to keep quiet

• Focus groups yield less systematic results

• Focus groups are best used to identify issues and develop surveys rather than as the only source of data

Who collects the data

• Outside contractors – surveys and desk studies

• Local survey researchers

• Local academics for desk studies

• National Statistical Offices

• Government ministries

• NGOs/CSOs

Data collection and dissemination exercise

• Pick a Governance issue in your country

• Choose the levels and types of the data you will collect – Macro, Micro, De jure and De facto

• Identify the types and sources of data

• Choose the data collection method/s

• Identify who will collect the data

• Select several options for sharing the results with different audiences