View
80
Download
0
Category
Preview:
Citation preview
CAPI Data Quality in
Namibia and Lesotho
Gregory Martin Methodology and Software Development Branch
International Programs Population Division U.S. Census Bureau
1
U.S. Census Bureau
International Programs
Promote international development and
capacity building on a reimbursable basis
through:
Technical assistance
Training
Software and methodological tools
2
What is CSPro?
Census and Survey Processing System First release in 2000
Initially designed for key-from-paper
Product Suite Data entry
Edits and imputations
Tabulation
Public domain
3
Who Uses CSPro?
Used in over 160 countries by:
National statistical offices (NSOs)
Other ministries (health, education)
Non-governmental organizations (NGOs)
Universities
Hospitals
Private sector
4
CSPro Applications
Censuses (population and housing, agriculture, and economic)
Demographic and labor force surveys
Health and nutrition surveys
Household income and expenditure surveys
Major international projects: Demographic and Health Survey (DHS), ICF
International
Multiple Indicator Cluster Survey (MICS), UNICEF
5
CAPI History
Windows Mobile (2006)
Unicode support (2012)
CSEntry Android application (2014)
6
CSPro Data Entry Features
Develop and test on Windows
Target multiple platforms
Multiple languages
Tightly controlled path
Powerful programming language
Data synchronization
7
USAID-Funded Survey Project
Namibia Statistics Agency Namibia Household Income and
Expenditure Survey (NHIES): 2015/2016
Rolling sample survey Demographic, economic, and
consumption modules Multiple visits to a household Roughly 10,000 households 10” Samsung tablets
8
UNFPA-Funded Census Project
Lesotho Bureau of Statistics
Lesotho Population and Housing Census: 2016
Single enumeration period
Around two million people
7” and 8” tablets from Lenovo, xTouch, and Click
9
CAPI Considerations
Designing applications as a distributed team
GPS and mapping
Case management
In-field automated sampling
Monitoring the quality of data collection
Data synchronization and backups
IT support
10
Distributed Application Design
CSPro application files are all text files and work well in revision control systems
Teams used Git with the SourceTree UI
Namibia: Privately hosted using Bonobo Git Server
Lesotho: Publicly hosted using GitHub:
https://github.com/lso-statistics/Lesotho-census-2016.git
11
Distributed Application Design
Advantages: Revision history
Changes to programs tracked, documented, and reviewed
Sense of ownership of and responsibility over modifications
Ability to simultaneously work on programs Undo ability
Disadvantages: Took some time for teams to adjust to paradigm shift Occasional difficulties merging changes
12
Distributed Application Design
13
Mapping
Used Google Earth to display enumeration maps
GPS points collected during listing overlaid on base maps to determine whether households were in boundary
Not all tablets had data plans so Google Earth maps had to be cached when Wi-Fi was available
14
Mapping
“The PSU boundaries are difficult to distinguish since only a small footpath or a dry creek separates one PSU from another. Having a GPS unit is the only way to tell where one PSU ends and another begins.”
15
Mapping
16
Case Management
In Namibia: Teams consisted of one supervisor and two interviewers,
hired for the duration of the survey Interviews occurred in an area over two weeks Headquarters staff assigned enumeration areas to
supervisors Interviewers listed households and then transmitted the
household listing to the supervisor The supervisor merged the listing files and then a 12
household sample was selected by algorithm Supervisors made household assignments, which were
assigned to a single interviewer, not split
17
Case Management
Assignments could be reassigned, and any data previously collected was maintained
Data collection for a household was split between interviewers and supervisors The household roster was transmitted from the interviewer to the
supervisor to ensure the data quality of the supervisor’s data collection
Interviewers had to validate all household data before transmitting it to a supervisor
Supervisors had to validate all enumeration area data before transmitting it to the headquarters server
If data did not pass validation tests, staff had to explain why it was submitted incomplete or with consistency problems
18
Case Management
19
Case Management
Case management and data flow was tightly controlled, but the system allowed for some flexibility
Headquarters staff could provide codes to unlock certain actions Working in an area meant for a
different supervisor Resampling an enumeration area Submitting incomplete data for an
enumeration area
20
Automated Sampling
Systematic sampling routine built into the program
Reduced errors of supervisors not properly following the sampling algorithm or introducing bias in household selection
Ensured that replacement households came from same stratum as replaced household
21
Monitoring Data Quality
In Namibia:
The programs had many consistency checks and supervisors were not required to review collected data
Regional supervisors could look at data on a case-by-case basis after the data had been transmitted to headquarters
22
Monitoring Data Quality
In Lesotho:
Looser consistency checks in the programs
Supervisors reinterviewed roughly five households per enumeration area
Reinterview program compared the supervisor-entered data with the data originally entered
23
Monitoring Data Quality
In Lesotho: Supervisors viewed reports
showing the status of assignments in an enumeration area
Supervisors could also view basic demographic reports
Reports generated using D3.js and displayed on the tablet’s web browser
24
Data Synchronization
In Namibia: Supervisors had SIM cards and controlled all
transmissions to headquarters MD5 hashes stored so only modified data was transferred
from headquarters
Only newly collected data sent to headquarters
Interviewers did not have SIM cards and updated their programs and files via the supervisor
Transmissions between supervisors and interviewers done via a locally created Wi-Fi hotspot and later Bluetooth
25
Data Synchronization
26
Data Synchronization
Lesotho was similar to the Namibia model
Enumerators sent data via Bluetooth to constituency supervisors
Supervisors sent data via FTP to headquarters
Files were updated from headquarters to supervisors to enumerators
27
IT Support
In Namibia:
An IT coordinator was in charge of two provinces, or 4-6 teams
Headquarters staff set up the 100+ tablets in a single day
28
IT Support
In Lesotho:
50 IT coordinators distributed throughout the country, in charge of around 6,000 tablets
IT staff took two weeks to set up all of the tablets
Pondered the use of a MDM for deployment in the future
Enumerators could use WhatsApp to ask IT staff questions
WhatsApp also used to send announcements to staff
29
General Comments
Interviewers generally liked using tablets to conduct a CAPI survey Data quality was improved due to consistency checks in the programs
In-field coding using lookups eliminated the need for a time consuming coding process at headquarters
Lesotho used three different brands of tablets, which led to some challenges, including subpar GPS reading on cheaper tablets; Namibia, using only one brand, had an easier time managing the equipment
CAPI worked well for Namibia’s rolling survey, allowing easy updating of the programs and the ability to add additional questions in later survey rounds
The programming skills required were greater than for key-from-paper surveys, and required enhanced IT knowledge
CAPI development using CSPro will become easier when there are census/survey templates that organizations can utilize
30
Thank You
31
Thank You
32
Recommended