Upload
theodore-hunter
View
220
Download
4
Tags:
Embed Size (px)
Citation preview
Quality Control
Data Processing OperationsScanning data capture and quality assurance
Quality in the Data Process
Geoffrey Greenwell, Data Processing AdvisorIPC
P-R-O (TQM)
P ersonal Commitment to Excellence R eject Cynicism or Satisfaction O wn the process
Conceptual Overview
C-E-S-S (Six Sigma, Lean Manufacturing)
C areful Design E valuate Continually S hift Perspectives S helf Life
Conceptual OverviewForm Flow
InformationExformation
Data EntryManualScanning
EditsStructurePre-editConsistency
Tabulation controls
Data Archive
Form flow
•Flow Charting is a fundamental tool for careful design.•Flow charting is the mapping of the process.
•A flow chart is defined as a pictorial representation describing a process being studied or even used to plan stages of a project. Flow charts tend to provide people with a common language or reference point when dealing with a project or process. deming.eng.clemson.edu
•EXAMPLE
•Process Flow Chart- Finding the best way home•This is a simple case of processes and decisions in finding the best route home at the end of the working day.
Primary Symbols
Form FlowSoftware for flowcharting:
•ABC Flowcharter•Visio•Corel Flow•(Microsoft word has basic flow chart symbols)
Form Flow
External information flowExformationAll the processes involved in managing questionnaires andproducing progress reports.
Internal information flowInformationAll the processes involvedin managing the electronic processes and producing FEEDBACK.
Paper Flow Data flow
Ex-Form FlowLook at the “nodes”. In this case, the exchange of forms is a “pressure drop.”
1
1. Field office to data entry center.2. Center to Date entry group supervisor.3. Supervisor to data entry person4. Problems/trouble shooting.5. Storage6. Retrieval
2A
2B
2C
4
3
5
6
Ex-Form Flow
Every node is a potential loss of control in the process.
Ways to control:
•Operational control forms (electronic systems)•Progress reports•Assign responsible persons
Ex-Form Monitoring
Control Forms: Input and Output
Define a fundamental unit based on geographic criteria.
Census: Enumeration area=a box of forms=an electronic batchSurvey: A region=a time cycle=an electronic batch
• All forms are traced and verified to a master control form.• The flow of the form is followed through the phases of the data process.• The process has an audit trail all the way to the individual data entry station.
Ex-Form FlowA division of labor into logical, efficient and affordable processes designed to optimize the efficiency of the production line.
Labor Intensive Solutions
P=f(K,L) Classic production function.Capital and labor inputs
SpaceTrained personnel (division of labor)Low tech supplies: boxes, paper, labels
In-form flow
Network serversData entry stationsCablingPrinters
An electronic management system for:A. Processing the primary survey or census instrument.B. Provide a tool for managing the ex-formation.C. Provide feedback in the form of reports (end of information).
Capital Intensive Solutions
In-form flow
Form Flow (In or Ex?)•Transferring the box from the project vehicle to the forms depot.
•Designing a flow chart for a data entry program.
•Assigning a form to a data entry operator
•Scanning or keying in a form
•Two EA boxes are not closed well and the forms scatter during transport.•Backing up data on the central server.
•Supply clerk checking the inventory of paper; folders; labels and filling out a form to re-stock needed items.
Manual Data Entry
• Cost Consideration
A. Training for all Clerks
B. Monitoring systems and supervisory time.
C. Verification of Clerk’s Work
D. Keying costs
Manual Data EntryDesign Considerations:
• Data File Structure Levels Multiple items vs. records
•Data Dictionary (Critical through all stages!)Good and consistent variable labelsLogical and efficient variable namesWell defined value sets (several value sets)
•Screen FormatsFont Size and typeSoft backgroundsField and text positioningField SizePhysical form emulation
• PathLogical PathItems vs. fieldsLevels
Manual Data Entry
• Skips To skip or not to skipCensus vs. Surveys
• On-line edits/Error Messages and Warnings
Census vs. Survey
CAPI (Computer Assisted Personal Interviewing)
Manual Data Entry
• Controls the process of carrying out the interview by the enumerator.• Removes the ex-form process by directly keying in responses into a portable computer.
CATI (Computer Assisted Telephone Interviewing)
• Controlling the data entry personnelEstablish objective measures (extract information from Log Files)
Manual Data Entry
Speed and accuracy (8000 keystrokes/hour)
• Heads up vs. Head down keying
• Census vs. Survey Constraints
Censuses: High Volumes (Time primary quality constraint)
Surveys: Low Volume (Place is the primary constraint)
Manual Data EntryVerification procedures:Verification is a duplication of the data entry process in order to compare two identical records for inconsistencies
Dependent vs. independent
Dependent verification duplicates the data entry process and compares the data files “on-line” and corrects the files when an error is encountered.
Independent verification is a complete re-entry of a form followed by a full comparison of the two data files.
Verification is dynamic and is adjusted to the learning curve. 100% at the outset and may drop to 2-3% at the end.
ScanningRefer to: A Comparison of Data Capture Methods by Sauer
Machine quality is very important.• Glass optics and color corrected instead of plastic optics
(light sensors and diffusion)• Resolution (DPI) 300• Bit depth: higher bit the better ability to interpret grayscales• Scanning speed• Optimal environment: 60-85 F and 40-60% relative hum.
Scanning
Rotary Scanners move the form. Best for censuses and surveys.Other Options to consider:•Automatic document feeder•Multi feed detector•Exit hoppers•Color bulbs•Image processing
ScanningCharacter reading: OCR, ICR and OMR
• OMR-Optical Mark Reading. Reads a mark from a questionnaire.
• OCR-Optical Character Recognition. Converts characters through photosensitive sensors and software enhancements.
• ICR-Intelligent Character Reading. ICR is pattern based character recognition and is also known as Hand-Print Recognition. (Software differences). Remembers patterns.
Note: OCR and ICR usually require “constrained handwriting” orBLOCK capital letters.
Scanning
Advantages and Disadvantages
Speed of process Loss of process controlTechnological Innovation Minimal technological transferMinimize human error Maximize machine error
Cost
World Bank CWIQ (Core Welfare Indicators Questionnaire)
ScanningQuality Issues
•Pencil type, paper jams, damaged forms•Accurate character recognition dependent on form quality and image•Field level accuracy vs. character level accuracy•Machine Maintenance•Software deficiencies (Voting)•KFI (Key from image) for character correction•KFP (Key from paper) for form and field correction•Confidence level reports (recognition rates)•See page 18, Sauer for Quality Control issues
Coding
Coding of open ended questions like: Occupation and activityrequire coding.
In Scanning this can be done from the image. In Manual entry it is usually the first step.
It requires its own process and supervision.In Scanning it can be seen as a parallel operation.In Manual Entry it is linear.
Now that you have: A well designed dictionary with the simplest and most efficient structure and well defined variable labels with easy to use variable names and well defined value sets clearly designed, simple and user friendly screens emulating the forms and flowing logically with programmed skips and have clear interactive messages should you use them and have defined productivity targets for your census or survey and a system to objectively measure productivity and reward accordingly and verified the work and finally rigorously subjected it against the PROCESS rule of quality…
you still need to check for errors.
Edit FlowI. Verified
Files
II. Structure
Consolidate
II. Pre-Consolidate
III. Consistency
IV. Pre-Tabulation
Data Processor Statistician Data Processor Analyst
Structure Edit Controls:• Performed after verification
• Control totals used to check completeness
– File totals compared with manual counts
• Corrections done with questionnaires
• Limit checks to rendering questionnaires clear enough
for computer processing (processability) only.
Pre-edit Consolidation Controls:
• Follow pre-established geographical priorities
• Check control totals
• Use operational control data base
• Avoid geographic coding (geocode) conflicts in
joining files
Consistency Edit Controls:• Develop consistency specifications
• Prioritize variables
• Monitor corrections
• Use control tabulations
• Re-run output file
Pre-tabulation Consolidation Controls:
• Consolidate to facilitate tabulation
• Check control totals for each record type
• Use standardized forms for operational control
• Avoid geocode conflicts in joining files
Tabulation Controls:• Computer Program Specifications
• User Approval of Tables
• Tables Grouped by Characteristics
• Tables Checked Against Control Totals
• Control Tables Show Weighted and Un-weighted Numbers
• Geographic Subtotals Match
• Standardized Control Forms for Production Control
• Final Table Review
• Data Dissemination to User Community
Data ArchivingData and Metadata
metadata are often called "codebooks“Metadata is the data which defines the data
http://www.icpsr.umich.edu/DDI/ORG/index.html
DDI-Data Documentation Initiative
The definition of an international standard to defineMetadata in the social sciences.
Data Archiving
The DDI defines a hierarchy of information related to a Census or survey.
The DDI defines these by using XML (Extensible Mark up language)
XML defines document tags much the same way as HTML.
Example of Codebook Structure:
http://www.icpsr.umich.edu/DDI/CODEBOOK/codedtd.html
Data Archiving
All processes need to be documented. This includes:CodebooksData entry manualsProgram documentationEdit programsImputation rulesTracking/process manuals
Final ConceptsLean Manufacturing: A quality control system for monitoringprocess flow.
Six Sigma: A statistical system developed by Motorola to establish process problems and error tolerances and methods to correct.
Final ConceptsLean Manufacturing:
•Workplace organization•Standardizing work/work stations.•Division of labor to increase process flow.•JIT-Just-in-Time delivery•Pull systems
Six Sigma:•Means six deviation tolerance for error: 3.4 error events out of one-million•Measurement (quantifiable) system for process improvement.
P-R-O (TQM)
Personal Commitment to Excellence Reject Cynicism or Satisfaction Own the process
Conceptual Summary
C-E-S-S (Six Sigma, Lean Manufacturing)
Careful Design (Flow Charts) Evaluate Continually (Six Sigma) Shift Perspectives (Lean Manufacturing)Shelf Life (Data Archive)
From John Henry
The man that invented the steam drillThought he was mighty fineBut John Henry made fifteen feetThe steam drill only nine, Lord, LordThe steam drill made only nine.
John Henry hammered in the mountainHis hammer was striking fireBut he worked so hard, he broke his poor heartHe laid down his hammer and he died, Lord, Lord.He laid down his hammer and he died