Data Quality CONCEPTS_ 20th Jan 2015

Accurately Measuring Progress

Khulisa Management Services (Pty) Ltd

DATA QUALITY CONCEPTS & PRINCIPLES

Khulisa Management Services

1st Feb 2015


WHAT IS DATA?

Data

• result of measurements

• building block for information and knowledge

• refers to a collection of numbers / other outputs produced from activities

• can be qualitative or quantitative

DATA INFORMATION KNOWLEDGE


DATA is deemed to be of Good Quality if it accurately represents the situation to which it relates

GOOD QUALITY DATA


DATA QUALITY

Data Quality

• Refers to the worth or accuracy (Value) of the information collected

• Focuses on ensuring that data management processes are of a high standard

• Data reflects TRUE performance

• Data is: Appropriate

Organised

Timely and Available

Accurate and Complete

Cost Effective


DATA QUALITY - VCRIPT

1. VALIDITY / ACCURACY

• Data is considered correct• Data measures what it intended to measure

• Minimizes errors: during, recording, transcribing etc.

5.PRECISION

• Data has sufficient detail• Error measurements: source bias / error; instrumentation error; sampling error; transcription error and manipulation error

4.INTEGRITY

• Data generated is protected from deliberate bias or manipulations (personal or political)

• Truthfulness of data

2. COMPLETENESS

• Information system providing the results is appropriately inclusive

• Represents the complete list of eligible persons or units and not just part thereof

3. RELIABILITY

• Data generated is based on protocols and procedures that do not change according to who is using it

• Data is measured and collected consistently

• CONSISTENCY + INTERNAL QUALITY CONTROL + TRANSPARENCY

6.TIMELINESS

• Data that is up-to-date (current) and when info is available on time

• Timeliness is affected by rate of updating; rate of activity changes and when info is used or required

• FREQUENCY AND CURRENCY


DATA QUALITY DIMENSIONS CRITERIA DEFINING CHARACTERISTICS

ValidityPrecise definitions for data to be collectedValidation of data sourcesDesign of data tools

Completeness

Completeness of received filesNo missing informationNo blank fieldsNeed procedures for addressing incomplete and missing data

Reliability

Consistency of tools and processes including TechnologyDocumentation of DMS and Standard Operating ProceduresDocumentation of Data Quality PlanImplementation of quality control checksCommunication of data problemsHR recruitment and training

Professionalism – objectivity in analyses policies and practicesTransparency – documentation and QC stepsEthical standards – avoids incentives, confidentiality of data



Quality• Quality in ISO 9000:2000 is defined as

the degree to which a set of inherent characteristics fulfils the requirements.

• With this definition the implication is that quality is relative to what something should be (requirements) and what it actually is. (To all stakeholders and not only to some.)


DATA MANAGEMENT SYSTEM

SOURCE

COLLECTION

COLLATION

ANALYSIS

REPORTING

USE

Definition•Where the data comes from

Examples• Registers or filesDefinition

•Process of gathering data e.g. Excel Files or Activity Forms etc Definition

• Aggregation of data into summarized formats

Examples• Excel Spreadsheets’ formulae

Definition•Review and analyse data

• Examples• Trend Analyses• Issues Analyses

Definition•Descriptive information, •presenting data

Definition• Evidence based decisions


BMGF - INDICATORS


Poorly Defined Indicators

• Undermine the validity of data such that

(i) incorrect data may be collected or

(ii) the data collection process is not correct for the data that is meant to be gathered.

(iii) each person interprets the indicator in his/her own way and goes about their activities according to his/her individual understanding.


INDICATOR USEFULNESS - CRITERIA

CRITERIA DEFINING CHARACTERISTICS

RelevanceIndicators are in line with Project/Programme activities, objectives, goals and visionDetermines what data is produced and required resources

InterpretabilityProvides sufficient information to allow users to properly interpret statistical information e.g. the concepts, variables, classifications and methodology

AccessibilityReflects how readily the indicator data can be located and accessedLabeling, filing and dissemination process are adequate

Coherence

The coherence of indicators reflects the degree to which it can be successfully brought together with other data within a broad analytical framework and over timeThe use of standard concepts, classifications and target populations promotes coherence, as does the use of common methodology across stakeholders


Common Risks to Data Quality: Manageable Risks

• Manageable risks = risks whose mitigation is within an organisation’s control.

• Common Manageable Risks : Absence of (or incomplete) SOPs Absence of (or poor) clear, operational definitions of

indicators Manual aggregation of data (using calculators & MS Word) Poorly designed data collection tools No backup of electronic data Absence of clearly defined margins of error for data quality Failure to keep an error log Absence of (or inconsistent) data quality checks Absence of (or incomplete) documentation of the DMS Absence of (or incomplete/inconsistent) Version Control

Mechanisms Data Management Staff shortages or high staff turnover Lack of (or poor) training in Data Quality/management)


Common Risks to Data Quality: Acknowledged Risks

• Acknowledged risks are those risks that are beyond an Organisation’s control.

• Examples Data collection & reporting for indicators

outside an organisation’s normal operations. Delays in external data collection Lack of data quality control for external data High turnover of staff


Data Collection• Designing data collection tools as simply and clearly as possible, for example Using tick boxes to minimise the need to write

Colour coding forms Dates /Time Period for the data being recorded on the tool

Including version numbers and date of release on tools


Data Collection• Limiting data collection manipulation. • Including signoff fields (and dates) for all

handlers of the data collection tools (e.g. data collectors, verifiers/supervisors, and “receivers” of the tool)

• Operationalising indicator definitions to improve Validity of data collected

• Using the same data collection tools across all sites implementation. Where different tools are used e.g. at different

types of facilities - inter-tool reliability should be established.


Data Collation:• Developing simple, clear, easy-to-understand, and easy-to-use

collation tools include version number and dates of release for tools include quality-check sign-off fields, and fields for dates of collation Use electronic collation tools, e.g MS-Excel, MS-Access – to the maximum

extent possible -- Minimise manual collation processes• Documenting all collation processes and procedures, through SOPs,

including Write detailed instructions on how to complete the tool /collation process how to handle missing data timeframes required for collation.

• Train data handlers in basic computer literacy so they can comfortably use applications like Excel.

• Incorporate cross-checks (verifications) by a second individual of all collations

• Using the same collation processes over time.• Keeping records of all collation processes including any errors found. • Data to be securely stored either under lock and key or password

protected


Data Analysis: Good Practices

• Applying appropriate analysis methods, including correct arithmetic formulae.

• Securing expert advice on the most applicable analysis methodology to use,

• Using current data. • Disclosing all conditions and assumptions affecting

interpretations of data. • Training staff in analysis methodologies. • Recording dates when analyses are carried out • Second person cross checking all analysis before data are

used or submitted • Maintaining an audit trail of analysis procedures

conducted – including documentation on analysis procedures and outputs.

• Adequate storage of records - both soft and hard - of all analyses procedures and outputs.


Data Reporting: Good Practices • Checking all data before final reporting• Copying other people with an programme when submitting final

reports • Reporting /Feedback to all levels within a programme. • Developing and adhering to reporting formats.• Developing and adhering to strict schedules for completing and

sharing reports at each level of the DMS. Include time for informing sites/offices of incomplete /missing

data Sending regular reminders to sites/offices for timely submission

of data required for reports.• Keeping copies of all data used to prepare reports.• Recording dates when reports are submitted • Informing report recipients of data limitations (e.g. missing data,

weaknesses)• Reporting format aligned with type of data received

i.e. if qualitative data received then the report mostly qualitative.



Khulisa Management Services (Pty) Ltd

Khulisa Management ServicesPhone: +27 (0)11 447-6464Email: [email protected]: [email protected]

mailto:[email protected]

mailto:[email protected]

Documents

Data Quality CONCEPTS_ 20th Jan 2015