Using Information for Health Management; Part I- Health Information Systems Strengthening

2

Learning objectives

– the information cycle; tools and processes for turning data into action

– the relationship between data use and data quality

– hierarchy of standards / essential data set– common reasons for compromised data quality,

and various counter measures– different information products for

communicating different meanings

• Reflecting on the data that you have been working with …. What do you think are the steps that have been taken to get the data into the Kenya HMIS?

3

Client record cardsCollection

Processing

Presentation

Action

Tally sheetsEasy way of counting identical events that do not

have to be followed-up (e.g. headcounts, children weighed)

Collection

Processing

Presentation

Action

6

RegistersRecord data that need follow-up over long periods such as ANC,

immunisation, Family Planning, Tuberculosis (TB)

Collection

Processing

Presentation

Action

Collection

Processing

Presentation

Action

Key issues

Registers for CoC

Tally sheets

Tick registers

Reports

weekly,monthly,quarterly

Collection

Processing

Presentation

Action

• And now that the data is there – what now?

10

Information cycle; from data to action

11

Collection

Processing

Presentation

Action

12

Collection

Processing

Presentation

Action

• Data set based on minimum indicator set• Standard definitions• Data sources & tools

• Data quality checks • Data analysis: indicators

• Tables• Graphs• Reports

• Interpret information: comparisons trends• Decisions based on information• Actions

• Why do you think we need this information?

13

Planning cycle INDICATORS

Linking Planning with Information

Information cycle

Collection

Processing

Presentation

Action

Data Collection and Collation in a Health Facility (Zambia HMIS Procedure Manual)

15

Data collection – at the source of data creation (point of care)

Service data collected by nurses and doctors in-between attending to patients

Usually several (manual) steps before it is in any database/storage– Tally sheets– Tally sheet totals at end of month– Monthly summary forms which are

reported to the next levelOften too much to collect for already

overworked staff16

Collection

Processing

Presentation

Action

What data elements should be collected?

Cannot be obtained elsewhere (e.g. survey) Are easy to collect (cost vs usefulness) Do not require much additional work or time Can be collected relatively accurately Is part of one or more indicators

Collection

Processing

Presentation

Action

Essential data sets (EDS)Collection

Processing

Presentation

Action

Hierarchy of standards

Example of a National Data Dictionary

• ZA National Data Dictionary

19

Key issues

Top-down vs bottom-up approaches

Who to involve in discussions

Maximalist vs minimalist approaches

EDS Example: vaccination data

Input (community and facility levels)– Staff attendence, vaccines, to whom, when, where

Process (district)– # Children Vaccinated

Output (province)– Coverage of child immunization

Outcome (national)– Decreased incidence of vaccine preventable diseases

Impact (international)– Decreased mortality, healthier children

Collection

Processing

Presentation

Action

Prioritising data in the EDS:

Finagle’s Law:

The information you have is not what you want;The information you want is not what you need;The information you need is what you can get;The information you can get costs more than

you want to pay!

22Balancing varying information needs

Collection

Processing

Presentation

Action

Comparability of collected data

Stable standardised definitions– To ensure spatial comparability between different facilities,

districts, provinces and nations– To ensure comparability over time

What do you think about this statement:“Revising poor indicators /data sets /data elements may not be advisable due to cost and loss of backward comparability”

Collection

Processing

Presentation

Action

Characteristics of the aggregated anonimised DHIS data

24

Org.Unit Immunization Coverage May 2012

Whatever 70

Notsogood 40

Verybad 15

Superduper 98

Spatial /organizational dimension

Org.Unit A May 2012

Data Element 1 70

Data Element 2 65

Data Element 3 62

Data Element 4 98

Phenomenological dimension

OrgUnit AMonth 2012 Immunization Coverage

Jan 83Feb 80Mar 70Apr 52May 64Jun 60Jul 54

Aug 43Sep 37Oct 39

Temporal dimension

Where do we get data from?Routine data collection

– Routine health unit and community data• Activity data about patients seen and programmes run,

routine services and epidemiological surveillance• Semi-permanent data about the population served, the facility

itself and staff that run it– Civil registration (vital events being integrated with health e.g.

India)Non-routine data collection

– Surveys– Population census (headcounts proportion/facility catchment’s

area)– Quantitative or qualitative rapid assessment methods

Collection

Processing

Presentation

Action

Data Sources in the HMN data warehouse concept

Collection

Processing

Presentation

Action

In Summary: Data collection

32

Collection

Processing

Presentation

Action

Input: Using data sources and tools to collect quality data

Common problems:- Too much to collect- Poor understanding of data

collection tools- Timeliness of reporting- Low data quality

Output: relevant data

Data Processing:• What

observations can you make about your experience in processing the data so far?

33

Processing; assuring data quality and calculate indicators

- Turning data into information- How to assess data quality?- What are indicators, and why do we need

them?

37

Collection

Processing

Presentation

Action

Why checking data is vital?• Use of inaccurate data leads to

– Wrong priorities (focus on the wrong data)– Wrong decisions (not applying the right actions)– Garbage in = garbage out

• Producing data is expensive– Waste of resources to collect poor data

Collection

Processing

Presentation

Action

Routine data should be..

Reliable: Correct, Complete, Consistent

Timely: fixed deadlines for reporting

Actionable: no action = throw data away

Comparable:same numerator and denominator definitions used by all data processers BUT striving for comparability can compromise local relevance

Collection

Processing

Presentation

Action

Complete data?

Spatial: submission by all (most) reporting facilities Timely: is the data available within the required time Temporal: can you do analysis over time?

Collection

Processing

Presentation

Action

41

Timely data?• Late reports weaken the potential for comparison,

action can be too late,but still useful for documenting trends;

• Better to use the data that you have even if incomplete: “Perfection is the enemy of good”

Collection

Processing

Presentation

Action

Correct data? Are we collecting the data

we need? The data values seems

sensible/plausible? The same definition

applied uniformly? Are there any preferential

end digits used?

JAN FEB MARCH APRIL MAY JUNE JULY

250 230 245 225 230 240 250

Collection

Processing

Presentation

Action

Consistent data?

Data in the similar range as this time last year or similar to other organization units

No large gaps or missing data

No multiplicity of data (same data from multiple sources –which one to trust?)

Collection

Processing

Presentation

Action

What are the causes of poor data quality?

- Too many forms to fill out that are not useful to health workers- Absent data collection tools (Nigeria)- Data collection tools are poorly designed and hard to understand- Too many steps of manual aggregation and transfer of figures (next slide)- Limited feedback on data quality to those who collect it- Data is not used

45

Collection

Processing

Presentation

Action

46

Mate KS, Bennett B, Mphatswe W, Barker P, Rollins N (2009) Challenges for Routine Health System Data Management in a Large Public Programme to Prevent Mother-to-Child HIV Transmission in South Africa. PLoS ONE 4(5): e5483. doi:10.1371/journal.pone.

ZA_Auditor General Findings on DQ:DQ Improvement Assessment

Doctor or nurse interacts with patient

Patient record

Data transcribed to

Sub-set of data recorded in register and/or tally sheet

Data capture in DHIS

Step 1

Step 2 Manual recording

Data quality affected by

Monthly summaries collated

Step 5

Monthly summary report compiledStep 3

Step 4

Data analysis and feedbackStep 6

Incomplete, illegible, undated data

Multiplicity of DCT’s, duplicated,

non-standardised

Inability to collate data accurately

Inability to collate data accurately

Data capture errorsIncorrect data elements

activatedValidation not done

No feedbackLittle data analysis by

program managers

Doctor or nurse interacts with

patient

Patient record

Sub-set of data recorded in register and/or tally sheet

Data capture in DHIS

Step 1

Step 2

Strategies to improve DQ

Monthly summaries

collated

Step 5

Monthly summary report

compiledStep 3

Step 4

Data analysis and feedbackStep 6

Incomplete, illegible, undated data

Multiplicity of DCT’s, duplicated,

non-standardised

Inability to collate data accurately

Inability to collate data accurately

Data capture errorsIncorrect data elements

activatedValidation not done

No feedbackLittle data analysis by

program managers

Training and skills development

Financial

Technology

In-service training and

formal courses

Supervision

Supervision

1) Use of DHIS daily data capture, eTools

2) Electronic sign-off of data

3) Facility level capture of ART & TB data

Supervision

Supervision

Supervision

1)Improve printing of DCT

2) Hardware & software at facilities

3) HIS Staffing

Formal courses:Data validation,

feedback, check it, etcSupervision

Data capture formsCorrect data

element activation

In-service training and

formal courses

In-service training and

formal courses

In-service training and

formal courses

Auto-reports as “push” feedback

Doctor or nurse interacts with patient

Patient record

Data transcribed to

Sub-set of data recorded in register and/or tally sheet

Data capture in DHIS

Step 1

Step 2Manual recording

eTool Scenarios: Excel Aggregation

Monthly summaries collated

Step 5

Monthly summary report compiledStep 3

Step 4

Data analysis and feedbackStep 6

Excel Aggregation Tool in facilities

Electronic data transfer

Easy to install and scale across facilities with computers on

site

Doctor or nurse interacts with patient

Patient record

Data transcribed to

Sub-set of data recorded in register and/or tally sheet

Data available in DHIS

Step 1

Step 2 Manual recording

eTool Scenarios: DDC in DHIS14

Monthly summaries collated

Step 5

Monthly summary report compiledStep 3

Step 4

Data analysis and feedbackStep 6

Daily data capture on DHIS14 in facilities

Electronic data transfer

• Already available for Midnight Census in hospitals

• Requires DHIS in facilities – useful for some larger PHC facilities

Doctor or nurse interacts with patient

Patient record

Data transcribed to

Sub-set of data recorded in register and/or tally sheet

Data available in DHIS

Step 1

Step 2 Manual recording

eTool Scenarios: DDC in DHIS2

Monthly summaries collated

Step 5

Monthly summary report compiledStep 3

Step 4

Data analysis and feedbackStep 6

Daily data capture on DHIS2 on central server

Electronic data transfer • Revolutionises the

availability of data and feedback processes;

• Aligns NIDS with DC tools immediately

• Use of tablets could replace paper registers

Doctor or nurse interacts with patient

Electronic Patient record

Data transcribed to

Sub-set of data recorded in register and/or tally sheet

Data capture in DHIS

Step 1

Step 2

Electronic data transfer

eTool Scenarios: EPR systems

Monthly summaries collated

Step 5

Monthly summary report compiledStep 3

Step 4

Data analysis and feedbackStep 6

• Potential expansion of the EPR systems to accommodate all kinds of chronic illnesses

What can be done to improve data quality?

1. Assess the cause by using theInformation Cycle as the basis

2. Programmatic Issues– Essential dataset– Feedback routines– Use of Information

3. Database validation mechanisms– Min/Max rules in software– Data validation rules, check for consistency in logic of data– Completeness and timeliness reports

56

Minimum and Maximum Values

0

500

1000

1500

2000

2500

3000

Jan Feb March April May June July

Num

ber

Minimum and Maximum Values

Maximum

Minimum

Collection

Processing

Presentation

Action

Indicators

Calculated by combining two or more pieces of data, so that– They can measure trends over time– They can provide a yardstick whereby facilities /

teams can compare themselves to others (spatial, organizational)

– monitor progress towards defined targets– Good for measuring change

To do this, indicators need to have a numerator and denominator

57

- measure service COVERAGE and QUALITY

Collection

Processing

Presentation

Action

58

Indicator typesCollection

Processing

Presentation

Action

Type Description ExampleCount Indicator

Number of events without denominator

Number of new HIV+ cases

Proportion Indicator

Numerator is contained in denominator

Immunisation coverage of children under 1 year of age

Ratio Indicator

Numerator is not contained in denominator

Number of maternal deaths per 100,000 live births in same time period

Rate Indicator

Frequency of the event in a specified time in a given population

Number of maternal deaths per 1,000 women of reproductive age in the population

– Maternal mortality ratio?“the number of maternal deaths per 100,000 live births in same time

period.”Numerator: Number of deaths assigned to pregnancy-

related causes during a given time intervalDenominator: Number of live births during the same time intervalMultiplier: 100,000

Millenium Development Goals have a set of proposed indicators [weblink]

denominatorindicator = numeratorX 100 = %

Collection

Processing

Presentation

Action

60

Atop the line – numerators(activities / interventions / events / observations / people)

a count of the event being measured

How many occurrences are there:

morbidity (health problem, disease)

mortality (death)

resources (manpower, funds, materials)

Generally raw data (numbers)

Collection

Processing

Presentation

Action

61

Under the line – denominators (population at risk)

size of target population at risk of the event

What group do they belong to:

- general population (total, catchment, target)

- gender population (male / female)

- age group population (<1, >18, 15-44)

- cases / events – per (live births, TB case)

Collection

Processing

Presentation

Action

An ideal indicator RAVES !!!

62

Collection

Processing

Presentation

Action

63

Indicators should beRELIABLE gives the same result if used by

different people

APPROPRIATE fits with context, capacity, culture and the required decisions

VALID truly measures what you want to measure

EASY feasible to collect the data

SENSITIVE immediately reflects changes in events being measured

Collection

Processing

Presentation

Action

Indicator OperationalizationDefining the sources of the data – both numerator & denominator (how is it to be collected?)

Determining the frequency of collection and processing of the indicator (How often should it be collected, reported, analyzed?)

Determining appropriate levels of aggregation(To where should it be reported and analyzed?)

Setting levels of thresholds and target

What will be the nature of the action (decision) once the indicator reaches the threshold?

Collection

Processing

Presentation

Action

Processing

65

Collection

Processing

Presentation

Action

Input: Relevant data

Processes:- Quality checks- Aggregation to relevant levels- Calculation of indicators- Analysis of data =>

information

Common problems:- Much irrelevant data- Low data quality- Limited knowledge of data

needs and analysis

Output: data converted to information