Using Information for Health Management; Part I- Health Information Systems Strengthening

2

3

Learning objectives

– the information cycle; tools and processes for turning data into action

– the relationship between data use and data quality

– hierarchy of standards / essential data set– common reasons for compromised data quality,

and various counter measures– different information products for

communicating different meanings

Data, information, knowledge

Data – observations and measurements

Information (processed data)– facts extracted from a set of data (interpreted data),

– data brought together to demonstrate facts– Meaningful and useful

Knowledge– contextualized information

– actionable

42

42 / 60 = 70%Fully Immunized Children < 1y

Where 60 is the target population

42 / 60 = 70%Fully Immunized Children < 1yFor month May 2010Organization Unit: Whatever

With additional contextual insight we may act upon this information

Org.Unit Immunization Coverage May 2012

Whatever 70

Notsogood 40

Verybad 15

Superduper 98

Spatial /organizational context

Month 2012 Immunization Coverage for WhateverJan 83Feb 80Mar 70Apr 52May 64Jun 60Jul 54

Aug 43Sep 37Oct 39

Temporal context

Information cycle; from data to action

10

Data collection

11

Input: Using data sources and tools to collect quality data

Common problems:- Too much to collect- Poor understanding of data collection tools- Timeliness of reporting- Low data quality

Output: relevant data

Processing

12

Input: Relevant data

Processes:-Quality checks-Aggregation to relevant levels-Calculation of indicators- Analysis of data => information

Common problems:- Much irrelevant data- Low data quality- Limited knowledge of data needs and analysis

Output: data converted to information

Presentation

13

Input: Analysed information, to be presented to communicate meaning

Common problems:- Importance of information is lost through poor presentation- Data can be misrepresented or misinterpreted

Output: various information products- Charts- Tables- Maps

Action

14

Input: Information products

Process: Interpretation of information, prioritizing action, making action plan, and execute action through budgets and plans with goals and targets

Common problems:-Incomplete information-Lack of good targets-No plan for evaluation

Output: Decision-making for health management

Data collection – at the source of data creation (point of care)

Service data collected by nurses and doctors in-between attending to patients

Usually several (manual) steps before it is in any database/storage– Tally sheets– Tally sheet totals at end of month– Monthly summary forms which are

reported to the next level

Often too much to collect for already overworked staff

15

What data elements should be collected?

Cannot be obtained elsewhere (e.g. survey) Are easy to collect (cost vs usefulness) Do not require much additional work or time Can be collected relatively accurately Is part of one or more indicators

Comparability of collected data

Stable standardised definitions– To ensure spatial comparability between different facilities,

districts, provinces and nations– To ensure comparability over time

Note:Revising poor indicators /data sets /data elements may not be advisable due to cost and loss of backward comparability

18Balancing varying information needs

Essential data sets (EDS)

Hierarchy of standards

EDS Example: vaccination data

Input (community)– Staff attendence, vaccines, to whom, when, where

Process (district)– Vaccination of children

Output (province)– Coverage of child immunization

Outcome (national)– Immunization rates going up

Impact (international)– Healthier children, less disease. Synergies

Where do we get data from?Routine data collection

– Routine health unit and community data• Activity data about patients seen and programmes run,

routine services and epidemiological surveillance• Semi-permanent data about the population served, the facility

itself and staff that run it– Civil registration (vital events being integrated with health e.g.

India)Non-routine data collection

– Surveys– Population census (headcounts proportion/facility catchment’s

area)– Quantitative or qualitative rapid assessment methods

Data Sources in the HMN data warehouse concept

Client record cards

Tally sheetsEasy way of counting identical events that do not

have to be followed-up (e.g. headcounts, children weighed)

25

RegistersRecord data that need follow-up over long periods such as ANC,

immunisation, Family Planning, Tuberculosis (TB)

Reports

weekly,monthly,quarterly

Processing; assuring data quality and calculate indicators

- Turning data into information- How to assess data quality?- What are indicators, and why do we need

them?

28

Why checking data is vital?

• Use of inaccurate data leads to– Wrong priorities (focus on the wrong data)– Wrong decisions (not applying the right actions)– Garbage in = garbage out

• Producing data is expensive– Waste of resources to collect poor data

Routine data should be..

Reliable: Correct, Complete, Consistent

Timely: fixed deadlines for reporting

Actionable: no action = throw data away

Comparable:same numerator and denominator definitions used by all data processers BUT striving for comparability can compromise local relevance

Complete data?

Spatial: submission by all (most) reporting facilities

Temporal: can you do analysis over time?

Does provided services cover the full population? Many indicators depend on population figures as denominators

Correct data?

Are we collecting the data we need?

The data values seems sensible/plausible?

The same definition applied uniformly?

Are there any preferential end digits used?

JAN FEB MARCH APRIL MAY JUNE JULY

250 230 245 225 230 240 250

Consistent data?

Data in the similar range as this time last year or similar to other organization units

No large gaps or missing data

No multiplicity of data (same data from multiple sources –which one to trust?)

Timely data?

Late reports weaken the potential for comparison, action can be too late,but still useful for documenting trends

What are the causes of poor data quality?

- Too many forms to fill out that are not useful to health workers

- Data collection tools are poorly designed and hard to understand

- Too many steps of manual aggregation and transfer of figures

- Limited feedback on data quality to those who collect it

- Data is not used

35

Accuracy enhancing principles Capacity building through training

(90% of HISP activities)

User-friendly collection/collation tools Feedback on data errors (but not only!) Availability of processed data Local Use of information Essential Data Set Clear definitions; standards Information sharing

CHIEFDOM LEAGUE TABLE 2ND QUARTER APRIL – JUNE 2009

20.893.632.4114.34391.4Total

14143.228.3100.038.273.82954.9Bumpeh

12123.738.677.8101.268.02961.1Upper Banta

12123.760.5100.057.453.72671.8Ribbi

884.364.086.689.492.64049.8Kori

884.336.5100.077.693.24580.4Kargboro

884.386.5100.0140.769.73555.6Kamaje

884.332.193.092.4110.33761.4Bagruwa

664.735.6100.0120.8201.64888.3Lower Banta

664.778.2100.046.796.552118.4Kowa

334.833.091.791.7106.846140.3Timidale

334.871.375.093.4162.75590.3Kaiyamba

334.845.9100.086.390.557134.9Dasse

225.048.1100.086.2154.362124.3Fakunya

115.393.386.696.6170.94598.2Kongbora

RankingRankingAverage Score

% Exclusive Breastfeeding

at Penta3

% MMRC Submitted

% 2nd Dose of IPT

% 3rd ANC Visit

% PHU Delivery2nd

Quarter

% FullImmunized 2nd Quarter

Chiefdoms

What can be done to improve data quality?

Systemic changes

– Essential dataset

– Feedback routines

– Information for action

– Promote information use at all levels

Data validation mechanisms

– Min/Max rules in software

– Data validation rules, check for consistency in logic of data

– Completeness and timeliness reports 38

Minimum and Maximum Values

0

500

1000

1500

2000

2500

3000

Jan Feb March April May June July

Num

ber

Minimum and Maximum Values

Maximum

Minimum

Indicators

Calculated by combining two or more pieces of data, so that– They can measure trends over time– They can provide a yardstick whereby facilities / teams

can compare themselves to others (spatial, organizational)

– monitor progress towards defined targets– Good for measuring change

To do this, indicators need to have a numerator and denominator

39

- measure service COVERAGE and QUALITY

40

Indicator types

– Example: How is maternal mortality rate defined?“the number of maternal deaths per 1,000 women of reproductive age

in the population (generally defined as 15–44 years of age).” – What about maternal mortality ratio?“the number of maternal deaths per 100,000 live births in same time

period.”

Numerator: Number of deaths assigned to pregnancy-related causes during a given time interval

Denominator: Number of live births during the same time interval

Multiplier: 100,000

Millenium Development Goals have a set of proposed indicators [weblink]

denominatorindicator =

numeratorX 100 = %

42

Atop the line – numerators(activities / interventions / events / observations / people)

a count of the event being measured

How many occurrences are there:

morbidity (health problem, disease)

mortality (death)

resources (manpower, funds, materials)

Generally raw data (numbers)

43

Under the line – denominators (population at risk)

size of target population at risk of the event

What group do they belong to:

- general population (total, catchment, target)

- gender population (male / female)

- age group population (<1, >18, 15-44)

- cases / events – per (live births, TB case)

An ideal An ideal indicator indicator RAVESRAVES !!! !!!

44

45

Indicators should be

RELIABLE gives the same result if used by different people

APPROPRIATE fits with context, capacity, culture and the required decisions

VALID truly measures what you want to measure

EASY feasible to collect the data

SENSITIVE immediately reflects changes in events being measured

Essential indicators: determines the essential data set at each level

Hierarchy of standards

Indicator Operationalization

Defining the sources of the data – both numerator & denominator (how is it to be collected?)

Determining the frequency of collection and processing of the indicator (How often should it be collected, reported, analyzed?)

Determining appropriate levels of aggregation(To where should it be reported and analyzed?)

Setting levels of thresholds and target

What will be the nature of the action (decision) once the indicator reaches the threshold?