Planning Protocols and Procedures (Monitoring and Evaluation Updating Plan) First National Course on Public Health Emergency Management 12 – 23 March 2011

Planning Protocols and Procedures(Monitoring and Evaluation

Updating Plan)

First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman


Learning objectives

By the end of this session, you should be able to:

Discuss basic principles of evaluation

Identify key lessons learned for health emergency managers from recent disasters

Identify areas for action in HEM capacity development


How can we Define Evaluation? (1)

The classic perspective: o Concerned with the achievement of objectives

The broad perspective:o Achievement of objectives is a key, but it is only part of what an evaluation might be concerned with


How can we Define Evaluation? (2)

Key elements are:

The need for systematic collection of information

The wide range of topics to which evaluation can be applied

To be effective, the evaluation results has to be used by someone

The wide variety of purposes of evaluations


What’s the Difference between Research & Evaluation?

Research & evaluations use the same toolbox

However, for a different purpose• Research aims to prove…• Evaluation aims to improve…


Why do we do evaluations?

It’s all about interventions or programs to:

Inform planning

Define progress

Examine efficiency

Examine effectiveness or achievement

Inform decision-making


How Could you Find out the Purpose of an Evaluation?

Who asked for evaluation?

Who pays for the evaluation?

Why do these people want an evaluation?

What are the decisions that need to be made?

What information is required to facilitate decision making?

Who is going to be affected by evaluation outcomes?


Needs assessments Before planning to understand context & needs

Program monitoring Compliance with policy/plan Validity of assumptions & pre-conditions

Formative evaluation Efficiency

Summative evaluation Effectiveness Decision-making

Classifying Evaluation Purposes


Evaluation Logic

Purpose Why do we do this?

Questions What do we want to know?

Objectives What information is needed?

Methods The way we collect/analyze information

Results Processed data or information

Conclusions Answers to questions

Recommendations Suggestions for decision-making


What Methods can you Think of to Collect Information?

Documentation research

Observations

Surveys interviews

Individual interviews

In-depth interviews

Focus group discussions


Quantitative Data Sources & Methods

Survey interviews

Indirect measures

Characteristics of samples of a community or subgroup

Observation

Conditions within samples of a

subgroup

Documented information on population groups


Qualitative Data Sources & Methods

Individual interviews

Focus groupsIn-depth

interviews

Key informantsIn-depth knowledge

Individual perspective intra-cultural variation

Community perspective

normative viewObservation

Behaviour

Indirectmeasures

Documented

information


About Methods

Questions will tell you what information you need

Information needed will tell you what methods could be used

Local context & resource limits will tell you what methods are feasible

Keep it as simple as possible


In Summary

To find out something useful about an intervention or program, use whatever you have in your toolbox that will get the job done…


Small Group Discussions

Based on personal experience, what are the lessons learned in terms of HEM?

Compare experiences & build consensus on common lessons learned

List on a flipchart your group results


SYNOPSIS OF EVALUATIONS ON THE HEALTH EMERGENCY RESPONSE TO SEA

REGION TSUNAMI

KEY LESSONS LEARNED FOR HEM


Sources

The Phuket Conference Papers (2005)

Tsunami Evaluation Coalition (2005)

The WHO-SEARO Tsunami evaluation (2006)


Lessons Learned in Context of HEM

• Early warning systems

• Health info system

• Capacity assessment

• Risk reduction

• Policy

• Needs assessment

• Emergency planning

• Incident command system

• Resource management

• Risk communication

• Donation management

• Operations management

• Recovery & reconstruction

• Capacity development


Early Warning Systems

Coordination structures need to be developed

Flexible surveillance systems need to be promoted

Surveillance systems need further development in terms of local capacity building:

PersonnelInfrastructure


Health Information System Management

Surveillance systems focused too exclusively on CD

Need to develop surveillance systems that are capable of covering all aspects of health

Surveillance and monitoring must be strengthened


Capacity Assessment

Private & other sectors should be involved in disaster reduction

Preparedness, mitigation measures & capacity building are more cost-effective than enhancing damage response

Vulnerability indicators need to be defined

I.e. safe hospitals


Risk Reduction

The importance of risk reduction need to be advocated

Health sector & facilities’ ability to recover from disaster stresses need to be improved

There is a need to develop vulnerability indicators

Integrate mitigation activities into the recovery process

PHC staff should receive training in disaster risk reduction


PolicyCountries that did relatively well in HEM in terms of national & international coordination had:

Well-developed policies & systems Strong health systems Clear lines of authority

Problems encountered post tsunami are not new:

Policy without enforcement is powerless Coordination policy-practice too often doesn’t happen

Relevant indicators on effectiveness, efficiency, costs, & benefits of both preparedness & response are needed


Needs Assessment-1

A pragmatic approach to needs assessment

Assessment must be relevant to decision makingAssessments must be timely

Invest in national assessment capacityStrengthen the capacity of national and local authorities to carry out cross-sector needs assessments


Needs Assessment-2

The need to streamline the international assessment mechanism

DysfunctionalCompetitiveNot shared

The need for a common information system of beneficiaries

Mainly statisticsIt does not tell who needs whatIt does not facilitate matching needs & offers


Emergency Planning & Response-1

Disaster preparedness is crucialThe quality of early responses is affected by the degree of preparedness

Lack of effective coordination: most common complaint

InternalExternalBetween policy & practiceAcross sectors & agencies

Information managementReporting formats should facilitate decision-makingMedia relationships including pre-disaster


Emergency Planning & Response-2

Flash appeals: a solution causing problems…

Disaster preparedness must include training on Flash Appeal Timing of funding was a problem Imbalanced demands: curative care & public health Funding worsening inequities

Consider psychosocial support mechanisms Health system deficiencies Alien expertise Community-based mechanisms


Incident Command System Operations

A 24-hour contingency response system should be in place

WHO internal relationships in response to the Tsunami raised questions:

Coordination Sharing of information Chains of decision making

First teams in the field should always be composed of experienced staff and not consultants


Resource Management-1

Recruitment & briefing of consultants calls for care

There is scope to strengthen regional & national financial resource mobilization capacity including:

Disaster preparednessMitigationResponse


Resource Management-2

WHO should assure its leadership in public health in its relations with UN agencies and NGOs

MOH should assure its leadership in public health in its relations with other ministries and international donors

Collaboration with the military is important and should be fostered


Risk Communication

Working with the media is important so that reporting is:

Technically soundEpidemiologically sound

We need to enhance the relationship with the media through:

Training of HEM managersRegular briefing of media Sharing some of the lessons learned


Donation Management

Make international response more relevant to needs assessment

Humanitarian response is an unregulated industry

Setting priorities and distribution of tasks among actors requires significant improvement

Assessing needs is pointless if decision making is not reformed


Operations Management

Standard Operating Procedures (SOPs) are essential and need to be developed & reviewed

Situation reports need to be standardized and regular

Logistics, supplies & donations can be troublesome


Recovery & Reconstruction-1

Planning & preparedness can reduce the impact of disasters

There is a need for a strong public health infrastructure

Planning and coordination of recovery & reconstruction is essential


Recovery & Reconstruction-2

Early responses do affect long term system functioning with consequences for health services

Researchers & implementers need to speak the same language

Relationships between national & local governments and international agencies are crucial for the course of development


Capacity Development-1

Provide leadership

Strengthening organizational & management capacities

Human resource development: International & national level staff are the sole barrier to improved health sector capacityNot by quick fix short courses Incorporate “Health Management in Disasters” in curricula of all relevant disciplinesNot HEM but HM knowing emergencies


Capacity Development-2

Develop partnerships & networksProfessional associationsMulti-sectoralNational RegionalInter-regional

Develop & share knowledgeCase studiesResearchContinuous learningHEM integrated in health curricula


Putting it together-1Build capacity in risk management & vulnerability reduction

Policy & legislation developmentDevelop disaster management lines of authority & controlAllocate resources for risk mgt & vulnerability reduction

Address the need for improved needs assessments

Relevant to decision-makingMulti-sectoralPopulation-basedInclusive


Putting it together-2

Enforce public health practice: Psycho-social support Gender equity Child health Management of the dead & missing Involvement of health volunteers IEC for watsan

Develop benchmarks & standards of practice

Vulnerability indicators Surveillance systems Needs assessments Indicators on effectiveness, efficiency, costs, & benefits of both preparedness & response



Improve management & coordination of responses

Internal External assistancePolicy-practice

Develop logistics of suppliesLegislationRelevanceCapacity building



Improve donation managementTimelySustainableRelevantFlexible

Foster contribution of other sectorsCivil-military liaisonsPublic-private sector liaisonsNGOs’ inclusion not marginalization



Develop media relationsIEC prior and during disastersCombat disaster mythsGuidance on media relations

Developing capacityHEM leadershipHEM capacityHuman resourcesNetworking & partnershipsKnowledge development


Capacity Development in HEM


Plenary Discussion

At the end of the day…considering:

The roles of HEM in disasters and

The lessons learned from disasters

How would describe the strengths & weaknesses in terms of HEM for your country?

Documents

Planning Protocols and Procedures (Monitoring and Evaluation Updating Plan) First National Course on Public Health Emergency Management 12 – 23 March 2011