Se30 improving hw ist - harvesting good practices and lessons learnt

1

Session overview and objectives Diana Frymus Health Systems Strengthening Advisor Office of HIV/AIDS USAID

2

3

Workshop objectives:

1. To launch the first global improvement framework for health worker IST

2. Harvest and share experiences, strategies, and lessons learnt on

addressing key IST challenges for health workers

3. Facilitate networking, shared understanding and learning between

health workers, Ministry of Health representatives, training institutions,

donors and partners to improve IST

Expected outcomes:

1. Participants will learn about the IST improvement framework and how

to use it to improve in-service training systems for health workers

2. Participants will enhance their understanding of how to evaluate in-

service training program outcomes and impacts

3. Participants will gain new knowledge and multi-stakeholder

connections to improve in-service training that will enable health

workers to provide better care

4. Participant experiences, lessons learnt and evidence will seed a new

global wiki on in-service training improvement

Launch of the in-service training improvement framework: What, why and how to apply Tana Wuliji Senior Quality Improvement Advisor USAID ASSIST Project, University Research Co., LLC (URC) @TanaWuliji

4

5

The in-service training improvement framework Towards more effective, efficient and sustainable training

Developed and launched by

International Center for AIDS Care and Treatment Programs (ICAP); International

Pharmaceutical Federation (FIP); IntraHealth; International Training and Education

Center for Health (I-TECH); Jhpiego; Sciences for Health (MSH); Pan-American Health

Organization (PAHO); Pathfinder; World Medical Association (WMA); USAID; University

Research Co. (URC)

What is the in-service training (IST) improvement framework?

A set of practice recommendations to improve

in-service training effectiveness, efficiency and

sustainability that were developed and agreed

through international expert consensus

6

Significant proportion of country HRH investments

Needed for scale-up of health services (HIV, TB, MCH)

Growing attention of countries in reviewing IST portfolios

Global landscape: Need for evidence-

based guidance for improving IST

How was the IST improvement framework developed? Delphi process

1. Develop and revise recommendations

2. Consolidate recommendations

3. Targeted literature review on

recommendations

4. Rate agreement on recommendations

25 member

consensus group

June – December

2011

Delphi process Development and review of recommendations through 5 rounds of peer review and ratings (Delphi process) and literature review

External open consultation: 4th round

• November 3-29 2011, online

• 26 countries

• 86/119 complete responses

Final consensus: 40 recommendations across 6 themes by 25 member expert panel

Improvement framework +

guidance Expected January 2014

How was the IST improvement framework developed?

IST improvement framework themes

10

1. Strengthening

training institutions

and systems

2. Coordination of

training

3. Continuum of

learning from pre-

service to in-service

4. Design and delivery

of training

5. Support for

learning

6. Evaluation and

improvement of

training

Using the IST improvement framework: Examples

• Nigeria- IST Assessment of PEPFAR Nigeria Portfolio

(CapacityPlus/IntraHealth)

• Afghanistan- Informed organization of National Training

Standards for the Afghanistan General Directorate of Human

Resources In-Service Department (HSSP/Jhpiego)

11

National IST strategy development workshop,

Oromia, Ethiopia, August 2012. Photo: USAID

HCI.

• Ethiopia- IST Rapid

Assessment to inform

Ethiopia Federal

Ministry of Health

efforts and strategies

to improve,

institutionalize, and

standardize IST

(HCI/URC)

In-service training journal series: Human Resources for Health Journal

12

Launched October 1st 2013 by

USAID, USAID ASSIST, Jhpiego,

I-TECH and IntraHealth

Open access: www.human-

resources-health.com/series/IST

Design and delivery of training: Recommendation: 4.3 IST should be based on sound, evidence-based learning principles and methodologies that offer the best opportunity to produce sustainable performance improvement within the workplace.

Explanatory note:

In order for IST programs to most effectively improve health worker competencies,

they should apply state of the art learning principles and methodologies appropriate

to the context that are based on the latest evidence……

Examples:

• For the development of clinical decision

making skills, a RCT comparing an

interactive workshop-based training to

simulation training found that simulation

training resulted in better skill performance.

This is reinforced by systematic reviews

that identify clinical simulations as an

effective technique for developing

psychomotor and critical thinking skills.

Daniels et al, 2010; Nestel et al, 2011;

Issenberg et al, 2005

Resources:

• The Johns Hopkins Evidence-

Based Practice Center

recommends multiple techniques,

multiple media, interaction and

repetition. Marinopoulos SS et al,

2007.

• The Learning for Performance

tools provide guidance on

designing training for desired

performance

http://www.intrahealth.org/page/le

arning-for-performance

http://www.intrahealth.org/page/learning-for-performance






Expert Consensus Group Marita Murrman, International Center for AIDS Care and Treatment

Programs (ICAP); Mike Rouse, International Pharmaceutical

Federation (FIP); Rebecca Bailey, Shaun Noronha, Kate Tulenko,

IntraHealth; Gabrielle O’Malley, Tom Perdue, Frances Petracca,

International Training and Education Center for Health (I-TECH);

Julia Bluestone, Peter Johnson, Edgar Necochea, Jhpiego; Karen

Chio, Gail Naimoli, Management Sciences for Health (MSH); Rosa

Maria Borrell, Pan-American Health Organization (PAHO); Cathy

Solter, Graciela Salvador-Davila, Pathfinder; Julia Seyer, World

Medical Association (WMA); Diana Frymus and Lois Schaefer,

USAID; Tana Wuliji, Akuba Dolphyne, Tisna Veldhuizen Van Zanten

University Research Co. (URC); Habib Benzinan, Alan Lyles, Linda

Ippolito, Hugo Mercer, Independent experts.

14

Lightning talks

15

1. Strengthening


and systems

2. Coordination of

training

3. Continuum of

learning from pre-



of training

5. Support for

learning

6. Evaluation and

improvement of

training

1. Strengthening


and systems

2. Coordination of

training

3. Continuum of

learning from pre-



of training

5. Support for

learning

6. Evaluation and

improvement of

training

• Engage national authorities, regulatory and professional bodies

• Build capacity and strengthen local infrastructure and trainers

• Support CPD systems

Theme 1: Strengthening training institutions and systems

Ethiopia’s experience with in-service training institutionalization and standardization Dr Samuel Hailemariam, Health Systems Strengthening Advisor, USAID Ethiopia

17

1. Strengthening


and systems

2. Coordination of

training

3. Continuum of

learning from pre-



of training

5. Support for

learning

6. Evaluation and

improvement of

training

• Coordinate IST

• Minimize duplications: Coordination mechanism

• Reduce disruption to health services

• IST tracking mechanism

Theme 2: Coordination of training

iHRIS system for tracking training Rebecca Bailey Team Lead, Health Workforce Development, CapacityPlus, IntraHealth

19

Coordination of Training

iHRIS system for tracking training

Sam Ngobua, Director, CapacityPlus, Nigeria

Rebecca Bailey, Team Lead, Health Workforce Development, CapacityPlus

Nigeria - Context

• Substantial US Government investment in in-service training (IST) of health workers in HIV/AIDS-related topics.

• A 2012 assessment – based on the IST Framework – found limited coordination of IST among PEPFAR-funded implementing partners.

• Recommended introducing an information system to track IST across implementing partners in order to decrease duplication, improve coordination, and ensure that the right health workers receive the right kinds of training.

Objectives of the Training Information System

• Track trainees to avoid double training of the same health worker

• Coordinate training between PEPFAR-funded implementing agencies

• Track availability of Master Trainers

• Ultimately reduce cost and improve efficiency

http://www.google.com/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&docid=6YSw7Y_wbU-61M&tbnid=8z0-xUcmC3GArM:&ved=0CAUQjRw&url=http://www.gedefoundation.org/training-on-hivaids-counseling-and-testing/&ei=DIOBUs_tLY7OkQfym4CIDQ&bvm=bv.56146854,d.eW0&psig=AFQjCNHW_q6G3aSPpQLlncx9Rr_z0bKjpQ&ust=1384305732036566

iHRIS Train Prototype for Nigeria

• Web-based application developed on the open-source application platform of iHRIS

• Open-source, user friendly and user operating system independent, affordable and customizable

• Sustainable through local capacity building

• Backed by IntraHealth informatics experience in over 23 countries

23

Welcome Page – Implementing Agency

iHRIS Train Nigeria 24

View and Edit Training Courses


View Calendar of Scheduled Training Courses


View Summary page of Participants


Search for other Implementing Agencies


List of Participants Report


CONCLUSION


• iHRIS Train Nigeria is a work-in-progress.

• It is scalable to meet the needs of the PEPFAR-funded implementing agencies and the Federal Government of Nigeria for an In-Service Training Information System.

• Affordable and sustainable

• Backed by the multi-national experience of CapacityPlus/IntraHealth Informatics.

Questions for Knowledge Café Discussion

• Is there a need for an electronic IST

tracking system in your country or for your

training programs?

• What characteristics or functions are most

desirable in a training information/tracking

system?

1. Strengthening


and systems

2. Coordination of

training

3. Continuum of

learning from pre-



of training

5. Support for

learning

6. Evaluation and

improvement of

training

• Synergies between pre-service education and IST systems

• Consistency in approaches and content

Theme 3: Continuum of learning from pre-service to in-service

Lessons learnt by professional associations, regulatory bodies, nursing schools and Ministries of Health from strengthening CPD systems Carey McCarthy Health Systems Scientist, CDC supported African Regulatory Collaborative

34

1. Strengthening


and systems

2. Coordination of

training

3. Continuum of

learning from pre-



of training

5. Support for

learning

6. Evaluation and

improvement of

training

• Needs based: aligned with plans, understand performance barriers

• Compliance with policies, strategies and laws

• Evidence based methodologies

Theme 4: Design and delivery

Findings from an integrative review of literature on training design and delivery Julia Bluestone Senior Technical Advisor, Jhipego @juliabluestone

36

Effective In-Service Training Techniques,

Frequency, Setting and Media:

Synthesis of an Integrative

Review of the Literature

What evidence is there to

support educational

techniques, Frequency (single

vs. repetitive), setting, and

media used to deliver

instruction?

What evidence exists about

the outcomes (knowledge,

skills, attitudes, provider

behaviors, patient outcomes)

of continued health

professional education?

In the context of continuing health

professional education

Julia Bluestone, MS

Peter Johnson, PhD

Judith Fullerton, PhD

Catherine Carr, DrPH

Jessica Alderman, MPH

James BonTempo, MS

What surprised us? (Or didn’t!)

Didactic-”low to no” outcomes

Simulation effective

Practice and feedback ‘dosage’

matters

Shorter, but more often

Workplace-setting better

39

Question:

Given the current system is

designed for group-based training,

what changes do we need to make

to implement, shorter, repeated,

simulation-heavy, workplace-based

training?

40

1. Strengthening


and systems

2. Coordination of

training

3. Continuum of

learning from pre-



of training

5. Support for

learning

6. Evaluation and

improvement of

training

• Share resources and materials

• Support trainees post-training

• Life long learning skills

Theme 5: Support for learning

Point of care performance support tools linking standards to resources needed to support them Julia Bluestone Senior Technical Advisor, Jhipego

42

SSDI Services

Mentoring in Malawi

Support for Learning

SSDI Services

Question:

How do we prepare, support and incentivize busy health care workers to provide workplace-based mentoring or support to others?

1. Strengthening


and systems

2. Coordination of

training

3. Continuum of

learning from pre-



of training

5. Support for

learning

6. Evaluation and

improvement of

training

• Build in evaluation to inform continuous improvement

• Evaluate against defined criteria

• Engage key stakeholders and trainees

Theme 6: Evaluation and improvement of training

Training Evaluation Framework and Tools (TEFT) Gabrielle O’Malley Director for Implementation Science, I-TECH

49

Is training making a difference?

International Education and Training Center for Health

INDIVIDUAL

SITUATIONAL FACTORS

ENVIRONMENT

HEALTH SYSTEM / POPULATION

ORGANIZATION

TRAINING EVALUATION FRAMEWORK

Training

Population level systems improvements

Population level performance outcomes

Population patient health Outcomes (impacts)

Organization systems improvements

Organization performance outcomes

Organization patient health outcomes (impacts)

Individual knowledge, attitude, skill outcomes

Individual performance outcomes

Individual patient health outcomes

INDIVIDUAL • Trainee background,

knowledge, experience, education

• Intrinsic motivation • Family demands

HEALTH SYSTEM/POPULATION • National, regional, community

systems – labs, supply chain • National, regional policies • Partner programs • Available health workforce,

including informal private, attrition issues

• Pre-service program • Retention factors, e.g. pay scales

ENVIRONMENT • Political instability • Prevalent disease • Natural disasters • Food availability • Seasonal changes • Patient access to food,

transportation • Available community support

resources

ORGANIZATION • Management support • HR – staffing levels, salaries,

burnout • Available drugs, supplies,

equipment and infrastructure • Facility systems –

appointments, records, flow, referrals

• Patient needs

Training Evaluation Framework (TEFT)

6 planning steps with tools and guidance

Planning steps Tools 1. Identify anticipated

outcomes Framework and template

2. Address situational factors Worksheet

3. Refine the scope of the evaluation

5 Considerations tool

4. Define evaluation questions, objectives, and indicators

Worksheet

5. Choose evaluation design and methods

Resource tables

6. Plan the evaluation Planning template

Training Evaluation Framework and Tools How can evaluation of IST outcomes be strengthened to inform improvement in training? What are some of your successes and some of your needs in identifying results of your training programs?

http://www.go2itech.org/resources/TEFT

http://www.go2itech.org/resources/TEFT

Knowledge cafe

54

1. Strengthening training institutions and systems: Dr Amir Aman, Samuel Hailemariam

2. Coordination of training Sam Ngobua, Rebecca

Bailey 3. Continuum of learning

from pre-service to in-service

Carey McCarthy

3 x 10 min discussions on

theme of your choice

4. Design and delivery Julia Bluestone 5. Support for learning Lois Schaefer 6. Evaluation and

improvement of training Gabrielle O’Malley,

Francie Petracca

Summary of discussions Lois Schaefer Senior Technical Advisor for HRH, USAID

55

56

Documents

Se30 improving hw ist - harvesting good practices and lessons learnt