1

IMPROVING THE QUALITY OF HIV/AIDS SERVICES

FACILITATOR’S GUIDE FOR

QUALITY IMPROVEMENT TRAINING

Developed by Shelemo Shawula Kachara (MD, MPH)

(Organizational Development Specialist) FOR

TECHNICAL ASSISTANCE TO CDC/PEPFAR-FUNDED NEW PARTNERS INITIATIVE PROJECT

JUNE 2012 KAMPALA, UGANDA

2

Contents I. Introduction

A. Purpose B. Audience C. Learning Objectives D. Curriculum Overview E. Facilitators F. Materials G. Preparation H. Agenda II. Workshop Sessions Session 1: Introduction, Objectives and Scope of the Workshop Session 2: The Basics of Quality and Quality Improvement Session 3: An Organization’s Mandate and Quality Improvement Session 4: Standards of Services and Operations Session 5: Measuring Quality Session 6: Supportive Supervision Session 7: Information Management Session 8: Concepts and Principles of Quality Improvement Session 9: Quality Improvement Model Session 10: Institutionalizing Quality Improvement

Annexes Annex I: PowerPoint Slides for All Sessions Annex II: Survey Forms A. Organization’s Standards and Quality Improvement Practices (Pre-Workshop) B. Individual Participant’s Quality Improvement Experience (Pre-Workshop) C. Daily Evaluation of Workshop Sessions Annex III: Case Studies and Other Information A. The Dimensions of Quality of HIV and AIDS Services B. Case Study on Supportive Supervision by the Community Development Organization C. Generic Draft Checklist for Integrated Supportive Supervision D. Quality Improvement Model Case Scenario for Group Exercises

3

Improving the Quality of HIV and AIDS Services I. Introduction

A. Purpose To develop an organization’s systems and processes for setting standards, measuring and monitoring performance against the standards and improving the quality of services at facilities, in communities and in program management

B. Audience Department heads, program managers, M&E officers, HR/admin and finance staff relevant to program implementation C. Learning Objectives General Objective: To strengthen the capacity of NPI grantees to assess and continuously improve the quality of HIV and AIDS prevention, care and support services and program management Specific objectives:

• To define quality and to introduce the concept of continuous improvement and a framework for managing quality

• To develop or revise quality assessment and improvement tools and techniques to continuously improve HIV and AIDS services

• To build the capacity of NPI grantees to coordinate quality with their implementing partners • To create momentum for quality improvement (QI) through better understanding its

principles and practices D. Curriculum Overview

QI training covers three broad areas: defining, measuring and improving the quality of HIV and AIDS services. The components are summarized in the following table. Defining quality

• Concepts of quality, improvement framework, outcomes of quality of care • Content, dimensions and perspectives of quality of care • Standards of quality: performance or service standards, reviewing/revising and

updating quality standards for orphans and vulnerable children (OVC) and people living with HIV and AIDS (PLWHA)

Measuring quality

• Principles of and practices for measuring quality • Tools and techniques to measure quality (indicators, tools and processes) • Data and/or information sources and managing information, identifying

problems and data-based solving (decision making) Improving and sustaining

• Improvement approaches • Model for improvement (Plan-Do-Study-Act)

4

quality

• Steps and processes for improvement • Translating knowledge of continuous quality improvement into practice, • Institutionalizing and sustaining quality improvement

This is an integrated training program designed to address HIV and AIDS service delivery (technical programs) and program management. The training enables an organization to assess and revise its management to facilitate the smooth implementation of quality HIV and AIDS services in communities and facilities. Four rounds of QI training were provided from October 2009 to October 2010. Each round was conducted over 5 days. Three of the four included field visits to implementing organizations in addition to the sessions. This enabled participants to gain practical skills in supervision and giving constructive feedback to service providers and managers. E. Facilitators Four facilitators were recruited and oriented for the sessions and to lead field visits with an estimated ratio of 1 facilitator to 8 participants.

F. Materials

Prepare participants’ folders with the agenda, hand-outs of the PowerPoint presentations (Annex I), the daily training evaluation form and summaries of the pre-training surveys (Annex II) and case studies and supportive supervision checklists (Annex III).

G. Preparation

1. Pre-training surveys: The Organization’s Standards and QI Practices Survey (Annex II Part A) and the Individual Participant’s QI Experience Survey (Annex II Part B) are to be completed and submitted before the workshop. The survey findings are very useful in selecting appropriate topics, in designing the contents and in delivery. Summarize the findings in each survey separately and present them to participants in Session 1.

2. Training materials and logistics: PowerPoint Presentations (Annex I), case studies and supportive supervision checklists (draft) (Annex III), LCD projector, flip charts with stands, markers.

3. Evaluations (Annex II Part C) are to be completed in the last 30 minutes of each day then summarized every evening and presented at the beginning of the next day.

H. Agenda The agenda and the scope of the sessions are as follow.

5

QUALITY IMPROVEMENT TRAINING Day1: Introduction and Definition of Quality of Health and Social Services Time Themes Topics/Activities Facilitators 8:30-9:00 Registration One of the facilitators or admin staff assists with the registration 9:00-9:30 Opening remarks; introduction Introduction (participants and facilitators), participants’ expectations and

concerns; share the goal, objectives, contents and expected outcomes. Set “rules of freedom” (group norms); ask for 2 volunteers for recap next day

9:30-10:00 Explore participants’ understanding and organization’s QI practices

Present a summary of pre-training survey results

10:00-10:30 Tea Break 10:30-12:30 The basics of quality Quality of health care or public health services; defining quality of care,

dimensions and perspectives of quality; the concept of improvement, a framework for quality improvement to define, measure and improve quality

12:30-13:30 Lunch Break 13:30-14:30 Defining quality: standards and

targets of performance

Terms and definitions, types of standards, guidelines. Definition or contents of service standards (packages, minimal critical activities or essential actions). Standards of input-process-output. Identify service standards. Scope of standards: international, national or local standards, program or project standards. Standards for home-based care (HBC) and OVC. Communicating quality standards (creating awareness and demand to enhance providers’ understanding and compliance; clients or beneficiaries understand and demanding the services. Make standards KNOWN TO ALL!

14:30-15:30 Standards and quality improvement/quality assurance

Relationship between quality (standards measuring improvement) and indicators of quality/or performance

15:30-16:00 Tea Break 16:00-17:00 Review performance standards Develop performance standards by levels based on service delivery setting

(national office, district office or community level, beneficiaries and providers); introduce and give guidance for group exercise: Identify roles and responsibilities of different actors in service delivery set up. Reviewing standards and tools takes place on Day 2 of the training

17:00-17:30 Evaluation and close the day Day 2: Review, Understand and Update the Standards for the Quality Improvement 8:00-9:00 Recap Day 1 learning and

summary of the evaluation Recap day 1, feedback on evaluation

9:00-10:30 Review OVC standards 10:00-10:30 Tea Break 10:30-12:30 Review home-based care (HBC) Performance and quality indicators by management and service delivery levels;

6

standards work in country/organization groups if participants are coming from various countries or organizations; group presentations of the key contents of their standards

12:30-13:30 Lunch Break 13:30-15:30 Review/update tools (checklists)

for OVC Group work and presentation ( similar groups)

15:30-16:00 Tea Break 16:00-17:00 Review/update tools (checklists)

for HBC Group work and presentation ( similar groups)

17:00-17:30 Evaluation, Close the day Day 3: Measuring Quality 8:30-9:00 Recap day 2, feedback on

evaluation Recap day 2, feedback on evaluation

9:00-10:30 Measuring quality

How do we know the quality we are providing? Concepts and principles of measuring quality including what to measure such as performance against standards and targets (inputs, how services are delivered, any requirements to be met by the clients in order to access services, anticipated outputs and outcomes)

10:30-11:00 Tea Break 11:00-12:30 Measuring quality Conducting effective supervision. Objectively verify services provided to OVC

and PLWHA. Observe while care provided; review care records; interview beneficiaries, families and communities; review of tools (from previous day): interview prompts, checklists, forms, observation guide, template to document or keep records of findings; group work and presentation OR field visit to implementing organizations to practice supportive supervision or review service delivery and provide constructive comments

12:30-13:30 Lunch Break 13:30-15:30 Measuring quality Supportive supervision: understand the role; case study of effective

supervision; group work and presentation OR field visit

15:30-16:00 Tea Break 16:00-17:00 Measuring Quality Supportive supervision: Role play/presentation (observers brief); select five

volunteers on Day 2 OR field visit to implementing organizations to practice supportive supervision and providing constructive feedback to providers

17:00-17:30 Evaluation, Close the day Day 4: Measuring and Improving Quality 8:00-9:00 Recap day 3, feedback on

evaluation Recap day 3, feedback on evaluation

9:00-10:30 Information management How do we make use of information from quality assessments? Data handling,

7

data processing, storage and analysis; preliminary gaps in services provided (problem identification, analysis and data use )

10:30-11:00 Tea Break 11:00-12:30 Concepts and principles of QI Concept of improving quality; cost of quality (cost of correcting errors, cost of

not doing the right job, cost of change or improvement); continuous QI (CQI) team and process

12:30-13:30 Lunch Break 13:30-15:30 QI model and collaboration PDSA—Plan—Do-Study—Act cycle; quality collaboration (community of

practice). Apply PDSA in case scenarios and/or examples from real life experiences of the facilitators and participants OR take case study of supervision a step further

15:30-16:00 Tea Break 16:00-17:00 QI model and collaboration Continued 17:00-17:30 Evaluation and Close the day Day 5: Institutionalize and Sustain Quality Improvement 8:00-9:00 Recap day 4, feedback on

evaluation Recap day 4, feedback on evaluation

9:00-9:30 Institutionalizing QI Reflections on lessons learned so far and brainstorming how participants will apply the learning in their organizations, their day-to-day work to improve quality of services. This brief PowerPoint presentation will lead to practical exercises to identify, analyze and plan for improvement

9:30-10:30 Improving practice

Applications of PDSA through case scenarios or examples from real life experience from facilitators and participants OR take case study of supervision a step further

10:30-11:00 Tea Break 11:00-12:30 Improving practice

Work on their projects

12:30-13:30 Lunch Break 13:30-15:30 Improving practice

Work on their projects (wrap up)

15:30-16:00 Tea Break 16:00-16:30 Evaluation and close of training

8

II. Workshop Sessions

DAY 1

The Basics of Quality and Quality Improvement Session 1: Introduction, Objectives and Scope of the Workshop A. Objective

• To create a conducive training atmosphere by welcoming and introducing participants and facilitators and sharing objectives and contents

B. Guide Topic Format Timing Welcome the participants and introduce facilitators

Plenary presentation 5 min

Introduce participants Self-introductions 10 min Expectations/concerns Plenary discussion 10min

Overview, objectives, agenda and participants’ folders

Plenary presentation 20 min

Summary of pre-training questionnaires

Plenary presentation 15 min

Total: 1 hour C. Preparation and Supplies

• Participants’ folders with agenda, hand-outs and supplementary materials • LCD projector • PowerPoint slides 1.2.-1.4 • Pre-training surveys: Organization’s Standards and QI Practices (Annex II Part A) and Individual

Participant’s QI Experience Survey ( Annex II Part B) • Flip chart and markers

D. Facilitators’ Notes for Session 1 1. Welcome to the workshop

One of the facilitators welcomes everyone to the workshop and tells them that the workshop has been designed to introduce QI concepts and to strengthen QI practices for HIV and AIDS programs. S/he introduces him/herself. If the workshop is organized for only one organization and its partners, the facilitator invites the organization’s director or a senior management staff to welcome all participants and facilitators. Then the facilitator invites other facilitators to introduce themselves one by one.

9

2. Introduction of participants Ask participants to introduce themselves individually by name and job title. 3. Participants’ expectations and concerns Give each participant two cards of different colors. Tell them to write their expectations on one card (specify color) and concern(s) on the other card. Make sure everyone understands which color is meant for expectation(s) and which for concern(s). Also write the instructions (colors) on the flip chart. Post the cards on the wall for easy review and to link them to training. 4. Overview, objectives, agenda and participants’ folders Present the workshop’s general and specific objectives by using slides 1.2 and 1.3. Emphasize the fact that the goal (general objective) of the workshop is to build the technical capacity of NPI partners to plan and implement a sustainable quality improvement system. Tell participants that there are four specific objectives and go through them. Using slide 1.4, present the general contents of the workshop. At the end of this slide, ask participants to pull out their agendas (in their folders) and go through the schedule. Using the table of contents for the folders, indicate the supportive documents for each session. 5. Pre-training survey summary Start by thanking participants and their colleagues for completing and returning the pre-training survey. The Organization’s Standards and Practice Survey (Annex II Part A) is completed by the management team of the organization and the Individual Participant’s QI Experience Survey (Annex II Part B) was completed by the prospective participants. Briefly go through the survey questions to refresh their memories and tell participants how the findings were summarized by paying attention to responses from each of them. Present the summaries you have prepared in advance and give practical examples to support the points. Keep the responses anonymous unless you think the processes and practices will have positive influences on other participants. Conclude by telling participants that their responses to the survey questions guided the development of the contents and flow of this workshop. Link the survey to various sessions of the workshop and how the contents will be addressed in the five days of training. 6. Wrap-up Conclude the session by telling participants that the workshop is highly interactive and that facilitators will use a variety of participatory learning approaches including well-designed, interactive lectures; case studies; role plays; field visits and reviews of standards; tools and documents that they can adapt for their organization’s future use.

10

Session 2: The Basics of Quality and Quality Improvement A. Objectives

• To understand the concept and building blocks of quality of care as a measure of contents, dimensions and perspectives

• To introduce the concept of quality improvement, its principles and framework

B. Guide Topic Format Timing Review objectives Plenary presentation 5 min Defining quality Self-introduction 10 min Quality in daily life Group work and presentation 10min Properties of quality Plenary discussions 60 min

Quality of health care and public health services

Plenary discussions 15 min

Quality assurance/improvement Plenary discussion 15min

Total: 2 hours C. Preparation and Supplies

• LCD projector

• PowerPoint slides 2.2- 2.20 • Flip chart and markers

D. Facilitators’ Notes for Session 2 1. Review objectivesIn plenary, start by reviewing the objectives, i.e., to orient all to the meaning of quality in health care. The session tries to create a shared understanding of the ever-changing or dynamic nature of quality. It explains why it is important to focus on improving or maintaining good quality and it reviews the concept of and a model for quality improvement. Tell participants that the session is organized into different segments starting with the general concept of or definition of quality as it applies to daily life or how we perceive quality in our common practices such as selecting a restaurant. We then will apply a similar analysis to the HIV and AIDS programs they are implementing. 2. Defining quality In plenary, ask participants to state their understanding or definition of the quality of “something.” They can choose what to illustrate. Ask what they have in mind when they talk about quality. Some might talk about training similar to this workshop or a piece of fabric or a cloth or a building or a food item. Write key points on the flip chart. Using slide 2.3, share a general definition of quality from a dictionary, e.g., “a degree/grade of excellence or worth” (American Heritage College Dictionary © 1993) OR “the degree to which social services meet or exceed the needs and expectations of the intended user or clients.” Briefly discuss by

11

building on what the participants brought out. 3. Quality in daily life, health and HIV and AIDS programs While sitting, divide participants into groups of 4–5. Display the scenario on slide 2.4. Give them 10 minutes to discuss it and report back in plenary. Write their responses on the flip chart. Guided by slide 2.5, document key features or responses from the group presentation. Allow participants to take a minute or so to think about their programs. Then ask them to what extent features or aspects of quality in daily life (from slides 2.4 and 2.5) are applicable to their health or HIV and AIDS care and support services. In most instances, participants may have stated that all or nearly all features of quality from the scenario are applicable to HIV and AIDS programs. That is right; however, some features are more applicable to restaurants or vice versa (slide 2.6). Appreciation of various features or aspects of quality may vary from person to person. These are very important considerations; tell the participants that the discussion will be organized into logical blocks or properties of quality. 4. Properties of quality By using slide 2.7 and the preceding discussions, present the fact that quality varies depending on the nature of the product or service (contents), the aspect(s) of the service under consideration or discussion (dimensions) or the views of the person who produces, manages or seeks the service or product (perspectives). Ask for participants’ experiences, comments or examples of how these properties make sense to them. Using Slide 2.8, ask participants to think about their own programs. e.g. prevention, HBC, OVC, treatment (technical program) or management such as administration, finance or human resources. Ask them to list the services they provide. Slide 2.9 helps to further a discussion on the contents of HIV and AIDS services and the programs they are implementing, and how quality considerations might vary depending on the service delivery arrangements, i.e. whether the organization implements its program directly, through partnerships and networking or through grantees. These arrangements affect aspects of the service and need careful designing for maximum effect. Use slides 2.10 and 2.11 to start a discussion on the different dimensions of quality such as safety, appropriateness, completeness, relationship, access, continuity and sustainability. Ask participants to define each of the terms or dimensions. At the end of the discussion, tell participants to refer to “Dimensions of Quality of HIV and AIDS Services” (Annex III Part A) in their folders for standard definitions of these dimensions. Slide 2.12 helps to start and conclude a brief discussion about various views on quality. Discuss how quality varies based on the views of different persons. Four key perspectives are those of the provider, of the client or beneficiary, of the manager or of another stakeholder who plays an indirect role. Use examples from the preceding properties and elaborate the points. 4. Quality of health care and public health services

12

Start off by defining quality of health care or public health services (social services). Slide 2.13 shows one of the many definitions of quality of care or social services: “The proper performance, according to standards, of interventions that are known to be safe, affordable and have the ability to produce desirable impacts” (Roemer and Aguilar 1988). This definition has three essential components: standards of performance, evidence-based (safe, affordable and effective) interventions and proper performance as judged against the standards. Using Slide 2.14 explain quality of operations (management). Like technical interventions, operations need to have clear systems and internal processes that are proven to be effective (produce good results in similar settings) with minimal cost. Properly managing resources to reduce waste and to have more impact (efficiency) is another essential component of quality management. Further, elaborate quality of care and public health services by using slides 2.15 to 2.17. Use slide 2.17 to emphasize the fact that the numerical achievement of the planned targets such as numbers or percentages is very important; however, this achievement is in no way complete without looking at the stories of how well each of the targets was achieved—including both the desired and undesired effects of the interventions. 4. Quality assurance and quality improvement Use slide 2.18 to introduce quality assurance (QA) and QI. Tell participants that they have very similar concepts, principles and approaches. Both QA and QI are standard approaches and require meeting performance standards, measuring performance against standards and improving any gaps identified. These are essential components of QA/QI. In literature and documents these terms are used interchangeably; they are applied interchangeably in this workshop. The only distinction between them is who sets the standards. QA focuses on standards, measurement and recommendations set by external bodies such as accreditation commissions and periodic external performance audits while QI focuses on internal processes in which the organization commits itself to continuous quality improvement by organizing an internal team(s) that takes charge of setting standards, developing performance indicators, measuring and monitoring performance against standards and targets and making necessary adjustments to improve performance. Use slide 2.19 to introduce the improvement cycle: a reiterative process of setting standards and measuring, improving and sustaining performance and the quality of health care, public health services or management. Slide 2.20 helps to conclude this session by emphasizing the importance of the organization’s mandate/mission and standards of performance and the importance of measuring and improving quality. Tell participants that this session tried to give a condensed version of the five-day workshop’s contents and that the rest of the sessions will go into greater depths in each area. 5. Remind participants to bring their organization’s by-laws, constitution, strategic plan or other

essential documents to the next session.

13

Session 3: An Organization’s Mandate and Quality Improvement A. Objectives

• To understand the relationship between an organization’s mandate, performance standards and

quality of services • To understand how quality improvement will contribute to an organization’s mandate, goal and

objectives • To apply the relationship between mandates and quality in measuring, monitoring and

improving the quality of HIV and AIDS services

B. Guide

Topic Format Timing Review objectives Plenary presentation 5 min Understand organization’s mandate Plenary, participatory discussion 10 min Understand organization’s vision and mission

Plenary, participatory discussion 10min

Understanding roles and responsibilities

Plenary discussions 20 min

Understanding standards and quality Plenary discussions 15 min

Total: 1 hour C. Preparation and Supplies

• LCD projector

• PowerPoint slides 3.2–3.25 • Flip chart and markers • Organization’s strategic documents (constitution, bylaws, strategic plan or others) that participants

should bring with them

D. Facilitators’ Notes for Session 3 1. Review objectives

Start the session with slide 3.2. The session has three complementary objectives: to understand the terms and concepts of the organization’s strategic existence (big picture), i.e., mandate, vision and mission, roles and responsibilities and performance standards; to analyze how quality improvement is associated with and contributes to the organization’s existence and finally, to apply the knowledge they will gain from this session to the subsequent sessions. Tell the participants that this session is critically important for understanding how both technical programs and management are related, and how both components of an organization are indispensable. The emphasis in this workshop is on ensuring that quality is integrated into both technical programs and management!

14

2. Understanding an organization’s mandate Using slide 3.3, ask participants to define or share their understanding of the term mandate. Then ask them to apply the common definition to an organization to define its mandate. Slide 3.4 defines mandate as, “…overall duties and responsibilities, power and authority of an organization granted by the law of a country.” It further describes formal as well as information expectations about what an organization does or does not do. Discuss this concept further by using slide 3.5. Slide 3.6 explains why an organization’s mandate is important. There are two essential elements. First, mandates provide a legal standing and set boundaries and broad expectations for the organization’s existence. Second, mandates are formal foundations for the organization’s identity, vision or mission. The programs, structures/institutions, operations and management systems of an organization perform within the provisions or restrictions of the organization’s mandate, so the mandate precedes and supersedes the organization’s mission and goal. Tell participants that mandates are documented in constitutions, bylaws, memoranda of association or similar documents; the names might vary from country to country or by type of organization. Conclude this discussion by asking for and giving examples from organizations. Share slide 3.7 with the list of mandates. Ask for comments. Then give 10 minutes to each organization to review their documents and share their mandates. Use slide 3.8 to capture key mandates from group presentations. 3. Understanding an organization’s vision and mission Using slide 3.9, ask participants to define or share their understanding of the terms vision and mission and how they are related to mandates. Allow a few minutes for the discussion. Note key points on the flip chart. Slides 3.10 and 3.11 compare and contrast mission and vision. The key points are that the mission is the organization’s reason for existence and justification for what it is does. The vision is how the organization wants to look and how it should act in the future to fulfil its mission. In other words, the vision communicates enthusiasm and provides the organization with something to aspire to. Slides 3.12 and 3.13 offer further descriptions of vision and mission. A clear vision and mission give an organization a strong sense of purpose. Use slide 3.13 to share an example of vision and mission statements, then ask participants to share theirs. Do participants clearly understand and can they articulate their organization’s vision and mission? If not, emphasize the fact that they would be much more motivated and engaged in the organization’s future if they had a clear understanding of and were able to articulate its vision and mission. Tell the participants that it is very important to communicate their vision and mission statements as widely within the organization as possible. A variety of channels can be used, e.g., official documents, letter heads/footers, websites, brochures, etc. Conclude this discussion by relating the vision and mission to the organization’s mandate. In short, tell them that mandates set legal boundaries for an organization. Any change in the status of the mandate

15

must be documented and presented to a statutory body for approval. On the other hand, while the vision and mission should be developed and remain within the mandate, an organization can change them without notifying a statutory body. This gives it flexibility. 4. Understanding an organization’s responsibilities Using slide 3.15, ask participants to share their understanding of what an organization’s responsibilities are. Allow a few minutes for the discussion. Note the key points on the flip chart. Slide 3.16 describes an organization’s responsibilities as specific duties emanating from its mandate or statements of what the organization and its institutions are meant to do. In performance planning terms, responsibilities are the key result areas (KRA) in the organization’s mandate. The mandate dictates duties and responsibilities which dictate performance expectations. Tell participants that responsibilities are relatively clear and measurable, but an organization might focus on a few or all areas of its responsibilities depending on its growth and capacity. Using slide 3.17, share the responsibilities of the Ministry of Health (MOH) of Ethiopia and compare them with its mandate on slide 3.7. Ask participants to state the responsibilities of their respective organizations. List key responses using slide 3.18. Slide 3.19 adds another very important dimension on how an organization’s responsibilities are translated into programs and operations. Based on the response to slide 3.18, ask participants to what extent their current programs are aligned with their organization’s mandate, role and responsibilities and to explain their responses. Applying the discussions to their real life situations is important for keeping the session relevant and participants engaged. 5. Understanding performance standards and quality of services Using slide 3.20, ask participants what they understand by the term performance standards. Allow a few minutes for the discussion. Note key points on the flip chart. Slide 3.21 describes and characterizes performance standards as documented statements of expectation that tell how the organization should fulfill its responsibilities. As such, standards are linked to the organization’s duties and responsibilities. Standards specify what tasks and when, where, how, with what tools and at what cost the organization must do them to fulfil its responsibilities. Standards state the resources and technical and financial inputs required to accomplish the tasks. Generally, performance standards are set for technical/health/service delivery and for management. Slide 3.22 provides performance standards for MOH Ethiopia (mandate on slide 3.7 and responsibilities on slide 3.17). Standards have been developed for two of its many responsibilities: 1) planning, monitoring/evaluating and reporting and 2) research and development. Performance standards are very specific for the MOH and for those two responsibilities, but the list is not exhaustive. There may be many more expectations to meet these two responsibilities. At this point, ask the participants to imagine how long the list would be for the entire MOH. When we talk about standards, we don’t talk about statements scattered here and there but rather documents of many pages listing specifications/expectations. Ask participants to give examples of performance standards for their respective organizations. List the key points on the flip chart or show slide 3.23. Using slide 3.24, link mandate, responsibilities,

16

vision/mission, performance standards and quality. This consolidates the hierarchical relationship: mandates—roles and responsibilities—standards—quality of care. Slide 3.25 links performance standards to quality: Quality is the degree to which health care or social services meet or exceed the standards. That leads to the summary slide 3.26 which gives a bird’s eye view of the different components discussed in the session and shows how they are very important in comprehensive quality improvement/assurance and how quality assurance helps fulfill an organization’s mandate. Quality of care would suffer if one or more of the components were deficient in design or function.

17

Session 4: Standards of Services and Operations A. Objectives

• To understand the basic role that service standards can play in quality of care • To review service standards to develop or update tools for quality assessment

B. Guide: Topic Format Timing Review objectives Plenary presentation 5 min Definition and importance of performance standards

Plenary, participatory discussion 10min

Developing standards (approaches and process)

Plenary, participatory discussions 10min

Performance standards (examples) Plenary, participatory discussions 15 min

Levels and scope Plenary, participatory discussions 10 min Review standards Plenary, participatory discussions 10 min Fill out daily evaluation form 30 min Plenary Total: 1 hour 30 min C. Preparation and Supplies • LCD projector • PowerPoint slides 4.2–4.28 • Flip chart and markers • Organization’s performance standards, policies or procedures, service delivery standards or

management guidelines for HIV and AIDS programs

D. Facilitators’ Notes for Session 4 1. Review objectives

Start the session with slide 4.2 and the two complementary objectives: why performance standards are essential for quality of care and quality improvement and the importance of regular reviews and updates of performance standards to meet changing needs and quality of health care. Give a brief overview of the presentation and the many terms related to or interchangeably used with the term performance standards listed on slide 4.4. Tell participants that they will actually take time to review performance standards for various programs and management areas of their organizations. 2. Definition and importance of performance standards Using slide 4.5, define performance standards. Link this discussion with the definition and description of performance standards in the preceding session. Ask participants why standards are important. Using slide 4.6, further discuss how standards create shared understanding among implementers and managers and how they form the basis for measuring and improving performance and quality of services. Ask whether the statement on slide 4.6 makes sense

18

to them. Encourage them to share real life experiences with how the presence or absence of performance standards in their organizations has helped or affected their performance or quality of services. 3. Developing performance standards Ask participants how performance standards are developed and whether any of them has participated in setting standards. Note their responses on the flip chart. Use Slide 4.8: Standards are developed by technical experts based on contemporary and accepted knowledge and evidence in the field; therefore the term “evidence-based standards” is used. Technical experts are usually organized into technical working groups under national or international standards or regulatory bodies. Many standards are available free of charge, and organizations usually access them through literature searches or by contacting the organizations that set them. An organization has the option to develop new standards or to adapt or adopt existing ones. Further describe and characterize standards by using slides 4.9 to 4.11. Generally, standards should be simple, clear, credible, measurable and achievable. Using slides 4.12 to 4.19, give examples of performance standards for different programs areas: prevention, HBC and managing operations. 4. Levels and scopes Using slide 4.20, ask for participants’ experience on how standards and performance expectations vary by organization or program or depending on the hierarchy within an organization. For example, national performance standards and expectations are different from regional, district or community level standards. Using slide 4.21, explain that standards vary in scope and specificity. Simply put, standards at higher levels have a broad scope and less specific descriptions while locally they have narrow scopes and specific descriptions. The same is true within an organization: broader at organizational or departmental level and specific or detailed at team/individual employee level. Therefore, it is very important to understand the intended purpose or audiences for standards and guidelines. Further discuss this by using slides 4.22 and 4.23. Ask participants to apply the discussion to their situations. Take a few minutes to think through their programs/operations or departments/teams within the organization and if they operate nationally/regionally or in communities. Do they have standards relevant to each level or context? (Note: this leads into the next discussion on reviewing standards) 5. Reviewing performance standards Without losing the momentum from the previous discussion, start by asking participants to share what they think about their organizations with regard to availability of performance standards and how relevant they think their standards are. Further explore participants’ understanding by asking why it is important to review performance standards (slide 4.24). Emphasize the fact that standards are useful only they are relevant at various levels of an organization and if they are up to date. As new evidence appears, practices should be aligned with them so standards should change. It is very important for an organization to have regular schedules and systems (process and clear steps) to keep their standards up to date! In support of this

19

discussion, go through the contents of slides 4.25 and 4.26. Slide 4.27 has key roles and inputs from various service delivery levels. Conclude this session by dividing participants into working groups to review their performance standards by applying the knowledge they gained from discussions so far. (Note: participants should come with their organization’s standards.) 6. Daily evaluation Tell participants to fill out the evaluation forms in their folders; a summary will be presented tomorrow. Ask for two volunteers to recap Day 1 at the beginning of Day 2.

DAY 2 Review, Understand and Update Standards for Quality Improvement

A. Objectives • To create a common understanding about the importance of periodically reviewing and updating

standards to keep them relevant to the program and/or the organization’s operations • To create an opportunity for the participants to apply the knowledge and skills they learned in

Session 4 on reviewing the standards and tools for measuring and improving quality of programs • To encourage organizations to revise and customize their technical and operational standards • To assist participants to develop, review and refine tools for measuring the quality of health services

B. Guide Topic Format Timing Recap Day 1 Review objectives and expected outputs

Plenary Plenary presentation

30 min 5 min

Introduce the topics Plenary presentation 20 min Technical standards #1 Group work and presentation 120 min Technical standards #2 Group work and presentation 120 min

Operational standards Tools

Group work and presentation Group work and presentation

60 min 60 min

Daily evaluation Individual participants 30 min Total: 7 hours 30

min C. Preparation and Supplies

• Participants come with their programs’ technical and operational standards and tools • Copies of the standards and tools for each group • LCD projector • Flip chart and markers

D. Facilitators’ Notes for Day 2

20

1. Review objectives Start by sharing the objectives for the day. Tell the participants that the day is dedicated to addressing standards of services and that they will review, understand and update their technical and operational standards and review and update the tools to measure performance/quality against those standards. Briefly explain the four objectives and tell them the expected outcomes are revised and updated standards and tools and improved knowledge, understanding and skills that will lead to compliance.

2. Introduce the topics There will be four interdependent presentations: two on technical standards, one on operational standards and another on the tools for measuring and improving quality. HIV and AIDS Technical Standards: The technical standards to be reviewed will vary according to the participants’ programs. For example, NPI programs implemented HIV and AIDS prevention, OVC care and support, community and home-based care (HBC) and HIV counseling and testing (HCT); however, a similar approach can be applied to review, understand and update standards in other program areas Operational Standards: 1 An organization has diverse management practices and policies and procedures; the focus of this training workshop is to review, understand, update and integrate key management performance standards to successfully implement and manage programs. The aim is not to review all the operational policies and procedures of the participants’ organizations but rather to give insights and practical approaches for understanding operational/management standards.

Broadly, the focus is on:

• Administrative policies and procedures such as travel, IT, Information management and communication protocols;

• Financial management including policies and procedure, best practices and compliance issues related to the program(s) and roles of non-finance staff in financial management of the programs;

• Methods and responsibilities of teams procuring and managing supplies (buffer, lead time, etc.); • Human resource management including recruitment and placement, orientation, training and

building skills and performance planning and management. You may also discuss strategic relationships that the organization may attain through specific program/project (s) and protocols for documenting and disseminating information to key stakeholders.

3. Review technical standards #1 • Divide participants into groups by organization and make sure that the groups work from the

correct copy of their program performance standards or guidelines: OVC, HBC, Prevention, HCT or other HIV program standards.

• In the group, ask them to answer the following general questions or give descriptions of what they will be reviewing:

• Briefly describe the level of the particular standard (global such as WHO/UNAIDS/PEPFAR or national/regional/district or community

1 Interchangeably used with standard operating procedures/guidelines or management performance standards

21

• Describe the technical scope of the standards, i.e., thematic areas or what the standards cover

• Describe the minimum essential services2 recommended by the standard (e.g. OVC care and support per PEPFAR guidelines) What are the minimum essential services the organization is providing (directly or through its partners)?

• More specifically, focus on the minimum essential services that the organization or its partners are implementing. Describe and analyze performance expectations.

• What are the minimum activities or statements of expectation or illustrative activities under each essential minimum service that your organization is implementing?

• Note any specific roles, strategies and performance targets in the standards you are addressing.

• Discuss the partnership strategy in place to bridge the gap between what your organization provides and what the standard recommends.

• Compile recommendations on how to improve the standards and implement changes. The recommendations will be shared with the management team in the organization to improve the contents and applicability of the standards.

• Prepare a presentation for the other participants. Focus on the key areas of your standards, lessons learned and steps forward.

• Summary: complement the presentations and discussions • Encourage participants to work on and finalize their standards in their groups especially if they

don’t have their own or they are in draft form.

4. Review technical standards #2 • Still in the same groups, ask participants to choose one other technical standard to review • Set the group to work on it. • Tell them to use the steps they used on the first standard. • Encourage them to briefly share their insights in plenary.

5. Review operational standards • In the same groups, ask participants to choose an operational guideline to review. • Tell them to use the steps they used on the first two standards. • Encourage them to briefly share their insights in plenary.

6. Review tools

• In the same groups, ask participants to choose a tool to review. • Tell them to use the steps they used previously. • Encourage them to briefly share their insights in plenary.

7. Daily evaluation Tell participants to fill out the evaluation forms in their folders; a summary will be presented tomorrow. Ask for two volunteers to recap Day 2 at the beginning of Session 5.

2One or more components of comprehensive services such as the seven service components of OVC care and support in PEPFAR guidelines.

22

23

DAY 3 Measuring Quality

Session 5: Measuring Quality A. Objective

• To introduce the concept of measuring quality • To discuss the importance of measuring quality • To discuss how to measure quality and what to measure

B. Guide

Topic Format Timing Recap Day 2 learning, feedback on evaluation Review objectives

Plenary presentation Plenary presentation

30 min 5 min

Measuring quality: defined Plenary, group work, discussions 10min Why measure? Plenary, group work, discussions 10min What to measure Plenary, group work, discussions 30 min How to measure Plenary, group work, discussions 35 min Total: 2 hours

C. Preparation and Supplies

• LCD projector • PowerPoint slides 5.1–5.32 • Flip chart and markers

D. Facilitators’ Notes for Session 5 1. Review objectives Start with Slide 5.2. Tell participants that this session concentrates on measuring—measuring quality, why measuring quality is important, what to measure and how to measure quality against the agreed standards or clients’ expectations. 2. Measuring quality defined Slide 5.3 indicates how quality is viewed and is difficult to measure and addresses any lingering thoughts by asking participants to agree or disagree with the statements and/or what they think about measuring quality. Tell participants that measuring quality is a complex subject, that it is important to design the framework as to what to measure and how to measure it carefully. This discussion leads into slide 5.4 defining measuring quality. 3. Why measuring quality is Important

24

Define measuring quality of care using slide 5.4. Measuring quality is the act of or attempt to quantify the current level of performance against expected standards. It is the process of assessing whether health or social services are consistently meeting service standards. Slides 5.5 to 5.10 further elaborate on measuring quality and justify why measuring performance or quality is so important. 4. What to measure Slides 5.11 to 5.21 help to facilitate in-depth discussions on what to measure in quality of care. Essentially, measuring quality means measuring the inputs, process and results (output, outcome and impacts). The slides go step-by-step through the components stated earlier. Exercises for self-reflection, buzz-group discussions and group work with presentations are meant to enhance participation and better understanding of the topic. 5. How to measure quality This is the last section of this session and emphasizes the importance of (i) standards and targets; (ii) indicators; (iii) tools such as checklists, interview guides or record review guides and (iv) clear processes or techniques for measuring performance and quality of care. These four elements are covered in detail in slides 5.22 to 5.37 Use slides 5.22 and 5.23 to form a basis for measuring quality of services by relating it to standards, indicators and checklists:

• Standards: Quality cannot be measured without a clear definition or standard. • Indicators: Show or indicate whether the standards are met or not; show levels of achievement

against standards; show what, when, how many/much and are derived from the performance standards

• Targets: Quantified or numerical levels of achievement against standards • Checklists : Documents listing various indicators organized for measuring

Slide 5.23 graphically presents the relationship between performance standards, quality indicators, tools and measurement. Further elaborate the relationship by using an example on standards, indicators and checklists (tools) from HIV prevention services. Tell participants many more examples can be drawn quickly by using similar approaches. Using slide 5.24, elaborate on indicators as key inputs for measuring quality. Without indicators, one cannot be sure how good or bad the quality of care is. Good indicators need to be sensitive to pick out desired/unwanted changes in performance. Using slides 5.26 and 5.27 qualify indicators and give examples of good ones. Use slide 5.28 to give further details on targets. Define targets and give examples by using slide 5.29. Go to the group exercise on slide 5.30. Tell the participants that they will have time to apply the concepts of indicators and targets to their organizations. Put 4–5 participants in groups according to their organizations. Ask them to reflect on their quality/performance standards and develop indicators for the area they choose. Tell them that they will share their work with the other groups in 15 minutes. Assign facilitators to sit with each group or move around to assist participants. In plenary, encourage comments that will help each group refine their indicators.

25

Building on Session 2 (slide 2.18), refresh their memories on QI/QA. Tell them that there are many ways to improve the quality of health care and of HIV and AIDS services. Broadly, this improvement comes from internal teams or external authorities (or technical experts). The teams or external experts may apply various means to improve quality. Slide 5.31 introduces various approaches to QA/QI: assessment, monitoring and external evaluation. Elaborate on them.

• Assessment: Measuring the quality of care at a certain point. A quality assessment describes the current level of performance with the objective of improving it and is often an initial step in QA that may include providing feedback to health workers on performance, training and motivating staff to undertake improvements, and designing solutions to bridge quality gaps.

• Monitoring: A process for regularly collecting and analyzing data on a set of indicators, using systematic methods to monitor the quality of healthcare, and emphasizing measurement and analysis and not individuals

• External Evaluation: People outside of the organization measure quality to validate what has been measured internally.

Use slides 5.32 to 5.36 to discuss specific techniques for measuring quality such as interviews, focus group discussions, record reviews and observations and feedback for service providers. One or a combination of these techniques can be applied in quality improvement. These slides discuss each technique, its advantages and its practical short comings. As none of the techniques is complete or perfect on its own, it is good to try combinations in varieties of situations. Slide 5.37 gives important tips for measuring and monitoring quality in a learning organization. These tips include the importance of plotting data overtime (information presentation), using existing data sets without waiting for “perfect information,” sampling and pilot testing before scaling up an improvement plan, integrating quality measurement (and monitoring) into daily routines and the complementary effect of numerical data and storytelling in quality improvement.

26

Session 6: Supportive Supervision A. Objectives

• To define and characterize supportive supervision; explain steps and techniques and give guidelines on how to provide constructive feedback

• To provide opportunities for practical supportive supervision through field visits or an analysis of a case study and encourage its application in quality improvement

B. Guide Topic Format Timing Review objectives Plenary presentation 5 min Define supportive supervision Plenary discussion 20 min Features of supportive supervision Plenary discussion 15 min What makes supervision effective? Plenary discussions 20 min Key steps in supportive supervision Plenary discussions 20 min Case study analysis Daily evaluation form

Plenary discussions, group work and presentations Plenary

3 hours 10 min 30 min Total: 5 hours

C. Preparation and Supplies

• LCD projector • PowerPoint slides 6.1- 6.13 • Flip chart and markers • Supportive supervision case study (Annex III Part B) • Draft integrated checklist (Annex III Part C)

D. Facilitators’ Notes for Session 6 1. Review objectives Start the session with slide 6.2. The objective is to create a shared understanding about supportive supervision and to provide practical experience through case studies and field visits. Give a brief introduction as to how the session will unfold. Using slide 6.3, tell the participants that the supportive supervision component covers about one day. It may take one of three approaches after a brief session guided by a PowerPoint presentation:

1. An analysis of a three-part case study 2. Practical supervision in the field for selected organizations 3. Role plays showing the various components

Tell the participants that this workshop uses the first of the three options. 3. Define supportive supervision Explain to participants that one of the most common problems with supervision is that everyone understands or interprets it differently. Because participants in this workshop have different lengths of

27

service, educational and/or career backgrounds, they might have their own definitions or interpretations of supportive supervision, so it is important to have a shared understanding of what it means. Ask participants to give their own definitions of supportive supervision and note their responses on the flip chart. Using slide 6.4, show participants one of the definitions of supportive supervision widely used. Supportive supervision is, “The process of guiding, helping and encouraging staff to improve their performance so that they meet the defined standards of performance of their employer.” The important features in this definition that need to be pointed out are the following.

1. Supervision is a process. It is not a one-time event, but is rather a connected series of events over a period of time.

2. Supportive supervision involves guiding, helping and encouraging staff. It recognises that the only way to improve staff performance over the long term is to promote in them the motivation to perform well and to give them guidance and help they need to do so.

3. Supportive supervision involves guiding, helping and encouraging staff to improve their performance. This recognizes that there is only one person who can improve performance: the individual himself or herself. The staff member has to want to do well and has to recognize that it is important that she/he performs well. Furthermore, improving performance is usually not a one-time event. It may take time for staff members to perform well in all aspects of their jobs, but they can do it little by little.

4. Supervision involves helping staff to improve their performance so that it meets the standards of their employer. This is a crucial factor. If there is no clear statement of what people are meant to do (job description) and how they are meant to do it (expected standard of performance), then how are they to know what they should be doing and how is the supervisor able to assess whether they are doing the right things in the right way?

3. Features of supportive supervision Using slides 6.5 and 6.6, discuss the key features of supportive supervision in contrast to traditional supervision or inspections in the health sector. Emphasize the fact that supportive supervision is a participatory assessment of performance and problem solving and analysis that motivates staff. By its nature, supportive supervision focuses on sharing responsibilities to attain the organization’s mission and encourages two-way communication and joint problem solving. 4. What makes supervision effective? Show slide 6.7 and note their responses on the flip chart. Emphasize the fact that there are many factors in management, systems, processes and tools, knowledge, skills and experience that affect supervision as summarized on slide 7.8. Go through each of the bullet points to reflect on some of the key factors such as management commitment, standards, processes and tools, attitude and practices of service providers and supervisors. 5. Key steps

28

Effective supportive supervision involves several steps each with many different components. For example, performance standards form the basis for any supervisory system and developing effective, evidence-based standards requires knowledge, skills, time and commitment. For the sake of this workshop, supportive supervision is divided into five steps:

1. Developing the system (including an annual plan) 2. Preparation by supervisory teams (supervisors and services providers) 3. Conducting supervisory visits 4. Concluding the visits 5. Following up after each visit

As stated above, each of the steps is composed of several complementary components. Use slides 7.9–12 to discuss and summarize each step. 6. Case study analysis Supportive supervision combines several management approaches and techniques for measuring quality. Full supportive supervision training usually takes 4–5 days. A combination of approaches such as role plays and field visits to practice supervision have proved to be very effective. This three-part case study was developed to provide the knowledge and skills for effective supportive supervision. The story is drawn from the practical experience of many organizations. The case study shortens training to one day, but the knowledge and skills can be effectively transferred. (Annex III Part B):

• Part One: Service delivery arrangements and system for supportive supervision • Part Two: Conducting supportive supervision • Part Three: Concluding supervision and developing a framework for follow-up

Divide participants into three or four groups of equal sizes and assign a table to each group. Ask each group to read the first part of the case study then discuss and take notes of key steps or processes or good practices. Ask the groups to write their answers on the flip chart and post them on the wall. The participants then make a gallery walk in a clockwise direction to review each group’s work (5 minutes/group). In plenary, highlight both the good practices they learned and how they will improve their existing systems. Repeat this process for parts 2 and 3. a. Part one learning objective:

• By reviewing the case study participants will learn the various components essential for supportive supervision systems

b. Part two learning objective: • By reviewing the case study, participants will learn how to conduct effective supervision and

provide constructive feedback

c. Part 3 learning objective: • By the end of this session, participants will learn how to develop and implement a follow-up

action plan and how to share information or findings from the visit for effective program management

29

7. Daily evaluation Tell participants to fill out the evaluation forms in their folders; a summary will be presented tomorrow.

Ask for two volunteers to recap Day 3 at the beginning of Session 7.DAY 4 Measuring and Improving Quality

Session 7: Information Management A. Objectives

• To define key terms and concepts and to, discuss data collection, analysis and action planning in the course of quality improvement

• To discuss sources of information and the importance of using existing information in quality improvement and to explore feasible approaches for routinely integrating quality indicators and targets for quality of care into the monitoring and evaluation system

B. Guide Topic Format Timing Recap Day 3 learning, feedback on evaluations Review objectives

Plenary presentation Plenary

30 min 5 min

Introduce information management Plenary discussion 30 min Definition and key components of information management

Plenary discussions, group work and presentation

30 min

Linking QI to the M&E system Plenary, group work, discussions

25 min

Total: 2 hours C. Preparation and Supplies

• LCD projector • PowerPoint slides 7.1–7.21 • Flip chart and markers

Facilitators’ Notes for Session 7 1. Review objectives Start the session with slide 7.2. Tell participants that the session will define information management, processes and tools for data collection and analysis and data-based improvement in quality of care. Using slide 7.3, emphasize that QI is data driven. Data are important for comparing performance and achievements against performance/quality standards, for identifying and analyzing gaps in performance as well as for identifying appropriate solutions to close the gaps. 2. Introduce information management In plenary go through the terms and definitions on slide 7.4: data, information, management and systems. This slide helps as an ice-breaker before group work. Ask participants to familiarize themselves with these terms.

30

Divide participants into groups of 6–7. Display slide 7.5.

Scenario (Slide 7.5) You are the program manager in a far-flung district in your region of 1 million people. There are four health facilities in the district. A recent client satisfaction survey showed that CD4 count results for routine follow-up in the ART clinic was turned in every 3–4 hours. This is a huge gap compared with the median of 1 hour in similar facilities in other districts of the region. You tried to solve the problem, but you have concerns because the problem has persisted. You are to meet with the district health management team to discuss this issue. Questions: What data will you present to the team, and how do you plan to solve this problem? (provide examples from your experience)

Facilitate group work by asking questions such as the following.

• How did you (your team) come to suspect there was a problem? • How did you know whether the problem was real or apparent? • What step(s) did your team take first? • Did you look at existing information sources or key informants? • Did you find the information you were looking for? • How can that affect your data collection? • If you don’t have data, how can you collect data to verify the problem exists? • Data can be collected routinely through reports or actively through supervision and field visits.

Assuming you have collected data, what will you do with it? How can you process, store and retrieve the information you need? How can you make sense out of your data?

Ask groups to present their work. Allow a brief discussion and comments. Summarize the responses by

asking: • How did you collect information? • What did you do with it? • Who else was involved in collecting and using this information? • Can you find the same information if you look for it for the next visit?

3. Definition and key components Using slide 7.7, define information management. A common definition is: information management is the process and practices of planning for, collecting, processing, analyzing, and storing, retrieving and using information for making decisions. Explain that information management can be described as a cycle composed of four major components (slide 7.8): 1. Planning for collecting data; 2. Collecting data; 3. Processing and analyzing data and 4. Using data for action (decision making). Discuss each component in greater detail guided by slides 7.9–7.19. 4. Linking QI to the M&E System By this point, participants have learned that QI is evidence based and that current information is required for ongoing quality improvement. Every organization has a monitoring and evaluation system,

31

and creating a separate information system for QI is usually costly and not effective. Using slide 7.20, highlight the importance of integrating QI data needs into the organization’s M&E system. Slide 7.21 helps to brainstorm what information is required for QI but is not collected by routine M&E. Ask participants how they are going to strengthen their existing systems: what additional indicators, data collection and reporting systems are required to cater for quality improvement?

32

Session 8: Concepts and Principles of Quality Improvement A. Objectives

• To discuss the concept and principles of quality improvement • To introduce the four basic steps in quality improvement

B. Guide Topic Format Timing Review objectives Plenary presentation 5 min Rationale for quality improvement Plenary discussion 10 min Define QI Plenary discussion 10 min Benefits of QI Plenary discussion 20 min Principles of QI Plenary discussion 10min QI methods and approaches Plenary discussion 20 min Steps in quality improvement Plenary discussion 15 min Total: 1hr 30min C. Preparation and Supplies

• LCD projector • PowerPoint slides 8.1- 8.24 • Flip chart and markers

D. Facilitators’ Notes for Session 8 1. Review objectives

Start the session with slide 8.2. Tell participants that this session is meant to lay the foundation for the quality improvement component of this workshop by providing concepts, rationale and key steps. 2. Rationale for quality improvement Use slide 8.3 with the quotation from the Institute of Medicine (IOM) to start a discussion on why they think it is important to talk about quality improvement. As IOM states, there is a chasm between the quality of health we have and the quality we need. A chasm is a divide, a gulf, a gorge, a rift or a break. This term tells us the magnitude of gap. The IOM is a US organization in a resource-rich setting, but there is a huge divide between the quality of health care they have and the quality they need. Ask participants, if they were to grade quality of service in their health care where they would put it on a scale of 1–100 and why? Note the responses on the flip chart. The rationale for QI and/or continuous improvement is to bridge the gap between the existing quality of services and what it should be. Using slides 8.4 and 8.5, tell participants that this can be achieved by changing systems of care to meet the needs and demands of clients. It is important to note that not all changes are improvements and the importance of making planned and effective changes to achieve improvements. 3. Define quality improvement

33

Slide 8.6 gives a comprehensive definition of quality improvement: Quality improvement is the complete process of identifying, describing and analyzing strengths and problems and then testing, implementing, learning from and revising solutions. Ask participants to take a moment to reflect on this definition and give their inputs. Impart the importance of the continuous nature of quality improvement. Slide 8.7 helps to make this point clear: “Even if you’re on the right track, you’ll get run over if you just sit there!” (William Rogers, 1879-1939)3 4. Benefits of QI Quality improvement has several benefits for staff, managers and clients. Using slides 8.8 and 8.9, discuss the benefits for staff (implementers) and managers as well as the benefits for clients. 5. Principles of QI With the guidance of slides 8.10–8.15, share the four key principles or critical issues for QI. They are: (i) focusing on clients’ needs, (ii) understanding work in terms of processes and systems, (iii) solutions by teams of health care providers and (iv) testing and measuring to bring about desired changes. Each of these principles is described in detail on the slides, and notes are also available on the PowerPoint format. In addition, the following points summarize the gist of each. • Client focus requires designing and delivering services that meet the needs and expectations of =

clients.

• Processes and systems recognize the importance of decision making and the flow of information in health-care delivery systems. The system view tries to understand how each component behaves as well as how different components influence each other. It is imperative to understand how the entire system works and not just to master the system’s components.

• Testing and data use recognize QI as data-based decision making where hypotheses test for the

underlying causes of problems and their solutions. QI is a scientific method of problem solving using data and accurate information. Data collection and analysis are the backbones of the system.

• Team approach is a collaborative effort where everyone in the organization contributes. With the

system view in QI, the performance or underperformance of a team or a team member affects the organization’s performance.

6. QI methods and approaches Ask participants to share their knowledge and experience about what works better or what actions/methods can improve the quality of care or HIV and AIDS services (slide 8.16). On the flip chart, write all responses. Share several methods of QI including integrated supportive supervision similar to what they did yesterday. Many other actions and interventions have improved performance and the quality of services such as training providers, training and capacity building for program managers who

3 William Rogers (1879-1939). Source: http://www.leadershipnow.com/initiativequotes.html

34

in turn support service providers and periodic and planned performance reviews. Performance improvement also emphasizes the availability of equipment and facilities, clear job expectations or job aids, training and capacity building and feedback and support from managers and supervisors. Certification, licensing and accreditation are methods for QA (by external assessors) while most other interventions are internal and can be done by managers, supervisors and peers. Using slides 8.17–8.21, introduce approaches to QI, i.e., how QI methods are specifically applied in various situations. There are four QI approaches: individual-based, rapid team (ad hoc team), systematic team and process improvement. These approaches vary depending on the complexity of QI issues to be identified and addressed. For example, responsible individuals accomplish their tasks on a daily basis and routinely solve problems within their knowledge, skills and responsibilities while a rapid team/ad hoc team is formed to investigate non-recurrent problems that are too complex for individuals. 8. Team approach to QI As stated above, there are four basic approaches to QI depending on the complexity and interdependence of the problems identified. The team-based approach is the most common and is very important for sustainable and continuous quality improvement. Using slides 8.22–8.28, systematically address the team-based approach. Make the session interactive by asking open-ended questions to help participants reflect on their experiences. 9. Steps in QI Using slide 8.22, discuss the four steps in QI: identify; analyze; develop (solutions) and test and implement (solutions). These four steps are components of all quality improvement efforts small or large, simple or complex. In the following session, these steps will be addressed in detail with examples and exercises.

35

Session 9: Quality Improvement Model A. Objectives

• To understand the four steps in QI and develop the skills to apply the QI model (Plan-Do-Study-Act) in service delivery

• To strengthen participants’ skills in QI through hands on practice of PDSA using examples from their programs

B. Guide Topic Format Timing Review objectives Plenary presentation 5 min QI model Plenary discussion 25 min Components of QI model Plenary discussion, group work and

presentations 240 min

Collaboration Plenary discussion 35 min Daily evaluation form Plenary 30 min Total: 3 hours 35

min C. Preparation and Supplies

• LCD projector • PowerPoint slides 9.1- 9.34 • Flip chart and markers

D. Facilitators’ Notes for Session 9 1. Review Session Objective

Start the session with slide 9. 2. Tell participants that this session is devoted to the QI model of Plan-Do- Study-Act (PDSA), and a step-by-step discussion of each of its components. The session will be highly interactive and practical with many exercises and questions for reflection and experience sharing to deepen learning. 2. Quality improvement model Using slide 9.3, start the session by refreshing participants’ memories about the four key steps in quality improvement: determine what to improve (identify), understand the problem (analyze), hypothesize solutions (develop) and test and implement those solutions. Remind them that these four steps are basic to all improvement activities. The fourth and last step of testing and implementing goes through four interdependent and iterative processes: planning (Plan), implementing the plan (Do), checking whether it is working or not (Study) and intervening depending on whether it is working or not (Act). PDSA is thus the basic model for improvement. Slide 9.4 links the four steps into the PDSA cycle. Slides 9.5 and 9.6 further describe and analyze the PDSA cycle and shed more light on its components. 3. Components of the QI model

36

The PDSA model has four interdependent components summarized as follows (slides 9.7–9.14). Slides 9.7–9.14

• Plan involves identifying, and analyzing problems using scientific methods, team work and innovative solutions. It involves both identifying the correct problems and the solutions for them as well as planning how to implement the solutions. Plan asks questions about who does what, how, when, where, at what cost, etc.

• Do is essentially implementing the plan.

• Study is examining the effect of solutions on the problems. Is the situation improving due to the intervention (s), getting worse or remaining the same?

• Act is proceeding according to the findings from the study. In general terms, adapt, adopt or abandon the intervention based on the evidence.

Important to Note: 1. Slide 9.6 contains all key points on each component.

2. The session is organized such a way that each component is introduced and discussed and then a

practical exercise allows participants to apply the lesson to an example from their own programs. It is imperative that facilitators give time for these exercises. (Annex III Part D and slides 9.15–9.18)

4. QI Collaborative Using slides 9.22–9.32, discuss quality improvement as a multi-team and/or multi-center approach in which many improvement interventions are tested, implemented and refined concurrently by using repeated PDSA cycles. Lessons are shared through learning sessions. Collaboration helps to rapidly test and implement successful QI interventions. 5. Daily evaluation Tell participants to fill out the evaluation forms in their folders; a summary will be presented tomorrow. Ask for two volunteers to recap Day 4 at the beginning of Session 10.

37

Session 10: Institutionalizing Quality Improvement A. Objectives

• To discuss the concept and practice of institutionalizing quality improvement • To create a shared understanding about leadership commitment, structure and systems in

sustaining quality improvement

B. Guide Topic Format Timing Recap Day 4 learning, feedback on evaluations Review objectives

Plenary presentation Plenary presentation

30 min

5 min Understand institutionalizing QI Plenary discussion 10 min What it takes to institutionalize Plenary discussion 20 min Participants’ reflections Plenary discussion 30 min Guidance for the rest of the session Analyzing and planning for QI in each organization Daily evaluation

Plenary Discussion Group work and plenary presentation Plenary

15min 5 hours 30 min

Total: 7 hours 20 min C. Preparation and Supplies

• LCD projector • PowerPoint slides 10.1–10.8 • Flip chart and markers

D. Facilitators’ Notes for Session 10 1. Review objectives Start the session with slide 10.2. Tell participants that this session is meant to create a shared understanding about sustaining QI by institutionalizing it. Tell them that this will be followed by a day-long guided analysis and planning for QI in their respective organizations. 2. Understanding institutionalizing QI In plenary ask participants how they define the term institutionalizing (slide 10.3). Note responses on the flip chart. Simply defined, institutionalization is making something an established custom or an accepted part of the structure of an organization as opposed to something practiced by individuals or groups that would cease if those individuals left the institution. Institutionalization ensures that good practices continue. The term can apply to many practices such as organization-to-organization relationships and collaboration, QI/QA activities or other practices. Slides 10.4 and 10.5 give a definition and features of institutionalizing QI. It is a systematic approach of establishing a culture of quality within the organization and makes QI an integral, sustainable part of the organization’s performance 3. What it takes to Institutionalize

38

In plenary, ask participants what facilitates institutionalizing good practices in their organizations. Note their responses. Using slide 10.6, summarize key factors: internal environment, organization’s structure and support structures. As stated, these include the commitment of management to the cause (quality health care), systems and process; facilitative policies, procedures and guidelines and human, financial and material resources. 4. Participants’ reflections This is meant to deepen understanding by reflecting on the concept and success factors for institutionalizing QI in their organizations. Using slide 10.7, ask participants to think about their individual responsibilities in their respective organizations and what and how they can contribute to institutionalizing QI. Note their responses and comments. 5. Planning quality improvement As stated above, Day 5 is the last day of training and is dedicated to identifying and analyzing QI gaps in their respective organizations and to developing, testing and implementing interventions. Distribute the following guide to all participants. Day 5 Apply the QI Improvement Model to Your Organization/Program

Objective • To apply the PDSA model to improve your organization’s or program’s performance

Instruction: Please strictly follow all the steps. Document everything as your group will make a presentation in plenary.

Guidance

I. Identify Problems (9:30-10:30)

a) Review the programs or services your project provides. Also review service delivery arrangements.

a. What servicers does your project provide? Is there any other organization that provides services that complement them?

b. Identify the roles and responsibilities of stakeholders involved in setting standards for service delivery and management.

b) What are major problems that have affected the quality of the services you provide? a. List key problems that affect the quality of your HIV and AIDS services b. In identifying these challenges, think about or apply parameters such as dimensions and

perspectives of quality; input, process or results or combinations of these factors on your program.

c. You can use examples or evidence from your M&E system or a recent supervisory visit. c) Select three major problems you would like to address in your project right after this workshop. d) Rank them in order of priority and then choose one of them.

II. Analyze (10:45-11:30) Analyze the o chosen problem using one of the following methods.

39

• System modeling (input-process-output), • Flow charts • Cause-effect analysis (fish bone or 5 whys)

40

III. Develop interventions (11:30-13:00) a) Design an alternative course of action and develop the interventions for the problem you have

identified. b) Key considerations

a. Process, approaches and techniques to monitor progress b. Your information sources and how you will access, use and manage information c. Key quality indicators you would like to integrate into your M&E system in order to track

progress c) How will you institutionalize and sustain improvement?

IV. Test and implement (14:00- 15:30)

a) Develop a detailed plan for implementing your interventions using the PDSA model, but for practicality, stop your written plan at the P-stage. You can consider all risks and assumptions such as resistance, resource shortages, inherent failures, etc. at this stage

b) REMEMBER: you will continue with the next three steps (DSA) at your organization or work site c) Present your work to the group in a plenary session.

6. Daily evaluation Tell participants to fill out the evaluation forms in their folders.

41

Annex I: PowerPoint Slides Session 1: Introduction, Objectives and Scope of the Workshop Slide 1.2. Overall Objective and Scope To strengthen the capacity of NPI grantees in assessing and continuously improving the quality of HIV and AIDS prevention, care and support services and of key management areas Slide 1.3. Specific Objectives

• To define concepts of quality, continuous improvement and a framework for quality management

• To familiarize participants with service standards and processes and steps in setting standards to assist developing and/or revising quality standards and tools to measure the quality of HIV and AIDS services

• To build the capacity of NPI organizations (and their partners, where applicable) to coordinate the quality of services implemented by partners

• To create momentum for quality improvement with a better understanding of the principles and practices involved

Slide 1.4. Training Contents

Defining quality • Quality, improvement concepts, framework and outcomes of quality of care

• Content, dimensions and perspective of quality of care • Service standards • Reviewing, revising and updating service standards

Measuring quality • Principles of measuring and monitoring quality • Tools and techniques • Information management, problem identification • data-based decision making

Improving quality Concept, principles and approaches • Improvement model(Plan-Do-Study-Act)

Sustaining quality improvement

• Translating continuous quality improvement knowledge into practice, • institutionalizing and sustaining quality improvement

42

Session 2: The Basics of Quality Slide 2.2. Objectives

• To understand the concept and properties of quality as a measure of the contents, dimensions and perspectives of care

• To introduce the concept of quality improvement, its principles and framework Slide 2.3: What is Quality?

• a degree/grade of excellence or worth (American Heritage College Dictionary © 1993) • the degree to which social services meet or exceed the needs and expectations of the intended

users/clients

Slide 2.4.Quality in Daily Life

• Assume you have a favorite restaurant where you always have your best traditional food. You have visited that restaurant for the last five years. You always feel confident when taking your family, friends and highly valued guests there. You have a favorite dish. You do not hesitate to order the dish of your choice in that particular restaurant. Your family, friends and guests affirm your choice of the restaurant to be the best!

• What makes this restaurant that favorable? Slide2.5. Quality in Daily Life

• Some of participants’ responses • Taste. (Think about why?) Right ingredients in the food, how the food was prepared or cooked

(technical ability of the chef) • Never get sick after eating the cuisine: safe/healthy • Attendants are friendly: hospitality, care, courtesy, communications • You get excellent food whenever you go—by yourself, with your friends, family and esteemed

guests: sustainable • No unnecessary delays: timely • The setting: scenery and facilities are nice • The price is reasonable

Slide 2.6.Quality of Care or Services

• To what extent are the above aspects applicable to health or HIV and AIDS care or services? • All or nearly all features of quality from the case scenario are applicable to HIV and AIDS

programs; however, some features are more applicable to restaurants. Appreciation of the various features may vary from person to person.

Slide 2.7 Properties of Quality

1. Content of care: types of services under consideration • How important are the services to clients seeking them

2. Dimensions of care: specific aspects or specifications

43

3. Perspectives of care: point of view or concerns or desires • e.g. think about taking an esteemed guest to your favorite restaurant

Slide 2.8. Quality: Contents of Services Examples:

1. HIV and AIDS prevention • • • •

2. Home-based care • • •

3. Managing operations • • •

Slide 2.9: 1. Content of Services

• Content of care and service delivery arrangements

a. Services provided on site or by the project: What can our organization or project offer? How

comprehensive are our services?

b. Services provided through partnerships, networking and referrals: How can the rest of the

services be accessed by beneficiaries? What barriers are there?

c. Services delivered through grants, sub-contracting , intermediary organizations and mechanisms

for capacity building • The specific contents of care and delivery arrangements should be clearly documented in standards

of services. Slide 2.10 and 2.11: 2. Dimensions of Quality

Safety Compassionate relationship

Completeness Appropriateness

Access Continuity Sustainability

Participation (in decisions)

Effectiveness Efficiency Technical performance

Adapted from Franco et al. (2003). (See Annex III Part A)

44

Slide 2.12: 3. Perspectives of Quality a. Providers

– doing the right things (evidence-based standards),

– doing things right (safe and timely, credible communication) b. Beneficiaries /families/communities

– doing what matters to the beneficiaries c. Quality from managers’ perspectives

– all the above PLUS timely inputs, competent providers, beneficiaries who are aware they can demand their rights

d. Other stakeholders

– What do they value? Slide 2.13. Quality of Care Defined

A proper performance, according to standards, of interventions that are known to be safe, affordable

and have the ability to produce desirable impacts

(Roemer and Aguilar 1988) Slide 2.14. Quality of Operations

• Ensuring that internal processes are carefully managed, measured and analyzed for quality services

• A great deal of focus is on the effectiveness and efficiency of these systems, e.g. systems for procurement, for staffing, etc.

Slide 2.15. Quality

Quality is …. – Doing the right thing,

– In the right way,

– At the right time,

– For the right person,

– With the best possible result.

Slide 2.16. Quality is…

• Achieving a predetermined standard or target

– Standards and targets should be set with clients and in consideration of their requirements

• Quantity and quality of inputs and infrastructure

• How tasks are organized and performed: managing

45

• Quality of output and outcome: how inputs and processes interact and give results

Slide 2.17. Quality of care…. behind the numbers!

• Quality of care is collection of stories: • Stories that individuals tell about how their lives were changed due to care • Stories about our activities that positively impact the lives of people who need care

• Stories that delight providers with a sense of accomplishment or success • Stories that can be converted into figures or facts

Slide 2.18. Quality Assurance and Quality Improvement

• Quality assurance: refers to external validations such as accreditations, certifications and regulatory approaches

• Quality improvement: refers to an internal, self-assessment against standards and progressively and continuously improving quality of care

Slide 2.19. Quality Assurance/Improvement Framework

Slide 2.20. Key Questions for Quality Improvement

• What are we trying to address? • mandates, content of care/standards, checklists • How will we know that a change is an improvement? • measuring quality, supportive supervision, information on quality • What changes can we make that will result in improvements?

Standard Setting or Defining

Quality

Measuring Quality

Improving Quality

Sustaining Quality

Improvement

Outcome

46

• PDSA cycle Session 3: Mandates and Quality of Care Slide 3.2. Objectives • To understand the relationship between mandates, performance standards and quality of services • To understand how quality improvement will contribute to an organization’s mandate, goal and

objectives • To apply the relationship between mandates and quality in measuring, monitoring and improving

the quality of HIV and AIDS services Slide 3.3. Mandate, Organization’s Mandate • What do you understand by the term mandate? • What is an organization’s mandate?

Slide 3.4. An Organization’s Mandate • The mandate is the overall duties and responsibilities, power and authority of an organization

granted by law. • It is what an organization is formally and informally required to do or not to do.

• Formal requirements are likely to be codified in laws, by-laws, ordinances, constitutions or articles of incorporation or other legal documents and the policies and procedures of organizations.

• Informal mandates may be embodied in the traditional and cultural norms, ethics or expectations of key stakeholders. (Bryson, 1995)

Slide 3.5. An Organization’s Mandate • List of authoritative statements regarding what an organization is allowed and/or not allowed to do • Legally recognized or lawful privileges and obligations of an organization • A cause an organization will be accountable for by law • A framework of accountability

Slide 3.6. Importance of an Organization’s Mandate • Sets the boundaries

• Knowing what the mandate is, an organization also knows its limits and boundaries. • A clear mandate therefore helps in formulating a clear mission and sets the framework for

possible actions. • Establishes the organization’s identity

• Before an organization identifies its mission and values, its formal and informal requirements must be clear.

Slide 3.7.Example of Mandates • Ministry of Health in Ethiopia

• Initiates policies and laws, ensures the enforcement of laws, regulations, and directives of the federal government with regard to health matters

• Undertakes scientific and operations research

47

• Prepares plans and budgets and upon approval implements them • Enters into contracts and international agreements in accordance with the law • Give assistance and advice, as necessary to regional/provincial executive organs on health

matters Slide 3.8. Examples of Mandates

• Examples of mandates of the participating organizations • •

Slide 3.9. Vision and Mission • What is an organization’s vision? • What is an organization’s mission? • How is the mission/vision related to an organization’s mandate? Slide 3.10. Organization’s Vision and Mission • In short, a mission is a purpose while a vision is a dream.

• An organization’s mission clarifies its purpose, i.e., what it is doing and why it is doing what it does.

• The vision clarifies how the organization wants to look and how it should act in the future to fulfill its mission. The vision communicates enthusiasm and provides the organization with something to aspire to.

• Vision and mission differ in that the vision is the cause and the mission is the effect. Slide 3.11. Vision • The vision statement articulates the ideal future of the organization and the community that it

serves. • The vision statement implies that work still needs to be accomplished, and it lends credibility and

motivation to the mission statement. Slide 3.12. Mission • The overall purpose of the organization

• Explains why the organization exists • States what the organization does now in the most general sense • Sets parameters for what the organization does not do.

• An inspirational description of what an organization would like to achieve or accomplish in the mid- or long-term future.

• Serves as a clear guide for choosing current and future courses of action Slide 3.13. Examples of Vision and Mission • Our Vision: We envision a healthy, productive and prosperous society with community members

who are self-reliant and able to determine their own destinies. (Matibabu Foundation-Kenya) • Our Mission: We are a community-based health and integrated social development organization.

We design and implement integrated health and social services to empower communities to take charge of their health, improve access to quality integrated health services and strengthen links to other essential services. (Adapted from, MF-K)

Slide 3.14. Examples of Vision, Mission

48

• The visions and missions of the participating organizations

Slide 3.15. Responsibilities • What do you understand by the term responsibilities? What are your organization’s responsibilities? Slide 3.16. Responsibilities • Key result areas of the organization’s mandate • The specific duties that emanate from the mandate • Statements of what the organization and its institutions are meant to do Slide 3.17. Examples of Responsibilities (MOH Ethiopia)

1. Expand health services in the country 2. Establish centers and undertake health and nutritional research 3. Determine standards for health services 4. Issues licenses to and supervise hospitals and health services established by foreign organizations

and investors 5. Determine the qualifications of health professionals and issue certificates of competence 6. Initiate, control and promote the study of traditional medicine

Slide 3.18. Responsibilities (examples from the participants) • List the roles and responsibilities of your organization and share in plenary (10 minutes)

• • •

49

Slide 3.19. Responsibilities • Group reflections

• To what extent are your current programs aligned with your organization’s mandate, roles and responsibilities?

• Please explain. Slide 3.20. Performance Standards • What do you understand by the term performance standards? Slide 3.21. Performance Standards • Performance standards are documented statements of expectations that tell how the organization

should fulfill its responsibilities • Specify what, when, where, why, how to meet or fulfill the responsibilities • Include both technical/health or contents of services, measurement and management • List resources and technical inputs required to accomplish tasks •

Slide 3.22. Performance Standards (Examples)

Responsibilities Performance Standards

Planning, M&E, reporting

• Every 5 years, MOH will lead the Strategic Health Sector Development Plan (HSDP) within the stated guidelines and deadlines

• Provide technical and financial support to all regions to adopt/adapt and implement

• Produce annual technical and operational plans in line with the HSDP • Organize annual review meetings, mid-term and final evaluation of

HSDP per the agreed timeline in the plan • Prepare semi-annual and annual performance reports and distribute

them to all stakeholders

Research and development

• Set priorities, develop guidelines and organize necessary resources for operations research to improve delivery and quality of health care

Many others • •

Slide 3.23. Performance Standards • Examples from participating organizations will be reviewed in subsequent sessions

50

Slide 3.24.Putting it Together

Slide 3.25. Quality

• the degree to which health care or social services meet or exceed the standards Slide 3.26. Summary

Mandates Responsibilities Standards Quality, Quality

Improvement

The authority granted

to an organization to

organize and function

Duties or functions of the

organization, its

departments and

structures

Individual roles and

responsibilities

Clear statements of

expectations, of

what is to be

performed

why

who

how

results

- Performance (process

and results) that meets

expectations or

specifications in

standards

-QI: what can be done if

performance fails to

meet expectations?

51

Session 4: Standards for Services and Operations

Session 4.2. Objectives • To understand the basic role that service standards can play in quality of care • To review service standards to develop or update tools for quality assessment

Slide 4.3. Scope of the Discussion • Objectives of the session • Terms, definitions and why standards • Developing standards • Examples of standards • Scope of standards • Service delivery arrangements • Operations management • Reviewing standards

Slide 4.4. Key Terms Standards technical or operational

Performance expectations Performance Standards

Standard operational procedures

Mandates, roles and responsibilities

Guidelines

Standard operational procedures

Job descriptions, job aids etc Many more

Slide 4.5. Definition • Statements of expectations

– What is meant to be done – How it should be done, by whom, when, where, to whom and with what resources or inputs

• Statements of intent according to which services should be provided in order to obtain the expected results

Slide 4.6. Why Standards?

If there is no clear statement of what people are meant to do and how they are meant to do it, how can

they know they are doing the right things in the right way, and how is their performance to be assessed?

(Anonymous)

Slide 4.7. Why Standards?

• What does this statement mean to you and can you relate it to your daily performance?

Slide 4.8. Developing Standards

52

• How performance standards are developed – Standards are developed by technical experts (technical working groups) based on

contemporary and accepted knowledge and evidence from the field – Standards are developed by relevant national or international bodies such as MOH, WHO,

accreditation commissions, etc. – An organization can also develop its performance standards or use standards set by other

organizations by reviewing literature sources

Slide 4.9. Criteria • Developed based upon evidence and best practices and should be appropriate to local settings • Standards are identified for each health service and management function • Define or list the contents of the cluster of services

– e.g. peer education; planning, recruitment and retention, training, supervision, management • Might consist of integrated services

– e.g. prevention and referrals for HCT for the disabled

Slide 4.10. Criteria

• To be valid, credible and able to be surveyed, the standard should – Clearly identify the compliance expected – Be specific, measurable and time bound-for self-assessment and external assessment – Be associated with quality of care provided to users – Use simple language—no jargon—in the definition

Slide 4.11. Criteria

• Standards are resource neutral, i.e., both resource rich and resource constrained settings can meet the indicator

• Standards have target audiences or intended users who expect the standards to be met • Standards need to be practical tools to guide performance

Slide 4.12. Examples of Standards

• The curriculum development team shall assess the needs of the target audience through surveys, focus groups and by informal means with the relevant representation of the group included before developing a new curriculum.

• Peer educators shall always hold information about peers and their concerns in confidence. Confidentiality is assured, except in cases where the young person is in danger or involved in illegal activities.

• Peer educators shall always provide correct and factual information to peers based on standards.

Slide 4.13. Prevention Standards

Desired outcome

• Appropriate access to

HIV education,

Critical activities

• Identify, engage and build the capacity of all stakeholders

including counselors, schools, community, FBOs, CBOs and local

53

information and

prevention services by all

persons with disabilities

government officials to address HIV inclusiveness

• Identify care givers in need of HIV prevention education

• Identify and address barriers to access prevention services for

different categories of disabled

Slide 4.14. Prevention Standards Desired outcome

• Appropriate access to HIV education, information and prevention services by all persons with disabilities

Critical activities • Improve knowledge of service providers about disability issues to

change attitudes on discrimination, exclusion, prejudice and stigma

• Ensure confidentiality for persons with disabilities by training the disabled as peer educators

Slide 4.15. Standards for HBC • Quality standards in HBC are addressed for three categories of persons:

– The ill person, – The family members – The HBC team

Slide 4.16. Standards for HBC • The community HBC teams should raise the awareness of HBC services among health workers, the

broader community and ill persons and their families

• The teams should play roles to ensure that ill persons and their families (including orphans) have access to health and social welfare regardless of their ability to pay

Slide 4.17. Standards for HBC: specifics • Educating ill persons, families, community HBC teams, health workers and the broader community

about the provision of care • Provision of basic nursing care and comfort such as cleanliness for the ill person, hygiene, adequate

beds and clothes regardless of their ability to pay • Access to psychosocial support and counseling (+VCT) for ill persons, families , HBC teams (caring

for the care givers)

Slide 4.18. Standards for Operations Management • Standards for proper financial management

– budgeting, internal controls, accounting/documentation, reporting • Standards for human resource management

– recruitment, personnel management , separation • Standards for proper office operations

– procurement, equipment maintenance

54

Slide 4.19. Finance, Human Resources and Administration Desired outcome Gaining credibility and legitimacy, ultimately building trust and confidence which will attract more donors and partners

Critical activities • Specify responsibility and authority and ensure clarity on the

organization’s values, policies, rules and regulations as well as expected results and resources

• Provide guidance and support to the staff by offering regular and timely management information, training and development

• Continuously monitor and assess responsibilities and authority

Slide 4.20. Scope of Standards • How standards vary by level in an organization or program

Slide 4.21. Hierarchy in Developing and Implementing Standards

Level Roles and Responsibilities

What is the role of national coordinators?

• Develop, adapt, revise and update standards • Resource mobilization, training and capacity building, periodic

supervision and support, advocacy What is the role of regional or district coordinators?

• Adapt standards to local contexts, resource mobilization/distribution, training and capacity building

• Partnerships, advocacy, supervision and support, referrals and networking, training for providers, supplies, budgets

What is the role of care providers

• Provide quality care

Slide 4.22. Scope of Standards

Slide 4.23. Standards and Service Delivery Arrangements

International Standards

National Standards

Org Performance

Standards

Guidelines

• International standards

• National standards (and also

regional and district standards

• An organization’s performance

standards

• Operational and service

guidelines

55

• Service delivery arrangements are who does what to make services available or accessible to beneficiaries

• Standards are performance expectations: – Who is involved? – Who does what and how? – How are beneficiaries meant to access services?

Slide 4.24. Standards and Service Delivery Arrangements

Slide 4.25.Reviewing and Updating Standards • Standards are useful only they are relevant and up to date

– Relevant to the current program and the implementers – Up to date with the current evidence and best practices in the field – Aligned with the organization’s roles and responsibilities

Slide 4.26. How to Review Standards • In an organization

– Culture of learning and development

56

– A system of surveillance on evidence and best practices – Ad hoc team

• Process – Plan and schedule reviews and updates, e.g. every 4 years – Decide on how and when the review starts and ends – Establish a review team – Involve relevant stakeholders

Slide 4.27. Steps In Reviewing Standards

1. Review service delivery arrangements including numbers or levels of national, regional, district or

community implementing stakeholders. Who is involved? How are they organized?

2. Determine the key roles and responsibilities of implementing stakeholders.

3. Develop or adapt standards and guidelines for each level to fulfill its roles and responsibilities.

Slide 4.28. Group Exercise

• Guidance: – Use laptops and soft copies of your prevention, HBC or management standards. – Identify implementing stakeholders at various levels (national, regional, district and

community or household). – Determine what roles they have in the delivery and quality of preventive/HBC services or

identify the key roles and responsibilities for the stakeholders from your service standards. – List or develop standards for each.

Session 5: Measuring Quality

Slide 5.2. Objectives • To introduce the concept of measuring quality • To discuss the importance of measuring quality • To discuss the process of measuring quality, i.e., what to measure and how to measure it

Slide 5.3. Measuring Quality • Quality of care is not a popular subject even though it is very important. • Measuring quality is not as straightforward as measuring quantitative targets, inputs, activities, or

outputs.

Slide 5.4. Measuring Quality Defined • The attempt to quantify the current level of performance against expected standards

OR • The process of assessing whether health or social services are consistently meeting standards

57

Slide 5.5. Measuring Quality • Focuses on the difference between expected and actual performance to identify opportunities for

improvement—the first step in improving quality • Answers the question: How will we know that a change is an improvement?

Slide 5.6. Measuring Stories • Measuring quality is counting the stories

– Measuring inputs and tasks performed – Measuring “how good,” “how acceptable,” “how worthy” – Measuring emotions like sorrow, shame, stigma, joy – Measuring consistency of supplies, emotions of providers and clients and support provided

to providers

Slide 5.7. Importance of Measuring Quality "The only man I know who behaves sensibly is my tailor; he takes my measurements anew each time he sees me. The rest go on with their old measurements and expect me to fit them." (George Bernard Shaw)

Slide 5.8. Participants’ Reflections • How do you interpret Shaw’s quote? • Why is measuring quality important?

Slide 5.9. Importance of Measuring Quality

Measurement is the first step to control and eventually to improvement. “If you can't measure

something, you can't understand it. If you can't understand it, you can't control it. If you can't control it,

you can't improve it." (H. James Harrington)

Slide 5.10. Importance of Measuring Quality • To identify and track progress against a set targets/indicators • To identify opportunities for improvement • To compare performance vs internal and external standards, across sites and organizations • For accountability, i.e., that we are achieving the intended results

Slide 5.11. What to Measure

Questions: Reflect on any measurements in your project/life.

• What aspects did you measure?

58

Slide 5.12. What to Measure

Inputs Process /Activities Outputs Outcomes

Your planned work Your intended results

Slide 5.13. What to Measure: Inputs • Focuses on resources, materials, supplies • What do we need to carry out activities/processes? For example,

1. for a prevention program– IEC materials, peer educators , funds 2. For management/operations; performance appraisal tools, procurement tools

• Emphasizes availability and adequacy of resources

Slide 5.14. What to Measure: Process • A step in attaining output and outcome • Working as planned, following steps, consistency of approach • Quality improvement focuses on the process • The voice of the system/process

– Tells us how the process is working? What is the system telling us at each step? • What was done and who was targeted?

Slide 5.15. Why focus on process? • Principle: Most problems are found in processes and systems, not in people • Poorly designed process = poor quality • Failed processes affect employees as well; they get frustrated

– Their workloads increase (more irrelevant work). They can be unfairly blamed. They don’t get the satisfaction of doing a good job and may lose motivation.

59

Slide 5.16. Staff Recruitment Flow • A flowchart is a graphic representation of how a process works showing, at a minimum the

sequence of steps. A flowchart helps to clarify how things are currently working and how they could be improved. It also assists in finding the key elements of a process, while drawing clear lines between where one process ends and the next one starts.

Slide 5.17. Exercise • Think of a process or an activity that you have carried out as part of your work. • Draw a flow chart that clearly identifies the steps that were followed (from start to finish). • Share it with the rest of the group.

Slide 5.18. What to Measure: Outputs • Immediate results from a process or an activity • Output is a program result, e.g. how many staff have had their performance appraisals done, how

many staff meetings have been conducted, how many people attended HIV awareness meetings, number of grantees who have submitted monthly reports

• Highlight success in achieving set targets • Confirm that an activity was conducted and that targets were achieved

Slide 5.19.What to Measure: Outcomes • Outcomes are benefits or changes in participants’ or clients’ knowledge, attitudes, values, skills,

behavior, condition or status. • Tell whether the changes are leading to improvement • Are evidence of a good process/system

Important: Outcome measures tell you whether changes are actually leading to improvement that is,

Vacancy arises Successor

Yes

Promote

No Advertise Receive application

Shortlist

Interview

Ref Check

Job offer

Contract

Staff placement

60

helping to achieve the overall aim. Outcome measures attempt to capture the voice of the customer. They answer the questions: How is the system performing? What is the result? (Are timesheets filled correctly, completely? Are reports complete and accurate and do they follow the format? Are reports submitted in a timely manner?) Ask participants to add examples.

Slide 5.20. What to Measure: Impact • Measures the long-term effect of programs on clients /target populations • Reflects on how the overall goal is achieved • Requires a long period to measure (5+ years) • Can be attributed to many factors • Measures consistency of performance over years

Slide 5.21.Considerations in Measuring Quality

Improvement should not negatively affect other areas • Need for balanced improvement across programs • Ensure that services are provided to the right target groups

Slide 5.22. How to Measure Quality • Standards: Quality cannot be measured without a clear definition or standard.

• Indicators, targets and checklists: Indicators for measuring quality show what, when, how

many/much, etc. and are derived from the performance standards. Indicators can also show

targets. Checklists are documents consisting of various indicators organized for measuring.

Slide 5.23. Relationship between Standards and Measuring Quality Standards

(The foundation to compare the actual performance/ Quality)

Measuring (The link or means of

verifications between the expectations and actual

performance)

Actual Performance and Quality (Results of

Measurement)

List of expectations and process to achieve the expectations

List of items to check or questions to ask to measure the level of performance. The questions are designed based on the expectations in the standards. Commonly used tools are Checklists. Appropriate tools (checklists and techniques or process of measurement should be applied in measuring performance and quality

List of expectations met or achieved compared to the standards

61

Slide 5.24. Examples

Standards/Expectations Measuring Quality

Measuring Tools (Checklists )

How

HIV prevention team must revise the curriculum every three years based on a needs assessment the and involvement of target audiences

1. Do you have a prevention curriculum? (Y/N). 2. Do all peer educators consistently comply

with the 10 steps in the Manual? (Y/N) 3. How frequently is the curriculum revised? (---

-) 4. When was the curriculum revised last? (-----) 5. Did the prevention team conduct a needs

assessment before the revision? (Y/N) 6. Who was involved in the needs assessment?

The curriculum revision or update? (-----)

Supervision, document review, integrated or program performance review meetings, program audit or evaluation

Slide 5.25. Indicators • An indicator is a variable that measures one aspect of a program or project. • Indicators are based on specific standards of care derived from guidelines. • Indicators can be \at different levels: inputs, process, output, outcome and impacts.

Slide 5.26. Characteristics of Good Indicators 1. Valid: Accurate measurements 2. Reliable: Minimal measurement errors 3. Precise: Clearly defined, unambiguous 4. Measurable: quantifiable using available tools and methods 5. Timely: Provide measurements at relevant time intervals that are appropriate in terms of program

goals and activities 6. Important: Linked to achieving the program’s objectives

Slide 5.27. Examples of Quality Indicators • Number of peer educators who received /completed refresher training in the last six months • Proportion of volunteer home-based care providers who consistently comply with the standards of

HBC • Number of organizations that have standard operational procedures aligned with USG regulations • Number of partners who conduct quarterly client satisfaction interviews

Slide 5.28. Targets • Are quantified levels of performance, goals, objectives or outputs that a program plans to achieve

by a certain date • Can be expressed in percentages or numbers

62

• Are precise measures of progress, usually based on numbers related to questions like, "By when will it be done? How much will be done or achieved? How many people or what numbers will be affected? (e.g. 80% of staff will have their performance appraisal conducted by the end of 2010.)

Slide 5.29. Examples of Targets • Convert process indicators into monitoring and evaluation indicators

– Do you have a prevention curriculum? (Y/N). – Number or proportion of peer educators who have or could prove they have a copy of the

prevention curriculum on the day of supervisory visit. (Target: 75%, 85%, 100% etc) • Do all peer educators consistently comply with the 10 steps in Manual? (Y/N)

– Proportion of peer educators who can correctly demonstrate the 10 steps (annual target 80% )

Slide 5.30. EXERCISE • In groups by organization, look at one of the standards you developed yesterday. • Identify at least two indicators for that standard with clear targets (think of quality indicators).

Slide 5.31. Approaches • Quality assessment: Measuring the quality of care at a certain point in quality improvement; a

quality assessment describes the current level of performance with the objective of improving it. A quality assessment is often an initial step in a larger quality assurance effort that may include providing feedback to health workers on performance, training and motivating staff to undertake quality improvements and designing solutions to bridge quality gaps.

• Quality monitoring: A process for regularly collecting and analyzing data on a set of indicators using systematic methods to regularly monitor the quality of health care that emphasizes measuring and analyzing and not individuals.

• External evaluation: Using people outside of the organization to measure quality to validate what has been measured

Note: All the three approaches use a number of techniques.

Slide 5.32. Techniques for Measuring Quality • Participants’ experiences

– List techniques you have used in measuring the quality of your programs or in management – What are the strengths and limitations of the techniques you have applied?

Slide 5.33. Techniques • Interviewing providers, clients, families, community members, opinion leaders • Observing service delivery (by experts, peers, supervisors) • Supportive supervision • Auditing individual client or patient records • Reviewing data from the information system • Written tests, simulations with standardized clients or patients such as case scenarios, problem

solving and demonstrations

63

• Adding others to the list like financial audits • Combining various data collection methods to overcome the intrinsic biases of each method alone;

however, the cost of applying different performance assessment methods also varies widely

Slide 5.34. Interviews • A technique that involves oral questioning of providers, clients, families and communities

– Directly capture the perspectives of participants, staff, clients – May be structured or unstructured

• Provide opportunities to explore topics in depth through clarification • Data from interviews may be summarized on the interview guide or recorded on tape

Considerations for interviews: Interviews are subject to error in a number of circumstances.

1. The setting can easily disrupt and/or inhibit clearly expressed information 2. Interviewer’s knowledge and skills may limit clear understanding or reporting of information 3. Recall error by the interviewee may distort information 4. Too much information, especially from in-depth interviews, may be difficult to analyze

Slide 5.35. Observation • Provides first-hand data on programs, processes, behaviors by directly observing an

activity/process – Provides a holistic perspective including aspects the staff/clients may not be willing to

discuss • Observations may use checklists, and other structured protocols to record data • Having more than one observer is ideal to ensure completeness and accuracy

Observations are subject to a varying degree of “ observational effects” or “desirability bias” • Subjects under observation are thought to perform better or possibly worse than they might in

everyday practice. • They provide answers they perceive the interviewer wants to hear because they are aware that

their performance is being assessed. • The presence of observers might also have the effect of making providers nervous and undermine

their performance. • Providers’ performances may vary from one client or patient to the next or from day to day,

depending on the clients’ or patients’ characteristics.

Slide 5.36. Reviewing Records • Provides information that may not be obtainable or certifiable through other methods • Relies on accuracy and accessibility of the records; not subject to recall bias • Confirms data collected through other methods • Can be used to generate questions for interview guides • Verifies compliance with standards: good for comparison between standards (documented) and

compliance with standards (thru other methods)

Considerations • Records may not be easily accessible • Clients’ records may be incomplete

64

Slide 5.37. Tips on Measuring Quality Tip 1: Improvement requires change, and change is by definition a temporal phenomenon. A great deal of information about a system and how to improve it can be obtained by plotting data over time—data on length of stay, volume, patient satisfaction—and then observing trends and other patterns. Tip 2: Remember, measurement is not the goal: improvement is the goal. In order to move forward to the next step, a team needs just enough data to know whether changes are leading to improvement. If we look for the perfect, we will wait and not ever achieve the good. Tip 3: Sampling is a simple, efficient way to help a team understand how a system is performing. In some areas, the patient volume is typically low enough to allow tracking key measures for all; however, sampling can save time and resources while accurately tracking performance. For example, instead of monitoring the time from registration to seeing the provider continuously, measure a random sample of 10 to 20 patients per month. Tip 4: Useful data are often easy to obtain without relying on information systems. Don’t wait two months to receive data on patients’ average length of stay in hospital. Develop a simple data collection form, and make collecting the data part of someone’s job. Often, a few simple measures will yield all the information you need. Tip 5: Use qualitative and quantitative data. In addition to collecting quantitative data, be sure to collect qualitative data that often are easier to access and highly informative. For example, ask the nursing staff how adherence counseling is going or how to improve on no-shows. Or, in order to focus your efforts on improving patient and family satisfaction, ask patients and their families about their experiences at clinic visits. An interesting discussion at this point would be to ask the group if they have asked patients and/or their families about their experiences at clinic visits. What did they learn that surprised them or that validated their own perceptions? How could they use this information for improvement? The purpose of this conversation is to begin to discuss the broader view one receives from talking with others, specifically patients and their families, and how this broader view can challenge and confirm our own experiences. In addition, this discussion could center on how this information might be utilized to improve the system.

65

Session 6: Supportive Supervision Slide 6.2. Objectives • To define and characterize supportive supervision; explain steps and techniques to conduct effective

supervision and give guidelines on how to provide constructive feedback

• To provide opportunities for practical supportive supervision through field visits or an analysis of a case study and encourage its application in quality improvement

Slide 6.3. Introduction • This session can take one of three forms after an introductory session:

– a three-part case study – a field visit to conduct supervision, provide feedback and prepare report – role plays

This workshop will use a case-study approach.

Slide 6.4. Supportive Supervision • What do you understand by the term supportive supervision? Definition: • Supportive supervision is the process of guiding, helping and encouraging staff to improve their

performance so that it meets defined standards.

Slide 6.5. Features of Supportive Supervision • Participatory problem solving based on clear standards and tools

– motivational • Focuses on

– shared vision and responsibilities for quality care – two-way communication and shared decision making – providing support that staff need to perform well; mentoring and building capacity – Systems and processes rather than individuals as sources of gaps.

Slide 6.6. Why Supportive Supervision? • Motivation: Supportive supervision with constructive feedback increases the motivation of

providers or supervisors to improve the quality of care. Supervisors are more welcomed and accepted.

• Verification: By observing, actively listening and reviewing records, supervisors can verify the quality of services provided.

• Solutions: Solving problems becomes a routine function of providers, and external supervision decreases with time.

• Opportunity: Supervisors can assess inputs, processes of care and outputs to fulfill both clients’ needs and staffs’ rights.

66

Slide 6.7. What Makes Supportive Supervision Effective? • Ask participants. • Key factors:

– Mindset of best performance (quality) and commitment to it – Clear standards and tools for service delivery and measuring performance – Agreed upon process, clear to all involved parties for planning, scheduling and

communicating

Slide 6.8. What Makes Supervision Effective? – Competence and supportive attitude of supervisors as role models – Objectivity: observation, record reviews, interviews with beneficiaries, communities and

stakeholders – Active engagement of all parties in planning, implementation and follow-up

Slide 6.9. Steps in Supportive Supervision

1. Developing systems for supportive supervision – Clear roles and responsibilities – Training and capacity building of supervisors and supervisee(s) – Tools (checklists, formats, templates) – Standards and guidelines (how-to manuals) – Annual plan, budget and agreed upon schedule

2. Preparing the supervisory team – Review information on performance of supervisee(s) and agree objectives for each visit – Handle the logistics for the visit – Gather materials and supplies for the supervisee(s)

Slide 6.10. Steps in Supportive Supervision

3. Conducting supportive supervision – Site visit – Assessment ( interview, record review, observations) – Community visits/clients and community interviews – Feedback***

Slide 6.11. Process for Supportive Supervision

4. Constructive feedback – Recognize good performance or achievements first. – Ask the staff how they think they did (usually supervisors are the fiercest critics). – Be very specific with criticism; give a specific example of what was not done correctly. – Explain why it is important that the correct procedure is followed (explain the implications of

NOT following the correct procedure). – Check that what you have said has been correctly understood. (Ask staff to repeat it in their

67

own words).

Slide 6.12. Steps in Supportive Supervision

5. Concluding the visit – Analyze the information. – Identify problems, set priorities and develop a plan of action to improve performance. – Give feedback***

6. Implementation and follow-up – Commitment to fulfill roles and responsibilities – Follow-up and support – Regular assessment of progress and planning

Session 7: Information Management

Slide 7.2. Objectives • To define key terms and concepts in information management and discuss the process of data

collection, analysis and action planning in quality improvement

• To discuss sources of information and the importance of utilizing existing information to improve quality and explore feasible approaches to integrate quality indicators and targets for quality of care into the routine monitoring and evaluation system

Slide 7.3. Role of Data in Quality Improvement

• Ask participants what they understand by this.

– Quality improvement is data driven. Data give a starting point, keep momentum up and record achievements. Data can be viewed as “push” and “pull.”

Data Data pushes quality improvement by helping to identify and analyze problems

Data

Data pulls quality improvement by helping to identify and analyze opportunities

68

– The more effort you put into understanding and utilizing data, the more you will be rewarded in terms of solving the right problem in the right way. (The Victorian Quality Council, 2008. www.health.vic.gov)

Slide 7.4. Defining Terms and Phrases • Data: Observation of facts. Stories and numbers • Information: processed data on person, place, time, cost etc • Management: planning and coordinating activities towards a goal • System: an organized method that follows a set of recognized procedures to achieve a certain goal

– Information system: collecting, processing, storing and retrieving information and using it to make decisions

Slide 7.5. Group Exercise • You are the program manager in District X and you have concerns about the quality of services

delivered by your team. You are to meet with senior management to discuss this issue.

Questions: • What information will you present to management, and how do you plan to solve this problem?

Provide examples

Facilitate group work by asking questions like the following. • How did you (your team) come to suspect there was a problem? How did you know whether the

problem was real or apparent? • What step(s) did your team take first?

– Did you look at existing information sources or key informants? – Did you find the information you were looking for? – How can that affect your data collection?

• If you don’t have data, how can you collect data to verify the problem exists? – Data can be collected routinely through reports or actively through supervision and field

visits. – Assume you have collected data, what will you do with it? How can you process, store and

retrieve the information you need? How can you make sense out of your data?

Slide 7.6. Participants’ Responses • Summarize the responses by asking

– How did you collect information? – What did you do with it? – Who else was involved in collecting and using this information? – Can you find the same information if you look for it next time (for next visit)?

Slide 7.7. Information Management Defined Process and practices of planning for, collecting, processing, analyzing, storing, retrieving and using information for making decisions

Slide 7.8. Information Management Cycle

69

Slide 7.9. Planning • Gathering information for a particular problem • Determining the need: type and amount of information • Source of information

– Do we have existing information such as M&E systems, reports? Where is it located? How can it be useful?

– Do we need to collect new or additional information, and if so, how? • If you need to collect additional information, decide what data is required.

– Develop indicators based on the service standards or on the problem you are going to solve. – Determine how that information will be collected. – Consider the data flow, and determine in what format the data will reach you.

Slide 7.10. Collecting • Information is like water. Too much, you drown in it; too little, you die of thirst. (Anonymous). • Developing infrastructure and systems:

– Chose appropriate data collection methods. – Design and test data collection tools. – Orient and train data collectors. – Design a data storage and retrieval system.

Slide 7.11. Collecting • Collect only useful information:

– Appropriate : relevant information for goal of improvement – Quality: complete, accurate and clear information – Timely: up-to-date or indicates current/recent status, events, activities – Amount: sufficient to make conclusions

Slide 7.12. Collecting • Techniques for collecting information:

70

– Directly observe service providers: observation checklists – Interview clients: questions focusing on limited number of issues – Focus group discussions: discuss one issue at a time – Self-evaluations by service providers

Slide 7.13. Processing and Analyzing • Processing

– Data cleaning or validating, entering, compiling/aggregating, tabulating results • Analysis

– Review data collected in comparison to program objectives and targets – Compare your program targets with actual performance.

Important: Analysis does not mean using a complicated computer package. It means taking the data that you collect and reviewing it in light of the questions that you need to answer. Guiding questions in analysis: 1. What question do you want to answer? 2. What technique should be used?

Slide 7.14. Presentation • The part of the information management system that includes analyzing and displaying information

on program and management performance by using tables, graphs, charts and maps • Very important in quality improvement efforts

Slide 7.15. Why Presentation? • To facilitate timely and accurate decision making • To disseminate information and enhance a shared understanding of performance • To improve communication and skills of staff • To understand trends and be able to forecast • To make comparisons easy

Slide 7.16. Considerations in Presenting Information

What and why to present The type of information to be presented

How? Appropriate tools or methods, formats and language

Where to display? Where appropriately utilized

When to update? Maintain and update on a regular basis.

71

Slide 7.17. Using Information for Actions

The Continuum and Relationships

(Source: The Victorian Quality Council, 2008. www.health.vic.gov.au ) The key message from this slide is the following.

1. Data have no meaning without reference to either space or time. The context is very important, e.g., if one sees the number 5, one can immediately associate it with cardinal numbers and relate it to being greater than 4 and less than 6, whether this was implied by this particular instance or not.

2. Information is a relationship between data elements. Relationships form patterns. Information relates to descriptions, definitions, or perspectives (what, who, when, where). Putting a lot of emphasis on data collection without equal attention to analysis and use is futile.

3. Knowledge is the ability to realize and understand patterns and their implications. Knowledge comprises strategy, practice, method, or approach (how).

4. Decisions can be made without complete knowledge though knowledge is important for making good ones. It is better to make a timely decision with incomplete knowledge than to wait for complete knowledge.

5. Actions are the ultimate purpose of data collection and analysis for decision making.

Slide 7.18. Using Information for Actions • Make adjustments in programs and management based on information

– Quality, quantitative targets, resource allocations – Disseminate information – Review and redesign the system

• Data Quality – The data you collect are meaningful only if they are of the highest possible quality. Data

quality must be monitored at every single step of the process and should not depend on only one person.

– The quality of the data determines the usefulness of the results.

Slide 7.19. Tips for Effectively Using Information • Collect useful, not perfect data, • The purpose of data is learning, not evaluating. • Use paper/pencil; do not wait for information technology. • Use sampling. • Use qualitative data rather than wait for quantitative data. • Stratify by provider, location, population, patient. • Plot data over time.

Slide 7.20. Reflections and Learning

Data Information Knowledge Decision Action

72

• Linking M&E and QI systems – We have seen QI is evidence based. Routine information, if used, forms the basis for

measuring and monitoring the quality of services in continuous manner. – Every organization has an M&E system at a minimum for donor reporting.

Slide 7.21. Reflections and Learning • Linking M&E and QI systems

– How can we link up the existing M&E and continuous QI systems? (Please draw clear steps and what is required to ensure the steps are implemented.)

73

Session 8: Concepts and Principles of Quality Improvement

Slide 8.2. Objectives • To discuss the concept and principles of quality improvement • To introduce the four basic steps in quality improvement

Slide 8.3. Concept

“Crossing the Quality Chasm’’ IOM 2001: “Between the health care we have and the care we can have

lies not only a gap, but a chasm’’….….. ‘’the problems come from poor systems, not bad people.’’

Slide 8.4. Concept • In QI, it is essential to make changes to the system in ways that permit it to produce better results. • QI identifies unnecessary, redundant or missing parts of \ processes and attempts to clarify and/or

simplify them.

Slide 8.5. Concept

“While improvements require change, not every change is an improvement.”

Slide 8.6. Improving Quality Defined

• “…quality improvement is the complete process of identifying, describing and analyzing strengths

and problems and then testing, implementing, learning from and revising solutions.” (Source:

National Child Welfare Resource Center for Organizational Improvement-NCWRCOI).

Slide 8.7. “Even if you’re on the right track, you’ll get run over if you just sit there!”

(William Rogers, 1879-1939)

74

The point is that even when doing something well, one must always improve, so improvements are

continuous.

Slide 8.8. Benefits of QI • Can help program implementers and managers

– To define performance standards and operational procedures – To assess performance – To improve program performance and effectiveness

• Makes it possible to address the different dimensions of care

Slide 8.9. Benefits of QI • Safety: avoids harming clients • Effectiveness: services benefit those who receive them • Client-centered: care is respectful, responds to individual patient preferences, needs and values • Timeliness: minimizes client waiting time and avoids delays • Efficiency: focus on cost of quality • Equity: should avoid bias and should be fair

Examples

• Timely financial reporting results in the timely disbursement of funds from donors which builds trust and the sustainability of programs.

Slide 8.10. Principles of QI • The focus is on clients’ needs. • Providers understand their work in terms of processes and systems. • Problems are solved by teams of health-care providers. • Providers test hypotheses and use data to measure change.

75

Slide 8.11. Client focus • Gather information through client interviews, observations or focus-group discussions. • Design services and the delivery system to meet the needs and expectations of clients.

Important: First, in addition to the actual services that clients require for their health, clients also have expectations or desires for services that providers might not necessarily understand. This presents a challenge to not only deliver the health care that is needed, but also to deliver it in a way that is acceptable to clients. Secondly, a focus on clients does not just involve making them happy. Clients also need information in order to access services and to make appropriate decisions. It is also important to recognize that clients have different needs. Although needs vary by individuals, socio-economic and cultural factors and service delivery settings, the common concerns are captured through dimensions of quality of care.

Slide 8.12. Process and Systems • A process is a sequence of steps through which inputs from suppliers are converted into outputs for

customers, e.g., decisions, materials and information flow • A system is the sum total of all elements that interact together to produce a common goal. A system

is the arrangement of people, materials and procedures associated with a particular function or outcome. It consists of inputs, processes and outputs.

Slide 8.13. Systems View Input Process Output

Trained personnel

Set standards, Plan for quality Poor HR systems(no supervision/no performance appraisals

Compliance with standards, improved efficiency, improved health outcomes Poor results/not meeting targets/poor client handling

• Effective change takes into account how parts of a system are coordinated and linked rather than focusing on just one part. Training alone does not help to improve quality greatly though it helps to the extent that lack of knowledge or skills affected performance. The performance of other components—infrastructure, logistics, supplies—is equally important. A systems view is essential!!

Slide 8.14. Testing Hypotheses and Using Data • QI is testing hypotheses

– Making hypotheses about the effectiveness of the interventions – Testing hypotheses through experimenting and interpreting the results

• Data collection and analysis are the backbones of the system. – Data are used to identify and analyze problems and to develop, test and implement

solutions.

Slide 8.15. Teamwork • “A bundle of 50 lemons is a weighty load for one person but fun for 50” (Ethiopian proverb)

– sharing responsibilities eases burdens and shares accomplishments – “Unity is a strength.”

76

Slide 8.16. Methods • What are methods or actions that improve the quality of health services? • QI (or QA) methods can be used in combination or in isolation: integrated supportive supervision,

training (staff and management), performance audits and feedback, redesigning and improving processes, job aids or reminders, certification, licensing, accreditation

Slide 8.17. Individual Approach • Individuals identify apparent problems in their everyday work, recognize their ability to fix them and

feel empowered to make necessary changes. These problems are not interdependent; one person can make and implement the decisions necessary to address them.

Slide 8.18. Rapid Team Approach • A series of small incremental changes in a system is tested and possibly implemented to improve

quality. – Can be applied in any circumstances by an ad hoc or temporary team – Needs people with experience – Draws upon existing data sources

Problem: typing errors in a report Solution: get specialists who are colleagues to proof read it before it is sent

Slide 8.19. Systematic Team Approach • Used for complex or recurring problems that require detailed analysis

– Frequently results in significant changes to a system or process – Involves data collection, a considerable amount of time and resources – Involves a detailed study of the causes of problems and then developing solutions accordingly

– A hall mark – Best used by an ad hoc team with a goal over a period of time

Example: To improve the delivery of HIV services for a project, set up a continuous QI (CQI) team and disband it when its work is finished

Slide 8.20. Process Improvement Approach • Used to respond to core processes of a system

– A permanent team that continuously collects, monitors, and analyzes data to improve a key process over time

– In organizations where resources are permanently allocated to QI – Various stakeholders contribute to the analysis

Example: a program committee for different partners over a project’s life. Emphasize that increasing complexity determines the approach to use

77

Slide 8.21.Choosing a QI Approach (an algorithm)

Slide 8.22. Team Approach to QI • Participants’ reflections: Why is the team approach used in quality improvement?

(Summarize participants’ responses on the flip chart)

Slide 8.23. Why a Quality Improvement Team? • Services consist of interdependent steps and processes that are executed by different people; so are

improvements in the services • Quality lapses often occur in interactions between people and steps • Staff can often identify problems and generate ideas to resolve them if given the opportunity

Slide 8.24. Why a Quality Improvement Team? • A team is made up of owners (doers) and stakeholders (receivers) of a process. • Accomplishing things together increases the confidence of each team member which in turn

empowers organizations. • Participation improves ideas, increases buy-in and reduces resistance to change.

Slide 8.25.What Needs to be in Place for a QI Team?

1. Management commitment: Led by a sponsor: holds a QI planning meeting. 2. Relevant composition: Team members should be relevant staff who are affected by the problems or

areas selected for improvement. 3. Leadership: The team leader is usually a senior manager responsible for the process.

– Team leader updates sponsor: sign off

Does the problem exist in the core process?

Is the problem interdependent?(can one person solve it?)

Is the problem complex and recurring?)

No

No

No

Rapid Team Problem Solving

Yes

Yes

Process Improvement

Individual Problem solving

Systematic Team Problem Solving

78

Slide 8.26. What Needs to be in Place

4. Training: The team leader and facilitator are trained prior to start-up. – The team should be provided with training in initial tools. – Subsequent “just in time” training will be provided by the team leader and facilitator as

required.

5. Facilitator: Each CQI team should have a facilitator or a pool of facilitators.

Slide 8.27. What Needs to be in Place? 6. Record keeping: A team leader or secretary will keep a file of all documents related to the team’s

activities/accomplishments. – Records include meeting minutes, sign-off documents, copies of questionnaires, all charts,

graphs – When solutions have been implemented, the team can meet less frequently (every 6 months)

to evaluate performance/modify solutions

7. How many meetings and for how long? – Determine according to need

Slide 8.28. Team Ground Rules and Guidelines • All decisions are made by consensus. • Some issues are confidential. • Listen to others. • Come prepared for the meeting. • Be an active participant. • Improve how the group works together; evaluate your meetings near the end. • Keep records of your work. • Always remember the sign-off process.

Slide 8.29. Steps in Quality Improvement • Four Basic Steps

– Identify: determine what to improve – Analyze: understand the problem – Develop: hypothesize about what changes will solve the problem – Test or implement: to see if the hypothesis yields improvement; based on the results, decide

whether to abandon, modify, or implement it Important: Although QI approaches vary depending on the complexity of problems, they all follow these four basic steps: identify, analyze, develop, test or implement

Session 9: QI Model and Collaboration

Slide 9.2. Objectives • To understand the four steps in quality improvement and develop skills for applying the quality

improvement model (Plan-Do- Study-Act)

79

• To strengthen participants’ skills in quality improvement through hands-on practice of PDSA by using examples from their program settings

Slide 9.3. Steps in Quality Improvement • Four basic steps

1. Identify: determine what to improve 2. Analyze: understand the problem 3. Develop: hypothesize about what changes will solve the problem 4. Test or implement: to see if the hypothesis yields improvement; based on the results,

decide whether to abandon, modify, or implement it

Slide 9.4. Key Questions, Steps and Model for QI

Slide 9.5. Plan-Do-Study-Act (PDSA) Model • Also known as the Shewhart Cycle, the Deming Cycle or the Learning and Improvement Cycle • A leading model in health-care improvement efforts • Is a time-tested quality improvement tool • The model conceptualizes the continuing cycle of improvement.

4.Test &

Implement

Plan

Do

Study

Act

What are we trying to accomplish? How will we know that a change is an improvement? What changes can we make that will result in improvements?

Identify

Analyze

Develop

80

Slide 9.6. The PDSA Cycle for Quality Improvement • Once a team has set its goal, established its membership and developed measures to determine

whether a change leads to an improvement, the next step is to test a change in the real work setting. The PDSA cycle is shorthand for testing a change by planning it, trying it, observing the results and acting on what is learned. This is the scientific method used for action-oriented learning

Plan: objectives, questions and, predictions (why). Plan to carry out the cycle (who, what, where, when) DO: Carry out the plan (on a small scale). Document problems and unexpected observations. Begin an analysis. Study: Complete the analysis of the data. Compare data with predictions. Summarize what was learned.

Act: Adapt? Adopt? Abandon? Next cycle?

• Reasons to test changes: Testing will increase your belief that the change will result in improvement, will help you decide which of several proposed changes will lead to the desired improvement and will help you evaluate how much improvement can be expected from the change. It will also allow you to decide whether the proposed change will work in the actual environment; to decide which combinations of changes will have the desired effects; to evaluate costs, social impact and side effects from a proposed change and to minimize resistance upon implementation.

Slide 9.7. Step 1: Plan • Set goals for the quality improvement cycle: questions, predictions, data to be collected, and the

who, what, when, where of the project

• Identify reasons for Improvement: brainstorm, clarify and eliminate; agree on the main problem area that will inform other areas

• After measuring the quality of services, staff discuss and consider activities that will lead to an

improvement. Ask, will this work? Why? A problem may have multiple causes; it’s important to identify the most important one and to develop a work plan to implement the solution. Who will do what, where, how and when?

Slide 9.8. The Problem, Goal and Target • Problem: Gaps in the existing support supervision system in CDO • Goal: To strengthen supportive supervision practices in CDO by the end of 2010

Act Plan Study Do

81

• Project target: Program management, service delivery and operations will follow agreed upon schedules and utilize shared supervision checklists for all supportive supervision.

Slide 9.9. Analyzing the Problem: Use the Fishbone

Slide 9.10. Matrix for Setting Priorities

Problem Evaluation criteria4 Total score ( the products of the evaluation criteria)

Priority

level Feasibility5 Effectiveness6 1. Gaps in the existing supportive supervision system in CDO

5 5 25 1

2. Shortfall on the budget line for supportive supervision

2 4 8 3

3. No budget for recruiting additional accountant

4 4 16 2

4 Evaluation criteria. 5 Feasibility: 1 lowest and 5 highest 6 and Effectiveness: 1 lowest and 5 highest

No supportive supervision (SS) plan and schedule

No SS guideline in CDO

Staff skill in SS is limited

No budget allocated for capacity building in SS

Gaps in the existing Support Supervision system in CDO

82

Slide 9.11. Expected Changes • Develop and use an integrated checklist and template for an action plan. • Develop a user manual that outlines how the check list and other tools will best be put to use. • Annual operation plan with schedule and budget for quarterly supportive supervision. • Communication and agreement on the schedule with implementing partners.

(These are the expected changes at the planning stage. What did you have in mind, the changes at the end of the cycle or after the planning stage?)

Slide 9.12. Get Practical with Countermeasures (Interventions) • Cause:

• Countermeasure: Practical method: 1. Hold a meeting with management on the proposed measure 2. Do monthly follow-up on supportive supervision (Is it working?)

Slide 9.13. Build in Sustainability

How will changes be sustained? • Ownership: project is based in the department responsible for ensuring quality • Replication and scaling up: the improvement will be replicated in other programs • Leadership and commitment: the team champion is the director of the organization

Slide 9.14. Complete with an Implementation Plan • Agree on implementation dates, the responsible person and the budget required to implement the

proposed actions. • Agree on indicators for measuring and monitoring progress against targets or best practice scores. • Agree on a forum for sharing and learning, e.g. interim review meetings.

Slide 9.15. Group Exercise

Read the short story on Ruth in your folders (Annex III Part E).

Step 1: Plan • Apply the quality improvement model and make a plan to solve the problems Ruth and the other

five clients are facing.

Slide 9.16. Step 2: Do

Carry out the plan and document problems and unexpected observations, re-assess and review. – Analysis must be evidence based: collect data or use an existing data source, e.g. M&E data – Ask why the issue is a problem. Seek at least 5 whys. – Focus on the causes you can control.

83

Note: As stated on slide 9.8, implement the change according to the plan made in the first step and document any challenges or other observations.

Slide 9.17. Step 3: Study

Complete the analysis of the data, compare it with predictions, summarize lessons, decide on actions • Collect data to see whether interventions bring about desired improvements • To measure and compare the change, use a graph or a table. • Base the change on evidence. • Compare the initial predictions with what has happened. • Is there anything unexpected, any lessons?

Note: Questions to ask: What worked and what didn’t work? What should be kept, changed or abandoned? Use the results to plan the next cycles. The end of one cycle should lead to the beginning of another.

Slide 9.18. Step 4: Act • Determine the adjustments to be made to your intervention based on the study. • Decision categories: adopt, adapt, abandon

– All other projects implementing similar prevention activities should have a CQI team, staff designated for data collection, a simple tool for monitoring activities

Note: a. Refine: The change may need to be modified until it’s ready to be conducted on a larger scale. b. Scale up: Once good results are achieved, the change can be implemented on a larger scale. c. Abandon: If the change wasn’t at all effective, abandon it and start with another.

Slide 9.19 .QI Collaboration

(See PowerPoint Slide)

Slide 9.20. QI Collaboration

(See PowerPoint Slide)

Slide 9.21. QI Collaboration

(See PowerPoint Slide)

Slide 9.22. QI Collaboration (Disseminating Lessons and Scaling up Improvements)

(See PowerPoint Slide)

Session 10: Institutionalizing Quality Improvement

84

Slide 10.2. Objectives • To discuss the concept and practice of institutionalizing quality improvement • To create a shared understanding about leadership commitment, structure and systems in

sustaining quality improvement

Slide 10.3. Institutionalize • What do you understand by the term institutionalize? • What does it take to institutionalize quality improvement?

Slide 10.4. Institutionalize Defined QI institutionalization is an ongoing process where activities related to defining, measuring and improving quality become formally and philosophically integrated into the structure and functioning of an organization or system.

Slide 10.5. Institutionalize • Establishing a culture of quality within the organization and make quality improvement an integral,

sustainable part of the organization’s performance • A long-term or strategic approach that systematically develops essential elements.

Slide 10.6. What it Takes to Institutionalize Eight Essential Elements for Institutionalizing Quality Improvement

Internal Enabling • Policy • Leadership • Core values • Resources

Organizing for Quality • Structure Support Functions • Capacity building

• Information and communication • Rewarding quality

Slide 10.7. Participants’ Reflections • How can you contribute to institutionalizing quality improvement in your organizations?

– Think about your responsibilities in the organization or program. – Consider the essential elements of institutionalizing quality improvement. – Discuss what will be possible for you to do immediately after this workshop.

• In plenary, share your thoughts/plans and incorporate your thoughts into your plan later in the planning session.

85

ANNEX II: SURVEY FORMS A: Organization’s Standards and QI Practices Survey 7 Instruction: Mark YES if quality improvement standards, processes, tools and practices are available and used in your organization, department, project or program as applied.

Please indicate your position (check the appropriate function in the box to the right): Technical ______ Administrative ______ Financial ______ Management ______

ITEM YES NO COMMENTS 1 Are there service delivery standards, guidelines or protocols for

your HIV and AIDS programs (Prevention, VCT, and home-based care)? (standards state what to be done, how, who, where and when) ,

1.1

List standards

1.2 Where appropriate, are the program management and service standards in line with government, or World Health Organization, or President’s Emergency plan for AIDS Relief (PEPFAR)?

2. Do you have a policy manual, standard operational procedures or documented processes for finance, administration and human resource management?

2.1. List what you have:

2.2 Are the standards in line with national standards or international (e.g. IFRS8) standards?

3 Is there a process to assess compliance with standards? 3.1

Describe the process:

4 Does the organization have any approaches9 for measuring and improving performance (both for management and for programs)?

5 If yes to question 6, which model is the organization using? Performance Improvement Review (PIR)

7 To be completed by the management teams and senior staff 8 IFRS: International Finance Reporting Standards 9 Approaches: include processes for measuring and analyzing performance against standards, planning and implementing improvements; measuring results to see if changes have improved; and action plans planning for further improvements.

86

Client Oriented Provider Efficient (COPE) Supportive supervision Performance audit Staff appraisal Other (specify):

6 Are there quality indicators and targets, and best practices to assess compliance with the standards?

7 Do you have checklists for supervision that involve service delivery, program management, finance and administration?

8 Does the organization have a designated quality improvement team? (If no, this concludes the checklist.)

9 Has the team been trained to monitor performance against standards?

10 How often does the team collect data on indicators? 11 How often does the team meet to compare results against

targets or performance expectations?

12 Is action planning a routine part of the team’s work? 13 If the team has a documented plan for quality improvement,

does it include the estimated cost of the intervention?

14 Does the team have a budget for interventions? 15 If the team does not have a budget, who meets the cost? 16 If the team have a documented plan for quality improvement,

who leads or is responsible for the oversight?

17 Describe any changes the organization has made that have been monitored and documented as improvements in the last 12 months.

B: Quality Improvement Training: Individual Participant’s QI Experience Survey

______________________________________________________________________________ Please indicate your position (check the appropriate function in the box to the right):

Category A. Technical/programs Category B. Admin/Finance/HR/Management

_____________________________________________________________________________________ 1. Category A staff, have you received quality improvement or quality assurance or quality management training in the past? If yes, write the calendar year of the training____________

2. Category B staff, have you received any training in financial management, performance improvement

or performance appraisal?

If yes, write the calendar year of the training____________

87

3. Please state three key lessons you had learned from the QI training that you received.

3.1.

3.2.

3.3.

4. In your own words, please define quality of services.

5. Have you ever participated in developing or reviewing service standards, standard operational

procedures or administration, financial or program management policies and procedures?

If yes, please specify

6. Have you ever participated in quality improvement or quality assurance or quality management

meetings? (Yes/ No)

If yes, please explain where and your role?

7. Have you ever participated in a program review or management review, identifying gaps and solving

performance problems? (Yes/No)

If yes, please explain

8. Have you ever participated in supervision that involved service delivery, financial and program

management? (Yes/No)

If yes, please explain

88

9. Do you have special dietary requirements?

If yes, please specify

10. Do you have any special health considerations?

If Yes, Please explain

89

C: Daily Evaluation of the Sessions Day #

Instructions for rating: Circle the number that you feel best describes your experience at the workshop.

1. Very poor 2. Poor 3. Satisfactory 4. Very good 5. Outstanding

Aspects to evaluate Session # Session # Session # Session #

1. Congruence between

the session’s objectives and your expectations

1 2 3 4 5 1 2 3 4 5 1 2 3 4 5

2. Session content • Organization of the

sessions 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5

• Relevance or applicability of the topic to your organization

1 2 3 4 5 1 2 3 4 5 1 2 3 4 5

3. Time allotted for the session

1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5

4. Facilitator • Facilitation

approach 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5

• Facilitator helped me to understand the content of the session

1 2 3 4 5 1 2 3 4 5 1 2 3 4 5

• Gives clear instructions

1 2 3 4 5 1 2 3 4 5 1 2 3 4 5

• Knows the subject 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 5. Overall achievement of

the session’s objectives 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5

6. Quality of handouts and reference materials

1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5

7. What were the major strengths of the session?

What changes would improve

90

the session?

8. The two things that you

learned today that were very useful

9. The two things you didn’t understand clearly

10. Evaluate the accommodation, conference facilities, meals and refreshments and provide us with comments

91

92

ANNEX III: CASE STUDIES AND OTHER INFORMATION

A. The Dimensions of Quality of HIV and AIDS Services The Dimensions

of Quality

Definition

Safety The degree to which risks related to care are minimized: do no harm

Access The lack of geographic, economic, social, cultural, organizational or linguistic barriers

to services

Effectiveness The degree to which desired results or outcomes are achieved

Technical

performance

The degree to which tasks are carried out in accordance with program standards and

current professional practices

Efficiency The extent to which the resources needed to achieve the desired results are

minimized and the reach and impact of programs are maximized

Continuity The delivery of ongoing and consistent care as needed, including timely referrals and

effective communication among providers

Compassionate

relations

The establishment of trust, respect, confidentiality, and responsiveness achieved

through ethical practice, effective communication and appropriate socio-emotional

interactions

Appropriateness The adaptation of services and overall care to needs or circumstances based on

gender, age, disability, community context, culture or socio-economic factors

Participation The participation of caregivers, communities and children in the design and delivery

of services and in decision-making regarding programs that directly affect them

Sustainability The degree to which the service is designed so that it can be maintained by the

community in the foreseeable future in terms of direction and management as well

as procuring resources

Adapted from Franco et al (2003)

93

B: Case Study on Supportive Supervision by the Community Development Organization Part One: Service Delivery Arrangements and Systems for Supervision

Please read the case and then answer the following questions: 1. What were the important steps in CDO’s supervision? 2. Can you suggest any steps that could be taken by CDO to improve supervision? 3. Review supervision in your organization and make notes on steps to improve it.

The Community Development Organization (CDO) is an African NGO with 15 years of experience

implementing integrated development programs. It has 35 staff: program technical (22), administrative

support (10), volunteers (2) and an intern (1). CDO is implementing the first year of a 5-year project in

HIV prevention and behavior change communication (BCC), voluntary counselling and testing (VCT) and

home-based care (HBC) for people living with HIV and AIDS (PLWHA) with the goal of reducing HIV

transmission in the general population and making a difference in the lives of PLWHA by providing

quality care and support services.

CDO partners with 10 community-based organizations (CBOs) for its program implementation in the 20

districts of the country. Hope for the Affected (HOPE) is one of the 10 and is implementing prevention

and HBC in five villages in a district. At the head office, HOPE has 10 staff including an executive director,

a project coordinator, 2 prevention program supervisors, 2 HBC supervisors and admin support staff. It

also has 5 volunteer HBC providers and 10 volunteer peer educators. The program reaches 250 PLWHA.

HOPE received an award from CDO at the beginning of CDO’s comprehensive program. The last

supervisory visit to HOPE and other partners was in August 2010. A subsequent visit was scheduled for

November 2010.

The CDO country director recently attended standard-based quality improvement training organized by

an umbrella organization in the country. He learned about the importance of integrated supportive

supervision in assessing and improving the quality of programs, managing finance, administration and

support services. He felt there were many gaps in the existing supervisory system at CDO and was

determined to improve and strengthen supportive supervision practices. The following week, he

organized training to share lessons from the training he had attended with an emphasis on effective

supportive supervision. This brief training program enabled the team to describe and analyze the

existing supervisory system and identify several weaknesses including lack of a plan, a budget, an agreed

94

schedule and checklists10 and the fact that supervision focused on visiting partners’ HQ offices and

addressed only program management. Neither service delivery nor financial management or support

were assessed. To strengthen the system, the staff and CDO management established a task force to

develop a draft schedule and comprehensive supervisory checklists for an integrated assessment of

service delivery, program management and the finance and administration systems. They also agreed

that the subsequent supervisory visits should include observing volunteers peer educators and HBC

providers as they offer services to their clients.

The monitoring and evaluation officer was assigned to lead the task force team that was composed of the prevention program coordinator, the HBC program coordinator and the finance and administration mmanager. In the next two weeks, the task team developed and shared the first draft of integrated checklist for assessing service delivery, program management and the finance and administration systems. The checklist had five major components: 1) interviews with CBO HQ staff for organizational management and program implementation; 2) reviewing record keeping and knowledge management; 3) reviewing the general administrative, financial and reporting systems; 4) observations and feedback guides (to observe service delivery and give feedback to providers) and 5) interviewing stakeholders: the beneficiaries, their families, community members and others who implement HIV and AIDS prevention and HBC services in the district.

Summarized the findings by adding the scores for each question asked based on the checklist; 1 for “Yes” and 0 for “No.”

Items to check

No

(0)

Comment

Please interview [specify whom ] to complete items

[from-to]

1 0

1 0

Please review [specify document or record ] to complete items [from-to]

1 0

1 0

10 The checklist is composed of questions for interviewing HOPE staff, review their record keeping and knowledge management, final records and reporting system, guides to observe and give feedback to the volunteers in providing services, interviewing beneficiaries, families, community members and other stakeholders who implement prevention and HBC services in the district.

95

S Add all the 1 responses together and enter the total in the space provided.

Σ (Score for Yes)

% Score= Σ (Score for Yes)/ Σ (all scores)

Observe Service Delivery on 1 n 2 on 3

#

ervations 1 and 0 and enter the total score in the space

provided

Σ (Score for Yes)

They also developed a template for an action plan with columns for the baseline and target scores.11 The task force a step further and developed a draft user manual that outlined how best the checklist and other tools could be put in use.

The checklist was disseminated to staff for review, and they provided their comments the third week after the training. Then the task force revised and finalized the checklist by incorporating the comments from the staff. Another workshop was organized to build consensus on the contents of the checklist and the user manual and to agree on the implementation of supervisory system. The final version of the checklist was disseminated to all staff. All staff were happy with it and decided to discontinue use of all other checklists to avoid confusion.

In June 2010, the CDO management and staff developed the Year 2 operational plan to begin in July 2010. The plan also included a schedule and budget for quarterly integrated supportive supervision. Tentative dates for the visits and supervisory teams were determined. The schedule was agreed upon by CDO’s implementing partners. Two week before the visit to HOPE, the supervisory team composed of one senior finance and administration officer, the prevention program coordinator and the HBC program coordinator met to discuss the upcoming visit, and to share responsibilities for preparation. One of the team members was asked to be the team leader and to review HOPE’s work plan and reports to identify strengths and areas of concern. The second member was to communicate to HOPE to confirm the visit. The third team member was to make copies of the checklist that had been recently developed. Each of the team members accomplished their assignments. At their second meeting one week before the visit, they discussed and summarized HOPE’s plan and performance in the last six months. They also reviewed findings from the last supervisory visit and updated the implementation status of agreed actions and the role and contributions of CDO in assisting HOPE to implement agreed changes (agreed actions). The team agreed on the issues that they thought they should focus on during the upcoming visit. The team also noted that the most important difference between the coming and previous visits would be that the CDO team would go beyond assessing program management by interviewing HOPE

11 See the action plan template in Part 3 of this case study

96

staff at HQ and would include reviewing records to assess documentation and reporting system, observing how volunteers provide services in communities and seeking inputs from the beneficiaries, the families, the communities and other bodies implementing prevention and HBC services in the district. They felt happy as they had a comprehensive checklist to guide all these steps. They understood that supportive supervision requires hard work, and that motivated them to see the effect of the newly designed system by applying all the steps specified in the guidelines (the user manual for supervision).

97

Part Two: Conducting Supervision

Please read the case study and then answer the following questions. 1. What were the important steps in CDO’s supervision? 2. Can you suggest any steps that could be taken by CDO to improve their supervision? 3. Review supervision in your organization and make notes of steps to improve it. 4. List or take separate note of the problems supervisors would come across during community

visits.

On the specified day, a vehicle was ready to take the supervision team to HOPE HQ. The team took with them copies of the integrated checklist and the report with the completed checklist and follow-up action plan from the last visit. On arrival, they found all HOPE staff available and prepared for the visit. After brief exchanges on general issues of life and work, they reviewed the report and follow-up action plan from the last visit. Both teams reviewed the actions implemented by HOPE and support provided by CDO. Guided by the new checklist, the team covered administration, finance, HR management, logistics and supplies and program management. Where necessary, they reviewed available records and systems. During this process, HOPE informed the supervisory team that they held weekly meetings to discuss the day-to-day management of the organization; however, minutes of the meetings were not consistently kept in the record. The team also observed that the staff and volunteers were not consistently filing in their timesheets and that some of their supervisors were neither checking nor signing them and yet they were receiving their salaries and allowances on time. Regarding procurement, there was a policy that was clear on thresholds and authority, but most procurement was not following the procedure, and important documents were missing from the procurement vouchers. HOPE had also overspent on some budget line items without prior approval from CDO, a sign that they were not managing their budget well. The CDO finance and administration manager underscored the importance of good administrative and financial systems for the overall growth of HOPE as an organization and for attracting more donors and avoiding disallowances. He decided to spend more time with HOPE’s accountant to provide more one-on one support. This was completed in the morning of the second day. On the afternoon of the second day, the team from CDO took HOPE prevention and HBC supervisors to visit communities to observe services delivered to communities and to PLWHA. The HOPE staff took the checklist and guides to observe service delivery and to interview the beneficiaries, families and communities as part of their community visit which is the usual practice. The HBC program supervisor from HOPE guided the team. On arrival at the community, they found two HBC volunteers prepared to meet the teams from HOPE and CDO. The HBC program supervisor introduced the teams to the volunteers. One of the providers led the team into the first household. They found a caretaker giving nursing care to the patient. The volunteer greeted the caretaker and offered to help. She also told the caretaker and the patient about the visitors. She received consent from the patient and caretaker to go

98

inside the house and to observe what she does for her client. This client is usually visited weekly; the last visit was five days ago. The volunteer assessed the general condition of the patient for any particular improvement or worsening of the patient’s condition since the last visit and whether the patient was taking his medications properly. The team observed the two volunteers providing services to five beneficiaries. The team from CDO asked if they could also observe two other patients who were in better condition and didn’t require weekly visits. In all the households, the team and the HBC supervisor were checking whether the volunteers were correctly applying the skills they learned at HBC training in line with HBC service delivery standards and guidelines. On a couple of occasions where the HBC supervisor saw mistakes, she politely showed the volunteer how to apply the procedures correctly and complemented the volunteer on assessing the patients’ situations. In every household visited, the volunteers took moments to talk to the caretaker and the patient. At the end of the observation, CDO and HOPE staff and volunteers thanked the patients and the caretakers for their consent. After the HBC observations, the team provided feedback to HBC providers. The HBC supervisor started by outlining the strengths she observed and then highlighting the gaps. In discussing the gaps she gave opportunities to the volunteers to explain whether they understood the incidents the same way the HBC supervisor did. Her feedback was concluded by informing the volunteers that she would come the following week to demonstrate the proper handling of excreta from PLWHA (nursing care) and other patient support outlined in the service delivery guidelines. The team from CDO and staff from HOPE added their observations. Comments from the families were discussed only with HOPE staff to avoid unwanted reactions from the volunteer HBC providers; however, the HBC supervisor took note of the comments to for her mentoring visit the following week. This was the conclusion of the second day of the visit. On the morning of third day, the teams from CDO and HOPE visited another community where they observed the activities of a peer educator (PE). They attended a peer education session in the community and visited a prevention committee. The PE introduced his visitors to his audience. Through his mobilization efforts, the PE had brought in 20 youth, 17 of whom were boys. The PE explained to his audience what HIV and AIDS are, the means through which they are spread and health features of someone with HIV. The audience did not seem to be hearing something new. He went on to outline the risk factors contributing to the spread of HIV. These included, among others, having unprotected sex with someone whose status was not known, having sex with multiple partners, alcohol abuse, accepting favors in exchange for sex and having sex with an older partner. During question time a female participant asked the PE what one would do if her friends laughed at her because she had not sex at 17 years. A male youth wanted to know where one would get condoms for protection. A third youth wondered whether a disabled girl could have HIV. All questions generated laughter. Most participants wanted to know where they could go to find out their status or get condoms. The PE told them to visit the health center across the valley where these services were offered. The session ended with the PE distributing some fliers. The team from CDO wanted to know what guideline the PE was using to carry out his session and whether another session was planned. The PE had a few notes written in his handbook to refer to and was to move to the next village for a similar session next week. The team then moved on to visit the voluntary prevention committee in the village

99

which was happy to receive the visitors. The team appreciated their service and enquired about their progress in HIV prevention efforts. The committee revealed that the youth do not listen to their advice and that young girls are lured into having sex with older men for material favors. The supervisory team helped the committee address their concerns and promised that the PE will visit them more regularly to assist with their work.

100

Part Three: Concluding Supervision and Developing a Framework for Follow-Up

Please read the case study and then answer the following questions. 1. What were the important steps in CDO’s supervision? 2. Can you suggest any steps that could be taken by CDO to improve their supervision? 3. Review supervision in your organization and make notes of steps to improve it.

The afternoon of Day 3, both the CDO team and all relevant staff from HOPE sat in the HOPE office. They shared responsibilities among themselves. Step 1: Computed Scores to Quantify Performance One group added up the scores, calculated percentages for different sections of the checklist and prepared and presented the information in bar charts. Another group summarized findings from the observations (observation guides) and notes from the family and community interviews. They presented their work to the entire team. All staff present were fascinated by the chart of findings which made comparing different sections very easy. This second presentation enabled the team not to lose the details thanks to the graphical presentations.

Step 2: Listing Key Issues The summaries generated a comprehensive list of 101 problems to be addressed. The team felt that was too many to address all at once. One of the members suggested they look at the list and eliminate some problems by combining them with others. With combinations and eliminations, the list came down 75. Another member of the team suggested they explore each issue a little further to discover why it was happening. This helped the team to identify that some were arising from common sources. For example, lack of orientation on good administrative and financial practices of HOPE staff was leading to many gaps in administrative and financial management, and a lack of follow-up after the training provided to HBC volunteers was the most important cause for their performance gaps. Lack of strong follow-up and support for staff was identified as the key factor for staff leaving HOPE. With further exploration, the list came down to 60 Issues. They were pleased with how they had successfully combined problems. Then they decided to sort the issues by the urgency according to concerns expressed by communities and families. On further analysis, the team determined that 75 percent of the 60 problems were not complex and could be solved by individual staff with support in devising and implementing solutions. They clearly marked those problems. Finally, the team settled down to address 15 complex and critical issues that required involving CDO and team work at every level. They further analyzed the underlying causes (root cause analysis) for these problems. Before closing Day 3, they formed five groups of three members each to analyze root causes

101

and suggest solutions.12 On Day 4, they met at 8:30 in the HOPE office. The groups were given half an hour to consolidate their analyses and to present them to the group. Two note takers were assigned to complete the first three columns of the action plan template (below): the issues (problems), the underlying cause (root cause), and suggested solutions (agreed actions). The team finally completed the plan by using the template.

Supportive Supervision Action Plan

Name (Supervising Org) ___________________ Name (Supervised)

_______________________ Level

supervised

Area assesse

d (Mgt,

Prog...)

Current Score

Issues (Problems)

Root caus

e

Agreed Action

s

Responsible

(Level, Person)

Dates for follow-up support

Target Score (expected Improvement)

After action planning, the CDO supervisory team sat with HOPE staff to assess the four-day visit. The HOPE team acknowledged that this visit was unique in many regards, i.e., in how it was organized, the tools used, the number of days it took and the range of issues covered. The visit to the community was the best exercise they had ever experienced. They greatly appreciated the supervisory skills of the team from CDO. They also recognized the mutual respect they showed. Most importantly, the gaps were treated as “ours” unlike in past visits where the gaps passed unnoticed or were treated as “the problems of HOPE.” They also felt that the visit had been tough but fair and that everyone had been given the opportunity to explain the problems and to participate in the decisions about what needed to be done. At their next meeting, the director of HOPE proposed that they use the same checklist to do a self-assessment to fix problems without waiting for CDO to come and supervise. The CDO team returned from the visit the afternoon of Day 4. The team developed a comprehensive report based on their findings and shared it with CDO management and staff. Then management assisted by the supervisory team developed a detailed plan to provide technical support and resources to HOPE in the coming three months. Monthly progress reports were suggested to make sure that all the actions that should be taken by CDO were completed. A copy of the report and the detailed follow-up plan were sent to HOPE the following week.

12 See the template.

102

C: Generic Draft Checklist for Integrated Supportive Supervision Instructions: Interview managers, staff, service providers and beneficiaries; review records and observe service delivery guided by the checklist

Part I: Operations Management Items to check Yes

=1 No =0

Comments

Operational standards exist? (check) - Administration - Financial management - HR management - Program management guidelines

Have board meetings been held according to the terms of reference for Board? (Check Minutes)

Have regular senior management meetings been held? Are there minutes of senior management team meetings? Are senior management team decisions shared with relevant

staff?

Do you have a strategic plan? (Check to see how long it runs.) Do you have a documented annual plan? (Check to see.) Do you have a supervisory plan/schedule? (Check how

frequently?)

Do you use supervisory checklists? When was the last supervisory visit to or from you? Was supervisory feedback given (report, plan of action)? Has follow-up support been offered since the last visit?

Do you have a staffing plan? (check) Is your program/project/org fully staffed according to the plan? Does every staff member have a job description? Are job descriptions filed in individual staff files? Record Review Randomly pick three job descriptions, review and document

the contents JD 1 JD 2 JD3 Comments

• Job title • Role and responsibilities • Qualifications and experience required • Salary grade • Reporting line or supervisory role • Up-to-date ( updated annually) Is there a staff training plan? Does a performance management system exist? (Review) Do staff receive timely supervision, feedback and support on

their performance?

Do procurement, logistics and supplies system exist? (Review.) Is there an equipment maintenance plan? Is the asset register available and updated as required?

103

Items to check Yes =1

No =0

Comments

Is there a system for monitoring vehicle use? Does a financial documentation, reporting and management

system exist? (Review13)

Is annual budgeting done by both program and finance staff? Do budget management tools exist and are they used? (Check

for budget tracking, forecasting and cash flow.)

Are internal controls in place for • accounting

• segregating duties • Inventory

• Management Is there a systematic documentation system for transactions?

(Check the documents available.)

Do you maintain bank accounts separately for different donors?

Is there an approval matrix?

Part II: HIV and AIDS Prevention, Care and Support Items to check Yes

=1 No =0

Comments

HIV Prevention Services Identified target population for HIV prevention

program/services?

Identified model for HIV prevention? (Review) Guidelines or standards appropriate for the model/target

audiences?

Are guidelines/standards up-to-date? Are guidelines relevant for to program’s context? Guidelines are consistently applied by implementers Appropriate prevention messages designed Prevention materials produced Supplies for prevention materials are reliable Prevention program staffed with clear leadership Staff has clear performance targets for prevention program Regular program review conducted System for supervision and support or quality assurance

(Review)

The prevention service is achieving its program goal/target

Observe Service Delivery: Observe a peer education session,

13 Review: standards that exist compared with best practices

104

Items to check Yes =1

No =0

Comments

take notes and give feedback based on the standard • Strength 1:

• Strength 2

• Strength 3

• Gap 1:

• Gap 2:

Interview clients/beneficiaries: Interview three clients, take notes and give feedback to the peer educator

• Strength 1:

• Strength 2

• Strength 3

• Gap 1:

• Gap 2:

• Gap 3:

Community-based/home-based care component Are there guidelines for home-based care? Is there a system to monitor the quality of care provided?

If yes, please explain how?

Are there adequate staff including volunteers to implement HBC?

Are staff/volunteers trained in relevant aspects of HBC? Are PLWHIV and their families provided with information on

services delivered/expected?

Does the organization have a holistic approach to care? physical? Spiritual? Psychosocial?

If yes, give examples or name services provided: If no, please explain how your clients access other services

Is a system in place for assessing the needs of clients? (Explain.) Are there adequate resources to provide safe community HBC

to clients?

Are clients’ records kept in a safe and confidential manner? Are all essential supplies and medication for clients available? Is there a formal mechanism to identify and address clients’

complaints/views? (Explain.)

Is there a system for referring clients to other service providers?

Is there adequate communication among

105

Items to check Yes =1

No =0

Comments

management/staff/volunteers/clients and communities to facilitate provision of quality HBC?

Observe Service Delivery: Observe an HBC volunteer providing service for a client, take notes and give feedback based on the standard

• Strength 1:

• Strength 2

• Strength 3

• Gap 1:

• Gap 2:

Interview Clients/beneficiaries: Interview three clients, take notes and give feedback to the peer educator.

• Strength 1:

• Strength 2

• Strength 3

• Gap 1:

• Gap 2:

• Gap 3:

Information system Is there a functioning information management system for HIV

and AIDS prevention, care and support? (Review the system.) • Data collection tools

• Staff trained • Reports/data received timely • Overall data quality acceptable

Is information analyzed timely and regularly? Is there evidence of information use to monitor program

progress? (Give an example.)

106

D: Quality Improvement Model Case Scenario for Group Exercises A Better Life for Ruth and Other People Living with HIV and AIDS in the Village Rahema is a volunteer HBC provider in an AIDS-stricken village located along a highway from the country’s biggest sea port. She provides support to 15 people living with HIV and AIDS. She is HIV+ and a committed provider. Last week, she visited Ruth in a one-room house Ruth had rented from another person living with HIV and AIDS. Ruth was a commercial sex worker and left her home village and relatives years ago. She is chronically ill, depressed and has lost the meaning of life. She has no friends, neighbours or visitors but Rahema. A community based organization (CBO) recruited and trained volunteers and started providing support and prevention education (key messages) to PLWHA in the village. Ruth is one of the HBC clients for this CBO. She was provided with bedding, clothes, kitchenware and household detergents. She also receives a weekly food ration (flour, cooking oil, salt, a kilo of sugar and tea). She was taken to the nearby health centre where she was started on ART; however, her condition has not improved much in the last six months. There are five other clients in similar conditions in the village who Rahema looks after. Assume Ruth and other five clients are supported by your program. Why do you think Ruth’s health is not improving? 1. Apply the quality improvement model and PLAN to solve the problems Ruth and the other five

clients are facing. 2. Apply the quality improvement model and DO something to solve the problems Ruth and the other

five clients are facing. 3. Apply the quality improvement model and STUDY the effects of your plan on solving the problems

Ruth and the other five clients are facing. 4. Apply the quality improvement model and ACT on your planned intervention to make necessary

adjustments to solve the problems Ruth and the other five clients are facing