Case management versus M&E in the context of OVC programs:

What have we learned?

Jenifer Chapman, PhDFutures Group/MEASURE Evaluation

Overview

Case management tools 101

Update on the Child Status Index

Key questions

What is a case mgmt tool?

Paper-based instrument

Completed by direct service providers (generally low-literate volunteers)

Assesses child’s well-being along priority dimensions (e.g. health, social relations, etc.)

Care plan documentation

Purpose of a case mgmt tool

Purpose: To improve quality of care Highest priority = Case workers’ needs

o Untested hypothesis: case workers will make better decisions if they use a tool

May also support M&E / reporting needs

What is included?

Information collected Client contact information / demographics

Wellbeing information that is changeable over time

Care plan: services & referrals provided

Information flow Most important use is at local/SDP level

Some information may flow up regional level

A plethora of tools...

Child Status Index (MEASURE Evaluation)

Child Support Index (Pact)

OVC Wellbeing Tool (CRS)

Child Status Matrix (FHI)

Parenting Map (TSA)

Etc.

We need to be cautious

Some CM tools are being applied for purposes beyond case management Targeting (identifying beneficiaries)

Program monitoring (recording services provided)

Evaluation (aggregating wellbeing scores)

Exercise caution in using a CM tool for other purposes

Child Status Index Assessment & Update

About the CSI 5 years ago, CSI was designed for low-literate

home visitors to capture children’s status across the 6 domains of PEPFAR OVC programming

Early hopes that CSI could meet a range of information needs

CSI has been implemented for different purposes: case management to program evaluation

CSI is used in at least 16 countries O'Donnell K, Nyangara F, Murphy R, Nyberg B. Child Status Index. A Tool for Assessing the Well-Being of Orphans and Vulnerable Children—MANUAL. Chapel Hill, NC: MEASURE Evaluation; 2009.

CSI Assessment: Phase I

Rationale: to systematically assess how programs are implementing & using the CSI and understand OVC program field needs for additional tools to meet care, support, and M&E demands

Study questions For what purposes are OVC programs using CSI?

What are the advantages and limitations of CSI?

What are the unmet M&E needs of OVC programs?Cannon & Snyder. 2012. The CSI Usage Assessment. Chapel Hill, NC: MEASURE Evaluation.

Summary of Findings: Phase 1

25 interviews with senior program staff in 13 countries

Program staff find the CSI useful

CSI implemented mainly by volunteers

Information collected via CSI is rarely used by volunteers except for targeting (not recommended)

Care plans and referral protocols are inadequate

Summary of Findings: Phase 1

Variation in CSI implementation, data use due to: Unclear purpose & guidance on CSI use with

desire to assess impact

Variability in training approaches

Insufficient support/funding for technical assistance, follow-up, and training (data management, analysis)

CSI is important, but one tool in the toolkit

So now what?

Study Phase II Problem: Lack of information on the utility of the CSI at the

community-level as a job aid & input from CCWs

Purpose: To understand how CCWs and care teams make decisions about children (including role of job aid / data)

Methods: Interviews with CCWs and team leads in five countries, among organizations using/not using the CSI

Revision of CSI Guidance

6 Core CSI functions (we think)

1. Builds rapport between service provider and beneficiaries

2. Orients service provider to the holistic needs of the child and encourages referrals

3. Strengthens informed care decisions by systematically considering and documenting child’s needs

4. If applied regularly with the same child, may show a child’s progress over time in particular domains

5. May be helpful in community-level planning and resource allocation decision making

6. May reveal emergency situations (a score of 1 in any outcome area)

6 Core CSI functions (we think)

Probably inappropriate CSI uses

Targeting Unnecessarily complex

May lead to expectations of action/enrolment

1st contact with child may not be reliable

Evaluating regional or national program impact Children’s needs/status are assessed relative to their

local community, and not to national standards

Probably inappropriate CSI uses

Producing a single combined score for the child CSI assessment should be presented as 12

independent measures

Risk varies across domains

CSI scale values are not equal-interval, but ordinal

Evaluating implementing organizations

Back to Case Management tools generally…

Some key questions Are case management tools effective at improving

care decision making? For all types of case workers? Formal? Informal?

How does training in both case management and tool use factor?

What specifically about a case management tool improves care decision making? For whom?

Does the benefit outweigh the burden?

Some key questions II

Are CM tools useful in managing case workers?

Are some CM tools useful also for targeting beneficiaries, monitoring outputs (services delivered, and evaluating impact? What are the risks?

The research presented here has been supported by the President’s Emergency Plan for AIDS Relief (PEPFAR) through the United States Agency for International Development (USAID) under the terms of MEASURE Evaluation cooperative agreement GHA-A-00-08-00003-00. Views expressed are not necessarily those of PEPFAR, USAID or the United States government.

MEASURE Evaluation is implemented by the Carolina Population Center at the University of North Carolina at Chapel Hill in partnership with Futures Group, ICF International, John Snow, Inc., Management Sciences for Health, and Tulane University.