Kabell Konsulting ApS - presentation WBI BBL 1.3.06 Leadership for Results Evaluating Leadership Interventions Kabell Konsulting ApS, Dorte Kabell

Kabell Konsulting ApS - presentation WBI BBL 1.3.06

Leadership for ResultsEvaluating Leadership Interventions

Kabell Konsulting ApS, Dorte Kabell


Leadership work in Madagascar

• Cabinet level retreats: 2003, 2004 and 2005• Leadership training program for ministers and

SGs (AoLF, IPAC): 2004• Transformational leadership program (Dean

Williams): 2004-2006• Rapid Results Initiatives: 2005-2006• Training of Chefs de régions: 2005


Evaluation in Madagascar

What:Evaluation of AoLF• Immediately after event• 6 months later • A year after

Evaluation of IPAC• Immediately after event• A year later

Evaluation of CabinetRetreats • Immediately after event

How: Needs assessment (by

IPAC) Questionnaires (pre- and post)Structured and open interviews (with participants, non-participants and donors) Document review Direct observation


Key Evaluation Findings (1)

2. Giving self confidence and empowerment at individual level through knowledge of useful management tools and practices, making participants better able to prioritize and manage time and hence stress

4. Facilitating the introduction and internalization of a new results oriented culture, as evidenced in the national vision, budget and strategy and various other new documents.

3. Improving cooperation among individuals and their respective institutions and clarifying roles and responsibilities

1. Creating a strong foundation for increased performance in terms of institutional structure and performance culture as well as a widely shared vision


Key Evaluation Findings (2)

• The “burning platform for change” • A clear commitment at the highest level of

government• A shared vision for the way forward • A well coordinated donor response including

synergies between WB/WBI initiatives • An increasingly client-owned program where

needs have been clearly identified and addressed

Issues to consider for replicability:


Evaluating Evaluation

• Limited participation from beneficiary in evaluation, but strong interest in results

• Retrospective evaluation, but limited monitoring data available

• Evaluation focused on “product”, not process• Outcome was lessons learned. Limited capacity

building - drawing lessons learned does not mean learning to do things differently

• Lack of a common conceptual framework


Evaluation Recommendations

• Incorporate M&E from the start - a process from conception to follow-up

• Evaluation should contribute to building capacity

• Evaluate process & product• Realistic expectations – mindful of trade-

offs• Common conceptual framework for

complementary initiatives


WBI

OPCS/LICUS PREMEvaluation

Country Focused Evaluation Framework


Integrated Country Focused Evaluation Framework

Capacity development at 3 levels

Individual Organizational Institutional/ Enabling Environment

Objectives, based on diagnostic work

Improved

Diagnostic capacity

Improved Coordination, shared goals

Improved

Trust (Demand)

Interventions Dean Williams, AOLF

IPAC

Cabinet Retreats, Rapid Results

PGDI, Judicial reform

(Supply)

Expected results/Indicators

(Does S meet D?)

Example: Madagascar


Intervention specific M&E

Design of M&E adjusted to each intervention

• Transformational leadership

• Peer-to-peer engagement

• Collective learning events/retreats


Align M&E with design & delivery

Initial contact – assure

demand

Scoping mission

Development of concept note

Follow-up(anchoring)

Interim period preparation

Event delivery

Step 1

Step 2

Step 3

Step 4

Step 5

Step 6

Baseline

Results-framewor

k

Observation, questionaires

Analysis

Feedback

M&E


Evaluation Design

Consider

Trade-offs

• Approach (participative)

• Methodology

• Method (self-assessment)

• Tools (facilitated focus groups)


Tool-box

• Survey, questionnaire, interviews, content analysis

• Internal (participants), external (key informants)

• Evocative (“I feel I have x, y, z)

• Evidential (“It seems she has x, y, z”)

• “Work stories”

• Self assessments


Issues for Debate

• Is it feasible to include M&E from start?

• Is it realistic to expect that evaluation can build capacity?

• How to determine trade-offs between flexibility and accountability?

• How to value un-anticipated outcomes and effects?

Documents

Kabell Konsulting ApS - presentation WBI BBL 1.3.06 Leadership for Results Evaluating Leadership Interventions Kabell Konsulting ApS, Dorte Kabell