Advanced Intentional Design Workshop OM LAB 2014

outcomemapping.ca

Advanced Intentional Design Workshop

OM LAB 2014Jan Van Ongevalle , BelgiumHeidi Schaeffer, CanadaZiad Moussa, Lebanon

September 23, 2014

Workshop ObjectivesThe workshop aims at deepening an understanding about:

1.How the individual steps of the Intentional Design process can help with a range of uses including for clarifying a theory of change.

– Systems thinking tools, engaging boundary partners, nested OM systems, planning for OM – step 0

2.How OM tools can be combined with other approaches to add value or nuance in strategy development and identifying monitoring priorities

– Using only selected parts of OM, fusing OM and LFA, using tools of theory of change to strengthen intentional design.

3.How to avoid that the ID becomes too heavy & Tips for writing progress markers and working with strategy maps

Agenda (tentative)9:00 - 9:15 Welcome, Overview of the Day9:15 - 10:00 Harvest Questions & Experiences .10:00 – 10:30: Strengthening intentional design as framework for actor centred theory of change (OM usefulness study, using principles and tools from systems thinking )Break -10:30-10:4510:45-12:30 Strengthening intentional design as framework for actor centred theory of change (systems thinking continued, engaging boundary partners, preparing for OM, nested OM frameworks)Lunch – 12:30-14:0014:00-16:30: •Using only parts of OM and combining OM with other approaches (LFA, TOC)•How to avoid that the ID becomes too heavy •Tips on writing progress markers and working with strategy maps

1. The ‘Socratic Wheel’ - harvesting questions/expectations and

experiences from participants about the intentional design process

1.1. Socratic Wheel Tool• integrates quantitative and qualitative information, • gathers individual information and perspectives as well

as group assessments,• involves participants in assessing and contextualizing the

findings as they are created,• is sensitive to cultural differences,• has greater diagnostic rigor than many traditional

participatory methods, and can be scaled up for more rigorous analysis,

• Adapt for particular evaluation question and context.

1.1. Knowledge of Intentional Design

• 1. Vision and Mission• 2. Boundary Partners• 3. Outcome Challenges and Progress Markers• 4. Strategy Maps• 5. Organizational Practices

1.1. Solar Eclipse Game

2. Strengthening the intentional design as a

framework for clarifying an actor centred theory of

change and for stimulating dialogue and

building trust

2.1. OM and systems thinking

In Michael Quinn Patton, Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use, Guilford Press, June 2010

“No go. The evaluation committee said it doesn’t meet utility specs. They want something linear, stable, controllable, and targeted to reach a pre-set destination. They couldn’t see any use for this.”

2.1. OM and systems thinkingUnderstanding Changes/Evolutions in relationships between the system components

2.1. OM and systems thinkingUnderstanding the appropriateness of the boundaries used to delimit the system





2.1. OM and systems thinking

The different perspectives on what changes and how it is changed in the system

2.2. OM and systems thinkingThe era of “proven models” is (hopefully) long gone

The added value of ID is the “clarification of intent” with regards to: Non linear development (small actions can potentially entail large reactions)How to STRATEGIZE? How to develop STORIES OF CHANGE? Dynamic nature of development interactions (interactions within, between and among subsystems is continuous) Emergent developments re-shape the system

EMERGENCE

2.2. OM’s perceived usefulness for dealing with multiple actors and relationships

2.2 The concept of spheres of influence• Helps to discuss and clarify

expectations, roles and responsibilities of different programme actors.

• Stimulates conversation and dialogue

• Building trust

• How? (What do OM users say - some quotes from OM usefulness study)

• Do’s and dont’sStrategic partners, donors, …

Implementing team

Boundary partner 1

Boundary partner 2

Boundary partner 3

Beneficiary 1 Beneficiary 2

Beneficiary 3

2.2 Quotes from OM effectiveness research

• Simple but powerful stakeholder analysis tool that helped them to discuss and clarify expectations, roles and responsibilities of different programme actors.

• The stakeholder analysis according to the spheres of influence does not remain a side activity in preparation of the subsequent planning or formulation of the programme.

• The OM framework provided programme implementing teams with a ‘language’ to explain the content and the purpose of a programme to programme stakeholders.– “OM helped to clarify expectations of boundary partners in terms of support

needed and contributed to programme staff working in a more participatory way with boundary partners” (Eco health programme South East Asia)

– “OM has provided a language to explain the programme to programme stakeholders.” (Teacher Education and Vulnerability Programme, VVOB Zimbabwe)

– “OM forced everybody to talk about where the programme might go”(Alberta Rural Development Programme)

2.2. Enhancing stakeholder participation in the intentional design

Sharing experiences on how to engage boundary partners and other stakeholders in the development of an intentional design:

Shift & share and wise crowd approach

2.2. Enhancing stakeholder participation

o Boundary partners in conflicto Setting priorities in complex system changeoWorking with BP’s in government or highly

bureaucratic settingsoWhen dealing with large informal international

networko Dealing with OM terminology and jargonoWhen the OM workshop didn’t manage to go to the

bottom of things?o……..?

3. Adapting OM to a programme’s needs and

combining OM with other approaches

3.1. 3.1. Examples of adapting outcome mapping to a Examples of adapting outcome mapping to a programme’s needsprogramme’s needs

The example of World Solidarity: The example of World Solidarity: Planning and monitoring capacity Planning and monitoring capacity development: www.wereldsolidariteit.be development: www.wereldsolidariteit.be

The example of MCNV: The example of MCNV: using outcome using outcome mapping for programme evaluationmapping for programme evaluation

The example of VVOB Zimbabwe:The example of VVOB Zimbabwe: Linking logical framework and Outcome Linking logical framework and Outcome MappingMapping

2727

STEP 8: Monitoring Priorities

STEP 9: Outcome Journals

STEP 10: Strategy Journal

STEP 11: Performance Journal

OUTCOME &PERFORMANCE MONITORING

INTENTIONAL DESIGN

STEP 1: Vision

STEP 2: Mission

STEP 3: Boundary Partners

STEP 4: Outcome Challenges

STEP 5: Progress Markers

STEP 6: Strategy Maps

STEP 7: Organizational Practices

INTENTIONAL DESIGN

STEP 12: Evaluation Plan

EVALUATION PLANNING

3.1. World Solidarity M&E of capacity development

Partners used outcome challenges and progress markers to plan their own capacity development process.

Strategy journals were incorporated in the outcome journals.

Strategy maps were not used for planning but for visualising exisiting support strategies of the global CSO

Outcome journals are used as basis for reflection meetings were progress on outcome challenge and progress markers is discussed.

3.1. MCNV evaluation of health programme3.1. MCNV evaluation of health programme






INTENTIONAL DESIGN

STEP 1: Vision

STEP 2: Mission






INTENTIONAL DESIGN


EVALUATION PLANNING

Partner organisations reflected on the achievement of their own outcome challenges

Made use of the learning questions of the outcome journal






INTENTIONAL DESIGN

STEP 1: Vision

STEP 2: Mission






INTENTIONAL DESIGN


EVALUATION PLANNING

3.1. Vulnerability & Teacher Education programme- VVOB Zimbabwe

Instead used specific objective and key result areas with result indicators

Information from the outcome journals and strategy journals is used to report on result indicators

3.1. OM-LFA fusion VVOB Zimbabwe3.1. OM-LFA fusion VVOB Zimbabwe

VISION Specific objective

Intermediate Result (e.g.

Policy development)


staff development)


early childhood development)

Boundary Partner (e.g.

Student support

structures)


College administra-

tions)


Student bodies)

Sph

ere

of in

tere

stS

pher

e of

influ

ence

Programme implementing

team

Outcome Challenge

Progress Markers

Outcome Challenge

Outcome Challenge

Progress Markers

Progress Markers

Program activities &

outputs

Program activities & outputs

Program activities &

outputs

Spher

e of contr

ol

Organisational practices

Monitoring of results

through result indicators and

specific objective indicators

Monitoring of achievement of progress markers by boundary partners through outcome journal

Monitoring of

programme activities

and organisational practices

through strategy journals

1. % of the colleges with a detailed staff development plan

2. % of the colleges that organise staff development activities related to OVC at least once every term

3. % of the subjects in which OVC themes have been integrated)

Intermediate result (e.g.

staff development)

Boundary partner (e.g. staff devpt.

Com.)

1. Coordinate functioning of staff development committee

2. Organise OVC related staff devpt activities for lecturers

3. Assist curriculum development

4. OVC related inservice training for TP mentors

MSC with lecturers and

student teachers

3.1. Example of operational planning with budget for OM-3.1. Example of operational planning with budget for OM-LFA fusion (see also handout)LFA fusion (see also handout)

Q1 Q2 Q3 Q4

SO To develop capacity and improve quality and relevance of teacher preparation in function of educational needs of orphans and other vulnerable children 329.954 €

/ Sub-tot 0,00 €LOG (+ Detail if possible)LOG 0,00 €PSW (+ Detail if possible)PSW 0,00 €VORM (+ Detail if possible)VORM 0,00 €KTZ-Z (+ Detail if possible)KTZ-Z 0,00 €KTZ-N (+ Detail if possible)

KTZ-N

0,00 €

/ Sub-tot 15.198,00 €LOG (+ Detail if possible)LOG 3.424,22 €PSW (+ Detail if possible)PSW 2.158,48 €VORM (+ Detail if possible)VORM 8.115,30 €KTZ-Z (+ Detail if possible)KTZ-Z 500,00 €KTZ-N (+ Detail if possible)

KTZ-N

1.000,00 €

IR 1 Increased capacity of MHTE & colleges to

develop and implement enabling policy on

OVC

1) Policy in support of the programme is developed, approved and in use in 50% of the colleges2) 25% of the colleges have a strategic document that supports all activities in terms of the programme3) 30% of colleges have implemented action plans towards the realisation of the OVC strategic plan4) 30% of colleges have an adequate M&E system in place for the evaluation of the programme

Planned

Planned x x

Realised

xAll teacher education colleges: college administrations, academic boardsMinistry Headquarters: department for teacher educationVVOB support team members

Realised

Outcome 1: Ministry Department of Planning and Institutional Development facilitates the development and

implementation of OVC policy at level of colleges and the ministry

The programme expects PID in the Ministry of Higher and Tertiary Education to:1. Play active role in monitoring and evaluation of the programme.2. Mobilise resources for the programme (e.g. establish position of project coordinator responsible for OVC issues at MHTE, ….).4. Support OVC related policy development (e.g. review existing policies to incorporate OVC and gender issues; put in place a reporting mechanism on abuse of OVC by both PID and PRD).5. Networking (e.g. collaborate with other stakeholders on OVC; strengthen linkages and collaboration with the National Plan of Action for OVC; …)

Outcome 2: College administrations in collaboration with their academic boards and the department of teacher

education facilitate and support development and implementation of college based OVC policy

The programme expects college administrations in collaboration with their academic boards to:1. Participate in OVC related activities2. Support and facilitate the development and implementation of OVC policy at college level. 3. Facilitate establishment and functioning of IT support structures and IT policy4. Moderate and approve reviewed syllabi5. Play an active role in the management and monitoring and evaluation of the OVC programme.6. Co-fund OVC related activities7. advocate for harmonisation of activities across OVC related initiatives.

x

ChronogramResponsible)

(person/institution) for

implementation

2009 MYP approved

Means

grouped per

cost category

per activity

Quality Education and Vulnerability Outcomes

3.2. OM and Theory of changeThe essence of Theory of Change is linking activities to INTENDED OUTCOMES

I am cutting rocks I am building a temple

3.2. OM and Theory of change

Theory of Change is often seen as a deeper reflective process and dialogue amongst colleagues and stakeholders that make more explicit people’s underlying assumptions of how and why change might happen as an outcome of the initiative.

In the Intentional Design you are actually fleshing out your Theory of Change!!

Even better, you are not shooting at the whole wide world, but at your Boundary Partners

3.2. OM and Theory of changeBasic Elements of a Theory of Change

3.2. OM and Theory of change

• Can you give examples of where the project or programme is working really well? What is making it work well? (Vision)

• What are the long-term changes that need to happen in the target group’s lives? (Mission, Outcome Challenges)

• Who and what (groups, structures, systems, relationships, processes, …) need to change? What are the barriers to those changes? (Boundary Partners, Progress Markers)

• How does the programme try to influence these changes? (Strategy Maps, Organizational Practices)

• How will we know if we have brought about change? (Journals)

3.2. OM and Theory of changePolicies

and Strategies

Measures and

Mechanisms

Outcomes Goal

Progress Markers

Outcome Challenges

Organizational Practices

4. How to avoid that the ID becomes too heavy & Tips for writing progress

markers and working with strategy maps

4.1. Simplifying the Intentional Design: the case of VVOB zimbabwe

Two main challenges with the results of the OM workshop:1.Seven boundary partners within eleven teacher training colleges accross the country. This gives seventy seven boundary partner groups.2.Rather long lists of progress markers and support strategies were developed for each boundary partner during the initial outcome mapping workshops.

4.1. Simplifying the Intentional design: the case of VVOB Zimbabwe

The intentional design was adjusted in three major ways to make it more practical. 1.The various progress markers for each boundary partner were categorised in major categories that had emerged during the analysis of the baseline survey. This reduced the number of progress markers and also made the progress markers more comprehensive descriptions of process behaviours which were easier to monitor over time.2.The strategy maps were realigned to the programme’s intermediate result areas. This reduced the long lists of strategy maps and removed overlaps between identical support strategies for different boundary partners. (see handout)3.Organisational practices were included in the support strategies and are aligned to a specific intermediate result area on organisational learning at the level of the implementing organisation and the boundary partners.

4.1. Original list of progress markers

4.1. Outcomes of the baseline survey for co-curricular student clubs

Co-curricular student bodies

10

5

11

7

1

54

facilitating thefunctioning of the

club

providing OVCrelated services in

the college

networking communityoutreach

nu

mb

er o

f co

lleg

es

High

Medium

Low

4.1. Exercise: simplifying Intentional design (see handout)

• You are given the original list of progress markers for the co-curricular student bodies from the VVOB Zimbabwe case.

• Task: In groups, simplify the list by clustering the progress markers under the larger progress markers categories that had emerged from the baseline survey results: i.e. 1) functioning of the club; 2) providing services, 3) networking 4) community outreach.

4.2. What Can Patterns of Change Look Like?

Nyangaga Julius and Heidi Schaeffer

March 2011

4.2. PMs about Changes in Behaviour

• Changes in relationships• Changes in actions and interactions• Changes in practices• Changes in Policies

• Other?

4.2. A PM Framework• P1 Preparation for the Journey: building

Knowledge and Capacity • P2 The owned journey begins: building support,

collaboration and networks• P3 The owned journey proceeds: sustained

continuous actions. Institutionalization, Policies and/or Culture Change

4.2. Outcome Engineering• Level 1: Knowing that there is a journey to take (P1) • Level 2: Taking the first Steps (P1 level)• Level 3: Investing your own resources (all)• Level 4: Overcoming resistance to the change (all)• Level 5: Identifying with the journey by joining with

others with a similar approach (P2 level)• Level 6: Leaving a legacy (now an expert for others)

(P3 level).

4.2. 32 sets of progress markers

4.2. P1: Knowledge acquisition processes and practices

• ...attending forums where (the project) elaborates about the technology

• ...raising questions and issues that (the Project) will address to encourage (the BP’s) uptake of the technology

• ...seek out additional information on water and watershed issues from external sources

• ...requesting position papers from the relevant departments to solicit input into decisions

4.2. P2: Getting involved, build support & enroll others

• ...brokering or developing partnerships with other agencies to take local action

• ...establishing mechanisms to share and review work programmes across departments, especially on research projects

• ...establishing and expanding the membership base of the national organization in Indonesia

• ...organize ‘popular education’ to increase critical thinking of their members

4.2. P3: Owned journey continuous and sustained

• ...contribute to the improvement of the methodology internationally to continually make it even more effective

• ...generate their own funds and re-invest in (related) community projects

• ...developing and putting in place a communication policy guiding how information is shared within the organization

4.2. Change is continuous

4.2. Food System Case Story2 NGOs, Meal Exchange and Sierra Youth Coalition coordinate student efforts on ten university campuses with the aim of changing university food procurement policies to support local, sustainable food systems. Applied student research will be used to increase the collective understanding of local, sustainable food procurement.

4.2. Food System Case StoryA broad objective of the project is to bring together students and organizations addressing food issues in universities.

This will take place through increased effectiveness of work through the formation of campus food strategy groups and campus food ‘charters’ that set out guiding principles for sustainable food service.

4.2. Initial Evaluation Questions

• 1. Is the work affecting campus food procurement networks?

• 2. What lessons are we learning about our strategies and what is working well?

• 3. What changes are the campuses, MX, and SYC making together through the project?

• 4. What effect are the changes likely to have on our organizations (MX and SYC), our network of partners, and the broader FS movement?

4.2. Participatory Mapping with BPs

4.2. Actor-Centred PM&E The multi-stakeholder approach allows me to see this work in its complexity and the importance of understanding and working with people on the ground to make change. But also, to focus on fruitful relationships who actually want to support the work we're doing and will be leverage points for us

4.2. Baseline Institutional BPs

4.2. PMs for Institutional BPs 1=accept an invitation to participate in a meeting 2=request further information and share information with their colleagues 3=regular attendance at meetings4=commit to work on a shared project/goal5=appoint representative to the group or a formal committee with defined terms of reference

4.2. Progress Marker Tracking

CFSG Progress Tracking Template - Fall 2012 - 2nd Cohort

https://drive.google.com/folderview?docId=0Aj03SPx07ilJdHRjT3hYYzM2UXhXaG1WUHhKekdCY0E&ddrp=1&id=0BzJg_TZTjdA1WEhFWkxVTG5NYlU#

4.2. Challenges of Actor-Centred

“In our introductory relationship building we are encouraged to listen, state where we're coming from, that we're here to support, and not offer too much direction. Sometimes this is perceived as "wishy-washy" and our food services and contract service director would like clarity on what specifically we want. How can we be specific, yet receptive at the same time?

4.2. Canada World Youth Evaluation

• Knowledge; Organizational and Communication Skills; Learning Skills and Technical Skills

• 0=No Impact• 1=Very Small Impact• 2=Small Impact• 3=Moderate Impact• 4=Important Impact• 5=Very Important Impact

4.3 Six routes to supporting capacity development – a complementary approach towards strategy maps.

• Route 1 : Supporting organisation from the North (N-S)

• Route 2 : local capacity development provider

• Route 3: networks, platforms, alliances

• Route 4: Between partner organisations or other organisations from the South (S-S)

• Route 5: Organisations in the North or trainings in the North

• Route 6: the partner orgnaisation organises its own capacity development processes.

19-04-23 63

Supporting organisationSupporting

organisationsupporting organisation

Partner A

Final Beneficiaries

Local CD providers

1

2

2

3Partner B

Organisations in the North or trainings in the North

4 5

6

Network

4.3. Belgian experience – Frequency of the 6 routes among Belgian NGOs

0% 5% 10% 15% 20% 25% 30%

Route 1

Route 2

Route 3

Route 4

Route 5

Route 6

4.3. Exercise:

• Exercise: score yourself for the various routes of supporting capacity development (see handout)

Documents

Advanced Intentional Design Workshop OM LAB 2014