Developing, Testing and Measuring Theories of Change:The WEIMI Experience to Date

Mary Picard, ConsultantCARE USA, Gender Unit31 Jan. 2012

WOMEN’S EMPOWERMENT IMPACT MEASUREMENT INITIATIVE

Content

Who and what is WEIMI?Progress on developing theories of change for a women’s empowerment programProgress on testing theories of changeProgress on measuring theories of changeSpotlightingCommon challengesCommon issuesNext phase

Who and what is WEIMI?

Core objective:to develop the necessary capacity, tools, guidance and practice to measure and demonstrate the impact of CARE’s work on women’s empowerment

First developed by the Program Impact Unit; moved to the Gender Unit

Who and what is WEIMI?• During this period, 4 COs – Niger, Bangladesh, Egypt and

Tanzania – were directly assisted through CO visits and considerable work done on their theories of change and indicators

• Each of these countries has peripheral learning partners

Progress: Developing a Theory of ChangeMeasurability depends on the completion of these elements:

Sub-impact groupsImpact GroupImpact Goal

Impact + DOC-level indicators

Pathways of Change

Domains of Change

Critical Hypotheses Breakthroughs Contextual

Analysis

A Coherent Set of Initiatives

CARE Tanzania Example

IMPACT GROUP: Marginalized and vulnerable women and girls living in rural underserved and environmentally restricted areas at critical life stagesSub-impact groups:

Who are agriculture and forest-dependentMining or industry-dependent or living adjacent to minesFisheries-dependentPastoralist and agro-pastoralist

Impact Goal Marginalized and vulnerable women and girls, at critical life stages, in rural under-served and environmentally restricted areas are empowered to live sustainable, healthy and secure lives.

Indicator Prevalence of underweight children under 5 years of age (MDI impact under Food Security)

Domain of Change (DOC)

DOC 1:The Impact Group (IG) has access to basic services, resources, skills, knowledge and confidence to diversify their livelihoods, and become resilient to economic, political and environmental shocks.

DOC 2:Cultural and social norms recognize and uphold rights of the impact group, enabling them to participate equally in family and community decision-making.

DOC 3:Civil society, private sector, local and national governance systems and institutions are responsive to the needs and rights of the IG.

DOC 4:Critical ecosystems and natural resources (forest, marine, watersheds, agricultural and range lands) on which marginalized and vulnerable women & girls depend are healthy and intact.

Indicators per Domain

% households that do not rely solely on agriculture for their livelihood, by sex of the head of household (MDI outcome #3)

% men and women reporting meaningful participation of women in decision-making in a domain previously held by men (domain__________)(MDI outcome #11)

Number of effective spaces in which the IG participations meaningfully (planning, budgeting, monitoring)(place holder for governance indicators; current draft #8)

% local actors with meaningful participation in productive, natural resource management at community level, by sex(MDI outcome #31)

CARE TANZANIA THEORY OF CHANGE

Pathways under DOC 1:

The IG has access to basic services, resources, skills, knowledge and confidence to diversify their livelihoods, and become resilient to economic, political and environmental shocks.

Indicator:

P1: IG is using quality basic services (education, health, ).

% of rural population reporting to be satisfied with basic services disaggregated by sex (adjusted from MKUKUTA II, Cluster 3/goal 2 Indicator)

P2: IG implements sustainable integrated land use plans and practices.

Proportion of households implementing integrated sustainable land use plans disaggregated by sex of the head of the household.

P3: IG is utilizing opportunities for the IG for on- and off-farm economic activities as a result of accessing and controlling productive resources.

% of IG who have one or more profitable on-farm and off-farm economic activities (MKUKUTA cluster 1 indicator # 22, does not exist in MKUKUTA II any more)

P4: IG is able to predict and prepare for economic, political and environmental shocks.

% households with capacity to cope with environmental shocks without depleting their assets, by sex of household head.(MDI + outcome #4)

Progress: Developing a Theory of Change

Other elements of a program are not conditional to testing the theory of change but complete the development of the program:

A Situational analysisIdentification of target groupsIdentification of stakeholder groupsStrategies or promising practices

Progress: Developing a Theory of Change

Where are pilot countries now in developing the theory of change?

Still transitioning towards testing the ToC

Progress: Testing the Theory of Change

All countries have undergone some sort of matrix exercise to link the current set of initiatives to the theory of change, usually to pathways. It is one level of hypothesis testing.

Progress: Measuring the Theory of Change

• Establishing an impact measurement system includes attention to:• Data Management• A knowledge management & learning strategy• Harmonization of M&E systems between projects and

programs• A set of guidelines on M&E to align projects to programs• A central database that is rationalized to the program

• Ongoing Analysis, Reflection and Learning• Staff Capacity and Systems

Progress: Measuring the Theory of Change

• Progress on Data Management• All 4 COs have had an Impact Measurement

Readiness Assessment (IMRA) done by Tom Barton• Increasing overlap and synergy between projects

contributing to the same program• Niger has developed a KM & L strategy at CO level

and progressed much further in data mgt• Some headway on IT front to facilitate shared

access to documents

Progress: Measuring the Theory of Change

Ongoing Analysis, Reflection & Learning• Which involves (a) structuring and systematizing

the process of gathering people for reflection “in action” and “on action”; and (b) a process for routine documentation of learnings

• The IMRA is a foundation for making needed changes.

• COs are aware of what needs to change and are in the process of adapting their behaviors or routines.

Progress: Measuring the Theory of Change

Staff Capacity and Systems• Almost all have re-aligned their staffing

structure• Most are still working hard to re-align program

support systems to the program approach• Most are aligning their design or new business

with program theories of change

SpotlightingGood exchange between pilot country and learning partners, e.g., Niger with Benin and MaliNiger measured “self-efficacy” as an impact indicator in its baseline study for its CO ToCBangladesh will be testing a methodology for contextual analysis

SpotlightingTanzania finalized critical hypothesis:

Women in VSL groups build the self-esteem, social and economic capital that enables/shifts in gender roles in HH and community, which makes them to [sic] raise their voices and participate in the community decision-making.

Both Niger and Egypt have a similar hypothesis to this

Common Issues

• Domains of change that appear commonly:• Increased access to basic services and skills• Access to economic opportunities • Change in socio-cultural norms or practices• Civil society and/or government institutions are responsive

• Common indicators:• % of men and women reporting ability of women to

effectively control productive assets• % of men and women with changed attitudes towards

gender-based violence • % of men and women reporting meaningful participation of

women in public sphere (in adapted forms)

Common Themes

These appear explicitly at different levels of the theories of change:

Importance of laws that support gender equityWomen’s access to land (and other natural resources)Gender relations in the household Women’s access to SRH rightsThe role of civil society actors in the social change process Women and girls’ access to education services Women’s participation in the public sphere

Common Challenges

Not enough time to focus on thisGetting focus – e.g., prioritizing and validating one or two critical hypotheses for testingFinalizing and validating indicatorsUnderstanding breakthroughsRationalizing pathways of changeRefining and prioritizing “critical hypotheses”Facilitating a shared understanding of the work across the Country OfficeFunding for this work

Next Steps

Include and capture the experience from learning partnersFinding points of connection with other initiatives in CARE (e.g., Pathways, WE-RISE) in order to consolidate experiencesGenerating exchange based on common interest, within and beyond WEIMI COs / partnersDocumenting the experienceFinding creative funding opportunities

THE END

THANK YOU!