A Logic Model for

Evaluation Capacity Development

draft !

Abridged version of slides presented at the

Canadian Evaluation Society 2011 Conference Edmonton, Alberta

May 2nd

by

Michael Obrecht and Robyn Sachs with acknowledgement of the contribution of

Wendy Ryan

Notes on this Abridged Version of Slides

•  This version focuses on ideas from the literature and our draft logic model

•  The version that we presented at the conference also included slides on: –  The evaluation capacity development project that inspired our

work; and,

–  Models developed by other authors

Why a Logic Model?

•  A simple logic model for evaluation capacity development should be useful when:

–  Assessing needs for development of evaluation capacity

–  Discussing approaches to improving evaluation capacity

–  Examining progress in capacity development

•  A logic model will complement other types of model that are already documented in the evaluation literature

Drafting the Logic Model

•  We brainstormed on inputs, activities, outputs, and outcomes by reflecting on our experiences with evaluation capacity development in India and Canada

•  We reviewed the literature on evaluation capacity development

•  We cross-checked our model against existing models

Points from the Literature

•  Up-front specification of the goals for evaluation capacity development is crucial

•  Evaluation capacity development is a long, slow process

•  It involves consideration of education, experience, motivation, personality, and soft-skills of participants

•  A balance of formal training, mentorship, and hands-on experience is required

•  Two or more years of capacity development are needed before someone is ready to lead an evaluation

… continued

•  Organizational culture, attitude, and readiness will affect evaluation capacity development

•  Evaluation capacity development can stimulate organizational learning and change

•  Evaluation use reinforces capacity development

•  It can be especially challenging to develop evaluation capacity in a government environment

First Draft of a Logic Model

•  We focused on getting the broad categories clear

•  Our working definitions of inputs, activities, outputs and outcomes were flexible

•  Our exposition of the draft model will focus on inputs and activities

People

I 

Structures

Context

•  Identify •  Engage •  Assess •  Train •  Mentor •  Conduct

I •  Plan •  Govern •  Monitor •  Inspire

I •  Adapt

•  Staff •  Managers •  Capabilities •  Frameworks •  Evaluations

I •  Plans •  Policies •  Assurance •  Receptivity

I •  Adaptation

Sustainable  evalua.on prac.ce   

•  Portfolios •  Experience •  Careers

I •  Stability •  Use •  Demand

I •  Flexiblity

    Inputs      Ac*vi*es      Outputs                    Outcomes 

Structures

People

I 

Structures

Context

•  Identify •  Engage •  Assess •  Train •  Mentor •  Conduct

I •  Plan •  Govern •  Monitor •  Inspire

I •  Adapt

•  Staff •  Managers •  Capabilities •  Frameworks •  Evaluations

I •  Plans •  Policies •  Assurance •  Receptivity

I •  Adaptation

Sustainable  evalua.on prac.ce   

•  Portfolios •  Experience •  Careers

I •  Stability •  Use •  Demand

I •  Flexibility

    Inputs      Ac.vi.es      Outputs                    Outcomes 

Structures

Availability

Skills and experience

Motivation

Interests

People

I 

Structures

Context

•  Identify •  Engage •  Assess •  Train •  Mentor •  Conduct

I •  Plan •  Govern •  Monitor •  Inspire

I •  Adapt

•  Staff •  Managers •  Capabilities •  Frameworks •  Evaluations

I •  Plans •  Policies •  Assurance •  Receptivity

I •  Adaptation

Sustainable  evalua.on prac.ce   

•  Portfolios •  Experience •  Careers

I •  Stability •  Use •  Demand

I •  Flexibility

    Inputs      Ac.vi.es      Outputs                    Outcomes 

Structures

Resources

Reporting Relationships

Senior Management

support

Understanding

Demand

People

I 

Structures

Context

•  Identify •  Engage •  Assess •  Train •  Mentor •  Conduct

I •  Plan •  Govern •  Monitor •  Inspire

I •  Adapt

•  Staff •  Managers •  Capabilities •  Frameworks •  Evaluations

I •  Plans •  Policies •  Assurance •  Receptivity

I •  Adaptation

Sustainable  evalua.on prac.ce   

•  Portfolios •  Experience •  Careers

I •  Stability •  Use •  Demand

I •  Flexibility

    Inputs      Ac.vi.es      Outputs                    Outcomes 

Structures

Culture

Legislation

Politics

People

I 

Structures

Context

•  Identify •  Engage •  Assess •  Train •  Mentor •  Conduct

I •  Plan •  Govern •  Monitor •  Inspire

I •  Adapt

•  Staff •  Managers •  Capabilities •  Frameworks •  Evaluations

I •  Plans •  Policies •  Assurance •  Receptivity

I •  Adaptation

Sustainable  evalua.on prac.ce   

•  Portfolios •  Experience •  Careers

I •  Stability •  Use •  Demand

I •  Flexibility

    Inputs      Ac.vi.es      Outputs                    Outcomes 

Structures

Identify need for personnel

Reassign or recruit

Determine developmental needs

Provide training and mentoring

Conduct evaluative work

People

I 

Structures

Context

•  Identify •  Engage •  Assess •  Train •  Mentor •  Conduct

I •  Plan •  Govern •  Monitor •  Inspire

I •  Adapt

•  Staff •  Managers •  Capabilities •  Frameworks •  Evaluations

I •  Plans •  Policies •  Assurance •  Receptivity

I •  Adaptation

Sustainable  evalua.on prac.ce   

•  Portfolios •  Experience •  Careers

I •  Stability •  Use •  Demand

I •  Flexibility

    Inputs      Ac.vi.es      Outputs                    Outcomes 

Structures

Overall plan for the ECD

Evaluation and monitoring policy

Evaluation planning

Evaluation quality control

Support innovation and creativity

G 

People

I 

Structures

Context

•  Identify •  Engage •  Assess •  Train •  Mentor •  Conduct

I •  Plan •  Govern •  Monitor •  Inspire

I •  Adapt

•  Staff •  Managers •  Capabilities •  Frameworks •  Evaluations

I •  Plans •  Policies •  Assurance •  Receptivity

I •  Adaptation

Sustainable  evalua.on prac.ce   

•  Portfolios •  Experience •  Careers

I •  Stability •  Use •  Demand

I •  Flexibility

    Inputs      Ac.vi.es      Outputs                    Outcomes 

Structures

Identify response to critical cultural and other environmental

variables

G 

Recommended Readings

•  Special issue of the Canadian Journal of Program Evaluation on evaluation capacity building: –  Volume 23, Number 3, 2008

•  Preskill, H., and Boyle, S. A Multidisciplinary Model of Evaluation Capacity Development. American Journal of Evaluation (Online). September 2008

•  We welcome your feedback!

•  For further information:

–  Michael Obrecht michael@obrecht.ca

–  Robyn Sachs sachs.robyn@gmail.com