THEMOST SIGNIFICANT CHANGE

(MSC) TECHNIQUE

RARE, 2013 - Assessment Task 2

A dialogical approach to information gathering for monitoring and evaluation.

Group Presentation Member’s Profiles

Dorothy - located in Dili, East Timor.

Working in international aid within the security sector.

Karen - located in Jakarta, Indonesia. Currently working in international aid within the education sector on programs funded through Australian Aid.

Carol - located in rural Victoria, Australia. Working for an NGO within the welfare sector.

Our Stories

Presentation Content

1. Description of MSC.

2. Purpose of MSC.

3. Appropriate contexts and uses of MSC, and inappropriate contexts and uses of MSC.

4. Strengths of MSC.

5. Weaknesses of MSC.

6. Example of the MSC approach.

7. MSC activity for the class.

Introduction

• The Most Significant Change technique (MSC), is located in the monitoring form of evaluative inquiry.

• MSC can be used in a program cycle for monitoring and for program evaluation.

• MSC collects qualitative data rather than the traditional quantitative reporting in program monitoring.

1. What is MSC?

Planning

Implementation

Monitoring

Outputs

Quantitative Reporting

The Most Significant

Change technique is

a participatory form

of program

monitoring and

evaluation.

Typical Program

Cycle

…What is MSC?

• MSC contributes to program monitoring and evaluation processes through capturing participants stories, in their own words.

• The story recording and subsequent hierarchical analysis assist workers and other stakeholders to understand the impacts of the program.

• Program improvement and worker learning are thus enhanced through the MSC technique.

…The MSC Story

MSC Time Line

1994: Davies began to develop the methodIn Bangladesh

1996: Called the ‘Evolutionary Approach to Organizational Learning’ (Davies)

1998: Termed ‘monitoring without indicators’ (Guijt, Arevalo, & Salsdores)

1999: Also known as ‘The Story Approach’ (Dart)

1999: First used in Australia, called the ‘Most Significant Change’ Technique (MSC) Dart et. al.

2009: Further use in Australia – The Ambassador Project, Whitehorse, Victoria

2000: MSC approach internet discussion group established

2005: Published in the American Journal of Evaluation (Dart & Davies)

2. What does the Most Significant Change (MSC) technique do?

• Uses a concept of ‘dynamic values inquiry’.

• Searches for, and deliberates over, the value of significant program outcomes.

• Directs work toward explicitly valued directions and away from less valued directions.

…Collecting SC Stories

Formulating an open ended question to collect Significant Change (SC) stories:

1. “Looking back over the last month…

2. …what do you think was…

3. ..the most significant…

4. …change…

5. …in the quality of people’s lives…

6. …in this community?”

…The MSC Process

Participant stories collected

Stories discussed & placed in domains

SC stories chosen & reasons stated for choices

Qualitative program outcomes and impacts

Verification, quantification & report

MSC Program

Cycle

Incorporates feedback loops within the project, and/or organisational hierarchy.

…MSC Valued Directions

Voices of participant's

Workers hear effects of work and reflect on practice

Stages of participatory discussion, review and SC selection

Selection group reflects on practice

Funders & managers reflect on desirable outcomes

A picture of what participant's value, stakeholder’s value, and funder’s value.

Cycle of improvement

& learning

3. Where does MSC fit?

Introduction

• The evaluator must always choose approaches and methods which are the most appropriate for the situation.

• MSC implementation can be adapted to suit some situation specific characteristics.

• However, MSC is more useful and better aligned to certain contexts than others.

…Where does MSC fit?

Everywhere? Anywhere?

• MSC has been implemented in developing and developed countries (e.g. Australia and Solomon Islands).

• Across the range of organisations and institutions e.g. government, community groups and businesses.

Therefore, is MSC appropriate for any country, program, project or institution?

…Where does MSC fit?

Contexts

Two elements to consider for appropriateness of context for MSC for the program, project or institution are:

• Level of complexity.

• Internal characteristics.

…Where does MSC fit?

Context - Level of Complexity

Appropriate Inappropriate

Highly Complex Simple

Multilevel Single level

Multiple Stakeholders Few Stakeholders

Divergent Values Focussed Values

Conflicting Opinions Agreement

Ambiguity Clarity

…Where does MSC fit?

Context - Internal Characteristics

Characteristic Appropriate Inappropriate

Culture of Program, Project or Institution

Learning Culture Accountability Culture

Improvement Oriented Success Oriented

Open to New Approaches

Closed to New Approaches

Understanding of MSC

Accurate Inaccurate

Realistic Expectations Misguided Expectations

Initial Pilot Attempt Full Implementation

…Where does MSC fit?

Context - Internal Characteristics

Characteristic Appropriate Inappropriate

People Leadership Stable Changing Leadership

Senior Personnel Support Commitment and Involvement

No Senior Personnel Support, Commitment or Involvement

Resourcing Adequate finances Considers MSC ‘cheap’ with inadequate provision

Knowledgeable Personnel

Considers MSC ‘easy’ -lacking knowledge

Realistic allocation of time for all MSC phases

Considers MSC ‘quick’ with unrealistic time allocations

…Where does MSC fit?

For anything?

• It is possible to use MSC for planning, reporting, public relations, communication, monitoring and evaluation.

But what are the more appropriate uses for MSC?

…Where does MSC fit?

Uses

Appropriate Inappropriate

Complementary technique within a suite of M & E methods

Only technique used for M & E

Focus on extreme experience Focus on average experience

Program Improvements Retrospective evaluation for a finalised program or project

Reflective Practice

Program Outcomes and Impacts

Fluid & Emerging Outcomes Known Outcomes

Changing Focus Static Focus

…Where does MSC fit?

Uses

Appropriate Inappropriate

Capturing Unexpected Change Changes Expected

Attaining Shared Vision Vision Established

Participatory Approach Confidentiality Issues

Evaluation Capacity Development Evaluation Knowledge with Experts

Capturing ‘voice’ of beneficiaries Ignoring beneficiaries

Qualitative – Hearts & Minds Quantitative – Statistics & Numbers

4. Strengths of MSC

• Inclusive and participatory at all levels.

• Transparent.

• Reflects a strong story telling tradition: Common in many non-Western cultures.

• Contributes to organisational learning:Provides opportunities to discuss what the project was trying to achieve and how it can be improved.

Can help improve program management.

• Non-threatening:No right or wrong.Genuine input from participant’s perspectives.

…Strengths of MSC

• Highlights unexpected changes.

• Provides data on impact and outcomes.

• Highlights social benefits.

…Strengths of MSC

5. Weaknesses of MSC

• Costly:Time.Human resources.

• Captures extreme cases – not the average.

• Validity and reliability of stories.

• Language – can be a barrier/impediment to the process.

• Difficulties with domains – “fuzzy”.

…Weaknesses of MSC

6. MSC In Action

Check out this link to see a great example of MSC in action:

http://www.youtube.com/watch?v=NkuJ69zKScU

To engage you further in the MSC technique we have put together some activities for you to try.

Please refer to the activity section within the written component of our MSC presentation.

7. MSC Class Activity