Embed Size (px)
Citation preview
Independent evaluation of
Paris Pact Phase III
A Partnership to counter the trafficking and
consumption of opiates originating in Afghanistan
Reconvened
Policy Consultative Group Meeting
Vienna, 3 September 2012
Mr Johannes Chudoba (Evaluation Team Leader)
Mr Marcio Carvalho (Evaluation Specialist)
Overview
• Introduction
• Methodology
• Background / Context
• Evaluation Findings
• Conclusions
• Recommendations
Introduction
• Independent two-month evaluation
• Two evaluators
• Emphasis on improving Phase IV
• Focus on effectiveness & engagement with stakeholders
• Scope:
• Paris Pact Initiative (PPI)
• UNODC GLO K31
Methodology
• Comprehensive qualitative and quantitative approach: • 50+ documents & ADAM reviewed, 123 responses on web-
survey, 64 interviews and 23 questionnaires & 1 field visit.
• 44 partner countries, 14 partner organisations and UNODC.
• Triangulation
• Participatory process
• Criteria: • Relevance
• Effectiveness
• Sustainability and Impact
• Partnerships and Cooperation
• Efficiency of UNODC Project GLO K31
Methodology Limitations and Constraints
• PPI is not a development project…
• Time:
• For the PPI/Project to show results in a 24-month framework
• For conducting the evaluation
• Data environment
• Impact
Background / Context
Paris Conference
• Paris Pact Phase I: Partnership / consultative framework
Moscow Conference
• Paris Pact Phase II: Strengthened analytical capacity in
priority locations for evidence based policies
• Paris Pact Phase III: Refining the tools: more focused
evidence for action
Vienna Conference
Global Afghan Opium Trade - A Threat Assessment, 2011
Global potential opium production, 1997-2011 (tons)
WDR 2012
Prevalence of the use of opioids (heroin, opium and
non-medical use of synthetic opioids) in 2010
WDR 2012
UNODC Project GLO K31
1. Consultative Mechanism
2. National Strategic Analysts (NSAs)
3. Automated Donor Assistance Mechanism (ADAM)
• Budget: USD 2,972,290
• Duration: 24 months from June 2010
Evaluation Findings
Relevance
• “The extent to which the objectives are consistent with
beneficiaries’ requirements, country needs, global priorities and
partners’ and donors’ policies.”
• Web survey: PPI remains the primary forum on opiates
originating in Afghanistan.
• Questionnaires: PPI is vital for continuously raising
awareness and building political will among stakeholders
• UNODC GLO K31 Project:
• “the Paris Pact could not exist without the project’s support”
Relevance
Effectiveness (PPI)
To serve a diverse international partnership as a platform
for information exchange; 88% 10%
To provide the venue for consultations among experts
and policy makers on issues related to opiates
originating from Afghanistan; 88% 9%
To provide evidence for strategic planning, policies, and
action; 81% 15%
To promote good practices on counter narcotics
enforcement and opiates demand reduction; 79% 17%
To identify weaknesses and recommend priority actions
at the geographic and thematic level; 86% 11%
To coordinate counter narcotics related technical
assistance. 73% 21%
Effectiveness
29%
21%
22%
28%
22%
26%
28%
26%
43%
39%
31%
33%
33%
36%
17%
13%
11%
13%
15%
17%
7%
10%
10%
8%
9%
9%
8%
6%
3%
2%
5%
7%
7%
5%
8%
14%
11%
15%
13%
14%
10%
16%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
InfoonPPIprocess
Mee nginforma on
Specialmessages/updates
Recommenda ons
TAprojects
Drugsandcrimenews
NSAsproducts*
Highlyrelevant Relevant Somewhatrelevant Limitedrelevance Notrelevant N/A
What kind of information provided on ADAM is relevant to your needs?
• 71% of respondents to the web survey indicated that the
project serves the partnership effectively and very
effectively.
• 81% of respondents that have participated in PPI meetings
rated UNODC’s role in the meetings as good or excellent.
Effectiveness UNODC Project GLO K31
Sustainability and Impact
• “The continuation of benefits after major development assistance has
been completed, and probability of continued long-term benefits.
• Positive and negative, primary and secondary long-term effect produced,
directly or indirectly, intended or unintended.”
• Among the most sustainable efforts are ones which are difficult
to quantify: improved communication, contacts and trust.
• Similarly, increased awareness, a stronger willingness to find a
common language, greater consensus, buy-in into a multi-lateral
process are all sustainable (though not irreversible).
Sustainability and Impact
Partnerships and Cooperation
73%
84%
91%
82%
67%
82%
17%
10%
6%
13%
19%
11%
Avoiding the duplication of efforts
Creating synergies among initiatives on
counter-narcotics
Enabling communication among experts
Enabling coordination among policy
makers to facilitate the…
Facilitating cooperation on joint
operations
Exchanging lessons learned andpromoting best practice
Perception of PPI support in terms of collaboration between partners
Efficiency of UNODC Project GLO K31
• Challenging to measure “efficiency”: results defined
broadly, “monetary value” of elements partners value most
is difficult to assess.
• Respondents were unable to respond directly on questions
of efficiency: Inferred that UNODC must support the
project well.
• Project budget allocated and spent as planned, with the
exception of the P3 position in the PPCU (not filled in
Phase III).
• Respondents mentioned good support for meetings, and
observed that the project has delivered more than could be
expected.
Conclusions
A.PPI is relevant and should continue. Most relevant value-added: PPI’s
contribution to convergence around balanced, comprehensive approach, and
political commitment generated.
B. PCG: Providing consistent, clear, solution-focused guidance, building on
expertise of EWGs; has a particular responsibility to lead partners’ efforts.
C.EWGs: more focused & better geared to providing actionable
recommendations following 2009 evaluation; room for improvement remains:
follow-up.
D.NSAs connect policymakers, experts & projects, are main contributors to
ADAM. Further efforts to connect them better with partners & projects under way.
E. ADAM: One of the most advanced tools of its kind; usage has increased in
Phase III, but is still below critical mass. Considerable room for further improvement
remains in terms of usage.
F. Connection between capitals/headquarters and the field, between policy-
makers and experts, and between donors and priority countries, deserves to be
clarified and strengthened, as one of the potential greatest benefits of the PPI.
G.PPI has been relying on a relatively narrow funding base compared to the
number of partners; the implementation period of just 24 months has
complicated project logistics.
Recommendations
1. Build on the Vienna Declaration: Clearer focus; renewed
commitment; balanced, comprehensive approach; more
consistent follow-up.
2. Project document for Phase IV: Clear, actionable
objectives & responsibilities for stakeholders;
strengthened PPCU to facilitate follow up.
3. Reconfirm roles of PCG, EWGs and UNODC; consistent
chairs for PCG and EWGs a year in advance; focal points
in capitals to follow up between meetings;
4. PPCU and NSAs to facilitate follow up with chairs and
partners, including beyond Vienna.
5. Nominate experts to rosters / consistently send them to
EWGs; strengthen follow-up mechanisms via PPCU;
6. PPCU: maintain rosters and facilitate follow-up to EWG
recommendations.
7. Define and endorse NSA’s role in Phase IV as “liaison
officers” (continuing research & capacity-building);
additional countries?;
8. UNODC: better define role of NSAs & integrate them
further with other projects.
9. Engage more directly in using ADAM for interaction
between meetings, including to facilitate preparation and
follow-up;
10. PPCU to conduct thorough assessment of best use of
ADAM as an information platform.
11. Consistently keep experts in priority countries engaged in
the PP process; and ensure follow-up and sustainability;
12. UNODC: strengthen PPCU’s connection with other
projects.
13. Enable PPCU’s work through more predictable,
equitable, long-term funding;
14. UNODC to consult with partners on long-term funding
for a Phase IV of four years.
Main Issues
• Clarity / commitment
• Consistency / follow-up
• Capacity / resources
Independent evaluation of
Paris Pact Phase III
Mr Johannes Chudoba (Evaluation Team Leader)
Mr Marcio Carvalho (Evaluation Specialist)
