1

Cohesion Policy

2007 - 13

Ongoing Evaluation

Budapest, 7 May 2010

Veronica GaffeyHead of Evaluation

DG for Regional PolicyEuropean Commission

2

Cohesion Policy

2007 - 13

What is Ongoing Evaluation?What is Ongoing Evaluation?

• An ongoing programme of evaluation, designed to meet needs

• Complete flexibility – evaluate what you want (themes, priorities, programmes) when you want

3

Cohesion Policy

2007 - 13

From Mid Term to OngoingFrom Mid Term to Ongoing Evaluation (1)Evaluation (1)

• Mid Term Evaluation 2003 – Strengths:– Organisation and planning; Partnership– Allocation of significant resources– Increased evaluation capacity

• Weaknesses:– Rigid deadline– Breadth of requirements– Quality concerns– Methods not sufficiently rigorous (too descriptive, focus

on financial results, little generation of new evidence)

4

Cohesion Policy

2007 - 13

From Mid Term to Ongoing From Mid Term to Ongoing Evaluation (2)Evaluation (2)

• Rationale for Move towards Ongoing Evaluation:– Breadth militates against depth

– Lack of use of rigorous methods

– Evaluation as a result of obligation rather than needs

– Increased evaluation capacity

• Key elements:– Evaluation Plan – multi-annual, to be adapted according

to need (goal is good evaluations not perfect plans!)

– Need for anticipative approach

– Evaluation should become a management tool

5

Cohesion Policy

2007 - 13

Progress in Ongoing Progress in Ongoing Evaluation 1/2Evaluation 1/2

– Nearly all MS have evaluation plans (or plans to evaluate!)

– Evolving evaluation capacity – some “new” Member States now among the “best”

– (Assessment based on capacity, quality and extent of activity in cohesion policy evaluation)

6

Cohesion Policy

2007 - 13

POLAND

AUSTRIA

HUNGARY

SWEDEN

7

Cohesion Policy

2007 - 13

Progress in Ongoing Progress in Ongoing Evaluation 2/2Evaluation 2/2

– Increasing interest in EFFECTS

– Commission has established an expert evaluation network to synthesise results – annual reports from end 2010 onwards

– Interest in more rigorous methods – not possible to apply across all interventions

– Smaller scale evaluations can help open up the market

– Good practice examples will be published on Evalsed

– Ongoing development of methods section of Evalsed too

8

Cohesion Policy

2007 - 13

More Rigorous MethodsMore Rigorous Methods• Quasi-experimental design (randomised control

trials?)

• Control groups suitable only for some sectors (e.g., social programmes, enterprise support, urban?)

• Different solutions needed for transport, environmental infrastructure (ex post CBA?), innovation and complex programme effects

• Improvement of models – macro & sectoral

• Complemented by theory based evaluation approaches (why does it work?)

9

Cohesion Policy

2007 - 13

Looking ForwardLooking Forward• Commission undertakes exploratory work using quasi-

experimental techniques

• We encourage Member States, who might even experiment with RCT (France, Hungary, Poland, UK – Wales currently doing some quasi-experimental work)

• Similar approach starting on evaluating innovation

• Encourage more rigorous approaches in other sectors (e.g., ex post CBA)

• The approach to ongoing evaluation facilitates this

• Regular synthesis of evidence

• Sharing results and gradual building up of a body of more rigorous evaluation results (Commission role – support, publicity, seminars, “naming and praising”)