ETF Yearbook 2008 - policy learning in action

8/14/2019 ETF Yearbook 2008 - policy learning in action

1/136

ISSN: 1725-9061

ETF YEARBOOK 2008

POLICY LEARNING IN ACTION


2/136

THE EUROPEAN TRAINING FOUNDATION (E

HELPS TRANSITION AND DEVELOPING COTO HARNESS THE POTENTIAL OF THEIR HRESOURCES THROUGH THE REFORM OF ETRAINING AND LABOUR MARKET SYSTEMCONTEXT OF THE EUS EXTERNAL RELATI


3/136

ETF YEARBOOK 2008POLICY LEARNING IN ACTION


4/136

Europe Direct is a service to help you

find answers to your questionsabout the European Union

Freephone number (*):

00 800 6 7 8 9 10 11

(*) Certain mobile telephone operators do not allowaccess to 00 800 numbers or these calls may be billed.


5/136

FOREWORD

As we put the finishing touches to this fifthand final ETF Yearbook, it is interesting toreflect on the way the publication hasevolved. We started in 2004 with a verystraightforward aim: to give ETF operationalstaff the opportunity to present analyses andinsights based on their project or country

experience. Not only would this encouragecolleagues to reflect on their work but itwould also provide readers with an insightinto critical issues related to the reform ofvocational education and labour markets intransition countries.

Over the following two years, we decided tofocus that reflective practice on a specific

theme. In the 2005 Yearbook we chose towrite on the dual role of teachers and trainersin reform; while in 2006 we switched ourfocus to the contribution that skillsdevelopment can make to poverty reduction.Through these themes, we continued toapproach the writing of the Yearbook as anopportunity to bring into focus the ETFsparticular approach to working with our

partner countries through the methodology ofpolicy learning.

Perhaps it was inevitable therefore that the2007 and 2008 Yearbooks would lookmore deeply at that methodology to theextent that policy learning itself became the

methodology on which we haincreasingly based our work aourselves to others in our intecommunity.

The process of producing thehas been a developmental ex

the ETF and could be said to progress from that of a technassistance body to that of a rcentre of expertise in human development. The lessons wlearned along the way are onmuch wider application than twhich the ETF works and so our readers will be able to be

applying the reflections on outo their own working contextsthough these may be.

One aspect of the ETFs evolyears has been our increasinwith other multi-lateral institutthe community of thinkers whwork. Therefore it was approp

should invite colleagues frominternational community to coreflections on policy learning.to thank our four external conPart 2 for accepting the invitathereby adding to the value thpublication brings to our colle


6/136


7/136

CONTENTS

FOREWORD

PART 1

INTRODUCTION: MAKING POLICY LEARNING WORK

1. WHAT CAN WE LEARN FROM POLICY LEARNING?

Borhne Chakroun

2. THE ETFS INTERVENTIONS IN JORDAN, A FINE LINE BETWEEN ADVICE AGUIDANCE

Grard Mayen

3. POLICY LEARNING THE EXPERIENCE OF IMPACT ANALYSIS IN TURKEY

Sren Nielsen, Outi Krkkinen, Recep Varcin and Arjen Vos

4. ETF PEER LEARNING: FROM POLICY LEARNING TO POLICY CHANGE INPARTNER COUNTRIES

Margareta Nikolovska and Arjen Vos

5. DEVELOPING KYRGYZ VET POLICY AND STRATEGY THE CHALLENGE O

FACILITATING POLICY LEARNING PROCESSES


8/136

PART 2

INTRODUCTION: HOPES AND PROMISES OF POLICY LEARNING

8. THE PROMISES AND PITFALLS OF PEER LEARNING

Ronald G. Sultana

9. SNAPSHOTS OF POLICYMAKING IN A CHANGING ENVIRONMENT

Slavko Gaber

10. LEADERSHIP AND POLICY LEARNING COMMUNITIES: PROMOTIN

ANIMATION

Louise Stoll

11. ENGAGING POLICY: NEITHER A BORROWER NOR A LENDER BE

Andy Hargreaves

12. LETTER TO A NEW EDUCATION MINISTER

AFTERWORD

Peter Greenwood

BIBLIOGRAPHY PART 2

ETF YEARBOOK 2008 POLICY LEARNING IN ACTION


9/136

INTRODUCTION:MAKING POLICY LEARNING

WORK

Debate is increasing among donors andrecipient countries on the need to secureownership of development policies bysupporting local policymakers and otherstakeholders in leading policy developmentand implementation. At the same time,concern is growing about the capacity ofgovernment (and other stakeholders andinstitutions) to design and lead reforms.

Several approaches in the European Unionand elsewhere attempt to theorise andprovide methodological tools forinvestigating the process by which

Sector-Wide Approach (SWArequires a stable macro-econsituation, realistic sector stratcoordination of external suppgovernments. All these develled the ETF to adopt policy lemajor principle of cooperationcountries.

Policy learning emphasises thengagement of national stakedeveloping their own policiesimplementation plans. It also the active engagement of the

Part 1


10/136

can lead to sustainable development

policies in partner countries.The first part of the Yearbookdrawslessons from an unprecedented andsystematic approach to putting policylearning into action. It is particularlyunprecedented in that, first, it coverscountries in transition whereas internationalliterature on policy learning refers mainly to

Western countries; second, it covers awide range of countries and regions(Morocco, Kyrgyzstan, the Western Balkancountries, and Turkey) with differentcontexts and governance systems.

The particular focus of all articles in thisfirst part of the yearbook is on vocationaleducation and training (VET). Initialexperiences are presented from ETFattempts to facilitate VET reforms usingpolicy learning principles in a number ofcountries. The authors invited to contributeto this volume are representative of thatlarger group in that they see the decisiveinfluence of policy learning principles ontheir work in and with partner countries.

They were all directly involved in theprojects and initiatives described. Somechapters are the result of collective workinvolving colleagues from the ETF andexperts from partner countries.

In the opening chapter, Borhne Chakrounpresents a synthesis of policy learning fromtheoretical and practical perspectives. He

argues that policy learning inevitably setsup different kinds of tension: between theprocess and the end result, between thetime needed for learning and the urgentneed to develop policy and between therole of facilitator and advisor. He concludes

practitioner, the tensio

and advising roles.In Chapter 3, Sren NOuti Krkkinen and AGreek concept of agomarket and politics mprivate emotions and opinion and political cthat an innovative eva

a sharper focus on orlearning platforms mastakeholder involvemnetwork learning, leadconsistent follow-up d

Margareta Nikolovskoffer another angle finterpret policy learnETF peer learning prfurther insight into hoprinciples can be appinternational experieexamines several pein the Western Balkapeer learning procesfurther by critically q

between peer learninin the countries conc

In Chapter 5, Vaclav Beishembaeva and San attempt to supporttraining reform in Kyrgpolicy learning approaETF. They focus spec

elements of policy lea(i) rationale behind thlearning principles in VET reform; (ii) main change; (iii) lessons rfacilitation role of the



11/136

always work in practice as it has beendesigned in theory.

In the final chapter of Part 1,Jean-Raymond Masson and Slava PevecGrm apply the policy learning perspectiveto analyse ETF experience in new EUmember states. The authors first look atcommon definitions used in EU policymessages and how these match the

reality of partner countries, identifyingmisconceptions and ambiguities that candegrade the effectiveness of support.They then examine more closely theexperience of Slovenia, which receivedEU support throughout the 1990s and theearly years of the new millennium, in anattempt to distil lessons which are thenfed back into a general conclusion and aset of recommendations for EU assistancefor VET reform and for ETF work ingeneral.

ABOUT THE AUTHOR

Mounir Baati, Anar Beishem

Borhne Chakroun, Outi K

Vaclav Klenha, Jean-Raymo

Grard Mayen, Sren Nielse

Nikolovska and Arjen Vos aworking at the European TraiFoundation.

Slava Pevec Grm is AssistanDevelopment at the SloveniaInstitute for Vocational EducaTraining.

Peter Schuh has been involvresources development projedeveloping countries for over

Recep Varcin is Associate PAnkara University and DirectoAnkara University Centre for human resources managemecounselling.

INTRODUCTION: MAKING POLICY LEAR


12/136


13/136

1. WHAT CAN WE LEARNFROM POLICY LEARNING?

Borhne Chakroun

1. INTRODUCTION

This yearbook takes a close look at thestrategic approach the ETF has adoptedfor its interventions in partner countries:policy learning. It is also an attempt by ateam of believers to define what is meant

by the term and what it can offer topolicymakers. This effort is necessary if weare to save the concept of policy learningfrom becoming a mere buzzword usedindiscriminately in all situations.

This opening chapter aims to draw lessons

This chapter attempts to answquestions about policy learnin

1. What can we learn from apolicy learning into practicrange of partner countries

2. To what extent does policy

to policy change in partne

The first section explores the basis of policy learning and thwhy it can sometimes fail to dcontinues with a look at how come to adopt policy learning


14/136

policy learning. They link the emergence ofpolicy learning to the desire to challengeconflict-oriented theories. The review takes inthe following concepts: political learningdeveloped by Heclo (1974), policy-orientedlearning developed by Sabatier (1987),lesson-drawing analysed by Rose (1991),social learning discussed by Hall (1988)and government learning developed byEtheredge and Short (1983). Bennett and

Howlett conclude that the concept of policylearning has been overtheorised andunderapplied and that the relationshipbetween policy learning and policy changecontours and components has only begun tobe investigated and understood.

Although it is useful to refer to the ideas ofthe above mentioned authors, I would like

to retain another perspective, put forwardby Raffe and Spours (2007) and Grootings(2004), that sees policy learning as a wayfor governments or systems of governanceto inform policy development by drawinglessons from available evidence andexperience.

In the context of the ETFs partnercountries this means examining to whatextent policy learning approaches havehelped those countries to inform policy.This includes the capacity of partnercountries to learn from their ownexperience and from that of othercountries. Thus policy learning should leadto policy change and increasing policy

effectiveness (Grootings, 2004).

Although the words policy and learning areby no means newcomers to the debate onhow to reform VET systems, in recentyears they have moved from the periphery

of its importance for pit may not always be ninstance, many donorpiecemeal approach iprojects in favour of aapproach. This aims fsustainability and ownpolicies (Europeaid, 2Declaration on Aid Efa recent example of t

of the declaration, minand developing countbe guided by developpriorities established when it comes to choeffective ways of deliv

I consider it useful to ewhat we can call polic

partner countries befoETF decided to put popractice and to use it afor its interventions in

1.2 Policy learning f

countries

Policy learning failurein part, reflects the faborrowing from abroapolicymakers to find fproblems (see Philipsdetailed discussion). Sbriefly discusses the fmodel of supporting pless developed count

transfer of knowledge

John Deweys theory perhaps throw some efforts to understand (Dewey, 1938, p.28).



15/136

experiences also runs the risk of artificiallygenerating dispersive, disintegrated,centrifugal habits.

International development aid provides awealth of examples of these kinds ofexperiences. How many countries, forexample, have benefited from thegenerous gift of a ready-made curriculumand skills needs analysis when their ability

to make good use of it is extremely limited?How many countries have decided toimport the foreign VET model only to find itso alien to their specific context so as to bepractically useless? How many ministrieshave been plagued by infighting betweendifferent project management units so thatevery donor-funded project ends up beingtotally disconnected from the rest?

In their discussion of policy learning failurein secondary education in the UK, Raffeand Spours (2007) found that it wascaused by the inability and unwillingness tolearn from past experience and by learningfrom the experience of other countries butonly superficially. Chapter 5 mentions the

loss of policy memory in Kyrgyzstan thatoccurred after the DACUM curriculareforms. Chapter 6 also mentions thedifficulties in helping the Moroccan teamlearn from past experience in spite of thepolicy learning nature of the whole process.

Policy learning failure is also linked to themodel of governance in place in each

partner country (see for example thediscussion of the Arab HumanDevelopment Report, 2005). In mostpartner countries, the process ofpolicymaking is far from transparent.Policymakers often work with little

organisations and to the ETFwith its long experience of esobservatory functions in moscountries (see Chapter 7 for discussion of this). Three imelements tend to be weak or almost all partner countries: systematic data collection anpolicy evaluation studies andWhile the first two points are

systematically highlighted (spress), the research dimensialways been ignored. In almocountries, research capacitietotally lacking, thus deprivingpolicymakers of a vital sourcknowledge and expertise.

Therefore it seems there are

overlapping factors that lead learning failure in partner coumisleading experiences causintervention, highly politicisedgovernance which are not copolicy learning and the limitedbase.

2. THE ETFS APPROPOLICY LEARNING

The beginnings of policy learintervention approach for the traced back to the ETF Advismeeting in 2003 where the coattracted considerable attenti

result, the ETF and its partneadopt policy learning as a toosupporting national reforms (The rationale given was that reforms of vocational educatitraining will only be successfu

1. WHAT CAN WE LEARN FROM POLICY


16/136

The second was that partner countryinstitutions could engage in a policylearning process by involving newstakeholders and making use of their policymemory as well as international experiencein the field. The third was that the role ofthe ETF would be to facilitate this process.

2.1 Learning paradigms

Analysis of policy learning generallyfocuses on the process of policy changerather than the underlying process oflearning. The challenge for policy learningwas how to get policymakers activelylearning from local and internationalexperiences. Grootings (2004) summed upthe main characteristics of the learningparadigm as active learning. Recent

theories on learning argue there are manyways people learn apart from simplyreceiving information from a teacher.These hold that learning is first andforemost a situated social activity (Laveand Wenger, 1991; Wenger, 1998), that alot of tacit learning takes place which is noteasy to define or quantify but which is there

when needed (Schn, 1983) and thatlearning is dynamic and that good learningdepends on experiential learning (Kolb,1984).

Once again Deweys theory of experiencecan provide useful insight. By seeingexperience as a continuous and highlyinteractive process of exchange between

individuals and their environment, Deweyargued that people assign their ownmeaning to information. They do so basedon what they already know and only retainwhat is relevant for them. By so doing, theyconstruct their own understanding of reality

learning environmengood learning outcom2004).

2.2 The role of partn

Following several initiYearbook, 2005), the launch a new generatprojects in 2007. The

policy learning approacould contribute to VEpartner countries. ThrMorocco, Kyrgyzstan to participate.

An initial problem waand explain the concinto different languag

connection between the process of makinthe partner countriessecond hurdle. Chaprelationship betweenand policy learning asome reasons why thcan often fail to live u

3. THE ETFS ROFACILITATOR

Although there was athat facilitating a policshould be the guidinginterventions (Grootin

& Nielsen, 2006, receProgrammes), the quthe ETF should go abspecified. A critical isthat many people seepolicy advice and pol



17/136

This volume as a whole presents a strongargument for continuing to change the roleof the ETF from delivering policy adviceand providing policy solutions to policylearning interventions that help partnercountries to help themselves.

3.1 Key lessons

There are of course many lessons to be

learned from the analyses and findings inthe following chapters. However some keypoints stand out and could provideinspiration for future ETF work with partnercountries. Four key questions must beconsidered when undertaking a policylearning process:

n Who are the policy learners?n What is the content of the learning?n What are the key actions or processes

of learning?n What is the impact of policy learning on

resulting policies?

3.1.1 Policy learners

In his discussion about policy learners,Peter Hall (1988, cited in Bennett &Howlett, 1992) suggests that both stateand societal actors are the main actors ofpolicy learning.

In Chapter 5, the authors give a broaddescription of what they call thestakeholder working group and the range

of actors government officials, schoolmanagers, local experts and socialpartners involved in the policy learningprocess in the project featured.

Chapter 3 mentions the important role

society and researchers. Thiherald an emerging new formgovernance of the VET systepartner countries. I will returnin more detail later.

3.1.2 The content of learnin

Several researchers have medifficulties of pinning down th

learning in policy learning pro(Freeman, 2006). In their anaseveral policy learning approaBennett and Howlett (1992) s[] existing theories vary onobject is. While all see learningeneral increase of knowledgpolicies, some see this in terminstruments, some in terms o

programmes, and some in tergoals or some combinations oelements. Heclo (1974) and (2004) believe that to a certaiis learned is policy itself. For the basic assumption underlconcept of policy learning is nthat policies can be learned b

policies are learned policies.1993) thinks of it in terms oflesson-drawing, where the laction-oriented conclusion abprogramme or programmes inelsewhere (1991, p.7).

Several chapters in Part 1 prexamples of two types of lea

contents which broadly correones outlined above. The firsthe definitions of Heclo and Grefers to the actual processepolicymaking. The second suthe content learnt consists of



18/136

Task Force members were excellent intheir analysis, interaction and involvement,it was difficult for them to move from aresearch format to the actual formulation ofthe policy and strategy paper, in spite ofthe facilitating efforts of the ETF, theynote.

Finally there is also evidence of learningfrom past failures from the new Member

States (see Chapter 7). Some of thisevidence is direct from the horses mouth.In Part 2, Slavko Gaber, a formerSlovenian education minister, drawsextensively on his first hand experience ofpolicymaking and highlights severalreasons why it was difficult to applylessons from other EU countries.

3.1.3 Learning actions and processes

Two processes problem solving andreflection are to be found throughout thiscollection of articles. Although both areintegral parts of the process of learning,they are of a very different nature.

Problem solving

Learning is embedded in complex, real-lifesituations: The learning occurs as anattempt to solve a real-life problem. Theneeds to design (Kyrgyzstan), develop(Morocco) or evaluate policy (Turkey) arewhat drive the policy learning processrather than the desire to apply abstract

concepts and principles. As Chapter 4shows, real collaboration calls for a sharedtask where the partners can work togetherto produce something none could haveproduced on their own.

learning is situated leinvolves participating activities which in turnintercultural dialogue negotiation (Morocco)authors state that peepracticed in the Balkathan policy borrowingknowledge consists othan merely identifyin

and which do not. A lawhat motivates the lesuch participatory prothey hope to gain fromprovides some insighat the opinions of diffenational policy task foin particular provides the teams motivation

established by the prewill give us authority aa big responsibility.

The role of policy learSome general observon how the policy leahelp establish a syste

policymakers of partnlearning. I would arguchapters show that thwill only work if local take control over the process. Therefore afacilitation is one whegradually relinquishefacilitator introduces

strategies for solvingasks questions that thotherwise ask, therebto expand their horizochapters reveal clearthe role of facilitator a



19/136

ideas by contrasting them with fresh ones.Policy learning, particularly when it takesplace as part of a peer learning event, is amatter of understanding other systems tobetter understand your own2. This comesover strongly in Chapter 4; where theauthors affirm that paying attention to thetype of policy to be implemented in aparticular country context and to thepeople involved in implementation

increases policymakers understanding ofhow different aspects of policies, peopleand places interact and combine inparticular ways to shape implementationprocesses and outcomes. Feedback andobservations from policymakers fromneighbouring countries on national policyissues in VET support self-reflection andbetter understanding of the specificities of

their own systems.

Dismantling myths: Participants in theBalkan case see the policy learningexercise as a way to dismantle the myth ofthe superiority of EU VET systems and tolearn from neighbours. This was not thecase of the Kyrgyz team who had access

to too many visions when they draftedtheir policy paper. In this case, as pointedout in Chapter 6, looking at too manypolicies prevented them from seeing thecharacteristics of their own system It startsand ends with visions and principles spicedwith different concepts taken fromdocuments related to the EU VET policyframework, and fails to integrate and build

on available national evidence, note theauthors.

3.1.4 The impact on policy

Bennett and Howlett (1992) believe the

perspectives see policy learninstrumental for policy changeand Spours (2007), policy leaimpact not only on the policiebut also on the way they are

As the following chapters shoidentify or quantify how muchchange is due to policy learniI would argue that the eviden

changes the way policy is maclearer.

n Kyrgyzstan: Chapter 5 proevidence of changes to thpolicymaking. In Kyrgyzstaformer Soviet republics, thexperience of policymakinpreviously been done cen

Moscow. As a result of ponew VET policy task forcestakeholders was set up bgovernment.

n Morocco: In spite of the neon some of the political dimthe work done, the depth oand cooperation establish

departments represents aachievement for the policyinitiative.

n Turkey: The main conclusiois that policy moved, throuassessment approach to aagora where not only poimportant but practitionersresearch activities are too

The present discussion is stilThis volume is based on a smexamples, some of which areunderway. It is also true that must pass before we can rea



20/136

learning to engage a wide range ofstakeholders from the public sector(including policymakers and practitioners),the private sector and civil society ingeneral. To varying degrees, all chaptersshare a view of policy learning as acollective enterprise which leads to a moredemocratic process of consultation andgreater ownership.

Second, policy learning is also aboutdeveloping national capacity to leadreforms. Capacity building occurs throughlearning to use different instruments suchas scenario building or impact evaluation. Itcan also draw in new stakeholders such associal partners, researchers or practitionersand help them move from the periphery tothe centre of policymaking (Lave and

Wenger, 1991).

Third, it would seem logical to expect thatan active learning process whichencourages interaction, collaboration andreflection will also promote a betterunderstanding of policy issues and lead tobetter designed policies.

4. CONCLUSIONS

Policy learning processes are powerfultools for promoting collaboration betweendifferent stakeholders and sharingexperiences. As we have seen, they alsoencourage situated problem solving and

reflection. This is especially importantwhen the aim is to develop nationalcapacity to lead processes of reform.

Nevertheless policy learning inevitablysets up different kinds of tension:

between the time nethe urgent need to debetween the role of fadvisor. Several chatimescales for policyand Spours (2007, precognition that polishould reflect the neas well as political animperatives.

As for the relationshiplearning and policy chchapters do not providevidence that participlearning processes cachange. However thechanges in the way p

Among these are incl

stakeholders, promotdecision-making and introducing new toolspolicymaking.

Policy change is difficdifficult to quantify. Thlearning interventions

described was that inencouraged stakeholdinvolved in steering thdevelop their own undissues. Using a policypaves the way for a ngovernance. By promactive learning which a policy learning proc

build national capacitpolicy learning has pleat the ETF and that wexplore its richness. Ican make a valuable facilitating the policy d



21/136

2. THE ETFSINTERVENTIONS IN JORDAN,A FINE LINE BETWEENADVICE AND GUIDANCE

Grard Mayen

1. INTRODUCTION

Policy learning refers to a change inthinking, not just any change in thinking,but a structured, conscious change in

thinking about a specific policy issue. Thisdefinition provided by Ren Kemp andRifka Weehuizen (2005) refers mainly tothe process that governments (meaningthe public sector) may develop to deal withpolicy changes.

A brief description of the Jordeducation landscape and reffollowed by a section on the role played by social partnerdesign of new policies in tech

vocational education (TVET)ETFs role as policy learningdescribed along with a comppicture of the work it is doingparticularly though the identifcritical factors and reflections(S h 1983) Th


22/136

and administration to make Jordan aknowledge society able to cope with the

effects of globalisation. This ambitious reformprogramme set targets for the period from2006 to 2015.

The National Agenda identified, among otherissues, several labour market and TVETchallenges and targets that cover:(i) reducing unemployment; (ii) absorbing the

annual inflow of new job seekers (projectedto grow at 4% per annum); and (iii) increasingthe participation of women in the labourmarket. This is combined with the need todevelop the capacity of enterprises(particularly small and medium sized ones) toadapt to technological changes and providequality products in a globalised market. In thiscontext, the Ministry of Labour has been

given the mandate to lead the employmentand TVET strands of the National Agenda.

In parallel, the Ministry of Education throughthe Education Reform for the KnowledgeEconomy project (ERfKE) and the Ministry ofHigher Education through the HigherEducation Reform for the Knowledge

Economy project (HERfKE) are engaged inwide scale reforms. Both ministries deal withthe segments of technical and vocationaleducation within their jurisdictions.

3. EMERGING NEWGOVERNANCE MODEL

In Jordan, the development of human capitalis now considered in a lifelong perspective.The main challenge of the reforms is thedevelopment of individual competences thatrespond to the needs of labour market.Leading the reform is seen as the collective

that its drivers (particuas social partners) are

the driving seat. Furthanalogy, they also needriving licence.

This poses two majorresistance of the publthe new stakeholderssecond, their lack of c

this responsibility. Bohave been consideredheart of the policy leainitiated by the ETF inactivities are carried ocascading participatostakeholders from a wprivate, non-governmsome extent, donors)

4. POLICY LEARPROCESS

Policy learning activitserve the parts of Jorthat aimed at develop

economy based on thcapital.

4.1 Task forces

The first step in this pthat participants woulpossible, be those whimpact on institutiona

Continuity was an impbecause understandinhad to be shared by estep of the process.

The participatory app



23/136

1. Technical learning. These learningactivities have led to the development of

instruments needed to better governTVET sector reform (such as monitoringindicators, a mid-term expenditureframework and others), and awarenessraising about the sector-wide approach.The primary targets were operationalstaff from the Ministry of Labour andsystem governors.

2. Conceptual learning. This was directedat a very wide range of stakeholders. Itincluded the presentation of the majorconcepts and strategies needed tomanage a sector reform where theprivate sector and social partners playan important role. It helped learners toidentify the crucial elements required by

a strategy to create a quality-orientedTVET sector.

3. Social learning. This primarily targetednewcomers such as social partnersand, to some extent, private sector andpublic training providers who had tolearn to take new responsibilities, adopt

new ways of interaction, and becomefamiliar with policy approaches.

These different learning processes wereorganised taking into account the mainprinciples of policy learning as applied bythe ETF. It helped partners in Jordan tolearn from their past experiences, learnfrom other countries and learn from doing

(Grootings, 2004).

4.3 Learning from the past to inform the

future

At the request of the EC Delegation in

2007 where stakeholders revithat was prepared by the Nati

Human Resources Developm1990s. Participants included dpublic, private and social partninteracted in round table discu

This exercise had a double objprovide stakeholders with stratinformation and regional and in

experiences in shifting from a sto a demand-driven TVET modemployment as a key issue, anincorporate their views (thus sttheir ownership) in the developvision in a participatory proces

A team of ETF experts facilitateproceedings and the introductio

reform concepts, such as qualitqualification frameworks and insystems, which were rather newthe stakeholders. However, polfacilitation involves more than tinvolves linking these new concnational institutional context. Thhelped Jordanian counterpartsconcepts in the context of theirsystem, assessing the role of swell as the technical processesimplications of the envisaged c

The following vision was elab

The vision for the Employme

Technical and Vocational EdTraining (E-TVET) sector is tefficiency of the sector in accwith the government vision toJordan as a knowledge econthe needs of the labour mark

2. THE ETFS INTERVENTIONS IN JORDAN, A FINE LINADVICE AN


24/136

Subsequently, based on the samepartnership approach and directly connected

to the strategy, a five-year sector reformaction plan (2007-2012) was elaborated inthe form of a logical framework.

4.4 Learning for the future

Following the workshops, a TechnicalCommittee composed of public and private

representatives was formed whichreviewed in detail all expected sectorreform results and translated them into adetailed action plan with a number ofactivities linked to each of thestakeholders. The ETF supported theirwork with regular review meetings thatcontinually emphasised the implications ofthe new vision and action plans on the

work of different institutions andstakeholders. The sector reform action planwas presented and approved by the TVETCouncil in August 2007.

All public institutions then set-up their owntechnical committees to translate the sectorreform action plan to their own situation.One interesting result of the entire exerciseis the way in which the participatoryprocess initiated by the ETF has taken holdin Jordan. It is now common to invite otherstakeholders to review plans and actions,while referring to the sector reform actionplan has become second nature. Thesocial partners themselves are showingincreasing interest in discussing the issues

linked to human capital development, to theextent that two of them (the Chamber ofIndustry and the General Federation ofJordan Trade Unions) have now asked theETF for support in internal capacitydevelopment.

improves coordinationeffective support to co

priorities. As one exaand the Canadian InteDevelopment Agencyactivities to the sectorwhich they had thorouthrough the process eETFs role in sector rementioned in World B

6. POLICY ADVIPOLICY LEARNFACILITATION

At the beginning of throle was described aspolicy learning proces

cover the ETFs workthough. A companionapplied: the learning pand, if policy learningsome impact on the swhole has gained conof the sectors, cultureof Jordan.

The ETF teams role influenced by earlier ealongside diverse cocontexts. The basic aETF support should bthe competences of thcounterparts: the morachieving a goal or de

more ETF support waversa.

This introduces the inscaffolding that was (1985). He described



25/136

scaffolding has been used by severalresearchers when analysing adult training

(Mayen, 1999; Kuningel, 2007) and mightbe an appropriate frame in which toanalyse the activity of ETF staff andexperts in charge of facilitating thelearning process. Policymakers in partnercountries can be regarded as policylearners (Grootings, 2004) and policylearning can be facilitated by creating a

relevant support system. Bruner et al.(2006) developed detailed descriptions ofan interactive system of exchange inwhich the tutor operates with an implicittheory of the learners acts in order torecruit his attention, reduces degrees offreedom in the task to manageable limits,maintains direction in theproblem-solving, marks critical features,

controls frustration, and demonstratessolutions when the learner can recognisethem (p. 207).

Bruner and his team (Wood, Bruner andRoss, 1976; Wood, 1980) demonstratedthat where support is contingent on theactivities of the individual and related towhat the individual is currently trying to do,then considerable progress may be made.

7. POLICY FACILITATION AS ASCAFFOLDING PROCESS

In addition to and in parallel with thesupport provided to the participatory

discussions described above, the ETFcontribution to policy learning also includedcapacity building activities for Ministry ofLabour staff, developing a managementstructure within the ministry to deal with thereform process, contributing to policy

donors), and on the other hancharge of implementing the re

process.

This coincided with other ETFthrough different projects sucObservatory Function, the NaQualification Framework ProjPartnership, and MEDA-ETEall contributed to the policy lea

among stakeholders. The key to apply the same principles ofto all interventions. This requirecoordination of all activities as communication with local and iconsultants working on behalf

By contributing to the developpolicy papers for the minister

conceptual steps to develop tgovernance model mentionedministry staff became involveThey also gradually became in drafting the actual docume

ETF interventions looked at pstaff skills, as well as the conpolicy agenda.

The process aimed at creatinsystem for staff to help them and master new concepts antheir capacity to manage teamthose concepts and turn themThe Jordanian staff have takeresponsibility for the process

presenting at and facilitating ETFs initial role is now entirehands of the team members.positive consequence is the iof staff skills in drafting policy



26/136

stakeholders themselves both on specificconcepts and general reform issues.

A pressing policy agenda meant thatlearning activities were combined with theproduction and delivery of policy notes,policy presentations and discussions bothwith national stakeholders and internationaldonors. Again, the skills of the Jordaniancolleagues and the quality of the

deliverables they produced improved to theextent that they became self-sufficient.

For example, one urgent and fundamentaltask was supporting the ministry in definingthe new governance and sector reformmanagement structure, based on aprocess calling for more effectiveness indecision making. This ranged from

describing the sector to defining in detailthe mechanisms that govern theimplementation and monitoring of thesector reform. This process required broadconsultations with public and privatestakeholders and donors before resultswere presented to decision makers.Throughout this iterative process,knowledge was shared with ministry staffregarding new concepts. Views ofstakeholders were seriously taken intoconsideration to ensure that a smoothimplementation of the process would be inline with local traditions. A collaborativeapproach was initially adopted for certainmatters so as to gradually develop a bodyof knowledge that eventually enabled the

Jordanian counterparts to deal with policyissues in a more constructive waythemselves.

Another example concerns unemploymentdata analysis followed by the publication of

sector expected to be Council as decision-m

is expected to becomethat serves the decisioETF now acts only as with most of the work pcontributors.

8. THE EFFECT

LEARNINGEvaluating the impactis always critical. Thisthe space nor the inteNevertheless, some eindication of how the above have contribute

n Previously not conare now embeddedocuments, concestakeholders and pinclude concepts ssystems, social paassurance, entrepqualification framepartnership and ceTheir adoption is ilthat policymakers mentioning their wimplement them aother actors (othersame political leve

n Participatory methby the ETF are no

institutions. This inthe way public instrole of social partncontribution to the

n Social partners aregranted new powe



27/136

private sector social partner said thathis institution had understood the

crucial role of human resourcesdevelopment and its readiness to adaptits structure to deal with it. Anotherexample comes from a keyrepresentative from the GeneralFederation of Jordanian Trade Unions,who questioned why his institution hadnot been invited to the Parliament to

discuss the TVET law. Both of them aremembers of the Reform SteeringCommittee and have participated in anETF study visit to the EU to learn moreabout social partnerships. A finalexample concerns the awarenessamong social partners of the need forfurther capacity building before a placein the driving seat can be legitimately

claimed. It is illustrated by a request tothe ETF from three social partners to besupported in building internal capacities.

9. LEARNING BENEFITS FORTHE ETF

Policy learning is not a one-way trip. It isabout collective learning. It should aim tobe, and thought of afterwards as a win-winexperience. A number of positive returnson investment for Jordanian stakeholdershave been outlined above. For the ETFthe learning process has been a richexperience which provides a deeperknowledge of the internal functions of a

partner country TVET sector (the technicallearning). It has given an increasedunderstanding of strategy andcoordination mechanisms amongstakeholders and an indication ofsubsequent levers to act on (the

The purpose of this chapter wportray the ETF as a lone adv

policy learning. The ETF is oncontributors to this process. Pis a long-term process that halaunched and will be improvelarge variety of interventions mechanisms. Many other intecontribute to policy learning aJordanian counterparts to bui

resource development systembest for them. Compared to teassistance from donors, the Econtribution is very limited in resources. Nevertheless, the added value that few other indemonstrate: the broad expeand the thorough knowledge in the sector.

10. CONCLUSION

In this chapter, policy learninconsidered as a process whepolicymakers have progressivtake account of the importancrole including the role of new meaning the social partners, process. It has enabled the sto move progressively from ththe centre of decision-makingThe effect of policy learning opolicymaking processes has described while the difficultieslearning process both for gov

social partners has been high

Although identifying and analof policy learning on policies there are several indicators othe policy goals through the l



28/136

of partner countries, the facilitation role hasbeen portrayed using the scaffolding

metaphor. Facilitators should regulate theirsupport in terms of their counterpartsabilities and capacities.

It has been the policy learning approacheswhich have led the ETF to look more

carefully at the procesin Jordan and to pay a

policy advisor with thepeople help themselvquestion of moving froacknowledges the neone which actively recimportance of collabo



29/136

3. POLICY LEARNING THEEXPERIENCE OF IMPACTANALYSIS IN TURKEY

Sren Nielsen, Outi Krkkinen,Recep Varcin and Arjen Vos

1. INTRODUCTION

Turkey has undergone tremendouschanges in recent years and the educationsystem in particular is undergoing radical

reform. Many foreign donors are active inthe country and there is a wealth of policyproposals available for policymakers andstakeholders. But ongoing changeprocesses are so dynamic that valuableachievements risk not being taken forward.

innovative approach that comfocus on the organisation of pplatforms with the facilitation learning processes around anactivity may create enhanced

and horizontal network learnilead to a policy process that yconsistent follow-up decision-stronger stakeholder involvem

The concept of policy learning


30/136

can and should play in their cooperationwith colleagues in partner countries.

Policy learning is sharing experience fromthe past to develop knowledge for thefuture. It also covers the distillation ofknowledge from other countries andknowledge that is produced locally into newknowledge. It contributes to creatingcoherent system-wide reforms that fit into a

local context and it facilitates system-deepreforms of VET systems because itenables all stakeholders to learn new rolesand develop new working routines.Developing concrete approaches that canmake policy learning work in practice is achallenging task. The ETF, working in aprivileged position at the interface betweenresearch and practice, can take such

approaches forward as a mediating agencybetween the two fields. Targeted impactanalysis could be yet another tool for policylearning, if properly designed andconducted in a participatory approach.

Reform of the organisation of vocationalteacher education is an essential part ofoverall VET reform in Turkey. The designof such systemic vocational teachereducation reform offers many angles fordiscussion. For analytical purposes, we willapproach the activity from twoperspectives. Both are central to theargument developed here. First theinnovative design and results of an impactanalysis of the EU-funded vocational

teacher training reform project,Modernisation of Vocational Education andTraining (MVET), is analysed. Then theextent to which evaluations can provide ameaningful and proactive contribution topolicy learning, rather than serving merely

arguments developedfor future work toward

and proactive use of pprogramme evaluatio

2. THE MVET PRREFORM OF THORGANISATIONVOCATIONAL T

TRAINING IN TUAs one among very fecountries, Turkey hasof its vocational teacha central part of geneproject focussing on g(SVET) and the ModeVocational Education

project sought synergmonitoring committee

The MVET project focimprovement of the qof vocational teacher was designed to addrneeds:

n to strengthen coopemployer of vocatiMinistry of Nationaprovider of vocatio(the Higher Educaof university educa

n to ensure that vocrelevant pedagogi

skills related to then to link vocational t

overall framework development in Tuand in-service train

n to align Turkish vo



31/136

Quality Assurance Framework forVocational Teacher Training (QATT).

Furthermore, the project also contributed tothe development of the policies andstrategies of vocational teacher training,which identified the need for significantchanges to the structure of vocationalteacher training courses. The strategyrecommends a move towards a one yearconsecutive pedagogical course forprospective vocational teachers who havethe minimum qualifications and experienceidentified by the Ministry of NationalEducation. Moreover, vocational teachergraduate qualifications need to be clarifiedand linked to the labour market, because atpresent only 5% of graduates get jobs asvocational teachers. The paper alsoproposes reviving the sleeping vocational

teacher training committee as an advisorybody that should link the policies of YKand the Ministry of National Education.

3. KNOWLEDGE CREATIONAND THE USE ANDMEDIATION OF KNOWLEDGE

A CHALLENGE FOREVALUATIONS

What really matters in countries intransition is supporting and encouraginginitiatives and organising changeprocesses. Open approaches tounderstanding change are thereforeimportant as these may better help us to

assess development oriented practices anddesign future policies that are adequate forthe country in question.

Traditional evaluation projects run byexternal education or VET scientists will

assessments of educationalpointing out that its relevanc

and policymaking is often to(OECD, 1995 and 2004).

Science and practice often dosame language. They have dPractitioners are mostly concpractice, educational scienceconcerned with theory, and thscientists and practitioners ofdifficult to communicate. The scientists is to produce new knew recognition they seek ttask of educationalists is to qlearners who can master andimportant and valuable knowskills as defined by society. Interms, they seek to make thin

Researchers, policymakers practitioners share a commocommitment to developing meducation systems, howevedistinct knowledge needs. Inbetween and within the threcommunities is made compldifferent priorities, agendas perspectives. Practitioners for empirical evidence offeriprecise answers that can bePoliticians typically look for results that are presented inthat they can be used for podecision-making8. In educaand practice we need to red

application gap that separafrom practice and devise nemechanisms for the implemeresearch findings by policympractitioners9.

3. POLICY LEARNING THE EXPERIENCE OF IMPACT ANALYS

OO 2008 O C G C O


32/136

way of producing knowledge. It is arguedthat the mode 2 concept is expanding

because of the increasing need forinteraction between science and othersocial sectors such as economy, politics,education etc.

Mode 2 research emphasises theinvolvement of both researchers andpractitioners in the knowledge productionprocess. The authors argue that mode 2research is contextualised in a newpublic arena which they call agora10.Research has moved from operating inisolation to the agora a place where themarket and politics meet and are mixed,and where private emotions and opinionmeet public opinion and politicalconsensus. Such contexts, which the

authors call transaction spaces, havemany similarities to the field ofeducational research, and the concepthas also found considerable resonancein education11. This school is veryinteresting for the ETF, as its main targetis not to produce new knowledge per sebut to build on, communicate andincreasingly ensure that knowledge is aguide for practice.

It is argued that in many applications theformer hard truth ideal of science is nolonger functional because today knowledgeproduction is distributed widely among anincreasing number of knowledgeenvironments.

From this it follows that a lot can be gainedby stimulating local capacity for carrying outaccompanying research that is closelyrelated to policy learning activities. We willhave to accept the in principle

There is an echo of thevaluation experience

work carried out by thof Human Relations (has been described aevaluation. It seeks toactivities in a way thato the development ofprogramme or policy. developmental evaluaimprovement and devgenerating reports an

judgments for externaaccountability12.

A developmental appinvolves a number ofoffers those evaluatethe evaluation agend

those directly involvecontribution that evaltheir future developminvolves an active feefeedback should be pdiscussed on a regulthe evaluation procesdevelopmental approimplementation. Expeunless early decisionhow to use the resultcannot be taken for gthe eventual implemefrom the outset and bthe evaluation. Fourtcommitment to actionprocess of undertakin

itself seen as a legitimstudy, because the aamong evaluators anparticipants may oftePatton (2002) arguesthe specific findings c


3 POLICY LEARNING THE EXPERIENCE OF IMPACT ANALYS


33/136

will be developed and applied to a specificand concrete case, the Modernisation of

Vocational Education and Training (MVET)project in Turkey. In the efforts to enablethe use of new knowledge produced andeffective mediation, researchers,policymakers, practitioners, beneficiariesand donor agencies are brought together inthe arena the agora where each hassomething to learn and teach.

4. THE APPROACH TO THEMVET IMPACT ANALYSIS

A fundamental question that affects manyevaluative assessments is: who is theclient? The approach to the impactassessment of the MVET project in Turkey

was therefore carefully discussed with theEC Delegation, key Ministry of NationalEducation (MONE) actors and seniorrepresentatives from the Council for HigherEducation (YK) at meetings in Ankarabetween 7 and 10 May 2007. It was agreedthat the activity should be developmentoriented and focus on how MVET projectresults can be taken forward by theMinistry of National Education and YK intheir development of further policies and inensuring that valuable results are actuallyimplemented.

The approach was based on a policylearning philosophy, where nationalownership, fit into context and sustainability

are important. The current situation, thenext targets, and the kind of supportneeded to reach these targets were focalpoints. Within the spirit of the agora asoutlined above, the ETF and all projectstakeholders and shareholders worked

Another strategic choice agreTurkish key players and the E

was to use the impact analyslearning platform for launchinlearning activity by building thfocus of the impact analysis ologic and not on the MVET pilogic.

Policymakers and practitionepointed out that one possible the fact that the Ministry of NaEducation does not have a devocational teacher training poapplied to the evaluation as mto the actual implementation project. The main emphasis tto be put on the projects polistrategy component (compon

Key players in Turkey suggesstructuring the impact analyspolicy and strategy paper, stapractical, pragmatic MVET wrecommendations and involvpolicymakers and stakeholdelooking discussions. The entiwas anchored around the Ste

However, a short, descriptiveand evaluative survey of the and its achievements also neproduced. This was mainly doexperts who worked togetherusing questionnaires, intervieobservations and semi-struct

discussions. These tools werand developed together with Group members, most of whoproject beneficiaries in technifaculties. The data collection was also meant to constitute




34/136

serve and encompasses differentapproaches. Monitoring has a chronicling

and appraising function, while evaluatingex-post involves making a retrospective

judgment, measuring already existingphenomena. Impact analysis is a forwardlooking activity based on retrospectiveanalysis with a prospective purpose. This isa particular type of assessment whoseresults are meant to be taken forward.

The following understanding or definition ofassessment was used for the MVETevaluation task: Impact analysis is carriedout to determine the significance or valueof something by careful appraisal andstudy. It is a development process thatenlightens the specific policies, processesand practice of its stakeholders and

contributes to collective learning.

Two general objectives of assessments orevaluations can be differentiated:accountability and improvement.

Accountability relates to inspectionexercises, which seek justification for theactivities undertaken. They should look atwhether best value was achieved for the

money invested and provide a measuringstick for judging the activity. The objectiveof improvement, in contrast, aims toprovide the actors with a thorough reviewof the achievements against the potential ofan activity. An impact analysis aims atproviding recommendations for furtheractivities and should promote instruction

and development.

Turkish policymakers and practitionersshared our view that the improvementdimension in this phase of rapid reformwas the most important. Whatever

often a major barrier tconclusions from eva

on single projects.

Knowledge of what acproject is above all a input into its ongoing systematic follow-up o(sustainability) and thnext phase of activitieare therefore clear pri

The impact analysis aillustrated in the follow

What is the

project

about?

What

the pr

resu

Aims (needsindicators)

Objectives(performanceindicators)

Output

(outputindicato

l Targe

l Resu

l ProduDelive

Although they were cproject impact analysconcentrate on the liobjectives and outpuemphasis was on theoutputs and outcomefrom the project.

The purpose of the aregister what differenresults could producimpact and sustainabproject were the mai


3 POLICY LEARNING THE EXPERIENCE OF IMPACT ANALYS


35/136

this reform, and (iii) to enhance nationalownership and ensure the sustainability of

such reforms. This was achieved byestablishing a Steering Committee thatwould oversee the actual assessment,organising meetings with university facultystaff, carrying out dissemination events,and presenting findings andrecommendations at a national event inJanuary 2008.

6. FINDINGS AND POLICYOPTIONS FOR FUTUREREFORM INITIATIVES

Less than 5% of teacher training graduatesend up as vocational teachers. Graduatesare underrated when offered alternative

jobs. Vocational teacher training facultiesneed restructuring to become technologyfaculties as part of the higher educationreform. And the relations between YK(the Higher Education Council in charge ofuniversities) and the Ministry of NationalEducation are problematic. These factorsset the scene for policy development, forthe ministry, for universities and for

students.

The MVET project achieved most of itsobjectives (Kiraz et al., 2007). Therecommendations on the components ofcurriculum, teaching methodologies,student-centred learning, pedagogicalequipment and quality assurance pose a

heavy menu for follow-up policy decisions.However, the in-depth interviews carriedout in 14 university environments throughthe impact analysis indicate that most ofthe activities can now be delivered byTurkish institutions and experts.

puts the following issues on tpolicy options for Turkey13.

1. At the VET system level, tproposed to solve the probvocational teacher overprounderrated employment ugraduation is two-fold: (i) runiversity faculties into tecfaculties providing the highqualifications needed by thlabour market; (ii) a comptechnical qualification levethese in line with EuropeaIt is proposed to make thisnational qualifications framdevelopment process andtime provide an input to ferestructuring of higher edu

Turkey in the spirit of the Bprocess. Due to the fact thoverhaul of qualifications arestructuring of faculties hbeen undertaken, it has bedifficult to reform vocationtraining in isolation.

2. The relationship between National Education and Y

be reinforced. YK is respvocational teaching qualifiuniversities, while the MinNational Education is its csituation is similar in manyin Turkey there must be a stronger interface betweensupply. The outcomes of t

project and the developmevocational teacher trainingonly through agreements alevel. So a top priority is toplatforms and modalities fcooperation. A revitalised




36/136

change focus from a pilot project logicto a policy logic. According to

policymakers and practitioners inTurkey, the Ministry of NationalEducation does not have a developedvocational teacher training policy andstrategy and therefore finds it difficult tocope with the implementation of newdonor-led initiatives while at the sametime allocating the necessary funds forbroader implementation.

4. There is a need to establish aspecialised VET centre in Turkey whichalso covers vocational teacher training.There are 25 university-basedvocational teacher training institutions,but there is no institution that canensure that conceptual, methodologicaland organisational learning is

consolidated, institutionalised and takenforward in a routine way. Vocationalteaching and learning is a rich researchfield that needs to be developed.International expertise is not optimallyused and institutional capacity has to becreated to consolidate and cascadeMVET outputs to accumulate newhome-grown expertise in the field. A

national VET centre should accumulateexpertise and focus on research,innovation and development. Also inTurkey, there is a lack of didacticalthinking on vocational subjects whichcan best be developed in centres ofexpertise. A VET centre should beplaced in one of the universities

involved in vocational teacher training. Itcould start by taking the strategic lead inthe broader implementation of theMVET project results, accumulating allinternational and national donor-drivenproject outcomes and serving as a

there are differencdepartments and f

among all MVET pwe can identify thedevelopment needuniversities. Horizonetworks is recombe structured. Thedevelop initiatives horizontal learningwhich faculty staff and engage in capefforts.

6. There is a need toin two directions: tenvironment that wvocational studenta professional, comteacher today mea

the demands of thsystem, having thevocational knowledunderstanding of hdevelop in the futuwith schools are cchallenge for vocais to produce teachis relevance for vo

While initiatives hasome universities faculty-company licase with faculty-sThe easy solution National Educationvocational school teacher competen

with the faculties trespond to these n7. There is a huge de

teacher training coaverage, current coteachers to attend




37/136

training initiate the training process bygoing to the vocational schools and,

together with teachers and principals,identify the actual training needs.Learning partnerships between trainingproviders and vocational schools couldprobably emerge. For vocational teachertraining faculties, feedback mechanismswould allow experiences to bechannelled back and used as input intoordinary teacher education programmes.

8. In Turkey, as in most other countries,the concept of continuous professionaldevelopment of teachers is almostexclusively seen as continuing trainingprovided by centralised deliverysystems. In reality, continuousprofessional development is a muchmore promising strategy, it is much

cheaper, and it re-establishes therecognition of teachers as professionalsand stakeholders of reform. Itintroduces action learning principles,horizontal learning from each other, andon-the-job learning supported byexternal consultancy. Such activitiescan be coordinated by schools and canfit into individual schools year plans

where time is set aside for thesepurposes. This will require adecentralisation policy with morefreedom for schools to organise theirown staff competence development onthe basis of locally identified needs andwithout central approval procedures. Anapproach along these lines could lead

to innovative local initiatives which willin turn foster professional developmentefforts.

ORGANISATION OF POLICY

platforms that enable key acstakeholders to discuss futu

before, during and after the The mode 2 based agora moutlined earlier has enabledcreation of a participatory prevery stage of the assessmefollowing instruments were iin the impact analysis desigeach heading we sum up posteps.

7.1 National Steering Group

To ensure true national ownMVET impact assessment snational Steering Group thainvolved from the outset, takinitial discussions of the eva

design, monitoring the assesactivities and linking the resimpact assessment with chafuture policy design and impThe Steering Group includedrepresentatives from the HigEducation Council (YK), thNational Education, the TurkEmployment Organisation (I

new Vocational Qualificationuniversity faculties and the EDelegation. They met in Octdiscuss the design and orgadata collection in university in December 2007 to discusfindings and recommendatioSteering Group members al

in the final conference on 24January 2008 and gave feedimpact assessment. Importawere discussed at these methe findings were thoroughlyThese included some seriou



38/136

7.2 Data collection initiating dialogue in

all university vocational teacher training

faculties

An initial assessment activity focused onthe review of all available projectdocuments. A survey instrument wasprepared, containing qualitative andquantitative questions. A draft of thequestionnaire was mailed to allbeneficiaries for comments. Then a sitevisit schedule was prepared for 14faculties at 11 locations, in all covering 36departments/training programmes thathad been supported by the MVET project.In each faculty two days were spent onaverage. In each department, a jointmeeting was organised with faculty staffexplaining the purpose of the study.

Rather than selecting a sample, theassessment team carried out face-to-facesessions with 195 interviewees in thedepartments.

The two days spent in each environmentlead to in-depth discussions with numerousstaff members and, equally importantly,raised awareness among faculty staff of

impending changes.

Next steps

Due to the design of the impact analysisthe evaluators helped to create strongerinteraction and networking between thefaculties. Such embryonic networking is a

strong asset for Turkey and should bestrengthened and better structured in thefuture, also in the areas of policy design,implementation and monitoring.

7.3 Regional workshops

on future policy actionbrought together the d

sides of local vocationAfter a short introductproject impact analysteacher training policydiscussed in line withkey issues raised in thstrategy paper.

Next steps

Fundamental reform aexisting conditions areIndependent from (buMVET project, YK isreforming vocational tfaculties. The graduatproblem is broadly ac

common view is that sbe teachers should taand a half of additionatraining after graduatifaculties should be invof change. Faculties cgraduates are furnishnecessary credentialspractical training, whic

their employability givand job profile.

7.4 Policy agenda se

The MVET impact anfocus than just consovaluable results of the

designed in a forwardto influence the vocatpolicy agenda in Turkneglected or ineffectivprogrammes and by boptions to the attentio



39/136

Next steps

The ETF contributes to the Turkisheducation system by offering public adviceregarding future policy directions. Thisadvice takes the form of policy learningworkshops, structured discussions with keypolicymakers, country analyses and anannual country plan. The wider circledeveloped around the impact analysis thestrategic learning platforms has created acommon language, understanding, anddeparture point for future activities.

7.5 Dialogue among authorities

The MVET project was long andsometimes difficult before and even duringits implementation. Although it targets the

modernisation of vocational teachertraining at higher education institutionswhich fall under the jurisdiction of theHigher Education Council (YK), the mainformal beneficiary is the Ministry ofNational Education because it is theprospective employer of vocational teachertraining graduates. The relationshipbetween these two entities has remained

weak but has improved during the project.The Ministry of National Education is not ina position to develop or implement highereducation policy. The project has helped toopen up dialogue.

The ETF has tried successfully to gatherboth parties around one table. It has

proposed reviving the vocational teachertraining committee as a tool to implementthe necessary reforms. The formalapproval by the Ministry of NationalEducation and YK of this committee is apromising first step.

policy implications. The broadfrom the impact assessment

shown its value as a vehicle flearning events. Especially hibetween YK and the MinistrEducation have been and arefacilitated. Partly as a conseqconvening efforts, YK has ineducation reforms and embrasignificance of VET for produinnovative and talented humaprerequisite for coping with thsociety.

The national Vocational TeacCommittee met for the first tim2008. It is a unique platform wMinistry of National Educationsocial partners will discuss ne

teacher training strategies. Twill be to move from a discusto action that will lead to chanvocational teacher training posystem. The ETF will monitornecessary promote progress.

7.6 National conference wit

international networking op

A national conference was coAnkara on 24-25 January 200aim was to discuss vocationalearning and the new role of tcentred around three questioTurkey now? What will be theHow do we get there? The c

discussed challenges, such alearning paradigm, and the cusituation in Turkey, with findinMVET project impact analysisteacher training reform. Next discussed in functional workin



40/136

was right to join international networks. TheMinistry of National Education supports this

idea and will see how a national vocationalteacher training network can be set up andfinanced. In his final comments on theMVET impact analysis, now taken forwardby the conference, Dr. Abdullah Snmez,Dean of the Technical Education Faculty ofthe University of Gazi, summed up thepolicy learning implications by underliningthat MVET is much more than a project: ithas provided a new awareness and a newpolicy platform for Turkey, on whichongoing discussions on the reform ofvocational teacher training can now takeplace. Engaging the national VocationalTeacher Training Committee and theTurkish network linked to the EU-levelCedefop vocational teacher training

network (VET TTnet) could become animportant driver in shaping andimplementing the necessary policy reforms.

8. CONCLUSION

This chapter has discussed an approach toimpact analysis based on a policy learning

logic. We have tried to use the policylearning philosophy as a guideline for thedesign of evaluations of donor-led projects.

A lot of effort and money is spent onevaluations in transition countries and thechapter has demonstrated that, rather than

just serving an accountability function,evaluations can be used to generate

proactive contributions to policy learning.Forward looking impact analysis which isorganised as development evaluation andhas improvement as its main objective canprovide national policymakers andstakeholders with a review of the main

It taught the lesson thshould not be carried

evaluating body if it isopportunities. Transabe established whichpolicymakers as wellthe process. From thelearning platforms muset up that engage nastakeholders in dialogthe assessment. Thistherefore incorporateas a national steeringcollection as a collecregional workshops, dialogue among authfinal conference as stplatforms.

This method worked winstrumental in overcobarriers in the knowleIt has brought togetheproviders of vocationaas well as establishinvocational teacher tradiscussions in and wito come. The Ministry

Education and the ECAnkara therefore askeundertake a similar imEU funded VET reform2009.

The approach to the awith much contextual

an array of policy optiTurkey. Its results conof evaluation of majorfinanced projects canreform achievements taken forward and tha


41/136

4. ETF PEER LEARNING:FROM POLICY LEARNING TOPOLICY CHANGE INPARTNER COUNTRIES

Margareta Nikolovska and Arjen Vos

1. INTRODUCTION

In this chapter we reflect on the ETFstwo-year experience with the methodologyof peer learning and its impact on

participants, national policy and policychange. We argue that the ETFmethodology makes peer learning apowerful policy learning tool for theparticipants policymakers and VETpractitioners and leads to greater

n What are the specific featETF peer learning methodwhich factors are importanpolicy learning process?

n How successful has the E

learning approach been?n Has the learning experien

policymakers, education apractitioners who participaimpact on their organisatiopolicy change?



42/136

Evaluations of peer reviews (Gordon andThompson, 2005) indicate that policy

recommendations are not much used andhave little influence on the policy process.The same evaluations confirm that thelearning experience of peers matters most

all participants concurred that individualknowledge creation and the learningprocess itself are the most importantoutcomes of the peer review project. TheETF concluded that its peer reviews can

still be a useful instrument for policylearning where there is a strongcommitment on the part of the nationalgovernment and the reviews are carriedout every four or five years.

The ETF peer learning methodology hasthus not been developed to replace peer

reviews, but as an attempt to find anotherpragmatic and efficient tool that will buildon existing local knowledge and capacitiesintegral to the specific environment ofpartner countries education reformpolicies, and to local education and VETsystems. Learning and policy are central tothis new approach15.

Peer learning is a modest and flexibleinstrument which recognises that thelearning process may be more effectivethan policy recommendations in report formwhere the objective is to strengthen thecapacity of policymakers and VET expertsto develop and implement policies.

The first ETF peer learning exercise in2006 concentrated on the policy issue ofVET financing in Albania, Kosovo16 andMontenegro. Policymakers and VETexperts from Albania, the former YugoslavRepublic of Macedonia, Kosovo and

concerned and also tolearning. The ETF is n

position to strengthenpolicy learning and onprocesses in countrieeducation or VET sys

The ETF sees policy instrument to support partner countries to achange in their educa

systems. Policy deveand implementation son broad ownership ainstitutional structuresstakeholders to partic2007). However, as thprogramme is not simlearning environments

enable them to formua result of their learnitwo issues to be addrThe first is the challenretain an environmenlearning for all participETF peer learning mesecond concerns the role as policy learning

policy learning instrumchange and this is m(Grootings, 2007). Boconnected to how theknowledge of how to learning that will lead change.

2. THE CHALLEPOLICY LEARNPOLICY CHANG

Within the internation

4. ETF PEER LEARNING: FROM POLICY LEARNING TO POLICPARTNER


43/136

inclusion policies, where peer learning isdefined as mutual learning processes

based on the systematic evaluation of goodpractice and assessment of selectedpolicies or institutional arrangementscoming under the various National ActionPlans (INBAS/NIZW, 2005).

The framework of ETF peer learningbroadly follows the logic of the EUapproach to the concept. However, given

the fact that the ETFs role in SouthEastern European VET reform is tofacilitate policy learning by makingavailable instruments and sources foreducation and training, and by assistingand guiding stakeholders participation andinteraction, ETF peer learning is based onthe principle of a learning platform carefully

created and facilitated around major policyissues of concern in the participatingcountries. By involving policymakers andVET experts/practitioners as peers,conditions are created for better targetedcapacity building around the policies inplace and policy outcomes.

The direct involvement of both

policymakers and VET experts wasintroduced in 2006, using the followingmethodology:

1. A common issue for policy learning isdecided in cooperation with thecountries involved.

2. A country background paper isprepared by the participants in the formof a self-study document.

3. A thematic concept paper is elaboratedby the ETF on the selected topic forpeer learning.

4. Peer learning events are organised in

Four main goals underpin curactivities in peer learning in th

pre-accession region:

n improved mutual knowledgunderstanding of VET sysand developments;

n promotion of networking, eexperience and cooperatioVET experts, stakeholderspolicymakers, leading to a

policy options suitable for and traditions;

n increased awareness andopportunities for learning freform experiences in EU States and (potential) cancountries;

n linking national policy refo

to the EU Instrument for PAssistance (IPA) program

The peer learning approach cas a tool to reinforce stakehocapacities to formulate and imsystemic education reform poadded value of this approachorganisation of a concise reg

platform for country stakeholdenhanced potential for reflectreform activities as well as a debate on the EU Education 2010 agenda. The two roundlearning activities have focussystem-level problems VET2006 and policy impact on sc

as experiments in both the policy learning, and how policcan contribute to policy chang

Nedergaard, in an article on mlearning processes in which h

PARTNER



44/136

engagement of national stakeholders indeveloping their own policy solutions. This

assumption is based on the principle thatVET reforms in transition countries will onlybe successful and sustainable if policydevelopment, formulation andimplementation are firmly based on broadownership and fit within the existingcountry contexts and policies.

The ETF peer learning methodology

assumes that learning is as much a socialas an individual process. The participatinggroup of selected policymakers and VETexperts and practitioners benefit frommutual insights and opinions. Although thegreatest impact of peer learning activitieshas been on those directly involved in theETF exercise, it is also important to

understand the related aspect of thesignificant role that this methodology canplay in the policy process in partnercountries, and how it can be embeddedinto comprehensive policy learningstrategies that include a series of mutualknowledge sharing activities18.

To expand on this, we simply need to ask

to what extent the policy learningincorporated in the ETF peer learningmethodology is likely to be used in thepolicy cycle and for systemic reforms in thecountries that have taken part in theexercise. To understand this we need todelve deeper into the various aspects ofpolicy learning and policy change.

3. ETF PEER LEARNINGMETHODOLOGY: ELEMENTSOF THE POLICY LEARNINGPLATFORM

The first of these is thpolicy issue around w

learning exercise is stfocuses on an importaagenda of the participwhich ministries of edcommitted. The VET 2006 was perhaps noattention in planning rawareness of the neenew reform plans has

the focus was on the policy development, sthe effective transformproposals into practicinvolvement in policy implementation stageattempted to advancewere not, or not suffic

the countries concernthese policy issues win the daily work of thpolicymakers and VElearning could have aan important motivatin

A second element, thpeer learning team,

constructed in discussETF and partner counprinciple on which therests is, as mentionedand knowledge creatias an individual procethe team is an individand professional expe

responsibilities. At theparticipants each reprpolicy cycle. The selecrucial. Policymakersinfluence on or respodevelopments, but it i



45/136

in their schools. A basic knowledge ofEnglish was required for effective

communication, although arrangements forsome translation within the team havebeen necessary19.

The way in which the peer learning teaminteracts and the relationships that emergebetween its members help to create aplatform that unleashes some specific lifeand professional experiences. To create an

effective learning environment, trust and acertain level of informality within the peerteam have to be built up. The membershave different experiences and differentperspectives on the selected policy issue.Policymakers expect that new policyreforms will be implemented in the waythey have planned them, whereas school

directors are inclined to implement reformsaccording to the needs of their school.These differences in perspective play animportant role in the creation of rich policydebates.

Thirdly, the peer learning events in theparticipating countries are the core of theETF peer learning methodology. Each

country hosts a peer learning event, so theparticipants are successively hosts andguests. The peer learning events arecarefully planned and structured aroundthe concept of stakeholder involvement inthe policy process in the country visited.The list of stakeholders in education is verylong and includes the parliamentaryeducation commission, ministers anddeputy ministers, heads of department inthe ministry, legislative bodies, teachersunions, social partners, companies, variousassociations of parents, teachers, students,etc. The question of how different

context to another, gives dynlearning experience and show

context is very much colouredstakeholders in the policy proParticipants feel responsible,a peer learning event, of tryinthe situation in their country, wother countries they are the ocritical observations and quescombination gives them a strocommitment to policy issues

learning, using comparisons countries as a crucial driver inprocess.

4. IMPORTANCE OF TPOLICY ISSUE

As mentioned above, the selepolicy issue is crucial for linkinlearning to policy discussions icountries and for stimulating ththe peers representing differengroups. The common denominrationale of appraisal firmly empractice. This is how the ETF pmethodology approaches the p

aspect of the policy process, othrough the lens of a problem wrelevant to the policy agenda.

This focused approach starts wdevelopment of a so-called sedocument, which usually provioverview of the major developm

country relating to the policy issintention is also to include a pranalysis, a stakeholder analysidentify the objectives and impissues that are of most interestparticipating country. The coun



46/136

Box 1: Peer learning methodology in practice

In the 2007 ETF peer learning exercise, the policy issue selected wof education polices: impact on schools and school management. Thas been applied in different contexts in relation to the countries invKosovo and Turkey. Each of these countries selected a country teatwo school principals (one from a pilot and the other from a non-piloschool), as well as a VET expert. The team consisted of peers fromand one guest peer from the former Yugoslav Republic of MacedoMontenegro20.

The team moved from country to country and took part in the three events to observe and discuss the policy issue from different pointsunderstanding of how policies travel up and down the various interbetween the centre (i.e. ministry of education as main policy design(major unit where policies are implemented), was the main topic forlearning team. A powerful way of understanding the implementationask two related questions:

n How do reforms change schools?n How do schools change reforms?

Other questions that guided the exercise included:

n How do school-level actors and stakeholders make sense of newdo they interpret them?

n Do they faithfully implement policies that have been centrally de

letter, or do they transform them, possibly by adapting them to senvironment, the schools institutional culture and the surroundinneeds?

n What opportunities exist for school-based actors to contribute tothe policies?

n Are there cases where school-based actors subvert policies thatat a higher level than the school? If so, how and why does this h

n Are such transformations and adaptations, accommodations and

educationally sound, or are they problematic?n How do new policies implemented in pilot schools travel to non-all?

n Who decides when a policy has been successfully implementedand on what grounds?21



47/136

asking questions, while at the same time itbuilds up commitment on the part of the

hosts. The purpose of the self-studydocument is thus to have team membersanalyse national policies, legislation orprogrammes that have been implemented.It should identify the key problems, keyindicators and qualitative information,describe the formulated policy intentions,priorities, barriers and the effects ofimplementation, and set out the most

important tasks and decisions still pending.

Although the self-study country report is anessential part of the peer learning project,providing background information for thediscussions and creating strong stakeholderinvolvement, the ETFs two years ofexperience shows that the country teams do

not always succeed in developing the paperbeyond the level of a simple policy report anda (self-)critical analysis. The focus is normallyclear and targeted, but often the report isquite descriptive and too open about thechallenges of policy implementation andpol

Documents

ETF Yearbook 2008 - policy learning in action