Proposal for a structure and process for transnational ... fileProposal for a structure and process for transnational European peer reviews Results of the work undertaken by the thematic

Proposal for a structure and process for transnational European peer reviews

Results of the work undertaken by the thematic group on Peer review

European Network for Quality Assurancein Vocational Education and Training

Report drafted by Maria Gutknecht-Gmeiner

© European Network for Quality Assurance in VET, 2009. All rights reserved.You are welcome to use this material but please remember to quote ENQA-VET in all references.

This report reflects the opinions of the participants of the thematic group and does not constitute an official European Commission position.

2

TABLE OF CONTENTS

Page

FOREWORD …………………………………………………………………………………3 ACKNOWLEDGEMENTS ………………………………………………………………….4 EXECUTIVE SUMMARY……………………………………………………………………5-6

1. POLICY CONTEXT ……………………………………………………………………...7-8

2. GENERAL BENEFITS OF (TRANSNATIONAL) EUROPEAN PEER REVIEW...9-10

3. CONTRIBUTION OF EUROPEAN PEER REVIEW TO EUROPEAN VET POLICY ………………………………….………………….…..11-15

3.1 European Peer Review and the EQARF …………………………………......11-14 3.2 European Peer Review and the Bordeaux Communiqué ………...………...14-15 3.3 European Peer Review and the cooperation between VET and HE ……....15

4. PROPOSAL FOR A PROCESS AND STRUCTURE FOR EUROPEAN PEER REVIEW ……………………………………….………...….16-25

4.1 Common principles for European Peer Reviews …………………..………...16-18 4.2 Institutional structure …………………………………………………………….18-20 4.3 Task and functions ………………………………………………………………21-25 4.4 Funding …………………………………………………………………………...25

5. OUTLOOK: EUROPEAN PEER REVIEW 2010-2013 …………………………..26-27 5.1 Participation of Member States and current forecast ………………………..26 5.2 Scenarios …………………………………………………………………………26-27

6. SCENARIOS FOR TRANSNATIONAL EUROPEAN PEER REVIEWS……….28-40 6.1 Scenario 1: Grass-roots model ……………………………………………….28-32 6.2 Scenario 2: Decentralised coordination model ……………………………..32-35 6.3 Scenario 3: Transitional cooperative model ………………………………..36-39

7. RECOMMENDATIONS AND CONCLUSIONS ………………………………….40-41 7.1 Recommendation by the thematic group on peer review ………………….40-41 7.2 Conclusions: further steps …………………………………………………….41

ANNEX 1: Peer Review and the EQARF –a detailed analysis …………………42-44 ANNEX 2: Bibliography ……………………………………………………….……...45-47 ANNEX 3: List of Participants ………………………………………………….……48

3

FOREWORD

The work undertaken in the context of the ENQA-VET work programme (2008-2009)

which has been funded by the European Commission within the framework of the Life-

long Learning Programme, has had a two-fold character. At one level it has sought to

produce policy-useful material to support Member States in developing the instruments

and tools which will be important for the implementation of the EQARF Recommenda-

tion. At a second level, the intensive cooperation between countries in areas of shared

policy concern, has contributed to a greater level of understanding of each others sys-

tems, mutual learning, and an increased culture of quality assurance in Member States.

This process has involved policymakers and VET providers as well as the social part-

ners as key players in developing appropriate responses to policy implementation is-

sues in regional and national contexts within the Member States.

The work of the thematic groups has provided an opportunity to mobilise policymakers

and specialists in areas of key policy interest for improving the quality of VET, such as

the common understanding of indicators, peer review the development of guidelines to

support quality, and how to make VET more attractive. The outcomes of this work will

play a significant role in supporting Member States in developing their national plans

for the implementation of EQARF, as well as informing the policymaking process at

Member State and European levels.

The results of the thematic groups will also be of particular interest to the European

Commission as it addresses the issue of how to best support the successful implemen-

tation of EQARF in the coming years.

The reports of the thematic groups are not designed to be static however. They repre-

sent best thinking at a particular moment in time. They will now be used at a more gen-

eral level, involving actors and countries who are at different stages of development, as

they reflect on what effective implementation of the EQARF will mean in their context.

This material will feed into this process and in turn support the development of the in-

cremental policy learning which is a key part of the open method of coordination as

applied to education and training.

Sean Feerick Director European Network for Quality Assurance in Vocational Education and Training (ENQA-VET)

4

ACKNOWLEDGEMENTS

The Secretariat would like to thank the members of the thematic group who have con-

tributed to the preparation of the outcomes which are published in this report. Their

commitment in sharing expertise and experience and their sustained collaboration be-

tween meetings has ensured the policy relevance and usefulness of the material pro-

duced.

The research and publications produced by both ETF and CEDEFOP have been of

particular importance in supporting the work of the thematic group. We would like also

to thank the representative of ETF, Margareta Nikolovska, for her input into the work of

this thematic group.

In particular we would like to thank the Österreichische Referenzstelle für Qualität in

der Berufsbildung, ARQAVET – the Austrian Reference Point for Quality Assurance in

Vocational Education and Training- for hosting the work of the thematic group, Leena

Koski for chairing the group and Maria Gutknecht-Gmeiner for facilitating communica-

tion between members and preparing the final report.

5

EXECUTIVE SUMMARY

Between 2004 and 2009, a European Peer Review procedure has been developed and

tested in European cooperation in a series of three Leonardo da Vinci projects (Peer

Review in initial VET, AT/04/C/F/TH-82000, 2004-2007; Peer Review Extended, LE-

78CQAF, EAC/32/06/13, 2007; Peer Review Extended II, LLP-LdV/TOI/2007/AT/0011,

2007-2009). From the outset, European Peer Review aimed at contributing to Euro-

pean policies regarding quality in VET (formerly laid down in the CQAF, currently in the

EQARF Recommendation).

The success of the Peer Review projects, the interest of a majority of the ENQA-VET

Member States and the high potential of Peer Review in the advancement of the

EQARF and related European policies now calls for an implementation of transnational

Peer Reviews at the European level. To promote further exploitation of the results of

the three Leonardo da Vinci projects and to maximise the benefit from current national

efforts to implement the European Peer Review, the Thematic Group on Peer Review

set up in the ENQA-VET framework has developed a structure and process for future

transnational European Peer Reviews.

The proposal made by the group reflects a high demand from VET providers from at

least 18 Member States to start transnational Peer Reviews by 2010 (about 250 trans-

national Peer Reviews were forecast in a survey of ENQA-VET Member States. The

proposal includes a section on the history and policy-relevance of European Peer Re-

view and describes elements of a structure and common principles as well as tasks

and functions pivotal for successful implementation of transnational European Peer

Reviews.

Three scenarios are outlined differing in allocation of tasks and responsibilities to the

different actors, namely the European coordinating body, national lead bodies in par-

ticipating Member States (possibly the QANRPs), and the VET providers. The scenar-

ios are: a “grass-roots model” (scenario 1), a “decentralised coordination model” (sce-

nario 2), and a “transnational cooperative model (scenario 3). While scenario 1 relies

on grass-roots activities by VET providers with no European coordination and support,

scenarios 2 and 3 encompass a coordinating body on the European level which in the

“decentralised coordination model” would primarily provide technical and administrative

support, while in the “transnational cooperative model” a more active and stringent co-

operation between the different actors is foreseen with a coordination body that also

provides expert support for both the national lead bodies (coordination, dissemination,

6

operative support of transnational Peer Reviews) and the VET providers involved

(training, Peer database, matching).

The Thematic Group recommends the implementation of scenario 3. It is not only the

most effective and efficient model in the longer term but, most importantly, will ensure

that the full potential of European Peer Review in terms of a sustainable contribution to

the implementation of the EQARF and other related European policies can be ex-

ploited. The Thematic Group was unanimous in this recommendation.

If implemented properly, European Peer Review as a practical, tangible and concrete

tool to implement the EQARF recommendation has the potential to become one of the

main pillars of European VET quality policy in the next 5 to 10 years encompassing all

relevant actors – VET providers, social partners and other stakeholders as well as the

VET systems of the participating Member States.

7

1. POLICY CONTEXT

Peer Review is a highly recognised and longstanding instrument of external quality

assurance in the realm of higher education. Within vocational education and training,

however, its use has so far been marginal. In the wake of the “Copenhagen Declara-

tion”1, the potential of Peer Review for the vocational education and training sector was

recognized in the mandate of the Technical Working Group on Quality in VET2. Be-

tween 2004 and 2009, the assignment to make promote and use of Peer Review also

at the VET provider level was taken up in three Leonardo da Vinci projects on Peer

Review: Peer Review in initial VET, AT/04/C/F/TH-82000, 2004-2007; Peer Review

Extended, LE-78CQAF, EAC/32/06/13, 2007; Peer Review Extended II, LLP-

LdV/TOI/2007/AT/0011, 2007-2009.

In the European Peer Review projects, the Peer Review methodology prevalent in

higher education was transferred and tailored to the VET sector (www.peer-review-

education.net). A common European standard for Peer Review of VET providers was

agreed upon by the 15 participating European countries and laid down in the “Euro-

pean Peer Review Manual for initial VET”3 which offers extensive information and guid-

ance on the Peer Review process at provider level. Peer Review was defined as a vol-

untary external evaluation supports European VET providers in their efforts to improve

the quality of their provision and enhance their accountability. Peer Review combines

self-evaluation with an external professional assessment conducted by peers – experts

of equal standing with those being externally evaluated.

Between 2006 and 2009 the European Peer Review of VET providers was successfully

piloted in 26 transnational Peer Reviews involving 11 European countries, demonstrat-

ing its efficacy and utility for improving the quality of VET provision and contributing to

ENQA-VET’s objective to build sustainable European cooperation and exchange of

good practice within and among Member States, between VET and HE and between

VET providers and relevant stakeholders.

The highly satisfactory results of the Peer Review pilot projects have led to a call for

the continuation of transnational Peer Reviews of VET providers at the European Level

1 Council Resolution of Dec. 19, 2002, OJ C 13, 18.1.2003 2 The mandate comprised the assignment to “promote the exchange of good practice and the use of volun-tary peer review at different levels. […]“. European Commission (2002/2003): Increased cooperation in vocational education and training. Mandate of the technical working group on quality in VET.

8

– i.e. of Peer Reviews in which at least one Peer from another country participates as

part of the Peer Team – both by VET providers and policy makers from ENQA-VET

Member States (cf. e.g. the Pécs Conference conclusions and the conclusions of the

project “Peer Review Extended”4).

The European Peer Review projects were closely followed by the TWG on quality in

VET (2003-2004) and its successor ENQA-VET (cf. ENQA-VET Work Programme

2006-2007, 8). Within the ENQA-VET 2008-2009 Work Programme, a Thematic Group

on Peer Review was set up with the aim to develop a proposal for a sustainable struc-

ture and process for transnational European Peer Reviews

At the same time, the European Peer Review developed in the three European projects

is currently being piloted for further implementation within the overall national/regional

quality assurance systems for VET in several European countries, among which are

Austria, Finland, Italy, Hungary and the Spanish region of Catalonia5. In other countries

such as the United Kingdom, grass-roots developments of Peer Review between VET

providers have also achieved widespread application.6

In order to make further use of the ground-work laid by the three LdV projects and to

exploit the synergies with current European VET policies, continued European coop-

eration on Peer Review in VET is essential.7

For the further implementation of transnational European Peer Reviews both the de-

velopments on the European level – ENQA-VET activities, the establishment of Quality

Assurance National Reference Points (QANRPs), the endorsement of the EQARF in a

recommendation of the European parliament and the council – and on national level

(where applicable) – set-up of national structures and procedures to implement Peer

Review – have to be taken into account.

3 Gutknecht-Gmeiner, Maria; Lassnigg, Lorenz; Stöger, Eduard; de Ridder, Willem; Strahm, Peter; Strahm,

Elisabeth; Koski, Leena; Stalker, Bill; Hollstein, Rick; Allulli, Giorgio; Kristensen, Ole Bech (2007): Euro-pean Peer Review Manual for initial VET. Vienna, June 2007. 4 Pécs Conference Conclusions Sept.14, 2007,

cf. www.peer-review-education.net /TCgi/TCgi.cgi?Target=home&P_Kat=18. 5 Gutknecht-Gmeiner, Maria (ed.) (2007): Implementing Peer Review as part of the CQAF. Scenarios for

Peer Review Implementation in Austria, Spain (Catalonia), Finland, Hungary, and Italy. Contributions by Koski, Leena; Molnar-Stadler, Katalin; Allulli, Giorgio; Tramontano, Ismene; Camps, Josep; Canyadell, Pere; Vienna. 6 Cf. http://excellence.qia.org.uk/page.aspx?o=resourcecentre.

7 Incidentally, this would also make for a best practice example of how Leonardo da Vinci has moved from

innovation laboratory stage to system implementation on the European level – on the Member States’ level this transfer has already been established through national implementation.

9

2. GENERAL BENEFITS OF (TRANSNATIONAL) EUROPEAN PEER REVIEW

Within the new governance paradigm of steering through objectives and evaluation on

the VET provider level, Peer Review might prove to be a powerful complement (or even

substitute) for other (and often more expensive) forms of external evaluation (full in-

spections, external evaluation carried out by professional evaluators or consultants

etc.).

It builds upon the self-assessment of VET providers and combines the internal ef-

forts for quality assurance with an external evaluation. Contrary to other external

assessments, evaluations, and inspections, Peer Review is voluntary and focuses on

quality development and improvement. Furthermore it taps on the professional know-

how of the VET practitioners and ensures that the know-how and results generated in a

Peer Review remain within the profession and are easily disseminated – both the re-

viewed institution and the Peers (and the institution they come from) benefit from a

Peer Review.

Thus, as a spin-off, Peer Review also highly contributes to the professional develop-

ment of the teachers, trainers, counsellors and other practitioners from VET who act as

Peers. Apart from the personal development, taking part in a Peer Review and the re-

quired Peer Training can be considered a demanding crash course in educational

evaluation and quality management.

Last but not least, the European Peer Review has so far met with high acceptance by

VET providers. The European Peer Review is considered a professional, open and

improvement-oriented procedure of external evaluation which tackles the crucial

issues of VET provision – learning and teaching – and leads to improvement. European

Peer Review directly contributes to the build-up of a professional quality ethos and

culture within VET.

Because of the inherent cooperation between VET providers from countries, transna-

tional Peer Review additionally fosters cooperation and mutual understanding be-

tween members states. It leads to the exchange of good practices and the transfer

of innovation at both VET provider and system levels.

10

Graph 1: Added- value of transnational Peer Reviews on the different levels of

VET

Peers VET providers VET Systems European level

Contacts to colleagues and VET providers from other countries

Contacts to VET providers from other countries

Internationalisation: Bottom-up European Cooperation between teachers/trainers and VET providers

European integration: Bottom-up European Cooperation between teachers/trainers and VET providers

International experi-ence; Insights into other VET systems

International coop-eration as a prereq-uisite for success in an increasingly glob-alised environment

International coopera-tion as a prerequisite for success in an in-creasingly globalised environment

Promotion of quality in VET on a European level

Good practice and innovation transfer from other countries



Good practice and inno-vation transfer between Member States

Professional develop-ment in an interna-tional setting

Higher credibility of external evaluation through transnational Peers and a common recognised European evaluation methodol-ogy

Higher credibility of external evaluation through transnational Peers and a common recognised European evaluation methodol-ogy

Mutual understanding and trust through a common, higher credibil-ity and transparency through a recognised European evaluation methodology

Professional develop-ment of teach-ers/trainers on interna-tional level, develop-ment of a professional quality ethos in VET

Professional develop-ment of teach-ers/trainers on interna-tional level, develop-ment of a professional quality ethos in VET

Contribution to all major European policies in VET

Sustainable coopera-tion and networking



Sustainable cooperation and networking

Source: Gutknecht-Gmeiner 2009

11

3. CONTRIBUTION OF EUROPEAN PEER REVIEW TO EUROPEAN VET POLICY

Peer review has important benefits at European, national/regional and VET providers’

level. Peer Review supports the implementation of EQARF in Member States by

� contributing to the improvement of the quality of VET provision

� promoting a shared quality culture at VET provider level

� stimulating mutual learning from good practice across Europe

� fostering European Networking of VET providers and encouraging mobility

� enhancing cooperation of VET providers and Member States at European level

� supporting mutual trust among and within Member States

� disseminating good practice, enhancing synergies, and fostering the transfer of

innovation both between VET providers and Member States.

As a bottom-up approach to quality assurance between VET providers Peer Review

also complements and supports other quality initiatives and activities on the European

or Member States’ level.

3.1 European Peer Review and the European Quality Assurance Reference Framework (EQARF)

The Peer Review procedure described in the European Peer Review Manual fully im-

plements the elements and principles of the Common Quality Assurance Framework8

by providing a systematic approach to external evaluation following the quality

assurance and improvement cycle (European Peer Review Manual, p. 3). It thus

puts into practice the quality criteria at the Provider level outlined in the EQARF (Annex

1)9.

In particular, the European Peer Review constitutes a promising new instrument for

external evaluation and assessment of VET providers. It combines internal and ex-

ternal evaluation and thus complements the currently prevalent methodology of self-

evaluation. It is considered an attractive and motivating methodology which pro-

motes bottom-up cooperation between VET providers and nurtures the development

8 Technical Working Group ‘Quality in VET’ (2004): Fundamentals of a ‘Common Quality Assurance

Framework’ (CQAF) for VET in Europe. European Commission, Directorate-General for Education and Culture, Vocational training: Development of vocational training policy. 9 Recommendation of the European Parliament and the Council on the establishment of a European Qual-

ity Assurance Reference Framework for Vocational Education and Training, Brussels, 2.4.2009, PE-CONS 3746/08.

12

of a professional quality-improvement culture within the VET sector. European

Peer Review by definition also relies on and necessitates exchange and networking

between VET providers within and across Member States. Mutual learning from good

practice and innovation transfer lie at the heart of European Peer Review. The ex-

perience of Peer Review indicates that it also represents a vehicle for further capacity-

building and sustained organisational improvement. In the past, Peer Reviews have

also led to further cooperation between VET providers including the establishment of

transnational mobility schemes.

It also complements the quantitative approach underlying most quality assurance

mechanisms by offering a criteria-based qualitative external evaluation which yields

valid results which are immediately understandable to the evaluated VET provider and

therefore also directly applicable in practice.

A European Peer Review framework will rely on the European Quality Assurance

National Reference Points (QANRPs) establishing synergies between Peer Review

and other activities of the EQARF network.

Graph 2: Contribution of Peer Review to the European Quality Assurance Refer-ence Framework

European Peer Review fully supports the dissemination and voluntary implementation of

the EQARF in national and transnational VET practice

Quality Criteria implemented in the European Peer Review

Dissemination and practical use of quality criteria enhanced through Peer Review

Peer Review as a

new methodology ofexternal evaluation

(Further) development and use of descriptive

indicators

Peer Review process

(Analysis of Self-Report; Peer Visit)

Follow-up(Consequences of review report and follow-up activi-

ties)

Peer Review Report (strengths

& improvement areas)

Self-assessment of VET providers

(Self-Report)

Peer Review

Methodology

�� Applicability of a common European standard for Peer Review within and between Member States which respects the rich diversity of national VET systems.

�� Contribution to mutual trust, increased cooperation and transparency across Europe.

�� Peer Review as an instrument on the way towards a European Area of VET.

13

European Peer Review has so far proven to be applicable in 15 European countries not

only within the national systems but also in transnational European exchange while

fully respecting the diversity of national systems and the individual cultures,

needs and requirements of VET providers all over Europe.

If Peers from other countries are involved – as is the case in transnational European

Peer Reviews –, Peer Review enhances transparency and comparability – both regard-

ing the evaluation methodology and the quality areas/indicators used – and directly

stimulates exchange and cooperation between VET providers on a transnational

level.10 Consequently, European Peer Review can be regarded a relevant tool for

assessing VET provision at institutional level within and across countries. It

helps to “identify and to assess good practices, to assess how good practices can be

effectively transferred, and facilitates mutual learning” between VET providers (cf. Fun-

damentals of a ‘Common Quality Assurance Framework’ (CQAF) for VET in Europe, p.

13).

As a voluntary and improvement-oriented instrument, Peer Review also complies

with the general principle of voluntary participation laid down in the EQARF.

Most importantly perhaps, the European Peer Review as a point of reference and

common framework for implementing Peer Review at VET provider level will in-

crease transparency and comparability thus enhancing mutual learning and mutual

trust. The acceptance of the European Peer Review in more than 15 European coun-

tries so far has been reached through a bottom-up voluntary consensus-building proc-

ess throughout the development and testing of the European Peer Review in the Leo-

nardo da Vinci projects (2004-2009) and in the Thematic Group on Peer Review of

ENQA-VET (2008-2009).

A European Peer Review Network comprising VET providers and other stakeholders

from all over Europe who engage in mutual reviews should contribute to the develop-

ment of a common European Area of VET (cf. Helsinki Communiqué).

The 14 Quality Areas and exemplary indicators at VET provider level developed for

use in transnational Peer Reviews (cf. European Peer Review Manual; Peer Review

Tool-box) adhere to the systematic criteria of the quality cycle proposed by the

EQARF (cf. European Peer Review Manual p. 36; cf. also Annex 1 of EQARF). In par-

ticular, they relate to the majority of the descriptive indicators of the EQARF and are in

10 Cf. Gutknecht-Gmeiner, Maria (ed.) (2007b): Implementing Peer Review as part of the CQAF. Scenarios

for Peer Review Implementation in Austria, Spain (Catalonia), Finland, Hungary, and Italy. Contributions by Koski, Leena; Molnar-Stadler, Katalin; Allulli, Giorgio; Tramontano, Ismene; Camps, Josep; Canyadell, Pere; Vienna.

14

line with policy rationales underlying the reference set of selected quality indicators for

assessing quality in VET (Annex 2 EQARF. To fully benefit from the use of the Peer

Review quality areas, criteria and exemplary indicators developed for Peer Reviews for

the implementation of the EQARF, some further development and fine-tuning to the

EQARF criteria will be helpful.

The Quality Areas have so far met with acceptance in all countries involved in the

European Peer Review projects; practice tests in eleven countries have shown their

practical usefulness and applicability. European Peer Review puts the quality and ef-

fectiveness of the core processes of VET – learning and teaching – in the fore-

ground of evaluation and improvement activities (cf. European Peer Review Manual, p.

35).

If adapted and directly linked to the EQARF set of indicative descriptors, the exemplary

indicators used in transnational Peer Reviews can help pave the way to a common

European understanding of quality criteria and indicators for VET providers on the

European level.

3.2 European Peer Review and the Bordeaux Communiqué, the EQF and ECVET

European Peer Review also explicitly supports and promotes the priority areas pro-

posed in the Bordeaux Communiqué11

� implementing the tools and schemes for promoting cooperation

� heightening the quality and attractiveness of VET

� strengthening European cooperation arrangements

by providing Peer Review as a powerful key European quality assurance instrument (p.

8f.), which promotes cooperation and enhances mutual learning, is flexible to different

needs and highly accepted at VET providers’ level. Improvement of VET provision re-

sulting from European Peer Review as well as a shared evaluation methodology with

Higher Education will heighten the visibility of high-quality VET and increase its attrac-

tiveness.

Quality assurance of VET Provision also lies at the heart of European instruments in

VET such as the European Qualifications Framework (EQF) and the European Credit

11 Bordeaux Communiqué on enhanced European cooperation in vocational education and training. Com-

muniqué of the European Ministers for vocational education and training, the European social partners and the European Commission, meeting in Bordeaux on 26 November 2008 to review the priorities and strate-gies of the Copenhagen process, p.7ff.

15

System for Vocational Education and Training (ECVET). Peer Review specifically

makes a substantial contribution to the implementation of EQF and ECVET by

� supporting a quality-improvement culture at VET providers’ level

� providing a highly accepted, formative external evaluation, supported by qualitative

analysis

� providing a framework for the identification, support, and exchange of good prac-

tices

� enhancing transparency of VET quality and promoting mutual trust.

3.3 European Peer Review and the cooperation between VET and Higher Educa-tion

Peer Review also fosters "cross-fertilisation" (cf. ENQA-VET Work Programme

2006/2007) and cooperation between VET and Higher Education through a shared

external evaluation methodology. The common benefits and challenges of Peer Review

for Higher Education and VET have also been stated in the Peer Review Workshop

conclusions of the 2006 Graz Conference on Quality in VET and HE12. Additionally,

mutual participation in Peer Reviews of VET and HE experts enhances mutual under-

standing and cooperation between the two sectors.

12 WS 5 “Peer Review in VET and Higher Education”, Conference „Quality Assurance in Higher Education and Vocational Education and Training”, 11/12 May, University of Graz eu2006.bmbwk.gv.at/veranst/qualitaetssicherung_dok.htm.

16

4. PROPOSAL FOR A PROCESS AND STRUCTURE FOR EUROPEAN PEER RE-VIEW

The proposal for a process and structure for European Peer Review is based on an

analysis of elements needed for a sustainable conduct of European Peer Reviews on

the European level. The proposal includes

� an elaboration of common principles and an institutional structure which in-

cludes all relevant actors for European Peer Reviews as the overarching element.

� a list of tasks and functions which can be attributed to the different actors, and

� proposals for funding including a list of possible sources.

An overview of this “Peer Review architecture” is given in Graph 3 (page 21).

4.1 Common principles for European Peer Reviews

The proposal for common principles devised by the Thematic Group on Peer Review

should apply to all actors involved. They comprise general principles for conducting

European Peer Reviews and principles for cooperation between the different actors.

4.1.1. General principles for conducting European Peer Reviews

Two overall principles frame the conduct of European Peer Reviews:

1) European Peer Reviews should be conducted in a way to support and promote the

further implementation and development of the EQARF and other relevant European

policies concerning Quality in VET (cf. also above IV. Contribution of European Peer

Review to European VET policy).

2) In the conduct of European Peer Reviews, the aims, principles, requirements and

guidelines set out in the European Peer Review Manual are adhered to.

Specifically, this means the compliance with the following principles:

� European Peer Review is a voluntary activity.

� European Peer Review supports the improvement of VET provision through a high

quality external evaluation and the set up and implementation of adequate follow up

procedures.

� European Peer Reviews relate to the EQARF quality criteria and indicative descrip-

tors and refer to the EQARF set of quality indicators. With some adaptation, they

will useful for direct implementation of the EQARF.

17

� Involvement of peers as persons of equal standing with those being peer reviewed.

� Objectivity and impartiality of peers and avoidance of conflicts of interest.

� Confidentiality and protection of individual data rights and principles of fairness dur-

ing the process and in the use of results.

� Transparency of the Peer Review procedure and assessment criteria to all persons

involved.

� Promotion of openness, integrity and sincerity as a prerequisite for mutual learning.

� Awareness of cultural influences and diversity of VET systems.

� Quality assurance, monitoring and continuous improvement of the European Peer

Reviews process.

4.1.2 Principles for cooperation

The principles for cooperation are based on the "general principles for conducting

European Peer Reviews" (cf. above V.1.1) They help clarify the structure for transna-

tional European Peer Reviews by highlighting the tasks and responsibilities of the ac-

tors on the different “levels - e.g. individuals (like Peers) and VET providers, na-

tional/regional etc. bodies, the European Coordinating body and ENQA-VET/the Euro-

pean Commission – and define interfaces, information flows and decision-making

processes. Two aspects of cooperation must be taken into account: 1) the cooperation

between the different "levels" and 2) the cooperation between the Member States. At

this stage, a set of general principles is proposed which needs to be elaborated once

tasks and responsibilities have been attributed.

General principles for cooperation include

� compliance with the subsidiarity principle

� compliance with the proportionality principle because European Peer Review does

not replace or define national Peer Review approaches

� mutual trust between the actors involved

� open access to information

� transparency of decision-making

� effectiveness and efficiency

18

� avoidance of unnecessary bureaucracy

� evaluation of the cooperation structure and processes and

� provisions for systematic improvement of cooperation structure and processes.

4.2. Institutional structure

The institutional structure comprises

� the VET providers from participating European countries

� a national/regional leading body which has the tasks and responsibility of coordinat-

ing and supporting national activities and liaising with the European level

� a coordinating body for European Peer Reviews on the European level, and

� ENQA-VET (res. the EQARF network) and the European Commission.

VET providers are the primary target group of European Peer Review activities. It must

be remembered that it is for their benefit that European Peer Reviews are coordinated

at European level. Resources, bottom-up initiatives as well as requirements and inter-

ests of VET providers must therefore play a dominant role in the set-up of European

Peer Reviews.

The national/regional lead bodies perform an important function as a liaison between

national activities and the European level. They also involve social partners, re-

gional/local/sectoral authorities and other relevant stakeholders. To avoid duplication of

efforts and funding, the QANRPs as important liaison and supporting institutions on the

national level may also assume the role of national lead bodies for Peer Review. Their

concrete tasks and responsibilities remain to be defined in detail as soon as the gen-

eral approach to the implementation of European Peer Review has been decided.

A coordinating body on the European level has been deemed a necessary and pivotal

element of the institutional structure by the members of the Thematic Group on Peer

Review. The coordinating body safeguards the feasibility and quality of European Peer

Reviews and supports European coordination and learning. Again, the concrete tasks

and responsibilities of this body will depend on the roles and functions of the other ac-

tors involved.

ENQA-VET in coordination with the European Commission can function as the entity

which commissions and supervises European Peer Review activities. The commission-

ing and establishment of a Coordinating Body (including a tendering procedure) is one of

19

the functions which should be assumed on the European level. In the establishment of a

coordinating structure, the following tasks have been identified:

� elaboration of evaluation criteria for the Coordinating Body selection

� tender procedure and selection of the Coordinating Body

� detailed cooperation procedures between VET providers, national lead bodies, co-

ordinating body and ENQA-VET/EC.

20

Gra

ph 3

: Ele

men

ts o

f a E

urop

ean

Peer

Rev

iew

“ar

chite

ctur

e”

Act

ors/

Are

as

VET

pro

vide

rs

Nat

iona

l/reg

iona

l etc

. bod

ies

Eur

opea

n co

ordi

natin

g bo

dy

EQ

AR

F ne

twor

k/E

C

Com

mon

prin

cipl

es

a)

Gen

eral

prin

cipl

es

Inte

grat

ion

into

Eur

opea

n V

ET

qual

ity p

olic

ies

and

prin

cipl

es fo

r con

duct

ing

Pee

r Rev

iew

s, a

pply

to a

ll le

vels

b)

Coo

pera

tion

prin

-ci

ples

P

rinci

ples

for c

oope

ratio

n in

faci

litat

ing

trans

natio

nal E

urop

ean

Pee

r Rev

iew

; app

ly to

all

leve

ls

−

oper

atio

nal r

espo

nsib

ilitie

s in

cl. m

onito

ring

(cf.

belo

w)

− na

tiona

l coo

rdin

atio

n

− op

erat

iona

l res

pons

ibilit

ies

incl

. mon

itorin

g (c

f. be

low

) −

coor

dina

tion

of n

atio

nal

leve

l/VE

T pr

ovid

ers

− re

porti

ng to

EQ

AR

F ne

t-w

ork/

EC

Gen

eral

dec

isio

n-m

akin

g S

truct

ure

−

nom

inat

ed b

y M

S/re

gion

s −

inte

grat

e Q

AN

RP

, if p

ossi

ble

Sele

cted

by

EQ

AR

F ne

twor

k an

d/or

EC

follo

win

g C

all f

or

Tend

er

Cal

l for

Ten

der f

or

Coo

rdin

atin

g B

ody

Common principles, Structure, Cooperation

Coo

pera

tion

base

d on

coo

pera

tion

prin

cipl

es

fine-

tune

s de

finiti

on o

f rol

es a

nd re

spon

sibi

litie

s of

diff

eren

t act

ors

with

in th

e ov

eral

l stru

ctur

e c

oope

ratio

n be

-tw

een

diffe

rent

act

ors

on d

iffer

ent l

evel

s A

ND

bet

wee

n M

embe

r Sta

tes

Info

rmat

ion/

D

isse

min

atio

n

− di

ssem

inat

ion

and

info

rmat

ion

by a

nd

amon

g V

ET p

rovi

ders

−

netw

orki

ng b

etw

een

VE

T pr

ovid

ers

− w

ide

scop

e of

info

rmat

ion

and

diss

emin

atio

n ac

tiviti

es

on n

atio

nal l

evel

−

coor

dina

tion

with

Eur

opea

n le

vel a

nd o

ther

MS

pend

ing

Eur

opea

n fu

ndin

g

− w

ide

scop

e of

info

rmat

ion

and

diss

emin

atio

n ac

tiviti

es

on E

urop

ean

leve

l −

coor

dina

tion

with

MS

and

E

QA

RF

netw

ork/

EC

Sup

port

info

rmat

ion

and

diss

emin

atio

n

Sup

port

supp

ort i

n co

mm

uniti

es

of p

ract

ice

(e.g

. ex-

chan

ge o

f goo

d pr

ac-

tice,

men

torin

g)

− op

erat

iona

l tas

ks a

nd re

-sp

onsi

bilit

ies

in s

uppo

rting

V

ET p

rovi

ders

on

natio

nal

leve

l

− op

erat

iona

l tas

ks a

nd re

-sp

onsi

bilit

ies

in s

uppo

rting

V

ET

prov

ider

s on

Eur

opea

n le

vel

− M

onito

ring

and

QA

of n

a-tio

nal a

ctiv

ities

−

Sup

port

for Q

A o

n th

e E

uro-

pean

leve

l

− M

onito

ring

and

QA

of E

uro-

pean

act

iviti

es

− S

elf-e

valu

atio

n to

geth

er

with

nat

iona

l bod

ies

− ex

tern

al m

eta-

eval

uatio

n −

impa

ct a

naly

sis

Tasks and responsibilities

QA

and

Lear

ning

Exc

hang

e of

goo

d pr

actic

es a

nd n

etw

orki

ng; C

ertif

icat

ion/

Qua

lity

Sea

l/Lab

el

Impl

emen

tatio

n of

pro

cedu

res

for c

hang

e (le

arni

ng fr

om e

valu

atio

n re

sults

) Fu

nd-

ing

Sha

red-

cost

s-pr

inci

ple*

* ow

n fu

nds

of

VET

pro

vide

rs

natio

nal f

undi

ng

Eur

opea

n fu

ndin

g

Sou

rce:

Gut

knec

ht-G

mei

ner2

009,

base

don

the

resu

ltsof

the

Them

atic

Gro

upon

Pee

rRev

iew

mee

ting

ofD

ecem

ber3

-4,

21

4.3. Tasks and functions

4.3.1. Information and dissemination

Information and dissemination on all levels is a crucial prerequisite for successful con-

duct of European Peer Reviews.

A positive attitude must be created through a suitable information and dissemination

strategies for the different target groups: national bodies and experts, VET providers,

Peers.

The sharing of experiences and good practice plays a central role, networking and di-

rect contacts between national bodies, VET providers and potential Peers from differ-

ent countries is essential.

Special attention must also be paid to those countries with no Peer Review experience

so far, facilitating first-hand experience of Peer Reviews is a promising strategy.

The following activities have been identified:

� Use of the European Peer Review Manual as the primary source of information on

European Peer Review in information and dissemination activities.

� Use of other material developed by the European Peer Review projects, if neces-

sary, translation of this material.

� Creation of information and dissemination material for target groups, translate it into

different languages and disseminate it to the different target groups (Brochures,

Flyers, Websites, Reports etc.)

� Offer of special information and support for countries with no Peer Review experi-

ence

� Information and training of national lead bodies

� Organisation of a Peer Learning Activity on "Peer Review” on the European level

� Organisation of conferences and workshops for different actors (special events or

mixed audiences; regular / virtual / national / European ), ¨

� Special attention must be paid to VET providers - the actors on national level are

more import in first phase in those counties where there is no experience on Peer

Reviews,

� Offer of opportunities for networking between VET providers

� Organisation of (national and/or international) workshops for different actors

22

� Offer of a possibility of individual actors to take part in Peer Reviews as observers

for learning purposes on condition of VET providers’ explicit approval and strict in-

dependence of the observers

� Information and training activities can be combined for different actors, esp. VET

providers and Peers

� Encouragement of dissemination by VET providers (e.g. conferences, personal

contacts…)

� Use of existing channels, media and events.

4.3.2. Support

“Support” denotes the practical operative support given to VET providers who want to

conduct a European Peer Review. Information and dissemination clearly also falls

within this category but has already been dealt with above.

The following tasks and responsibilities have been identified which may be performed

on the national and/or the transnational level depending on the concrete distribution of

functions:

� Preliminary: Define quality criteria for Peer Training and for institutions/trainers of-

fering Peer Training

� Preliminary: Nomination of training institutions

� Preliminary: Training of trainers

� Preliminary: Define recruitment and selection criteria and procedures of VET pro-

viders and Peers (by using know-how and good practice from European Peer Re-

view projects).

� Preliminary: Establish a procedure to allocate peers from the Peer Register to dif-

ferent Peer Reviews

Support for VET providers

� Recruitment of VET providers: soliciting applications of VET providers (cf. Tool-box

of the European Peer Review Manual)

� Processing of VET provider applications: collect information, include in

list/database

23

� Selection of VET providers (if available “places” do not match the number of appli-

cants), establishment of a set of criteria upon which this selection is based

� Forward Information on selected VET providers to other relevant actors/institutions

� Counselling of VET providers, perhaps also provision of training for VET providers if

needed (support in self-evaluation, in preparation and conduct of Peer Review)

� Facilitating exchange between VET providers and Peers and among VET providers

(e.g. moderated electronic platform, workshops, conferences)

� Supply of suitable forms, handouts and checklists to support VET providers (cf.

Tool-box of the European Peer Review Manual)

� Sign contracts or Memorandum of Understanding with VET providers

� If applicable: Reimbursement/funding for VET providers

Support for Peers

� Recruitment of Peers: soliciting applications of Peers

� Processing of Peer applications, establishment of a national/regional etc. list of

Peers

� Forward information on selected Peers to other relevant actors/institutions

� Integration of information on Peers in a common European Peer Register

� Provision of Peer Training

� Facilitating exchange between VET providers and Peers and among Peers (e.g.

moderated electronic platform, workshops, conferences)

� Supply of suitable forms, handouts and checklists to Peers (cf. Tool-box of the

European Peer Review Manual)

� Contracts with Peers

� If applicable: Reimbursement/funding for VET providers

Matching of Peers and VET providers

� Matching of Peers and VET providers: establish a procedure to allocate peers from

the Peer Register to different Peer Reviews

24

4.3.3. Quality Assurance and Learning

Quality assurance and learning as horizontal themes must be embedded in all activities

and phases (cf. e.g. section V.3.2. Support).

In this section, activities of monitoring, assessment, reporting, evaluation and learning

are tackled (Phase 3 and 4 of the quality circle).

� Preliminary: Define how (much) reporting and monitoring should take place; define

format of reporting and monitoring

� Define criteria and indicators for quality assurance and learning

� Define levels of confidentiality for different actor

Documentation/monitoring/reporting

� Collect Self-Reports (cf. Tool-box of the European Peer Review Manual)

� Collect Peer Review Reports (cf. Tool-box of the European Peer Review Manual)

� Establish a feedback procedure, send out and collect feedback questionnaires from

VET providers and Peers

� Collect Meta-evaluation of Peers (cf. Tool-box of the European Peer Review Man-

ual)

� Collect and record information on Peers: data on involvement in Peer Reviews,

feedback from VET providers.

� Record and assess information on Peer Reviews (by case)

� Reporting / monitoring on Peer Reviews on overall progress

Evaluation and monitoring on the meta-level

� Commission and monitor meta-evaluation on process and impact of European Peer

Review (including cooperation structures and processes)

� External monitoring of the coordinating body’s operation

Learning

� Preliminary: Define structures and procedures for continuous improvement and

learning concerning the conduct of European Peer Reviews

25

� Implement procedures for learning through e.g. exchange of good practices and

networking, conferences, workshops etc.

4.4. Funding

Funding will be on the basis of a shared-cost-principle and thus come from different

sources:

The contribution of own funds of VET providers – as the primary beneficiaries – seems

appropriate. Support from national funds (e.g. for costs which cannot be borne by the

VET providers themselves) is to be defined by the Member States.

Funding from the European level should be dedicated to the transnational aspects of

Peer Review, i.e. (co-)funding the costs of transnational Peers and the costs of Euro-

pean training (if offered), co-funding of exchange and networking of VET providers and

Peers on the European level, co-funding of coordination between national lead institu-

tions and the European coordinating body.

European funding could come from different sources:

� Some costs of Peer Review activities as part of ENQA-VET (res. ENQA-VET suc-

cessor) activities could be funded within the future budget of this network. This in-

cludes in particular funding for the coordinating body, if applicable, and coordination

of national lead bodies (e.g. QANRPs).

� The lifelong learning programme could be tapped for certain activities: e.g. partner-

ship projects between VET providers (LdV, Comenius); mobility (Peer Reviews)

and further training (Peer Training) of teachers/trainers (LdV, Comenius), confer-

ences, workshops, dissemination and valorisation etc.

Ad lifelong learning programme: Earmarking of funding for European Peer Review or

joint proposals should increase efficacy and efficiency on the operational level. The

relevant bodies responsible should be involved in the planning and budgeting of future

European Peer Reviews.

26

5. OUTLOOK: EUROPEAN PEER REVIEW 2010-2013

5.1. Participation of Member States and current forecast

In a survey of ENQA-VET Member States (Jan.-Feb. 2009), so far 18 Member States

have voiced their interest in participating in further Peer Review activities on the Euro-

pean level, 14 of which have also given a first forecast on the number of transnational

European Peer Reviews for the four-year-period between 2010 and 2014, which add

up to almost 250 Peer Reviews within this period (as of March 20, 2009).

Table 1: Forecast for transnational European Peer Reviews 2010-2013

Year 2010 2011 2012 2013 Total

No. of Peer Re-views

43 56 70 78 247

Source: Gutknecht-Gmeiner 2009 (figures as of March 20, 2009)

Member states who have no concrete plans to conduct European Peer Reviews state

that they want to be included in the activities with a view to full participation in the fu-

ture.

Thirteen Member States plan to use the QANRP as liaison and coordinating institution

for European Peer Reviews on the national level. The remaining states have either not

established a QANRP yet or have so far denoted another institution as responsible

institution for coordinating European Peer Reviews.

Interested Member States are: Austria, Bulgaria, Czech Republic, Denmark, Estonia,

Finland, Italy, Hungary, Lithuania, Luxembourg, Malta, the Netherlands, Norway, Ro-

mania, Slovenia, Spain, Sweden, and United Kingdom.

5.2. Scenarios

Different scenarios for the distribution of tasks and responsibilities with different funding

structures are possible. Cooperation structures and procedures will have to be adapted

once the overall scenario has been decided upon.

5.2.1. Landmarks

The following landmarks provide some orientation:

Coordinating Body: In order to match its function of safeguarding the common Euro-

pean aspects of transnational Peer Reviews, the coordinating body at European level

(scenarios 2 and 3) will have to be commissioned by a suitable institution on the Euro-

27

pean level – most likely ENQA-VET (res. its successor) or the European Commission

in cooperation with the Member States to ensure legitimacy of the coordinating body.

Clear terms of reference for this assignment should guide the selection process. Fund-

ing for this coordinating body will probably involve some community funding (possibly

also from existing programmes) and own funding.

VET providers: If European Peer Review is to be implemented as a voluntary external

evaluation which supports quality improvement at VET provider level, a considerable

part of the decision-making will lie with the VET providers: Primary ownership of the

Peer Review process remains with the VET providers who engage in transnational

European Peer Reviews. Apart from voluntary participation, VET providers also have a

say on the topics to be evaluated as well as on the selection of the national and trans-

national Peer(s).

Concerning the dissemination, coordination, operative support and monitoring of Peer

Reviews, three different scenarios have been elaborated by the Thematic Group on

Peer Review.

28

6. SCENARIOS FOR TRANSNATIONAL EUROPEAN PEER REVIEWS

6.1. Scenario 1: Grass-roots model

6.1.1. Characteristic features

Scenario 1 is a “grass-roots model” in which no co-ordination and support for Peer Re-

views is provided at the European level. It will be entirely up to the individual VET pro-

viders to build up their own networking and coordination structures and to acquire fund-

ing for transnational Peer Reviews. Conduct of transnational Peer Reviews will be ad

hoc; no systematic sharing of experiences and joint learning will be possible.

If the implementation of such a model of transnational Peer Reviews is successful, un-

related, individual networks with no or little interconnection will be established (the sce-

nario was therefore also dubbed „chaotic“ by the members of the Thematic Group on

Peer Review). Lack of consistent coordination also means that learning on the transna-

tional level (if it exists) will be piecemeal and dissemination and transfer of innovation

not possible on a larger scale curtailing a possible contribution of transnational Peer

Reviews to European policy development and implementation, especially with regard to

the EQARF.

6.1.2. Role of key players

It is evident that the initiative of VET providers lies at the heart of this scenario. There

are at least three bottlenecks in this scenario: the dissemination of information to VET

providers which is not taken care of in this scenario, coordination and support for VET

providers (e.g. in finding suitable Peer Review partners from other countries), and fund-

ing. VET providers who cannot afford to pay for transnational activities (e.g. in coun-

tries with little financial autonomy for VET providers) will thus be at a disadvantage.

In countries implementing Peer Review VET providers might be assisted in the above

tasks by national bodies like the QANRPs or other suitable institutions. The Quality

Assurance National Reference Points (QANRPs) might take up co-ordination and sup-

port functions at the national level and give individual support for transnational activities

– possibly also through bilateral/multilateral agreements with similar bodies from other

countries.

Yet, if countries consider some kind of bilateral or even multilateral cooperation in order

to promote transnational Peer Reviews, these activities are not linked to a broader

European strategy and/or network. Likewise, the inclusion of the social partners into

29

transnational Peer Review activities will be at the discretion of individual institutions

and countries and not an integral part of transnational Peer Review activities.

In countries in which Peer Review is not supported by authorities, transnational Peer

Reviews will be individual, happenstance and often one-off activities of VET providers

with no overall mechanisms for monitoring, exchange and learning between the indi-

vidual institutions since information flows are not coordinated. No quality assurance of

Peer Reviews is provided.

6.1.3. Funding procedures/sources

Funding procedures and sources in this scenario will vary widely, by and large VET

providers will be called for to find funding, for the transnational part the Lifelong Learn-

ing Programme could perhaps be tapped (e.g. partnership projects), yet individual ap-

plications will have to be organised and submitted by different Peer Review networks.

In some countries which attribute high importance to transnationality of Peer Reviews,

national co-funding might also be available.

No funding provisions need to be made on the European level.

6.1.4. Contribution to the achievement of the goals, descriptors and indicators of the

EQARF

On the whole Peer Review shows a very high correlation with the principles and provi-

sions of the EQARF (cf. above chapter IV.1 and Table 9 in the Annex).

In this section, the potential of the specific scenario of transnational Peer Review im-

plementation to contribute to the implementation and further development of the

EQARF is assessed with a view to the particularities of the scenario. The table below

should be read complementarily to the analysis of the correlation between Peer Review

and EQARF provided in Table 9 in the Annex.

Table 2 : Scenario 1: Contribution to EQARF

Contribution to EQARF Inconsistency or contradiction to EQARF

Contribution to continuous quality improve-ment (EQARF Recommendation passim)

Role of QANRPs not clear, QANRPs poten-tially not included in coordination of Peer Re-view activities (EQARF Recommendation, p. 8ff., 4; Annex 1, p. 1)

30

Grass-roots implementation is in compliance with a voluntary approach (EQARF Recom-mendation passim)

Systematic and coherent involvement of VET stakeholders and esp. social partners not pro-vided for. (EQARF Recommendation, Annex 1 – relevant for all phases of the quality cycle)

Focused on and consistent with national/local training needs (EQARF Recommendation, Annex 1, p. 1)

No sustainable impact on the system level (national and European)

VET system can benefit from the VET provid-ers experience depending on national ap-proaches to quality in VET. (EQARF Recom-mendation, Annex 1, p.1).

Inconsistencies also exist with the arguments set forth in the Explanatory Memorandum (COM (2008) 179 final):

Individual and uncoordinated activities contradict the provision that transparency and consis-tency of policy and practical developments “cannot be achieved by the Member States individu-ally”. (p.8)

Possible bilateral/multilateral agreements between Member States threaten transparency, cf. p.8: “bilateral agreements … would result in an extremely complex and non-transparent overall structure at the European level”.

6.1.5. SWOT analysis: advantages and disadvantages of scenario 1 from a European perspective

Table 3 : SWOT analysis of scenario 1: Grass-roots model

Advantages (Strengths) Disadvantages (Weaknesses)

Grass-roots implementation is in compliance with a voluntary approach

No overall dissemination and no consistent and reliable support for VET providers

Lighter and faster because it would require less bureaucracy;

Necessary initiative by VET providers ensures “ownership” of Peer Review by VET providers

No common agreements (or at best bilateral or multilateral agreements) between Member States lead to intransparency.

No consistent quality assurance leads to in-consistencies, therefore also no direct compa-rability of different initiatives

Focused on and consistent with national/local training needs.

No European yield of ad hoc transnational European activities.

Little direct contribution to the EQARF and other European policies, in contradiction to some EQARF provisions.

No funding on European level must be pro-vided for.

No funding (or only temporary piecemeal fund-ing from LLP programme) available for VET providers.

31


No overall coordination and cooperation costs and efforts on European level.

Possible exclusion of VET providers from countries with little financial autonomy of VET providers and no national coordination struc-ture for Peer Review

Economic inefficiency of parallel efforts to organise transnational Peer Reviews, duplica-tion of administration

Finding suitable network partners and Peers is not supported.

No monitoring and learning on European level

Absence of a unified research base forecloses evidence-based policy-making

Ad Threats

One potential threat may lie in the possibility that, in the absence of a European coor-

dination, other international organisations and/or networks might step in and “reap the

harvest” from projects and initiatives which have hitherto been supported by European

funds and which have from the start been designed for implementation not only on the

national level of participating countries but also on the European level.

Opportunities Threats

Some grass-roots approach should be inher-ent in all scenarios since the VET providers and their improvement needs are at the centre of Peer Review. VET provider initiatives should be supported in all scenarios.

Further implementation of transnational Peer Review is in jeopardy if only a grass-roots approach is used – transnational Peer Review simply might not continue on a relevant scale.

Additionally, even a bottom-up implementation can bring a benefit for the VET system as such: VET system can benefit from the VET providers experience depending on national approaches to quality in VET.

Without any organised support and coordina-tion, the grass-roots scenario highly depends on the individual competences and capacities in VET providers to engage in transnational Peer Review.

If a national body for Peer Review exists, the quality of transnational Peer Reviews is also dependent upon the national competences and mandate of the selected coordinator (and the policy context in the given country).

32

6.1.6. Conclusions

The grass-roots scenario has some considerable advantages – mainly that no coordi-

nation efforts are necessary. However, this approach to implementation does not allow

exploiting the potential of transnational Peer Reviews in terms of contributing to the

implementation of the EQARF and other related policies.

Bilateral agreements account for inconsistencies in approach, lack of funding and sup-

port can thwart any further implementation. No overall evidence base on transnational

Peer Review will be collected foreclosing mutual learning and further research and de-

velopment.

6.2. Scenario 2: Decentralised coordination model


Scenario 2 is a “decentralised co-ordination model” in which a coordinating body on the

European level provides a basic framework for transnational European Peer Reviews.

This body coordinates the national lead bodies, gives technical support and ensures

that common procedures (including quality criteria for Peer Reviews) are followed in all

participating countries. It is also responsible for setting up appropriate monitoring and

evaluation schemes on the European level.

There exists a common agreement between the participating Member States and the

coordinating body as to the specific structures, procedures and quality criteria govern-

ing the conduct of transnational Peer Reviews. The European coordinating body does

not, however, give expert support for the preparation and conduct of Peer Reviews or

organise trainings or other supportive events. It may provide technical support for dis-

semination and networking (e.g. administrative support for conferences etc.).

The main responsibility for dissemination, operative support and monitoring lies with

the national bodies.


In this scenario, the QANRPs may play a decisive role as national lead bodies manag-

ing and coordinating Peer Reviews. They would be responsible for almost all tasks and

functions described above: information, dissemination, communication, support in im-

plementation, exploitation of results, and management of funding. Some coordination

33

activities will be facilitated by the coordinating body (e.g. organisation of meetings);

however, the coordination concerning practical political and operative issues must be

dealt with by the QANRPs who receive no expert support from the European level.

The QANRPs will also be responsible for the integration of social partners and other

important VET stakeholders in the implementation, revision and further development of

transnational European Peer Reviews.

VET provider participation will be channelled through the QANRPs. Participation of

VET providers from countries with no national coordination body is not foreseen. It is

not clear whether and how initiatives of VET providers outside the common structure

can be integrated.


Funding will come mainly from the national level – be it own funds of VET providers or

national co-financing.

There may be some common funding schemes elaborated by the national lead bodies

with the support of the coordinating body e.g. in the area of reimbursement of travel

costs for Peer Reviews or conference attendance.

6.2.4. Contribution to the achievement of the goals, descriptors and indicators of the

EQARF




The collation of a comprehensive evidence-base is not ensured in this scenario.

A voluntary approach is ensured through com-mon agreements and the commitment of the national lead bodies (EQARF Recommenda-tion passim)

Full-cycle learning (including phase 4 “review”) is not ensured in this scenario.

Coordination among the national lead bodies ensures that transnational Peer Reviews re-main focused on and consistent with na-tional/local training needs (EQARF Recom-mendation, Annex 1, p. 1)

Full impact on European and Member States level is not ensured in this scenario.

With a coordinated approach VET system will more fully benefit from the VET providers ex-perience, transfer of innovation between coun-tries is possible (EQARF Recommendation, Annex 1, p.1).

Exploitation of Peer Review’s full contribution to the implementation of the EQARF is not ensured in this scenario.

34


Involvement of VET stakeholders and esp. social partners is provided for on Member States level through the national lead bodies. (EQARF Recommendation, Annex 1, p. 1)

Bottom-up mutual learning and cooperation between Member States facilitated.


Table 5 : SWOT analysis of scenario 2: Decentralised coordination model


A coordinated approach ensures that transna-tional Peer Reviews will remain linked to and contribute to policy developments on the European level, in particular the EQARF im-plementation.

Decentralised coordinated implementation is in compliance with a voluntary approach which is ensured through common agreements.

National bodies are aware of the needs of the VET providers and are able to provide tailored support on site. They can also link Peer Re-view activities with other QA-activities on a national level.

High dependence on the commitment and capacity of the national lead bodies.

Coordination among the national lead bodies ensures that transnational Peer Reviews re-main focused on and consistent with na-tional/local training needs.

Some compromises between participating countries might seem necessary.

Common agreements between Member States ensure transparency and comparability, facilitating European cooperation on all levels.

Integration of VET provider initiatives and, in general, of VET providers from countries with no national lead body is not provided for.

Consistent quality standards and procedures are part of the common agreements. They are monitored by the European coordinating body.

No quality assured standardised training.

Little bureaucratic and administrative load on the European level.

Some bureaucratic burden possible, especially on the national level.

Funding is taken care of primarily on the na-tional level.

No funding for VET providers from non-participating countries. No possibility to influ-ence funding in “hard times” (possible choice for national Peer Review).

No extra funding on European level must be provided for the transnational Peer Review and the QANRPs.

Some costs are incurred for the coordinating body and the coordination activities between the QANRPs (however: high synergies with other QANRP-activities expected)

35


Economic efficiency through coordination and well-managed interfaces between the different actors.

No expert support provided by coordinating body, puts a considerable work-load on QANRPs.

Finding suitable network partners and Peers is not supported on the European level and will need extensive coordination between the na-tional lead bodies.

Some monitoring on the European level is supported by the coordinating body, the main responsibility lies with QANRPs.

Responsibility for promotion of the European perspective not clear.

Collation of a unified research base seems possible within the monitoring carried out by the European coordinating body.

Some data not in the hands of the coordinating body (e.g. Peer Databases), collation of a comprehensive database therefore will require extra efforts.


The decentralised coordinated approach relies heavily on existing structures; no special ex-pertise of the coordinating body is needed. There is little investment in structures at the beginning.

Yet, with no expert support on the European level, coordination between the national lead bodies can become burdensome. National lead bodies might also be overwhelmed by the tasks and functions to be performed on the national level in order to support transnational activities.

With no expert body on the European level, the European dimension could become less prominent over time since national lead bodies represent primarily their own systems.

Grass-roots initiatives should be possible within all scenarios of transnational Peer Re-views. If this is not provided for, some of the most motivated and eager may not be able to participate if their national structures do not give them support.

6.2.6. Conclusions

The decentralised coordinated scenario will be rather easy to implement based on

common agreements which can be taken from the proposal at hand. Without enough

expertise provided, however, problems might arise during implementation - concerning

e.g. the scope of the tasks of the national lead bodies (which include expert support in

transnational Peer Reviews) and considerable coordination requirements between the

national bodies. The whole approach hinges primarily on the commitment and capaci-

ties of the national lead bodies. Additionally, no provision is made to encompass VET

providers from countries with no national lead bodies or VET providers who have

started their own Peer Review network and want to integrate it into a larger framework.

36

6.3. Scenario 3: Transnational cooperative model


Scenario 3 is a “transnational cooperative model” in which the coordinating body at

European level plays a more comprehensive role. While operative tasks and responsi-

bilities by and large remain with the national bodies – who clearly are closer to the VET

providers and can provide support on-site much easier – there are also offers at the

European level, as for instance European Peer Training or workshops for VET provid-

ers. A wider range of support activities (e.g. training, workshops, networking events

etc.) on the European level highlights the European dimension for all actors involved,

especially for the primary target group – VET providers and Peers. The coordinating

body should also be involved in recruiting, thus opening an additional avenue for par-

ticipation for VET providers and Peers from countries with no national bodies responsi-

ble for Peer Review. Extensive monitoring (including collaborative self-evaluation of

coordinating structures and processes) is provided by the coordinating body who may

also commission a central external evaluation of transnational Peer Review implemen-

tation.


The coordinating body in this scenario can be described thus:

The coordinating body

� Is managed by a steering group of experts which is composed of representa-tives from Member States (management board)

Structure

� Takes responsibility for periodic dissemination of outcomes Information/

Dissemination

� Supports and coordinates (if necessary) information and dissemination in par-ticipating countries

Information/ Dissemination

� Establishes and maintains a website related to developments in peer review Information/

Dissemination

� Acts as source of contact for establishing transnational peer groups Support

� Provides for consistent methodology and support evaluation activity Support

QA and learn-ing

� Manages, administrates and maintains a database of transnational peers Support

QA and learn-ing

� Provides peer review training at transnational level through face to face and electronic methods

Support

� Provides a centralised source for research activity QA and learn-

ing

37

� Supports developments, including training, at national/transnational level QA and learn-

ing

� Is a central source for responding to issues related to funding Funding

The main responsibility for dissemination and operative support will still lie with the na-

tional bodies who will receive expert support from the European level, though. Again,

the QANRPs may play a decisive role as national lead bodies managing and coordinat-

ing Peer Reviews on the national level (recruitment of VET providers, Peers, informa-

tion flows to transnational level). Coordination between QANRPs will be facilitated by

the coordinating body that will actively promote the European dimension and can act as

a referee or moderator in moot points. The coordinating body – with the support of the

QANRPs will thus be responsible for all activities on the European level. A monitoring

database of Peer Reviews and Peers will be maintained by the coordinating body and

made available to the QANRPs. This database will also serve for further research and

evaluation.

The integration of social partners and other important VET stakeholders in the imple-

mentation, revision and further development of transnational European Peer Reviews

will be managed by the QANRPs and – on international level – by the coordinating

body.

VET provider participation will be channelled through the QANRPs. Participation of

VET providers from countries with no national coordination body is foreseen. Initiatives

of VET providers outside the common structure can be integrated through the coordi-

nating body at European level.


There will be split funding: For the national parts of the transnational Peer Reviews (i.e.

preparation of Peer Reviews by VET providers and national bodies, recruitment and

remuneration of national Peers, administrative support by national bodies), own funds

of VET providers and/or national co-financing will be used.

For transnational activities (training, travel cost of transnational Peers, networking and

dissemination activities as well as the administrative support by the coordinating body)

common funding schemes (including possible use of the LLL (Leonardo da Vinci) Pro-

gramme) will be elaborated which allow for consistent and sustainable implementation

of transnational Peer Reviews.

38

6.3.4. Contribution to the achievement of the goals, descriptors and indicators of the EQARF




none

A voluntary approach is ensured through com-mon agreements and the commitment of the national lead bodies (EQARF Recommenda-tion passim)

Coordination among the national lead bodies ensures that transnational Peer Reviews re-main focused on and consistent with na-tional/local training needs (EQARF Recom-mendation, Annex 1, p. 1)

QANRPs play a major role as national lead bodies for Peer Review leading to synergies and a coherent implementation of Peer Re-view and other activities within the EQARF.

With a coordinated approach VET system will more fully benefit from the VET providers’ experience, transfer of innovation between countries is supported (EQARF Recommenda-tion, Annex 1, p.1).

Involvement of VET stakeholders and esp. social partners is provided for through the national lead bodies on the Member States’ level and additional coordination on the Euro-pean level. (EQARF Recommendation, Annex 1, p. 1)

Bottom-up mutual learning and cooperation between Member States is facilitated.

The collation of a comprehensive evidence-base is ensured in this scenario providing a basis for full-cycle learning (including phase 4 “review”).

Full impact on European and Member States level is ensured in this scenario.

Exploitation of Peer Review’s full contribution to the implementation of the EQARF is en-sured in this scenario.


Table 7 : SWOT analysis of scenario 3: Transnational cooperative model

39


Provides for consistent, efficient peer review methodology across all Member States, in-cluding training

Possible disjunction with policies and devel-opments at national level

Encourages transparency of processes and systems

Requirement for consistent levels of commit-ment for all Member States

Provides for consistency of quality assurance Potential to be bureaucratic

High benefits of a clear and easily approach-able source of information, support and access to peer contacts on the European level.

Coordination and cooperation efforts to im-plement structure and process on European level

Provides structure for collaborative reportage of development of impact and outcomes

Costs of maintaining coordination body and structure

Single evidence-base

Provides for effective access and support for VET providers/countries new to peer review

Provides for consistent mechanism to meet objectives and targets linked to European educational policy

Single or at least coordinated funding base

Provides for consolidation of experience, re-search opportunities and new initiatives

Delimits competition and encourages transeu-ropean collaboration and networking

Enables comprehensive and systematic evaluation strategy

Single administration base provides economic efficiency


Through a transnational cooperative approach Transnational Peer Reviews have the potential to become one of the most important pillars of VET quality improvement on the European level in the future providing common ap-proaches and cross-fertilisation between dif-ferent countries and the European level.

No funding available for coordinating body and its activities

Not all member states are equally committed to cooperate with the European coordinating body which could result in different degrees of access to support for VET providers.

Difficulties in identifying suitable institutions with the necessary expertise and in ensuring continuity.

6.3.6. Conclusions

Because of its consistency, sustainability, and its potential to contribute to European

integration in the area of VET (transparency, cooperation, and mutual trust) this is the

scenario favoured by the Thematic Group on Peer Review (cf. below).

40

7. RECOMMENDATIONS and CONLUSIONS

7.1 Recommendation by the thematic group on peer review

The Thematic Group unanimously recommends the implementation of scenario 3 due

to

� its efficacy, (economic) efficiency and the synergies it creates (which are especially

important in times of economic crisis), and

� the fact that in this scenario the full potential of Peer Review in terms of a sustain-

able contribution to the implementation of the EQARF and other related European

policies can be exploited.

Additionally, scenario 3 will also ensure smooth cooperation and fruitful networking

between the different actors involved, high-quality implementation of transnational Peer

Reviews and evidence-based evaluation and learning.

If implemented properly, European Peer Review as a practical, tangible and concrete

tool to implement the EQARF recommendation has the potential to become one of the

main pillars of European VET quality policy in the next 5 to 10 years encompassing all

relevant actors – VET providers, social partners and other stakeholders as well as the

VET systems of the participating countries.

Table 8: Overview: Comparison of the three scenarios

Scenario 1

Grass-roots model

Scenario 2

Decentralised coor-dination model

Scenario 3

Transnational co-operative model

Contribution to EQARF and other related European policies

low medium high

Contribution to Euro-pean networkingamong VET providers and national VET systems

low medium high

Quality assurance and learning on Euro-pean level

none medium high

Integration of QANRPs in European Peer Review structure

no (structure does not exist)

yes yes

Integration of social partners in European Peer Review structure

no (structure does not exist)

yes yes

41

Scenario 1

Grass-roots model

Scenario 2

Decentralised coor-dination model

Scenario 3

Transnational co-operative model

Overall efficacy low medium high

Economic efficiency low medium high

Coordination and implementation effortson European level

none some some

Funding needs on European level

none basic* basic + *

* To be defined in detail in a subsequent planning phase; use of LLL Programme (LdV) funding.

7.2 Conclusions: further steps

The following recommended further steps have been identified by the Thematic Group

on Peer Review:

� Choose one scenario

� Elaborate a European project plan for the implementation of the chosen sce-nario including timelines, allocation of responsibilities and tasks and funding

� Possibly buttress the European project plan with a feasibility study which also takes into account existing experiences in national implementation of Peer Re-view

� Disseminate Peer Review on European level and attract member states and/or individual VET providers to join transnational European Peer Review activities.

42

ANNEX 1: Peer Review and the EQARF – a detailed analysis

NB: The majority of the indicative descriptors in Annex 1 of the EQARF Recommenda-tion are also covered in the framework of Quality Areas developed for European Peer Reviews.

Table 9: European Peer Review: Contribution to and consistency with the EQARF Recommendation

Contribution to and consistency with EQARF

Reference

Contribution to systematic and continuous quality improvement at the VET provider level with impor-tant contributions to overall improvement of VET systems.

EQARF*: passim.

European Peer Review is geared towards maxi-mum correspondence with the CQAF (inception phase of Peer Review starting 2003) and currently the EQARF.

It has been developed, tested and revised through intensive European cooperation within the last 5 years and is now ready for large-scale implementa-tion.

A plethora of support and guidance material has been developed to facilitate the conduct of high-quality Peer Reviews on national and transnational level.

CQAF and quality cycle:

EQARF: passim, see especially Annex 1)

EQARF, pp. 8 (3)

Guidelines and quality standards

EQARF Annex 1, p.2 (guidelines on VET sys-tem level)

See also Explanatory Memorandum, p.6 where the European Peer Review is mentioned and p.9 (guidance and support material)

Peer Review provides a new and promising qualita-tive methodology for external evaluation which can be implemented in all existing quality systems and contributes to their further development.

External evaluation

EQARF: p. 4 (9) ; 5 (11), Annex p.3

Qualitative methods

EQARF: 5 (11)

Adequacy for the sector

EQARF Annex p.3 (System level)

See also Explanatory Memorandum, p. 4

The European Peer Review procedure itself follows the systematic quality cycle. It builds upon self-assessment at VET provider level thus combining internal and external evaluation.

CQAF and quality cycle

EQARF: passim, see especially Annex 1

Combination of internal and external evalua-tion

EQARF: esp. 5 (11), Annex p.3 (system level and VET providers)

Explanatory Memorandum, pp. 9; 10

A common European Peer Review procedure in-creases transparency of quality assurance and

Transparency

43


Reference

improvement systems and approaches in VET, to promote mutual trust and facilitate mobility.

EQARF: passim, esp. p. 4 (9), 6 (15)

Mutual trust and mobility

EQARF: passim, esp. p. 4 (9), 6 (15) Explicit and transparent quality assurance system EQARF Annex 1, p.1

See also Explanatory Memorandum, p. 4

Peer Review is an external evaluation in a network and builds upon cooperation between VET provid-ers. It aims at facilitating mutual learning and inno-vation transfer. Important VET stakeholders are included into the evaluation activities.

Peer Review is designed to foster a bottom-up commitment to quality improvement and a quality culture across institutional boundaries. A participa-tory approach is supported with clear responsibili-ties of management.

“Culture of quality improvement”

EQARF, pp. 4 (8), 7 (18), 8(1)

“Cooperation and mutual learning, partner-ships”

EQARF, p. 10 (1), 6 (15), 10 (10)

EQARF Annex 1, p.1, p.2

“Stakeholder and social partner inclusion”

EQARF, pp. 8 (2)

QARF Annex (VET system and VET provider) p. 1, p.2, p.3, p.4.

Staff participation; allocation of responsibility

QARF Annex, p. 1;

Exchange of good practices/innovation transfer

EQARF, p. 5 (13)

Explanatory Memorandum p. 4

European Peer Review is an adequate and attrac-tive external evaluation tool for phase 3 of the qual-ity assurance and improvement cycle: evaluation and assessment.

As an open and “friendly” methodology which is based upon professional assessment by col-leagues it focuses on the improvement needs of the VET provider and provides directly understand-able and acceptable feedback. This in turn en-hances the commitment to improvement (phase 4 of the quality cycle).

CQAF and quality cycle

EQARF: passim, see especially Annex 1

Evaluation and review

EQARF Annex pp. 3&4

Explanatory Memorandum, pp. 9; 10

European Peer Review is in compliance with the voluntary approach – both on VET provider and system level (EQARF, Explanatory Memorandum, p. 4)

EQARF p.11

Explanatory Memorandum, p. 4

Peer Review respects the principle of proportional-ity and builds upon existing systems – therefore no replacement of existing systems or creation of new systems is necessary.

EQARF p.11

Explanatory Memorandum, p. 8

A set of European Quality Areas for VET providers have been developed and tested for use in Euro-pean Peer Reviews. The Quality Areas cover all relevant aspects of VET provision and are further specified by criteria and exemplary indicators.

EQARF, Annex 1

44


Reference

Peer Review is a concrete proposal and initiative developed under the Leonardo da Vinci programme which is ready for implementation across Europe.

EQARF, p. 10, 2 “Promote and participate together with the Member States in the Framework network, contributing to policy development in this area through concrete proposals and initiatives, as appropriate.”

* EQARF: Recommendation of the European Parliament and the Council on the establishment of a Euro-pean Quality Assurance Reference Framework for Vocational Education and Training, Brussels, 2.4.2009, PE-CONS 3746/08.

** Memorandum: Explanatory Memorandum, In: Proposal for a Recommendation of the European Parlia-ment and of the Council on the establishment of a European Quality Assurance Reference Framework for Vocational Education and Training, Brussels 9.4.2008, COM (2008) 179 final.

45

ANNEX 2: Bibliography

European policy documents

- Recommendation of the European Parliament and of the Council of 18 June 2009 on the establishment of a European Quality Assurance Reference Framework for Voca-tional Education and Training (2009/C 155/01).

- Proposal for a Recommendation of the European Parliament and of the Council on the establishment of a European Quality Assurance Reference Framework for Voca-tional Education and Training, Brussels 9.4.2008, COM (2008) 179 final.

- Council Conclusions on Quality Assurance in Vocational Education and Training, 18 May 2004.

- Technical Working Group ‘Quality in VET’ (2004): Fundamentals of a ‘Common Qual-ity Assurance Framework’ (CQAF) for VET in Europe. European Commission, Direc-torate-General for Education and Culture, Vocational training: Development of voca-tional training policy.

- European Network on Quality Assurance in VET: Work Programme 2006-2007.

- European Network on Quality Assurance in VET: Work Programme 2008-2009.

- Declaration of the European Ministers of Vocational Education and Training, and the European Commission, convened in Copenhagen on 29 and 30 November 2002, on enhanced European cooperation in vocational education and training.

- Council Resolution of the European Ministers of Vocational Education and Training on enhanced European cooperation in vocational education and training of Dec. 19, 2002 (OJ 2003/C 13/02, 18.1.2003).

- Maastricht Communiqué on the Future Priorities of Enhanced European Cooperation in Vocational Education and Training (VET) (2004) (Review of the Copenhagen Decla-ration of 30 November 2002), December 14, 2004

- Helsinki Communiqué on Enhanced European Cooperation in Vocational Education and Training (2006): Communiqué of the European Ministers of Vocational Education and Training, the European Social partners and the European Commission, convened in Helsinki on 5 December 2006 to review the priorities and strategies of the Copenha-gen process.

- Bordeaux Communiqué on enhanced European cooperation in vocational education and training. Communiqué of the European Ministers for vocational education and training, the European social partners and the European Commission, meeting in Bor-deaux on 26 November 2008 to review the priorities and strategies of the Copenhagen process.

- Conclusions of the Conference „Quality Assurance in Higher Education and Voca-tional Education and Training”, 11/12 May, University of Graz.

- Proposal for a recommendation of the European Parliament and of the Council on the establishment of the European Qualifications Framework for lifelong learning, COM (2006) 479 final, Brussels, 5.9.2006

- Commission staff working document “European Credit System for Vocational Educa-tion and Training; a system for the transfer, accumulation and recognition of learning outcomes in Europe”; SEC (2006) 1431, 31.10.2006.

- ETF Yearbook 2007: “Quality in Vocational Education and Training: Modern Voca-tional Training Policies and learning processes”, ETF, Turin, 2007.

46

Documents on Peer Review

- Peer Review Website: www.peer-review-education.net

- Gutknecht-Gmeiner, Maria; Lassnigg, Lorenz; Stöger, Eduard; de Ridder, Willem; Strahm, Peter; Strahm, Elisabeth; Koski, Leena; Stalker, Bill; Hollstein, Rick; Allulli, Giorgio; Kristensen, Ole Bech (2007): European Peer Review Manual for initial VET. Vienna, June 2007.

Available in English, Catalan, Czech, Danish, Dutch, Finnish, German, Hungarian, Italian, Portuguese, Romanian, Slovenian, Spanish, and Turkish from: www.peer-review-education.net/TCgi/TCgi.cgi?Target=home&P_KatSub=6

- Gutknecht-Gmeiner, Maria (2007a): Peer Review and the CQAF. Peer Review as an innovative methodology for external evaluation in VET and its contribution to the fur-ther development of the „Common Quality Assurance Framework“(CQAF). Contribu-tions by Allulli, Giorgio; Koski, Leena; Väyrynen, Pirjo; Molnar-Stadler, Katalin; Camps, Josep; Canyadell, Pere; Vienna.

- Gutknecht-Gmeiner, Maria (ed.) (2007b): Implementing Peer Review as part of the CQAF. Scenarios for Peer Review Implementation in Austria, Spain (Catalonia), Finland, Hungary, and Italy. Contributions by Koski, Leena; Molnar-Stadler, Katalin; Allulli, Giorgio; Tramontano, Ismene; Camps, Josep; Canyadell, Pere; Vienna.

Further publications/reports/articles

- Allulli, Giorgio; Tramontano, Ismene (2005): Valutazione con i “Peer Review”, in: Journal “Il Sole 24ore Scuola”, Edizioni II Sole 24ore, Jan 26, 2005, anno VI No 2.

- Brinek, Gertrude (2007): Peers oder PISA – Schulqualität am TGM, In: der Technologe, Mitteilungen des Verbands der Technologen, 4/2007, 195. Folge, 27.

- Dousset-Ortner, Eva (Spring 2006): Peer Review als Instrument der Qualitätssicherung und Qualitätsentwicklung an der Tourismusschule Wien 21, in: PIB Newsletter, 14-15.

- Dousset-Ortner, Eva (2006/2007): Erster Peer Review an den Hertha-Firnberg-Schulen durchgeführt. Projekt. Die Schulen testeten das Peer-Review-Verfahren, ein Instrument der Qualitätssicherung, in der Pilotphase, in: Wissenplus, 3-06/07, 27.

- Dimas, Cristina (2005): Peer Review – Qualidade na formação. CECOA investe em novos instrumentos para garantir of futuro dos jovens, in: CECOA-Newsletter, April/June 2005.

- Giordani, Roberta (2005): A Vienna il "Don Milani" rappresenta l'Italia, in: "L'Adige" and "Trentino".

- Gutknecht-Gmeiner, Maria (2008a): Externe Evaluierung durch Peer Review. Qualitätssicherung und -entwicklung in der beruflichen Erstausbildung, Wiesbaden: VS Verlag (Dissertation, Universität Klagenfurt 2006).

- Gutknecht-Gmeiner, Maria (Hrsg.) (2008b): Peer Review in der Berufsbildung. Projekte und Erfahrungen – ein Reader, Vienna.

- Gutknecht-Gmeiner, Maria (2007): Peer Review als Verfahren zur externen Evaluierung von Schulen. Definition und Gütekriterien, In: Zeitschrift für Evaluation 1/2007, 31-60.

47

- Gutknecht-Gmeiner, Maria (2006): Evaluierung durch Peer Review fördert die Qualität der Ausbildung, In: Berufsbildung in Wissenschaft und Praxis (BWP) 6/2006, 11-15.

- Gutknecht-Gmeiner, Maria (2006): Peer-Review in der beruflichen Erstausbildung in Europa. In: Basel, Sven, Giebenhain, Dagmar und Rützel, Josef (Hg.): Peer-Evaluation an beruflichen Schulen - Impuls für dauerhafte Schulentwicklung durch Öffnung nach Außen, Paderborn, 117-139.

- Gutknecht-Gmeiner, Maria (2006): Peer-Review zur Qualitätsevaluierung im Schulbereich – ein europäisches Projekt des öibf, In: Schul-NEWS, Newsletter des Schulkompetenzzentrums der Kinderfreunde 4/2006, 8-10.

- Gutknecht-Gmeiner, Maria (2006): Peer Review in Initial Vocational Education and Training. Externe Evaluierung durch Kollegenschaft, In: Weg in die Wirtschaft 3/2006, 17-19.

- Kristensen, Ole Bech (2005): Peer Reviews i grundlæggende erhvervsuddannelse, in: Hæfteklammen, 26-27.

- Oliviero, Paola (2006): Dentro l’Europa. L’esperienza Peer Review nel CIOFS-FP, in: “Citta” CIOFS-FP magazine, December 2006, 16.

- Puchhammer-Neumayer, Victoria (2007): Gelebte Qualität – die Abteilung Wirtschaftsingenieurwesen aus europäischer Sicht, In: der Technologe, Mitteilungen des Verbands der Technologen, 4/2007, 195. Folge, 1-3.

- Zemanek, Jutta (2007): Peer Review in der Berufsbildung, in: Weg in die Wirtschaft, No. 2/2007, 13-14.

48

ANNEX 3: List of Participants

Title Name Country Organisation name

Mr Michaela Jonach Austria (present at 1st meeting)

ARQA-VET

Mr Gabriela Nimac Austria (present at 2nd and 3rd meet-ing)

ARQA-VET

Mr Stanislav Michek Czech Republic National Institute of Technical and Vocational Education

Ms Leena Koski Finland Chairperson

Finnish National Board of Education

Mr Thomas Reinholz Germany Staatliches Schulamt

Ms Katalin Molnar-Stadler Hungary National Institute of Vocational and Adult Educa-tion

Ms Grainne Cullen Ireland Further Education and Training Awards Council

Ms Maria Vittoria Marini Bet-tolo

Italy Ministero della Pubblica Istruzione

Ms Desiree Scicluna-Bugeja (present at 1st meeting)

Malta Malta Qualifications Council

Mr Willem de Ridder The Netherlands ROC Aventus

Ms Unni Teien Norway Norwegian Directorate for Education and Train-ing

Ms Dana Stroie Romania National Centre for Technical and Vocational Education and Training Development

Mr Shawn Mendes Sweden (present at 1st and 2nd meeting)

Swedish National Agency for Education

Ms Asta Modig Sweden (present at 1st meeting)

Swedish National Agency for Education

Ms Linda Wilson (via e-mail) UK LSIS _ Learning and Skills Improvement Ser-vices

Mr Christopher Lambert UK City College Norwich

Ms Margareta Nikolovska ETF European Training Foundation

Ms Maria Gutknecht-Gmeiner TG Expert

European Network for Quality Assurancein Vocational Education and Training

Contact DetailsFor further information please visit www.enqavet.eu or contact the ENQA-VET Secreteriat.

FETAC, East Point Plaza, East Point Business Park, Dublin 3, Ireland.

T: 00353 1 865 9546 F: 00353 1 865 0072E-mail: [email protected]: www.enqavet.eu

This project has been funded with support from the European Commission.