Joint degrees/Double degrees – how to assess?Presentation at Nordic Bologna Seminar on Quality Assurance of Learning OutcomesStockholm, 8 November 2010

Simon Holmen Reventlow ClemmensenDanish Evaluation Institute

Agenda

• Terminology• NOQA project on joint evaluations of joint master’s

programmes• TEAM II project on accreditation of joint programmes• Methodologies applied• Main conclusions• Challenges• Recommendations• Questions?

What is EVA?

• Established in 1999• Based on former experience with programme

evaluation of HE• Covers the entire Danish educational system• Independent institution under the Danish

Ministry of Education• Board with right of initiative• Own budget

• Staff of 90+

International cooperation

• European Association for Quality Assurance in Higher Education (ENQA)• Standards and Guidelines for Quality Assurance in the

European Higher Education Area (ESG)

• Nordic Quality Assurance Network in Higher Education (NOQA)• Yearly joint projects on quality assurance in the Nordic

countries

• European Consortium for Accreditation (ECA)• Promotes mutual recognition of accreditation and

quality assurance decisions

Terminology I – joint programme

• The European Consortium for Accreditation in higher education:

“A joint programme is a programme offered jointly by different higher education institutions irrespective of the degree (joint, multiple and double) awarded”

Terminology II – joint degree

• The Recommendation on the Recognition of Joint Degrees : • “A joint degree should, for the purposes of this

Recommendation, be understood as referring to a higher education qualification issued jointly by at least two or more higher education institutions or jointly by one or more higher education institutions and other awarding bodies, on the basis of a study programme developed and/or provided jointly by the higher education institutions, possibly also in cooperation with other institutions.

NOQA project: Joint Master’s Programmes – Joint Evaluations• 2007 initiative from Nordic Council of Ministers• 6 joint programmes funded out of 41 applicants• 19 institutions involved in all Nordic countries• NOQA tasked to propose one or more joint

methodologies for evaluation of the programmes• Project group with representatives from Finland,

Norway, Sweden and Denmark – Iceland represented in Steering Group

• Project concluded autumn 2009.

TEAM II: Accreditation of joint programmes• Organised by ECA, funded by European Commission• Project started in 2009 and has just been concluded• The main aims of TEAM II was to explore recognition of

qualifications from joint programmes and facilitate accreditation of these programmes.

• Ambition: It should only be necessary to apply for accreditation of joint programmes once

• 5 pilot accreditations covering programmes leading to bachelor’s, master’s and research master’s degrees

Methodologies I:NOQA• 2 pilot evaluations – NordMaG & Norpath• Self-evaluation, site visit with external experts and

report• Common criteria and procedures for both pilots• Extensive list of criteria developed on the basis of

criteria employed in all Nordic countries• Criteria for jointness developed by project group• Resulted in two programme-specific reports and one

project report• Outlined three models for quality assurance of joint

programmes

Methodologies II: Criteria for jointnessNOQA• Specific criteria for jointness developed for NOQA

project• Formal agreement• Common funding strategy• Mobility• Practical preparation• Sharing of information between students• Employability• Quality assurance

Methodologies III:TEAM II• 5 pilot accreditations of joint programmes• One accreditation agency in charge, at least one other

invited as observer• All pilots followed ESG standards with self-evaluation,

site visit, external experts and final report• In general, self-evaluation reports showed insufficient

self-evaluation – mostly descriptive documents • Found very different emphasis on learning outcomes in

general and specifically whether emphasis is on ’intended’ or ’achieved’ learning outcomes

• Resulted in a methodological report, a conference report and the extension of Qrossroads to 13 countries

Methodologies IV:TEAM II• Common principles – different methodologies were employed

for each pilot• procedure, criteria and experts mainly from one country• procedure and criteria from one country, but applied by

experts from elsewhere• procedure and criteria based on (shared) European

professional practice requirements • new procedure developed with (‘traditional’) criteria

based on a minimum set + the extras required for another country

• based on one country’s procedure but with experts nominated by all and with a new set of criteria specifically developed for joint programmes

• Conclusion: All were in principle feasible

Main conclusions I

• Common for both projects:• Joint programmes ought to be quality assured jointly• Joint quality assurance is, in principle, feasible• Still a long way to go before it is practically doable• Significant legal challenges to joint quality assurance

• Require legislative changes in most countries• Require far-reaching agreements and strong mutual trust

• Strong national traditions and approaches to education and quality assurance

• Necessary to distinguish between criteria that could, should and must be applied

Main conclusions II

• Specific to NOQA:• Nordic systems of quality assurance divergent, not

convergent• No systematic quality assurance of Nordic programmes

so far• Little knowledge of the extent of joint Nordic

programmes• Criteria for jointness strengthen joint quality assurance

• Specific to TEAM II:• Modular approach may be the best way forward• Many ways to meet accreditation requirements• ECA principles entirely workable for most

Challenges

• Legal issues – divergence of national regulations• Regulated professions• National educational traditions and systems• National principles and models for quality assurance• Recognition of degrees and evaluation outcomes• International experts using their own context rather

than the relevant framework as point of departure• Differences in style, content and legal status of

accreditation reports• How many site visits?

Recommendations

• Common for both projects:• Any joint quality assurance system must build on the ESG• Further develop mutual trust between quality assurance

agencies• Promote recognition of joint programmes as equal to national

programmes• Important to use diverse and multinational expert panels

• Specific to TEAM II:• Further develop mutual recognition agreements• Increase access to information about programmes in other

countries – e.g. Qrossroads• Consider establishing a coordinating agency for accreditation of

joint programmes

Models for joint quality assurance of joint programmes• NOQA:

• Evaluation and accreditation• Nordic Quality Label• Audit of the programme’s quality assurance procedures

• TEAM II:• Modular approach with compulsory core elements