1

ENQA Debriefing of expertsAgency reviews

Experiences of review team member

Stefan Delplace, secretary general EURASHEstefan.delplace@eurashe.eu

Barcelona

17 March 2009

2

ARACIS Romanian QA & Accreditation Agency

Background/Terms of reference Purpose of this review: ENQA membership

criteria met? One-time individual experience Challenges/Unresolved issues Interpretation of ESG Conclusions

2009

3

Background/Terms of reference

ARACIS successor of former CNEAACandidate status ENQA membership now terminatedReview (result) by EUA in 07-08 was not entirely compatible with ENQA standards Review by ESU (08): focused exclusively on student involvementTwo private agencies formulated accusationsLegislation on Romanian QA & accreditation agency

2009

4

Purpose of the review: ENQA membership criteria met?

Type A review: criteria ENQA & thus of ESG,

by ENQA team ‘Substantial compliance with ESG’ if

individual criteria met? Role & tasks of agency in context of

Romanian HE system End result to allow national agency to

operate in accordance with the law

2009

5

On this individual experience

Suitability of ENQA pre-review proceedings Abundance of preparatory documentationInformal contacts & formal settingsOn-site visit to premisesRoom for improvement : ESG as an evaluation

instrument

2009

6

CHALLENGES/UNRESOLVED ISSUESFOR REVIEWS OF AGENCIES

Mix-up of responsibilities of different parties in QA process

Suitability of legal provisions for QAEvidence if a ESG criterion is met?Systemic evaluation: collection & checking of such

evidenceHow to measure the real & long-term impact of QA

provisions?

2009

7

March 2009

On-site interpretation of ESG• the ‘involvement of SH’ (also non-academic) is a verifiable item, but still undervalued (2.2)• ‘moderating conclusions’ may be purely formal, and an excuse for

cumbersome proceedings (2.3)• ‘fit for purpose’ not only to apply for processes, but also to the overall

European dimension (2.4)• Cultural differences determine the degree of openness concerning

‘reporting’ requirements (2.5)• Consistency in ‘follow-up procedures’ is subject to general

interpretation of purpose of an external review (2.6)• Wealth of information to be gathered from QA reviews: its (‘system-

wide’) value for all SH is underestimated (2.8)

8

March 2009

On-site interpretation of ESG

*Programme/institutional reviews require different

mindset & proceedings (3.1)• Grip on non-accredited HEIs not guaranteed by existence of ‘official agency’ (3.2)• Competitive position of official agency depends on its wide range/ scope of ‘QA

activities on a regular basis’ (3.3)• Current available resources no guarantee for permanent compliance; staff development on all levels (3.4)• Mission statement is an ‘action paper’ towards SH (3.5)• ‘Independence’ & ‘accountability’ not to be seen as conflicting principles, but truly complementary (3.6/3.7)

9

March 2009

Conclusions

new ‘lay-out’ of ESG was overdue review proceedings seem adequate feedback on HE structure & role of QA more stakeholders in HE than we are

aware of?