1 Some Basic Observations: External Evaluation and Accreditation as a Specific Case of External...

1

Some Basic Observations:

External Evaluationand Accreditation as a Specific

Case of External Evaluation

Prof. Dr. Jürgen Kohler, Greifswald (Germany)

2

I. Terminology and Correlation

1. Definitions (Approximation)

• Evaluation: a systematic appraisal (of a HEI institution, its programmes, etc.) by experts with a view towards identifying features of quality in relevant areas in order to induce and support improvement

• Accreditation: a type of evaluation (see above) with a view towards identifying the accomplishment of certain ‘standards’ (‘threshold concept’) – usually in order to judge on the validity (admissability, in various kinds: e.g., right to operate; funding, ‘quality label’) of the object to be appraised

3

2. Common Features

• Purpose: Instruments of Quality Assurance

• Focus: Open; e.g., programmes, institutions, processes, or specific features (such as student support systems)

• Methodology, in major aspects: Self-assessment report; peer/expert review; site visit; judgment on existing features of quality; conclusions; final report

4

3. Differences: in theory

• Outcome: advisory (e) – granting/denying rights (a; usually)

• Style: collegial (e) – authoritative (a)

• Atmosphere: (more) frank (e) – (more) self-protective (a)

5

II. Bridging Differences

1. Levelling theoretic differences in practice:

• De facto, accreditation reports often also include elements of (mere) advice/ recommendation

• Evaluation findings may be included in accreditation fact-finding

• Due to de-facto overlap: Style of collegiality and atmosphere of frankness in evaluation processes may suffer

6

2. The accreditation concept of matching threshold standards revisited:

to be detailed hereafter

7

III.  Quality concepts – different or convergent?

1. (Main) Quality Definitions and Options:

• excellence

• fitness of, and for purpose

• matching directives (complying with curricular templates)

• meeting thresholds (complying with standards)

• client/customer satisfaction

• value for money/time invested (efficiency)

• individual enhancement (transformation)

• (institutional) capacity for change

8

2. (Traditional) Approach of Accreditation (?):Compliance/Matching (threshold) standards

Model template/threshold (t): features a(t) + b(t) + c(t) + … + z(t)

Sketching the concept:

Criterion: compliance/identity

Concrete programme (p): features a(p) + b(p) + c(p) + … + z(p)

9

Fitness of purpose

Fitness for purpose

(1) Objectives: valid

(2) Concept: fitting

(3) Implementation: true

(4) Monitoring: honest

(5) Enhancement: timely

10

4.    Moving accreditation towards fitness of/for purposeapproach

• Per se/as such: shift in methodology of programme accreditation

• Via institutional approaches: shift from programme level to institutional steering of quality related processes (‘quality audit’);

Sketching the concept:

11

programme (object of activity)

iteration/enhancement objective – concept – implementation – monitoring

process steering the quality cycle

institution (active subject)

actors action interaction(quality culture and management support; int’l and ext’l communication, transparency, decision-making, setting milestones, et al)

12

IV. Making Evaluation and Accreditation Match

Programme approach Institutional approach

compliance autonomous process of development

holistic entity

model template/ threshold oriented

fitness of and for purpose-oriented

study programme steering

(not con-sidered here)

13

Explanation:Italics, ordinary: tradit. accreditationItalics, bold: evaluation, plus ‘new’ accreditation

(of programmes)Bold print: institutional capacity for steering

programme quality (evaluation – accreditation/audit)

14

2. Identity via common denominators (I); e.g.: Role of qualifications framework

15

The European Higher Education Qualifications Framework: System Survey© J. Kohler, Greifswald/Germany

pro forma in substance

cycles qualifications descriptor elements

3rd

cycl

e2n

d cy

cle

degree

degree

degree

specific competences

specific competences

specific competences

Transparency (e.g. diploma supplement); reliability; verifiability; quality assurance

gene

ric/

gene

ral a

cade

mic

, so

cial

, per

sona

l com

pete

nces

subj

ect r

elat

ed (

prof

ile)

quan

tita

tive

: (cr

edit

poi

nts,

EC

TS

)

qualitative

lev e

l 1le

vel 2

leve

l 3

or other certification

attribution

1st c

ycle

orientation: learning outcomes

shor

tpr

ogra

mm

e (o

ptio

nal)

16

3. Identity via common denominators (II); e.g.: Role of ECTS

Entrepreneurial University/Individuality

Individual Programme/Curriculum

Macro-(programme-)-level

Micro("module"-)level

quantitative qualitative

ECTS 1a

Bachelor:180/240Master: - 300

quality /recognition/mobility

1 ECTS-functions1a Regulatory effects: 1b Procedural effects:

a) targeted reflection on quality: workload/per module def. of module by learning outcomes learning outcomes as development of competencies

b) compilatory: transferability by means of accumulative (assessment) structure ECTAS

Learning Outcomes Competencies 2: academic

quality + employability + citizenship+ personal development

2 a) employability ./. acad.qualityb) competencies ./. disciplinary material subject related generic („social skills“)

„Product“,Programme:acad. quality + employability

3 a) transparency diploma supplement

b) common denominators: competencies; quantities

c) Lisbon Convention: "presumption of quality"

d) scope for differentiation

coherence(menue, not cafeteria)

"tuning" (?)

process 1b Imp

lem

enta

tion

, pro

gram

me-

rela

ted

-aim

s-s

kill

s-d

idac

tics

Technical: -frames (standardisation);Substantial: -accumulation

Licensing/Quality assurance:-accreditation-evaluation

Modelling „Bologna Reforms“ via ECTS © J.Kohler, Greifswald/Germany

3

x ECTS = y h workl. manage- 30 800/900 h ability