17
1 Some Basic Observations: External Evaluation and Accreditation as a Specific Case of External Evaluation Prof. Dr. Jürgen Kohler, Greifswald (Germany)

1 Some Basic Observations: External Evaluation and Accreditation as a Specific Case of External Evaluation Prof. Dr. Jürgen Kohler, Greifswald (Germany)

Embed Size (px)

Citation preview

Page 1: 1 Some Basic Observations: External Evaluation and Accreditation as a Specific Case of External Evaluation Prof. Dr. Jürgen Kohler, Greifswald (Germany)

1

Some Basic Observations:

External Evaluationand Accreditation as a Specific

Case of External Evaluation

Prof. Dr. Jürgen Kohler, Greifswald (Germany)

Page 2: 1 Some Basic Observations: External Evaluation and Accreditation as a Specific Case of External Evaluation Prof. Dr. Jürgen Kohler, Greifswald (Germany)

2

I. Terminology and Correlation

1. Definitions (Approximation)

• Evaluation: a systematic appraisal (of a HEI institution, its programmes, etc.) by experts with a view towards identifying features of quality in relevant areas in order to induce and support improvement

• Accreditation: a type of evaluation (see above) with a view towards identifying the accomplishment of certain ‘standards’ (‘threshold concept’) – usually in order to judge on the validity (admissability, in various kinds: e.g., right to operate; funding, ‘quality label’) of the object to be appraised

Page 3: 1 Some Basic Observations: External Evaluation and Accreditation as a Specific Case of External Evaluation Prof. Dr. Jürgen Kohler, Greifswald (Germany)

3

2. Common Features

• Purpose: Instruments of Quality Assurance

• Focus: Open; e.g., programmes, institutions, processes, or specific features (such as student support systems)

• Methodology, in major aspects: Self-assessment report; peer/expert review; site visit; judgment on existing features of quality; conclusions; final report

Page 4: 1 Some Basic Observations: External Evaluation and Accreditation as a Specific Case of External Evaluation Prof. Dr. Jürgen Kohler, Greifswald (Germany)

4

3. Differences: in theory

• Outcome: advisory (e) – granting/denying rights (a; usually)

• Style: collegial (e) – authoritative (a)

• Atmosphere: (more) frank (e) – (more) self-protective (a)

Page 5: 1 Some Basic Observations: External Evaluation and Accreditation as a Specific Case of External Evaluation Prof. Dr. Jürgen Kohler, Greifswald (Germany)

5

II. Bridging Differences

1. Levelling theoretic differences in practice:

• De facto, accreditation reports often also include elements of (mere) advice/ recommendation

• Evaluation findings may be included in accreditation fact-finding

• Due to de-facto overlap: Style of collegiality and atmosphere of frankness in evaluation processes may suffer

Page 6: 1 Some Basic Observations: External Evaluation and Accreditation as a Specific Case of External Evaluation Prof. Dr. Jürgen Kohler, Greifswald (Germany)

6

2. The accreditation concept of matching threshold standards revisited:

to be detailed hereafter

Page 7: 1 Some Basic Observations: External Evaluation and Accreditation as a Specific Case of External Evaluation Prof. Dr. Jürgen Kohler, Greifswald (Germany)

7

III.  Quality concepts – different or convergent?

1. (Main) Quality Definitions and Options:

• excellence

• fitness of, and for purpose

• matching directives (complying with curricular templates)

• meeting thresholds (complying with standards)

• client/customer satisfaction

• value for money/time invested (efficiency)

• individual enhancement (transformation)

• (institutional) capacity for change

Page 8: 1 Some Basic Observations: External Evaluation and Accreditation as a Specific Case of External Evaluation Prof. Dr. Jürgen Kohler, Greifswald (Germany)

8

2. (Traditional) Approach of Accreditation (?):Compliance/Matching (threshold) standards

Model template/threshold (t): features a(t) + b(t) + c(t) + … + z(t)

Sketching the concept:

Criterion: compliance/identity

Concrete programme (p): features a(p) + b(p) + c(p) + … + z(p)

Page 9: 1 Some Basic Observations: External Evaluation and Accreditation as a Specific Case of External Evaluation Prof. Dr. Jürgen Kohler, Greifswald (Germany)

9

Fitness of purpose

Fitness for purpose

(1) Objectives: valid

(2) Concept: fitting

(3) Implementation: true

(4) Monitoring: honest

(5) Enhancement: timely

Page 10: 1 Some Basic Observations: External Evaluation and Accreditation as a Specific Case of External Evaluation Prof. Dr. Jürgen Kohler, Greifswald (Germany)

10

4.    Moving accreditation towards fitness of/for purposeapproach

• Per se/as such: shift in methodology of programme accreditation

• Via institutional approaches: shift from programme level to institutional steering of quality related processes (‘quality audit’);

Sketching the concept:

Page 11: 1 Some Basic Observations: External Evaluation and Accreditation as a Specific Case of External Evaluation Prof. Dr. Jürgen Kohler, Greifswald (Germany)

11

programme (object of activity)

iteration/enhancement objective – concept – implementation – monitoring

process steering the quality cycle

institution (active subject)

actors action interaction(quality culture and management support; int’l and ext’l communication, transparency, decision-making, setting milestones, et al)

Page 12: 1 Some Basic Observations: External Evaluation and Accreditation as a Specific Case of External Evaluation Prof. Dr. Jürgen Kohler, Greifswald (Germany)

12

IV. Making Evaluation and Accreditation Match

Programme approach Institutional approach

compliance autonomous process of development

holistic entity

model template/ threshold oriented

fitness of and for purpose-oriented

study programme steering

(not con-sidered here)

Page 13: 1 Some Basic Observations: External Evaluation and Accreditation as a Specific Case of External Evaluation Prof. Dr. Jürgen Kohler, Greifswald (Germany)

13

Explanation:Italics, ordinary: tradit. accreditationItalics, bold: evaluation, plus ‘new’ accreditation

(of programmes)Bold print: institutional capacity for steering

programme quality (evaluation – accreditation/audit)

Page 14: 1 Some Basic Observations: External Evaluation and Accreditation as a Specific Case of External Evaluation Prof. Dr. Jürgen Kohler, Greifswald (Germany)

14

2. Identity via common denominators (I); e.g.: Role of qualifications framework

Page 15: 1 Some Basic Observations: External Evaluation and Accreditation as a Specific Case of External Evaluation Prof. Dr. Jürgen Kohler, Greifswald (Germany)

15

The European Higher Education Qualifications Framework: System Survey© J. Kohler, Greifswald/Germany

pro forma in substance

cycles qualifications descriptor elements

3rd

cycl

e2n

d cy

cle

degree

degree

degree

specific competences

specific competences

specific competences

Transparency (e.g. diploma supplement); reliability; verifiability; quality assurance

gene

ric/

gene

ral a

cade

mic

, so

cial

, per

sona

l com

pete

nces

subj

ect r

elat

ed (

prof

ile)

quan

tita

tive

: (cr

edit

poi

nts,

EC

TS

)

qualitative

lev e

l 1le

vel 2

leve

l 3

or other certification

attribution

1st c

ycle

orientation: learning outcomes

shor

tpr

ogra

mm

e (o

ptio

nal)

Page 16: 1 Some Basic Observations: External Evaluation and Accreditation as a Specific Case of External Evaluation Prof. Dr. Jürgen Kohler, Greifswald (Germany)

16

3. Identity via common denominators (II); e.g.: Role of ECTS

Page 17: 1 Some Basic Observations: External Evaluation and Accreditation as a Specific Case of External Evaluation Prof. Dr. Jürgen Kohler, Greifswald (Germany)

Entrepreneurial University/Individuality

Individual Programme/Curriculum

Macro-(programme-)-level

Micro("module"-)level

quantitative qualitative

ECTS 1a

Bachelor:180/240Master: - 300

quality /recognition/mobility

1 ECTS-functions1a Regulatory effects: 1b Procedural effects:

a) targeted reflection on quality: workload/per module def. of module by learning outcomes learning outcomes as development of competencies

b) compilatory: transferability by means of accumulative (assessment) structure ECTAS

Learning Outcomes Competencies 2: academic

quality + employability + citizenship+ personal development

2 a) employability ./. acad.qualityb) competencies ./. disciplinary material subject related generic („social skills“)

„Product“,Programme:acad. quality + employability

3 a) transparency diploma supplement

b) common denominators: competencies; quantities

c) Lisbon Convention: "presumption of quality"

d) scope for differentiation

coherence(menue, not cafeteria)

"tuning" (?)

process 1b Imp

lem

enta

tion

, pro

gram

me-

rela

ted

-aim

s-s

kill

s-d

idac

tics

Technical: -frames (standardisation);Substantial: -accumulation

Licensing/Quality assurance:-accreditation-evaluation

Modelling „Bologna Reforms“ via ECTS © J.Kohler, Greifswald/Germany

3

x ECTS = y h workl. manage- 30 800/900 h ability