Ekkehard Nuissl von Rein

Quality Assurance by External Evaluation

of Leibniz Institutes

Strasbourg, 15th November 2005

1. The Leibniz Association

2. External Evaluation Procedure of the Leibniz Association

3. Results of the External Evaluation

Applied Research

Fraunhofer Society

BasicResearch

Max Planck Society Helmholtz Association

Leibniz Association

Problem-orientedResearch

German Publicly FundedNon-University Research System

1. The Leibniz Association

The 84 research institutes and scientific service facilities of the Leibniz Association are divided into sections:

A: Humanities and Education Research (14 institutes)

B: Economics, Social Sciences and Regional Infrastructure Research (17)

C: Life Sciences (21)

D: Mathematics, Natural Sciences and Engineering (23)

E: Environmental Sciences (9)

1. The Leibniz Association

Leibniz Institutes

• address specific scientific problems, which are socially relevant e.g.

– Diabetes Research

– Climate Research

• offer consulting services for different reference groups, e.g.

– Economics

– Adult Education

• offer information or infrastructure services for research, e.g.

– Primate Centre

– Information Centre for Social Sciences

1. The Leibniz Association

Location

of the 84

Leibniz Institutes

Funding

• Non-profit research organisation

• Institutes are co-financed by the federal and

the 16 state governments

• Budget of about one billion €, thereof 20 % from

third-party funding

• About 12,000 employees, about 7,000 academics

1. The Leibniz Association

Section A

Section B

Section C

Section D

Section E

AdministrativeBoard

Leibniz Senate

Organization of the Leibniz Association

AdministrativeOffice

EvaluationOffice

President

Quality Assurance System

Scientific excellence and service quality is ensured by three elements:

• Internal quality management by the institutes

• Accompanying consulting and evaluation by the Advisory Boards of the institutes

• External Evaluation by the Leibniz Senate

1. The Leibniz Association

• Accountability: Assessment whether the institute meets the requirements of further public funding

formulated as evaluation criteria

• Improvement: Recommendation for the institute‘s development

Objectives of the external evaluation procedure are

2. External Evaluation Procedure

Evaluation criteria

1. Scientific Quality of research and service: determining programmatic focus areas and future prospects; theoretical and methodological foundation of research; work efficiency and effectiveness; raising third party funds

2. External Evaluation Procedure

Evaluation criteria

2. National and international significance: International standing; networking; promoting young scientists; political, economic and social relevance

2. External Evaluation Procedure

Evaluation criteria

3. Structure and organization: Efficiency of management and organization, human resource development, internal quality management

2. External Evaluation Procedure

Methods

• Analysis of available documents (budgets etc.)

• Peer review by an expert panel• Statistical methods, bibliometric data

(if available)• Visit of the institute‘s departments and

interviews with the institute‘s staff

2. External Evaluation Procedure

4 Steps of the external Evaluation Procedure:

(1) Institutes report on their work

(3) Leibniz Senate discusses and passes statements

(4) Political Committee (BLK) decides about further funding

(2) International expert panels evaluate the quality of work

4 3

21

The evaluation procedure is based on principles:

• Regularity

• Independence

• Transparency

• Acceptance

2. External Evaluation Procedure

Crucial Points of the Evaluation Procedure

• Appropriateness of criteria

• Selection of experts

• Acceptance of recommendations

2. External Evaluation Procedure

3. Results of the External Evaluation

41 institutes have been evaluated since 2002

The Leibniz Senate has issued 24 statements:

- 19 institutes will be supported for the next 7 years

- 4 will be re-evaluated in 3 years

- 1 institute will be partly merged

Statistics

Costs per evaluation

55,000 € on average

Number of experts 10 to 12

Experts‘ nationalities

A, AUS, CH, DK, F, GB,

GER (75 %), NL, S, USA

Professional work areas of experts

university research (65 %), non-university research (15 %), economy/industry (8 %), associations, public authorities

Evaluations per year

12 to 14

Duration of a complete evaluation

about one year

Quality Assurance by external evaluation?

Observations on institutes’ performance development:

In general institutes have

- focused their research and service profile

- increased scientific output, e.g. papers in reviewed journals

- strengthened international co-operations

- implemented internal quality management

- emphasized promotion of young scientists.

Detailed information can be found on our homepage

www.wgl.de/evaluation

e.g.:

The Senate’s statements and evaluation reports

Questionnaires, tables and information on the evaluation procedures as well as the Senate Evaluation Committee