Upload
hannah-dalton
View
213
Download
0
Tags:
Embed Size (px)
Citation preview
Quality Procedures in European Higher Education
Findings of the 2008 Survey
Achim Hopbach
Managing Director German Accreditation Council
Background
“QPP 1” in 2003: Major progress towards convergence despite
varying national priorities
Major developments since then: Quality Assurance in the European Higher
Education Area (ESG) in 2005 Common understanding of quality assurance
Framework for Qualifications of the European Higher Education Area (FQEHEA) in 2005
Common understanding of learning outcomes
Goals of the survey
to update the 2003 survey to assess the agencies’ opinion on their
compliance with Part 3 of the ESG to survey agencies’ external review
plans to survey agencies’ attitudes towards
the European Quality Assurance Register for Higher Education (EQAR)
Method Development of online questionnaire 2nd half 2007 Responses by 31st January 2008 1st analyses: Division of 71 questions into seven
sections (by May 08) Final analysis, draft report (by August 08)Organisation Project run by HETAC and QAA Steering group Project group (9 members responsible for different
chapters)
Responses
51 responses from 30 EHEA countries (2002: 36 from 23)
29 full member agencies (out of 34)
7 candidate member agencies (out of 11)
11 Associates 5 Affiliates
Findings: Agencies
Scope, remit and domain of agencies Agencies predominantly cover both
universities and other types of HEI (28 in 24 countries), and all fields of learning
35: (whole) country 7: part of a country (6: Spain, Belgium) 4: international agencies by definition
42: All HEI subject to compulsory external QA
Scope, remit and domain of agencies Agencies predominantly cover both universities
and other types of HEI (28 in 24 countries), and all fields of learning
35: (whole) country 7: part of a country (6: Spain, Belgium) 4: international agencies by definition
42: All HEI subject to compulsory external QAOne QA framework in national HEI systems
Status of agencies
44 out of 49 agencies based on law or recognised by public authority
only one responsible agency per country (except for federal systems of Spain and Germany)
QA as main task which is carried out regularly (42 out of 46)
Status of agencies
44 out of 49 agencies based on law or recognised by public authority
only one responsible agency per country (except for federal systems of Spain and Germany)
QA as main task which is carried out regularly (42 out of 46)
Legally based agency with sole responsi-bility as widespread European pattern
Function of agenciesFunctions Response
s %(very) important
Quality assurance 100 95.3
Quality enhancement 97.6 93.0
External QA of programmes 95.5 84.1
Collecting/Diseminating information on quality
95.5 70.5
External QA of institutions 90.9 77.3
Recognition/licensing of institutions 66.7 57.1
Development/Maintenance of discipline standards
66.7 41.9
Development/Maintenance of NQF 62.8 41.9
Deciding on the funding of iunstitutions 50.0 26.2
Recognition of national diplomas 47.6 42.9
Funding
Funding
Government funding is the main source of funding for agencies
Source Average %
No. of agencies
Government 70.63 41
HEIs (fixed subscriptions only) 37.35 15
Fees (for specific activities) 32.85 28
Grants (other than government) 2.25 12
Other 11.72 18
Findings: Quality Procedures
Activities
Most frequent types of QA:Evaluation, accreditation, audit
65% programme approach40% institutional approach
Either regular or never (very seldom occasionally)
90% are not confined to only one approach
Reference frame (1)
What defines the reference frame for the external quality procdure?
% of agencies
Legal regulation 74.5
Stated goals of the institutions 57.4
Guidelines of good practice 57.4
National Qualifications Frameworks 55.3
European (Bologna) Qualifications Framework 44.7
Standards defined by professional organisations
34.0
Subject benchmarks 27.7
Reference frame (2)
Notion of criteria and standards as a common feature in European quality assurance (2003 survey: emerging feature)
What specific criteria and standards are used for your agency‘s external quality procedure?
% of agencies
Agency‘s own published criteria and standards 87.2
European Standards and Guidelines 83.0
National criteria and standards 66.0
OECD/UNESCO Guidelines on Quality Provision in Cross-boarder HE
14.9
Areas addressed in the quality proceduresThe five highest impacting items programme level procedures: qualifications of staff; curriculum/syllabus; facilities and resources; internal QA procedures; mission/goals.
institutional level procedures: internal QA procedures; management, organisation; mission/goals; qualifications of staff; facilities and resources.
Areas addressed in the quality proceduresThe five highest impacting items programme level procedures: qualifications of staff; curriculum/syllabus; facilities and resources; internal QA procedures; mission/goals.
institutional level procedures: internal QA procedures; management, organisation; mission/goals; qualifications of staff; facilities and resources.
Isn’t one item missing?
Areas addressed in the quality proceduresThe five highest impacting items programme level procedures: qualifications of staff; curriculum/syllabus; facilities and resources; internal QA procedures; mission/goals.
institutional level procedures: internal QA procedures; management, organisation; mission/goals; qualifications of staff; facilities and resources.
Addressing learning outcomes in (external) QA is only at the beginning.
Consequences of QA
External QA leads to formal consequences (approval, funding) in more than 75% of cases.e.g. approval in 90% of accreditationsapprox. 80% of evaluations and audits
All the common procedures are applied for these purposes
Consequences of QA
External QA leads to formal consequences (approval, funding) in more than 75% of cases.e.g. approval in 90% of accreditationsapprox. 80% of evaluations and audits
All the common procedures are applied for these purposesConsequences of external QA are taken independently of QA-type
Stakeholder involvement:Responsibilities66.7% External stakeholders may influence
neither the conclusions nor the recommendations in reports
78.7% External stakeholders have no share in responsibility for the operations
Others: Influence mainly in way of participation in agency’s body, panels, etc.
Stakeholder involvement:Responsibilities66.7% External stakeholders may influence neither
the conclusions nor the recommendations in reports
78.7% External stakeholders have no share in responsibility for the operations
Others: Influence mainly in way of participation in agency’s body, panels, etc.
Stakeholder involvement is a common feature; it does not necessarily compromise independence of the agencies
Stakeholder involvement:Specification of processes and criteria• In more than half of all respondent agencies
“government”, “quality assurance agencies” and “student representatives” are involved; “industry and labour market” (36%) and “professional organisations" (32%). In 4 cases also government.
• The final decision on the specification of processes and criteria is likely to be made by the relevant agency (79%). In two cases it is the government alone.
Panels: responsibility
WHO PERFORMS THE FOLLOWING FUNCTIONS IN THE EXTERNAL QUALITY PROCEDURES
Panel
No. of respondents
Agency
No. of respondents
Choice of basic methodology applied 12 44Preparation of the guidelines for the self-evaluation
4 42
Preparation of the external quality procedure concept
11 42
Contact with the institution 11 41Planning of the site visit 23 36Preparation of the guidelines for the site-visit
16 38
Writing of the report 41 17
Panels: membershipWho are members of the external expert panel?
% of agencies
(2003)
National experts representing area of focus
76.1
International experts 73.9
Professional practitioners 63.0
Students 56.5 22.0
National experts representing institutions
32.6
Employers 45.7 36.0
Staff members of the agency 26.2 40.0
Other 26.1
Professional organisations 19.6
Graduates 15.2
Panels: training
Duration: 1.5 days on average
Characteristics and circumstances of briefing/training provided to panel members
% of agencies
Training/briefing is compulsory for all panel members
75.0
Training/briefing is compulsory for all panel chairs
50.0
Training/briefing provided is matched to the experience of each panel members
42.5
Training outcomes are assessed by the agency 27.5
Training is requiered before a person may be nominated to a panel
25.0
Training is provided using distance learning 22.5
Other 17.5
Findings: Future developments
Future developments
Expected developments in the QA sector by January 2010
Response%
Revision of external quality procedures 57.4
Establishment of National Qualifications Framework
48.9
Introduction of different methodologies for external quality procedures
46.8
Commencement of new higher education legislation
31.9
Reorgamnisation or merging of the agency 31.9
36 out of 48 agencies changed their approach recently or are about to do so; 27: significant changes (8 referring to ESG, 3 switching to another approach, 16 adding another approach)
Future developments
Expected developments in the QA sector by January 2010
Response%
Revision of external quality procedures 57.4
Establishment of National Qualifications Framework
48.9
Introduction of different methodologies for external quality procedures
46.8
Commencement of new higher education legislation
31.9
Reorgamnisation or merging of the agency 31.9
36 out of 48 agencies changed their approach recently or are about to do so; 27: significant changes (8 referring to ESG, 3 switching to another approach, 16 adding another approach)
QA in flux
The register
78% planning to apply66% think it’s a useful tool
Expected benefits Response%
International recognition
85
Credibility 82
International recognition
59
Accountability 56
National recognition 26
Concluding remarks
National QA systems and role of agencies legally based and well developed
Methodology well developed (but 4-stage-model not applied everywhere, due to national context)
ESG as major reference frameConvergence continues but national context matters! Still: QA in flux (1)
Developing external QA in a changing world:Student centred learning, LO, stakeholder involvement, accountability to society
Demands on QA get diverse Purpose of QA gets diverse
QA in flux (2)
Final question:Is there a gap between direction/stage of development of QA methods and function/ purpose of QA?
Is the purpose of external QA compromised by different usage of results?What does it mean for the type and design of methods applied?