International Developments in Academic Audit

Source: International Network of Quality Assurance Agencies in Higher Education INQAAHE Biennial Conference

Wellington 29 March - 1 April 2005

Themes:

o Effectiveness of Quality Assuranceo Quality Assurance in Transnational

Issueso Impacts on Diversityo Indigenous Peoples

Keynote Speakers:

o Professor Denise Bradley - VC University South Australia

o Professor Ranginui Walker - Retiredo Sir John Daniel - President & CEO, The

Commonwealth of Learning Organisation

Speakers

o Phil Meadeo Gareth Joneso Sarah Carro Emma Hamilton

Quality Assurance Agencies: New Guidelines

Standards & Guidelines for Quality Assurance in the European Higher Education Area:

February 2005

INQAAHE Guidelines for Good Practice: January 2005

Guidelines for Quality Provision in Cross-Border Higher Education jointly elaborated by

UNESCO and the OECD: March 2005

Standards & Guidelines - European HE

Part 1: Guidelines for internal quality assurance within HEIs

1.1 Policy and procedures for QA

1.2 Approval, monitoring & review of programmes

1.3 Assessment of students

1.4 Quality assurance of teaching staff

1.5 Learning resources and student support

1.6 Information systems

1.7 Public information

Standards & Guidelines - European HE

Part 2: Standards for External Quality Assurance of HE (cont.)

2.1 Use of internal quality assurance procedures2.2 Development of external quality assurance

processes2.3 Criteria for decisions2.4 Processes fit for purpose2.5 Reporting2.6 Follow-up procedures2.7 Periodic reviews2.8 System - wide analyses

Standards & Guidelines - European HE

Part 3: Standards for external quality assurance agencies (cont.)

3.1 Procedures in part 2 to be used3.2 Official status3.3 Activities3.4 Resources3.5 Mission statement3.6 Independence3.7 External quality assurance criteria and processes used by

agencies should be pre-defined and publicity available

3.8 Accountability procedures for their own accountability

Guidelines for Quality Provision in Cross-Border Higher Education

o UNESCO’S aim for cross-border education

• radically change cost structures to make it affordable• keep young people learning in their country and stay there• accessibility, availability, affordability needed

o 4 billion poor people in the world who want a better lifeo There is 40% participation in HE in developed

countrieso If there was 35% participation in under-developed

countries there would be 150 million extra students to serve

Guidelines for Quality Provision in Cross-Border Higher Education

o A need for strong partnership with local institutions to make sure programmes are relevant

o Providers need to take national priorities into account and refrain from academic dumping

o Highly distributed, culturally sensitive, small scale operations are called for

o A need to apply:o new technologieso e-learning and reusable learning objectso connectivity and free open source software o apply QA

Guidelines for Quality Provision in Cross-Border Higher Education

o Students need to be protected from dubious

supplierso Foreign suppliers must agree to accredit

graduates in own countrieso Must partner with local institutionso QA concernso Capacity building and long-term perspective

called for

Australian Universities Quality Agency (AUQA)

AUQA audits extended to State Accreditation Agencies

State Agencies have developed legislation to prevent misuse of the university label and to protect use of the terms “university” and “degree”

The Australian - Wednesday March 30 2005

College amends offshore uni linkBrendan O’Keefe

A Sydney hypnotherapy college has removed

references to an unaccredited overseas university

and the degrees it offered after a prosecution threat

from the NSW Department of Education and training

…

Australian Universities Quality Agency (AUQA)

AUQA uses National Protocols for Higher Education Approval Processes

Protocol 1 Criteria and processes for recognition of universities

Protocol 2 Overseas higher education institutions seeking to operate in Australia

Protocol 3 The accreditation of higher education courses to be offered by non-self accreditation

agenciesProtocol 4 Delivery arrangement involving other organisationsProtocol 5 Endorsement of courses for overseas students

AUQA - Strengthening of processes

o Visits to off-shore campuseso equivalence of entry & assessment standardso acknowledgement of graduates by professional accreditation

bodies for registration purposeso provision of learning and support services

o Visits to Partners operating in Australiao nature of MOUo use of university brand in advertisingo standards; credit transfero learning & support services

o Drilling down - use of web-site - seeking additional information

AUQA Good Practice

DatabaseExamples:Mäori @ Massey StrategyMassey University Workloads PolicyAcademic Policy Formation (Massey): Collegial

Participation in University GovernanceEnsuring Awareness of Plagiarism Policy (UNE)Improving Graduate Supervision and Completion Rates

(RMIT)Integrating Graduate Attributes into UNE Courses

Review of the Queensland Studies Authority

An alternative approach to standard audit with Self Review and Quality

Portfolio followed by visit by External Audit Team

Review of the Queensland Studies Authority

o Minister appoints an External Reviewero External Reviewer consults with Reference Group

comprising Key Stakeholderso Minister Releases Discussion Paper (prepared by

External Reviewer approved by Cabinet)o Public Submissions calledo Reviewer Prepares Report & states whether or not

recommendations are supported by Stakeholders

o Recommendations considered by Cabinet

INQAAHE Conference Indigenous Theme Professor Ranginui Walker

Powerful address advocating for a bi-cultural educational experience for Mäori

o Use of building and campus design structures sensitive to Mäori culture

o Learning of Mäori languageo Embedding of Mäori values in processes during provision of

educational services

Emerging role of World Indigenous Nations Higher Education Consortium WINHEC

Dr Manulani MEYERUniversity of Hawaii at Hilo

“Higher Education Quality Assurance and Indigenous

Epistemology: Triangulating Our Way to Meaning”

o The more specific we are about culture the more we head towards systems of universality, so specificity (not uniformity) leads to universality

o Uniformity in QA leads to complianceo Message to young indigenous scholars “when you excel, don’t

leave us for your culture will wither on the vine”

Elizabeth Bean(Lincoln University) Post-graduate Experience

“Do the post-graduate students agree with the academic auditors?”

University of Otago CEQ Good Teaching Scale 1995-2002

1995

-1998

1995

-1998

1995

-1998

1996

-1999

1996-1999

1996-1999

1997-2000

1997-2000

1997-2000

1998

-2001

1998-2001

1998-2001

1999-2002

1999

-2002

1999

-2002

1

2

3

4

5

CC3 CC7 CC18

Question

4 Year Moving Average

Comparison of the CEQ overall satisfaction question: Otago mean score and Australian national score

Australian

National Mean = 0D

eviation from the w

eighted mean

University of Otago teaching evaluations 2002: “How

effective was Dr X in teaching this course?”

% of satisfied or highly satisfied respo

nses

External research funding at the University of Otago 1994 – 2003

$0

$10

$20

$30

$40

$50

$60

1994 1995 1996 1997 1998 1999 2000 2001 2002 2003

M$

Number of PhD graduates at the University of Otago 1995 - 2003 Note – change in way data was recorded in 1999 results in an anomaly during that year

0

20

40

60

80

100

120

140

1995 1996 1997 1998 1999 2000 2001 2002 2003Number of PhD Graduatess

PBRF quality evaluation score for New Zealand universities 2003

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

New Zealand Universities

Quality Score (out of a potential of 10))))

BA C Otago D E F G

International Developments in Academic Audit

Summary

and

Discussion