1 Developing a Quality Culture

Developing a quality culture:

The basic framework

Mag. Oliver Vettori

March 2008

EQM

March 2008Evaluierung & Qualitätsmanagement in der Lehre 2

Introduction

‘The quality of higher education has proven to be at the heart of a European Higher Education area’.

(Berlin Communiqué)

EQM


Introduction

‘The concept of quality is not new: it has always been part of the academic tradition. It is the outside world that now emphasises the need for attention to quality…It is the relationship between higher education and society which has changed’.

(Vroeijenstijn 1995)

EQM


Introduction

Bologna process - European Higher Education Area:degree structure, student centred learning, student mobility

Accountability demands: evaluations, controlling, performance based funding schemes, employability rates

Institutional profiles and international competition:rankings, ratings, benchmarking

Changing role of the university managemententrepreneurial universities, increased autonomy

Rising importance of quality assurance

EQM


Quality notions

Quality

Value for moneyStakeholder

satisfactio

n

Tra

nsf

orm

atio

n

excellence

Fitn

ess for

pu

rpo

se

Management

Corporations, employers

Teachers

GovernmentStudents

Graduates

AdministratorsSociety at large

…

EQM


Differing views on Quality Cultures

control oriented

quality as an objective that canbe created and managed

clear understanding of quality required

top down implementation of quality standards & strategies

quality improvement can bemeasured – with unintended

consequences

participative

quality as an emerging result that changes in a dynamic way

permanent negotiations of quality notions required

participative development &implementation of quality st.

quality development can beinfluenced – with uncertainresults

EQM


Setting the frame…

Quality as a dynamic, multidimensional and perspective-bound concept

Culture as a complex, interaction-based, and continuously changing construct

Quality Cultures as stakeholder-dependent, historically grown and learning-oriented social

phenomena

The way(s) quality and culture can be related to each other have important consequences for

QM and QA strategies

EQM


Principles of a quality culture

Empowering the stakeholders to develop their own quality goals, initiatives and measures

Guaranteeing transparency and common standards without succumbing to a purely formal

quality approach

Showing trust without disregarding the risks involved

Strengthening reciprocal communication processes

Balancing the delegation and acceptance of responsibility

EQM


Fitness for purpose approach

Graduate surveys, round tables, feedback box, awards, student evaluation of teaching questionnaires, program monitoring, focus groups, advisory board, accreditation, process analysis, student support program, discussion groups, staff development, premiums, fast feedback, midterm review, external evaluation, tutoring, COMMUNICATION, peer review, implementation of standards, rules and procedures, online evaluation, reporting, accompanying research projects, drop out study, steering committee, interviews, data ware house, meta-evaluation, counselling etc.

EQM


concepts

organisation

operation

QA roles: the formal structure

Vice-Rector f. Academic Programs a. Student Affairs

Teaching Evaluation &

QM Office

Evaluation Board of the

Senate

conciliates

Reports to

lays foundation

EQM


QA roles in a quality culture perspective

Management

Administrators

Teachers

Students …….

dialogue

measurement improvement

EQM


QA roles: A few examples at WU Wien

Encouraging quality intiatives on departmental level

Rewarding initiatives for innovative and excellent teaching

Reinventing program directors and coordinators

Collaborating with staff development and controlling units as well as teachers and researchers throughout the university

EQM


… and some final caveats

Organisational beliefs and values are not homogeneous, not even within a stakeholder group

Quality cannot be decreed

Permanent control or standardisation are no guarantors for success and transparency

External and internal requirements cannot be satisfied by one and the same strategy

EQM


Part II

The students‘ role

EQM


Student Contributions

The students‘ role

give feedbackparticipate in surveys and selfevaluations

take part in decision-making and design processes

tutors and mentors self-assessment and self-evaluation

take responsibility

EQM


student contributions - examples

course evaluations, sounding boards, feedback boxes, fast feedback …

introductory tutors, learning coaches, buddy networks, teaching assistants…

members of self-assessment teams, members of external review teams, graduate surveys …

curriculum development boards, advisory boards, teaching & learning forums etc.

feedback

tutoring

evaluating

designing& deciding

EQM


Student self-assessment (1)

What did I learn today? What were my contributions in class today?What do I need help with?What do I want to know more about?What would you do differently, if you took this class Again?

EQM


Student self-assessment

• self-evaluation forms

• learning inventories

• reflection logs and learning diaries

• expectation exercise

• discussions

• peer assessment

EQM


Part III

The administration‘s role

EQM


Administrative contributions

The administration‘s role

providing quality services

supporting feedback loops

Engaging in self-Assessment and self-reflection

avoiding quality bureaucracy

establishing supportive structures

information and communication

EQM


feedback systems

develop suitable quantitative and qualitative quality indicators that hold relevance for all the actors involved

maintain a reasonable balance between the input (time, efforts) and outcome of evaluations

support the other actors to make sense of their findings

integrate evaluations into a broader system of quality assurance and development (follow ups)

keep the feedback processes as transparent as possible

EQM


necessary information

•student progression and success rates

•employability of graduates

•students’ satisfaction with their programmes

•effectiveness of teachers

•profile of the student population

•learning resources and their costs

•the institution’s own key performance indicators

ENQA Standards & Guidelines

EQM


Criteria for self-evaluation

(cost) effectiveness

efficiency

information quality

celerity

transparency

service-orientation

learning-orientation

…

EQM


Part IV

The teachers‘ role

EQM


Administrative contributions

The teacher‘s role

creating suitable learning environments

peer teaching & peer reviews

engaging in self-Assessment and self-reflection

implementing a feedback culture

take the role of learners

learning coaches

EQM


Goals of evaluations

Feedback assessment

improvement

developmentcontrol

Quality of content and didactics

Quality of teaching and learning processes

Staff satisfaction/student satisfaction

EQM


Course Evaluations

EQM


Student questionnaires

assessing the consistency of teaching goals, contents and methods

regular student feedback (questionnaires)

multifaceted item pool, flexibilised

questionnaires

Constant documentation/observance of

results and developments

objective

method

tools

monitoring

EQM


Additional Fast Feedback

early diagnosis of potential problems, assessing standards of knowledge/understanding etc.

flexible and individualised use:

dimensions/aspects

time

extent

counselling, support

objective

method

Support

EQM


Fast Feedback: Beispiel 1

Course:

+ ?What I liked about the course.. What could be improved…

My further comments

EQM


Fast Feedback: Beispiel 2

Course Date

Minute Paper

Please answer the two questions below:

What is the most important thing you What were the “muddy points”? learned today?

Documents

1 Developing a Quality Culture