Teaching Quality Indicators Project

Aims of today’s session:

To consider the scope of ‘Institutional Climate and Systems’

To consider indicators as defined by the TQI project and elsewhere

To consider a move from ‘indicators’ to measures or standards

The TQI project defines performance indicators as:

“… measures which give information and statistics context; permitting

comparisons between fields, over time and with commonly accepted

standards. They provide information about the degree to which teaching

and learning quality objectives are being met within the higher education

sector and institutions.”

Components

Structural

Human Resources

Political

Symbolic

Diversity Issues

Components

General Characteristics

Leadership

Employee Management

Organisational Glue

Strategic Emphasis

Criteria of Success

Components

General Climate

Work-life balance

Professional Life

Situations and Interactions at Work

Perceptions of organisational features like decision-making, leadership and norms about work. The

ways in which members perceive and characterise

the work environment

Values, underlying assumptions, beliefs,

expectations, collective memories, ways of thinking,

managerial styles, paradigms, and approaches to problem

solving.

Institutional Work Climate

Organisational Culture

Departmental Work Climate

Institutional Climate

Valetin, 2007

http://www.uprh.edu/~advance/resumen%20ejecutivo%20clima%20revisado.pdf

Other definitions of Performance Indicators:

A numerical measure of the degree to which an objective is being achieved.

A performance indicator defines the measurement of a piece of important and useful information about the performance of a program expressed as a percentage, index, rate or other comparison which is monitored at regular intervals and is compared to one or more criterion.

Performance indicators are one tool in many to answer the questions:

How do we know that we are achieving? Are we achieving our aims and objectives?

Performance Indicators can and have been problematic:

What happens if the performance indicator is the pass mark (say in a proposed school league table system) and schools are rewarded for an increase in the performance indicator? Bullen 1991

http://www.mapl.com.au/A1A.htm

Performance Indicator

The University has clear, well established and understood promotion and career progression processes and policies that

actively support contributions to learning and teaching.

Standards or Grades

Criteria Level 4(Yes)

Level 3(Yes, but)

Level 2(No, but)

Level 1(No)

Institutional academic promotion policies and procedures that recognise and reward the contributions made to learning and teaching.

All relevant policies and procedures recognise and reward contributions made to L&T.

Some policies and/or procedures that recognise and reward contributions made to L&T.

Some policies and/or procedures but none explicitly recognising and rewarding the contributions made to L&T.

No policies or procedures evident.

Alignment with the institutional strategic and operational plans.

Clear criteria on L&T and the level and quality of evidence required for progression.

Access to Study Leave and Conference Leave with L&T focus is actively supported and funded.

LTPI 4: Development and Implementation of Faculty Learning and Teaching Plans with or without Department Plans.

Weighting: 15 points

Expectation:

Evidence of a Learning and Teaching Plan for the Faculty (and Department plans where they exist), which is consistent with the University Learning and Teaching Plan.

Evidence that initiatives in the Faculty and Departments are aligned with the Faculty Plan.

Evidence of the development of new Learning and Teaching initiatives aligned with the Faculty Learning and Teaching Plan (possibly funded through grants).

UNSW 2007

http://www.learningandteaching.unsw.edu.au/content/userDocs/LT_Performance_Indicators_2007-2008.pdf

Ideal Performance Indicator Actual

Low

Low

High

High

PRESENCE

IMPACT

HI

LP

HP

LI

HI

HP

LI

LP

IMPACT = actually achieving key objectives.

PRESENCE = being seen to be doing something (which may or may not be wholly or semi-cosmetic).

Developers need both Impact and Presence.

Presence is more social or psychological.

Impact is a measure of effectiveness.

The Map is Not the Territory (PIs are not the performance):

Performance indicators are not the judge of performance; performance indicators and performance are not the same.

Mistakes (including poor performance) are an opportunity for improving performance rather than an opportunity for 'punishment‘ (e.g. reprimanding staff or cutting funds).

Objectives are what we want to achieve in relation to a need. They are typically Specific, Measurable, Achievable, Results orientated (written as something to be achieved) and Time bound.

In order to achieve objectives a process needs to be identified (step a, step b, step c, etc.)

Performance and activity information is about knowing if we are achieving our objectives, what do we know about what we are doing, and are we carrying out our strategies and to what standard?

Bullen 1991

http://www.mapl.com.au/A1A.htm

Other Quality Indicators - Benchmarks.

Benchmarks are standards used to make comparisons. Broadly they are:

Internal – between areas to identify best practice.External Competitive – between competitors.External Collaborative – comparisons between peers.External Trans-industry – comparison with best in class regardless of focus.

And there are several methodologies:

Ideal “Gold Standards” – assessment against idealised best practice.Activity Based – representative activities compared between providers.Vertical – quantification of workloads, productivity and performance.Horizontal – analysing a process of workloads, productivity and performance.