Upload
vokien
View
223
Download
0
Embed Size (px)
Citation preview
Topic 2 Creating HEI Performance
Indicators Framework
Indicators for measuring higher education performance
September 8, 2016
Beirut, Lebanon, Dr. prof. Tatjana Volkova
Indicators for measuring higher education performance1
National Erasmus + Office
Lebanon
Higher Education Reform
Experts
In collaboration with the Issam Fares Institute and the
Office of Institutional Research and Assessment
Ministry of Education and Higher Education
Directorate General of Higher Education
Agenda
1. PM Frameworks practices
2. Strategic PM framework and its connection with Quality
Management System
3. Barriers and enablers to the effective use of performance
frameworks
Indicators for measuring higher education performance2
1. Critical review of PI Frameworks
There are no commonly agreed PI
Framework;
It varies across HEIs depending on the
context (e.g. legislation requirements),
mission of the HEI, stage of development,
on the level of development; size of HEIs,
priorities of development, type of HEIs
(research intense, teaching intense),
leadership vision, etc.
It is up to HEIs to select areas of
strategic importance;
http://www.bersin.com/uploadedImages/Starter_Apps/Wiki/Lexicon/PM%20Maturity%20Model.jpg
Indicators for measuring higher education performance3
The primary purpose of the Key Performance Indicator
(KPI) Framework
is to support management decision making within the University.
“…. it seeks to provide Council, the Executive and senior management with informative insights into how UTS is performing as a whole against its strategic planning framework and how it is likely to perform in the future. UTS evaluates its performance against the KPIs biannually”
http://www.pqu.uts.edu.au/tracking-performance/key-performance-indicators.html
http://www.pqu.uts.edu.au/tracking-performance/key-performance-indicators.html
Indicators for measuring higher education performance4
Focus on Results and Creating Value:
An organization’s performance
measurements need to focus on key
results;
Results should be used to create
and balance value for students
and for key stakeholders—the
community, employers, faculty and
staff, suppliers and partners, and the
public thus contributing to improving
overall performance and builds loyalty.
Indicators for measuring higher education performance5
http://www.pqu.uts.edu.au/tracking-performance/_images/kpi-framework-2013-1.jpgIndicators for measuring higher education performance6
Indicators for measuring higher education performance7
htt
p://1
7986-p
ress
cdn-0
-31.p
agely
.netd
na-
cdn.c
om
/wp-
conte
nt/
uplo
ads/
site
s/2/2
016/0
7/c
urt
in-2
015-a
nnual
-report
-
kpis
Indicators for measuring higher education performance8
http://researchanalytics.thomsonreuters.com/m/pdfs/higher-ed-exec-
report.pdf
Indicators for measuring higher education performance9
Indicators for measuring higher education performance10
Indicators for measuring higher education performance11
http
s://ww
w.n
ottin
gham
.ac.u
k/h
r/guid
esan
dsu
pport/p
erfo
rman
ceatw
ork
/pdpr/d
ocu
ments/p
dprfr
amew
ork
Indicators for measuring higher education performance12
https://www.newcastle.edu.au/?a=185090
The Performance Expectations
Frameworks for Academic and
Professional Staff have been developed to
support the University's vision and
strategic direction. The aim of these
Frameworks is to clearly align staff
performance with the University's goal of
delivering excellence in research and
teaching.
Indicators for measuring higher education performance13
Findings reveal that the most frequently used
KPI include:
a specialized student
survey (used by 94% of
institutions);
overall retention (91%),
employee satisfaction
(85%) and
graduation rate (79%). https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcSZQmvPnKKX6F2LP4VWqCfNEA951-
Qdlqq_2p9efrXyNpM4-eOe
Indicators for measuring higher education performance14
The least frequently used KPI include
adult learner measures
(15%),
completion rate (9%),
diversity measures (9%),
and
tuition as compared to
peers (6%).
http://4.bp.blogspot.com/-
v_oY4GDl6zQ/Tc3QqTlPRuI/AAAAAAAAAAM/0RuAJSy8nNc/s1600/adult%252520learners.jpg
Indicators for measuring higher education performance15
9 categories are suggested by researchers to assign PI
related to academic performance (Paul J. Ballard)
1. Helping Students Learn
2. Accomplishing Other Distinctive Objectives
3. Understanding Students’ and Other Stakeholders’ Needs,
4. Valuing People
5. Leading and Communicating
6. Supporting Institutional Operations
7. Measuring Effectiveness
8. Planning and Continuous Improvement
9. Building Collaborative Partnerships
Indicators for measuring higher education performance16
Indicators for measuring higher education performance17
Indicators for measuring higher education performance18
Indicators for measuring higher education performance19
2. Strategic Performance management framework
What we are going to measure?
Why we need to measure?
Are those areas of
measurement are relevant to
particular HEI?
What kind of indicators to
select?
Are those indicators relevant?
http://www.studentpulse.com/imgs/640/scale/article-images/uid-140-1014000958-3649/a673b3.jpg
Indicators for measuring higher education performance20
Strategic PM
Takes a long-term view of HEIs
directions of development :
To select the strategic directions of
development in order to spot new
strategic choices;
To identify the best approaches for
setting goals;
To identify the winning approaches
that successful peers use and apply
them to their own strategic process.
http://www.dumblittleman.com/wp-content/uploads/2014/04/Decision-Making-Skills.jpg
Indicators for measuring higher education performance21
Indicators for measuring higher education performance22
In spite on diverse practices there are 4 main
strategic areas for performance consideration:
financial sustainability,
services delivered to
stakeholders,
quality of processes,
development of HR and
growth.
Indicators for measuring higher education performance24
STRATEGIC PM framework for HEI: Balanced approach
What we are going to achieve? Setting strategic directions: Vision, Mission, Values and Goals:
Financial perspective (financialsustainability)
Services to society (Teaching& Learning, Research, Researchtransfer, Service to society (incl. LLL, regional engagement)
Internal Processes perspective
Learning and Growth perspective
Indicators for measuring higher education performance25
Strategic performance framework development
1. Before you start the process of finding KPIs, you must acquire a very goodunderstanding of strategic directions of HEI (vision, mission, values and goals).
2. Once you have determined your strategic goals, you will select KPIs for each of these goals.
Indicators for measuring higher education performance26
If there are to many KPI, it will be difficult to manage and
measure.
So it needs to select the most important KPIs that have significant
contribution to HEI performance.
In order to class this list in descending order of relevance, the survey could
be conducted by involving parties who are directly involved in HEI activities.
In this case, a questionnaire, in which the responsible persons have to give a
mark to each criteria, could be distributed.
The three-point scale of “not important”, “somewhat important” and “very
important” could be used;
Its result is the selected KPIs according to its degree of importance.
Indicators for measuring higher education performance27
Setting targets and assigning KPIs (sample)
Indicators for measuring higher education performance
Financial perspective
Aim Strategic target Measurement KPI
To ensure financial
stability
To increase revenue
To decrease costs
To increase income from
tuition fees
To outsource cleaning
services
Increase by 20 % in
2017
Decrease by 5 % in
2017
28
Samples: performance indicators for ensuring
Financial sustainability
operating income ratio,
tuition fees from international students;
resources per student,
resources per faculty,
research income;
debt burden ratio,
viability ratio,
service expenses, etc..
Indicators for measuring higher education performance29
Setting targets and assigning KPIs (sample)
Indicators for measuring higher education performance
Research activities
Aim Strategic target Measurement KPI
To ensure industry
engagement
To increase industry
research income
Patenting income
Licensing income
Spin-out generation income
Consultation income
Increase by X % in
2017 (each area
needs to be
specified)
30
Tool for selecting appropriate PI
Indicators for measuring higher education performance31
The survey by involving parties who are directly involved in HEIs activities
would be helpful to select relevant indicators;
A questionnaire in which the responsible persons have to give a mark to each
metrics could be designed and distributed;
The 3 point scale of “not important” , “ somehow important” and “very
important” could be used;
As the result those with highest degree of importance could be selected;
The survey of key stakeholders on KPIs focus could be
conducted
http://www.nousgroup.com/images/news_attachments/COAG_Reform_Council_-
_performance_framework_research_project_140411.pdfIndicators for measuring higher education performance32
Strategic PM framework connection with Quality
Management System
PM is a tool to enhance
Quality of HEIs activities;
There is a clear connection
between PM and QM;
PM is aiming at monitoring
the quality results achieved by
delivering HEIs activities;
http://www.newdirectionsconsulting.com/wp-content/uploads/2012/08/ScientistQualityHabit.gif
Indicators for measuring higher education performance35
htt
p://w
ww
2.w
arw
ick.a
c.uk/fac
/sci
/wm
g/ft
msc
/cours
es/
mbe-
hom
e/m
odule
s/efq
m/e
fqm
_m
odel.j
pg
Indicators for measuring higher education performance36
Total Quality Management system
Indicators for measuring higher education performance37
https://www.elsevier.com/__data/assets/image/0005/33359/Snowball-Metrics-landscape.jpgIndicators for measuring higher education performance38
Research performance framework
Research inputs
• Research grants: research applications, research awards;
• Enterprise activities/ economic development: Industrial
income: industry engagement
Post-graduate education
Contributors: People: researchers and research assistants
Research process
• Research income
• Industry research income
• Post-graduate experience: facilities
• Organizations
Research output and outcomes
• Publications & citations, collaboration
• Patenting, licensing income, spin-out generation
• Completion rates
• Themes/schemes
Indicators for measuring higher education performance39
3. Barriers and enablers to the effective use of performance
frameworks
challenge to have all data needs met;
small sample sizes can make it difficult to demonstrate improvements or progress with any statistical certainty;
promoting a culture of evidence-based decision-making, while laudable, could also be a constraining factor;
for the success of major longer term reforms to develop a more sophisticated understanding of what the medium-term targets ought to be;
dilemma of simplicity and complexity;
Indicators for measuring higher education performance41
Enablers to the effective use of performance frameworks
Leadership awareness and communication platform;
PM system in place;
performance information that would be publically shared in an accessible format;
high performance have to be rewarded;
performance management culture;
benchmarking;
PM as ongoing process..
http://wpmu.mah.se/comdev/files/2011/05/WB-cartoon-page-4.jpgIndicators for measuring higher education performance42
Enablers to the effective use of performance frameworks
(cont)
Indicators for measuring higher education performance43
the potential within the public sector to benchmark universities is
almost certain to be considered by governments and funding bodies;
….it is not impossible that the development might occur of compulsory
institutional participation in benchmarking;
Although such an approach might be resisted by both benchmarking
specialists and many of the universities concerned, such a compulsory
approach would be consistent with the imposition of other quality
management and assurance mechanisms in many higher education
systems.
http://science.uniserve.edu.au/courses/benchmarking/benchmarking_manual.pdfIndicators for measuring higher education performance44
Types and ways of benchmarking
Indicators for measuring higher education performance45
There are 3 major types of
benchmarking:
Strategic
Performance
Process
BENCHMARKING TYPES
Indicators for measuring higher education performance46
• Long term view
• Identifying and applyingbest approaches to setstrategic directions
StrategicBenchmark
• Selecting areas of measurement
• Identifying indicators of measurement
• Measuring achievementsfor improvement
PerformanceBenchmark • Finding and application
of best practises in keymanagement processes
Process Benchmark
BENCHMARKING TYPES
Indicators for measuring higher education performance47
Determining what University is doing strategically is sometimes easier than trying to
learn how they deliver individual procedures.
Indicators for measuring higher education performance48
Strategic directions can be found in
annual reports, often publicly
available.
Selecting a partner University to
study makes it easier to approach
peers and ask for advice and
information on how organization
built its success.
Source: https://static.securityintelligence.com/uploads/2015/07/Security-Decision-Making-938x535.jpg
Performance BM
Indicators for measuring higher education performance49
In the selection of indicators care was taken to check whether they:
• measure what they claim to measure;
• focus on the performance of programmes or institutions;
• are defined so as to measure “relative” characteristics and thus avoid
dependence of institutional size;
• are already known as a result of benchmarking or ranking exercises;
• provide comparisons from one situation, system or location to another;
• draw on data which is available in existing databases or at higher education
and research institutions,
or which can be collected without too much difficulty (Federkeil et al., 2012c).
Ways for benchmarking
Indicators for measuring higher education performance50
It is important to choose the optimal
approach as it reduces the costs of
the activity and improves the chances
to find the ‘best practices’ you can rely
on
Internal/external
Functional
Generic
Competitive/ collaborative
Internal BM
Indicators for measuring higher education performance51
comparisons are made of the performance of different departments, campuses
or sites within a university
to identify best practice in the institution, without necessarily having an
external standard against which to compare the results;
This type may be particularly appropriate to universities where a high degree of
devolvement exists to the constituent parts of the institution, where a multi-
campus environment exists, or where extensive franchise arrangements exist
whereby standard programs are taught by a number of partner colleges in
different locations.
External (competitive) BM
Indicators for measuring higher education performance52
Comparisons of performance in key
areas is based upon information
from institutions which are seen as
“best in the class” or «competitors»
May be run by the institutions
themselves on a collective basis;
Consultant may be invited in order
to ensure continuity and sufficient
momentum
Higher possibilities of finding the
best practices
Functional BM
Indicators for measuring higher education performance53
Managers of functional departments
find it useful to analyze how well their
functional area performs compared to
functional areas of other Universities in
a selected number of activities;
It is quite easy to identify the best
strategic management, marketing,
finance, human resource, real estate,
library, research, internationalization or
IT departments in other Universities
that excel in what they do and to apply
their practices to your own functional
area http://www.wiesbadener-kurier.de/fm/819/thumbnails/CON_513989848_54831_M.jpg.21626268.jpg
Indicators for measuring higher education performance54
Estate
Indicators for measuring higher education performance55
AUDE (Association of University
Directors of Estates) states that it
promotes excellence in the strategic
planning, management, operation and
development of Higher Education
estates and facilities
(http://www.aude.ac.uk/home).
AUDE is critically involved in the Estate
Management Statistics (EMS) collection,
now transferring to HESA from IPD
Occupiers; a technical implementation
group has been set up and a member of
the AUDE Executive Committee sits on
the HESA HEI User Group.
In the area of estates and facilities, it is
common to benchmark against
commercial sector standards
https://www.hesa.ac.uk/benchmarking_report_nov10
http://www.uky.edu/iraa/content/tuition-and-fees-benchmarkcomparisonsIndicators for measuring higher education performance56
IT Benchmarking: Case of University of Westminster (UK)
Indicators for measuring higher education performance57
Delivered strategic IT outsourcing
to GOOGLE;
Implemented GOOGLE Apps
accross the whole University
Saved ~1 million pounds to date on
ICT costs
Improved email storage capabilities
Universities UK Efficiency and effectiveness in higher education, 2011, p.55
Functional benchmarking: Research management and
knowledge transfer
Indicators for measuring higher education performance58
Benchmarking research activity is widespread, undertaken at the level of
inputs, outputs and grant awards.
Full economic costing has also provided a potential resource of comparable
data
Some HEIs have Research Management Systems to monitor and manage their
research activity, thus allowing for available management information.
The research‐intensive HEIs share application and award data at institutional
level on a collaborative and confidential basis
Indicators for measuring higher education performance59
Generic benchmarking
Indicators for measuring higher education performance60
Comparing your University to some generic (or ‘gold’) standards, which
would indicate that your University is a learning organization, you would be
using generic benchmarking
‘Implicit benchmarking’ is likely to increase in future years as governments and
central funding agencies seek to apply benchmarking approaches to
universities. F. e. Research excellence initiative in Germany, UK
Benchmarks for Effective University Administration
Indicators for measuring higher education performance61
There is some evidence that:
successful BM is much more likely to take place in those organizations that are already relatively well managed and with a strong emphasis on continuous improvement philosophy and innovation;
are likely to be less effective in individualistic or centralized cultures where the long term commitment for benchmarking is unlikely to be found.
http://www.universitiesuk.ac.uk/highereducation/Documents/2011/EfficiencyinHigherEducation.pdf
Indicators for measuring higher education performance62
«.. the development of a common
framework for efficiency benchmarking
in higher education should also seek to
provide guidance on the link between
costs and outputs, so that institutions
can measure the services delivered
against the costs of delivery»
Source: Efficiency and effectiveness in higher education Universities UK
Benchmarks for Effective University Administration
Indicators for measuring higher education performance63
Not only are organizational cultures and structures important in determining
the appropriateness of benchmarking and related quality approaches.
The meanings attached to benchmarking processes, the organizational
assumptions on which they rest, and the consequent success of any
benchmarking initiatives are likely to be significantly affected by the
social culture of the university system concerned
Developing performance management culture
Indicators for measuring higher education performance64
However, developing a positive PM
culture that adds value will not
happen overnight rather it will evolve
over time to fit an organizations
requirements and to be accepted
and embraced by all employees.
http://pwc.blogs.com/milton-keynes/2016/01/unlocking-the-
value-of-performance-management-in-higher-education.html
http://www.michaelbarata.com/wp-content/uploads/2013/03/communication-.jpg
Building PM culture
Indicators for measuring higher education performance65