Higher Education 14 (1985) 235 239 Elsevier Science Publishers B.V., Amsterdam Printed in the Netherlands

235

EFFECTIVENESS AND EFFICIENCY IN H I G H E R E D U C A T I O N

J O H N COWAN Heriot- Watt University, Riccarton, Edinburgh EHI4 4A5, United Kingdom

ABSTRACT

The article begins from the basic concepts of efficiency and effectiveness in higher education. The author defines these terms, and their potential use, through a range of simple examples. Thereafter he develops an argument to suggest that it is important to distinguish between effectiveness and efficiency, and the measure for components of educational processes rather than attempt to identify global values which are descriptive of courses or departments.

I should begin this article by a frank admission that it is an account of a plain man's attempts to make sense of educational economics, starting from the very beginning. It sets out that thinking, step by step, with simple examples which helped me to clarify each step in my argument. That argument leads me to the conclusion that there is something very wrong with our present approach to the evaluation of higher education, in economic terms. And at that point, having identified the problem and suggested a tentative solution, my argument a n d m y article stop abruptly. For this statement is only a summary of the thinking of an engineer about important principles which are properly the domain of the economist. The engineer may be able to recognise and explain that we are on the wrong track; but he has neither the expertise nor the impertinence to develop an optional economic strategy in detail.

The heart of my concern is that, in educational discussions, the concepts of effectiveness and efficiency are often discussed as if they were synonymous. Yet, as an engineer, I know that they have different meanings - and that each is useful in a different way in describing performance. I am grateful to Alan Harding, Bradford University, for reminding me of the distinction - frequently - until eventually I began to think about what it might mean for me, in the context of my teaching.

Educational institutions tend to measure performance in global terms

0018-1560/85/$ 03.30 �9 1985 Elsevier Science Publishers B.V.

236

according to the cost per student, the student/staff ratio in a department, or the failure rate in a given course. I have long been dissatisfied with that approach, despite the fact that it is seldom questioned and generally deemed respectable in the United Kingdom. My confused worries were crystallised during a discussion with Senora Julia Gonzalez de Cruz, at the Centre for University Development at the University of Yucatan in Merida. The eventual outcome was this article.

ARGUMENT

Efficiency and effectiveness are both ratios; they each express different aspects of a given process.

Efficiency is the ratio of output to input; effectiveness is the ratio of the actual outcome to the possible or ideal outcome.

Examples: The efficiency of a particular transformer is 85%; it has an output of 85 units

of electrical energy for every 100 units o f input energy.

I f the effectiveness of a sales campaign is 45%, then, o f every 100 possible customers, 45 have been persuaded to purchase.

Both ratios may be expressed in a form which uses the same units through- out; in that case the value can be quoted as a decimal fraction or as a percentage. Since the ratio is then an expression of the contrast between the first and second values, there are obvious advantages in adopting a single basic unit of measure- ment. For the ratio then demonstrates the contrast in a familiar and immediately comprehensible way.

Examples: The efficiency of 85% indicates that only 15% of the input has been used

unproductively.

An effectiveness of 45% indicates that 55% of the potential has not been realised.

But there is no objection in principle to the use of different units - and that is often done, for valid reasons or as a matter of convenience, when figures of efficiency are quoted.

Example: A car engine could be said to have an efficiency of 50 milesper gallon at a

constant speed of 60 mph on a level track.

It is not, however, acceptable to summate components of the quantities in

237

the ratios, if the components are themselves expressed in differing units; for the sum thus obtained would not then be meaningful unless it was restated in a new unit of a less specific nature.

Example: 10 (apples) + 2 (oranges) only equals 12 (objects or pieces o f fruit); other-

wise the summation is impossible or meaningless.

The output o f a turbine cannot be measured by directly adding the kilowatt hours taken by an electric motor driven through a generator and the cubic metres o f water pumped mechanically by the turbine into an overhead tank.

If higher education is regarded as a process (with an as yet undefined product), the terms efficiency and effectiveness can be used in the same way, and subject to the same conditions. The immediate problem is to ascribe meanings to the quantities which are compared in the ratios.

Examples: The efficiency o f teaching could be regarded as the ratio of( the learning

which takes place as a result o f the teaching) to (the effort devoted to that teaching).

The effectiveness o f a lecture could be regarded as the ratio of(the summa- tion o f all the learning within the class group) to (the summation o f the instruc- tional aims for that class group, in that particular period o f instruction).

Notice immediately that problems arise educationally in specifying the units in which these quantities are to be expressed, and of course - in proposing valid ways of measuring the quantities involved. Notice also that, as the magnitude or complexity of the "process" increases, so does the summation of components become increasingly difficult. For it is then more and more likely that they will be measured in diverse units.

Examples: The outputs o f higher education could be said to include

- the total amount o f useful teaching (in all domains and at all hierarchical levels)

- the total amount o f useful learning (again in all domains, and at all hierarchi- cal levels)

- the increased ability o f graduates to meet educational, industrial or social needs

238

The inputs could be said to include - the attitudes, experience and competence o f the academic staff

the effort devoted to teaching, preparation and updating - the material and human resources other than the academic staff - the study effort by the students.

Not only is it difficult to suggest common units for the first and second quantities in our ratios: it is now also difficult to think of suitable units for any one of the above components of input or output.

Example: In what units could we measure and meaningfully summate learning which

includes understanding o f basic concepts, the development o f evaluative ability, a critical change in attitude and the acquisition o f certain manual skills?

One possible means of resolving this dilemma is to work with components of the process of higher education, rather than with the whole entity.

Example: In a lecture and tutorial devoted to the mastery o f the concept of"entropy",

it would be relatively straightforward to quantify all the teaching effort (includ- ing an allowance for overheads) in terms o f costs.

It would equally be possible to analyse the aspects o f entropy to be mas- tered, to weight them in an objective manner, and to devise a pair o f matched tests f rom which the gain ratio could be derived.

Gain ratio = Post-test score -pretest score

100% - Pretest score

That gain ratio could be acceptable as a quantification o f learning, in that particular topic. Thus efficiency could be measured and expressed in terms o f percentage gain ratio per unit cost.

The expedient of studying effectivenesses and efficiencies for parts of the process is much less than a total solution to the problem. But it offers a number of distinct advantages over the status quo: 1. One method of teaching, or one department, can be compared directly with

another. 2. Issues which require to be resolved are pinpointed. 3. Effectiveness and efficiency ratios for small components of higher educa-

tion are defined and measured in a meaningful form.

239

4. A greater emphasis than hitherto is placed on efficiency (which has seldom been properly explored in higher education) and relatively less emphasis on effectiveness (which has proved significantly similar in so many compari- sons of teaching methods).

5. The need for meaningful dialogue between society and education, to define the relevance and standards which are desired, is stressed. Without that prescribed target, it would be open to any institutions to be both efficient and effective - by setting themselves low targets!

6. It will be much easier to pinpoint the sources of inefficiency or ineffective: ness through the preparation of component figures for particular parts of the process.

Examples: I f the effectiveness o f teaching structural mechanics were to be monitored in

this way, I could 1. Compare resource-based learning with conventional methods o f teaching. 2. Enter into a purposeful debate with my colleagues, and with the profession,

about the relative importance o f different types o f objectives in this and associated courses.

3. Build on this work as a basis for developing overall figures for course effectiveness, and comparing that with a target profile.

4. Gain credit for this innovative method o f teaching if I achieve the same learning as elsewhere with less input.

5. Insist that accreditation o f courses, and discussions authorising funding, scrutinise and evaluate standards and relevance according to declared criteria.

6. Identify aspects o f the resource-based course in which there is room for improvement, and quantify my expectations o f the improvement which should be generated.

It seems to me that all of this would be a marked advance on the present systems of economic budgeting and monitoring in higher education. For there is little (save extreme simplicity) to commend the unquestioning use of student/ staff ratios, total resource allocations per student - or even class contact hours. In each case these global figures are compared with a subjectively decided norm which takes no defensible account of the differences between disciplines, courses, standards, intake quality or components of courses. Each of these so-called measures is calculated by adding quantities which are, explicitly or implicitiy, expressed in different units. They therefore aggregate to a meaningless total, which is then to be compared with a norm of dubious origin. Surely we would do better by making a determined effort to place our educational book-keeping on a business-like basis, step-by-step, and component by component?