School Roll Forecasts: Their Uses, Their Accuracy and Educational Reform

School Roll Forecasts: Their Uses, Their Accuracy and Educational ReformAuthor(s): Stephen SimpsonSource: Journal of the Royal Statistical Society. Series A (Statistics in Society), Vol. 152, No. 3(1989), pp. 287-304Published by: Wiley for the Royal Statistical SocietyStable URL: http://www.jstor.org/stable/2983127 .

Accessed: 24/06/2014 20:14

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact [email protected].

.

Wiley and Royal Statistical Society are collaborating with JSTOR to digitize, preserve and extend access toJournal of the Royal Statistical Society. Series A (Statistics in Society).

http://www.jstor.org

This content downloaded from 195.34.78.121 on Tue, 24 Jun 2014 20:14:32 PMAll use subject to JSTOR Terms and Conditions

http://www.jstor.org/action/showPublisher?publisherCode=black

http://www.jstor.org/action/showPublisher?publisherCode=rss

http://www.jstor.org/stable/2983127?origin=JSTOR-pdf

http://www.jstor.org/page/info/about/policies/terms.jsp


J. R. Statist. Soc. A (1989) 152, Part 3, pp. 287-304

School Roll Forecasts: their Uses, their Accuracy and Educational Reform

By STEPHEN SIMPSONt

Calderdale Metropolitan Borough Council, Halifax, UK

[Received June 1988. Revised December 1988]

SUMMARY Recent research has indicated the isolated development of school roll forecasts in each local education authority of Britain, each using similar methodology but with wide variations in data intensiveness, resources, documentation and responsibility for production of forecasts. This paper reviews the nature of this variation and the reasons for it. The current restructuring of schooling in Britain directly limits local scope to plan educational provision; the changing role of forecasting is reviewed. The paper develops measures of forecast accuracy that are related to the costs of inaccurate forecasts and applies these measures to data from a variety of local education authorities. The accuracy of complex and more simple models is compared. The accuracy of headteachers' estimates of their future school rolls is compared with centrally prepared estimates.

Keywords: ACCURACY OF FORECASTS; EDUCATIONAL PLANNING; FORECAST METHODOLOGY; LOCAL GOVERNMENT; SCHOOL ROLL FORECASTS

1. INTRODUCTION

Forecasts of pupil numbers are calculated annually by each of the 120 local authorities in Britain responsible for provision of education to children of statutory school ages: metropolitan districts and county councils of England and Wales, regional councils of Scotland and the local authorities of the Channel Islands and Isle of Man. These are all referred to here as local education authorities (LEAs).

These forecasts have two broad purposes.

(a) Forward planning: to foresee when educational structures will have outlived the conditions which gave rise to them, in sufficient time to do something about it. School reorganizations provide the best recognized policy issue for which forecasts are used in forward planning of educational facilities.

(b) Administration of resources: routinely to distribute resources between areas and schools equitably, in sufficient time for schools to use those resources efficiently. The teaching staff and other resources from the LEA revenue budget are commonly distributed earlier than the start of the school year in which they are to be used, on the basis of expected pupil numbers and other criteria. School capacity can also be adjusted according to forecasted school rolls to implement agreed educational policy.

Both for forward planning and for routine distribution of resources LEAs require forecasts of pupil numbers in each single age group in each local area of their authority. For the second purpose of distributing resources, short-term forecasts are

tAddress for correspondence: Chief Executive's Strategy Section, Calderdale Metropolitan Borough Council, Town Hall, Halifax, HX1 1UJ, UK.

? 1 989 Royal Statistical Society 0035-9238/89/1 52287 $2.00



288 SIMPSON [Part 3,

required for each individual school. Thus the forecasts are commonly multiregional cohort survival models based on a notion of geographical catchment areas to each school, in which the number of pupils in each year group is carried forward a year at a time to provide expected pupil numbers in each of the coming years.

There are three critical transition points in the model:

(a) joining school, when some estimate of children under 5 years old about to enter each primary school has to be made;

(b) transfer between schools, when parental choice of secondary schools is increas- ingly important;

(c) leaving school, when future rates of staying on to sixth form and other local tertiary education must be predicted.

This method is straightforward and similar in each LEA. The staff resources devoted to calculating school roll forecasts vary greatly, from less than a week annually by an LEA officer with no research training, to more than 6 months' effort by statistically qualified officers within relatively high powered research units. Such research units tend to use considerable computer resources. The relationship between the originator of forecasts and the nature of the forecast is explored in Section 2.

The level of sophistication in methodological refinements to the basic cohort survival model also varies greatly. Methodological issues include the estimation of migration due to new housing and to changes in use of the existing housing stock, the use of birth statistics and health authority infant registers to estimate intake to primary school, and the measurement of parental choice of school. Such methodological issues and others are surveyed in Jenkins and Walker (1985) and Simpson and Lancaster (1987). The material in this paper is an extension of the latter work. A review of the limited previous literature appears in Simpson (1987).

Current government proposals to restructure the provision of school education have major implications for the use of school roll forecasts. It is clear that within a restruc- tured school system the role of planning in educational provision will become more limited than its potential in past decades. However, comprehensive information on the future trends of school rolls will remain essential for the administration of an LEA's schools. It will also be important to maintain an LEA role in representing the needs of the local population as a whole. For this, the past, present and future trends of school population remain central, and monitoring the social differentials in access to education are more relevant than in the past. These questions are discussed in Section 3.

Forecasts of pupil numbers are used to determine provision for schooling before actual numbers are known because staffing and accommodation, for examples, have to be appointed or commissioned before the year in which they are to be employed. The accuracy of forecasts is therefore an important issue: errors in the forecasts mean that school provision is not according to policy. Since a quarter of all local government expenditure is on schools, a marginal improvement in accuracy of forecasts can mean not only a better implementation of policy but a significant release of resources for other educational purposes. Knowledge of regular inaccuracies in forecasts allows the provision of approximate confidence intervals for the guidance of forecast users, a degree of approximation arising since unexplained errors in the past are not a random sample from the errors in a particular LEA's forecasts.

Accuracy alone does not produce the best forecasts. These also require careful presentation to earn a place in the educational planning process. But accuracy



1989] SCHOOL ROLL FORECASTS 289

remains a primary aim of forecasters and one important means of distinguishing the results of different approaches to forecasting. Measures of accuracy and their application to some recent LEA forecasts are presented in Section 4.

2. CURRENT ORGANIZATION OF FORECASTING IN LOCAL EDUCATION AUTHORITIES IN BRITAIN

Table 1 illustrates some results from a survey of 120 LEAs (Simpson and Lancaster, 1987) to indicate who prepares school roll forecasts and the nature of those forecasts.

A little more than half of LEAs use computers in the calculation of forecasts. Although the calculations are quite simple, hundreds of schools are involved in each LEA and the model is suited to spreadsheet packages available in almost all local authorities. The need to control output is one factor that has limited computer use. Educational administrators who are keen to use the forecast results immediately they are produced are less likely to have access to computer equipment and programming skills, and may not be satisfied with the presentation of output designed by others. However, computer use for forecasts has increased in LEAs in recent years, partly impelled by the computerization of administrative data such as pupil records, school transfers and school roll counts (Streatfield and Jones, 1987).

Only a quarter of LEAs forecast further than known births allow, i.e. 4 or 5 years for primary schools, 11 years for schools with intake at 11 years old and so on. A small but significant number do not regularly forecast more than 1 year ahead at a time,

TABLE 1 School roll forecasting organization in LEAs, 1986t

No. of Computer Future Range of Heads Report on LEAs assisted births I year involved methods

(%) estimated only (%) (%) available (%) (%)

All LEAs 108 55 27 9 50 31 (100%)

Schools 26 38 12 12 71 21 administration (24%) officers

Other 27 30 21 21 46 23 educational (25%) administration officers

Education 42 71 29 0 49 42 research staff (39%)

Central or 12 83 67 8 27 33 planning department (11%) research staff

Computer division 1 (1%)

tThe survey gained responses from 114 of the 120 LEAs of Britain. Excluded from the table are the Channel Islands, the Isles of Scilly and the Isle of Man, and three London boroughs claiming to forecast only the LEA total of pupils. Percentages in the body of the table take into account occasional non-response on particular questions.




for distributing the following year's teaching staff. In many cases, LEAs do not investigate long-term trends because of doubt about future fertility. How- ever, the small cohort born in the mid-1970s will almost certainly imply a drop in primary school age pupils after a peak at the turn of the century, even if fertility rates rise as statisticians at the Office of Population Censuses and Surveys currently assume. Knowledge of long-term trends of this nature can be invaluable to local planning, providing a rationale for targeting maximum capacities.

Headteachers can provide local knowledge that is the most useful ancillary information to centrally held databases, for short-term forecasts of intake and choice of educational path at age 16. Many authorities consult headteachers when preparing forecasts and at the very least formally allow a challenge to centrally prepared forecasts before their use in distributing resources. However, half of the LEAs take no account of headteachers' knowledge of their local area, and this proportion rises to three-quarters among LEAs where forecasts are commissioned from research staff outside the education department.

Perhaps most importantly of all, less than a third of LEAs can provide a written report of the methodology used in forecasting pupil numbers. Even including LEAs where a 'brief' description is available (not shown in Table 1), the figure does not reach 50% of all LEAs. Many vital decisions are proposed and taken without an opportunity for interested parties to try out different assumptions in the forecasts and to derive alternative proposals for educational policy. Interestingly, the research units outside education departments, often having greater research resources, do not produce methodological reports significantly more often than others, at least as far as school roll forecasting is concerned.

3. RESTRUCTURING OF SCHOOL EDUCATION

Since the Education Act of 1944 until the beginning of the 1980s the framework for local education policy has been to provide school education for all children in the LEA area. Private and other non-LEA schools have been a small and constant minority in almost every LEA. Finance for LEAs has been such as to allow planning of capital expenditure according to medium- and long-term need. The legal rights of LEAs have permitted school admissions on the basis of geographical catchment areas providing community or 'neighbourhood' primary schools and, more recently, community secondary schooling.

Forecasting pupil numbers on a demographic basis for each school's catchment area has been both appropriate and capable of fulfilling the needs of educational planning and administration for regular estimates, sufficiently accurate to allow both short- and medium-term planning of expenditure directed to pupil needs.

However, the government is radically restructuring the provision of schooling by means of several reforms during the 1980s culminating in the major 1988 Education Reform Act (Her Majesty's Government, 1988) which was passed with little change from the original white paper in spite of the debate which it aroused (Simon, 1988). Each of the three main aspects of structural reform- opting out, open enrolment and local management of schools-together with changes in capital expenditure already in operation, lead away from need-based planning to administration of demand for pupil places as expressed individually by parents.




3.1. Opting Out All secondary schools and those primary schools with more than 300 pupils may,

on a simple majority of voting parents, opt out of LEA funding and be maintained by direct grant from the Department of Education and Science. A significant sector of local schools outside the LEA's information base but drawing on the pupil population in an unpredictable manner would make forecasting for either school sector extremely difficult. The government expects many schools to opt out by the mid-1990s and a special body to promote the development of grant-maintained schools began publicizing its work to all schools in 1988.

3.2. Open Enrolment The power of LEAs to fix a limit on each school's intake, used to plan the

contraction or expansion of a school system in a rational manner, is to be relaxed to very broad limits-the 1979 intake levels when secondary pupil numbers throughout the country were particularly high. The effects of relaxed admission controls on reducing parental choice by encouraging school closures have been quantified in Southampton by Mar Molinero (1988).

It is likely that non-demographic estimates of demand will become more widespread, including economic models and telephone surveys to estimate demand as in the USA (e.g. Cook (1985)). Geographically coded demographic data will none-the-less be needed to monitor the effects of open enrolment on access to education by comparing the number of pupils living in the school's neighbourhood with the number attending the school. Mar Molinero's work shows the emergence of schools whose intake is defined as much by the social background of parents as by geography. A child's receipt of free school meals provides a crude indicator of social class, one which is available in all areas.

3.3. Local Management of Schools Devolved financial responsibility for running individual schools (all secondary

schools and all primary schools with more than 200 pupils) reinforces the use of pupil numbers as an explicit criterion in the distribution of resources to schools. The precise formula for school budgets is decided locally, but a minimum of 75% of the total distributed budget must be divided according to age-weighted pupil numbers. The measures of school size, intake, sixth-form size and so on may be forecasts of pupil numbers as have usually been used to help to allocate teaching staff, or may be simply the number of pupils known at the time that the budget is set. The latter number has been considered in early experiments in Cambridgeshire, but it is a poorer measure of need than is a forecast. The discrepancy may be met by adjusting school budgets during the school year when actual pupil numbers are known, but such budget uncertainty adversely affects each school's ability to make best use of its resources. The use of forecasts in distribution of resources to schools is discussed further in Simpson (1988).

3.4. Capital Expenditure Rules already in operation (e.g. Department of Education and Science (1986))

prevent the commitment of capital expenditure by LEAs further than 3 years ahead.




Medium- and long-term forecasts of school rolls have thus become more difficult to translate into operational policies.

The influence on local educational statistics of the Education Reform Act is not one of simple reduction or increase but is more a changed focus; new statistics will emphasize a gradation of schools through common assessment of a core curriculum and publication of examination results, rather than demographic demand for a common standard of school. However, the vast majority of local educational statistics are generated by administration needs. Here the inputs to the financial formulae for local schools management, required for all LEA-maintained schools, may include school information that has not previously been brought together. Furthermore, although the Act does not demand that grant-maintained (opted-out) schools should provide information to the LEA, the LEA continues to be responsible for matters including capital expenditure and in-service teacher training that make a common database quite feasible.

Thus LEAs can, if their councillors wish, use the information at their disposal to monitor trends in school provision and the distribution of school age population. As school provision is seen to have a clearly graded quality, the question of equal access to education will come once more to the fore in educational debate.

4. ACCURACY OF SCHOOL ROLL FORECASTS

The benefits to an LEA from evaluating the inaccuracies of its past forecasts are considerable. They can be summarized as follows:

(a) system bias-identification of bias in the methods of forecasting, leading to an improvement in the overall methods;

(b) subsystem bias-identification of bias in the forecasts for particular schools or for subareas of the LEA, leading to improvement in future forecasts for those schools or subareas;

(c) confidence intervals-identification of the scale of inaccuracies to be regularly expected from forecasts, leading to (i) appropriate safety margins for users' consideration and

(ii) evaluation of current and alternative strategies for ensuring adequate resourcing of schools.

This section concentrates on defining measures of forecast inaccuracy related to the practical costs of inaccurate forecasts and on their application to a variety of recent LEA school roll forecasts. The effect of sophistication in forecasting method and the relative accuracy of headteachers' estimates and centrally prepared estimates are discussed. The application of knowledge of forecast errors to the administration of school resources, by evaluating various strategies for distributing resources based on estimates of future school rolls, is discussed in Simpson (1988).

Most LEAs have not been accustomed to assess their past forecasts in a systematic manner and therefore do not take advantage of the potential benefits for improved accuracy and use. Monitoring of past accuracy, when undertaken, is usually limited to a comparison of forecast and actual numbers for the total number of pupils in the LEA or in large subareas, with no bearing on the major use of forecasts, that of informing the distribution of resources between schools. This lack of self-evaluation is common to national population forecasts, but in business and the social sciences




surveys of forecast errors have suggested that simple methods often outperform complex methods and that combining forecasts from different methods frequently improves overall accuracy (e.g. Mahmoud (1984)). School roll forecasts offer ample scope for the measurement of accuracy because each year actual outcomes are observed through the count of school pupils maintained in school and LEA records. The data used in this section are those supplied by LEAs to the Forecasting Pupil Numbers Project and have not previously been published.

4.1. Costs of Inaccurate Forecasts Fig. 1 shows a typical school roll forecast. On the horizontal axis is the actual size

of the 132 primary schools in one district of an English county council. They vary from very small village infant schools of less than 20 pupils to large town primary schools of over 300 pupils. On the vertical axis is the school roll as forecast 1 year earlier. Clearly, the forecast is usually very close to the actual value.

However, there are some educationally significant errors shown, remembering that these 1-year forecasts would be used to set the staffing level of each school for the coming year, and perhaps the financial resources for a range of other necessities from pencils to natural science equipment. An underforecast of over 20 pupils may mean overcrowding when the school year starts. The LEA would usually then top up with a supply teacher, but the disruption to a school's internal planning caused by such a

400 +

360 +

320 +

FORECAST ONE

280 + YEAR PREVIOUSLY

28 */

240 +

200 +

160 +

1 20 + 2

I *3 4

I 3/

40 +

I */

10 + ACULSHOLRL

o0 * 80 1) (.0 200 240 280 320 360 400

Fig. 1. Forecasted pupil numbers and actual pupil numbers, for the 132 primary schools in LEA F (the roll in 1986-87, forecast 1 year previously)




iIncreasing

costs

NOTICEABLE NOTICEABLE INEQUALITY 0 INEQUALITY OF PROVISION PROVISION

LOSS OFI CONFIDENCE LOSS OF

CONFIDENCE

ISRUPTION 'LOST' OVERCROWDIN RESOURCES

No Under-forecasts Error Over-forecasts

Fig. 2. Costs of inaccuracy in school roll forecasts

late appointment is an educational cost of underforecasts. The larger overforecasts would signify that these schools had staff extra to their requirements according to council policy. Education authorities would not take away staff in such circumstances, so this is a financial cost of overforecasts, albeit balanced by an educational benefit of a lower pupil-staff ratio than expected; but the money might have been used elsewhere.

Fig. 2 summarizes the costs of overforecasts and underforecasts. In practice, there are extra costs associated with particularly large errors-loss of confidence in the planning process and significant inequality of provision-making the cost function parabolic. There is evidence (some of which is given in Section 4.4) that LEAs lay a high premium on every school's receiving at least the agreed resource level; the costs of underforecasts are thus relatively large and the cost function rises less steeply for overforecasts.

At this point it is worth noting that the only cost directly measurable in money terms is one relating to overforecasts and overprovision. A purely financial approach would indicate that forecasters should err on the underforecasts side, advice which is directly opposed to the educational premium on avoiding underforecasts and underprovision.

4.2. Measures of Accuracy Denham (1971) and Braden et al. (1972) have attempted to compute exact prior

confidence limits for school roll forecasts by Monte Carlo simulation and statistical distribution theory, but depended on arbitrary estimation of the probabilities that forecast assumptions themselves are correct.

It is more usual in studies of forecast accuracy (e.g. Armstrong (1978)) to apply a post hoc summary measure of accuracy such as the mean absolute deviation, mean- square error, explained R2 or other similar statistic. However, the discussion in




Section 4.1 of the costs of inaccuracies suggests that a single summary measure would not be appropriate to the use of school roll forecasts; it would confound the separate consequences of underforecasts and overforecasts. In the following applications, two measures are therefore used:

(a) the aggregate overforecast, or the aggregate 'excess' of pupils in all schools overforecast; for example, if five schools were overforecast by six pupils each, the aggregate overforecast is 30 pupils;

(b) the aggregate underforecast, or the aggregate 'shortfall' of pupils in all schools underforecast, defined in the same way.

These two measures may be expressed as a percentage of the LEA's total roll and can be monitored regularly without great expenditure of resources. From them, two other useful measures may be derived:

(a) the overall bias, or the aggregate overforecast minus the aggregate underforecast, the accuracy of the LEA total;

(b) the total error, or the aggregate overforecast plus the aggregate underforecast, the sum of all the errors whether overforecasts or underforecasts.

One might in addition monitor the number of schools with extreme errors (more than 10% of school roll or more than 20 pupils, say) and schools which had been over or underforecast for each of the past 3 years.

It may be questioned whether the actual school roll subsequently recorded can be fairly compared with a forecast since a forecast may have led the LEA to intervene with the very purpose of avoiding a forecast outcome, for example to maintain an equitable use of school accommodation. Similarly a forecast will normally apply to specific dates in the school year because the school roll varies as pupils join and leave; the greatest within-year variations are due to the intake of infants and to leavers at age 16. These issues are discussed in Simpson and Lancaster (1987), chapter 8, but do not affect the inferences drawn here in later sections.

4.3. Application to a Series of One Local Education Authority's Forecasts Fig. 3(a) shows these measures of forecast accuracy for the 60 primary schools of

LEA B, a metropolitan district, again for 1-year forecasts. Six sets of forecasts are shown, those made a year previously to each of the 6 years indicated.

The underforecasts in LEA B slightly increased over the 6 years, while the overforecasts decreased. The total error, around 4% of the aggregate LEA roll, decreased a little over these 6 years. However, because the educationally expensive underforecasts increased it is not clear that the forecasting improved during the period.

The overall bias, simply the aggregate overforecast minus the aggregate underforecast, was positive at the start of the period but virtually disappeared by 1986-87: this overall bias is all that LEA forecasters tend to examine at present. The elimination of the bias was due in this case to a fortuitous reduction in net in-migration to the district; the LEA had not included any adjustment for future migration in the methodology.

Figs 3(b)-3(e) apply the same assessment to the same LEA forecasts when neigh- bouring schools' results are grouped together, which is usual for planning school accommodation in each neighbourhood of the LEA. For these school groups, data




4_ Totel ~Errorsl Aggregate 4 errors as a percentage 3 O ver of the total forecasts LEA roll 2

a~~~~~~~~. . E . .. . ,. ... .. ..... .... ... . . .

8 1/82 82/83 83/84 84/85 85/86 86/87

(a)

3 X ~~Total Errors 3%I Aggregate [l

errorsasa 2%_ 2% I percentage Over- Total Errors ofthetotal 1 t -?vsnin ' j. , _

LEA roll Under-forec LEA roll T ...... .... ; " ...~~~~.... .... ...... ...... ...... o%~ ~ -~~~~~~ ~~0 l

81/82 82/83 83/84 84/85 85/86 86/87 81/82 82/83 83/84 84/85 85/86 86/87

(b) (c)

Aggregate Toa 5 %eo

errors as a 4 - Total errors _

percentage 4 %

of the total 3 X - ; _ ; % LEA roll forecast

2 2 % Total errors

1 Over _ .1% Overforecasts

1 yr 2 yrs 3 yrs 4 yrs I yr 2 yrs 3 yrs 4 yrs

(d) (e)

Fig. 3. Errors in the forecasts of primary school rolls in LEA B: (a) total school rolls (aggregate errors for the 60 primary schools, forecast 1 year before the year indicated); (b) total group rolls (aggregate errors for the 11 school groups, forecast 1 year before the year indicated); (c) group intakes (aggregate errors for the 11 school groups, forecast 1 year ahead of the year indicated); (d) total group rolls (aggregate errors for the 11 school groups, for forecasts of 1-4 years ahead; for each range the results displayed are the average of the forecast for 1985-86 and the forecast for 1986-87); (e) group intakes (aggregate errors for the 11 school groups, for forecasts of 1-4 years ahead; for each range the results displayed are the average of the forecast for 1985-86 and the forecast for 1986-87)

were also made available on school intake and on medium-term forecasts. Fig. 3(b) shows the same forecasts as in Fig. 3(a), with the same trend of increasing underforecasts and decreasing total error; the percentage errors are less because some of the individual school errors have cancelled each other out in the group totals.

Comparing the errors in forecast school intake (Fig. 3(c)) with those in the total school roll (Fig. 3(b)) it is clear that at least half the error in 1-year primary school forecasts is due to the difficulty of predicting the new intake of infants.

Fig. 3(d) shows that forecasts further into the future tend to have greater errors, as we would expect. However, the forecasts of intake are not less reliable as the range increases from 1 year to 4 years. This result suggests that the Health Authority Infant Register, frequently used as an indicator of future school intakes and used in LEA B for the forecasts 1 year and 2 years ahead, is not a good guide to children's changes of address after birth, at least when used for this purpose.




Aggregate errors, as a percentage of the total LEA roll

6%

5%

Total errors

4%U

3%U ver-forecasts

1% C ' Urider.forecaPts

A B C D E F G H LEA

Fig. 4. Errors from various LEAs' school roll forecasts: each forecast is of primary school rolls, made 1 year ahead, as frequently used for staff allocation and for controlling the new intake of pupils to schools (the data all refer to forecasts made in the 1980s, of all schools in the LEA or LEA district); the errors for schools in an LEA are aggregated and expressed as a percentage of the aggregate LEA roll; the LEAs are ordered from left to right in decreasing average size of school; the variety of forecasting circumstances are

(a) LEA type-metropolitan district, B; English county, A, C, D, E, F, G; Scottish islands region, H;

(b) computer assistance in forecasting-microcomputer, A, E, G; mainframe, C, D, F; none, B, H; (c) forecast responsibility-central unit, D; education research officers, E, F, G; school buildings

officers, A, B, C; schools administration officer, H; (d) staff equivalent annually used-2 weeks, H; 1-6 months, B, C, F, G; 6-12 months, D, E; 15

months, A; (e) forecast range (the display is of the first year only)-4-5 years, A, E, F, G, H; further (future

births estimated), B, C, D; (f) estimates of new housing-included, A, D, F, H; not included, B, C, E, G

4.4. Forecast Accuracy in Various Local Education Authorities Fig. 4 compares 1-year forecasts of individual primary schools, from data relating

to this decade for each of the eight LEAs in Britain which supplied this information. Less error may not mean a better forecaster, because different demographic and educational conditions make forecasting more difficult in some areas than in others. However, there are two interesting results from these comparative data.

Firstly, in every LEA except LEA H, the underforecasts are matched or more than matched by the overforecasts. There is a common positive bias: the forecaster has recognized the educational costs of underforecasting and tended to inflate the forecasts to avoid these costs.

Secondly, the total errors are remarkably similar, seven of the eight being between 3.5% and 6% of the total LEA roll. The similarity of total forecast errors is interesting




50 ++ + I + +

I + +

I + +

4 4- + +

40 + + +

I + +

I + +

I + +

3 + +* +

30 + + +

I + +

I + +

20 + + +

20 +

0 + - +

I + +*

I + +

10 + + I + **2+

I + 2*+2 +~ 22

SIMPLE COHORT **+ ** o

I PROJECTION ERRORS *+*2*+ I0* 2~-*+ +

I ** ** ~~~+ * +

I ~~~~~~+* ** +

-10 + + +

I ~ ~ ~~~+ +

I ~~~ ~~+ *+

I ~~~~~+ +

I + ** * ~++ -20 + +

I ~~~+ +

I ~~~+ +

I ~~~+ +

I ~~~+ +

-30 + + +

I ~~+ *+ I+ +

I+ +

I + +

-40 + + +

I + +

I + ~~~~RESEARCH +

.50 ++ UNITERRORS + +-4--- 4----F--- -- - 4--- -- --------------F----F----F--------------------.

-50 -40 -30 -20 -1 o 10 20 30 40 50

Fig. 5. Comparison of errors from a simple and a refined methodology: primary schools forecast 1 year ahead

because the means of forecasting varies from hand-calculated estimates with very crude assumptions to more highly sophisticated computer-assisted models in which more effort is expended each year.

LEA H, with relatively very small errors, is a Scottish isles authority. It has few schools, most of them with catchments precisely defined by the limitations of transport between islands. The authority's officers have near exact knowledge of pupils and residents.

If widely varying effort in forecasting produces similar accuracy perhaps the complexity currently employed in some LEA forecasting work is not very effective in gaining greater accuracy. Fig. 5 compares the errors from a relatively sophisticated forecast by a respected local authority research unit for its primary schools with a forecast for the same schools in the same year by a relatively crude method employing data on past trends only at aggregate level and only for the past 2 years. The main differences in method between the two models concerned estimation of migrants and of the new intake of infants to each school:

(a) migrants-the research unit forecast included adjustments for expected in-migrants to new housing planned, in progress, or recently completed; the simple cohort survival forecast included no adjustments for new housing developments;




(b) infants-the research unit forecast of new intake to each school was based on a local 'attendance factor', the ratio of the school's past intake to infants resident in the school's area, estimated from the recorded births in that area 5 years previously and adjusted for estimated net migration; the simple cohort survival model used only the trend in births at an aggregate local authority district level, applying it equally to each school's current intake, with no adjustment for the known information on births in each school's area.

The simple cohort survival model described here provides a useful bench-mark by which to measure the results of other forecasts.

Fig. 5 shows most schools clustered at the centre of the plot, indicating that their future roll was forecast 1 year ahead without much error by both the more complex and the simple models. Along the leading diagonal are other schools which were misforecast by both models equally. It is the schools away from the leading diagonal which were forecast differently by the two models. For several schools, the forecasts were more than 20 pupils apart. The two largest errors are for two schools severely overforecast by the research unit model, by 40 and 50 pupils; the simple model also overforecast them, but by two and 25 pupils respectively. The errors in the research unit's model were due to the mechanism for estimating the yield of in-migrant pupils for new housing developments. The planned housing was not completed on time, or yielded fewer extra pupils than the model had predicted. In the second school, it is possible that a change of parental choice away from that school also affected the accuracy of both models' predictions. In total the simple forecast had less overall bias and precisely the same total errors (as defined earlier) as the more complex forecast. It is possible that the errors due to the housing adjustment of the complex model would correct themselves in medium-term forecasts as the timing of house building is most crucial in the shorter term, but the messages remain that simplicity should not be ignored and that every forecasting system has room for improvement.

4.5. Accuracy of Headteachers' Estimates It was suggested in Section 2 that the local knowledge of an area possessed by

headteachers is the most useful ancillary information to centrally held databases that can improve LEA short-term school roll forecasts. Some LEAs regularly compile headteachers' estimates and in Fig. 6 these are compared with centrally prepared forecasts for the three LEAs which provided such data.

As previously, aggregate underforecasts and aggregate total errors are indicated, aggregate overforecasts being the difference between these two. In each of the three LEAs, the underforecasts are less in the headteacher estimates. However, the tendency to overforecast, to build in safety margins, already noted in the centrally prepared estimates in Section 4.4, is greater with headteacher estimates. There are some schools for which headteachers have better demographic information, e.g. from waiting lists and knowledge of pupils' siblings, which helps them to make more accurate forecasts. For others, a lack of organized information and the desire to protect resources produce less accurate forecasts. Headteacher estimates are an essential adjunct but not a replacement for central LEA school roll forecasting.




Aggregate errors 16% as a percentage of the total LEA roll

H

14%-

12% -

10%

8% H

6% - H

4% - Over-forecasts

2% c- .-\

H HH 0% Under-forecasts H

LEA D LEA F LEA I PRIMARY PRIMARY 6TH FORM SCHOOLS SCHOOLS

Fig. 6. Accuracy of forecasts prepared by the LEA and by headteachers, 1 year ahead: C, forecasts made centrally by the LEA; H, forecasts made by each headteacher

5. DISCUSSION

The paper has surveyed three areas of school roll forecasting that have not previously received attention.

At present, forecasting is the pursuit of each LEA, using similar methodology but without common development. It is common for an LEA's production of forecasts to collapse on the departure of key personnel who have held the knowledge necessary to school roll forecasting without documentation. There is at present no national centre for the development of tools for planning and administration at a local level.

The government's proposed restructuring of the school system has far-reaching implications for educational planning. Local policy tailored to local educational needs becomes more difficult to implement. While a comprehensive information base is still necessary to routine administration, it also becomes more political in its use by tracing the effects of market forces and central government policy on the provision of schooling and access to it.

The evaluation of past forecasts has been seen to be important and useful. Use- related measures of accuracy may be relevant to other multiregional forecasts that are used for resource distribution, including government forecasts of local authority populations. At present, little such evaluation is formally carried out by LEAs; past forecasts are not kept, as is reflected by the predominance of short-term forecasts available for analysis in this review. It has been shown, however, that simple but robust methods of forecasting are a more reasonable objective for development than




further complexity in the model itself. Complexity may better arise as a result of adjustments to an initial robust core forecast, to take account of headteachers' knowledge, expected large housing developments, pupils received from across LEA boundaries and other influences.

ACKNOWLEDGEMENTS

This paper is largely based on research undertaken at Sheffield City Polytechnic's Department of Education Management, funded by the Department of Education and Science. The author is grateful to the efforts of local authorities which supplied the data analysed in the paper. An earlier version of the paper was presented to the conference of the British Society of Population Studies, 1988. Particularly useful comments were received from a referee and from colleagues in Calderdale Education Department.

REFERENCES

Armstrong, J. S. (1978) Long-range Forecasting. New York: Wiley. Braden, B. et al. (1972) Enrolment Forecasting Handbook Introducing Confidence Limit Computations for

a Cohort-survival Technique. Newton: New England School Development Council. Cook, R. (1985) It is possible to predict enrolment. Momentum, 16, 40-42. Denham, C. (1971) Probabilistic school enrolment predictions using Monte Carlo computer simulation.

Final Report. Boston College, Chestnut Hill. Department of Education and Science (1986) Education capital expenditure 1987-88 to 1989-90. Letter

to Local Education Authorities. London: Department of Education and Science. Her Majesty's Government (1988) Education Reform Act. London: Her Majesty's Stationery Office. Jenkins, J. and Walker, J. R. (1985) School roll forecasting. In Information Systems for Policy Planning

in Local Government (eds J. R. England, K. I. Hudson, R. J. Masters, K. S. Powell and J. D. Shortridge), ch. 2.6, pp. 96-112. Harlow: Longman.

Mahmoud, E. (1984) Accuracy in forecasting: a survey. J. Forecast., 3, 139-159. Mar Molinero, C. (1988) Schools in Southampton: a quantitative approach to school location, closure

and staffing. J. Op. Res. Soc., 39, 339-350. Simon, B. (1988) Bending the Rules. the Baker 'Reform' of Education. London: Lawrence and Wishart. Simpson, S. (1987) School roll forecasting methods: a review. Res. Pap. Educ., 2, 63-77.

(1988) The use of school roll forecasts in LEA administration: the allocation of resources to schools. Camb. J. Educ., 18, 89-98.

Simpson, S. and Lancaster, D. (1987) Forecasting pupil numbers for educational planning. Sheffield Papers in Education Management, 71. Sheffield City Polytechnic.

Streatfield, D. and Jones, S. (1987) From here to technology: the use of computers in management and administration by Local Education Authorities. Report. London: Peat Marwick McLintock-National Foundation for Educational Research.

COMMENTS ON THE PAPER BY SIMPSON

Mr I. F. Plewis (Thomas Coram Research Unit, London): Dr Simpson's useful paper raises many questions about educational planning in the 1990s, and hence about the kind of educational statistics that will be produced, and needed, then. Views about what statistics are needed properly to evaluate the new educational policies will vary across interested parties. As the paper points out, local education authorities (LEAs) will have to deal with a changed situation, one in which preferences, expressed not so much by the consumers as by their parents, will come much more to the fore, with the needs of the community as a whole given less prominence. LEAs do have to




take account of consumer preferences at the moment, particularly with regard to the transition to secondary schools. Additionally, about 10% of secondary school pupils go into the private sector, and there is movement of pupils across LEA boundaries within the state sector, figures which will vary from LEA to LEA and from year to year. Nevertheless, in the future, LEAs will have to predict both the number of pupils requiring locally controlled education and the number of schools at their disposal, in the face of much more uncertainty than at present.

As well as coping with the problems of forecasting, LEAs will be expected to produce data from the national assessment programme for pupils aged 7, 11, 14 and 16. It is intended that these data will be used explicitly to compare schools although without, at the moment, any clear guidelines on how these comparisons should be made. It is not apparent what will influence consumer choice in the future although some evidence about the effects of the 1980 Act on widening choice is given by Stillman and Maychell (1986). Nevertheless, it is reasonable to suppose that these choices will be influenced and modified by the information provided by the LEA. Those LEAs which produce thoughtful, clearly presented and, above all, widely available information about their schools may well find that their forecasting and planning are made much easier than those LEAs which do not, or are not able to, devote resources to these tasks. One reason for this supposition is that all the evidence from the school effectiveness research shows that differences between secondary schools in examination performance are rather small once adequate allowance has been made for the intakes to these schools.

Regardless of the 1988 Education Act, it is right that LEAs should not only provide a range of information about the schools under their control for use by the community as a whole, but also that they are able to describe the sources of their data, and their methods of analysis, to those who would wish critically to examine them. If this is accepted, then it is disturbing to read in the paper that so few LEAs can provide a written report of the methods used to forecast pupil numbers.

Mr W. B. Wakefield (Department of Education and Science, London): The Edu- cation Reform Act places much greater responsibilities on headteachers and school governors for the management of their schools. This substantial change of emphasis raises new questions about school roll forecasts. Have they a place at all? Who should be producing them if they are required? If they are produced what value might they be to a local education authority (LEA)? The paper addresses these only in passing. There are two statements in the third paragraph of Section 5 but these are not related to the earlier analyses of the accuracy of forecasts.

For example, it is interesting to note from the previous section that the underforecasts are less in the headteachers' estimates but the tendency to overforecast, to build in 'safety margins', is greater. Can this be expected to continue? It could have been suggested that the procedures adopted for forecasting by headteachers might change substantially in the future. Some will assume, because of the school's high standing, that each entry form will be filled and total pupil numbers will be maintained: a simple and direct forecasting procedure. The consequences for other schools would then depend on the general level of population change in the LEA. It could have been argued that it will be more important for an LEA to have reliable school population forecasts either, depending on circumstances, to encourage schools of lesser standing to close or restrict entry for their own good or alternatively to arrange for the opening of a new school.




A possible scenario for the future based on the conclusions reached in the paper is that every headteacher should be encouraged to adopt simple but robust forecasting methods for the school's population 5 years ahead for sharing with governors and the LEA. Associated with the estimates would be statements on general assumptions and assumptions specific to the school. The LEA's responsibility would be to compare estimates produced by its own methods with the aggregates of school-produced forecasts and to make recommendations. It is disappointing that no attempt is made to put forward a scenario or alternative scenarios and to relate conclusions from the research to them. If such an approach had been attempted the two statements referred to earlier might have been substantiated and means of meeting them illustrated in the context of educational reform.

Experience has shown that the most difficult and sensitive aspect of school roll forecasting is producing assumptions about migrants. It would have been helpful for schools and LEAs if the author had included a section comparing the reliability of different methods of preparing migration assumptions comparing the procedures adopted by schools and LEAs. It could be argued that the preparation of such assumptions will be of even more crucial importance in the future.

Author's Reply Mr Plewis indicates, and I agree with him, that current educational legislation gives

less prominence to the needs of the community and more to those of the individual pupil and parent. However, a school's policies and management affect many pupils' and parents-they take effect on, and shape the educational provision for, an aggregate community of pupils, potential pupils and their families. School and community require information (and, to answer one of Mr Wakefield's questions, school and community require school roll forecasts) to help them to create schooling that is appropriate and resourced.

Specifically, there are two major uses for roll forecasts as indicated in Section 1. Firstly, they are essential if resources are to be distributed to schools to cater for the pupils expected to be in school at the time that the resources are used: budgets are set in an earlier school year than the year in which they are used thus current pupil number are not a good guide to need for resources. On this point the Department of Education and Science (DES) has given no advice to local education authorities (LEAs) in setting the formulae now legally required for distribution of resources. Secondly, as Mr Wakefield discusses, roll forecasts are an ingredient of decisions on the rationalization of school places, including the restriction of entry to schools to prevent overcrowding.

The recent legislation is putting many new demands on educational management and the paper argued that at the same time the legislation makes forecasting consider- ably more difficult. There is a danger therefore that school roll forecasting will be given less time and still less development by LEAs than at present. However, this danger in no way diminishes the importance of using roll forecasts in these two major areas of administration.

Parents will wish to know the competition for places when choosing schools; forecasts of demand for a school's places are useful information to them. I whole- heartedly agree with Mr Plewis that each LEA has responsibility to provide clear and thoughtful information to parents on each school's provision and record.




The preparation of migration assumptions for school roll forecasts was not the subject of this paper but was dealt with in detail in Simpson and Lancaster's (1987) report to the DES. Briefly, a school's cohorts compared from one year to the next provides a measure of recent local movement which is incorporated into forecasts by most LEAs. The in-migrants generated from expected new developments of less than 50 dwellings are rarely worth incorporating, since the net yield of extra pupils is small and highly variable. It is dependent on the nature of householders moving to the new houses as well as on the nature of the new dwellings. Recently relaxed planning controls make prediction of the occupants of small developments of dwellings a near impossible task and experience has shown that the effect on school rolls is most usually negligible. Where large estates or new towns are concerned, a standard approach is more feasible and is usually incorporated in local authority all-age population forecasts.

I agree with Mr Wakefield's suggestion that headteachers be encouraged to provide reasoned forecasts which can then be assessed and combined with centrally prepared LEA forecasts. Data are available to pursue the effects of different strategies of combining such forecasts, from LEAs where this is current practice. However, this approach would run into difficulties if headteachers were to forecast 5 years ahead as Mr Wakefield suggests. Schools usually do not have the required information on local infants. In times of crisis schools have occasionally mobilized volunteers to conduct a census of local infants, gaining better information that the LEA can hope to command, but it is unreasonable to expect such an outlay of volunteer resources on a regular basis. The LEA often does have access to indirect estimates of infant numbers from health authority records. We are led once again to the importance of the LEA in providing school and community with relevant information for their planning and decision making.

The increasing uncertainty of future demand for individual schools suggests that attention should be focused not only on accurate forecasts but also on planning strategies that reduce the educational costs of errors in forecasted school rolls, a point developed further in Simpson (1988). Mr Wakefield points to room for further appraisal of local educational administration. It would be helpful if the DES were to attempt to have the room filled. There remains a need for a national centre of support, development and training for local government educational planning in general, and local government demographic work in particular.

REFERENCES

Simpson, S. (1988) The use of school roll forecasts in LEA administration: the allocation of resources to schools. Camb. J. Educ., 18, 89-98.

Simpson, S. and Lancaster, D. (1987) Forecasting pupil numbers for educational planning. Sheffield Papers in Education Management, 71. Sheffield City Polytechnic.

Stillman, A. and Maychell, K. (1986) Choosing Schools: Parents, LEAs and the 1980 Education Act. Windsor: NEER-Nelson.



Documents

School Roll Forecasts: Their Uses, Their Accuracy and Educational Reform