Continuous Improvement in the Quality of Provision …...Continuous Improvement in the Quality of Provision and Learners’Performance Post-16 The ES will use the self-assessment and

Continuous Improvement in theQuality of Provision and Learners’Performance Post-16

A Guide for Providers on Self-assessmentand Development Planning

Foreword

Section 1 – Introduction

Purpose of the guide 1

Continuous improvement in the quality of provision and the performanceof learners post-16 – roles and responsibilities 3

New arrangements for self-assessment and development planning 8

Section 2 – Self-assessment

Overview 1

Purposes of self-assessment 4

Self-assessment framework and report 7

Self-assessment evidence and judgments 17

Responsibilities for self-assessment 32

Timescales for self-assessment 39

Procedures for agreement of self-assessment reports and development plans 44

Section 3 – Development Planning

Overview 1

Identifying and prioritising areas for improvement 5

Defining objectives and targets for improvement 6

Specifying actions, responsibilities and timescales for achieving improvements 9

Arrangements for monitoring and reporting on progress and outcomes 13

Annexes

Annex A LSC/ES requirements for self-assessment and development planning

Annex B Preparing for self-assessment and development planning – a checklist

Annex C Self-assessment report based on draft format of inspection reports

Annex D Performance indicators and evidence: some examples

Annex E Examples of good evaluative statements

Annex F Example of a development plan

Annex G Further sources of information and advice

i

Contents

This document was developed with:

Philip Cox and Rosemary MoorseLearning and Skills Development Agency

The Learning and Skills CouncilCheylesmore HouseQuinton RoadCoventryCV1 2WT

Foreword

iii

Post-16 education and training providersare expected to produce annual self-assessment reports and development plans.This is one of several publications designedto help providers to produce sound reportsand effective plans. It is written primarilyfor those providers who have little previousexperience of these activities, but it will beof some interest to those who have alreadystarted to produce them.

The Learning and Skills Council and itspartners published general guidance onrequirements for self-assessment anddevelopment planning in March 2001. TheEmployment Service, the Adult LearningInspectorate and Office for Standards inEducation have welcomed the productionof this guide for providers. It has beenprepared for the Learning and Skills Councilby the Learning and Skills DevelopmentAgency. The guide gives providers helpfuladvice on how to organise and carry outself-assessment, how to evaluate and gradeprovision, and how to write reports andlayout development plans. It includes usefulappendices on the preparation of reportsand the nature of the evidence that mightbe quoted in them.

Raising standards of learners’ performanceand continuous improvement in the qualityof provision is a central concern of theLearning and Skills Council. Theresponsibility for making improvementsrests with providers. Good self-assessmentand development planning are importantmeans by which providers can movetowards higher standards and betterquality. Through producing this guidancefor providers, the Learning and SkillsCouncil aims to strengthen providers’efforts to improve the quality of theirprovision and raise the standards attainedby learners.

Thanks are due to Philip Cox and RosemaryMoorse for their help. We hope you will findthe document useful.

Avril Willis

Director of Quality and Standards

Learning and Skills Council

Continuous Improvement in the Quality of Provision and Learners’ Performance Post-16

1

Section 1 – Introduction

Purpose of the Guide

1 New policy guidelines on self-assessmentand development planning for post-16providers were published in March 2001. Thisdocument offers further guidance to providerson their responsibilities for carrying out thiswork. More specifically the guide:

• identifies important shifts in policyand practice that have implicationsfor all providers

• offers advice on self-assessment anddevelopment planning, together withexamples linked to the requirementsset out in the policy guidelines

• provides signposts to further sourcesof advice and guidance.

2 Supplementary guides will be issued toproviders of adult and community learning andwork-based training. These guides will includeadvice on how to set about self-assessmentand development planning, for those with littleexperience of such activities.

Continuous improvement in the quality ofprovision and the performance of learnerspost-16: roles and responsibilities.

3 The Learning and Skills Council (LSC) andEmployment Service (ES) will seek to developan effective strategy for continuous

improvement in all post-16 provision,consistent with the aims set out in ‘RaisingStandards in Post-16 Learning’, which are:

• high levels of learner achievement

• excellence in teaching, training andother services

• learning represents value for money

• learning meets the needs of learners,employers, the local community andthe economy.

4 Providers will have the primaryresponsibility for improving the quality andstandards of provision and raising standards.For this purpose they will be contracted to:

• carry out an annual self-assessmentof all aspects of their provision

• produce a self-assessment reportwhich identifies strengths andweaknesses, and other aspects ofprovision in need of improvement

• agree with LSC and the ES adevelopment plan aimed at bringingabout improvements

• monitor and review theimplementation of the developmentplan on a regular basis.

5 The LSC and the ES will use providers’ self-assessment reports and development plans askey sources of evidence when making decisionsabout funding and contracts with the provider.

A Guide for Providers on Self-assessment and Development Planning


The ES will use the self-assessment anddevelopment plan to inform contractmanagement and continuous improvementactivities, as detailed in the ES QualityFramework and through the ES ContractManagement Framework. Through regularmonitoring visits and review exercises they will:

• make judgements about the qualityand rigour of the provider’s self-assessment process

• approve (or refer for further action)the provider’s development plan, andagree targets and milestones

• offer support (or impose sanctions)where deficiencies are identified inthe self-assessment report ordevelopment plan

• monitor and review progress againstthe approved plan.

6 The LSC and the ES will aim to bringabout improvement through their ownstrategies for improving the quality ofprovision, by identifying strengths andweaknesses, by interventions whereappropriate, and through spreading goodpractice.

7 The Office for Standards in Education(OFSTED) and the Adult Learning Inspectorate(ALI) will, through periodic inspections, offer anindependent account of the quality of teachingand training, the effectiveness of learning, thestandards achieved, and the efficiency andeffectiveness with which resources aremanaged by providers. The LSC and the ES willmake available to the inspectorates, copies ofproviders’ self-assessment reports anddevelopment plans (together with other data itholds on providers’) for use as part of theevidence base for inspection. Collegeinspections will no longer focus on validating

self-assessments. The inspectorates will seek tobring about improvement by identifyingstrengths and weaknesses and disseminatinggood practice.

New arrangements for self-assessment and developmentplanning

8 Many providers will be familiar with theprinciples and practice of self-assessment anddevelopment planning; others will not. The newarrangements apply to all providers. These aresummarised below.

9 All post-16 providers will be expected tocarry out self-assessment and developmentplanning against the requirements set out inthe national guidelines, published in March2001. (Key requirements are listed in annex Aand developed further in this document).

10 Providers’ arrangements for self-assessment and development planning will besubject to continuous monitoring and reviewby the LSC and the ES and four-yearlyinspection visits by OFSTED and/or the ALI.Providers will be required to develop qualityimprovement strategies in close liaison withLSC and the ES. These strategies must takeaccount of inspection findings.

11 The new arrangements have beendesigned to ensure that the interests oflearners come first and are of paramountimportance. The intention is to place learners,their needs, experiences and achievements atthe heart of these arrangements.

12 All provision for learners should beresponsive to the needs of employers, the localcommunity and the economy. To this end,providers will be expected to undertake theirplanning in concert with the LSC and the ESand other external agencies.

2


13 The emphasis in the new arrangements ison collaboration rather than competition.Providers should fully involve key partners inthe process of self-assessment anddevelopment planning.

14 Providers will be expected to promoteequality of opportunity in all aspects ofprovision. This will include adopting measuresto widen participation, removing barriers tolearning, and lessening inequalities inopportunities for employment.

15 All providers will be required todemonstrate financial probity and value formoney as key elements of their self-assessments.

16 In making judgements about the quality oftheir provision, all providers will be subject tothe same degree of rigor. The newarrangements, however, will be applied flexiblyso as to take into account the particular aimsof providers and the diverse needs of theirlearners.

17 The new arrangements will respect thefact that providers are at different startingpoints in their familiarity with self-assessmentand development planning. Support will beoffered to those providers with littleexperience in this area of work.

18 All providers will be expected to achievecontinuous, year-on-year improvements in thequality of their provision and to raisestandards. If standards are already very high,they should be maintained.

3


Section 2 – Self-assessment

Overview

1 Self-assessment should be a systematicprocess in which providers collect and analyseevidence in order to make judgements abouttheir performance in relation to agreed goals.The main purpose of self-assessment is self-improvement. Effective self-assessment enablesan organisation to identify its strengths andweaknesses, to compare its performance withthat of other providers, to identifyopportunities for improvement, to setobjectives and targets, and to prioritise theactions required to achieve these. It alsoprovides the means of identifying andresponding to the needs of learners and theirclient groups.

2 In planning for self-assessment andpresenting the self-assessment report providersshould identify:

• why self-assessment is being carried out

• which areas/activities will be subject to self-assessment

• how the self-assessment will be carried out

• who will carry out the self-assessment

• when the stages in the process will becarried out.

3 This document offers guidance on how toaddress each of these questions (with workedexamples) based on the policy guidelines onself-assessment and development planning. Achecklist is presented in annex B.

Purposes of self-assessment

4 Self-assessment should underpinorganisational development. It should thereforebe undertaken as an integral part of strategicand operational planning, not as a bolt-onactivity. It should also be integral to theprovider’s quality assurance arrangements,including any externally kitemarked qualityassurance standards that the provider hasachieved or is working towards (such as theEuropean Foundation for Quality ManagementExcellence Model, Investors in People, ISO9000,or Charter Mark).

5 Self-assessment is an important means ofdemonstrating public accountability. Thequality and rigor of the self-assessmentprocess will be examined critically as part ofthe LSC and the ES procedures for monitoringand review. Self-assessment reports willcontinue to form an important part of theevidence base for inspections. With appropriateexternal safeguards and support, self-assessment provides the key to achievingcontinuous improvement in provision andperformance post-16.

6 The purposes of self-assessment should beproperly communicated to all staff, learnersand others who use the provider’s services. Thebenefits (and costs) of self-assessment shouldalso be evaluated as part of the drive forcontinuous improvement.

Self-assessment framework andreport

7 Each provider should seek to develop aform of self-assessment that is responsive toits own organisational needs and the needs ofits client groups. There are four requirementsrelating to the self-assessment framework andreport that all providers must satisfy.

4


Self-assessment should deal with allaspects of the organisation’s activity, inparticular the quality of learners'experiences and the standards learnersachieve.

8 To meet this requirement, reports mustaddress all the areas of learning that make upthe provision. They should also includejudgements on all the key services thatcontribute to the experience of learners. Someareas of work such as guidance and support forlearners and equal opportunities, may need tobe evaluated as part of areas of learning and aspart of management. When makingjudgements on any aspect of provision andperformance, the emphasis should be onoutcomes and/or impact on learners and othercustomers rather than on policies andprocedures.

Providers must address the qualitystatements in the Common InspectionFramework (CIF) and the LSC and the ESquality and financial probity requirements.

9 The LSC and the ES requirements areintended to complement the qualitystatements set out within the seven categoriesof the Common Inspection Framework (seesection 38 of the policy guidelines on self-assessment and development planning, March2001). The requirements should also be read inconjunction with the Council’s new frameworkfor assessing provider performance.

10 The ALI has published draft guidance forproviders on how to interpret the CIF qualitystatements in the context of the differenttypes of provision for which it hasresponsibility: work-based learning, New Deal18-24, adult and community learning, andUniversity for Industry learndirect provision.The Handbook for Inspectors, published by theALI, and the Handbook for Inspecting Colleges,

published by OFSTED, also give furtherguidance on the new requirements.

11 It may not be possible or necessary toaddress each of the 60 quality statements inthe CIF. However, self-assessment reportsshould make clear how these statements haveinformed the provider's judgements and howthey have shaped the identification of keystrengths and weaknesses.

Self-assessment should take account of thequality improvement strategies of the LSCand the Council’s framework for providerperformance review.

12 The LSC has published a three-yearcorporate plan addressing needs and prioritiesfor learning and skills, and an operational plancovering target numbers for learners. LocalLSCs will publish an annual plan on needs andpriorities for the development of localprovision. The plans will be based on reportsfrom learning partnerships, national trainingorganisations, regional development agenciesand local authorities, and will need to bematched with providers’ plans for meeting theneeds of learners, employers and thecommunity. Providers should take account ofplanning at all these levels when evaluatingtheir self-assessments.

13 Providers’ frameworks for self-assessmentshould also take account of the Council’s keyassessment categories for providerperformance review. The key areas are:

• volume of provision agreed with thelocal LSC

• quality of education and training andthe standards achieved by learners

• equality of opportunity

• health and safety

• quality of leadership andmanagement

5


• continuous improvement

• quality of strategic planning

• financial assurance

• data management

• other priorities, including nationalinitiatives such as the basic skillsquality initiative.

The structure of reports should, as far aspossible, be similar to that of publishedinspection reports.

14 The LSC and the ES have not prescribed anentirely standard format for the self-assessment report. Providers should seek, as faras possible, to produce reports that correspondto the common inspection framework andinclude sections on learning areas and onleadership and management.

15 A summary of the key informationnecessary to meet OFSTED and the ALIreporting requirements is given in annex C ofthis document.

16 For all but the smallest organizations, theself-assessment report will need to beconstructed from a number of sub-reports.Providers should establish clear procedures forthe way in which this is to be done. It isimportant that the final report should beconcise and clearly expressed.

Self-assessment evidence andjudgments

17 Providers are required to evaluate andgrade provision using inspectorate gradingscales. In order to carry out this task it isnecessary for them to:

• make effective use of performancedata, including benchmarks

• provide clear evidence to supportjudgements

• be evaluative rather than descriptive,clearly identifying strengths andweaknesses

• be honest and objective.

These matters are addressed further below.

Effective use of performance data, includingbenchmarks, and management information.

18 Providers should collect data relating to allof the key criteria used in the self-assessmentframework and to all the activities/areas whichare subject to assessment.

19 It is essential that self-assessment reportsinclude data to support judgements aboutlearners' achievements and performance,including retention, achievement ofqualifications (part or whole) progress againstindividual learning goals, added value,attendance, punctuality and progression toother forms of education and training oremployment. The draft guidance published bythe ALI, Guidance on Inspection for Providers,gives advice on how to judge learners'achievements in the context of New Deal,work-based training, adult and communityeducation and UfI/learndirect.

20 Evidence should also be presented tojustify comments on the quality of teachingand training and the effectiveness of learning;for example, grades for lesson observation andinformation from questionnaires designed tomeasure learners’ levels of satisfaction withtheir provision. Many FE colleges havedeveloped effective arrangements for lessonobservation, based on inspection protocols, asan integral part of their quality assuranceprocedures.

21 Data should also be collected for assessingthe performance of providers’ support services.

6


Many providers have developed internal andexternal quality standards to help measure theeffectiveness of these services. Questionnairesdesigned to elicit staff and learners’ views ofthe effectiveness of these services may alsoprovide useful evidence. An example is given inannex D.

22 Year-on-year trends in providers’performance should be recorded to provideevidence of improvement. Performance shouldbe set against agreed targets and should,where possible, be compared against that ofother providers. Benchmarking data areimportant means of assessing performanceand setting appropriate target improvements.Providers should seek to use nationally andlocally derived data for these purposes(annexes D (2) and D (3)). It is recognised thatin some areas of work, such as learndirect andadult and community learning, benchmarks arenot yet available and will take time to develop.

23 Benchmarking is not just about comparingperformance. This in itself will not drive changeand lead to improved performance.Benchmarking is also about identifying,understanding and learning from the processesand practices that lead to superiorperformance in other organisations. It requiresthat providers ask such questions as: How goodare we? How good can we be? How can weget better? How can we learn from others?

24 Other activities and techniques whichmay help in analysing and improving providerperformance include:

• brainstorming – to generate ideasabout the possible causes of problemsand potential solutions

• cause and effect diagrams – toidentify the root causes of a problem

• pareto analysis – to prioritise the ‘vitalfew’ causes of a problem from the‘trivial many’

• process mapping – to help understandand streamline the operation of anywork process

• force field analysis – to identify thefactors helping or hindering change

• solution effect analysis – to identifythe effects of implementing aparticular solution

• failure prevention analysis – to helpanticipate problems before they occur.

Clear evidence to support judgements

25 The LSC and the ES, and the inspectorates,will wish to assess the quality and robustnessof the evidence quoted in self-assessmentreports and to decide whether it providesadequate justification for the strengths orweakness claimed. As a general guide, forevidence to be robust it should be:

• valid – evidence is appropriate anddirectly supports the strength orweakness identified

• quantifiable – internal and externalperformance measures are used,wherever possible

• sufficient – evidence is complete(selective or incomplete evidence cangive a false picture)

• current – evidence is recent enoughto give an accurate position at thetime of writing

• accurate – evidence is attributed tonamed and verifiable sources.

An extract of a report that meets these criteriais given in annex D (4).

7


26 It is necessary always to distinguish theactual evidence presented in support of ajudgement from the source of that evidence.The source of the evidence is in itself notsufficient. The example given in annex D (5)illustrates this important distinction. Theinternal and external sources from which theevidence is derived should, however, be statedclearly. Further information on possible sourcesof evidence for different types of providers isgiven in the ALI’s draft Guidance on Inspectionfor Providers. Where possible, providers shouldbase their judgements on more than one sourceof evidence.

27 Judgements should always reflect thevolume and range of provision under review. Itwould not be acceptable, for example, to claim'excellent retention' as a key strength if thearea concerned involved only a smallproportion of the learners.

Evaluative rather than descriptivestatements

28 Providers should consider the followingprinciples when addressing this requirement:

• always use appropriate evaluativeadjectives when identifying strengthsor weaknesses (eg, ‘good examinationresults’, not ‘examination results’)

• distinguish clearly betweenjudgements and statements of fact

• avoid vague language. A judgement isan evaluation

• distinguish between ‘strengths’ and‘norms’ (aspects of provision orperformance are only a strength if theyare above what is normally expected)

• seek to identify the most significantstrengths and weaknesses. Note inparticular those that have an impact

on learning and learners'achievements.

Examples to illustrate each of these keyprinciples are given in Annex E (1) – E (5).

29 Providers will be required to grade theirprovision, using the grade descriptors publishedby the inspectorates, and to record thesegrades in the self-assessment report. Providersare required to use the five-point numericalscale for everything other than lessonobservation for which the seven-point scaleshould be used. In order to avoid confusion, theALI now recommend the use of a five pointscale for teaching observation undertaken aspart of self-assessment. Grades should reflectthe balance of strengths and weaknesses andtake account of the relative importance of theissues under review. Poor levels of achievementon the part of the learners would, for example,outweigh other perceived strengths.

Honest and objective self assessment

30 This injunction is ultimately aboutdeveloping a self-critical organisation thatactively promotes feedback from staff, learners,employers and other interested parties, and isequally as confident about admittingweaknesses as claiming strengths. It alsoinvolves a provider’s capacity to distinguishbetween real strengths and normal or standardpractice (see above).

31 Providers should have establishedarrangements for moderating and validatingself-assessment judgements and grades. Does agrade 3, for example, mean the same acrossdifferent learning areas? Does it mean thesame for a particular area of learning across arange of providers? Internal moderation can becarried out by internal auditors or by teams orcommittees charged with validating the self-assessment reports and training should beprovided for this purpose. External validation is

8


important to ensure that standards are broadlyconsistent with those of other providers. It isalso useful in strengthening the expertise ofstaff who are working in the area under review.Providers should consider establishingnetworks within which they can work withother providers to establish agreed standardsand develop effective procedures formoderation and validation.

Responsibilities for self-assessment

Engagement of staff at all levels

32 In order to achieve continuousimprovement, all staff should be encouraged tomonitor and evaluate their own performanceand to identify areas for improvement. Theyshould also be given the time and trainingnecessary to carry out this work. Manyproviders underestimate the time needed tocomplete an effective self-assessment,particularly the time required to gather thenecessary evidence. Special considerationshould be given to the needs of part-time/volunteer staff in this respect. In planningthe self-assessment process, providers mustalso consider the part sub-contractors, keypartners and employers will play.

33 Where possible, and for all largerorganisations, teams should be established tocarry out the assessment of their own areas ofactivity. Typically, the teams will be:

• course and/or programme teamslinked to areas of learning

• functional teams for specialistservices, such as student services orfinance

• cross-functional teams offeringservices across departmentalboundaries.

34 Teams should be trained in the techniquesof self-assessment, particularly the skills ofassessing evidence and making soundjudgements. Each team should have a leaderwho will plan and manage the self-assessmentprocess and take responsibility for writing theself-assessment report.

Involvement of learners

35 Providers should demonstrate that theyhave fully involved learners in the self-assessment process. They should developeffective methods of gathering feedback fromlearners, including questionnaires, interviews,focus groups, workshops and complaintsprocedures. Surveys should also include theneeds of prospective learners and leavers’levels of satisfaction with their programmes.Providers should ensure that learnersparticipate fully in self-assessment teams andlearner-consultative committees and alllearners should be properly briefed on theoutcomes of the self- assessment process.These matters should be addressed instatements about learners’ ‘entitlements’ andresponsibilities.

Leadership and management

36 The self-assessment process must beeffectively led and managed. Governors, boardsof directors, trustees and senior managersshould be committed to the aims of self-assessment and seek to promote a climate oftrust in which individuals and groups feel ableto be self-critical about their performance.They should actively participate in the self-assessment process, in a manner consistentwith their responsibilities for raising standardsand improving the overall effectiveness andefficiency of provision. They should alsoapprove the final self-assessment report andevaluate the effectiveness of the self-assessment process.

9


37 An appropriate person should be assignedresponsibility for co-ordinating the variousaspects of the self-assessment process, forexample, for planning, timetabling, advising,facilitating, monitoring, reviewing, validating,report writing, editing, disseminating bestpractice. Ideally this would be a seniormanager with close access to the chiefexecutive or principal. In the case of a largeprovider, a group should also be established tooversee the self-assessment process.

All participants

38 All participants should be aware of theirresponsibilities within the self-assessmentprocess. They should also be properly briefedon the purposes of self-assessment, the scope ofthe self-assessment framework, how the self-assessment will be carried out, and thetimescales for self-assessment.

Timescales for self-assessment

39 Providers will be expected to establish andsustain a commitment to, and involvement in,continuous improvement so that self-assessment becomes a regular part of the dailywork of all staff and not an add-on extra or aone-off event. The aim should be to ensurethat problems are solved at source and thatways of achieving excellence are sharedthroughout the organisation on a continuousbasis. Self –assessment should also be integralto the normal planning cycles of the provider.

40 Providers will be required, as part of theircontracts, to carry out self-assessment anddevelopment planning across all aspects ofprovision at least once a year and to submitreports and plans to the LSC and the ES.

41 Providers can carry out self-assessment attimes which fit their strategic/operationalplanning cycles. An intended date forcompletion of the self-assessment report anddevelopment plan should be forwarded to theLSC and the ES so that appropriatearrangements can be made for providermonitoring and review.

42 Evidence from FEFC and TSC inspectionvisits suggests that the development of self-assessment as an annual process has encouragedmany providers to integrate self-assessmentinto their planning cycles. There is some wayto go before this is widely achieved, however.Many providers have underestimated the timetaken to complete the self-assessment, includingthe time necessary to gather evidence.

43 Providers who have not been required toself-assess in the past will be required to submitself-assessment reports and development plansin the new format by March 2002 at the latest.In some cases providers may be encouraged toproduce an updated self-assessment report inthe new format prior to inspection. Providersare currently given 6 weeks notice ofinspection and will want to take this intoaccount when planning the process.

Procedures for agreement ofself-assessment reports anddevelopment plans

44 Providers should send their self-assessment report and development plan tothe executive director of the local Learning andSkills Council. The report and plan will beassessed against the criteria given in RaisingStandards in Post-16 Learning: Self-Assessmentand Development Plans.

10


45 Providers should be informed of theoutcome of the assessment within four weeksof submission. Where the development plan isnot approved by the local LSC and/or the localES District Programmes Quality ManagementTeam, discussions will take place with theprovider. They will write to the providerindicating the weaknesses in the plan andsetting out what action needs to be taken toimprove the plan to an appropriate standard.

46 If the provider fails to produce anacceptable plan within agreed timescales andnecessary improvements are not made, theLSC and the ES will consider whether thestandards achieved by the provider aresufficient for the continued funding of theirprovision.

47 Implementation of the development planwill be monitored through routine visits bystaff from local LSCs and the localEmployment Service.

11


Section 3 – DevelopmentPlanning

Overview

1 Self-assessment should not be an end initself but a means of ensuring continuousimprovement. It must be complemented bydevelopment planning to address weaknesses,build on strengths and implement othernecessary changes identified through the self-assessment process. The planning andimplementation of changes prompted by self-assessment need careful consideration. It isestimated that 80% of improvement initiativesfail because of poorly thought-outdevelopment plans. All key stakeholders shouldbe involved in the planning process. It is alsoimportant that the interdependence ofindividual plans is properly understood andthat all plans become integral to the corporateplanning of the provider.

2 All providers will be required to producean overall development plan in which proposedactions are explicitly linked to self-assessmentfindings. The plan should be submitted to theLSC and the ES for approval, together with theself-assessment report, within agreedtimescales.

3 The LSC and the ES will discuss and agreetargets and milestones as set out in the planand may require amendments as a conditionof approval. They will also monitor providers’progress against the activities and targets setout in the plans. Following inspections,providers must update their development plansto take account of inspectors’ findings. Therevised plan must be submitted not later thantwo months following the published report.Providers in scope for re-inspection will berequired to produce a post-inspection plan,

again within two months of the publishedreport.

4 The development plan and updates shouldbe set out in tabular form and based on thecriteria set out in the policy guidelines (seeAnnex A). In summary, the plan should specifyactions and targets for improvement inspecified areas, together with assignedresponsibilities for carrying out the requiredactions within an agreed timescale. Anexample of a development plan that followsthis format is presented in Annex F. (The textin italics at the top of the plan providesprompts for each column). Appropriatearrangements should be in place formonitoring the implementation of agreedactions, for evaluating outcomes and judgingthe effectiveness of the self-assessment anddevelopment planning process. These mattersare addressed in the following sections.

Identifying and prioritising areasfor improvement

5 Development plans must be manageablein terms of their scope. Self-assessment willgenerate many areas for improvement.Attempts to address every area may proveself-defeating and lead to a loss of staffmorale. Priorities should therefore be set,focusing on those areas for improvement that:

• are paramount to learning andlearners’ achievements

• consolidate strengths and rectifyweaknesses promptly

• offer opportunities for simple ‘quickwin’ solutions

• offer major long-term benefits (butfor which careful planning is essential)

• reflect national or local priorities(including those identified by the LSCand the ES).

12


Development plans should also continue toaddress actions not completed in previousplanning cycles.

Defining objectives and targetsfor improvement

6 Providers will be expected to set clearobjectives and targets for improvement,together with criteria for judging whether theagreed actions have been successful.Statements of broad intention or aspirationshould be avoided. The objectives should beSMART; that is, they should be:

• specific – the objectives/targetsshould be clearly defined

• measurable – the objectives/targetsshould be capable of being measured

• achievable – there is at least a 50%chance of achieving theobjective/target

• results-orientated – plans are aimedat achieving improvement, notincreasing activity

• time-bound – specific dates and timesare set for achieving targets.

7 Providers will be expected to carry outbenchmarking activities when setting theirtargets. They will also be expected to maintaintheir high standards or, where necessary,improve them.

8 All providers will be expected to settargets for recruitment, retention, participationand achievement and to agree these with localLSC staff. It is anticipated that target-settingactivity will be extended over time to includeother measures, such as the levels ofsatisfaction of all those who use the provider’sservices, added value, equal opportunity, andvalue for money. Time will be needed to

establish baseline performance data acrosspost-16 provision. In the meantime, providerswill be encouraged to develop performancetargets for those activities that are importantto measure, rather than easy to measure.

Specifying actions,responsibilities and timescalesfor achieving improvements

9 Development plans should provide asound basis for bringing about improvement.For this purpose, they should specify all theactivities and tasks necessary to achieve theproposed objectives and targets forimprovement. Activities should be defined at alevel of detail necessary for the effectiveimplementation of the plan and should beordered into a logical sequence. The boundariesof development plans should be carefullydefined in order to assess their possible impacton other improvement initiatives. Care shouldalso be taken to ensure that the proposedactivities tackle the central issue. A keyquestion to ask is, ‘how is this activity going tomake a difference?’

10 The activities defined in the developmentplans should be properly costed and resourced.For this purpose an estimate should be madeof the number of staffing days and otherresources required for each task. The total costof the project should then be identified and abudget made available for the work to beundertaken.

11 Responsibilities for carrying out proposedactions should be recorded in the developmentplan. The person with overall responsibilityshould be identified and other team membersappointed to carry forward the proposedchanges. Responsibilities for monitoring theplan and evaluating outcomes should also beclearly defined. A ‘task programme’ should be

13


drawn up stating who does what and by when.Where necessary, appropriate training anddevelopment should be given to ensure thatstaff have the required knowledge and skills tocarry out this work.

12 All of those directly affected by theproposed changes should be consulted on theproposed developments and, whereappropriate, involved in their implementation.Any key partners involved in drawing up andreviewing the plan should also be identified.

Arrangements for monitoringand reporting on progress andoutcomes

13 Providers will be required to set specifictimescales for completion of each activity ortask. Plans for undertaking activities on an‘ongoing’ basis will not be acceptable. Whereappropriate milestones should be set forassessing progress, to ensure that the planstays on track so that it can be appropriatelyamended, where necessary.

14 The implementation of development plansshould be carefully monitored to ensure that:

• staff are fully involved in the process

• actions taken conform to the plan

• the reasons for departure from theplan are understood and agreed

• where necessary, appropriate actionsare taken to update or modify theplans.

15 Amendments to the approved plan shouldbe properly recorded. Development plans willneed to be updated in consultation with theLSC and the ES.

16 In evaluating and reporting on theoutcomes of improvement initiatives providersshould consider:

• the results achieved

• how far the results meet the targetsset

• unintended outcomes (positive ornegative)

• evidence of year-on-yearimprovements in performance

• strategies for rewarding improvementsin performance

• opportunities for sharing findings,experiences and best practice.

17 As part of the review process, providersshould also seek to evaluate the effectivenessof the self-assessment and developmentplanning process; for example, throughquestionnaires or discussion groups. Theultimate test is whether the benefits arisingfrom the self-assessment process outweigh thecosts. Providers should also assess whether thecapacity for self-assessment and improvementis enhanced by the process.

14


15

Paragraph references to policy guidelines onSelf Assessment and Development Planning(March 2001) given in brackets.

Self-assessment

Self-assessment process

The process should:

• encourage staff at all levels of theorganisation to evaluate theirperformance (39/50)

• involve learners, employers and othercustomers (50)

• culminate in the approval of thereport by chief executive, principal ordirector and the endorsement of thereport by the governing body or boardmembers where appropriate (50)

• be an integral part ofstrategic/operational planning andquality assurance arrangements. (51)


The framework and report should:

• deal with all aspects of theorganisation's activity, in particularthe effectiveness of learning and thestandard learners achieve (40, 50 &53)

• address all the quality statements inthe Common Inspection Frameworkand the LSC/ES quality and financialprobity requirements (6, 36 & 37)

• as far as possible, follow a structuresimilar to that of published inspectionreports. (51)

Evidence and judgements

The self-assessment should:

• make effective use of managementinformation and performance data,including benchmarks (50)

• provide clear evidence for judgements(50 & 53)

• be evaluative rather than descriptive(53)

• be honest and objective. (50 & 53)

Development Plans

Structure of development plan(57)

The plan should be presented in tabular form(57):

• areas requiring improvement

• targets for improvement

Annex A: LSC/ES Requirements forSelf-assessment and DevelopmentPlanning

Annex A: LSC/ES Requirements for Self-assessment and Development Planning

• actions required to bring aboutimprovements, including costingswhere appropriate

• clear statement of the expectedoutcomes of specific actions

• criteria for judging whether actionshave been carried out successfully

• person(s) responsible for ensuring thatactions are carried out

• priority given to each action

• timescales for the completion ofactions, with milestones whereappropriate

• arrangements for monitoring,evaluating and reporting on progress

• dates by which actions have to becompleted and outcomes achieved

• involvement of key partners indrawing up the plan and thesubsequent reviews.

Other requirements fordevelopment plans

Plans should:

• be manageable in scope (55 & 56)

• be linked explicitly to all actionsidentified in the self assessmentreport (57)

• take account of national and localpriorities (58)

• identify and prioritise areas forimprovement (55 & 57)

• be reviewed and updated regularly(61)

• provide a sound basis for bringingabout improvement. (55)

16


17

Purposes of self-assessment

• are the purposes of self-assessmentclearly defined?

• are these purposes properlycommunicated to and understood byall those involved?

• is self-assessment used as a vehiclefor improvement rather than being anend in itself?

• are the costs and the benefits of theprocess properly understood?


Does the self-assessment take account of:

• the needs of learners, employers, thecommunity and the economy?

• the criteria set out in the commoninspection framework?

• the additional criteria introduced bythe LSC and the ES?

• the continuous improvement strategiesand review processes of the LSC andthe ES?

Responsibilities for self-assessment

• is top leadership actively committedto and involved in the process?

• are teams established for all areas tobe assessed?

• are team leaders appointed to planand manage the self-assessmentprocess?

• are appropriate responsibilitiesassigned for the co-ordination of theprocess?

• are staff appropriately trained in self-assessment methods and procedures?

• do staff have reasonable time andresources to carry out the process?

• are learners actively involved in theprocess?

• are employers and others who use theprovider’s services involved in theprocess?

Timescales for self-assessment

• has an annual cycle of self-assessment been established andcommunicated to staff?

• has the provider established aplanning cycle in which self-assessment, quality assurance andstrategic/operational planning areintegrated?

• has sufficient time been allowed toundertake the process, including thegathering of evidence?

Annex B: Preparing for Self-assessment and DevelopmentPlanning – A checklist

18

Use of performance data tosupport self-assessment

• is performance data available for allcurriculum/services areas and alllearner groups?

• is performance measured againstagreed targets?

• does the provider measure trends inits performance?

• is performance’ benchmarked’ againstthe performance of other providers?

• is the data easily accessible?

Analysis of information and data

• how does performance compare withtargets?

• what are the trends in performance?

• how does performance compareagainst that of other providers?

• is current performance sustainable?

Judgements and grading

• are our strengths real strengths or dothey simply reflect what is normallyexpected?

• how do these strengths enhancelearning and achievement?

• are arrangements in place for themoderation and validation ofjudgements and grades?

• are self-assessment reports evaluativerather than descriptive?

Development plans

• do plans address all of the identifiedweaknesses?

• do the plans address actions notcompleted from previous cycles?

• are the objectives/targets forimprovement SMART?

• do plans tackle the root causes ofidentified problem?

• are individual and teamresponsibilities for action clearlydefined?

• are plans properly costed andresourced?

• are there clear timescales withinwhich actions are to be completed,including milestones?

• are plans documented thoroughly?

• are plans integrated with theprovider’s strategic/operational plans?

Monitoring implementation ofplans

• were the plans properly monitored?

• were the actions taken in line withthe plan?

• where appropriate, was action takento modify the plan?

Evaluating and reporting onoutcomes

• what were the results of the actionstaken under the plan?

• how did the results compare to theexpected/desired results?

• were there unintended outcomes(positive or negative)?

• did the benefits of the processoutweigh the costs?

Annex B: Preparing for Self-assessment and Development Planning – A checklist


19

Note that the structure for self-assessmentreports recommended by the ALI appears onpages 11/12 of the Draft Guidance onInspection for Providers. The OFSTED structureis presented as annex C of the Handbook forInspecting Colleges. Both documents areavailable from the respective ALI/OFSTEDwebsites.

Introduction or backgroundsection to the report

• the nature of the provider and itswork

• main aims/purposes/mission of theprovider

• location and any significant featuresthat affect provision

• number and range of learners

• programmes offered and areas oflearning

• provider's funding arrangements

• local/regional employment data andeducational achievement data

• a description of the provider's self-assessment processes

• a table of grades which includes: agrade for leadership and managementand grades for equality of opportunity

and for quality assurance; grades forlearning areas

• an overall judgement on the adequacyof the provision.

Reports on learning provision

There should be a report on each programmeor area of learning that includes:

• a brief overview of provision

• key strengths and weaknesses(including other areas forimprovement)

• a table of observation grades (whereappropriate)

• tables showing numbers andachievements for the past two orthree years with nationalbenchmarking data for the mostrecent year where available

• a grade for the provision.

These sections should give particular emphasisto questions 1-6 of the CIF but also addressrelevant aspects of key question 7. Whereappropriate, and depending on the volume ofprovision, there might be a separate reportwith a contributory grade for specificprogrammes within each area of learning(work-based learning for adults and/or youngpeople, New Deal, Adult and communitylearning, University for Industry learndirect

Annex C: Self-assessment ReportBased on Draft Format of InspectionReports

Annex C: Self-assessment Report Based on Draft Format of Inspection Reports

provision, ES provision, full-time 16-18provision)

Report on leadership andmanagement

This report should focus primarily on question7 of the CIF and related LSC qualitystatements and make particular reference tothe impact of provision on learnerachievements. It should give an overalljudgement on the quality of leadership andmanagement, identifying key strengths, keyweaknesses (and other areas for improvement),and record an overall grade. The report onleadership and management should includejudgements about equal opportunities andquality assurance with a contributory grade foreach.

It is likely that the self-assessment reports forlarger providers will be built up from a numberof sub-reports. Such providers may prefer tofurther sub-divide the contributory aspects ofleadership and management, for examplegovernance and value for money, or to includeadditional functional (departmental) reports.

Appendices

Appendices might include an organisationalchart, staff statistics and performance dataover a three-year period.

20

D2 Use of benchmarking data to support judgements on teachingand learning

r


21

D1 Use of performance indicators by learner support services

Annex D: Performance Indicatorsand Evidence – Some examples

Weakness/Strength: Highly effective, comprehensive and accessible learning support

Strength: Highly effective, teaching and learning in construction:

Evidence:

• All full-time students underwent initial screening

• Of those diagnosed as needing learning support, 98% were offered it within threeweeks of the diagnosis

• Retention rates and achievement rates for those receiving support were on a par withor higher than for non-supported learners on comparable programmes

• Mean score of 4.25 (out of 5) for this item on learner survey (increase of 25% on last year)

Performance is judged against internal performance indicators relating to:

– the number of students screened

– extent to which learners’ need are met

– retention and achievement rates for supported learners compared with non-supportedlearners

Learner survey data are used to judge users’ satisfaction with service

Evidence:

• Of 15 sessions observed in 1999/00 there were:– 3 x grade 1– 8 x grade 2– 4 x grade 3

• 57% were graded 1 or 2. This compares favourably with the national figure for99/00. It also shows an improvement over our last year's figure of 50%

• 9 out of 10 external verifier reports during the past year specifically highlightedeffective teaching and learning

Bold type shows how benchmarking data have been used to make comparisons and givefurther support to the judgements

22

Annex D: Performance Indicators and Evidence – Some examples

D3 Use of benchmarking data to support judgements on retentionand achievement

D4 Use of evidence which is valid, quantifiable, sufficient and current

D5 Actual evidence or source of evidence?

Weakness/Strength Evidence:

Strength: Highly effective teaching and learning in construction:

Poor retention and achievement on Starters 25 National dataadvanced GNVQ Retention 51% Retention 68%

Achievement 35% Achievement 43%

High achievement and retention on Starters 98/99 105 National datashort courses Retention 97% Retention 65%

Achievement 100% Achievement 83%Starters 99/00 71Retention 99%Achievement 100%

Evidence:

• Of the 15 sessions observed in 1999/00, there were:– 3 x grade 1– 8 x grade 2– 4 x grade 3

• 95% of learners surveyed in 1999/00 rated the overall quality of teaching and learningvery good or excellent (92% response rate from 205 learners)

• 9 out of 10 external validation reports during the past year specifically mentioned thequality of teaching and learning

Evidence is:

– valid - it directly supports the judgement

– quantified clearly and unambiguously using both figures and percentages

– sufficient - there are three separate sources of evidence from three differentperspectives number of learners observed is significant in each case

– current - all the data relate to the most recent year

– accurate - the source is the 1999-00 learner survey

Strength: Good retention on level 3 programmes

Actual evidence: Average of 89% retention on these programmes

Source of evidence: Retention records


23

E2 Distinguish between evaluative judgements and statement of fact

E1 Include an evaluative word or phrase that indicates whysomething is a strength or weakness

Annex E: Examples of GoodEvaluative Statements

Strengths

• Prospective learners receive comprehensive information and advice before the startof their programme (contains evaluative words)

• The centre has well-established QA procedures that are used effectively to bring aboutimprovements (includes evaluative word, refers to impact of procedures not simplytheir existence)

Weaknesses

• In many cases, teachers pay insufficient attention to the individual needs of studentsin planning their lessons and in teaching (includes evaluative words and clearlyidentifies the nature of the weakness)

• Additional support for identified learners is not consistently available (includesevaluative phrase. Includes reference to impact - students have identified needs thatare not being met)

Statement of fact:

• Work experience is included as part of the learning programme. (So what? Is itrelevant? Do learners learn from it?)

Evaluative judgement:

• All learners have access to a vocationally relevant work placement that is integral totheir programme and contributes to the practical assessment of learners

Annex E: Examples of Good Evaluative Statements

24

E5 Does the strength or weakness impact on the learner or othercustomer?

E3 Avoid vague or uncertain judgements

E4 Distinguish between real strengths and norms

Vague judgements:

• Students appear not to understand the criteria for assessment (Do they or don'tthey?)

• Some of the students' social facilities could be improved (Which facilities? Howserious is the problem? This is not a judgement of the situation as it is; it is aprediction.)

Evaluative judgements

• Many students do not understand the criteria for assessment

• Recreational facilities for students at the South campus are poor

Norms:

• Staff are appropriately qualified/well qualified

• Staff are enthusiastic/well informed

• Staff are committed to their work

It would be surprising if these things were not the case. Only refer to them if they areweaknesses.

• Effective collaboration with other agencies has led to an increase in the number oflearners joining basic skills classes

Raise retentionrates

Use of range ofactions as below:

Increase retentionrates on all f/tcourses by 3% andp/t by 2% to 87%and 85%respectively

June 2002 Head of school Seniormanagement team

Objectives Actions Measurable Timescales and Responsibility Monitoring, Costing and outcomes milestones for action responsibilities additional

resources

This is for a weakness identified in a self-assessment report and uses as an example poor retention of learners.

Weakness to be addressed: Poor retention

Continuous Improvem

ent in the Quality of Provision and Learners’Perform

ance Post-16

25

Annex F:Exam

ple of a D

evelopment Plan Additional

guidance providedbefore enrolment

New guidanceleaflet producedand all learnersreceive copies

Additionalrequirements setfor courseenrolment

Guidanceinterviews for alllearners

Aug 01 Course leader Head ofcurriculum areachecks guidance

£1000

All learnersrecruited meet newrequirements

Sept 01 Course leader Course leader

All Learners haveinterviews

Sept 01 Admission tutor Course leadermonitorsinterviewingarrangements

£2000

Close attendancemonitoring

Clear records ofattendance for alllearning

Oct 01 Course attendancecoordinators

Course leaderreceives weeklyreport fromcoordinators

£5000p/t coordinators

Annex F:Exam

ple of a Developm

ent Plan

26

Objectives Actions Measurable Timescales and Responsibility Monitoring, Costing and outcomes milestones for action responsibilities additional

resources

Criteria for learnersat risk agreed andat risk learnersidentified

Criteria published

Attendancecoordinatorsidentify all learnersmissing consecutivelessons and alertpersonal tutor

At risk learnersseen twice foraction planning

Aug 01

Sept 01

Course leader

Attendancecoordinators

Course leader

Report fromcoordinators

Increased tutorialtime for at risklearners

All at risk learnersattend weekly oneto one abovenormal level oftutorial support

Oct 01 Personal tutors Course leaderreceives monthlyreport from tutors

Retention ratesmonitored usingweekly aggregatefigures

Retention rate forcourse maintainedat or above 85%

Nov 01 – Nov 02 Course leader Head of curriculumarea monitors ratesand reports tosenior manager


27

Publications

Effective Self-assessment (FEFC 1999)A survey of college self-assessment withpointers for good practice

Self-assessment in Practice (FEDA, nowLSDA, 1998)A practical guide and supporting video forcolleges on how to self-assess

Self-assessment for Improvement (LSDA1999) Case studies and research on collegesusing self-assessment to improve practice

Good Practice Guide for Self-assessmentand Development planning (draft title:ES/DFES forthcoming) A guide for trainingproviders and those new to self-assessment

Self assessment for adult educationproviders (draft title: LSDA/NIACE September2001) A practical guide for LEA adult educationon self-assessment

Raising standards in post 16 learning: self-assessment and development planning (ALI/ES/LSC/ Ofsted 2001)

Useful websites andorganisations

www.ali.gov.ukAdult Learning InspectorateSite includes draft guidance for providers onwhat the CIF means to different providers

www.aoc.co.ukAssociation of Colleges

Represents FE colleges; advice, networking,support

www.dfee.gov.uk/post16Department for Education and Skills

Site for DfES post-16 policy and publicationsincluding Raising Standards in post-16 learning:self-assessment and development planning

www.fefc.ac.ukFurther Education Funding Council

Source for FEFC publications, inspectionreports and information on the FE Sector

www.learning-providers.netAssociation of Learning Providers

Represents training providers and voluntaryorganisations

www.LSagency.org.ukLearning and Skills Development Agency

Research and development supporting thesector

www.lsc.gov.ukThe Learning and Skills Council

www.lscbrief.org.ukUpdates on LSC and links to Training andEnterprise Network, Association of LearningProviders and other sites

www.niace.org.ukNIACE

The national organisation for adult learning

www.ofsted.gov.ukOfsted (Office for Standards in Education)

Includes the College Handbook on inspections

www.rqa.org.ukRaising Quality and Achievement

LSDA/AOC programme supporting qualityimprovement

Annex G: Further Sources ofInformation and Advice

© LSC 2001 October 2001Published by the Learning and Skills Council. Extracts from thispublication may be reproduced for non-commercial educationalor training purposes on condition that the source isacknowledged and the findings are not misrepresented.

This publication is available in an electronic form on theCouncil’s website: www.lsc.gov.uk

Reference REP/0068/01

Documents

Continuous Improvement in the Quality of Provision …...Continuous Improvement in the Quality of Provision and Learners’Performance Post-16 The ES will use the self-assessment and