Upload
anastasia-mitchell
View
218
Download
1
Tags:
Embed Size (px)
Citation preview
Why ‘outcomes’ aren’t enough
Tim Oates
Group Director
Assessment Research and Development (ARD)
Some system characteristics (indicators?)
1 buoyancy in the labour market for young people in their late teens, until the recent recession
2 a high level of ‘lifelong learning’ after cessation of compulsory schooling
3 decline in employer use of FE relative to the volume of employer-funded training
4 accompanying high flexibility in the structure and content of FE provision
5 persistence in structural skill shortages
6 collapse in provision of ‘classical apprenticeship’
7 the use of imported labour in skilled segments of the economy
8 partial success in the attempt to use HE as a dominant VET route (including variability in return)
9 poor resilience and responsive in the system in respect of offering places during a period of recession and high youth unemployment
Qualifications reform as a principal focus
Schools
Known locations of learning (schools)
Relatively easy to govern
Ownership clear – although parental/state responsibility an issue
Clear populations and cohorts
Legal obligation to attend
Solid research on pedagogy
Powerful instruments of curriculum control
VET (note HE VET ambiguity)
Diverse and shifting locations of learning (workplaces, colleges, PTPs)
Difficult to control, governance institutions unclear
Ownership of/responsibility totally unclear – employers, State, individual
Constantly shifting participation patterns
Complex patterns of requirements and incentives regarding participation
Very little work on workplace pedagogy
So qualifications reform will fix it? …. Unlikely
A reductive framework
QCF
- Notional learning time (size and weight)- Level- Sector
Rationale - Reduction in number of qualifications- Rationalisation of elements; new combination of common elements
Stimulated explosion in qualifications – 22,000? No….6,450 VQs
Increase in QCF category; no commensurate reduction in others
06-07 07-08 08-09 09-10 10-11
QCF 1000 1050 2480 6100 9700
Total 8150 9700 11500 15300 18100
‘Curriculum thinking’
Aims
Content
Methods
Assessment
Evaluation After Michael Eraut
Curricular and extra-curricular
Taught and untaught curriculum
Unstated curriculum (ethos, culture)
The constructed curriculum
Outcomes versus formation
Conceptual ground-clearing 1
The term ‘system’ – as in ‘education and training system’ is a metaphor
‘Systems-analysis’ is a theoretical and practical means of understanding the overall operation and affects of a diverse set of education and training arrangments
Should we aspire to manage these arrangements as a system – do we have the theoretical and practical tools to do this? Are we prepared for the constitutional consequences of a ‘systems management’ approach?
The implications of ‘systems-thinking’
Singapore and Korea – the characteristics of modern industrial strategy
UK attitude to industrial strategy
With the race for international investment going global, how should Britain address its manufacturing sector in order to boost the economy? And how much is this the government’s responsibility, rather than that of industry and businesses? Industrial strategy has been considered an unfashionable term in the past, with some unable to see beyond British industry being in a slow decline since the 1980s. This view could hamper the focus of policy-makers on manufacturing as crucial to better-balanced growth. Coupled with a devastating manufacturing skills gap, this should be carefully considered in the lead-up to the Budget.
Total politics Issue 56 March 2013
Conceptual ground-clearing 2
Implicit routes exist in seemingly ‘unified’ systems – you need to make choices of subject and focus
Seemingly ‘routed’ systems possess different levels of real flexibility – issues of outcomes (attainments), destinations and switching between routes
Conceptual foundation 1Situated cognition
Lave & Wenger GeeNewble & Clarke Seely Brown, Collins & DuguidSociolinguists - context and language acquisition BhatkinBrownBoreham and Samurcay
Anderson Reder & Simon (1998)
-issues of context –bound nature of learning, arguing that this is dependent on the precise content of knowledge being acquired
-issues regarding the extent to which abstraction is inappropriate for initial acquisition; iteration of abstract concepts and concrete exemplification is vital
-knowledge (and skill) is not resolutely context-bound; generalisation is possible
-context is characterised by complexity; there are issues of orientation and selective engagement
Specific skills associated with managing equipment, materials and compounds, measuring, observing, reporting
Levels of analysis
Generic issues of reliability
Occupational
Enterprise
Job role
Individual issues of currency
False distinction in that functional competence (held by individuals) can be described as generic but is necessarily constituted in an individual manner and is linked to the circumstances of its formation
Work process knowledge and distributed competence
Conceptual foundation 2Learning as well as outcomes
French VET system – ‘formation’ German Dual System – ‘Beruf’ English apprenticeship system – ‘timeserving’
‘Curriculum thinking’
Aims
Content
Methods
Assessment
Evaluation After Michael Eraut
Curricular and extra-curricular
Taught and untaught curriculum
Unstated curriculum (ethos, culture)
The constructed curriculum
Outcomes versus formation
‘Curriculum thinking’
Aims
Content
Methods
Assessment
Evaluation After Michael Eraut
Curricular and extra-curricular
Taught and untaught curriculum
Unstated curriculum (ethos, culture)
The constructed curriculum
Outcomes versus formation
The National Curriculum 1995
Chemical Reactions
i. that when chemical reactions take place, mass is conserved;j. that virtually all materials, including those in living systems, are
made through chemical reactions;k. to represent chemical reactions by word equations;l. that there are different types of reaction, including oxidation and
thermal decomposition;m. that useful products can be made from chemical reactions, including
the production of metals from metal oxides;n. about chemical reactions, e.g. corrosion of iron, spoiling of food, that
are generally not useful;o. that energy transfers that accompany chemical reactions, including
the burning of fuels, can be controlled and used;p. about possible effects of burning fossil fuels on the environment.
Science - key stage 3Materials and properties
National Curriculum 2007
Chemical and Material BehaviourIn their study of science, the following should be covered:
a. chemical change takes place by the rearrangement of atoms in substances;
b. there are patterns in the chemical reactions between substances;
c. new materials are made from natural resources by chemical reactions;
d. the properties of a material determine its uses.
Securing precision of the construct – science practical work
specific outcomes – the HE requirement for specific preparation – mastery?
persistence (Deakin Crick)
precision (Fotheringham)
methods of measurement
experimental design
experimental control and threats to validity
psychomotor co-ordination
principles and practices of safe working
ethics of investigation
observation
recording
teamworking and collaboration
‘beruf’
concepts of proof, adequacy, verification, falsification
The number of qualifications in England
Comparison with Germany 2005-2007
National Database of Accredited Qualifications (source: QCA-DfES) May 2007 5830 total
938 mainstream academic qualifications
4800 vocational and ‘other’ qualifications
Vocational qualifications in the German system (source: BIBB)
4070 total
150 FT State-specific qualifications
420 vocational qualifications in the Dual System of Apprenticeship (3yr employment-based programmes in which around 50% of the 16-19 cohort participate)
3,000 chamber of commerce qualifications
500 continuing education qualifications
Note – this is a careful comparison of ‘like with like’ – in neither case does it include industry-owned specific qualifications
The number of qualifications in England
My conclusion in 2006-07:
There is no basis for using 20,000 as the baseline figure for the English system. There is no compelling argument that the system is incoherent, compared with other benchmark nations
Comments collected from industry at that time: there are issues regarding gaps – particularly maths - and some duplication in information technology qualifications.
Rapid growth in NVQ2 care as a result of change of labour market requirement
Oxford work on Skills Survey 2000
Qualifications supply and combination
QCF implied combination and saliency
Policy maker: ‘There are hundreds which are not used…’
Me ‘So what?...they’re not used...’
Policy maker: ‘Remove all with fewer than 500 candidates…’
Me ‘what about the 50 who qualified in bomb disposal?...and…”
QCF lacked principles of combination
QCF lacked criteria of need (utility and saliency)
Material impediments to shared units
Relatively unfettered escalation in qualifications
Failure of principle or a failure of implementation?
Neurotic obsession with system tidiness.
Combination: purpose and needs
It is vital to differentiate:
•school-based VET (including work experience) as a component of compulsory general education
•initial VET in a full-time educational setting
•employment-based for young entrants to the labour market – focused VET for comprehensive labour market preparation
•continuing VET for adult employed workers
•VET for unemployed adults
‘Frameworks’ as a policy instrumentThe importance of higher order criteria
- Education and training ‘system’ is a metaphor- Qualifications arrangements - Qualifications frameworks
- Clear progression routes - Absence of inefficiencies- Absence of problems of knowledge and skill supply - High levels of recognition and trust- High levels of motivation and attainment
Frameworks have emerged as a relatively recent policy instrument
Understanding relations as well as objects
- Curriculum coherence - Curriculum control
- Explanatory factors- Control factors
Control factors
1 curriculum content (nc specifications, support materials, etc)2 assessment and qualifications3 national framework for qualifications4 inspection5 pedagogy6 professional development 7 institutional development8 institutional forms and structures (eg size of schools, education phases)9 allied social measures (linking social care, health care and education)10 funding 11 governance (autonomy versus direct control)12 accountability arrangements13 labour market/professional licensing 14 allied market regulation (eg health and safety legislation; insurance regulation)
Oates T 2010 Could do better: using international comparisons to refine the National Curriculum in England Cambridge Assessment
Curriculum control
A system must exercise control, it is not that individual agencies should take control:
‘…our purpose in introducing alternative ways to govern curriculum…is not to advocate one approach or another. As analysis by Cochran-Smith and Fries (2001) indicates, disagreements about teaching and, by implication, curriculum, often divides along ideological lines, an outcome that occurs no matter how pragmatic the veneer. A functional approach, by specifying in advance the criteria that an effective curriculum-governance system must meet, lessens the tendency to judge these systems in terms of the political values they represent (eg regulation vs deregulation, public interest vs private
interest…’
Schmidt W & Prawat R 2006 p656
Qualification frameworks as policy instruments
Key distinctions
Descriptive frameworks (accommodation – ‘relating’ frameworks
Prescriptive frameworks (change) – ‘realigning’ frameworks
30 levels in CINOP project
12 levels in Scotland – the rationality shown by empty cells
Functions
-showing relationships (level, ‘weight’, worth)
-indicating progression routes
-rationalising qualifications
Qualification frameworks as policy instruments
EQF – metaframework
Analytic function – labour market mobility as prime aim)
NQF – national framework
Developing equivalences and rationalising the system as awhole
QCF – formula for recognition in addition to framework
Reducing the number of qualifications, duplication and redundancy
Who is a framework for, exactly?
QCA website statement: ‘for learners to navigate the system’
Erosion of concept of formation
Ditch lessons, schools are toldGeraldine Hackett, Education Correspondent 24 06 2007
‘State secondary schools are being told to ditch lessons in academic subjects and replace them with month-long projects on themes such as global warming.
Quote from official “At the moment most schools are in the traditional mindset, which means they organise the day into a meaningful form take content and divide it up into fragments called timetables. They do it as it has always been done…The idea is to offer less education prescription and more opportunity to interpret the curriculum. Cutting across all subjects are curriculum dimensions; a set of themes including creativity, cultural understanding and diversity.”
Classical apprenticeship
Internal economics of classical apprenticeship Merits of long duration Formation through long duration Formation as a removal of the need to specify and assess externally Material efficiency of inflexibilities
Re-establishing curriculum thinking in general education as well as VET
QCF impacts and issues
High cost of QCF compliance
Small quals with genuine utility removed from the system
Some ABs forced out of business - undue market restriction
Material impediments to shared units
Relatively unfettered escalation in qualifications
Comments from contracted developers
Failure of principle or a failure of implementation?
Neurotic obsession with system tidiness.
Insights from German VET
External appearance of incoherence:
364 training lines (high labour market mobility)
Each set of standards revised on a sector basis (High saliency)
Qualification is associated with ‘formation’ not with certification
Restriction vital to the system
Low complexity in QA arrangements around qualifications
Social partnership essential to form of system (the institutional form of curriculum control)
(Reuling on modularisation 1995)
Is this for England? No naïve policy borrowing.
Modualisation and ‘outcomes orientation’
In general education
Greater clarity in learning outcomes
Early feedback on performance
Gains in motivation and engagement
Clear conceptual sequencing in material
But
A very significant increase in assessment load
A truncation of learning processes
Problems in establishing comparability
A narrow focus on assessment objectives, contributing to narrow instrumentalism in schools and colleges
Modualisation and ‘outcomes orientation’
In vocational education and training
Greater linkage between assessment and requirements of work
High specificity, precision and transparency in required outcomes
Validity leading to higher reliability
More flexible provision, reduced programme cost, meeting LM needs
Recognition of prior attainment
Qualifications capable of being used across all VET routes
But
Escalation in cost and complexity of quality assurance
Significant problems in reliability and comparability
Reducation of employers’ incentives to train (erosion of apprenticeship)
Failure to recognise more subtle ‘work process’ skills
The 40 functions Oates & Cole CEDEFOP ‘Changing Qualifications’ report (2010)
1 Documenting the outcomes of learning
3 Signalling
6 Controlling flows into specific occupations
7 Empowering citizens
8 Re-aligning control of professions
10 Measuring the performance of the education system
14 Guaranteeing the quality of provision, for funding agencies
16 Affecting the identity of learners
25 Influencing the content of learning programmes
35 Providing feedback to learners (formative and diagnostic function)
Post-16 Mathematics – an important case study
An emergent gap in qualification arrangements identified through domestic research and international comparisons
Research undertaken by King’s College London, sponsored by the Nuffield Foundation
An increase in the maths requirements of a broad range of science and social science degree courses
A level and AS qualifications too specialist and high level
GCSE repeat or retake demotivating, higher level, refined content required
‘Functional skills’ qualis inappropriate; higher level, refined content required
The NQF/QCF did not indicate or highlight deficit in qualifications ‘catalogue’ – specific research on needs and requirements was required
Concluding remarks 1 (of 3)
System tidiness is a misleading preoccupation
Vertical progression is vital; clarity in signalling and opportunity
Coherence and knowledge symmetry in sectors is vital
‘Level 2’ hairdressing, ‘Level 2 Engineering’ – why is this not immediately a scandal and a problem in the economy?
Labour market flexibility and skills supply vital – lessons from Germany
Bridging at key points in the system are critical
Real systems (‘arrangements’) are messy
Concluding remarks 2 (of 3)
High quality qualifications arrangements are not dependent on the existence of a qualifications framework
Dependence on qualifications frameworks can lead to over-simplistic approaches to the development of effective qualifications arrangements and effective learning provision – signs from the Richards Review
Multiple purposes for qualifications are likely – but curriculum coherence is vital; it requires sophisticated and evidence-based policy formation, management of ‘over determination’ and adjuvant policy, combined with subtle monitoring of impact and system performance
Concluding remarks 3 (of 3)
The assets of outcomes:
- drive to remove dysfunctional duplication - drives analysis of the composition of competence
But
- ‘level’ of analysis is crucial – theory matters - robust and explicit method and aim are vital - can encourage reductivist move from ‘formation’ and ‘beruf’ - can drive policy towards a consideration of ‘policy objects’ and a
neglect of ‘complex relations’