Upload
rcu
View
1.108
Download
1
Embed Size (px)
Citation preview
RCU, 3 Tustin Court, Port Way, Preston, PR2 2YQTel: 01772 734855 email: [email protected]
AoC Teaching, Learning and Assessment Conference
Understanding and improving Level 3 value added performance
19thth January 2016
Presented by Richard Boniface (RCU Ltd)
Martin Laverick (Harlow College)
RCU, 3 Tustin Court, Port Way, Preston, PR2 2YQTel: 01772 734855 email: [email protected]
Session overview
1. Introduction
2. Feedback on analysis of national level data
3. Value added at Harlow College – making it work in practice
4. Activity and discussion
RCU, 3 Tustin Court, Port Way, Preston, PR2 2YQTel: 01772 734855 email: [email protected]
Two new progress measures:
Value added measure for Academic and Applied General qualifications –based on L3VA
Completion and Attainment measure for Tech Levels and Technical Certificates (at Level 2)
Both measures will show the progress of students compared to others taking the same subject nationally
In addition:
the value added measure will compare students with the same starting points
The completion and attainment measure will treat non completion as a fail.
These measures will form the basis for new progress based minimum standards
New Progress Measures (2016)
RCU, 3 Tustin Court, Port Way, Preston, PR2 2YQTel: 01772 734855 email: [email protected]
Analysis of National Level 3 Value Added Data
Richard BonifaceRCU Ltd.
RCU, 3 Tustin Court, Port Way, Preston, PR2 2YQTel: 01772 734855 email: [email protected]
• Level 3 value added score for each qualification taken by a learner is:
Actual Level 3 performance MINUS Predicted Level 3 performance
• The predicted score is derived from a statistical model based on the performance of other learners who took the same qualification and had the same Key Stage 4 points score;
• The published data aggregates learner level data to give scores by qualification, qualification type and institution (academic and vocational);
• The baseline Key Stage 4 score was calculated at age 15 for 16,17 year old completers but age 16 for 18 year old completers.
Analysis of Level 3 Value Added Data
RCU, 3 Tustin Court, Port Way, Preston, PR2 2YQTel: 01772 734855 email: [email protected]
Academic Value Added Scores
Provider TypeNumber of
EntriesValue Added Score
Lower
Confidence
Limit
Upper
Confidence
Limit
Academy 358,046 0.03 0.03 0.03
Sixth form college 213,704 0.00 0.00 0.01
Independent school 124,777 0.12 0.12 0.12
Community school 91,557 -0.02 -0.02 -0.01
Voluntary aided school 59,208 0.01 0.00 0.01
General further education college 45,489 -0.13 -0.14 -0.13
Foundation school 43,728 -0.01 -0.02 -0.01
Sponsored academy 37,164 -0.09 -0.10 -0.09
Tertiary college 32,349 0.01 0.00 0.01
Voluntary controlled school 9,149 -0.01 -0.02 0.00
City technology college 1,316 -0.04 -0.06 -0.01
Free school - mainstream 772 -0.09 -0.11 -0.06
Agriculture and Horticulture College 701 -0.10 -0.13 -0.07
Free school - 16-19 628 0.30 0.26 0.34
Free school - UTC (university technology college) 202 -0.26 -0.33 -0.19
Art, design and performing arts college 183 -0.05 -0.12 0.02
Free school - studio school 102 -0.24 -0.31 -0.17
RCU, 3 Tustin Court, Port Way, Preston, PR2 2YQTel: 01772 734855 email: [email protected]
Academic Value Added Scores
0%
10%
20%
30%
40%
50%
60%
>-1
and
<=-0
.8
>-0.
8 an
d <=
-0.6
>-0.
6 an
d <=
-0.4
>-0.
4 an
d <=
-0.2
>-0.
2 an
d <=
0
>0 a
nd <
=0.2
>0.2
and
<=0
.4
>0.4
and
<=0
.6
>0.6
and
<=0
.8
>0.8
and
<=1
Perc
enta
ge o
f Pro
vide
rs in
Val
ue A
dded
Ban
d
Value Added Score
All
Academy
Sixth Form College
Community School
GFE/Tertiary
RCU, 3 Tustin Court, Port Way, Preston, PR2 2YQTel: 01772 734855 email: [email protected]
Vocational Value Added Scores
Provider TypeNumber of
EntriesValue Added Score
Lower
Confidence
Limit
Upper
Confidence
Limit
General further education college 73,750 -0.14 -0.14 -0.14
Academy 24,571 0.16 0.15 0.16
Sixth form college 22,017 0.14 0.13 0.14
Tertiary college 16,685 0.04 0.03 0.05
Sponsored academy 14,480 0.08 0.07 0.09
Community school 11,606 0.14 0.13 0.15
Foundation school 6,129 0.14 0.12 0.15
Voluntary aided school 4,837 0.20 0.19 0.21
Agriculture and Horticulture College 3,207 0.03 0.01 0.04
Voluntary controlled school 694 0.11 0.08 0.14
Independent school 611 0.22 0.19 0.26
Art, design and performing arts college 561 0.37 0.33 0.40
General further education college (special) 536 0.03 -0.02 0.07
City technology college 532 0.38 0.35 0.42
Free school - UTC (university technology college) 168 0.31 0.25 0.38
Free school - studio school 104 0.27 0.19 0.36
Free school - mainstream 13 0.32 0.08 0.56
RCU, 3 Tustin Court, Port Way, Preston, PR2 2YQTel: 01772 734855 email: [email protected]
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
>-1
.6 a
nd
<=
-1.4
>-1
.4 a
nd
<=
-1.2
>-1
.2 a
nd
<=
-1
>-1
an
d <
=-0
.8
>-0
.8 a
nd
<=
-0.6
>-0
.6 a
nd
<=
-0.4
>-0
.4 a
nd
<=
-0.2
>-0
.2 a
nd
<=
0
>0
an
d <
=0
.2
>0
.2 a
nd
<=0
.4
>0
.4 a
nd
<=0
.6
>0
.6 a
nd
<=0
.8
>0
.8 a
nd
<=1
>1
an
d <
=1
.2
>1
.2 a
nd
<=1
.5
Pe
rce
nta
ge o
f P
rovi
de
rs in
Val
ue
Ad
de
d B
and
Value Added Score
All
Academy
Sixth Form College
Community School
GFE/Tertiary
Vocational Value Added Scores
RCU, 3 Tustin Court, Port Way, Preston, PR2 2YQTel: 01772 734855 email: [email protected]
VA Scores by Qualification Type (GFE)GFE/Tertiary
Qualification Type
Number of
EntriesValue Added Score
Lower
Confidence
Limit
Upper
Confidence
Limit
A Level 53,197 -0.06 -0.06 -0.05
EDEXCEL BTEC Level 3 Extended Diploma 47,842 -0.05 -0.06 -0.05
AS Level (Not Continued to A2) 21,698 -0.03 -0.03 -0.02
Edexcel BTEC Level 3 90-credit Diploma 15,107 -0.05 -0.06 -0.04
EDEXCEL BTEC Level 3 Subsidiary Diploma 10,726 -0.29 -0.30 -0.28
EDEXCEL BTEC Level 3 Diploma 6,764 -0.33 -0.34 -0.32
CACHE Level 3 Diploma in Child Care and Education 1,809 -0.06 -0.08 -0.03
BTEC National Diploma 1,700 -0.13 -0.15 -0.10
Extended Project (Diploma) 1,433 -0.01 -0.04 0.02
EDEXCEL BTEC Level 3 Certificate 1,046 -0.13 -0.15 -0.10
Level 3 Diploma (ABC or UAL) 615 0.12 0.07 0.16
Applied GCE Single Award 610 -0.08 -0.11 -0.04
CACHE Level 3 Extended Diploma 498 -0.00 -0.03 0.02
UAL Level 3 Extended Diploma 483 -0.15 -0.20 -0.09
L3 Foundation Diploma in Art and Design 413 -0.05 -0.11 -0.00
UAL Level 3 Diploma 315 0.00 -0.06 0.06
City & Guilds Level 3 Extended Diploma 291 0.04 0.01 0.08
IFS Level 3 Diploma in Financial Studies 258 -0.28 -0.33 -0.23
IFS Level 3 Certificate 236 -0.05 -0.10 0.01
City & Guilds Level 3 Diploma 196 0.08 0.01 0.14
CACHE Level 3 Certificate in Child Care and Education 185 0.04 -0.02 0.10
Pearson BTEC Level 3 Diploma in Beauty Therapy (QCF) 173 -0.05 -0.11 0.01
Applied GCE Double Award 172 -0.37 -0.43 -0.31
Applied GCE AS level 166 -0.04 -0.10 0.02
Level 3 Subsidiary Diploma (RSL or City & Guilds) 155 0.16 0.10 0.22
International Baccalaureate 139 0.00 -0.05 0.06
IFS Level 3 Certificate in Financial Studies 96 0.24 0.14 0.34
ABC Level 3 Foundation Diploma 74 0.05 -0.08 0.18
BTEC National Award 73 -1.31 -1.41 -1.20
OCR Level 3 Cambridge Technical Extended Diploma 70 -0.49 -0.60 -0.38
BTEC National Certificate 64 0.54 0.41 0.67
City & Guilds Level 3 Subsidiary Diploma 54 -0.02 -0.10 0.05
OCR Level 3 Cambridge Technical Introductory Diploma 30 -0.09 -0.25 0.08
Pre-U Principal Subject 16 -0.05 -0.21 0.11
Pearson BTEC Level 3 National Award 15 0.28 0.09 0.48
OCR Level 3 Cambridge Technical Diploma 13 0.19 -0.02 0.41
Free Standing Maths 8 0.24 -0.15 0.64
OCR National Certificate 7 -0.97 -1.27 -0.67
RCU, 3 Tustin Court, Port Way, Preston, PR2 2YQTel: 01772 734855 email: [email protected]
VA Scores by Qualification Type (SFC)
Qualification TypeNumber of
EntriesValue Added Score
Lower
Confidence
Limit
Upper
Confidence
Limit
A Level 139,358 0.02 0.01 0.02
AS Level (Not Continued to A2) 58,852 -0.01 -0.01 -0.00
Extended Project (Diploma) 8,799 0.10 0.07 0.13
EDEXCEL BTEC Level 3 Subsidiary Diploma 7,583 0.17 0.16 0.17
EDEXCEL BTEC Level 3 Extended Diploma 7,148 0.15 0.14 0.15
Applied GCE Single Award 4,330 0.13 0.09 0.16
EDEXCEL BTEC Level 3 Diploma 3,639 0.20 0.19 0.20
Applied GCE AS level 1,205 0.11 0.05 0.17
EDEXCEL BTEC Level 3 Certificate 1,078 0.18 0.16 0.21
Applied GCE Double Award 882 0.09 0.04 0.15
Edexcel BTEC Level 3 90-credit Diploma 792 0.17 0.16 0.17
IFS Level 3 Diploma in Financial Studies 377 0.18 0.13 0.23
BTEC National Diploma 283 0.36 0.34 0.39
CACHE Level 3 Diploma in Child Care and Education 183 0.16 0.14 0.19
International Baccalaureate 153 0.01 -0.04 0.07
OCR Level 3 Cambridge Technical Introductory Diploma 149 0.03 -0.14 0.19
OCR National Diploma 146 -0.12 -0.12 -0.12
OCR Level 3 Cambridge Technical Diploma 104 -0.11 -0.32 0.10
City & Guilds Level 3 Diploma 43 -0.06 -0.12 0.01
IFS Level 3 Certificate 36 0.34 0.28 0.39
BTEC National Award 33 0.71 0.61 0.82
UAL Level 3 Extended Diploma 33 0.24 0.18 0.29
IFS Level 3 Certificate in Financial Studies 31 0.66 0.56 0.76
L3 Foundation Diploma in Art and Design 26 0.23 0.18 0.28
Pearson BTEC Level 3 National Award 25 0.20 0.01 0.40
Level 3 Diploma (ABC or UAL) 16 0.02 -0.02 0.07
Free Standing Maths 12 -1.27 -1.66 -0.87
CACHE Level 3 Extended Diploma 10 0.41 0.39 0.44
BTEC National Certificate 7 -0.72 -0.85 -0.59
City & Guilds Level 3 Subsidiary Diploma 7 0.17 0.10 0.24
ABC Level 3 Foundation Diploma 7 0.31 0.19 0.44
RCU, 3 Tustin Court, Port Way, Preston, PR2 2YQTel: 01772 734855 email: [email protected]
Academic Vocational
College Value Added Scores by Region
-0.30 -0.20 -0.10 0.00 0.10 0.20 0.30
North East
West Midlands
East Midlands
London
Yorkshire and The Humber
South East
East of England
North West
South West
Value Added Score
-0.30 -0.20 -0.10 0.00 0.10 0.20 0.30
London
Yorkshire and The Humber
East Midlands
West Midlands
East of England
North East
South West
South East
North West
Value Added Score
RCU, 3 Tustin Court, Port Way, Preston, PR2 2YQTel: 01772 734855 email: [email protected]
Value Added Scores by Deprivation
Academic Vocational
-1.5
-1.0
-0.5
0.0
0.5
1.0
1.5
0 10 20 30 40 50 60
Val
ue
Ad
ded
Sco
re
IMD 2010 16-18 Classbased L3 Deprivation Score
GFE / Tertiary College
Sixth form college
Land-based/Art andDesign Colleges
Least Deprived Most Deprived
-1.5
-1.0
-0.5
0.0
0.5
1.0
1.5
0 10 20 30 40 50 60
Val
ue
Ad
ded
Sco
re
IMD 2010 16-18 Classbased L3 Deprivation Score
GFE / Tertiary College
Sixth form college
Land-based/Art andDesign Colleges
Least Deprived Most Deprived
RCU, 3 Tustin Court, Port Way, Preston, PR2 2YQTel: 01772 734855 email: [email protected]
Value Added Scores by OFSTED Grades
Academic Vocational
-0.20 -0.10 0.00 0.10 0.20
Outstanding
Good
Satisfactory/Requires Improvement
Inadequate
Value Added Score
-0.20 -0.10 0.00 0.10 0.20
Outstanding
Good
Satisfactory/Requires Improvement
Inadequate
Value Added Score
RCU, 3 Tustin Court, Port Way, Preston, PR2 2YQTel: 01772 734855 email: [email protected]
• Schools were withdrawing students who they expected to do badly, improving their headline Value Added scores;
• Level 3 Value Added measure does not take into account the impact of deprivation;
• The lack of a common Key Stage 4 baseline year disadvantages GFEs with 18 year olds and does not show the true value that colleges add;
• Statistical model may not be accurate for learners with relatively low KS4 scores;
• Small sixth forms benefit from a ‘shrinkage factor;’
• Value Added scores included qualifications taken at other institutions.
Issues Raised from Analysis
RCU, 3 Tustin Court, Port Way, Preston, PR2 2YQTel: 01772 734855 email: [email protected]
• Common baseline introduced - only specific qualifications achieved during Key Stage 4 will be included;
• New methodology for predicting scores to be used which should be more accurate for students with low prior attainment (20 bands rather than multi-level modelling);
• Shrinkage factor removed;
• Introduction of a separate retention measure;
• Progress measure only applies to Academic and Applied General qualifications;
• Level 3 value added scores will be reported separately for each provider if a learner changes institution.
Changes for 2016
RCU, 3 Tustin Court, Port Way, Preston, PR2 2YQTel: 01772 734855 email: [email protected]
Harlow College
Martin LaverickDirector of Quality Standards and Assessment
RCU, 3 Tustin Court, Port Way, Preston, PR2 2YQTel: 01772 734855 email: [email protected]
• Values around
– A Levels +0.30
– Vocational +0.30
• A brief history
– Starting Point – Sport
Harlow College Experience
RCU, 3 Tustin Court, Port Way, Preston, PR2 2YQTel: 01772 734855 email: [email protected]
RCU, 3 Tustin Court, Port Way, Preston, PR2 2YQTel: 01772 734855 email: [email protected]
• Discuss and write down three things that are stopping your organisation from achieving high value added
– Caveat – these must be internal factors
• By 9 minutes each table should list these factors on separate post-it notes (min of 3)
• At 10 minutes stick them on the wall.
Activity
RCU, 3 Tustin Court, Port Way, Preston, PR2 2YQTel: 01772 734855 email: [email protected]
• Set a challenging target grade for the student based on KS4/GCSE results
– Predicted plus one?
– Target based on data from DfE, ALIS, ALPS, etc.
– Does the accuracy of the target matter?• Don’t get hung up on the correlation at individual student level.
– New Assessment Rules QCF and NQF Edexcel
• Negotiate a student target
Our Process
RCU, 3 Tustin Court, Port Way, Preston, PR2 2YQTel: 01772 734855 email: [email protected]
• Are your tutors/lecturers and managers held accountable?
– How?
• Do you have a central tracking system?
– Advice - have something simple that works
• In year progress– Which groups underperformed last year.
– Where are they are now?
– Can you predict the influence of other changes?
Monitor Student Progress
RCU, 3 Tustin Court, Port Way, Preston, PR2 2YQTel: 01772 734855 email: [email protected]
What evidence?
RCU, 3 Tustin Court, Port Way, Preston, PR2 2YQTel: 01772 734855 email: [email protected]
At student level?
Note: the headings for the
third and fourth columns
should be:
Personal Target Points
Harlow College Target Points
RCU, 3 Tustin Court, Port Way, Preston, PR2 2YQTel: 01772 734855 email: [email protected]
• Staff
– Teaching to the higher grade
– Not accepting the “just pass”
– Other reasons
• Students
– The basics• Attendance (>85% will cost you a grade)
– For every negative VA.
Changing Mindsets
RCU, 3 Tustin Court, Port Way, Preston, PR2 2YQTel: 01772 734855 email: [email protected]
• Daily targets that build on prior knowledge, are personalised/differentiated, and include support and intervention, as well as, stretch and challenge.
• Unit Days - students go home when s/he has met the differentiated SMART target(s).
One Approach
RCU, 3 Tustin Court, Port Way, Preston, PR2 2YQTel: 01772 734855 email: [email protected]
• Data/Data Capture
• Data Arrives too late
• Teachers don’t understand
• Not understanding what a distinction profile like
• Lack of Stretch and Challenge/personalised targets
• Vocational teachers understanding of calculated
• Value Added high achievers
• Too much focus on success – lack of aspirations
• Feedback on work doesn’t help learners improve
• Not collecting quals on entry
Summary and Questions