Seminar 6: Promoting use of Evidence and Data to Evaluate and Improve Closing the Gap in Educational Outcomes

Input 1: North Ayrshire Council

Assessment and Data Analysis Attainment Challenge Hampden 9th June 2016

Assessment & Data Analysis WorkstreamProject Objective• Teachers will raise attainment and achievement through innovative learning and teaching

approaches that are informed by the data that has been derived from their use of high quality assessments.

Aims• Develop a range of data analysis tools• Introduce improving teacher professional judgement programme• Establish a suite of baseline data to measure progress against• Form pan-Ayrshire families of schools to encourage more

structured collaboration, based on shared data• Introduce standardised assessments to support teacher

judgements• Increase parental and pupil involvement in data analysis to

inform planning

Project Benefits• Teachers will become more confident in their assessment and collection of data and in its use to

inform their practice.• There will be greater consistency across the authority in the content and quality of the information

that is being tracked and monitored, leading to improved understanding and knowledge of what works.

• Improved knowledge and application of socio-economic data within and across schools will lead to enhanced partnership working and planned targeted improvements being developed for those pupils whose needs are greatest.

• Teachers will undertake a learning programme that is designed to support them in improving their professional judgement and give them confidence in gathering and evaluating appropriate evidence, including from standardised assessments.

• The range and quality of information that is provided to pupils and parentswill be improved.

North Ayrshire Families of Schools

Factors included:

• Free Meal Entitlement• Footwear & Clothing Grant• Percentage in 5% most deprived datazones• Percentage in 10% most deprived datazones• Percentage in 15% most deprived datazones• Percentage in 20% most deprived datazones• Percentage in 30% most deprived datazones

Weighted rankings for each factor added.Four families of schools created based on this data – for targeting approaches to learning & teaching

Pivot Table Tools

Investigative tool for identifying vulnerable/other target groups of pupils in school

Notes:

The Pivot Tables are not very printer-friendly. You cannot edit the cells inside and it is very difficult to format it in any desired manner. If you wish to print out a certain table I advise to select and copy the desired table and "paste values" in a new sheet, where you can edit it as convenient, maybe adding a other information in separate columns or rows. Please note that you can also manipulate the rows and columns displayed in the pivot table as well (filter to certain categories only, hide certain rows/columns, change the order of the rows/columns or express the cells as numbers or percentage out of row/column/overall totals). You need, however, to develop a bit of familiarity with this facility in Excel to do that confidently. Excel Help files/tutorials will again be useful for that!

The SIMD ranking (decile, vigintile, etc.) for each pupil is found by matching up the pupil's postcode to the 6505 datazones in Scotland. SIMD deciles (vigintiles) are the 10% (5%) bands in which a certain datazone is ranked among the 6505 datazones in Scotland. E.g. SIMD decile 1 represents the 10% most SIMD deprived of datazones in Scotland. SIMD vigintile 1 represents the 5% most deprived of all datazones in Scotland. Please regard the SIMD decile for each pupil as complimentary information to the Free School Meals / Clothing Grant entitlement. While the SIMD ranking (or band) represents the level of deprivation in the area where the pupil lives, it does not necessarily imply that the pupil comes from a socially/economically deprived family. The personal level of deprivation is better described by the Free School Meals and Clothing Grant entitlement.

This spreadsheet is meant to give you easy access to your Pupil Census data using either filtering and sorting tools in the master sheet ("All pupils on the roll") or specially designed Pivot Tables that summarise the dataset by certain characteristics. Although you may construct your own Pivot Tables (using Excel Help files) I have constructed a few tables for you, using the most common pupil characteristics that will determine your target groups of vulnerable categories (e.g. Looked after children, pupils entitled to Free School Meals or Clothing Grants, pupils in certain SIMD deprived areas, geographical areas, previous school attended, etc.). Please check the notes below for other specific information and get in touch with Luoana Santarossa for any further assistance.

If you double-click in any of the cells inside the pivot table, a new sheet will pop up with the details of the pupils counted in that cell . You can delete the new sheet that popped up when you finish checking which pupils belong to that group, or you can copy and paste the list of names in another file if you will need it later. You are advised to keep the original file saved under the initial name and maybe make copies under different names if you wish to add additional sheets and/or tables that you find more

The "All pupils on the roll" sheet is edit-protected so that the initial database from the 2014 Census remains unchanged. Please note it does not include "Pupils on the roll of another school" and "Adults". It also provides only a pupil's main language spoken at home and the type (CSP, IEP, Other ASN) of ASN that a pupil may have, in order to prevent record duplication. For details on other languages spoken at home and on specific ASN needs, please use Custom Reports in Click+Go or ask Luoana Santarossa for further assistance.

Pivot Table Tools

Candidate Number School Name Forename Surname Gender

Year/Stage Date of Birth

Registration group Start Date

Student status Ethnic Origin

National identity

Asylum Status

Free Meals

In Receipt Of Clothing Grants Looked after Looked After By

090822578 St Matthew's Chevon Ballantyne F S1 08-May-03 1A 18-Aug-15 Pupil on roll o White - Scottish British No N090808931 St Matthew's Dean Black M S1 29-Apr-03 1A 18-Aug-15 Pupil on roll o White - Scottish Scottish No N

090821253St Matthew's Academy

Megan Elizabeth Campbell F S1 14-Jan-04 1A 18-Aug-15 Pupil on roll o White - Scottish Scottish No N

091046377 St Matthew's David Clyde M S1 01-Dec-03 1A 18-Aug-15 Pupil on roll o White - Scottish Scottish No N Looked after aw North Ayrshire090808966 St Matthew's John Cook M S1 03-Sep-03 1A 18-Aug-15 Pupil on roll o White - Scottish Scottish No N151073190 St Matthew's Philipp Euler M S1 07-Aug-02 1A 18-Aug-15 Pupil on roll o White - Other Other No N091035707 St Matthew's Shane Hampson M S1 14-Nov-03 1A 18-Aug-15 Pupil on roll o White - Other Scottish Yes Y090799916 St Matthew's Kian Harkins M S1 18-Dec-02 1A 18-Aug-15 Pupil on roll o White - Scottish Scottish Yes Y090809709 St Matthew's Tyler James Hollywood M S1 14-Nov-03 1A 18-Aug-15 Pupil on roll o White - Scottish Scottish Yes Y090822756 St Matthew's Abby Leckie F S1 10-Jul-03 1A 18-Aug-15 Pupil on roll o White - Scottish Scottish No N090821296 St Matthew's Madelaine Macmillan F S1 22-Nov-03 1A 18-Aug-15 Pupil on roll o White - Scottish Scottish No N090809768 St Matthew's Molly McLaughlin F S1 10-Sep-03 1A 18-Aug-15 Pupil on roll o White - Scottish Scottish No N

090799215St Matthew's Academy Alex David

Montgomery M S1 11-Sep-03 1A 18-Aug-15 Pupil on roll o White - Scottish Scottish No N

Pivot Table Tools

SIMD decile (10% band) Distribution - percentage

School Name St Matthew's AcademyLooked after (All)Free Meals (All)Intermediate Geography (All)Staged Intervention Level (All)

Count of Candidate Number SIMD DECILEYear/Stage 1 2 3 4 5 6 7 8 9 Grand TotalS1 16.95% 30.93% 19.07% 5.93% 2.12% 2.54% 8.90% 9.75% 3.81% 100.00%S2 16.43% 23.47% 23.94% 5.16% 1.88% 7.51% 13.62% 5.63% 2.35% 100.00%S3 17.78% 24.44% 20.00% 2.22% 4.00% 5.78% 12.89% 7.11% 5.78% 100.00%S4 22.13% 26.88% 15.81% 9.88% 2.77% 5.14% 9.09% 5.14% 3.16% 100.00%S5 18.32% 21.47% 20.42% 3.14% 2.62% 8.90% 14.14% 7.33% 3.66% 100.00%S6 19.23% 20.51% 16.67% 3.85% 2.56% 6.41% 11.54% 12.82% 6.41% 100.00%Grand Total 18.52% 25.04% 19.31% 5.26% 2.67% 5.89% 11.54% 7.69% 4.08% 100.00%

Double click in the cell to list all pupils in

S2 living in decile 7

Filter to the desired pupil category

You can change settings to view numbers (count) or averages (e.g. for

standardised score results)

Percentage can be taken out of row total, column total or entire

table total

You can pick row/column variables from all the variables in the initial

spreadsheet

Standardised AssessmentsSuite of assessments available to all schools:

CAT 4 GoldNew Group Reading TestProgress Tests in MathsProgress Test in EnglishProgress Test in Science

P1 BaselineSingle Word Spelling Test

Dyscalculia ScreenerDyslexia Screener

PASS surveys

New Group Reading Test

North Ayrshire Requirements:

P1 baseline

P3, P6 and S2 – Progress Test in Maths

P3, P6 and S2 – Progress Test in English

Leading Learning in Summative Assessment

Improving Teacher Professional Judgement

2016 - 2017

North Ayrshire Council

Purposes of the ITPJ Programme• To establish or further establish Teacher Learning

Communities (TLCs) as an effective professional learning model for improvement in schools;

• To support establishments in the development of coherent and progressive approaches to assessment for CfE;

• To support the development of the processes of summative assessment and moderation.

13

Purposes contdTo give confidence to teachers in:a) embedding formative practice in ongoing every day assessment;b) improving learner involvement fully in assessment processes;c) improving the planning of units / topics to ensure coherence in

teaching, learning and assessment;d) improving approaches to the gathering of appropriate assessment

evidence and using it for summative judgements; ande) improving teacher capacity to agree standards and participate in

moderation.

14

Date Session Summary

Support Session 3

1. How's it going with your planning for Assessment - feedback from tasks from session 2

2. Reflection of and planning for TLC Workshop 3 3. Personal planning for Support Session 4

Workshop 3 late JanuaryAssessment Design - selecting the task - identifying, adapting and designing effective assessment tasks

Support Session 4

1. Feedback - evidence from the task 2. Getting started with your TLC

3. Reflection of and planning for TLC Workshop 4 4. Personal planning for Support Session 5

Workshop 4 mid FebruaryAssessment Design - agreeing the success criteria for our assessment tasks - towards agreeing and sharing

standards

Support Session 5 1. Feedback - how's it going with your TLC?

2. Reflection of and planning for TLC Workshop 5 3. Personal planning for Support Session 6

Workshop 5 early MarchEvaluating the evidence generated by our planned

assessment tasks - agreeing and sharing standards

Masterclass with Prof Dylan Wiliam 23rd March 2016

Assessment (and what makes it good) Assessing - Task selection and presentation Evidence identification and accumulation

Recording and Reporting Putting it all together

Workshop 1 early November

Assessment for Curriculum for Excellence - current key messages - where am I now in relation to the key

criteria for effective assessment and agreeing and sharing standards

TLCTLC

Workshop Date

Outline of New Learning…Let's talk about…and tasks for Leaders between sessions

April Putting it all Together - accumulating the evidence Reviewing our summative assessment processes

Workshop 2

Sessions for TLC Leaders

Support Session 6

1. Feedback - how's it going with your TLC? 2. Reflection of and planning for TLC Workshop 6

3. Self-evaluation and future planning 4. Programme Evaluation

TLC Workshops in schools / educational establishments

Support Session 1

1. An introduction to the programme 2. Getting started with your TLC

3. Self-evaluation - Summative Assessment where am I? 4. Reflection of and planning for TLC Workshop 1

5. Personal planning for Support Session 2

late November/

early December

Planning for Assessment - how effective is our planning for formative and summative assessment?

Workshop 6

Support Session 2

1. How's it going with your planning for Assessment - feedback from tasks from session 1

2. Reflection of and planning for TLC Workshop 2 3 Personal planning for Support Session 3

Support Session 7 1. Individual presentations

2. Reflections on improvements 3. Next steps - plans for further improvement

7

North Ayrshire Progress Tracker

Next Steps

• The North Ayrshire Progress Tracker will now be used to generate professional dialogue within and across schools and as the basis of teacher-pupil learning conversations when planning next steps.

• Professional learning sessions will be organised for class teachers on incorporating data and feedback from standardised assessments more effectively into the planning for learning process.

• A full suite of data will be agreed and brought together, based on both universal and targeted approaches to raising attainment for all and for identified groups, to support schools more coherently with the robust evaluation of the various measures that are put in place.

• Data analyst and graduate intern will work with schools to support staff groups with their approaches to effective use of data.

• Further professional learning sessions (for an increasing number of staff) on maximising the use of data, what it tells us about what works, and how it contributes to planning for improvement.

• Early identification of vulnerable groups / SIMD 1,2 etc. at Early Years stage. Combining data from early numeracy and literacy assessments and developmental milestones data to plan a range of early interventions focused on closing the gap.

• Professional learning sessions will take place focusing on a shared understanding of standards in relation to CfE levels. This will be complemented by an enhanced approach to moderation across establishments.

GLASGOW PSYCHOLOGICALSERVICES – EVALUATINGNURTURING APPROACHESPromoting use of evidence and data to evaluate and improve closing the gap in educational outcomes Jenni Kerr (Senior Educational Psychologist)

Jenni Kerr 2016

IN TEACHING, ‘YOU CAN’T DO THE BLOOM STUFF UNTIL YOU TAKE CARE OF THE MASLOW STUFF’. ALAIN J. BECK

Create

Evaluate

Analyse

Apply

Understand

Remember

Jenni Kerr 2016

What Works?

Effective InterventionsEvidenced Based Practice

Effective Implementation

MethodsImplementation science

Enabling ContextsNurturing

Approaches

SOCIALLY SIGNIFICANT OUTCOMES

Jenni Kerr 2016

NURTURING APPROACHES IN GLASGOW

Nurture groups - primary schools Nurture corners – early years Nurture bases – secondary Whole school approaches – all

Jenni Kerr 2016

• 2001 Pilot of a nurture group in Anderston Primary

• 2010 68 nurture groups in primary schools• 2012 20 nurture corners in Early Years

establishments • 2012/13 All establishments received training on

the Nurturing Principles • 2015 14 Nurture Bases in secondary schools

3 Enhanced Nurturing Provision Units iin primary schools

Glasgow’s Story

5

Jenni Kerr 2016

Nurture group research • Evaluation – comparing pupils from 16 nurture

group schools and 16 control schools• Measures – baseline assessment (basic literacy),

Boxall profiles, SDQ, BIOS, Parentalquestionnaire

• Hypothesis– Pupils who attend NGs will show greater

improvement in emotional and behaviouralfunctioning (Boxall & SDQ)

– Pupils who attend NGs will show greaterimprovements in attainment (Baseline assessment)

– Pupils who attend NGs will show greaterimprovement in self esteem (BIOS)

Jenni Kerr 2016

Key findingsYoung people did significantly better:• Improved behaviour• Improved social & emotional wellbeing• Improved academic attainment especially

early literacy• A significant difference both in school and

wider settingReynolds, MacKay & Kearney (2009)

• Parents rated their children significantlymore positively after attending a NG (March &Healy, 2007)

Jenni Kerr 2016

Longitudinal effects of nurture groups

• The majority (89.5%) of young people that receivedthe nurture group intervention in 2007 remain inmainstream provision as at 2014

• Exclusion levels were low. 86.8% of the nurturegroup cohort had no exclusions

• Attendance levels were high. 87.9% attendanceopenings on average

• Academic attainment results were encouragingwithin the small sample of pupils old enough tohave sat national exams. 66.7% of these pupilsachieved a pass at Mathematics and EnglishSQA Level 3 or above.

Jenni Kerr 2016

SECONDARY RESEARCH

Aims To test the effectiveness of nurture groups in

secondary schools. To investigate the fidelity of implementation of

nurture groups in secondary schools.Measures pre and post Boxall profiles, pre and post SDQ

scores and contextual information inclusive of attendance and DHT referrals

Jenni Kerr 2016

KEY FINDINGS

The intervention has had a clear and statistically significant effect on almost all developmental strands

Supportive implementation included:- identification at transition, - access to training, - whole team approach, - focus on building relationships, - senior management support, - nurture policy in place

Jenni Kerr 2016

NEXT STEPS FOR SECONDARY RESEARCH

Incorporating more quantitative measures such as pre and post SDQ scores

looking further into the differences of implementation across secondary schools and the effect of these differences to establish what makes nurture in secondary schools successful.

Jenni Kerr 2016

Towards the Nurturing school

1. Children and young people’s learning is understood developmentally2. The classroom offers a safe base3. The importance of nurture for the development of resilience 4. Language is a vital means of communication5. All behaviour is communication6. The importance of transition in young people’s lives

Jenni Kerr 2016

WHOLE SCHOOL EVALUATION OF THENURTURING PRINCIPLES

All establishments trained in nurturing principles

All staff completed the pulsion survey (EVA) to evaluate nurture

More in depth evaluation using How Nurturing Is Our School (HNIOS)

- parent, pupil, teacher questionnaires and focus groups

Jenni Kerr 2016

Completion of a needs analysis

Introductory session

Consulting with stakeholders

Feedback and analysis

Improvement plan

Implementation plan for schools

Jenni Kerr 2016

EVALUATION OF QUALITY

Pre and post measures – attainment, relationships, emotional development

Quantitative measures Questionnaires Attendance/exclusion data Attainment

Qualitative measures (people’s views and direct observation) Focus groups Observations Questionnaires with evaluative questions Interviews with staff

Jenni Kerr 2016

METHODOLOGICAL CONSIDERATIONS

Have a clear implementation plan School readiness Establish a baseline

- pre and post measures should be used Develop clear questions

- Ensure the evaluation method will answer your questions Improvement plan Control group? Nurture is about building positive relationships

– think about impact on teachers and parents as well as pupils

Jenni Kerr 2016

• Any Questions ?

Jenni Kerr 2016