Strengthening Teacher Assessment Practices,
Learning and Evidence
Paul Blackwith Chris Harrison, Jeremy Hodgen,
Bethan Marshall & Natasha Serret
King’s College London
2
Reasons Behind Project
• Success of formative assessment projects • Current high stakes testing inhibits teaching
and learning (Harlen & Deakin-Crick, 2004, ARG) • Little support given to teachers in relation to
continuous summative assessment
• Changes in Scotland,Wales, Jersey and N. Ireland
3
KOSAP
• King’s-Oxfordshire-Summative-Assessment Project
• Investigate & strengthen teachers’ summative assessment practices
• Working in three schools well versed in FA• English & Mathematics departments• Head of department + interested teacher• Year 8 (13 year olds)
4
Formative Assessment
• This is classroom assessment which focuses on the learning as it is taking place and its function is to bring about improvement
• Characterised by Teacher-Pupil and Pupil-Pupil interactions, self-assessment, diagnosis, and contingency.
5
Summative AssessmentPurposes
Decisions about teaching setsInformation for the next teacherReporting to Senior Management TeamReporting to ParentsTarget setting
FA also has a role to play in these.
6
Phases of KOSAP
• Phase 1 (January 2005-July 2005)– Pilot– Investigating existing practice– Evaluating & devising new assessments
• Phase 2 (September 2005-July 2006)– Trial– Adapting, developing and implementing reformed assessment practices
• Phase 3 (September 2006-July 2007)– Implementation in departments
7
Focus
• Teacher’s literacy, skills and values of assessment
• Whether summative tasks could be selected/designed that were acceptable in terms of reliability and validity
8
Data sources 1
• summative assessments;
• fieldnotes/audio recordings of all project meetings;
• fieldnotes/audio recordings of intra-school standardisation and moderation meetings, as well as inter-school meetings preparing for these;
• classroom observations of alternative summative assessment events;
9
Data Sources 2
• regular interviews with the “12” core teachers;• focus group interviews with subject and
school teacher groups;• teachers’ writing and reflections, in particular
teachers’ reflective diaries;• interviews with teachers in departments;• pupil questionnaires
10
Research Questions• What do the summative assessment
practices of FA teachers look like?• How different are the SA practices between
English and Maths for these teachers?• How do the processes of moderation support
teacher learning about SA?• How can these ideas be implemented across
departments?• Can summative assessment tasks be used
without detrimental effects on FA and students?
11
Maths SA Practices
Maths teachers used end-of-topic tests.
Derived questions from external examinations & textbooks
Teachers had not considered quality of tests nor how appropriate these tests were for assessing specific skills and conceptual understanding.
12
Realism?(2)
I am going to target those things. If you can do these things, it is going to radically improve the probability that you are going to get a C. Now that’s not necessarily making them a better mathematician, but that is saying if you practice these skills and learn to do these skills and apply them on that exam paper, you are more likely to get a C which will then enable, will open doors for you beyond Yr11 here
13
Findings
Mismatch and confusion between purposes of assessment
We are caught in a trap. Report to parents a level. Students focus on just doing tests so we have a level to report. We have missed the point of what we are trying to achieve. It is less and less with monitoring progress and more about filling in boxes.
14
Findings• Teachers challenged their own existing and
external summative assessments
Maybe that’s what the investigation is trying to address. Think that these investigations have become too formalised. Something that lets them have a more creative element in their work and rewards kids that try things in a different way, who are willing to take risks with their maths
15
Sampling concerns:
I’d like to look over all students’ shoulders all the time
I want to know everything really
BUT as recipients they wanted:
a broad-brush level plus anything unusual
16
First Findings
• Maths teachers look at classwork through an assessment lens & try to hone approach
• Maths teachers began to pilot and trial alternative assessment tasks
• Tests would still feature and need improvement through professional input
Maths Portfolio
Design of six tasks, aimed for:curriculum cover
flexibility and comparability.
3 common tasks: one in each of number & algebra, space& shape, data handling;all to involve using and applying mathematics.(each of the three schools designed one of these)
3 non-common tasks
18
Reflections of maths teachers – 1
• Implementation of tasks – staff were a bit reluctant to do the projects, may have been partly to do with all 4 tasks being conducted simultaneously but post projects their views changed and this year development of investigation based tasks has become an issue that the KS3 staff have been keen to do as being part of performance management.
• How the investigation tasks have improved after the first one or two and the pupils would develop their thinking with little or no teacher intervention choosing their own approach to a problem. This highlights the need for them to be appropriately placed in the scheme of work for KS3 to link in with topics covered.
19
Reflections of Maths teachers – 2
• There is also a lack of confidence on the part of teachers in the school (in maths) in their own judgment. Levels from tests are trusted whereas teachers’ assessments are not and often a test mark agreeing with a teachers assessment is seen as vindication rather than the other way round. . . . Student perception of teacher assessment seems to be that it is a ‘guess’ which is then confirmed or otherwise by the test – this culture would take a lot of changing
20
Reflections of Maths teachers – 3
• It all points towards the ‘what does it mean to be good at maths’ question and how we give (get?) the students to show this – surely tests in a formal way (if properly constructed) have a role to play in allowing students to demonstrate this – and does also leave scope for teacher assessment – if the teachers are confident in this.
21
English SA Practices
English teachers used a portfolio of 6-8 pieces of work that they built up as part of their teaching
Moderation was done by one teacher on a sample of portfolios/work
Used a mixture of KS3 and GCSE criteria
22
Findings Teachers reformulated ideas of assessment in terms of
“what does it mean to be good at this subject for Y8?”
Being able to access very difficult text, being able to put themselves in a writer’s situation. Something you can’t teach... Come out with these amazing, creative ideas and you think, I couldn’t have written that. That cha-ching value. They teach you something. (Teacher K)
23
Findings• English teachers use CONSTRUCT REFERENCING
For a writing task, students have to write, in Yr. 7 for example, a persuasive letter to (King) Arthur asking him if they can join the round table as a knight. I just want to enjoy the letter-have they convinced me? Then let’s look at what persuasive techniques have they used. Were they effective? And then have they chosen good vocabulary. Then have they written in a fluent way and used punctuation.” (Teacher C)
24
Action re First Findings
• English teachers use portfolio system and replaced/reworked some activities
• English teachers began to readdress mismatch between KS3 and GCSE systems
- a greater emphasis on Speaking and Listening
- using GCSE criteria or rewriting KS3 criteria
• Introduce a ‘controlled piece’.
25
Quality in English
‘Insight’ ‘Flair’ ‘Confidence’ English teachers not concerned with reliability
Rhetoric of justification
Aim is to inform rich holistic judgmentsusing six portfolio tasks (one control conditions)
2 writing (non-fiction, fiction) 1 reading (lit.crit.)1 controlled piece (reading and writing)3 speaking and listening (group, role play, public
speaking )
Reflections of English teachers – 1
The project made me think more critically about what exactly I was assessing. The first question I remember being asked (‘what does it mean to be good at English?’) gave me a different perspective on assessment I find myself continually returning to this question. When I began teaching, I accepted that tests (were) established by the department with reliable methods – e.g. the timed essay for assessing a student’s reading of a text. The project has led me to question this assessment of reading in particular, since I have realised that the essay form can be more about the students’ writing ability, than their reading.
Reflections of English teachers – 2
. . .helped me to think (and enable others to think) much more deeply and critically about how to set quality tasks. Also enabled me to share success criteria in a way which is more valuable and engaging and apprentices students into the ‘guild knowledge’ that subject specialists have. Project has removed anxiety about delineating success only in terms of a neat, prescriptive check list.
28
Reflections of English teachers – 3
• . . the moderation and standardisation process was incredibly valuable in ensuring rigour, consistency and confidence with our approach to assessment. . . . teachers in the school were highly motivated by being involved in the process that would impact on the achievement of students in their classes (like the moderation and standardisation at GCSE – they felt that their professional judgment was valued and would impact on the bigger picture).
• It is important that teachers understand the skills that are being assessed (and make up what it is to be good at English) rather than being fixated on the form of the assessment. Teachers can then communicate the value of the assessment rather than just focussing on the test in isolation.
29
Subject differences
Curriculum ATs
KS3 and GCSE
Target: point(s) or horizon?
Analytic vs. holistic
Aggregation
Experience of summative assessment
30
Findings 1• Limited space for learners to do anything
different from classwork• Classwork activities did not discriminate and
provide evidence of a range of achievement (mainly in Maths)
• Tension between ‘doing assessment’ and using assessment information to help guide teaching and learning
• Teacher intervention could limit or facilitate student showing potential or capabilities
• Plagiarism, copying, outside help not serious problems
31
Findings 2
• Need for some ‘controlled’ assessments and some more informal assessments•Confidence that they can identify levels but concern that they are perceived as reporting on an ‘acceptable assessment’ •Judgements could be built up over activities rather than decided on ONE task/test
32
Strengthening processes of assessment
• Assessment literacy would include knowledge of the types, methods and purposes of assessment, and an understanding of the reliability and validity, of responses, scores and grades, and so on, and their implications for learners. Assessment skills would in turn include competence in different assessment methods, question design, item writing, feedback, moderation, facilitation of self and peer assessment and so on. Assessment values would include an endorsement of the importance of consistency, impartiality and transparency in assessment practice. (Gardner 2006, p.20, our emphasis)
33
Strengthening processes of assessmentcontinued
• A “partial” profession (Gardener)• Assessment literacy• Assessment skills• Assessment values
• Self-doubt, distrust, little inter-collegiate discussion• Simply “telling” teachers doesn’t work • Collaboration & design research• Recognise the systemic constraints on, and
affordances for, change
34
Implementation across departments
• Working on how they could ensure colleagues develop their expertise (learning journey)
• In English, colleagues accept new approach as a variation on current practice
• In Mathematics, some concerns as to why this needs to be done
35
ImplementationRecognition of the need to wrestle with the FA/SA interface
I wish we could bottle what we’ve done. Our teachers are happy to go along with this, but it’s so hard to help them deal confidently with the issues that arise for them in their own classrooms, without … You know ... the hindsight that we have through engaging with this. (Teacher K1)
Some of our staff just don’t get it. They’re okay with assessment for learning… They know we need to do tests. But they don’t really link the two. (Teacher I)