View
213
Download
0
Category
Tags:
Preview:
Citation preview
Prof Robert CoeSchool of Education
Tel: (+44 / 0) 191 334 4184Fax: (+44 / 0) 191 334 4180
E-mail: r.j.coe@dur.ac.uk
Assessment
Lecture for PGCESeptember 2012
2
Definition of a grade
‘An inadequate report of an inaccurate judgment by a biased and variable judge of the extent to which a student has attained an undefined level of mastery of an unknown proportion of an indefinite material.’
Dressell (1983)
3
Assessment
What different types of assessment are there?
Problematising assessment: assessment criteria; assessment for accountability; teacher assessment
Why does assessment matter? How can teachers use assessment to
promote learning?
© 2010 Robert Coe, Durham University
4
Background:Types of assessment and their problems
5
Assessment types
Functions of assessment To certify achievement (summative) To inform learning (formative) To control curriculum, teachers, students
(accountability) Types of assessment
Formal / informal Criterion / norm / self referenced Continuous / final / progressive
Questions of quality Validity: does it test what you want students to learn? Reliability: would it give the same result again?
6
Criterion-referenced assessment Define what you want learners to be able
to do Assess whether they can
‘Add numbers up to 20’ 11+3 3+11
Context is everything!
50-59
Approp-riate referen-cing and citation details.
Good in terms of structure, compre-hensibility and quality of presen-tation.
Good level of knowledge and understanding in terms of central ideas and concepts within the field of education theory, policy and context with appropriate supporting factual detail allied to an ability to make explicit the relationship of the ideas to the wider field of educational knowledge.
A relevant line of argument is presented and the relationship of this to the wider field of educational knowledge is made explicit.
Demonstrates a critical understanding of the appropriateness of educational* research and its relationship to the topic.
The theoretical justification for practice and the practical implications of theoretical ideas are discussed as are some of the dilemmas which arise from the relationship of educational theory to practice.
Evidence of sufficient reading focussing on a range of educational source material [e.g. government reports, research articles, books, personal experience etc.]
© 2010 Robert Coe, Durham University
7Teaching to the test: the ‘Texas miracle’
Klein et al, 2000
© 2010 Robert Coe, Durham University
8
Teacher assessment
Pros Based on sustained performance Based on assessment of authentic learning
tasks (validity) Incorporates a range of tasks Motivates students
Cons Judgements often have low reliability Hard to translate into absolute standards Halo effects Teacher workload Problems if teachers are judged by results
Rising standards
Performance of England in international surveys
Maths (age 10, TIMSS)
Maths (age 14, TIMSS)
Reading (age 11, PIRLS)
Science (age 10, TIMSS)
Science (age 14, TIMSS)
Reading literacy (age 15, PISA)
Mathematical literacy (age 15, PISA)
Scientific literacy (age 15, PISA)
480
490
500
510
520
530
540
550
560
19
94
19
95
19
96
19
97
19
98
19
99
20
00
20
01
20
02
20
03
20
04
20
05
20
06
20
07
20
08
20
09
20
10
Year
Sta
nd
ard
ise
d t
es
t s
co
re
25 point rise in PISA =
+£4,000,000,000,000 GDP
© 2010 Robert Coe, Durham University
11
Assessment:Perhaps the single most important determinant of learning
© 2010 Robert Coe, Durham University
12What factors influence achievement most?
John Hattie and his team have analysed data from all available studies on the effects of different strategies From 50,000 studies Involving over 200,000,000 students Compiled in over 750 meta-analyses
Which are most promising? Personalised learning? 0.20 Smaller classes? 0.21 Ability grouping? 0.25 Better headteacher? 0.30
© 2010 Robert Coe, Durham University
13
Assessment…
Influence Effect SizeSelf-report grades 1.44Feedback .72Providing formative evaluation to teachers .70Frequent/ Effects of testing .46Teaching test taking skills .22
www.suttontrust.com
‘Best buy’ strategies
Cost per pupil
Eff
ect
Siz
e (
mon
ths
gain
)
£00
10
£1000
FeedbackMeta-
cognitive
Peer tutoring Pre-school
1-1 tutoringHomework
ICT
AfLParental
involvementSports
Summer schools
After school
Individualised learning
Learning styles
Arts Performance pay
Teaching assistants
Smaller classes
Ability grouping
Promising
May be worth it No
way
© 2010 Robert Coe, Durham University
16
“… we have each been asked several times by teachers, ‘What makes for good feedback?’—a question to which, at first, we had no good answer. Over the course of two or three years, we have evolved a simple answer—good feedback causes thinking.”
(Black & Wiliam, 2003)
© 2011 Robert Coe, Durham University
17
Focus on learning
Does your ‘theory of learning’ explain why
ability grouping (setting) after-school clubs teaching assistants smaller classes
do not work?
© 2011 Robert Coe, Durham University
18
Do we care about learning?
Which of the following are evidence of learning? Students are busy: lots of work is done Students are engaged, interested, motivated Classroom is ordered, calm, under control
What do school students value most? Social interactions & status with peers Keeping out of trouble Pleasing teachers: good marks, neat writing,
polite Thinking hard about really challenging problems
© 2010 Robert Coe, Durham University
19Assessment can help focus on learning How can you know what your students
are learning? How often do they need to think hard? Learning is invisible, and is low on the
agenda of many teachers and students If you want to evaluate what students are
learning, you need appropriate tools to help you
© 2010 Robert Coe, Durham University
20
If you want your students to learn something difficult …
You need to know how many of them have ‘got it’
They need to know whether they have ‘got it’
If they haven’t, you need to be able to do something about it
© 2010 Robert Coe, Durham University
21
Formative assessment
Effect sizes between 0.4 and 0.7 Advice to improve formative
assessment: Feedback should focus on work, with advice for
improvement, not comparisons with others Pupils should be trained in self-assessment - to
understand learning aims Teaching should allow pupils to express their
understanding Create reflective dialogue between teacher
and pupil Tests should be frequent and relevant;
feedback and support to improve should be given
“Information used as feedback to modify teaching and learning”
Black and Wiliam, 1998
© 2010 Robert Coe, Durham University
22
Formative assessment in practice
Sharing success criteria with learners Classroom questioning Feedback through marking, eg comment-
only Peer-assessment and self-assessment Formative use of summative tests
Black et al, 2002
© 2010 Robert Coe, Durham University
23
Comments: instead or as well?
Comments group
Recalled comment (84%)
Failed to recall
Recalled comment (5%)
Recalled grade and comment (23%)
Recalled grade (53%)
Failed to recall
Comment and grade group
Butler, 1988
© 2010 Robert Coe, Durham University
24
“If you are going to grade or mark a piece of work, you are wasting your time writing careful diagnostic comments.”
Wiliam, 1999
© 2010 Robert Coe, Durham University
25
One evaluation of comment-only marking
“I would prefer to be in another form because we don’t get our test marks back”
Smith and Gorard, 2005
© 2011 Robert Coe, Durham University
26
Which feedback is best?
a) Well done, that is very good workb) Well done, that is very good work (for you)c) Well done, you have shown a high level of ability
in solving these problemsd) Well done, that is one of the best essays in the
classe) Well done, that is a big improvement on your
previous workf) Well done, I can see you have worked hard at thisg) Well done, I can see you have concentrated on
reading the questions carefully and it has paid off
© 2010 Robert Coe, Durham University
27
Feedback does not always enhance performance
relate performance to clear, specific and challenging goals
make people focus on the task, not themselves, nor compare them with others
compare their performance with their own past performance
be seen as informative, not controlling make people feel competent, but not
complacentCoe, 1998
Ideally, feedback should …
© 2010 Robert Coe, Durham University
28
Feedback does not always enhance performance
suggest reasons for performance that are alterable (eg effort, strategies)
follow soon after performance be specific and task focused be given to individuals on their individual
performance correct errors or inadequacies
Coe, 1998
Ideally, feedback should …
© 2010 Robert Coe, Durham University
29
Further advice on assessment
Feedback should relate to individuals’ (mis)understandings
Praise should be infrequent, credible, contingent, specific and genuine
Praise should be related to factors within an individual’s control
Don’t help too much: ‘minimal intervention’
Feedback should support a view of ability as incremental rather than fixed
Students often don’t understand what they are meant to be doing and how it is assessed Wiliam, 1999, 2000
Recommended