Prof Robert Coe School of Education Tel: (+44 / 0) 191 334 4184 Fax: (+44 / 0) 191 334 4180 E-mail: [email protected] Assessment Lecture for PGCE September

Prof Robert CoeSchool of Education

Tel: (+44 / 0) 191 334 4184Fax: (+44 / 0) 191 334 4180

E-mail: [email protected]

Assessment

Lecture for PGCESeptember 2012

2

Definition of a grade

‘An inadequate report of an inaccurate judgment by a biased and variable judge of the extent to which a student has attained an undefined level of mastery of an unknown proportion of an indefinite material.’

Dressell (1983)

3

Assessment

What different types of assessment are there?

Problematising assessment: assessment criteria; assessment for accountability; teacher assessment

Why does assessment matter? How can teachers use assessment to

promote learning?

© 2010 Robert Coe, Durham University

4

Background:Types of assessment and their problems

5

Assessment types

Functions of assessment To certify achievement (summative) To inform learning (formative) To control curriculum, teachers, students

(accountability) Types of assessment

Formal / informal Criterion / norm / self referenced Continuous / final / progressive

Questions of quality Validity: does it test what you want students to learn? Reliability: would it give the same result again?

6

Criterion-referenced assessment Define what you want learners to be able

to do Assess whether they can

‘Add numbers up to 20’ 11+3 3+11

Context is everything!

50-59

Approp-riate referen-cing and citation details.

Good in terms of structure, compre-hensibility and quality of presen-tation.

Good level of knowledge and understanding in terms of central ideas and concepts within the field of education theory, policy and context with appropriate supporting factual detail allied to an ability to make explicit the relationship of the ideas to the wider field of educational knowledge.

A relevant line of argument is presented and the relationship of this to the wider field of educational knowledge is made explicit.

Demonstrates a critical understanding of the appropriateness of educational* research and its relationship to the topic.

The theoretical justification for practice and the practical implications of theoretical ideas are discussed as are some of the dilemmas which arise from the relationship of educational theory to practice.

Evidence of sufficient reading focussing on a range of educational source material [e.g. government reports, research articles, books, personal experience etc.]


7Teaching to the test: the ‘Texas miracle’

Klein et al, 2000


8

Teacher assessment

Pros Based on sustained performance Based on assessment of authentic learning

tasks (validity) Incorporates a range of tasks Motivates students

Cons Judgements often have low reliability Hard to translate into absolute standards Halo effects Teacher workload Problems if teachers are judged by results

Rising standards

Performance of England in international surveys

Maths (age 10, TIMSS)

Maths (age 14, TIMSS)

Reading (age 11, PIRLS)

Science (age 10, TIMSS)

Science (age 14, TIMSS)

Reading literacy (age 15, PISA)

Mathematical literacy (age 15, PISA)

Scientific literacy (age 15, PISA)

480

490

500

510

520

530

540

550

560

19

94

19

95

19

96

19

97

19

98

19

99

20

00

20

01

20

02

20

03

20

04

20

05

20

06

20

07

20

08

20

09

20

10

Year

Sta

nd

ard

ise

d t

es

t s

co

re

25 point rise in PISA =

+£4,000,000,000,000 GDP


11

Assessment:Perhaps the single most important determinant of learning


12What factors influence achievement most?

John Hattie and his team have analysed data from all available studies on the effects of different strategies From 50,000 studies Involving over 200,000,000 students Compiled in over 750 meta-analyses

Which are most promising? Personalised learning? 0.20 Smaller classes? 0.21 Ability grouping? 0.25 Better headteacher? 0.30


13

Assessment…

Influence Effect SizeSelf-report grades 1.44Feedback .72Providing formative evaluation to teachers .70Frequent/ Effects of testing .46Teaching test taking skills .22

www.suttontrust.com

‘Best buy’ strategies

Cost per pupil

Eff

ect

Siz

e (

mon

ths

gain

)

£00

10

£1000

FeedbackMeta-

cognitive

Peer tutoring Pre-school

1-1 tutoringHomework

ICT

AfLParental

involvementSports

Summer schools

After school

Individualised learning

Learning styles

Arts Performance pay

Teaching assistants

Smaller classes

Ability grouping

Promising

May be worth it No

way


16

“… we have each been asked several times by teachers, ‘What makes for good feedback?’—a question to which, at first, we had no good answer. Over the course of two or three years, we have evolved a simple answer—good feedback causes thinking.”

(Black & Wiliam, 2003)


17

Focus on learning

Does your ‘theory of learning’ explain why

ability grouping (setting) after-school clubs teaching assistants smaller classes

do not work?


18

Do we care about learning?

Which of the following are evidence of learning? Students are busy: lots of work is done Students are engaged, interested, motivated Classroom is ordered, calm, under control

What do school students value most? Social interactions & status with peers Keeping out of trouble Pleasing teachers: good marks, neat writing,

polite Thinking hard about really challenging problems


19Assessment can help focus on learning How can you know what your students

are learning? How often do they need to think hard? Learning is invisible, and is low on the

agenda of many teachers and students If you want to evaluate what students are

learning, you need appropriate tools to help you


20

If you want your students to learn something difficult …

You need to know how many of them have ‘got it’

They need to know whether they have ‘got it’

If they haven’t, you need to be able to do something about it


21

Formative assessment

Effect sizes between 0.4 and 0.7 Advice to improve formative

assessment: Feedback should focus on work, with advice for

improvement, not comparisons with others Pupils should be trained in self-assessment - to

understand learning aims Teaching should allow pupils to express their

understanding Create reflective dialogue between teacher

and pupil Tests should be frequent and relevant;

feedback and support to improve should be given

“Information used as feedback to modify teaching and learning”

Black and Wiliam, 1998


22

Formative assessment in practice

Sharing success criteria with learners Classroom questioning Feedback through marking, eg comment-

only Peer-assessment and self-assessment Formative use of summative tests

Black et al, 2002


23

Comments: instead or as well?

Comments group

Recalled comment (84%)

Failed to recall

Recalled comment (5%)

Recalled grade and comment (23%)

Recalled grade (53%)

Failed to recall

Comment and grade group

Butler, 1988


24

“If you are going to grade or mark a piece of work, you are wasting your time writing careful diagnostic comments.”

Wiliam, 1999


25

One evaluation of comment-only marking

“I would prefer to be in another form because we don’t get our test marks back”

Smith and Gorard, 2005


26

Which feedback is best?

a) Well done, that is very good workb) Well done, that is very good work (for you)c) Well done, you have shown a high level of ability

in solving these problemsd) Well done, that is one of the best essays in the

classe) Well done, that is a big improvement on your

previous workf) Well done, I can see you have worked hard at thisg) Well done, I can see you have concentrated on

reading the questions carefully and it has paid off


27

Feedback does not always enhance performance

relate performance to clear, specific and challenging goals

make people focus on the task, not themselves, nor compare them with others

compare their performance with their own past performance

be seen as informative, not controlling make people feel competent, but not

complacentCoe, 1998

Ideally, feedback should …


28

Feedback does not always enhance performance

suggest reasons for performance that are alterable (eg effort, strategies)

follow soon after performance be specific and task focused be given to individuals on their individual

performance correct errors or inadequacies

Coe, 1998

Ideally, feedback should …


29

Further advice on assessment

Feedback should relate to individuals’ (mis)understandings

Praise should be infrequent, credible, contingent, specific and genuine

Praise should be related to factors within an individual’s control

Don’t help too much: ‘minimal intervention’

Feedback should support a view of ability as incremental rather than fixed

Students often don’t understand what they are meant to be doing and how it is assessed Wiliam, 1999, 2000

Documents

Prof Robert Coe School of Education Tel: (+44 / 0) 191 334 4184 Fax: (+44 / 0) 191 334 4180 E-mail: [email protected] Assessment Lecture for PGCE September