Developing an effective assessment strategy Peter Hartley, Professor of Education Development...

Developing an effective assessment strategy

Peter Hartley,

Professor of Education Development

University of Bradford

p.hartley@bradford.ac.uk

2

This session

• introduction

• what’s the problem?

• introduce current projects and initial outcomes: the PASS and TESTA projects

• identify important issues, suggestions and implications

3

Introduction

• myself

• good and bad news!

4

Introduction

• myself

• good and bad news!– The bad news – the jury is out!– The good news – growing evidence and

examples from new projects and initiatives with the opportunity to build on and extend others’ experience

5

What’s the problem? #1

• Consider the issues buried in the title of this paper– the implications of ‘developing’– effective ‘for whom’?– clarifying the purposes of ‘assessment’– the notion of ‘strategy’

6

What’s the problem? #2

• See the PASS Issues Paper– Please comment/feedback and use

• Would highlight:– Assessment ‘drives and channels’– What/why are we measuring: the ‘slowly learnt’– Limitations of grading systems

(e.g. marks are not numbers)– Implications for course structures/regulations

–

7

Introducing the projectsPASS project #1

• NTFS group project over 3 years– One year of development and investigation

and two years of implementation

• Consortium– Led by Bradford– 2 CETLs – ASKE and AfL– Plus Exeter, Plymouth and Leeds Met.

8

PASS project #2

• Outcomes and outputs– approaches to PBA– ‘choice and consequence’ guides– workshop and resources for local

implementation– case studies from different disciplines

9

TESTA project, #1

• NTFS group project with 4 partner universities

• ‘aims to improve the quality of student learning through addressing programme-level assessment.’

10

TESTA project # 2

• starting from audit of current practice, so far looked at nine programmes

• surveyed students using focus groups and AEQ – Assessment Experience Questionnaire – Graham Gibbs et al

• also using tool to identify programme level ‘assessment environments’ (Gibbs)

11

Issues to disentangle include:

• Defining assessment

• Assessment environments and their impact

• Defining ‘programme-based’ assessment

• Student perceptions and expectations

• The need for a strategic approach

• Grading and credit

12

Defining assessment: a challenge

– program outcomes “need to be assessed in complex, multidimensional student performances”

– “Multidimensional performance entails the whole dynamic nexus of the individual’s intentions, thoughts, feelings, and construals in a dynamic line of action and his or her entanglement in an evolving situation and its broader context. Such a context may be within or across work, family, civic, or other settings.”

– (Rogers, Mentkowski, & Reisetter Hart, 2006, p. 498).

13

Assessment environment and impact

• Interim findings from TESTA– variety of assessments can cause problems– Issues over understanding assessment

criteria, marker variation, and feedback– variation across programmes– QA ‘myths and traditions’ can get in the way

14

Starting to define PBA

15

Typical student perceptions and concerns (based on PASS)

• perceptions of ‘the course’ are variable.

• assessment experienced as ‘fragmented’.

• anxieties re move to more integrated assessment – perceived risk in terms of performance.

• Concerns about feedback and timing.

16

The need for strategy

• An example finding from Gibbs– ‘greater explicitness of goals and standards

was not associated with students experiencing the goals and standards to be clearer’

• And what did make a difference?

17

The need for strategy

• An example finding from Gibbs– ‘greater explicitness of goals and standards was

not associated with students experiencing the goals and standards to be clearer’

• And what did make a difference?• Formative-only assessment• More oral feedback• Students ‘came to understand standards through many

cycles of practice and feedback’

18

An example: Peninsula Medical School

• NB Case study forthcoming from PASS

• Includes:• four assessment modules that run through the 5

year undergraduate medical programme and are not linked directly to specific areas of teaching

• focus on high-quality learning (Mattick and Knight, 2007)

19

Issues re grading and credit

• Teaching/assessment links

• The assessment/credit link

• Credit cf accreditation

• Threshold cf grades in performance

20

And back to ‘effective assessment strategy’

21

And back to ‘effective assessment strategy’

• Will it explain to staff, students and external agencies:– How the course/programme assesses the

main outcomes?– How assessment and teaching are linked?– How assessment both supports ‘high-quality

learning’ and develops it over the course?

22

And finally …

• Other initiatives and events, e.g.– EARLI conference later this year– Growth of work on feedback

(e.g. use of audio as in ASEL)

23

And absolutely finally …

Contacts for PASS

• The project website

• Project Director: Peter Hartleyp.hartley@bradford.ac.uk

• Project Manager: Ruth Whitfieldr.whitfield@bradford.ac.uk