Learning outcomes

By the end of the session, you should be able to:

• Discuss how to assess student work accurately and fairly

• Identify types of unfair means

• Identify the characteristics of good feedback

Session overview

• Assessment: • Marking student work

• Using assessment criteria

• Moderation and external examiners

• Unfair means: identification and action

• Feedback to students: • The role of feedback in learning

• Giving useful feedback

Assessment and Feedback

What is the role of assessment and feedback in HE?

What role do you play in:

• Assessing students

• Giving students feedback

Responsibilities

“Departments are responsible for ensuring that all staff and any postgraduate research students involved in marking and moderation are adequately prepared for this activity, particularly those with less experience or who are new to the department.” From: Guidance for departments on the internal moderation of summative assessment tasks and assessed work

http://www.sheffield.ac.uk/lets/pp/assessment/marking

Assessment: Key concepts

Validity: The assessor is measuring what the assessment is intended to measure

Reliability: Consistent marking - as an individual and among markers

Teaching & Learning Activities

Intended Learning

Outcomes

Assessment Tasks

Constructive Alignment Biggs (2011)

Example Learning outcome

• Students will be able to identify different types of unfair means

Teaching and learning activities

• A lecture on types of unfair means

Assessment task

• Multiple choice exam questions which require identification of types

Decision makers and processes

Who decides

• Learning outcomes

• Assessment tasks

• Teaching and learning activities

When and how?

Answer…

• Course Directors/Module convenors

• When programmes/modules created or revised

• Approval via a university process

• Not part of your remit, but marking validly and reliably is(!)

Validity

• So how can we ensure the assessor (you as marker) is measuring what the assessment is intended to measure?

• Time to do some marking…

The design brief

Create an illustration for a greeting card using the theme “Emotions”. The illustration should use a range of colours and the design should be flexible to enable adaptation to a range of card formats.

Grade the illustration

• Merit?

• Pass?

• Fail?

Marking outcome

• Did we all agree on the grade to be awarded?

• Why, why not?

• If we all agree, have we marked accurately?

• Ask yourself how would we know? (think about evidence)

What were we measuring (outcomes)?

• Communicates specific emotions clearly

• Uses an appropriate range of emotions

• Uses a range of colours

• Uses space in a flexible way to enable adaptation to alternative card formats

• Mark again using these

• Communicates specific emotions clearly

• Uses an appropriate range of emotions

• Uses a range of colours

• Uses space in a flexible way to enable adaptation to alternative card formats

Marking outcome

• Did we all agree on the grade to be awarded?

• Why, why not?

• If we all agree, have we marked accurately?

• What else do we need to know?

Assessment criteria specify • The evidence used to determine the

achievement of the learning outcomes

• Evidence used to determine the level of achievement

Descriptor Merit Pass Fail

Clarity of emotions

Emotions are clearly identifiable with viewers highly likely to agree on all of the emotions represented, ambiguity is very unlikely.

Emotions are clearly identifiable with viewers highly likely to agree on some of the emotions represented, but a level of ambiguity is evident.

Emotions are not clearly identifiable with viewers highly unlikely to agree on any of the emotions represented; a high level of ambiguity is evident.

Range of emotions More than four different emotions are evident (the artist has attempted to represent more than three).

Three different emotions are evident (the artist has attempted to represent three).

Less than three different emotions are evident (the artist has not attempted to represent more than two)

Use of colour More than four colours have been used (not including the background).

Three or four colours have been used (not including the background).

Less than two colours have been used (not including the background).

Use of space

The illustration fits within the chosen format (rectangle or square or circle) and could fit in either of the other formats with little or no work required.

The illustration fits within the chosen format (rectangle or square or circle) but some work would be required to adapt the illustration to fit in either of the other formats.

The illustration fits poorly within the chosen format (rectangle or square or circle) and a significant amount would be required to adapt the illustration to fit in either of the other formats.

How did we do this time?

• Did we all agree?

• What issues does this raise for you?

• What can you do about them?

Valid marking

• Discuss any concerns about the criteria to your Module contact (this can help for the future)

• However, your opinion of the criteria is irrelevant from a marking point of view

• To ensure valid marking you must use the criteria

Checklist for markers:

• What is the assessment task?

• What teaching and learning activities have led to the assessment task?

• What learning outcomes are being assessed?

• What are the assessment criteria -evidence for achievement and levels

Review: Key concepts

Validity: The assessor is measuring what the assessment is intended to measure

Reliability: Consistent marking - as an individual and among assessors (markers) …and consistency across assessment activities e.g. exam papers across cohorts (but not your remit!)

Factors Affecting Reliability

With your neighbours….

…..brainstorm the factors that can affect reliability in assessing student work.

…..identify what can be done to improve reliability?

Processes

Standardisation

• To the marking criteria for individual marker consistency

• Ask how your department does this (e.g. samples of work)

Moderation

• Consistency among markers

Internal Moderation

“The process of summative assessment should involve internal moderation wherever practical to ensure that the initial judgements/marks have been arrived at accurately, consistently and fairly in accordance with the assessment criteria/marking scheme. The method of moderation may vary, and it is for the department to determine the most appropriate method for the type of assessment”. From: Guidance for departments on the internal moderation of summative assessment tasks and assessed work http://www.sheffield.ac.uk/lets/pp/assessment/marking

Support for the less familiar (you?)

Particular arrangements for moderation of work that is first marked by those who may be less familiar with the assessment process/the use of the department’s assessment criteria. This might apply to postgraduate research students, postdoctoral research associates, part time tutors

or any new members of academic staff. http://www.lets.dept.shef.ac.uk/flats/guidance_internal_moderation.pdf

External Examiners

The purpose of the external examiner system is to ensure:

• the assessment system is rigorous, equitable and is fairly conducted,

• student performance and degrees awarded are of an appropriate standard and are comparable nationally

Add to your checklist…

• What is the assessment task?

• What teaching and learning activities have led to the assessment task?

• What learning outcomes are being assessed?

• What are the assessment criteria -evidence for achievement and levels

• How will you be inducted?

• How will you be standardised (e.g. examples of assessed work)

• How will the assessment be moderated (and what is your role?)

Summary

So far we have looked at:

• Validity in Assessment

• Reliability in Assessment

Unfair Means 1. Plagiarism (either intentional or

unintentional)

2. Submitting bought or commissioned work

3. Double submission (or self plagiarism)

4. Collusion (distinct from collaborative group work)

5. Fabrication

6. Facilitating the use of unfair means

Is it unfair means? Which one?

1. Plagiarism (either intentional or unintentional)

2. Submitting bought or commissioned work

3. Double submission (or self plagiarism)

4. Collusion (distinct from collaborative group work)

5. Fabrication

6. Facilitating the use of unfair means

Detection?

1. Plagiarism

2. Submitting bought or commissioned work

3. Double submission (or self plagiarism)

4. Collusion

5. Fabrication

6. Facilitating the use of unfair means

If you suspect unfair means…

• Do not approach the student

• Your department will have process for reporting suspected Unfair Means

• Find out who you should raise them with (e.g. Module Tutor)

• Do not hesitate to raise suspicions, it will not be up to you to investigate – doing nothing will not help the student

• Find out University’s policy and procedures at http://www.shef.ac.uk/lets/design/unfair

Summary

So far we have looked at:

• Validity in Assessment

• Reliability in Assessment

• Unfair means

Types of Assessment and Giving Feedback

Forms of assessment

• What forms/types of assessment do you use?

• When do they occur within the teaching/learning experience?

Modes of Assessment

Summative assessment

• Sums up achievement

• Performance indicator

• High-stakes (this is it – cannot address issues)

• ‘Low level learning’

Formative assessment

• Informs students

• Emphasis on feedback.

• Lower stakes (time to address issues)

• ‘High level learning’

Formative or summative?

• Final grade from a module

• Feedback on an essay which is not graded

• Feedback on a mid-term exam which is graded

• Which would you find most useful form a student perspective?

Definition of feedback

Feedback exists in any process, activity or information that enhances learning by providing students with the opportunity to reflect on their current or recent level of attainment. It can be provided individually or to groups. It can take many forms. It is responsive to the developmental expectations of particular programmes and disciplines. Detailed opportunities for the receipt of feedback by students will therefore vary across the University, and at different stages of students’ programmes.

University of Sheffield’s Principles of Feedback

Feedback types?

In your experience…

• What sort of feedback do you give students? When, How and Why?

• How do they use the feedback?

Problems with feedback

• Comes too late to be applied elsewhere

• Focuses on presentation/superficial learning

• Quantity: Too much or too little

• Justifies the “loss of marks”

• Doesn’t lead directly to improved future performance

(Glover 2006)

What is good feedback? 1. Clarifies good performance

2. Helps close the gap between current and desired performance

3. Gives high quality information

4. Helps students to assess themselves

5. Encourages students to talk about their work

6. Motivates students

7. Leads to useful information for teachers Adapted from : Nicol, D. J. & Macfarlane-Dick, D. (2006). “Formative assessment and self-regulated learning: a model

and seven principles of good feedback practice.” Studies in Higher Education, 31(2), 199-218.

Tips for effective feedback • Tell your students you’re giving feedback

• Be specific

• Concentrate on points not person

• Be positive about things to continue

• ….and be specific about how to improve

• Don’t focus on excuses for poor performance

• Use learning outcomes and assessment criteria!!

Student engagement

Responsibilities for students around feedback:

• Our Commitment http://www.shef.ac.uk/ssid/ourcommitment

• Principles of Feedback

Acting on feedback:

• Some students may have little experience with thinking about what feedback means

• Personal tutors can help

Principles of Feedback

• The University of Sheffield has Principles of Feedback

• Approved by Senate

• A definition and six Principles

• Staff and student versions

• Your department’s processes should be aligned with these

Feedback resources

For further information and guidance on feedback, take a look at:

• Feedback and Assessment section of the online Toolkit for Learning and Teaching www.shef.ac.uk/lets/toolkit

• School of Architecture Feedback Handbook http://feedbackhandbook.wordpress.com

Sessions Learning outcomes

Can you:

• Discuss how to assess student work accurately and fairly?

• Identify types of unfair means?

• Identify the characteristics of good feedback?

Learning and Teaching Professional Recognition Scheme (LTPRS)

• Started 1 September 2014

• Provides a framework that enables our staff to achieve recognition from the HEA at:

– Associate Fellow

– Fellow

– Senior Fellow

• Encourages all who teach and/or support learning to align their practice with the UK Professional Standards Framework for teaching and learning in HE

• Provides national recognition of your commitment to professionalism

• It is a portable asset that has UK-wide relevance

Foundation Pathway

The aim of the Foundation Pathway is to enable staff who teach or support learning to consider learning as a process, and apply current and emerging methods/strategies appropriate to their particular subject and teaching context to help students learn.

• Structured, non-credit bearing route to recognition as Associate Fellow

• For any member of staff or PGR student who teaches/supports learning

• Assessed by Portfolio comprising three main components:

– Workshops and self-evaluation (including some STA sessions)

– Teaching observations

– Developing teaching philosophy

• BUT, can only include activities completed after 1 September 2014

What next?

If you wish to apply for recognition via the Foundation Pathway:

• You will need to attend an Orientation Workshop. Initial dates are:

– Thursday 16 October 2014 9.30-12.30

– Tuesday 21 October 2014 13.30-16.30

• Full information including the Foundation Pathway Handbook is available at www.shef.ac.uk/lets/cpd/ltprs/foundation

If you have completed some of the STA workshops prior to 1 September 2014 and have teaching experience, you may wish to apply for recognition via the Personal Pathway. Further information is available at www.shef.ac.uk/lets/cpd/ltprs/mypathway