Audio feedback design models and tips

Audio Feedbackdesign, models, benefits and tips

MELSIG: Audio Feedback Toolkithttp://melsig.shu.ac.uk/?page_id=1196

Andrew Middleton, NTFHead of Academic Practice & Learning Innovation, Sheffield Hallam University

@andrewmid

With thanks to Anne Nortcliffe, Michelle Blackburn and innovators everywhere

Introduction

We will find out about audio feedback

Consider many models for designing and using audio feedback

Understand when and why to use it within a feedback strategy

Hear what academics and students think about audio feedback

Consider the benefits of using audio feedback

Look at tips for giving audio feedback, technically and pedagogically

Definitions and descriptionsWhat do we mean by audio feedback?

What is audio feedback?

the recording and distribution of spoken feedback given on a student's work using diverse media

Definitions and descriptionsWhat do we mean by audio feedback?

each academic devises a model of feedback that works for them and their students

Audio feedback

takes many forms needs to be fit for purpose needs to be designed

Definitions and descriptionsWhat do we mean by form or model of audio feedback?

Forms and models

Media

TechnologyContext

Pedagogy

Academic designers will need to consider each of these dimensions

Audio – an adaptable spaceWhen to use audio feedback

Audio feedback is spoken…it has different properties to written feedback

Some studies ask – “is audio better than written feedback?” (Fawcett & Oldfield, 2016; Nemec & Dintzner, 2015; Voelkel & Mello, 2014)

Audio is a powerful part of your feedback toolkit Ask,

How does audio fit into your assessment strategy?

How does it combine with other methods?

Which peg will fit the round hole?

When to use audio feedbackAudio – an adaptable space

Micro

Basic technical matters of writing, e.g. spelling, grammar and referencing conventions

Middle

Supporting student’s ability to produce quality ideas and support these with evidence

Global

Overall structure, academic argument and organisation

(Hatziapostolou & Paraskikis, 2010; Ice et al., 2010; Stern & Solomon, 2006)

Correction Detailed Constructive Criticism

Why use audio feedbackIs it really effective?

What is audio feedback?

Effective and fit for purpose

Quick to turnaround and manageable

Timely Clear Personal and meaningful

Audio feedback is effective when designed for context

Analyse the contextBe clear about what you are trying to achieveWhen is the feedback needed and when exactly will the feedback have most effect?What do your students need to think about at this point?How can this connect to learning in other modules, now and later?What, if nothing else, do they need to hear?Motivation - why will the students want to listen and learn and what can you do make their use of the feedback more likely?

A designed spaceWe are designers of experience that leads to learning

It is what we make itAnalyse

ADDIE

Design for contextDesign specification - know specifically how you will give the feedback and how it will be usedUsage – what impact, when? How will it be accessed and where? Will it be reused? Why and when?Detail – decide what to include and what to leave out“Voice”, tone, intimacy – the special personal ingredientHow you will refer to the student's work in the feedbackHow you will relate it to other feedback or teaching activity


It is what we make it

Design

Analyse

ADDIE

Ask – if I could have just five minutes with this person, what would I say?

Develop for contextThink about your context as tutor-producerTime-based media are simple – saying is makingAssume a simple “Press the red button” mindsetKeep the message brief, clear, focussed and manageableRemember: voice is rich and personalBe direct, be close, let them know you care…How can voice promote their sense of belonging and becoming?



Design

Develop

Analyse

ADDIE

Implement – the feedback model (technical)Decide how the feedback will be made and distributedFind support, involve students and trial itTools people use audio recorders, smart phones and

tablets, PCs, USB headsets VLE, standalone audio software, apps



Design

DevelopImplement

Analyse

ADDIE



Design

DevelopImplement

Evaluate

Analyse

ADDIE

Evaluate – the feedback modelDid this achieve what you set out to do? Ask your students, check access data,

measure impact on engagement and impact on learning

Ask yourself – did this feel right? Evaluate… and improve

Refle

ction

mo

me

nt

A designed space Feedback is..?

Why do you give feedback..?

Because I have to

Justify my marking

Bring my module to a suitable close

Correct student mistakes

Explain and clarify knowledge

Reiterate what is important

Expand upon key ideas

(Re-)orientate, motivate and challenge my students

Develop self-directed learning and self-esteem

Feed-forward learning and content knowledge for future engagement

Because I have to

Justify my marking

Bring my module to a suitable close

Correct student mistakes

Explain and clarify knowledge

Reiterate what is important

Expand upon key ideas

(Re-)orientate, motivate and challenge my students

Develop self-directed learning and self-esteem

Feed-forward learning and content knowledge for future engagement

2 4 6 8 10

AN

OTH

ER M

OD

ULE

“Good assessment engages students with the curriculum; it creates opportunities for dialogue and ultimately stimulates learning.”see ESCAPE Project Russell & Bygate 2010

2 4 6 8 10

A M

OD

ULE

Assessment & Feedback Strategies

Formative low stakes

Summative high stakes

Feedback

How can we position feedback? e.g…

A designed space Understanding feedback in the context of learning, teaching and assessment

From: Russell, M. & Bygate, D. (2010). Assessment for learning: An introduction to the ESCAPE project. Blended Learning in Practice, March 2010, pp. 38-48.

How do we connect feedback across and through the course?

Why we use it

Academics say

The feedback is fresh. It's feedback that is alive rather

than something that is dead on

paper.

You can get some of the kindness that we intend

...into how you talk about it."

[It's] easier to more clearly indicate what has been

good or bad about their work

It's ever so easy to record. I've done it with my little MP3 audio recorder but

actually my mobile phone is a very good MP3

recorder and I've always got it with me.

from research conducted by Andrew Middleton and Anne Nortcliffe

Why we use it

Students say

I listen more when someone is talking to me

than if I'm reading it.

[A few] weeks later, I'm sitting down to do a bit more work on this

assignment project. I'm thinking "What did she say?" ... It's right

there. Press it. Brilliant!" Everything she'd said, all the suggestions she

made, were right there.

It's just there on your computer - at

home, at university.

Anywhere, any time.

We got the feedback back

really quick – I was surprised by that.

More later from students...from research conducted by Andrew Middleton and Anne Nortcliffe

Question: Where do we learn? How do we learn?Where and how will we listen?

Everywhere.

Anywhere.

In between and across locations…

Lecture theatres, classrooms, corridors, outside, pub, home office, student rooms, workplace, placements, professional settings, international settings, online, on the move…

Learning Space – places for learning

Home

Formal

Informal Work

Design for learning contextAudio feedback is part of a mobile pedagogy…Learning context can be significant

Refle

ction

mo

me

nt

Preparing students for feedback

Do you talk to your students about how to use the feedback you give them?

…About reading places and listening places?


Where do we teach? How do we teach?Where will we record?

Everywhere.

Anywhere.

In between and across locations…

Lecture theatres, classrooms, corridors, outside, pub, home office, workplace, placements, professional settings, international settings, online, on the move…

Design for teaching context

Feedback is often produced in spaces and situations that work for you, where you feel right


Home

Formal

Informal Work

Your teaching contextFeedback is often produced in spaces and situations that work for you

Tutorial conversations – offices, cafes, quiet cornersMarking scripts – office, home, waiting to pick the kids up…Assessing studio-based learning – in front of the work with images or videoProcesses and activity – observational video commentary e.g. Coaches Eye appScreencasts - usually at the PC incorporating student’s workAuthentic situations - WBL or field-based study, etc


Home

Formal

Informal Work

Teacher's Context SMARTPCQuiet

Authentici

Immediate

Live and active

Calm

Focused

Structure

QuietScale

SituatedSystematic

On task

Convenient

Expedient

Media-Enhanced Assessment

Digital media-enhanced Assessment for LearningExtending the opportunities for personal engagement and clarification

See MELSIG website: http://melsig.shu.ac.uk

Digital and social media can be used to enhance assessment for learning in many ways Course - orientation, support, challenge, motivation, reflection Audio Briefing - clarity, emphasis, meaning, perspective,

engagement Audio FAQs - one to many, clarification, belt and braces,

continuous engagement Assessment Objects - case studies, diagrams, real world artefacts Audio Summaries & Revision Notes - succinct and manageable,

peer reviewed, communal, multiple purposes over time Screencast Moderation – modelling and managing consistent

team marking Media-Enhanced Feedback - audio, screencast, smartphone,

video, web-based media, audio annotation

http://melsig.shu.ac.uk/?page_id=259

Commonly audio feedback is given by the academic to a student by recording at a PC with a headset using software such as Audacity (see Personal tutor monologue later).

Staff enjoy personally engaging students - showing they care

Getting close - the 'radio voice'

Quick to do (never edit - just start again or enjoy the rough edges)

It can be used to complement other objective methods, like assessment grids

Often distributed through the VLE

Audio feedback modelsTechnical overview

Audio feedback

Mostly audio feedback, but personal devices and apps create potential for many flexible methods.

Some academics use MP3 recorders.

Tutorials, informal 'corridor' tutorials and field-based activities can be recorded and shared with participants immediately.

Personal and portable devices can make it easier to manage give and turn around feedback while marking.

The mobile technology can help to capture previously ephemeral formative conversations

Easy, immediate distribution via apps to online services


Smartphone and tablet feedback

e.g. Voice Record appor GarageBand

A talk through recording of the PC screen showing student's work

Essay scripts, project reports, posters, photographs, wikis and websites... anything that can be displayed on screen.

Shows the academic giving due consideration to the students work.

Feedback on outputs of project group work can be returned for review by groups e.g. on group project plans - "cinema feedback"


Screencast feedback - 'talk throughs'

Open Screencast-o-Matic.com to make a screencast

https://screencast-o-matic.com/

Using small cameras with video options or web cams

Commentaries on action, performance analysis

Easy to do

Using web cams, academics at Reading and Plymouth liked 'being in the picture’ (see Jisc ASSET project)

Video feedback of:

workshops

labs or studio 'crits'

location specific work/ engagement

portfolios


Video feedback

Use web cams, video camera apps e.g. Coaches Eye

Some Office tools (e.g. Powerpoint) have options for audio annotation or screen recording

Apps like iAnnotate allow you to add audio annotations to PDF documents

Be careful, this can be time consuming. Consider producing a screencast as a more efficient way of producing audio feedback on work or use your word processor’s review and commenting tools.


Audio annotation

Skype

Google Hangouts

Webinar environments, e.g. Blackboard Collaborate, BigBlueButton, Zoom, etc.

All allow visual artefacts (pictures, diagrams, videos, charts) to be reviewed by one or more people.


Web-based media discussion boards

Works well as part of a marking approach

Tutor works through many submitted assignments

Can be used in combination with other methods, e.g. assessment rubrics, annotated scripts or generic feedback

Allows for a reflective and supportive voice that can help to challenge the student about to improve

Tone of voice can help to clarify marks and marginalia, adding meaning and encouragement

Usually, selected significant points are usually identified for each student in a 3-6 minutes audio file…

…though some commentators (e.g. Ribchester et al., 2007) believe that feedback should be extensive.

Audio feedback modelsPedagogical model

Personal tutor monologue

Use the Pause button

Tip: Mark commentary numbers on the student's text and reference these in the audio commentary (Rust, 2001).

More than formal tutorials.

Tutor-student conversations can be timely, rich and formative.

Conversations can be daunting to students who feel anxious about discussing their work and the formative opportunity can get lost.

Capture conversations between student and tutor, e.g. where the tutor is offering feedback on work-in-progress.

Students can focus on the live conversation in situ and together agree actions - and then review it later.

Recorded by either student or tutor as a summary conversation to highlight the points raised in a discussion about a student’s work.


Personal feedback conversations

e.g. studio, lab, or field conversations

Tip! - Ask the student to record and then send you a list of agreed action points.

Broadcast or generic feedback considers cohort performance

It is quick to turn around, e.g. use an early sample of marking

Tutor records a single summary message focused on strengths, weaknesses and clarification.

Members of the class are not singled out but encouraged to compare their own performance against points raised.

It can be distributed without concern over privacy.

Can be reused to prepare subsequent cohorts.

However, listeners must see relevance to them, so structuring feedback against criteria is useful.

Best when combined with personal feedback e.g. rubrics or self-assessment tools, and action planning activities


Broadcast or generic feedback

Tip! - Ask students to self-assess and then send three agreed action points based on how they estimate their performance.

Students give each other feedback, 1 - 1 or team-team

Giving constructive feedback is challenging (Gibbs, 1999)

Need to introduce students to how giving good feedback

Valuable as much for the person giving it as it is for the recipient.

Benefits

Promotes co-operation and shares knowledge and methods.

Encourages student professionalism and develops communications skills.

Knowing how to give feedback is a valuable skill.


Peer review audio feedback

Be aware:

Quality and consistency of the feedback may be disputed

May be rivalries within the cohort

So, award some marks for quality of feedback given

Tutors record tutor group conversations as,

Generic feedback – “How did the cohort do?”, or

Personal feedback – co-marking assignments, or

Modelling exemplary arguments

Two examples informed this model:1. First, demonstrating consistent marking and thinking

across the teaching team;2. Second, generating feedback given during a field trip

where the excitement and authenticity of the trip was not only evident in the student’s work, but also in the tutor’s reflection upon it.


Tutor conversations

Tip! – Good academic CPD promoting consistent tutor team thinking across module and course

a broadcast stem is produced and this is appended with a message targeted at individuals (Ribchester et al., 2007)

Tutor,

summarises cohort performance and clarifies general misconceptions

Copies the generic stem and appends personal feedback

(Copy & Paste in Audacity)


Generic+Personal

Audio feedback researchWhat do we mean by audio feedback?

Research on audio feedback Audio feedback is designed for context

Case studies tend to be used

Caution: Even with large data sets, difficult to generalise and extrapolate findings from one context to another.

Often confusion about causality, e.g. assumption that engagement causes learning

Lack of context including experience of academic innovator

However,

many common issues

much agreement about the benefits and principles

Audio feedback researchStudent experience of audio feedback

What some students* sayNearly all (92.6% of 78) indicated a preference for audio feedback

Several intersecting reasons:

1. Accessible - easier for them to listen to feedback than read it

2. Explaining - found audio more focused, clearer and more helpful

3. Interactive - tone of voice heightened comprehension

4. Supportive and personally engaging and sense of care

• From a data set collected by Anne Nortcliffe, SHU, 2013.

• Anne Nortcliffe is experienced at designing and using audio feedback.

• Her students in this case were engineers

“There was more room for explanation with the audio”“Hearing an actual voice explaining in detail an evaluation of my paper is much more effective than reading short little comments here and there on a paper.”

“a lot easier to understand exactly what I needed to do”“…you were able to elaborate…”"I can understand better through your tone”"It makes more sense"

“It’s like your in class”"It showed me what you thought of my paper"

Audio feedback researchLiterature

Audio feedback benefits (1) Timely – received when useful and available when needed

(Nortcliffe & Middleton 2008)

Clear and comprehensible - tone, nuance, and personal input add layers of meaning for the recipient, better than illegible handwritten(Carruthers et al., 2015; Laughton, 2013; Gould & Day 2013; Olesova & Richardson, 2011; Middleton et al., 2009; Davies et al., 2009; Davis & Ryder, 2009; Nortcliffe & Middleton 2008; Walker)

Formative - it reinforces the construction of knowledge and leads to ‘feedforward’ actions(Nortcliffe & Middleton 2008)

Personalised - the use of voice powerfully connects the tutor to the student, emphasising a sense of care and direct interest in the student’s work (Chalmers et al., 2014; Lunt & Curran, 2010; Dixon, 2009; Merry & Orsmond, 2008; Nortcliffe & Middleton, 2008; Ice et al., 2007; Rotherham, 2007)

Richer and more authentic - the giving of feedback is more easily situated in meaningful real world situations and activities (King, McGugan and Bunyan, 2008)

Audio feedback researchLiterature

Audio feedback benefits (2)

Promoting a dialogic relationship - the capture of spoken feedback emphasise the value of conversation and interpersonal interaction to learning(Orsmond et al., 2013)

Time efficient - recorded feedback is often quicker to produce than written feedback(Lunt & Curran, 2010; Dixon, 2009; Rotherham, 2009; Ice et al., 2007; Nortcliffe & Middleton, 2007)

Engaging and replayable - students like to pause, rewind and listen again(Rotheram, 2007)

High quality and great quantity - providing detailed explanations to clarify concepts and processes(Voelkel & Mello, 2014 ;Merry & Orsmond, 2008; Rotheram, 2009)

Tips!Audio Feedback Checklist

Preparation1. Decide on the best method for your context.2. Be realistic.3. Do a non-critical dry run and involve your

students if possible.4. Prepare students: rationale, learning

enhancement benefits, check it works technically, discuss how you expect them to use feedback.

5. Identify the right space and put a ‘Quiet Recording’ sign up if necessary!

Recording6. Keep close to the microphone to reduce

background.7. Monitor or test the recording level – is it clear

and loud enough?8. Use the Pause button for thinking time.9. Never ever edit! Just start again.10. Get used to the sound of your own voice!

Content11.Include the student's name, assignment title,

and date at the start of each recording. Name the recording systematically so you can browse the recordings to organise them.

12.Use the medium for its strengths – be sensitive, personal, clear and direct. Be interested in the student and their work.

13.Conclude with a summary of actions you expect them to take

14.Evaluate and develop the approach.15.Remind the student to store, revisit and

review the feedback.

Visit the Audio Feedback Toolkit http://melsig.shu.ac.uk/?page_id=1196

Particular thanks to Anne Nortcliffe and to academic innovators everywhere

http://melsig.shu.ac.uk/?page_id=1196

Education

Audio feedback design models and tips