Dr Nadia WagerDr Nadia Wager

Principal Lecturer in Principal Lecturer in Criminological Psychology Criminological Psychology

and and Portfolio Leader in Psychology Portfolio Leader in Psychology

and Sociologyand Sociology

Emotional, Attitudinal and Behavioural Emotional, Attitudinal and Behavioural Responses to Assignment FeedbackResponses to Assignment Feedback

Why the interest in feedback??

Gibbs (2005) - motivated to raise standards in Higher Education ???

The we need to begin conceptualising assessment as a learning modality as opposed to a measurement tool.

Ramsden (1991) the helpfulness of feedback is the single factor that distinguishes between students’ perceptions of their favourite and least favourite modules.

Hattie (1987) – meta-analysis of 87 studies of student achievement – feedback is the most potent factor in enhancing performance

BUT often not used by students – Wager (2004) – 29% of the students surveyed admitted to only looking at the grade and ignoring the feedback

Why the reluctance to engage with the feedback?

Strategic learning style

Feedback lacks clarity and direction

Leaves some students confused and feeling helpless

Emotional inhibitionGood grade – motivated to maintain a positive mood

Low grade unable to tolerate further assaults in self-esteem

Good practice according to the literature

Languageavoid ‘final’ vocabulary, e.g. good, right or poor -

does not permit change or repetition

Use formative rather than summative language, e.g. ‘do, don’t and how’ rather than ‘did, didn’t and why’ - gives direction for improvement

Focus of Feedbackcomment on the essay and not the author

Boud – power differences between lecturer and student

So… my new study

Two Parts:

Survey of student experiences of good and bad feedback

Experimental study examining the emotional, attitudinal and anticipated behavioural responses to feedback

Participants

Opportunistic sample of 47 psychology students from all three year groups

Asked to reflect on their last piece of assignment feedback

Questions on survey were open ended

Q(1): How did the feedback make you feel?

26 (55.3%) reported a positive emotional response

4.9% a negative response

10.6% relative indifference

19% inappropriate or missing data

Positive emotions: Three themes

motivating and empowering experiences

confirmation

experience of conflicting emotions

Motivating and empowering experiences

42.3% of the positive experiences

Made me feel positive for future assignments as it gave me something to work towards and improve upon.

Really pleased that I received positive feedback – chuffed and felt like I’d got more confidence to do it again.

It helped me understand better how to improve my grade and made me feel more confident.

It made me feel great and excited.

Confirmation

27% of the positive experiences

Reassured

Relieved

Very good. I knew exactly what my lecturer was looking for.

Proud of the mark considering I had put the effort in

Happy that it passed

The duality of emotions

Good, but I was also annoyed (with myself) because some of the feedback related to things I should have done, but I just forgot to do.

I was disappointed with the mark, but the feedback was good because it told me how to improve and where I went wrong

I felt happy with the positive comments and low with the negative ones

Negative Emotions

3(43%) referred to a sense of disappointment,

2 (28.6%) related a sense of unjust accusations,

1 (14.3%) describes a diminution in self-worth

1 (14.3%) describes a loss in confidence.

‘Unjust accusations’Down and like my work wasn’t good enough,

although I had worked hard. Also some wasn’t accurate so the teacher hadn’t read my work properly.

Upset and frustrated, I was accused of sticking too close to the text when I feel that was not the case.

Worthless - Very disappointed with the grade and the manner in which the lecturer addressed my work made me feel worthless.

Feeling okay…but confused

A bit confused, but overall okay.

It made me feel okay as it was very detailed but straight to the point.

Okay, the content was positive and well written, but I would have liked to have known how to improve further.

What was ‘good’ about the good feedback?

All 47 reported having experienced good feedback

7 themes emerged from the responses:

positive encouragement & highlighting areas for improvement 28.6%

offering guidance on how to improve - 25.7%writing with a future rather than a past orientation -

20%clarity of both expectations expressed by lecturers

and the wording of the feedbackopportunity to discuss the feedback with the lecturer.confirmation of success

The importance of clear directive guidance

It gave direction for the future, i.e. what was wrong and how to change it.

Telling me my mistakes and how to improve on them for my next coursework.

But other’s highlight how we can point out what but they really do need the how….

Well explained, but I still feel the need to get together with someone to work out how to improve.

Advice for the future and where it went wrong, however, I am unclear where to go with it next.

Less Favourable Experiences of Feedback

15% of the sample claimed that they had never received bad feedback!!!!

3 themes emerged; the lack of direction for improvement -23% the lack of detail – 23% the perception of being insulted – 12.8%

Lack of direction for improvement

The feedback said that some points were wrong, but it didn’t give an explanation for how to improve.

Where I felt I had put a great deal of effort into the work but was only told that I didn’t answer the question, with no indication of where I went wrong.

Perceived Insults

It only contained criticism so I didn’t benefit from it.

The lecturer was rude to me. To the point of giving me no positive feedback for my work only, negative.

Feedback that insinuates that because the level of work was good the student must have copied the text.

It was bad because it didn’t motivate me to work harder. Also because it put down my work.

It was a bit humiliating, not enough detail for improvement.

How did the good feedback help you?

3 super-ordinate themes

Empowerment – 58.7%reassurance – 8.7%confidence building – 15.2%

Empowerment and taking responsibility for own improvement

It motivated me to improve myself.

Enabled me to see what I needed to do on future assignments.

It means that I know what to look for in my work that could be improved.

It encouraged me to do more independent research for my exams.

Better understanding. It woke me up and made me want to work much harder.

Confidence Building

It gave me confidence, as I wasn’t sure how well my writing skills were due to the original feedback I had previously received. It made me feel that I wasn’t as bad as I had originally thought.

It improved my confidence because I can actually do the assignments.

It boosted my self-esteem and made me what to work harder to keep up the standard.

Reassurance

It meant that I am able to keep up with what I am doing right.

It pointed out that what I am doing is right.

It motivated me to continue with my style of writing and essay structure.

Bad feedback – did it hinder you???

22% did not believe that their experience of bad feed back had any negative impact on them

5.5% felt it had made them work harder

72.3% felt it had hindered their progress in some way – majority referred to the lack of guidance given and the consequent sense of helplessness

Lack of guidance

If I’d been given ideas on how to improve I could have understood where I had gone wrong.

I couldn’t see where mistakes were made

Yes, because I only know where I got it right and where I went wrong, but not why and how to improve. I don’t know what made the good areas good.

Others reported more extreme and enduring effects …

It made me feel really crap and I didn’t want to continue. It affected me quite badly.

Yes it reduced my confidence entirely. It made me reluctant to start my dissertation as I felt I could not write cohesively in an academic manner. This slowed the process of writing my dissertation and confidence that I could do it right.

I was left feeling very negative about the tutor, my course and the experience.

I wasn’t confident or comfortable handing in work to this lecturer again.

Felt low for days.

Felt slightly de-motivated when it came to writing the next assignment.

De-motivated me to work hard for this subject.

It made me feel that I wasn’t capable of achieving.

Students’ Advice to Lecturers

Avoid providing feedback to student through Blackboard with our ID numbers which everyone can see.

It has been good to see lecturers address feedback to the whole class if most of the class encountered the same problem with the assignment.

Provide an example of a good paper so that we can compare and understand how to achieve this level.

Be polite.

The experimental study…

An experimental study of student emotions, attitudes and anticipated behavioural responses on receiving fictitious feedback.

2 independent variables were manipulated in the study

To whom the feedback is directed - 2 conditionsaddressed to the author and written in the past tensewritten to the assignment and written in the future tense.

Whether or not guidance as to ‘how to improve’ statements

were provided.

Thus four different variants of the stimulus material.

Age was entered as a covariate

Example of feedback written addressing the assignment with a future orientation

All sources used were appropriate for this assignment however, remember to use original journal articles and some of the key texts highlighted on the reading list.

Many of the key arguments have been identified, although inclusion of one or two more may have benefited this assignment.

Clear referencing in the text, appropriate use of the Harvard system, just remember to be particular about the use of punctuation.

Now with ‘how to improve’ suggestions

All sources used were appropriate for this assignment however, remember to use original journal articles and some of the key texts highlighted on the reading list. The range can be broadened by looking at both theoretical and empirical material relating to the topic. There are some chapters or journals which are specifically theoretical (e.g. …….) whilst other journals tend to focus on empirical studies (e.g. …..) – try to use a selection of both. The electronic database collections can be particularly useful.

Many of the key arguments have been identified, although inclusion of one or two more may have benefited this assignment. Remember that reading around the topic when planning the essay should begin by being quite broad and often a focus on ‘review’ papers/ chapters can be quite helpful at the start. This will help the identification of all of the central arguments and important search terms.

Clear referencing in the text, appropriate use of the Harvard system, just remember to be particular about the use of punctuation. The full stop goes after the reference in brackets at the end of the sentence rather than before the bracketed reference [e.g. Cats are always

clever (Smith, 1984).] This means that we know that the reference relates to the preceding sentence.

Same comments directed at the author and in the past tense

You have selected appropriate sources, but the range is a little narrow.

You have missed a few of the key arguments.

Your referencing in the text was largely okay, however, your punctuation in relation to referencing was incorrect.

Now with ‘how to improve’ suggestions

You have selected appropriate sources, but the range is a little narrow. The range can be broadened by looking at both theoretical and empirical material relating to the topic. There are some chapters or journals which are specifically theoretical (e.g. …….) whilst other journals tend to focus on empirical studies (e.g……..) – try to use a selection of both. Make sure you attempt to use the electronic database collections.

However, you have missed a few of the key arguments. Remember that reading around the topic when planning the essay should begin by being quiet broad. Often focusing on ‘review’ papers/ chapters can be quite helpful at the start of reading for the assignment. This will help you to identify all of the central arguments and important additional search terms.

Your referencing in the text is largely okay, however, your punctuation in relation to referencing was incorrect. The full stop goes after the reference in brackets at the end of the sentence rather than before.

Assessing Emotions

Please indicate what emotions were evoked when you read the feedback whilst imagining that it related to your own assignment?

Pleased Not at all _/_/_/_/_/_/_/_/_/_ Extremely Angry Not at all _/_/_/_/_/_/_/_/_/_ Extremely Relieved Not at all _/_/_/_/_/_/_/_/_/_ Extremely Upset Not at all _/_/_/_/_/_/_/_/_/_ Extremely Proud Not at all _/_/_/_/_/_/_/_/_/_ Extremely Shamed Not at all _/_/_/_/_/_/_/_/_/_ Extremely Content Not at all _/_/_/_/_/_/_/_/_/_ Extremely Embarrassed Not at all _/_/_/_/_/_/_/_/_/_ Extremely Deflated Not at all _/_/_/_/_/_/_/_/_/_ Extremely Motivated Not at all _/_/_/_/_/_/_/_/_/_ Extremely Happy Not at all _/_/_/_/_/_/_/_/_/_ Extremely Depressed Not at all _/_/_/_/_/_/_/_/_/_ Extremely Inspired Not at all _/_/_/_/_/_/_/_/_/_ Extremely Irritated Not at all _/_/_/_/_/_/_/_/_/_ Extremely

2 scales: Positive and Negative Emotions

Assessing Attitudes towards the feedback

On reading the feedback, how likely you are to think the following, if this feedback related to your assignment?

a) The comments were fair Not at all _/_/_/_/_/_/_/_/_/_ Extremelyb) The comments were helpful Not at all _/_/_/_/_/_/_/_/_/_ Extremelyc) The comments were insulting Not at all _/_/_/_/_/_/_/_/_/_ Extremelyd) The comments were accusatory Not at all _/_/_/_/_/_/_/_/_/_ Extremelye) The comments were encouraging Not at all _/_/_/_/_/_/_/_/_/_ Extremelyf) The comments were meaningless Not at all _/_/_/_/_/_/_/_/_/_ Extremely

Two scales Favourable perceptions Unfavourable perceptions

Meaningless removed from the analysis

Anticipated behavioural responses

Having received this feedback, how likely do you think you would do each of the following in relation to the assignment and the feedback received?

Approach the lecturer and ask for more guidance Not at All _ / _ / _/ _ / _ / _ / _ / Very Likely

Think of the lecturer as incompetent Ignore the feedback Check over the assignment to see whether the comments were justified Ask someone else to read through your assignment to see if the comments

are justified Decide on a course of action to rectify one or two of the weaknesses Decide to start working earlier on future assignments. Seek further guidance from the Academic Skills Support Unit File the feedback sheet safely so that you can refer to it when preparing for

your next assignment Throw the feedback sheet away Discuss the feedback with a friend to help

workout how to improve

Behavioural Responses (Cont.)

Four factors/ subscales

De-motivated inaction

Seeking clarification

Empowered action Disbelief in the feedback

Analysis

A 2 X 2 MANCOVA was performed controlling for the age of the participant

Preliminary assumptions testing was conducted to check for normality, linearity, univariate and multivariate outliers, homogeneity of variance-co-variance matrices and multicollinearity. The following two violations were noted. Box’s Test of Equality of Covariance Matrices produced a p = .001, thus suggesting that this assumption had just been violated. Consequently, Pillai’s Trace statistic was used to determine significance (Tabachnick & Fidell, 1996) and the Bonferroni adjustment for multiple analyses was applied. Thus a significance level of p = .006 is required on the between test of between-subjects effects. However, the Levene’s test of Equality of Error variances, was also violated on one of the dependent variables (holding an unfavourable perception of the feedback) and thus a more conservative alpha level, (p =.003) was utilised in determining significance in relation to this variable.

Results

Both of the independent variables produced results that attained a level of significance or borderline significance.

Where, the Pillai’s Trace for the presence or absence of ‘how to improve’ suggestions revealed (F8,31 = 2.258, p = .050, 2 = .368)

Whether the feedback was addressed to the assignment or the author revealed (F8,31 = 2.084, p = .068, 2 = .350).

Main effect of ‘how to improve’ suggestions

Only two of the dependent variables attained a borderline level of significance.

Firstly, the inclusion of ‘how to improve’ suggestions was associated with an almost statistically significant higher score on favourable attitudes towards the feedback (mean = 17.14, sd = 7.42, F8,31 = 8.057, p = .007, 2 = .175), in comparison to when these suggestions are omitted (mean = 13.38, sd = 9.70).

And almost statistically significant in terms of the anticipation of engaging in de-motivated inaction. Thus, those who did not receive the ‘how to improve’ suggestions were more inclined (mean = 8.35, sd = 4.52, F8,31 = 7.761, p = .008, 2 = .170) to report de-motivated inaction in comparison to student who were presented with such suggestions (mean = 5.50, sd = 3.03).

Improvement advice

No improvement advice

Author

Assignment

4

6

8

10

12

14

16

18

The main effect of 'how to improve' suggestions upon favourable attitudes towards the feedback

Improvement advice

No improvement advice

Author

Assignment

2

3

4

5

6

7

8

9

10

11

The main effect of 'how to improve' suggestions on de-motivated inaction

Main effects of ‘to whom the feedback is addressed’ (i.e. author or assignment)

Addressing the feedback to the author and writing it in the past tense was associated with:

significantly less positive affect (mean = 29.0, sd = 16.19, F8,31 = 9.146, p = .004, 2 = .194)

and higher levels of negative affect (mean = 36.07, sd = 14.28, F8,31 = 7.959, p = .008, 2 = .173) in comparison to when the feedback is written to the assignment and in the future tense (positive affect mean = 40.93, sd = 15.15; negative affect mean = 23.07, sd = 11.68).

No significant interaction effect was observed for the two independent variables.

Improvement advice

No improvement advice

Author

Assignment

20

25

30

35

40

45

The Impact of 'How to improve' suggestions and to whom the feedback is addressed on Positive Emotions

Improvement advice

No improvement advice

Author

Assignment

20

22

24

26

28

30

32

34

36

38

Impact of 'How to improve' suggestions and to whom the feedback is addressed upon negative emotions

Conclusions

The ‘how to improve’ suggestions appear to impact upon attitudes and intended behaviour and thus we can imagine would be directly related to whether or the students attempt to use the information. It is proposed that the how to improve statements may actually enhance self-efficacy.

Addressing the feedback directly to the author, particularly in the context were either the mark is poor or there are significant areas for improvement can have a detrimental impact on the student – which may erode self-esteem and confidence.

Constructing ‘how to improve’ suggestions

Small group activity

Thank you…