Humanities Teaching and Learning Showcase

Innovation in on-line assessment and feedback: how we brought the WOW factor to French

grammar

Organisation of core French language modules

Fren10210 – 20 credits (3

contact hours)

2hr Written / grammar seminar

1hr Oral seminar

ILP Programme

20 credits = 200 hours = 6 hours outside classroom each week (h/w and Independent Learning Programme =

ILP)

WOW factor in this bit

Our ‘challenge’Students not engaging sufficiently in the building-

blocks of language learning – grammar and phonetics;

Limited class time to devote to such time-consuming basic elements;

ILP ‘encouraged’ with a week by week schedule of activities across all four skill areas, including reflective tasks and action planning, building toward a portfolio of tasks BUT students still not working regularly or systematically;

120-150 students in each year means we cannot take in and mark or assess ILP

Our proposed ‘solution’Creation of a strand of ‘Monitored ILP’ activities to

complement existing class and independent schedules;Weekly on-line (Blackboard) grammar and

oral/phonetics exercises: completion tracked by system;Roughly 1hr a week (30 mins x 2);Exercises live for 10 working days but attempted as

many times as desired;90% of tasks must be completed each semester; 5%

available each semester (so at end of year students come out with either 0%, 5% or 10% for this element of their assessment diet;

Minimum work for convenors to check completion stats on BB…

Problems we considered:Software solutions versus VLE compatibility= do the best we could with what we had / BB

trainingTime issues- Considerable tutor time investment= worked a ‘just-in-time’ system, got quicker as

went along, once it’s done, it’s done!Quantity versus quality (see Ferris, 2004)- the more one does, the better one gets?= provision of FEEDBACK = added-value

WOW = Weekly on-line WorkFormerly known (2011-12) as “Monitored

Independent Learning”“I think my independent learning should be

just that... independent, and not dictated by the university”

“I strongly disagree with the idea of a Monitored ILP. I was enraged when I found

out that the reason we have them was because last years' students said they were not motivated to do independent work. To be at university you have to be prepared and you have to WANT to do independent

work”.

What we came up with for grammar…1 x weekly self-test;Discrete grammar points – MCQ, matching,

gap-fill , jumbled sentence etc;Immediate feedback (as ‘personalised’ as

possible) for right and wrong answer;+ monthly targeted prose (short translation

paragraphs into French)No right or wrong answer, machine cannot

‘mark’ it…

What is the ‘added value’?Multi-pronged, multi-media feedback approach…

Motivate, stimulate, satisfy different and varying learning styles and needs

Fair copy + notes on grammar point being testedIndication of page numbers in set text

Link to a micro-tutorial (Powerpoint + narration and rehearsed timings)

‘Live’ correction of a staged student copy (OneNote + jing)

Demo!

So what did our students think?2 x surveys (2011-2012)Questions on all aspects of the (new) course – combination

of numeric and commentsWe will be looking at the results from these 4 (2 x

semesters combined):1.HONESTLY, do you think you would be doing as much

independentlearning if we did not have the monitored ILP section of the

course?

2.The weekly monitored ILP activities are helping me work regularly

and systematically outside the classroom.

3.The feedback given with the Monitored ILP exercises is helpful.

4.Did you find the 'Now watch Annie mark one of your translations live‘

feedbacks useful for the targeted prose exercises?

HONESTLY do you think you would be doing as much ILP if we did not have the Monitored ILP section?

Yes

No

No answer

19.2

79.7

2.2

%

80% say they wouldn’t be doing as much ILP

“If the activities weren't compulsory

and checked I definitely would not do as much so this is

so good for me because it forces me to put the extra time in. The exercises are

really great and varied. I really enjoy them (weirdly!) and they don't take too

long to do.”

The weekly monitored ILP activities are helping me work more regularly and systematically outside the classroom

Strongly agree

Agree

Neither agree nor disagree

Disagree

Strongly disagree

Unanswered

16.7

59.75.6

9.7

6.9

1.4%

76% agree or strongly agree82% strongly agree/agree or have no opinion

The feedback given with the exercises is helpful

Strongly agree

Agree

Neither agree nor disagree

Disagree

Strongly disagree

Unanswered

26.9

50.7

12.8

5.9

0.7

2.9%

78% of students agree or strongly agree90% of students agree, strongly agree or have no opinion

Did you find the ‘Now watch Annie mark one of your translations live’ useful for the targeted proses?

Strongly agree

Agree

Neither agree nor disagree

Disagree

Strongly disagree

I did not watch them

16.2

36.6

13.7

1.4

0

31.1

% 31% of students had not watched them, of those that had:

76%agreed or strongly agreed96%strongly agreed, agreed or had no opinion

Why did the students not watch me?No need, enough feedbackLaziness Technical difficulties Didn’t know about them

“Because having wasted my time completing them I don't want to spend any

more time on Blackboard.”

OTHER COMMENTS

I found the correction videos in which a translation is corrected showing the most common mistakes made a really useful way of picking up on personal mistakes

and remembering them - as opposed to just seeing the correct version and noticing differences between

one's own translation and the correct one.

Something which I find SO USEFUL are the powerpoints where Annie speaks and the work

where Annie marks it with you. I think these are great and I always listen to them. It makes so much

more sense when Annie explains them.

[…]how is it she knows absolutely every mistake I have made, without having even

looked at my work?!

What we have learnt – our studentsFirst years do not feel any ‘urgency’ to

complete the activities;

47%

20%

33%

0% 5% 10%13%

21%66%

0% 5% 10%

First YearOnly a third of

students got the max +10% available

Almost half the students got +0%

Second YearTwo thirds of students did

enough activities to get the max +10%

available

What (else) we have learnt – our studentsThey do not all thrive in the computer age:- It took them longer to get used to the system that we

anticipated (and we had a number of technical problems);

- A good induction and very precise instructions is needed;

- They did not believe the number of activities the system said they had done;

- Some of them want to do exercises with paper and a pen

Only some of them are mature enough to understand why regular and systematic work is necessary.

What we have learnt – the technologyBlackboard has not always proved 100% reliable (TDF

session: The Trouble with Blackboard – ways round!) Once deployed, a test cannot be changed, even if there is

an error in it: we lost all attempts at week 3 and had to ‘gift’ it to all students!

Away from the University, BB is supported differently by different operating systems; audio files proved particularly problematic;

Gradecentre not v. student friendly (paper version in dossier)

BB CANNOT count the number of activities students are doing. A spreadsheet must be downloaded and activities counted for each student;

Live-marking technique + jing screen capture is v. time-consuming, but does get quicker!

It seems to work –

0-3940-4950-5960-6970+

75575648

20

25

21.52317

10

21.52135

70

0 5% 10%

Result (%) in end of year FREN10210 written examination

Perc

enta

ge o

f stu

dent

s aw

arde

d 0,

+

5 or

+10

% fo

r W

OW

com

plet

ion

It seems to work –

0-3940-4950-5960-6970+

2516.396.7

3322.4

15.513.3

4261.2

75.580

0 5% 10%

Result (%) in end of year FREN20210 written examination

Perc

enta

ge o

f stu

dent

s aw

arde

d 0,

+ 5

or

+10

% fo

r W

OW

com

plet

ion

More info?Franc, C & Morton, A “The Use of VLE for

Monitoring Independent Learning in Large Cohort Provision: The Case of French Studies at the University of Manchester” (in Computer Assisted Foreign Language Teaching and Learning: Technological Advance, ICI Global, 2013)

Just email: annie.l.morton@manchester.ac.uk

Gloomy looking book!

Not gloomy –looking Annie and Catherine