The Mechanical Vibrations QuizAn example of formative assessment

Ines Lopez Arteaga

19th March 2012

Overview

• Background• Goals• Approach• Results• Conclusions

Course information

• 2nd year Mechanical Engineering

• Number of students +/- 200

• Course organization: • 4h/week Lectures• 2h/week Guided self-studies

• Assessment: written examination with book

• Material: course book and handouts

3

Situation 2008

• Average first-time pass rate 2000-2008: 45%

4

Improvements 2006-2008

• 2006: New course slides

• 2007: New examination format including theory

• 2008: Use of blackboard instead of slides

• Exercises and exams previous years with solutions available in Studyweb

5

Situation 2008

• Lecture attendance dropped from about 130 to 70 during the course

• Only 30% of the students attend guided self-study

• Students do not study course book

• Consequence:• Students know how to solve the exercises …• … but do not understand the concepts behind

6

Questions

• How to stimulate the students to be active with the course book and other material?

• How to repeat/review important concepts without being “repetitive”?

• How to give them feedback on their progress during the course?

• …. And all this at a “low cost” in terms of structural time investment.

7

Situation 2012

• Average first-time pass rate 2000-2008: 45%• Average first-time pass rate 2009&2010: 70%

8

Goals

• Activate students• Give feedback• Monitor student progress• Review concepts

Alternatives 10

Alternatives 11

Approach

• Ask short questions during lectures.

• Use sharepoint applications in Studyweb to collect answers and show statistics.

• Give the students a prize in the form of 0.5 extra point.

• Add a competitive element to make it more fun for the best students.

• The questions should not be too difficult. The goal is to help them learn and not to let them fail.

Course design: weekly structure 13

Day 1 Day 2 Day 3

Lecture (45min) Lecture (1 hour) Guided self-study (45 min)

Lecture (45min) Guided self-study (45 min)QUIZ (30 min)

Quiz design

• During lecture

• Students login to Studyweb

• Lecturer displays 6 to 8 (multiple choice) questions on the screen

• Students answer to survey on Studyweb and send their answers after the last question

• Lecturer shows the students’ answers on the screen and gives feedback on the correct answers

14

Quiz design

• After lecture

• Lecturer exports individual answers to Excel and calculates individual scores

• Lecturer publishes weekly scores in Studyweb

• Week 8: Final Quiz• Best students get a prize• Every student with a final score >55% gets 0.5 point extra

15

16

Main ingredients of quiz

• Peer-instruction: Students discuss with peers during quiz

• Formative assessment: Inmediate feedback during lecture

• Long term feedback in the form of published individual scores

• Reward: Additional 0.5 point on top of examination mark

17

The Mechanical Vibrations QuizTest 1

Week 1

Question 1

• How many degrees of freedom (DOF) has this

system?

a) 1

b) 2

c) 3

d) 0

1Z

0

m

)(ts1

)(ts2

2Z

Question 2

• How many degrees of freedom (DOF) has this

system?

a) 1

b) 2

c) 3

d) 0

1Z

0

m

)(ts1

2Z

Studyweb surveys 21

Inmediate feedback 22

Individual results 23

Publish weekly scores 24

Final Quiz

Situation 2012

• Average first-time pass rate 2000-2008: 45%• Average first-time pass rate 2009&2010: 70%

26

Conclusions student survey

• Most students are positive about the quiz.

• Most students say that the extra 0.5 point motivates them.

• The opinions about the competitive component and DAF sponsoring are very diverse.

• Some students indicate that discussing with their peers helps them understand better.

Conclusions student survey

• 56% respondents didn’t start studying earlier than normal, 21% did.

• 20% prepared better for the lectures, 58% didn’t.

• Only 13% prepared better for the guided self-study, whereas 57% didn’t.

• 54% thinks the feedback after the tests was a good indicator of the aspects they need to improve, 22% does not.

Facts

• Lecture attendance significantly higher than in previous years.

• Guided self-study attendance significantly higher than in previous years.

• Only 8 students passed in both 2009 and 2010 due to the extra 0.5 point.

• Students make less conceptual mistakes in their written exams.

Conclusions

• Most students say they did not change their studying attitude.

• But the exam results are significantly better than in previous years.

• Students are “forced” to think about a number of concepts themselves during the tests, which helps them understand better without giving them the impression they are working harder.

• Discussing with their peers helps them settle their knowledge better.

30

The Mechanical Vibrations QuizAn example of formative assessment

Ines Lopez Arteaga

19th March 2012

Results 32

• Average first-time pass rate 2009&2010: 70%