Maintaining student engagement with formative

assessment(using Questionmark)

Tracey Wilkinson Centre for Biomedical Sciences Education

School of Medicine, Dentistry and Biomedical Sciences

Plan

• why use objective assessments?• advantages of regular formative assessment• Questionmark – types of questions available– examples– have a go!

• exchange ideas• final word

Why use objective assessments?

• diagnostic (often early)• self-assessment (feedback to students)• continuous, formative (feedback to students

and teachers)• summative (counted towards final mark)

Formative assessment

• improves academic performance• feedback element particularly important– allows both teacher and student to monitor

progress

• engagement – summative element

• increasing requirement for online / computer-assisted assessment

Advantages of online tests– advantages for lecturer • reduced time commitment• logistics of individual feedback (especially for large

classes)• allow testing of wide range of subjects• more frequent assessment possible• analysis of questions

– advantages for student• convenient access• immediate feedback (individual)• monitor progress

Disadvantages of online tests

• time and effort to construct good questions• higher order questions (Bloom’s taxonomy) • written communication skills or creativity not

easily tested

Questionmark Perception Assessment Management System

– Queen’s pilot project– easy to use online assessment tool– more question types available (than QOL)

Examples from Questionmark

Have a go!In pairs, or on your own, discuss the types of questions that might be suitable in your particular discipline.

- true-false - multiple choice / response - matching - ranking - fill in / select a blank - text match - hotspot / drag and drop - Likert

Compare the questions you have constructed with your neighbour. Any suggestions for improvement? Any questions that might test higher cognitive ability?

Now try constructing a few simple questions in your own discipline(s).

Logic and reasoningTrain service suffers when a railroad combines commuter and freight service. By dividing its attention between its freight and commuter customers, a railroad serves neither particularly well. Therefore, if a railroad is going to be a successful business, then it must concentrate exclusively on one of these two markets.

For the argument to be logically correct, it must make which one of the following assumptions?

a. Commuter and freight services have little in common with each other.b. The first priority of a railroad is to be a successful business.c. Unless a railroad serves its customers well, it will not be a successful business.d. If a railroad concentrates on commuter service, it will be a successful business.e. Railroad commuters rarely want freight services as well.

American LSAT (Law School Admissions Test)

Answer = c

Logic and reasoningA small software firm has four offices, numbered 1, 2, 3 and 4. Each of its offices has exactly one computer and one printer. Each of these eight machines was bought in either 1987, 1988 or 1989. The eight machines were bought in a manner consistent with the following conditions:– the computer in each office was bought either in an earlier year than, or in the same

year as, the printer in that office– the computer in office 2 and the printer in office 1 were bought in the same year– the computer in office 3 and the printer in office 4 were bought in the same year– the computer in office 2 and the computer in office 3 were bought in different years– the computer in office 1 and the printer in office 3 were bought in 1988

If the computer in office 3 was bought in an earlier year than the printer in office 3, then which one of the following statements could be true?

a. The computer in office 2 was bought in 1987.b. The computer in office 2 was bought in 1988.c. The computer in office 4 was bought in 1988.d. The printer in office 4 was bought in 1988.e. The printer in office 4 was bought in 1989.

American LSAT (Law School Admissions Test) Sample Test, June 1999 Answer = b

Regular assessmentsHave any of you used regular formative assessment (paper, personal response, Queen’s online, other) with feedback?

In groups of two to three, discuss the ways in which you have used, or may be planning to use, regular formative assessment to enhance the learning in your classes. Be prepared to give feedback to the whole workshop.

Feedback – any areas of good practice?

Final word

• the students– uptake– find them useful– ask for more– pressure to do well as teacher sees them– good revision for tests

References

• Brown G, Bull J and Pendlebury M (1997) Assessing student learning in higher education. London: Routledge.

• Bull J and McKenna C (2004?) Good practice guide in question and test design.

• Fry H, Ketteridge S and Marshall S (2003) A handbook for teaching and learning in higher education. London: RoutledgeFalmer.

• Miller T (2009) Formative computer-based assessment in higher education: the effectiveness of feedback in supporting student learning. Assessment and Evaluation in Higher Education 34:181-192.

• Nicol D (2007) E-assessment by design: using multiple choice tests to good effect. Journal of Further and Higher Education 31: 53-64.

• Wiliam and Black (2003) In praise of educational research: formative assessment. British Educational Research Journal 29:623-637.