Effective strategies for online testing in large classes · PDF fileIn MATH2019 online tests do not count towards nal grade. ... Effective strategies for online testing in large classes

E�ective strategies for online testing in large classes

Jonathan Kress

School of Mathematics and StatisticsUniversity of New South Wales, Australia

Maple TA User Summit 2014

Jonathan Kress E�ective strategies for online testing in large classes

UNSW Australia and me

University of New South Wales

Sydney Australia

54000 students

Large Science and Engineering Faculties

25% international students

UNSW in top 8 of 35 Australian Universities

School of Mathematics and Statistics teaches approximately 3000�rst year students

Jonathan Kress

Senior lecturer in the School of Mathematics and Statistics

At UNSW since 2001

Creating and administering online tests since 2004


Courses using Maple TA at UNSWSchool of Mathematics and Statistics

MATH1131/1141 Science and Engineering (1700)assumed knowledge / class test preparation / online Maple /invigilated Maple lab

MATH1151 Actuarial Stuides (200)class test preparation / online Matlab

MATH1011 Lower level �rst year course (100)weekly tutorial preparation, online Maple

MATH1031 Maths for Life Sciences (400)class test preparation, Maple

MATH1041 Stats for Life Sciences (400)online quizzes, invigilated R or Excel lab

Higher Year Stats/Numerical Methods/Engineering Maths (2000)intro to matlab, various online tests

Faculty of Engineering

Higher year undergraduate and Masters Engineering (1000)mostly longer assignments


Why use online testing and Maple TA?

Regular testing encourages students to keep up to date

Provides immediate feedback, anytime, anywhere

Reduces labour on routine marking(What should be done with that saving?)

Allows repeated attempts according to student need

Personalisation of learning

encourages collaboration that isn't just copying

Analysis of student responses can be used to improve teaching

Very �exible algorithmic question generation and grading

It lacks features we don't want during invigilated testseg, messaging or access to course materials

Plan to use adaptive questions in the future


Di�erent scenarios for use of Maple TA

Formative � assessment for learning

Assumed knowledge quiz (placement test)for students in MATH1131 who should be in MATH1011

Class test preparationin the week preceding a written class test

Weekly tutorial preparationa short online test preceding each week's tutorial

Progressive (mastery) testsmust pass each test to access the next test

Practice followed by actual testunlimited practice but limited number of attempts count

Practice testsno marks awarded, feedback only

Summative � assessment of learning

Invigilated lab teststraditional exam conditions but Maple TA graded


How students view Maple TA

Students expect to be rewarded with marks.

Students expect things to work without having to read instructions.

Student confusion about the naming of �Maple TA� and �Maple�lead to an increase in student use of Maple!

From a MATH1131 survey in 2011:


Student comments (2013 and 2014)Feedback from quizzes was very helpful.

Constant tests are good.

Online tests are mostly just annoying and take up valuable time.

(Course could be improved by) more online questions. . .

The online tests and Maple tests instead of assignments were the bestfeatures for this course.

The online tests helps in getting a better grip over the subject.

(Best features of the course) on-line tests

I found the online tests provided a good revision tool as they providedimmediate feedback which I could use to improve.

Should have more smaller online tests on maple ta so that it forcesstudents to learn and be up to date with the course as it moves quickly.

Regular online and class tests prompt us to be continually up to date. . .

The online quizzes kind of force you to learn how to do each question, andsince you have 5 tries you can work on a question until you get it right.

More weighting on online tests. The time used to complete the onlinetasks was not appropriate in comparison to the percentage it makes up.

Online quizzes were well set out. It was useful to have several attempts.


Engagement

Do students engage with tests?

In MATH1131 online tests are worth approximately 1% each.Approximately 90% of students attempt each test.

In MATH2019 online tests do not count towards �nal grade.Between 1% and 10% of students attempt the tests.

Incentives

Marks count towards �nal grade0.5% per 30 minute online test seems su�cient!

Immediate feedback

Flexibility

allow some tests to be missed or failedeg only count best 8 of 12

allow repeated attemptsonly count best markallow plenty of time

More and smaller tests to avoid unnecessary repetition


Set and forget

Our main current use of Maple TA is to provide practice questions andfeedback to students with minimal intervention.

Provide written and screen capture video instructions.

Link to Maple TA provided next to link to instructions.

Maple TA class homepage tells students to read the instructions.

Compulsory �Using Maple TA� test must be completed �rst.


�Using Maple TA�


�Using Maple TA�


Set and forgetAllow multiple attempts.�Revision only� versions of tests available after deadline.Course outline states no extenstions or additional attempts allowed.Only best attempt counts.Allow some tests to be missed without penalty.Questions contain embedded instructions.


Tutorial preparation tests in MATH1011

Encourage regular preparationfor tutorials.

MATH1011 has separatealgebra and calculuscomponents.

Weekly tutorial preparationtests for the calculus introducedin 2004.

Amnesty for missed tests inweek 5 of semester.

Course survey commentsrequested similar tests foralgebra.

Evidence of increase in averagescore for �nal exam questionsrelated to online tutorial prepquestions.


Tutorial preparation tests in MATH1011


Class test preparation

Provide feedback to help students prepare for class tests.

Initially allowed 3 attempts but spent too much time dealing withspecial pleadings for extra attempts so increased to 5 attempts.


Invigilated tests

Maple component of MATH1131 is assessed with invigilated lab test

students book time for test on Moodlestudent ID cards are checkedcomputers (112) have exam mode with no internet access

Before 2010

paper based teststudent Maple worksheets collected, printed then marked by handand returned in tutorial classvery labour intensivestudents had no feedback prior to test

Since 2010

Maple TA testtest is available for unlimited practice on Maple TA before lab testtest restricted to lab IP rangeeasy immediate grading once all tests have �nished

Since 2014

MATH1041 Stats for Life Sciences has an invigilated lab test partlygraded by Maple TA.


Maple Lab Test


Allowing for special cases

Although we say no extenstions or additional attempts will begranted, in reality there will be special cases.

We use �revision only� versions for special cases.

�Revision only� tests also depend on �Declaration� and �Using MapleTA� tests.

�Revision only� test become available as soon as `for marks' deadlinehas passed.


Summary

Expect bugs in your questions.

Expect students to do uncontrolled tests collaboratively.

Students won't attempt �optional� tests.

Assume students won't read the instructions.

Assume students will start the test when they are not prepared.

But if you just want students to learn, the solution to all of this is. . .

don't worry!

O�er enough marks that the students care but you don't.

Provide lots of ways for lost marks to be recovered.

Have a strict non-intervention policy.


Summary

Expect bugs in your questions.

Expect students to do uncontrolled tests collaboratively.

Students won't attempt �optional� tests.

Assume students won't read the instructions.

Assume students will start the test when they are not prepared.

But if you just want students to learn, the solution to all of this is. . .

don't worry!

O�er enough marks that the students care but you don't.

Provide lots of ways for lost marks to be recovered.

Have a strict non-intervention policy.


Documents

Effective strategies for online testing in large classes · PDF fileIn MATH2019 online tests do not count towards nal grade. ... Effective strategies for online testing in large classes