Quizzes in Brightspace Online exams

HvA FT - april 2020

When to use? Section 1

01 When to use?

Before the outbreak of Covid-19, Brightspace quizzes could be used to test students' prior knowledge, to formatively test learning goals or as preparation for the upcoming exam.

During this specific time, the Programme Manager has the authority to decide on alternative forms of testing within existing modules without the consent of the OC and the representative advisory board of the faculty (deviation from the OER). The two criteria for a good testing program continue to apply; • The intended learning outcomes guide the testing programme and determine the form and content of the examinations,

compensation rules within the module, the assessment criteria and norms.• The assessment programme is a coherent and balanced set of formative and summative assessment forms, with which all the

intended learning outcomes are ultimately assessed at the final level.

In accordance with the above criteria, the learning objectives that have been laid down in the study guide always form the starting point while making new decisions. These learning objectives cannot be altered. This can mean though that partial tests may be cancelled (when the learning objective are also covered in another assessment) and that it is possible to deviate from the types of testing as listed in the OER and the study guide for the (resit) exams of blocks 2, 3 and 4 of this academic year.

Only continue reading this instruction if the management of your programme (Bachelor) has decided that this functionality (Brightspace Quizzes) may be used for summative testing in the current academic year!

Brightspace offers the possibility to create a digital quiz

01 When to use?

Online proctoring makes it possible to remotely monitor a person’s behaviour during an assessment. With online proctoring, a written/digital test is taken on the student's laptop and his/her environment and behaviour is monitored with video, sound and algorithms. This offers the possibility to allow secured tests to continue. Unfortunately, the AUAS has no experience with such tools and there is no possibility to implement this on a large scale in the short term. This excludes online proctoring.

The only possibility where tests can be taken online with remote supervision is when a supervisor is simultaneously watching the candidate while he or she is taking the test. In this way, a supervisor can talk to the candidate and ask him/her to focus the camera on a specific point, for example, for control purposes. The ratio is four to eight candidates per supervisor.

For privacy reasons, it is important to only use software with which the AUAS has obtained licenses for. The options are:• Microsoft teams • Brightspace Virtual Classroom • Skype for business

If your bachelor programme chose to use online supervision, make sure it very clear to the students how this procedure will take place, what rules and guidelines apply and what conclusions may be drawn based on observed 'suspicious behaviour'.

Where no online surveillance is used, the tests are by definition open book exams. Nothing is preventing the student from using resources such as the internet, study books, and notes. Exams should be geared towards this. In an unsurveillanced setting, students can (screen-)share via WhatsApp and other social media or ask to Google.

Consideration: Remote supervision or open book exams?

What to consider? Section 2

02 What to consider?

• How can you set up the question library in such a way that you can create a test that does justice to the test matrix (and thus tests the learning objectives)?

• What type of questions do you have? If there are questions that have parameters, you can change some parameters so to create multiple versions of that question.

• Are the test questions all stand-alone questions and can the order be completely random? Or do certain questions belong together in a specific order?

• Within the sections of questions you create, can questions be selected randomly or is there a chance of being asked almost similar questions in one exam?

Creating a quiz as an exam requires you as a teacher to think carefully about a number of settings:Considerations

Question library

• What measures can you take to reduce fraud? How can you set up an exam that is as dissimilar as possible to that of other students?

• Randomising questions;• Not allowing students to go back

to previous questions;• Disallow right click (copy / past)• Time pressure • Special access (make different

versions without informing students)

• Block off access to course material on the device students are taking the quiz on. Surveillance in Virtual Classroom is not possible when activating this option;

• Give every student only one attempt to the quiz so they can’t leave questions open and start over again;

• Would you like to prevent that students can see their own answers and/or results after taking the quiz?

• Students won’t be familiar with the procedure of exams in Brightspace. How are you going to prepare them for this? (inform and practice).

• Think of a way to give students access to the documents or formula sheet they might need during the exam if you choose make use of the option to prevent students to access Brightspace during the quiz.

• Are you going to inform students of the ‘code of conduct’ in the introduction of the exam: that they acknowledge to have read, understood and will comply with this by starting the test? Or not?

• How are you going to inform students about who they can contact in case of (technical) problems before and during the test?

• Are you going to ask students to check in advance if they see the specific test name under assignment, quizzes and if this is not the case, report to someone? Or how are you going to make sure there is a quiz set up for everyone?

Preparing students to take the quizCreating a quiz

How to set it up?Section 3

03 How to set it up?

Setting up a question libraryWhen you go via 'Activities' to 'Quizzes', you will see a heading with 'Question Library'. It is best to create the questions in here, because you can then easily reuse these questions by importing them into another quiz and they will be stored here. It is therefore advisable to first create the questions in the Question Library and then import them into a 'New Quiz'.

Step by step guide:

Step 1 Go via 'Activities' to the ‘Quizzes' tool

Step 2 Then click on 'Question Library'

Step 3 Click on the 'New' button and choose the top option 'Section'. Using sections is a way to categorise questions and give structor to your question library. Give a section title, this is for example ‘Learning Objective 1’ or ‘Topic X’. As you see it can be structured in different ways. The structure of your question library should reflect the structure you normally have for your exam (and test matrix). The section title is only important to list the questions in, so select 'Hide Section Title from Students'. Then click on Save.

Step 4 Now click on the ‘Learning Objective 1’ folder, the intention is to create the questions in this folder. Click on 'New' to create a subfolder, called ‘section’ in Brightspace. In that way you can make sure to order the questions on bloom levels or on topic’s within this learning objective (folder).

Step 5 • Click on ‘New’ to create a question. • Select the type of question you want to create (for example Multiple Choice Question, Written

Response Question or Arithmetic Question). • Describe your question, tick the correct answer and possibly assign a number of points. Click on

'randomize answers for each student' at a multiple choice question • Click on 'Options' in the upper right corner for extra options with the question. The hint or feedback

options are very useful for formative testing, but will not be used in online exams (summative).• Click on 'Save' and repeat these steps for each question. Click on 'Save and Copy' to use the

previous question as a template for a new question.• Click 'Done Editing Questions' when all questions have been created.

Click here for a video guide for setting up a question library.Click here for a video on how to create a Multiple Choice Question.

03 How to set it up?

Creating a quizCreating a Quiz requires you, as a teacher, to think carefully about a number of settings. If you create a new quiz you will see the tabs below (figure 1) The tabs properties, restrictions, assignment and submission review are essential when creating a summative quiz. We will go through these step by step in the following pages.

Figure 1: The four main tabs you will be making settings in while creating a quiz as an online exam.

The last tab ‘reports setup’ is important if you wish to export the answers of the students to excel, to grade them per question or to compare the answers given by students.

Step 1 Go to Activities and select 'Quizzes'Step 2 click on 'New quiz’

03 How to set it up?

Properties 

Step by step guide:

Step 1 Give a title to ‘Name’.This will make the title of the exam visible to students. When working with different versions, make sure that all versions have the same name. This way a student will not know which version they are working on, and communication with other students will become more difficult.

Step 2 Create a 'Category' if you want to organize the Quizzes on folders or choose the 'Category' you already created.

Step 3 Next, you will import the previously created questions from the 'Question Library' into the Quiz under ‘Quiz Questions’ you’ll find the button: ‘add/edit questions’.

Step 4 If you want to create several quizzes for a single course exam, click on the ‘add’ button and then on ‘section’ to add the version of the exam. Name the section in ‘Section Title’ Version A, B, C, D , etc., and click on ‘Hide Section Title from students’. In this way lecturers now what version of the exam the student took, but the students don’t. You can also use this option to create a section within the quiz, for example to inform the students when the open questions will start or for what questions they can use the formula sheet. In that case don’t select: hide section title or section text from students.

Step 5 Click on the ‘add’ button and select ‘question pool’ if you would like to create a section within your exam with randomised questions from your question library:• Enter a title, the number of questions to select, and the points per question for your question pool.• Browse in the Question Library to select the desired questions. • Click on Save• It is recommended that you organise your Question Library to include a folder for each topic or

learning outcome in the quiz so the test will match the test matrix (see setting up a question library). Step 6 Click on the 'import' button and then on 'Browse Question Library’ if you want to insert whole sections

of questions (all of them) or individual questions from the question library to the quiz. This is often chosen when questions need to be asked in a specific order and/or when they are linked to a case.Then choose which ‘Section’ or individual questions you want to add and click on ‘add' when you are done.

Step 7 Are you done with the questions? Then click on the top right: ‘Done Editing Questions’

03 How to set it up?

Step 8 Now you are back on the 'Properties' tab. You now have more options under the heading 'Quiz Questions': • If it's an exam with unconnected multiple-choice questions make sure you set each question to be

shown on a different page by setting ‘questions per page’ on 1. Click on add.• If its an exam with questions related to a case or questions that have to be presented to students

together do not use the ‘questions per page’ button, but set a page border by clicking on the line between the questions, it will turn darker purple and on the left side a triangle will appear that shows 'remove page break' when you put your mouse on it (don't click: you'll remove the page break).

• Click on ‘paging’. When this option is enabled, students taking the quiz will only be able to move forward through pages. They will not be allowed to move to a previous page once they have left it.

• Do not use the ‘Shuffle questions at the quiz level’ button if you have an exam with MC-questions and open questions: this can also shuffle sections what means that students won’t all start with the MC-questions and end with the open questions. Only use this button if it’s not relevant for the exam in what order the questions (or questions arranged in sections) are presented to the students.

03 How to set it up?

Step 9 Write under the heading ‘Description’ a clear instruction for the students. The description is shown before quiz availability. This is also the text that will be shown as a ‘description’ to students when you place the exam in the content structure (via ‘existing activities’ and ‘quizzes’). As an example for a mock exam:

“Read this instruction carefully, well before you take the mock exam, so that you know what is expected of you.

This test is mainly meant to get used to the system and is identical to the practice test as previously placed in a Word document under content. 

You may use the book during the open questions of the exam.

For the regular exam there is a duration of 100 minutes, this time will be visible during the intake. As soon as this time has elapsed, it is not technically possible to make any changes to your answers. 

The regular exam will consist of 32 MC questions / 3 open questions, this mock exam consists of 8 MC questions and 1 open question. Each question will be shown on a separate page. Make sure you answer the questions on all pages. WARNING: After you have gone to the next page, you cannot go back to the previous page.”

Note the difference between "Next page" and "Submit Quiz". With "Next page" you go to the next page with questions, with "Submit quiz" you submit the quiz and you cannot make any more changes. Make sure there are "9 of the 9 saved questions" before you submit the quiz.”

03 How to set it up?

Step 10 The information under the heading ‘Introduction’ is shown at quiz commencement. And this makes it the perfect place to place a code of conduct if you want to leverage it. Example of a ‘code of conduct’:

Please read the following paragraph carefully.

By starting the quiz you acknowledge that you have read, understood and will comply with this statement.

By testing you remotely in this fashion, we express our trust that you will adhere to the ethical standard of behaviour expected of you.

This means that we trust you to answer the questions and perform the assignments in this test to the best of your own ability, without seeking or accepting the help of any source that is not explicitly allowed by the conditions of this test.

By starting this quiz you acknowledge that you have read and understood, and will comply with this ethical standard.

Also, by starting this quiz you consent to the possibility that your teacher can call you (up to 3 weeks) after the exam, asking you to explain your answers, in case of a suspicion of fraud. 

Step 11 If you want your exam to show a header or a footer on each page you can set that up under ‘Page Header / Footer’

Step 12 Finally, under 'Optional Advanced Properties', click the ‘Disable right click’ button and disable access to Brightspace content via clicking on ‘If you turn on this option, users cannot access the Email tool, Instant Messages tool, or their alerts if they have a quiz attempt in progress’.

03 How to set it up?

Restrictions  

Step by step guide:

Step 1 Go to the Restrictions tab

Step 2 By default, the status of the quiz is set to 'Hide from Users'. Uncheck this, after you set start and end date to the quiz, matching the exam duration as communicated to the students in advance, to make the exam visible to students.

Step 3 Under 'Timing' you can indicate how long the quiz will take. This is set to 120 minutes by default and has to be set to 50 or 100 manually. With the option 'Enforced Time Limit' you set the maximum time for the exam and click the button: ‘Prevent the student from making further changes’ for what happens after the quiz attempt as exceeded time limit.

Step 4 When you create different quizzes (version A, B, C, D, …) you can use ‘Allow only users with special access to see this quiz’ to allocate students from the classlist to a specific exam, without them knowing what version they have and without them seeing other options in their Brightspace course. Make sure to double check that all students are allocated to an exam.

Step 5 To ensure that students with a test time extension have the 25% (or 50% for visually impaired students) extra time to which they are entitled you have to use ‘Add special access’:• Set timing to ‘enforced time limit’ and set the time limit for these students;• Select the specific students under ‘users’• Click on ‘add special access’

03 How to set it up?

Assessment  

Step by step guide:

Step 1 Go to the Assessment tab

Step 2 The setting ‘allow attempt to be set as graded immediately upon completion’ must be turned on for grades to be automatically sent to the grade book’. The score displayed in the grade book is only what the system can auto-grade (all questions except written response questions).

Step 3 Click on ‘add grade item’ or select one if you created one in ‘administration’ and ‘grades’. Name the grade item after the exam (incl. version).

Step 4 • In case of an exam without written response questions: click on ‘Allow automatic export to grades’ so the system can send auto-graded attempt scores directly to the grade book.

• In the case of an exam with written response questions: do not click on ‘Allow automatic export to grades’. You must set attempts as marked in the Grade Quiz page, after correcting the anwers, to populate the grade book with attempt scores.

03 How to set it up?

Submission Views  

Step by step guide:

Step 1 Go to the 'Submission Views' tab and click on ‘default view’

Step 2 Write at 'message' a text that you want students to see at the end, when submitting the exam. Step 3 Click under ‘view details’ at ‘show questions’ no and un click all other options for scores and statistics.

Step 4 Click on ‘save’

03 How to set it up?

Report

Step by step guide:

Step 1 Go to the ‘Reports setup' tab and click on ‘add report’

Step 2 Name the report. Step 3 Click on ‘question details’ and select:

• Display difficulty for each question;• Include text responses;• Display private comments from sections (to see what version of the exam it is);

Step 4 Set release on ‘immediately’ and release the report to everyone with the role ‘designing lecturer’

Step 5 Click on ‘Save’ and on ‘Save and close’

03 How to set it up?

Hiding the grade item

Step by step guide:

Step 1 Go to ‘administration’ and ‘grades’

Step 2 Go to ‘manage grades’ and repeat these steps for multiple versions:• Select the grade item for the exam and click on the triangle behind it, and click on ‘edit’• Go to ‘restrictions’ and make sure ‘hide from users’ is on.

03 How to set it up?

Embedding a quiz in the content

Step by step guide:

Step 1 Go to ‘content’ and create a module you want to put all exam information in.

Step 2 Click on 'Existing Activities'Step 3 Next you select in the drop down menu which created 'Activity' or 'External Learning Tools' you want

to add, in this case 'Quizzes'Step 4 Now select the Quiz you made for this exam.

How to check if the set up is correct?

Section 4

04 How to check if the set up is correct?

How to impersonate a dummy student?

Step by step guide:

Step 1 Go to ‘administration’ and ‘classlist’

Step 2 Search for ‘dummy’ Step 3 Click on the triangle behind the dummy’s name and click on ‘impersonate’.

Step 4 Now you can access the course from a student’s perspective. • Take the exam, look for flaws or errors, write them down so you can correct them later. • Check after completing the quiz if any results are viewable for the dummy student. It shouldn’t be. • Check also grades, if there a grade item connected to this exam viewable for the dummy student. It

shouldn’t be. Step 5 Go to the top right corner, click on the dummy’s name and click on the cross to end the impersonation.

Step 6 Correct the quiz or settings in grades if needed following your notes.

Ask the key user of your bachelor programme to add a ‘dummy student’ to your specific course, or send an e-mail to onderwijslab-ft@hva.nl with the request to add a dummy student in your specific course. The following steps only work when a dummy student is added to the classlist:

How to prepare students?

Section 5

05 How to prepare students?

• Announce the examination in time, preferably two weeks before the date of the examination. • Ask students who intend to participate to make themselves known (e.g., by enrolling in Academy Attendance), so that the

coordinator of the course can ensure that each student is enrolled in the classlist (Brightspace). This could be a taks of the educational coordinator of your bachelor programme.

• As a lecturer you must ensure that all students on above list are assigned to a version of the exam (see how to set up Restrictions, step 4).

• Make sure that all procedures and other relevant information related to the exam are available online at least one week in advance, including a mock exam to get familiar with the interface.

• Make sure your students are aware of the need to download the formula sheet or other supplies prior to the exam. Do this in the identical manner for the mock exam to prepare them.

• Provide a name and contact details so students know who to contact in case of (technical) administration problems or ambiguities.

• Please point out to students that if for any reason a student is unable to take this exam, he/she can report to the course coordinator. Per student, per situation it will be examined whether another form of examination can be offered (in consultation with the examination board) or whether another opportunity will be offered at a time to be determined.

Preparation

How to give students with technical issues a second attempt?

Section 6

06 How to give students a second attempt?

Step by step guide:

Step 1 Go to ‘activities’ and click on ‘quizzes’

Step 2 Click on the drop down arrow behind the specific exam and then click ‘edit’Step 3 In the Restrictions tab, you select 'Allow selected users special access to a quiz' and choose 'Add

Users to Special Access'.

Step 4 In the 'Users' area, select the learner that you want to grant a different number of quiz attempts. Click Add Special Access’.

Step 5 In the Special Access area, alongside the name of the learner, click on ‘edit’.

Step 6 Scrol downwards to Attempts en click on 'Override attempts allowed'.

Step 7 Change 'Attemps allowed' from 1 to 2, and click on ‘apply’

Step 8 Select the specific student.

Step 9 Click on 'add special access’

Step 10 Click on 'save and close'

This option can be used when a student presses the wrong button after one of the first questions, or within the set time and leaves the quiz environment in Brightspace. IMPORTANT: Make sure to give the student a clear instruction that it does not make sense to reanswer the questions that the student had made in the first attempt, these have been logged. Through this second attempt the student only gets the opportunity to finish his exam and can skip the questions he had already made in the first attempt.

How to grade it?Section 7

07 How to grade a quiz with written response questions?

Step by step guide:

Step 1 Go to ‘activities’, click on ‘quizzes’

Step 2 Click on the drop down arrow behind the specific exam and then click ‘grade’Step 3 Here you have two options:

1. Grade every submission one by one 2. Export all given answers of excel and use a filter to select the question you would like to grade first,

etc. Step 4 The two options below depend on the choice you made in step 3:

• If you have chosen for option 1 in step 3: you can enter the points the student got for each question manually in brightspace and it will calculate a grade automatically.

• If you have chosen for option 2 in step 3: you can enter the grades manually in grades via ‘administration’, ‘grades’. Make sure to write them down for the correct exam (version).

How to export the results to SIS?

Section 8

08 How to export grades to SIS?

Step by step guide:

Step 1 Go to ‘administration’ and click on ‘grades’

Step 2 Click on ‘export’Step 3 Use the following settings:

• All users• ‘Org Defined ID’• Select the second option: ‘Org Defined ID, username, firstname, lastname’ • Select ‘points grade’ • Select by user details only ‘firstname’ and ‘lastname’

Step 4 Select all the versions of the exam you want to export

Step 5 Click on 'export to Excel’

Step 6 Hide in excel the rows you don’t need. Make sure you have the student ID and the grade displayed, you can copy that information into SIS, like you would normally enter the grades.