1

DESIGNING EFFECTIVE DQS/TESTS

Dr. Jey Veerasamy

2

Bad DQs

Direct answers are in the book Direct answers can be found in the web

with minimal effort Totally open-ended with no right or

wrong answers Not relevant to the current week’s

material Questions like “Did you encounter any

issues when doing the assignment?”

3

Ideas for better DQs

Requires the students to do critical thinking Requires them to put 2 or more concepts

together to arrive at the responses Highlight the words “summarize in your

own words” if DQ related material can be found easily in the web.

Reasonably more difficult for an average student, compared to other individual assignments Wrong initial answers means more discussion!

4

DQ assignments

I want to focus more on other aspects of DQ assignments. How many DQs? What do we expect the students to do? What should the instructor do? How the discussions are managed?

5

DQ options

Basic: 2 to 3 questions Expanded: 5 to 10 questions Personalized: assign specific question

for each student for posting initial response

Empowered: designate each student as “DQ lead” for one question

6

DQ option: Basic

2 to 3 questions every week After first few posts, most students tend to

“beat the dead horse” for participation. Students in a large class may create a

thread for each DQ resulting in tons of threads in the forum.

Suggestion: Choose questions wisely. Create individual thread for each question in the weekly forum. Disallow creation of new threads.

7

DQ option: Expanded

5 to 10 questions every week This option provides most freedom to

students! Most students tend to clamor on “easy”

DQs and manage to “beat the dead horse” for participation.

Suggestion: Create individual thread for each question to maintain sanity in weekly forum.

8

DQ option: Personalized

Assign specific question for each student for posting initial response by Saturday.

While “beating the dead horse” is still possible, there will be at least one response for each question which encourages more participation.

Suggestion: Ask each student to create one new thread for the assigned question. Encourage students to work on “unassigned” questions for additional learning.

9

Personalized DQ assignment options

You can assign a question manually to each student every week.

More efficient option is to assign a number to each student for DQ selection for the whole course:

Student name DQ #

Jey Veerasamy 1

Mike Picerno 2

Glenn Cummings 3

… …

10

DQ option: Personalized …

We can come up with few variations. We can be more forceful/intrusive regarding comments. Assign 1 DQ for every 2 students and mandate

that each student should comment on other student’s initial response.

Assign one question for every student for DQ response and another question for mandatory comment.

I worry about the value of such mandatory comments.

11

DQ option: Empowered

Designate each student as “DQ lead” for one specific question. Ok, what does it mean?

Each student is required to Post initial response Follow-up on each comment Summarize and post final response based

on the discussions DQ grade will be based on all these

items.

12

DQ option: Empowered+

Instructor posts own version of DQ responses by Tuesday morning.

In addition to posting the final response based on the discussions, student is also required to compare it with instructor’s answer and comment on it in that final post.

Suggestion: Post instructor’s version of DQ responses in a new thread titled “DQ final response” and ask the students to submit their final posts there.

13

DQ option: Empowered+ …

I use this Empowered+ option and it has worked well.

A few students are intimidated by the process in Week 1, then they warm up and embrace it.

Less pressure on me – I can be more selective in responding to DQ related posts

Students feel a sense of “closure” when they see instructor’s version of DQ responses.

14

Explanation to students

There is a separate question for each student to maximize learning & participation (see your assigned question # under Assignments tab). You are in-charge for managing the discussions for your question, i.e. you should post your initial response in DQ thread of weekly forum by Saturday, then continue to refine, monitor & respond to others' comments. I will create the final DQ response thread on Tuesday and provide my DQ responses. Then you should write your final response based on the discussions and compare it with my response and submit it to the weekly forum by Sunday.

Complexity of these questions may vary. But, I am not keen on the perfect answers - in fact, there may not be any perfect answers for a few DQs! As long as you write a meaningful answer by Thursday, continue to follow up on others' comments & post a final response by Sunday as stated above, you will get full points for your DQ. Include the question # in the subject line of your initial post and include the full text of your DQ within your post as well. If you do not understand the question, ask for clarification from the Instructor.

I also encourage you to participate by commenting on others' DQ responses. All meaningful postings will count towards participation. Also, feel free to attempt any "unassigned" DQs for additional participation.

15

Grades in online courses

Concerns about grade inflation: every student scores > 90% consistently

Almost every one scores < 50%: this scenario does not happen often. While we can apply curve and adjust the final grades, it is not a good feeling.

What is the middle-ground? How to achieve it in our courses? If you are like me, we continue to struggle to find it.

16

Grades in online courses …

I feel uncomfortable to decide the final grade purely based on participation and weekly hands-on assignment. Why?

Participation does not automatically confirm critical thinking, it simply says that student is meaningfully active in the class.

Irrespective of whether weekly hands-on assignment is easy or hard, we encourage the students to approach us when they get stuck and we provide varying degree of help.

Some students refuse to ask for help, a few others bug us for everything. Should that really determine the final grade?

17

Why to use tests?

Well designed test can test the student’s critical thinking skills and help us achieve a meaningful grade distribution.

A few courses use weekly quizzes, which typically use multiple choice questions to test whether the student is familiar with various terms introduced in the week. They may even allow the students to take quiz again & again to improve the scores. Since its goal is narrow, such quizzes should not carry too many points.

18

Weekly tests

I introduced weekly tests to probe students' understanding of concepts and related details. I found my tests probed them bit too much

I felt bad for adding difficult questions to test & making students suffer. I asked myself how to test "students understand the concepts reasonably well"?

I came up with the following trade-off: Continue to use those difficult questions, but I gave additional questions for extra credit. For example, if the test is worth 40 points, I designed it for 50 points, designated 10 points as extra credit within the test. If some one did score more than 40, manually I clipped it to 40.

This has worked reasonably well - students seem to fear less about the tests and try to focus more on its contents. New method tests their understanding, but it does not punish for not knowing ALL the details.

19

Design good test questions

Multiple choice is generally not a good choice for tests. Instead of logical thinking, students may get into the game of elimination. If absolutely needed, have as many as 6 or 7 choices if possible.

Be as clear as possible so that the students won’t need further clarifications. After fine-tuning in a few sessions, it can become fully auto-graded item.

Avoid manul partial credit as much as possible, otherwise it can degenerate to weekly hands-on assignment.

Avoid repeated testing of same concept in several questions.

Ensure that all the “tested” concepts have been covered in the course or in pre-requisites.

20

Questions?