Assignment Marking via Online Self Assess

Margot Schuhmacher, Lecturer Higher Education Development Unit, Centre for Learning

and Teaching Support, Monash University

Robert Redpath,Lecturer School of Computer Science and Software Engineering,

Faculty of Information Technology, Monash University

Overview

• The Role of Assessment

• The Assessment Process

• Assessment Criteria as a Quiz

• Evaluation of the New Process by Students and Tutors

• Conclusions

Introduction

• Assessment for formal accreditation and student learning

• Student as primary stakeholder in the process

• Assignment submission via a learning management system

The Assessment Process

• The LMS replaces a traditional paper based process

• This provides benefits:– Flexible student submission– Persistent record

• BUT the student view of the marking process is still as a black box

Black Box View

The Tutor and an LMS

• Assignment is online

• Marking guide is separate

• The process is tedious

• Mark entry is separate step

Marking Guides

• Represented in many forms

• What if it is presented as a simple rubric?

Rubric Score for Essay Structure

Amateur Critical &

Essential

Desirable Extension

Descriptor Essay seems to have coherent structure

Essay structure has an introduction body and conclusion

All critical elements and a discussion

All desirable elements and extended with analysis

Points 1 2 3 4

Select X

Desired Improvements

• Can the technology incorporate marking criteria into the assignment?

Assignment tool in Vista

• Undergraduate database unit

• Students needed to implement a database

• Assessed on logical and physical design– Use of constraints– Valid syntax– Results of queries

Assessments Tool in Vista

• Quizzes question types include– Multiple choice (single and multiple selection)

– True False

– Short Answer

– Open-ended (paragraph)

• Feedback and results can be returned to students

Combine two tools?

Marking Criteria ExamplesThe constraints were:

1. Not explained

2. Had a poor explanation

3. Had a good explanation

4. Had a very good explanation

Did you use sensible sample data that allowed the queries to produce output?

1. Yes

2. No

Further Examples

Which of the following SQL Features did you use correctly and successfully in one or more of your queries? Please select all features used.

Compound/complex search conditions LIKE search criteria Date function (such as TO_CHAR, TO_DATE) Join 2 tables Join 3 or more tables ORDER BY clause GROUP BY with aggregate function Sub-queries

A Demonstration

• Using WebCT – Vista (MUSO)

A White Box View • The student sees the evaluation of their

assignment specifically related to the marking criteria

The Evaluation Process

• Trialled in undergraduate database unit

• Anonymous surveys of students and tutors

• 76 assignments

• 46 self assessments

• 12 student evaluations

• 3 (out of 4) tutor evaluations

Student Issues

• Preference for the new system?

• Ease of the process?

• Value of the process?

• Indication of performance from the quiz?

• Variation in grade expectation when result known?

Greater Student Satisfaction

• Early and strong indications of assessment

• Improved student feedback at both times of self assessment and assessment confirmation

• A greater understanding of the criteria applied

• Improved learning

Tutor Issues

• Ease of marking in comparison to previous approaches?

• Ease of management?• Ease of providing feedback?• Clarity of marking guide in comparison to other

units?• Preference for the process?• Need to override the student score?• Direction of adjustments

Tutors Prefer the Approach

• Ensures a clear assessment criteria

• Moderation carried out by confirming or overriding student self assessment

• An imposed structure exists for persistent records easing further moderation

Findings

• The criteria must be described clearly– Improves consistency

• The student receives immediate feedback– Self assessment is moderated by the tutor

• Persistent and central record – Ease of moderation

Conclusions

• An improved assessment process

• Clearer criteria

• More immediate feedback

• Improved learning experience

• A persistent record of the entire process

• Greater efficiencies in the process

Future Work

• The process trialled again in 2005

• Process improvements

• Process training/education required

• Investigate other forms of assessment criteria