Embed Size (px)
DESCRIPTION
Assessment of large student cohorts in an e-learning environment. Naveed A. Malik Virtual University of Pakistan. Overview. Background The problem Dimensions of the problem Major issues/conflicting requirements IT to the rescue. Background. - PowerPoint PPT Presentation
Citation preview
Assessment of large student cohorts in an e-
learning environment
Naveed A. MalikVirtual University of Pakistan
Background The problem Dimensions of the problem Major issues/conflicting requirements
IT to the rescue
May 24, 2010
Overview
LINC 2010
• Virtual University of Pakistan (VU) delivers education through innovative uses of IT
• VU conducts its courses under a formal semester system with a defined academic calendar
• Video lectures are delivered through:– free-to-air television on VU’s TV channels– Data DVDs– Youtube: http://www.youtube.com/vu
May 24, 2010
Background
LINC 2010
Student-teacher interaction is through a comprehensive Learning Management System ◦ http://vulms.vu.edu.pk
Examinations:◦ Formal proctored examinations◦ Twice a semester (midterm and finals)◦ Conducted at dedicated exam centers (f2f)
Background
May 24, 2010LINC 2010
Large student body◦ Nearly 60,000 students and growing!
Scattered all over the country◦ 95 cities and more…
Admissions twice a year◦ Average annual intake is ~25,000 students
160+ courses offered every semester
Problem Dimensions
May 24, 2010LINC 2010
• Full time students– Normal course load– Mostly as per scheme of studies
• Working students– Partial course load– Variety of course distributions– Limited study times/exam days/hours
• Stragglers and certificate students– Can take any course in any semester
Problem Dimensions
May 24, 2010LINC 2010
How can the University conduct mid and final term examinations in a
secure, convenient, time-limited and efficient manner?
The Problem
May 24, 2010LINC 2010
How to create a datesheet (exam schedule) that satisfies all students
How to ensure:◦ Secure distribution of question papers◦ Sanctity of examinations across the country◦ Efficient and secure collection of answer sheets◦ Grading within an acceptable time frame
Major Issues
May 24, 2010LINC 2010
Exam centers must run at capacity if exams are to be conducted in a reasonable amount of time (number of days)
but Working students prefer evening hours or
weekends for exams Full time students prefer morning times,
weekdays for their papers
Conflicting Demands
May 24, 2010LINC 2010
Commuting students prefer mid-day times! Students prefer to have gaps between
papers
Management Issues: Mismatch in exam start timings can lead to
examination leaks across the country Ensuring proper invigilation across all
centers is a nightmare
Conflicting Demands
May 24, 2010LINC 2010
Allow students flexibility in choosing their examination dates/times on a first-come first-served basis and run centers at full
capacity to minimize the total duration of the exams
The Idea
May 24, 2010LINC 2010
Allowing students to create their own individual datesheets on a first come first served basis:
◦ Can only work if students receive distinct question papers
◦ Can be done by creating a question bank and automating the process!
IT to the rescue
May 24, 2010LINC 2010
Question Bank Student created individual datesheets Distinct paper generation Encryption and security Conduct Grading Result
Key Concepts
May 24, 2010LINC 2010
Includes MCQs and Essay type questions Questions identified by
◦ Syllabus topic / Lecture number◦ Difficulty level◦ Type of skill being assessed◦ Etc.
New questions added on daily basis Old/overused questions retired after each
examination
Question Bank
May 24, 2010LINC 2010
Determine:◦ the total number of (Students*Subjects)◦ the capacity of exam centers (computer seats)
And hence
Determine and define grid of: ◦ Exam Centers◦ Exam days (including weekends)◦ Exam times (multiple sessions in a day)
Planning an Examination
May 24, 2010LINC 2010
Students log-in to http://datesheet.vu.edu.pk
Select the City where they wish to appear Choose Exam Center within the selected
city Their current semester subjects are
displayed For each subject, they:
◦ Select day/date◦ Select session time◦ Confirm
Making a Datesheet
May 24, 2010LINC 2010
Making a Datesheet
May 24, 2010LINC 2010
Making a Datesheet
May 24, 2010LINC 2010
Based on datesheet data, exam dept knows:◦ Center
Date Time
Subject, No. of papers required Subject, No. of papers required .....
..... .....
◦ .....
Paper Generation
May 24, 2010LINC 2010
Academic staffs provides paper parameters
System generates required number of distinct question papers
Paper Generation
May 24, 2010LINC 2010
All e-papers for a center are collected Dispatched by email / transferred to a CD
and sent via courier Cannot be opened before date/time and
then only with passwords which are NOT provided beforehand
Questions are doubly encrypted anyway and cannot be read except through software
Paper Generation
May 24, 2010LINC 2010
Exam software and encrypted data are loaded onto exam center servers using file password
Session password is sent to the invigilator by email/SMS on exam day 5-10 minutes before exam
Data for only a single session is unlocked Students log in and are given their question
paper according to their datesheet
Conduct
May 24, 2010LINC 2010
At end of each day, an exam file containing student answers is sent back to VU by email from each center
Contains data for all sessions for the day
Files are uploaded into the grading system and MCQs marked automatically
Conduct
May 24, 2010LINC 2010
Student attempts are collected and sorted by question
All attempts for one question are given to a single tutor for grading
Grading rubric/reference is provided by QB
Ensures anonymity and uniformity
Grading
May 24, 2010LINC 2010
Result is collated by the system Declared at one go Students receive their results in their
Learning Management System (LMS) gradebooks
Semester and subject positions and campus-wise results also declared
Result Publication
May 24, 2010LINC 2010
IT Works!
May 24, 2010LINC 2010
Question Bank
May 24, 2010LINC 2010
Question Bank
May 24, 2010LINC 2010
Planning an Examination
May 24, 2010LINC 2010
Planning an Examination
May 24, 2010LINC 2010
Paper Generation
May 24, 2010LINC 2010
Question Paper Criteria
May 24, 2010LINC 2010
Paper Generation
May 24, 2010LINC 2010
Conduct
May 24, 2010LINC 2010
Grading
May 24, 2010LINC 2010
