Upload
bono
View
19
Download
0
Embed Size (px)
DESCRIPTION
a. A University Strategy for Delivering Online Assessment. Dr Richard Parsons Director of the Learning Centre University of Dundee [email protected]. University of Dundee. More than 10 years substantial experience of online assessments - PowerPoint PPT Presentation
Citation preview
Learning CentreUniversity of Dundee Richard Parsons 20 April 2023
A University Strategy for
Delivering Online Assessment
Dr Richard ParsonsDirector of the Learning CentreUniversity of Dundee
Learning CentreUniversity of Dundee Richard Parsons 20 April 2023
University of Dundee More than 10 years substantial
experience of online assessments Blackboard clients – Academic
Suite (virtually 100% modules) Delivering up to 3500 formative
assessment sessions a day with QM Perception
Delivering 50+ summative exams a year with QM Perception
Range of online assessment activities recognised
Blackboard Server Configuration
Summer 2006 –
Bb upgrade 6.3 to 7.1 on Unix,
QM application from 3.4 to 4.1,
Bb - QM bridge software (twice),
QM repositories for assessments,
Blackboard building block deployments and upgrades,
Solaris and Oracle database upgrades,
Annual module rollovers,
Enhanced PDP programmes,
Medical School curriculum map integration,
University restructuring configuration changes
Learning CentreUniversity of Dundee Richard Parsons 20 April 2023
Online Assessment SystemAssessment
hardware design & deployme
nt
Staff and
student educatio
n
Central assessment support
Assessment software design &
deployment
Learning systems
integration
Clear policy and procedure
s
Learning CentreUniversity of Dundee Richard Parsons 20 April 2023
The Hardest Task
Remains writing quality questions Staff time is precious Students are more critical than ever before Assessments to be aligned Questions need to be prepared ahead of
time Quality runs to feedback, distracters,
variety, depth, currency Authoring Q within the software is
secondary
Learning CentreUniversity of Dundee Richard Parsons 20 April 2023
Staff and Student Education For 4 years at Dundee we
have run an Online Assessment course, online
Covers technical and pedagogical aspects
5 weeks long, 3 to 6 hours a week.
Formally accredited by our Education Faculty (10 SC)
Building a bank of staff skills
Most students need little guidance
Learning CentreUniversity of Dundee Richard Parsons 20 April 2023
Examples of Assessment Strategies
15 Q from pool of 100, max mark taken (Genetics) Take assessment first, then repeat in an open
fashion. Mark taken from first run. (Medicine) 10 numerical questions, each from a pool of 50
randomly generated Q (Life Sciences) Student revision – students invited to compose 2 Q
each with FB, all compiled automatically, checked & made available for open assessment (Env Sci)
Open assessment, one submission only (Law) Summative (range of departments)
Learning CentreUniversity of Dundee Richard Parsons 20 April 2023
Quality Hardware and Systems Central servers built & maintained by ITS LC involved in system design Three servers, one formative, one summative,
one shared repository HP BL20p Blades Dual Pentium 3400 MHz 6GB
RAM, dual NIC backplane. Databases
Formative on Oracle Unix serverSummative on Oracle Unix serverShared on Oracle Unix server
LC Administrator
Local Author (Staff)
Local Author (Staff)
Local Author (Staff)
QMP V4.2 CAA-02
Production Formative Repository
Blackboard 7.1
My Dundee
Participant
Participant
Participant
Participant
Participant
University of Dundee
Questionmark Perception V4 Systems Model
Oracle 10GQMP V4
Authoring Manager
QMP V4 Authoring Manager
QMP V4 Authoring Manager
QMP V4.2 CAA-01
Production Exam Repository
Oracle 10G Oracle 10G
QMP V4.2 CAA-03
DevelopmentRepository
Oracle 10G
Participant
Participant
David Walker, University of Dundee
QM / Bb integration
Learning CentreUniversity of Dundee Richard Parsons 20 April 2023
Clear Policy and Procedures Central online assessment system required
clear guidance Prepared draft Policy and Procedures Generally follows BS7988:2002 Developed for about a year, inc. consultation Passed at University Senate Excellent for ensuring quality Runs to 17 pages Additional documentation for Online
Assessment system administrators
Learning CentreUniversity of Dundee Richard Parsons 20 April 2023
Summative Procedures
Assessment must be tested Assessment loaded to both servers,
scheduled on one Central staff attend examinations Use special ”exam” accounts Use Perception Secure Browser Candidates complete Examination
Attendance form Immediately after the exam a
backup of results is taken Use two way radios or mobiles No system is fullproof, 99% success
target ?
Learning CentreUniversity of Dundee Richard Parsons 20 April 2023
Self & Peer Assessment Bb Building Block
Assessments are added and edited by the instructors in any course
One or more questions, each with one or more marking criteria Marking criteria as discrete values or range Feedback provided as score and text if required Students automatically allocated to mark peers (1 to many) Peer assessments are assigned automatically Self assessment is an option Marks and reports are visible to the Instructor All marks can be downloaded for analysis in excel Moderated results automatically passed to Bb Gradebook Exercise text is available for download and online plagiarism
checks Questions and criteria are stored for reuse and exchange
Learning CentreUniversity of Dundee Richard Parsons 20 April 2023
The Dundee My PDP resources
Each student can prepare an online portfolio based on templates, culminating in prioritised action points and an enhanced CV. Portfolios can be shared online.
Online resource to support PDP; timetabled induction sessions; scope
for faculty tailoring. Available for free under license.
Learning CentreUniversity of Dundee Richard Parsons 20 April 2023
Mobile Campus Map
Java game-based map Rich links with key campus
information Single download Popular with students
www.dundee.ac.uk/mobilemap
Learning CentreUniversity of Dundee Richard Parsons 20 April 2023
Additional Online Assessment Self & Peer Framework for plagiarism advice and
management Procedures for online submission Discussion board assessments ePortfolio assessment rubrics and tools Classroom/personal response systems Business intelligence activities Overall framework of OA support at Dundee
Student
This is nice and privateI am getting real feedback hereDo I know the answers?Can I learn from this?I am at home in this online environment
Distance Student
Is my technology compatible?Is my connection reliable?Is this a replacement for F2F?
Lecturer
Are my questions too simple?Is this technology reliable?Do the students find it tedious?Where can we obtain challenging questions?Are these assessments aligned with our learning outcomes?
Online Assessment Officer
Are all assessments up-to-date and correctly available?Is the server-client functionality all compatible?Will the next examination run smoothly?Are the staff education programmes ready?
IT Server Administrator
Is the server fully patched, reliable and secure?Is the OA software functioning correctly and reliablyDo we have monitoring and is it working?Are the logging and backup procedures working correctly?
eLearning Manager
Are the staffing levels correct?Is our software choice appropriate?Are the server specifications correct to cope with the expected load?Are staff education procedures in place?
IT Desktop Administrator
Does the desktop software operate correctly?Are the “exam accounts” secure and available?Are the PCs well maintained and reliable?
Examinations Officer
Are these results 100% reliable?secure?Do our procedures represent the best possible practice?
College Officer
Can we book an IT suite for 240 students on Wednesday the 25th?Two students have additional requirements.Will the results transfer directly to the SRS?
Individual concerns within an online assessment world
Traditional Lecturer
Who can electronically author my questions?Can we use textbook questions?Is this assessment driving learning?
Timetabling Officer
Can the proposed exams be fitted into the best rooms?Are assessments receiving appropriate prioritisation?
Richard Parsons, University of Dundee
Procedural
Technical
OA policy and procedures
Student and staff training
Examination practices
Room booking procedures
Student personal PCs
University student desktop
Question creation
OA application software & support
Technical database considerations
Operating system & web server
Hardware & infrastructure
Network
Question/assessment repositories
Elements Each addressed by a tasked individual(s) within a supported team, and each element able to cope with growth and changes
Revisit annually?
The entire operation has limited inertia, limited history, sustainable staffing, space, budget.
Represents a framework
ZZ + AA
XX + WW
Academic staff + XX + WW
College staff
Web guidance + VV
BB + CC
Academic staff + XX + WW
XX + WW
DD + EE
XX + WW
FF + GG
XX + HH
HH + II
Actors
13 specialist staff + academics & college staff
Richard Parsons, University of Dundee
Sept.
Dec.
March
June
Summer upgrade window
Testing & tuning
Modules begin
Annual framework
meeting
Mid semester window
Online exams
Online resit exams
Online exams
OA staff education module
The Annual Cycle of Online Assessment
•Growth may occur each year
•Changes are made to the systems
•A framework allows the process to be sustained
Richard Parsons, University of Dundee
Learning CentreUniversity of Dundee Richard Parsons 20 April 2023
Summary
Dundee has an institutional online assessment & some elements of a strategic framework
Software, hardware, P&P, staffing, integration and education. 13 specialist staff.
Secure and reliable, but straightforward to use for students and staff
Let the academics create the Qs to stimulate learning
Costs are neither insignificant or astronomical
Learning CentreUniversity of Dundee Richard Parsons 20 April 2023