Proctoring Service API - IMS Global Learning Consortium · } Increasing interest in proctoring of exams delivered electronically } A number of proctoring services to choose between

Copyright©1995-2017Ques7onmarkCorpora7onand/orQues7onmarkCompu7ngLimited,knowncollec7velyasQues7onmark.Allrightsreserved.Ques7onmarkisaregisteredtrademarkofQues7onmarkCompu7ngLimited.Allothertrademarksareacknowledged.

ProctoringServiceAPISteveLay,Ques7onmark


}  Increasinginterestinproctoringofexamsdeliveredelectronically

}  Anumberofproctoringservicestochoosebetween}  ProctoringservicetypicallyseparatefromtheAssessmentManagementSystem§  Some7mesreferredtoasthe‘TestDriver’

}  TheProctoringServiceAPIaimstostandardizethetouch-points

Introduc7on

Slide2


}  PhotoCaptureandCompare§  Theproctoringservicecapturesoneormorephotosofthecandidateduringtheexam

§  Photosarecomparedwithregistra7onphotosonfile§  Photosareavailableforlaterreview

}  RecordandReview§  Theproctoringservicecapturesacon7nuousaudioandvideostreamofthecandidateduringtheexam

§  Dataisprocessedandflaggedautoma7callyforanomalies§  Summarydataandfullstreamsareavailableforlaterreview

}  OnlineProctoring§  Thecandidateisconnectedwitharemotehumanproctoratthestartoftheexam

§  Theproctormonitorsthecandidatethroughouttheexamandrecordsanyanomalies

§  Summarydatafromproctorandsampleframesareavailableforlaterreview

ProctoringServiceExampleServiceTypes

Slide3

Teddy Funger

[email protected]

Slide5

Launching into an Assessment from the Proctoring PlatformWhat we’ve learned at ProctorExam

What are we trying to achieve?1. Security

a. The exam should only be accessible through the exam platformb. The exam attempt should always be correlated with the proctoring sessionc. The exam should not be accessible after the attempt

2. Ease of Usea. The student should take the least amount of steps to access the examb. The student should have explicit instructions to follow

3. Universalitya. The exam access procedure should be the same across exam platforms

The “Download Your Exam as a Document” method● Student can keep a copy of

the exam in any number of ways

● Teacher has to download exam document

● This isn’t our core competency!

The “Sign in to Blackboard” method● The student can access

the exam if the poke around blackboard enough

● The student can share their credentials with anyone

● Way too many steps!

The “Global Exam Password” method● Once the password leaks,

anyone can access the exam

● Exam session isn’t automatically linked to the proctoring session

The “Individual Password” method● Student can share their

individual credentials out of band if they are sneaky enough

The “Signed Exam Link” method● More difficult to

communicate through a side channel, as the link is fairly long

● Technically minded students can access credentials to use if there is also a login page

● Requires exam platform to provide mechanism to sign links

The famous “LTI Launch” method● Link points to url on

proctoring platform, which then redirects to a time limited(single use) signed LTI launch

● We can ensure that the student is sharing all streams before launching exam platform


}  ExamandQues7oncontent

}  Candidateresponses}  Detailednaviga7onpaths

}  Timinginforma7on

}  Photostakenduringtheexam}  Audio/Videocapturedduringtheexam

}  Screencaptures}  Applica7onusageinforma7oncapturedfromopera7ngsystem

}  Proctorincidentreports}  Flagsfromautomateddetec7onofanomalies

PerformanceDatavsExamSessionEvidence

Slide13

PerformanceData ExamSessionEvidence


ProgramAdministratorRegistersCandidatesfor

theExamSetsProctoringRequirements

CandidateTakestheexam

ProctorOverseesExamSessionHelpscollectExamSessionEvidence

ReviewerReviewsExamSession

EvidenceMakesdecisionsabout

outcomes

Roles

Slide14


}  Site-wideProctoringOp7ons§  Configuringproctoringop7onsthatwillapplytoallassessmentsdeliveredwithintheins7tu7on

}  AddProctoringRequirement§  Addproctoringrequirementtoanassessment

}  AssessmentSpecificProctoringOp7ons§  Modifydefaultproctoringop7onsforanindividualassessment

}  ScheduleanAppointment§  Pre-registerand/orscheduleanappointmentwithaliveproctororothertypeofresource-limitedproctoringservice

}  ScheduleBlocking§  Requestabookingfromproctoringsystemforagivennumberofcandidatesand7meslot

UseCases:Pre-exam

Slide15


}  ValidateUserSystem§  ChecktechnologyrequirementforaspecificPC,laptoporotherdevice

}  LaunchtheExam§  Launchtheexamthroughtheproctoringservice

}  ProctorIntervenes§  Controlarunningexamtoprotectintegrityoftheprocess

}  ExternalProctorLaunchofProctoringSystem§  Usersintheconsumersystemactaslive-onlineproctorsintheproctoringtool(agentprovidedownproctors)

UseCases:In-Session

Slide16


}  ExternalReviewerLaunchofProctoringSystem§  Reviewexcep7onsforanindividualassessment§  Reviewauditinforma7onforen7reassessmentprogram

UseCases:Post-Exam

Slide17


AnatomyofaProctoringSolu7on

Slide18

ProctoringAPI

CandidateCheck-in

Candidate’sBrowser

AMSCandidateExamDelivery

Proctor/ReviewerApplica7on

Proctor’swebbrowser

AMSProctorControls


APIDetails


IMSLearningToolsInteroperability(LTI)h_ps://www.imsglobal.org/ac7vity/learning-tools-interoperability

Slide20


}  AlreadyusedbyLMScommunity}  Designedtosolvetheproblemof‘launching’onewebappfromanother§  ToolConsumerlaunchesToolProvider

}  Onestop-shop:§  User,contextandroleinforma7on§  Resourceplacement(e.g.,mul7pleexamstosamegroup)§  Authen7ca7onviaOAuthsigning§  Returnlinksforsmoothhand-back

}  What’smissing?§  Communica7onofworkflowinforma7on:

�  Timeconstraintsonresource(exam)availabilitysetbyAMS�  Stateofproctoringservice,e.g.,appointmentbookingphase,appointmentreadiness

§  SafewaytocommunicatetheExamLaunchinforma7on

WhyLTI?

Slide21


ProctoringSolu7onCandidateflowinWebBrowser

Slide22

CandidateCheck-intoProctoringService

LTILaunch Redirect


AMSCandidateExam

Delivery

AMSCandidatePortal

ProctoringServiceWS

GET


ProctoringSolu7onCandidateflowwithSpecialProctoredApp

Slide23


LTILaunch Redirect


AMSCandidateExam

Delivery

AMSCandidatePortal

ProctoringServiceWS

GET

AppLaunch

ProctoringApp

AppLandingPage


ProctoringSolu7onProctorsregisteredwithProctoringSolu7on

Slide24

iframeorwindow

Proctor’sBrowser

AMSProctorControls

ProctoringServiceProctorPageProctorControlslinkreceived

fromCandidatelaunch


LTILaunch Redirect


AMSCandidateExam

Delivery

AMSCandidatePortal


ProctoringSolu7onProctorsregisteredwithAMS

Slide25

ProctoringServiceProctorPage

LTILaunchAMS

ProctorPortal

ProctoringServiceWS

GET

iFrameorPop-up

Proctor’sBrowser

AMSProctorControls


ProctoringSolu7onReviewerflow

Slide26

ProctoringServiceReviewPage

LTILaunchAMS

ReviewerPortal

ProctoringServiceWS

GET

Reviewer’sBrowser


}  Reviewuse-casedocument:§  h_ps://drive.google.com/file/d/0B-xMrCxugbJxUDJHaHVXaUtoRmM/view

}  Fillinyourpriori7es:§  h_ps://docs.google.com/spreadsheets/d/1I6Dg6sqysumQk1XVGaX6FPfu2ramAsi804B2hGPbu1Q/edit?usp=sharing

Whatnext?

Slide27


Jointheteam…

Slide28

Documents

Proctoring Service API - IMS Global Learning Consortium · } Increasing interest in proctoring of exams delivered electronically } A number of proctoring services to choose between