Does interoperability actually work? A case study carried out with Boon Low and Niall Barr

© 2001 By Default!

A Free sample background from www.pptbackgrounds.fsnet.co.uk

Slide 1

Does interoperability actually work?

A case study carried out with Boon Lowand Niall Barr

Niall SclaterNiall SclaterManager, Educational SystemsManager, Educational Systems

University of StrathclydeUniversity of Strathclyde

© 2001 By Default!


Slide 2

The IMS Question and Test specThe IMS Question and Test spec

For the exchange of questions and testsFor the exchange of questions and tests Processes include:Processes include:

– AuthoringAuthoring– StorageStorage– AdministrationAdministration– RenderingRendering

© 2001 By Default!


Slide 3

The packages testedThe packages tested

A leading commercial CAA package A leading commercial CAA package ((QuestionmarkQuestionmark PerceptionPerception))

The two leading virtual learning The two leading virtual learning environments (environments (WebCTWebCT and and BlackboardBlackboard))

A lesser known but highly innovative A lesser known but highly innovative rendering tool (rendering tool (Canvas ArenaCanvas Arena))

A university-built tool developed for testing A university-built tool developed for testing IMS QTI (IMS QTI (CETIS Rendering ToolCETIS Rendering Tool))

© 2001 By Default!


Slide 4

Scope of the evaluationScope of the evaluation

Concentrates on the Concentrates on the exchangeexchange of questions of questions between systemsbetween systems

Question creationQuestion creation ExportingExporting ImportingImporting

© 2001 By Default!


Slide 5

Question 1Question 1

Question 1:Question 1:Which electronic component allows alternating Which electronic component allows alternating

current (AC) to flow but prevents direct current (DC) current (AC) to flow but prevents direct current (DC) flowing?flowing?

A transistorA transistor A resistorA resistor A capacitor A capacitor [correct][correct] An inductorAn inductor

© 2001 By Default!


Slide 6

Feedback for question 1Feedback for question 1

Transistors are used to amplify or switch Transistors are used to amplify or switch both AC and DC currents, the correct answer both AC and DC currents, the correct answer is capacitor.is capacitor.

Resistors impede the flow of both AC and Resistors impede the flow of both AC and DC currents, the correct answer is capacitor.DC currents, the correct answer is capacitor.

Yes, capacitors prevent DC current flowing.Yes, capacitors prevent DC current flowing. Inductors have high impedance for AC Inductors have high impedance for AC

currents and low impedance for DC currents, currents and low impedance for DC currents, the correct answer is capacitor.the correct answer is capacitor.

© 2001 By Default!


Slide 7

Question 2Question 2

Which of these symbols represents a Which of these symbols represents a Field Field Effect TransistorEffect Transistor (FET)? (FET)?

11

22

33

44

© 2001 By Default!


Slide 8

CriteriaCriteria

Question creation – the ease with which the Question creation – the ease with which the two model questions can be created and the two model questions can be created and the degree of accuracy of question rendering degree of accuracy of question rendering

Exporting – the ease and accuracy with Exporting – the ease and accuracy with which the following applications export which the following applications export questions into the IMS QTI specification questions into the IMS QTI specification

Importing –Importing – the degree with which the degree with which the suite the suite of test questionsof test questions can be imported and how can be imported and how accurately the imported questions are accurately the imported questions are rendered.rendered.

© 2001 By Default!


Slide 9

Results of the evaluationResults of the evaluation

Question authoringQuestion authoring PerceptionPerception

– q1 entered without difficultyq1 entered without difficulty– q2 not so easy but workarounds foundq2 not so easy but workarounds found

WebCTWebCT– q1 fineq1 fine– q2 – feedback reduced to a single section for q2 – feedback reduced to a single section for

each answereach answer BlackboardBlackboard

– Feedback restricted to one section for the correct Feedback restricted to one section for the correct answer and one for the incorrect onesanswer and one for the incorrect ones

© 2001 By Default!


Slide 10

Results of the evaluation (2)Results of the evaluation (2)

ExportingExporting PerceptionPerception

– Worked fineWorked fine WebCTWebCT

– Requires use of a command line toolRequires use of a command line tool– Some minor errors in the exported QTI codeSome minor errors in the exported QTI code

BlackboardBlackboard– Questions exported within an IMS content Questions exported within an IMS content

manifest but in a proprietary formatmanifest but in a proprietary format

© 2001 By Default!


Slide 11

Results of the evaluation (3)Results of the evaluation (3)

ImportingImporting PerceptionPerception

– No problemsNo problems BlackboardBlackboard

– Didn’t workDidn’t work WebCTWebCT

– Content migration tool only for systems adminsContent migration tool only for systems admins– Extremely difficult to use even for themExtremely difficult to use even for them

RenderingRendering

© 2001 By Default!


Slide 12

Rendering: WebCT - PerceptionRendering: WebCT - Perception

© 2001 By Default!


Slide 13

ConclusionsConclusions

VLEs very disappointingVLEs very disappointing None of the other systems had problems with None of the other systems had problems with

q1q1 Questionmark has been more rigorous than Questionmark has been more rigorous than

the VLE vendors in implementing QTIthe VLE vendors in implementing QTI– But even it had some problemsBut even it had some problems

Does this bode well for interoperability?Does this bode well for interoperability?

© 2001 By Default!


Slide 14

Conclusions (2)Conclusions (2)

Still in early daysStill in early days Most problems easy to correctMost problems easy to correct Need cross-vendor collaborationNeed cross-vendor collaboration Stress-testing programmesStress-testing programmes VLE vendors may leave QTI compliance to VLE vendors may leave QTI compliance to

QuestionMarkQuestionMark

© 2001 By Default!


Slide 15

Further infoFurther info

www.cetis.ac.ukwww.cetis.ac.uk

www.sclater.comwww.sclater.com

Documents

Does interoperability actually work? A case study carried out with Boon Low and Niall Barr