24
Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. eAssessment in the Cloud, Sunshine or Thunderstorm? John Kleeman, Questionmark www.questionmark.com

eAssessment in the cloud

Embed Size (px)

Citation preview

Page 1: eAssessment in the cloud

Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

eAssessment in the Cloud, Sunshine or Thunderstorm?

John Kleeman, Questionmark

www.questionmark.com

Page 2: eAssessment in the cloud

Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Using “clouds” for computing tasks promises a revolution in IT similar to the birth of the web and e-commerce (McKinsey)

What does this mean for eAssessment in HE?

Technology pros and cons Installing eAssessment software locally

“On-premise” Using an eAssessment service “On-

demand”

The promise of the cloud

Slide 2

Page 3: eAssessment in the cloud

Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Questionmark offer both, so I can be neutral

Installable assessment management system

You Purchase servers & software

Install software

Install updates

Create your own eAssessment offering based on the software

Questionmark provides service at cloud data center

You Contract for the service

Connect to the service over the Internet

Delegate running the service 24/7 to Questionmark

Use the service for your eAssessment offering

Slide 3

Page 4: eAssessment in the cloud

Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Access to innovation Speed/flexibility of deployment Reliability and uptime Scalability and peaks in demand Security and cheating Governments accessing your data Data protection Ease of customization/integration Connectivity Can you change providers? IT bandwidth Cost, features and other factors

Key factors

Slide 4

Page 5: eAssessment in the cloud

Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Key advantage of On-demand is frequent updates without effort Support of mobile devices

Support of new interoperability standards

Support of uptodate browsers

Feature improvements

Access to innovation

Slide 5

Page 6: eAssessment in the cloud

Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

With an On-demand system, you are usually upgraded automatically

Most providers will Give notice of upgrades

Give opportunity to test

Make sure upgrade is upwardly compatible

Not a problem in practice with a good provider

What happens if I don’t want innovation?

Slide 6

Page 7: eAssessment in the cloud

Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Speed/flexibility of deployment

On-premise On-demand

Hardware needs purchase

System needs provisioning

Software needs installing

Needs planning and can be slow to deploy or increase scale

Turn on within days Increase capacity easily Reduce capacity easily Ramp up for temporary

courses or one-off events

Slide 7

Page 8: eAssessment in the cloud

Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

For high reliability/uptime, redundancy is key Power supplies

Internet connectivity

Firewalls

Servers

Database servers

Application servers

Offsite backup

Everything needs a backup On-demand will usually be superior

as long as participants have Internet connectivity

Reliability and uptime

Slide 8

Page 9: eAssessment in the cloud

Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Typical On-demand redundancy

Internet

Off Site Data Back Ups

Tapes

Page 10: eAssessment in the cloud

Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

What SLA is available? What are support hours? Do they match our hours? Is there a public status page? What is historical uptime? What protection vs power outages? How often are results backed up, e.g. hourly? Are back-ups off-site? Is every server load-balanced? Is there redundancy for everything? Is there monitoring for failures? What happens with planned maintenance? How strong is the provider’s disaster recovery plan?

Some key questions to ask a provider

Slide 10

Page 11: eAssessment in the cloud

Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Typical eAssessment load: Formative: steady during

year Summative peak at end

of term

On-premise Need to scale system to

cope with peaks

On-demand Gives an advantage

Scalability and peaks in demand

Slide 11

Page 12: eAssessment in the cloud

Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Just like a castle Security needs investment &

vigilance

UCISA: “In practice, data is probably more secure in cloud services than can be provided by in house solutions.”

Security and cheating

Slide 12

Picture by Simon Ledingham from geograph project, licensed under Creative Commons

Page 13: eAssessment in the cloud

Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Data center security Is it certified?

Are there 24/7 onsite guards?

Are all visitors escorted and videoed?

Fire suppression, 24/7 monitoring?

Tiered architecture, firewalls between each tier?

All traffic uses HTTPS?

Separate production and staging/development areas?

Can developers access production area?

Security built into application development?

Some key questions to ask a provider?

Slide 13

Page 14: eAssessment in the cloud

Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Be realistic! Governments mostly well-intentioned Does government want your assessment data? Does it matter if they have it?

If you are concerned

Make sure service provider in an acceptable country If you are concerned re US Patriot act, then these should all be

outside USA Service provider

Data center

Data center owner

Can governments access my data?

Slide 14

Page 15: eAssessment in the cloud

Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Data protection

On-premise On-demand

The institution controls and is responsible for data protection

Institution = data controller Provider = data processor

Questions for provider

Employee confidentiality? Background checks? Employee training/testing? Data encrypted in transit? One tenant protected from

other? Hard drives degaussed?

Slide 15

Page 16: eAssessment in the cloud

Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Employees background checked?

Employees sign confidentiality agreement?

How do they monitor third party suppliers?

Training (and testing) in data security?

Are end-of-life disks degaussed?

Is data encrypted if moved?

If US, is provider enrolled in Safe Harbor

Some key questions to ask a provider

Slide 16

Page 17: eAssessment in the cloud

Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

On-demand On-premise

Configure and template

Change core code X

Direct database integration X

Web services

E-learning standards

Connect to Moodle, Blackboard etc

Ease of customization/integration

Slide 17

Page 18: eAssessment in the cloud

Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

In campus universities, local connectivity can favour On-premise

Speed of light also small factor

But … Ray Fleming, Microsoft (2011): I saw some statistics yesterday that says over 85% of the use

of the Deakin University LMS takes place outside of the University. So there is a huge logic for putting things into the Cloud, as your users are out on the Internet – not necessarily in the institution when using your systems

Connectivity

Slide 18

Page 19: eAssessment in the cloud

Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Whatever system you use Questions

Assessments

Results

Are your data

On-premise and On-demand Make sure that the format of data is documented and open

On-demand Make sure that you have the rights to copies of your data

Can you change providers?

Slide 19

Page 20: eAssessment in the cloud

Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

eAssessment and the IT department eAssessment critical so needs professional maintenance

IT departments overloaded, IT skills in short supply

eAssessment is specialist IT skill

Tasks performed once per year difficult to train for

What can happen in On-premise eAssessment projects have to queue for priority in IT

Often a bottleneck

A challenge which On-demand resolves

Getting IT bandwidth

Slide 20

Page 21: eAssessment in the cloud

Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

When considering cost include: Hardware

System software

License costs

IT time

Support costs

Other factors Contract

Trust

Features

Training and SLA

Costs, features and other factors

Slide 21

Page 22: eAssessment in the cloud

Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

• Access to innovation

• Speed/flexibility of deployment

• Reliability and uptime

• Scalability

• Security and cheating

• Getting IT bandwidth

On-demand may be stronger

• Ease of customization/integration

• Connectivity

• Governments accessing your data

On-premise may be stronger

• Data protection

• Can you change providers?

• Costs, features and other factors

Needs review

Summary

Slide 22

Page 23: eAssessment in the cloud

Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

UCISA Cloud Computing Briefing Paper (2011) Download from www.ucisa.ac.uk

How to decide between On-demand and On-premise eAssessment (presented at 2012 CAA conference) http://caaconference.co.uk/pastConferences/2012/caa2012_

submission_23.pdf

Some further resources

Slide 23

Page 24: eAssessment in the cloud

Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Sunshine or thunderstorm: your questions

Slide 24