#ALTC “Systems are arcane, forced to work, unreliable and convoluted”

@elearninglpool

“Systems are arcane, forced to work, unreliable and convoluted”

The long and winding road of Electronic Management of Assessment

Dan Roberts, Alex Spiers & Tunde Varga-Atkins Centre for Innovation in Education

Annual Conference of the Association for Learning Technology September 2017 University of Liverpool, UK

@elearninglpool

Click to edit Master title style

Photo by Inha Leex Hale - Creative Commons Attribution License https://www.flickr.com/photos/46835425@N00 Created with Haiku Deck

ContextToolsResearchInteractivityCommunity Created Solutions!

Photo by Marc Wathieu - Creative Commons Attribution-NonCommercial License https://www.flickr.com/photos/88133570@N00 Created with Haiku Deck

( EMA ) is increasingly being used to describe the way in which technology is used across the assessment life cycle to support the electronic submission of assignments, as well as marking and feedback. (JISC 2014)

@elearninglpool

Photo by Pulpolux !!! - Creative Commons Attribution-NonCommercial License https://www.flickr.com/photos/46425925@N00 Created with Haiku Deck

Purpose of this session

2016 Survey of Technology Enhanced Learning for higher education in the UK

Benchmarking and review

@elearninglpool

Created with Haiku DeckPhoto by UCL IOE - Creative Commons Attribution-NonCommercial License https://www.flickr.com/photos/43313941@N06

EMA Policy Project Board 2016

APVC Helen O Sullivan

Academic Staff (Faculty Rep)Professional ServicesCSD (IT Services)CIE (Centre for Innovation in Education)Project Manager (CSD)

Works StreamsPilotsProcessesEquipmentSystemsExternal examinationCommunication

Click to edit Master title style

Photo by falequin - Creative Commons Attribution-ShareAlike License https://www.flickr.com/photos/61314442@N03 Created with Haiku Deck

1 Think 3 Pair 3 Share

@elearninglpool

What tools are used?

@elearninglpool

• Insert the stats from the Craig Goacher

@elearninglpool

@elearninglpool

Printing

Fall Down Protocol

Management of technologyLimitations of supported tools

MOBILE DEVICE MANAGEMENT

Return of MarksLanguage

Support

Tablet Annotation FINANCIAL SUSPENSION

Moderation

Blind MarkingDistribution of marking

M O D E R A T I O N

E x t e r n a l E x a m i n e r a c c e s s

Accessibility

@elearninglpool



@elearninglpool


How does your institutions deal with this type of complexity?

@elearninglpool

Photo by Joe The Goat Farmer - Creative Commons Attribution License https://www.flickr.com/photos/132604339@N03 Created with Haiku Deck

Liverpool: Data

@elearninglpool

Data collection methods• Staff survey (n=345)

• Staff interviews eMarking pilots;

• Report from eLearning Unit learning perspective

• Student survey (JISC Digital Tracker) (n=670) responses from LivUni students;

• Student enhancement projects (Guild/SSLC)

• Desk-top research, see benchmarking report.

https://www.jisc.ac.uk/rd/projects/student-digital-experience-tracker

@elearninglpool

Student more divided on eMarking (n=670)

@elearninglpool

Vignettes• Staff less keen on eMarking: barriers outweighing potential

benefits, or prefer paper-based Marking;

• Staff who would be keen on eMarking but are frustrated by the lack of functionality of systems;

• Staff who have tried hard to make eMarking work but abandoned it, or scaled it back;

• Staff eMarking despite major difficulties through workarounds;

• eMarking enthusiasts who are on board and engaged in eMarking;

• Staff with pedagogical considerations for opting out of eMarking;

• Staff who are impacted by eMarking with respect to physical and health problems or those with accessibility needs;

• Occasional markers – e.g. PGRs. See also external examiners.

@elearninglpool

Recommendations – technical

• Improvements to reliability – some outside of Uni control!

• Lobbying with software providers (long-haul!)

• Support & tools for group assessments

• See also Recommendations around health & safety, equipment in full report & EMA system downtimes.

@elearninglpool



@elearninglpool


Why can’t we come together to build something to fit our needs?What is the ideal system? Is there one?

@elearninglpool

Photo by oggin - Creative Commons Attribution-NonCommercial License https://www.flickr.com/photos/85755792@N00 Created with Haiku Deck

FIND OUT MOREGet in touch with us…

@elearninglpool

[email protected]

elearningatliverpool.wordpress.com

Thank you for listening and participating