The ITEAM Project (integrating technology enhanced assessment methods for improved student support and self-regulation)

Rolling out institutional technologies – Electronic Voting Systems (EVS) Sharing the lessons from the ITEAM Project: integrating technology enhanced assessment methods Julie Vuolo and Janet Webber

The University of Hertfordshire is one of its region’s largest employers with over 2,700 staff and a student community of 23,000, including more than 2,000 international students from over 85 countries

Background

• Internally funded Assessment and Feedback project 2010-11 • Formulation of Assessment-for-Learning principles • Recognition of effectiveness and efficiency issues • Use of EVS to reduce assessment load and encourage active

learning and engagement • Local use supplemented by provision of 3,845 electronic voting

handsets across 8 academic Schools • JISC funding provided support for further Assessment and Feedback

project 2011-2013 • Schools purchased own EVS handsets 2011-12 (2975) • Lifecycle – logistics of implementing across an organisation

Lifecycle

• Considers the different phases of implementing EVS across institution

• 5 phases in total • Not discrete; overlapping and interdependent • Work in progress!

Decision Phase • Consider pedagogical rationale for use • Decide implementation approach e.g. Top

down • Agree type and number of handsets required • Identify training needs • Ensure technical and administrative support

in place

Purchase Phase • Identify preferred supplier • Agree pricing framework • Confirm order-to-delivery times • Agree ordering process with procurement teams • Identify delivery point • Request named contact identified for receipt of

goods

Distribution Phase • Agree personnel responsible for assigning handsets

to students (barcode scanner to EVS database), ensure scanners available, train personnel in use

• Agree timing of distribution (not at registration) • Give essential information* to students at point of

distribution (EVS info card & EVS info sheet) • Ask Programme Tutors / Module Leaders to signpost

students to EVS support page • Ensure re-issued handsets have new batteries and

are cleaned before distribution

* Essential handset information includes what to do if it is lost and an URL for the EVS web page

Usage Phase • Professional staff /first line contacts know how to use the technology • Students know how to replace handset batteries & to recycle old ones • Faulty or lost handsets are replaced using the agreed replacement

procedure • Students are reminded that misuse of the handset may be construed as

academic misconduct

• Teaching staff happy with technology • Students know to bring their handsets to every class • Teachers explain to students how & why EVS is used • Technical support provided by Help Desk; L&T support provided by

Learning and Teaching Institute/Champions

Usage Phase • Test conditions are employed where appropriate • Formative vs summative • Feedback on tests is timely with opportunities to discuss • Timing of release of results agreed; errors in reporting

identified and rectified promptly

Return Phase • A process for return is agreed which includes a specified return

point and a return by date • Students are informed in advance (e.g. module handbook,

email, verbally), module teams re-enforce message • The handset is un-assigned from the student at the point of

return • Batteries are replaced / handset cleaned for re-issue • Students who fail to return the handset are recorded as debtors

at the discretion of the School • A list of student EVS debtors is sent to the Student Centre by

the agreed deadline

Lessons to share (?success factors)

• Walking the ‘handset journey’ in advance

How did I get here? Who are they giving me to? How will this person know how to use me? What happens if my battery is flat? What happens if I break? What happens if I am lost?

Lessons to share (?success factors)

• Identifying key players from pro-VC to goods inward to supplier to teacher to technicians to manufacturer and so on…..

• Take account of organisational developments/ changes

• Keep everyone informed, keep conversations going • Provide just-in-time support • Expect the unexpected, assume nothing!