Document title: Jisc PotsPan Project Final ReportLast updated: February 2013

Project Document Cover Sheet

PotsPan Project Final Report

Project Information

Project Acronym PotsPan

Project Title Pilot of the SWANI Project Administrative Network

Start Date 1st May 1012 End Date 30th April 2013

Lead Institution Coleg Sir Gâr

Project Director Dave Howells

Project Manager & contact details

Professor Tony Tooletony.toole@e-college.ac07964894790Coleg Sir Gâr, Graig Campus, Llanelli SA15 4DN

Partner Institutions Pembrokeshire College, Swansea Metropolitan, JISC RSC Wales

Project Web URL Project Website: http://potspan.pbworks.com/ Project Blog: http://potspan.blogspot.co.uk/

Programme Name (and number)

JISC e-Learning Embedding Benefits

Programme Manager Rob Englebright

Document Name

Document Title

Final Report

Reporting Period

May 2012 – April 2013

Author(s) & project role

Tony Toole, Project Manager

Date February 2013 Filename PotsPan Final Report

URL http://potspan.pbworks.com/w/file/63770491/PotsPan%20Final%20Report.docx

Access

Document History

Version Date Comments

v1 19/02/2013 First Draft

1

JISC PotsPan Project Final Report

JISC Project Final Report

Project name: Pilot of the SWANI Project administrative network (PotsPan)

Author: Tony Toole

Contact: tony.toole@e-college.ac

Date: February 2013

Contents

Acknowledgements 2

1. Executive Summary 3

2. Background 4

3. Aims and Objectives 4

4. Methodology 4

5. Implementation 5

6. Outputs and Results 5

Case Study 1: Coleg Sir Gâr 6

Case Study 2: Pembrokeshire College 7

Case Study 3: Swansea Metropolitan 8

7. Outcomes 9

8. Conclusions and Recommendations 9

References and Appendices

Acknowledgements

The PotsPan Project was funded by JISC as part of the e-Learning Embedding Benefits Programme. The project team would like to thank Rob Englebright, the Programme Manager, and staff at RSC Wales for their support and assistance. We would also like to thank all those in the WBL teams in the partner institutions and in particular, the excellent work by Lewis Jones who built the digitised WBL administration system.

2

JISC PotsPan Project Final Report

1. Executive Summary

The PotsPan (Pilot of the Swani Project administrative network) project was funded as part of the Jisc e-Learning Embedding Benefits Programme. The goal was to take the outcomes of the successful Swani (Secure Work-based learning through Networked Infrastructure) concept-proving project completed by Coleg Sir Gâr in 2012 and to pilot it more broadly across the regional partnership.

The Jisc Swani Project created a digitised administrative and management documentation system for Work Based Learning in Wales that included a digital signature system that satisfied the audit requirements of the EU and the Welsh Government.

The PotsPan project piloted the system with two of the partners in the Skills Academy Wales South Wales (SAWSW) WBL consortium and also evaluated the approach in a partner University for potential use in the administration of a new distance learning WBL programme.

The purpose of the PotsPan project was to take what was a successful concept proving project and develop it into a practical digitised WBL administrative system that would not only satisfy the needs of the SWASW partnership, but would also be of value to other institutions addressing similar issues.

The broad aim of the PotsPan project was to implement the digitised online WBL administrative system, created by the Swani project. The key objectives were to:

Refine, update and test the WBL document management system to ensure it met the efficiency, usability and data consistency design requirements;

Test and evaluate the system with cohorts of work based learners in the SAWSW partnership;

Explore the use of digital pens and electronic signatures as part of the WBL programme audit arrangements;

Contribute to the European debate about digitised document management and the use of electronic signatures for audit purposes.

The outcomes of the PotsPan project led the project team to conclude that the work had confirmed the main benefits of administrative system digitisation:

Digitisation of the administrative system eliminated the need for the repeated entry of core data and ensured data consistency;

Digitisation made the entire administrative system available to programme managers at all times and from any location;

The system delivered significant operational efficiencies, mainly from the avoidance of physically transporting learner files between different geographical locations.

The overall conclusion was that the PotsPan project had achieved its primary goal of exploiting the benefits of the original Swani project and demonstrating its value and viability in the practical administration and management of the work-based learning programme in South West Wales. Furthermore, it was felt that the benefits were transferable to other programmes with similar needs and hence contributed positively to the Jisc e-Learning Programme Embedding Benefits objectives.

3

JISC PotsPan Project Final Report

2. Background

The JISC SWANI Project1 successfully created a digitised administrative and management documentation system for Work Based Learning in Wales that included a digital signature system that satisfied the audit requirements of the EU and the Welsh Government.

The PotsPan project piloted the system with two of the partners in the Skills Academy Wales South Wales WBL consortium and also evaluated the approach in a partner University for potential use in the administration of a new distance learning WBL programme.

The purpose of the PotsPan project was to take what was a successful concept proving project and develop it into a practical digitised WBL administrative system that would not only satisfy the needs of the SWASW partnership, but would also be of value to other institutions addressing similar issues. In that context it directly contributed to the objectives of the e-Learning Embedding Benefits programme.

3. Aims and Objectives

The broad aim of the PotsPan project was to implement the digitised online WBL administrative system, created by the Swani project. The key objectives were to:

Refine, update and test the WBL document management system to ensure it met the efficiency, usability and data consistency design requirements;

Test and evaluate the system with cohorts of work based learners in the SAWSW partnership;

Explore the use of digital pens and electronic signatures as part of the WBL programme audit arrangements;

Contribute to the European debate about digitised document management and the use of electronic signatures for audit purposes.

4. Methodology

The PotsPan project methodology followed a straightforward testing and evaluation process that began with a period of system refinement at Coleg Sir Gâr where the document management system was hosted. The web designer and the project manager worked together to test each of the system functions and the usability of each of the forms, including how effectively they shared common data fields.

Workshops were held at both Coleg Sir Gâr and Pembrokeshire College with the WBL teams that would be testing the system in the field. Each team selected an appropriate cohort of work based learners and completed a testing and evaluation exercise that involved completing all the digitised versions of the administrative documents in parallel with completing the existing paper based versions. Each exercise concluded with a comparison of the two approached with the aim of judging the relative advantages in terms of efficiency and effectiveness. These comparisons would be presented in the form of case studies.

An additional exercise was carried out at Swansea Metropolitan where the use of electronic signatures was explored. It was felt that this solution for document verification would be eventually accepted by the EU for audit purposes, but that was not the case currently. The exercise at Swansea Met, being unconstrained by EU rules, was designed to take that agenda forward by testing and evaluating current electronic signature technologies and their appropriateness for educational administration, particularly in the distance learning context.

1 http://www.jisc.ac.uk/whatwedo/programmes/elearning/swaniltig/swani.aspx

4

JISC PotsPan Project Final Report

5. Implementation

Following the usability testing exercise at Coleg Sir Gâr, the two partner cohorts for the WBL implementation testing and evaluation were chosen and the process begun. The paper based administration forms were completed as normal and were then transcribed onto the digital system. The ease with which this was completed was noted, as was the potential saving in time with the sharing of common data fields.

A perception at the outset was that there were administrative processes, particularly with the registration of new learners, for example, where multiple forms were completed by hand that contained the same information in terms of learner name, address, NI number etc. An anticipated benefit was that the digitised system would only require these data fields to be entered once, would ensure data consistency and, where editing was needed at a later date, the edits would automatically update all forms that used that data.

The testing of electronic signature usage at Swansea Metropolitan was focussed on the development of a distance learning version of a WBL programme delivered to the armed forces in Germany. Of particular interest was the use of such signatures to verify the authenticity of written assessment materials submitted online.

The outcomes of the testing were recorded using a common case study template for ease of comparison. The outcomes were also shared on the project blog and wiki and were presented for wider dissemination at ALT-C and at the JISC Online Conference 2012.

6. Outputs and Results

The Outputs and Results for the JISC PotsPan project included an updated and tested WBL Document Management System, a series of three Case Studies that detail the evaluation and testing of that system, and a project Blog that provides a narrative account of how the project evolved. The final output was this report that provides an overall summary of activities, achievements and deliverables.

The primary deliverable was the JOOMLA based document management system that was originally developed during the Swani project. As a result of the PotsPan testing and evaluation activities the presentation of the documents was refined and the efficiency and consistency of data sharing was improved. The system can be viewed at http://sawsw.colegsirgar.ac.uk/.

Examples of the documents and the way they are presented are shown on the project blog at http://potspan.blogspot.co.uk/ and are available for download from the project Wiki at http://potspan.pbworks.com.

There were three structured testing and evaluation exercises carried out as part of the PotsPan project. Two of them tested the document management system, and the use of the digital pens for audit purposes, with selected WBL cohorts at two of the partner institutions. The third exercise explored the use of electronically signed documents, rather than digital pens, in the administration of a work based distance learning programme at a third partner institution.

The outcomes are presented in the form of three case studies. Single page summaries are included below and the full case studies can be downloaded from the project Wiki.

5

JISC PotsPan Project Final Report

Case Study 1: Coleg Sir Gâr

The evaluation at Coleg Sir Gâr involved a group of modern apprentices at the TATA steel plant in Llanelli and the digitised administrative system was carried out in parallel with the existing paper based system and the outcomes compared.

Summary

{to be added}

Conclusions

{to be added}

Full Report

The full case study report can be viewed at: {to be added}

6

JISC PotsPan Project Final Report

Case Study 2: Pembrokeshire College

The evaluation at Pembrokeshire College also involved a group of {to be added} at {to be added} and the digitised administrative system was carried out in parallel with the existing paper based system and the outcomes compared.

Summary

{to be added}

Conclusions

{to be added}

Full Report

The full case study report can be viewed at: {to be added}

7

JISC PotsPan Project Final Report

Case Study 3: Swansea Metropolitan

The evaluation at Swansea Metropolitan was different to the other partners in that it explored the use of electronic signatures for audit documents rather than digital pens.

Summary

The contribution to be made by Swansea Metropolitan University as a partner in the JISC PotsPan project was to pilot the use of digital signatures with management and administrative documents in the context of its merger with University of Wales Trinity Saint David during the project period. The intention was to explore the benefits of electronic signatures on documents that needed to be shared on multiple sites, particularly with regard to the delivery of work-based learning.

The importance of this pilot exercise was that it was not constrained by the EU requirement for hand written signatures that led to the use of digital pens for the other pilot exercises. It was therefore an opportunity to explore the use and effectiveness of secure electronic signatures on documents and how they could be used to authenticate digitally communicated administrative paperwork.

The basic features of a secure electronic signature system are that it should be able to confirm ownership of the document, that the signatory was authorised to sign the document and that the document has not been changed since the signature was applied. Typically the system will also identify the computer used to create the document and will verify the date and time the signature was added.

A further element of security in the systems accepted by international finance and commerce is the use of a third party Certification Authority (CA) that generates encrypted public and private key certificates. The private key certificate is held on the owners’ computer and ensures that the encrypted data includes verification information that is recognised by the receiving computer(s) with the public key. If any aspect of the document and signature security is not verified, then a ‘not valid’ alert will be shown.

Both commercial and open source document creation applications are available that include the ability to add secure electronic signatures to documents and provide CA services. An objective in the PotsPan project was to identify the most cost effective solution for the institutions and this case study shows how open source solutions are both available and effective.

The pilot exercise created and tested a series of documents with secure electronic signatures. These documents were sent electronically as attachments and uploaded to institutional websites and were shown to retain the encrypted data confirming signature and document validity.

Conclusions

It was concluded that the work confirmed that electronic signatures that authenticated digitally transmitted administrative documents were achievable in a cost effective way and that met commercially accepted levels of verification and security. It is believed that electronic signatures will eventually become acceptable for EU audit purposes and allow a fully digitised online administrative system to be implemented.

Full Report

The full case study report can be viewed at: http://www.slideshare.net/ttoole/pots-pan-case-study-swansea-met

8

JISC PotsPan Project Final Report

7. Outcomes

It is anticipated that the outcomes of the PotsPan project will have practical value, both for the SAWSW WBL partnership, and for other groups looking for similar administrative efficiencies using technology. However, there were certain compromises that needed to be taken for the project to achieve its objectives.

The project took a pragmatic approach to the digitisation design, given the non-acceptance by the EU of electronic signatures for audit purposes. Ideally, it would have prefered a fully electronic signature system to be accepted, but, given the current rules, it adopted the digital pen as a compromise solution.

This was not ideal as it still required paper versions of all the forms to be produced in order for them to be physically signed. It did allow, however, the entire system to be digitised whilst still satisfying EU requirements. This meant that the advantages of single data entry, data consistency and the ability for management to access the system online were achieved.

This latter factor was perhaps one of the most important from a benefit point of view. The paper based system was managed by Training Advisors and the documentation distributed geographically as a result. This was both a risk and made central management access to data very difficult.

However, the goal of achieving an acceptable electronic signature solution remains. The outcomes of the PotsPan project are being disseminated with a view to contributing to and furthering that debate. Presentations at Alt-C 2012 & the JISC Online Conference have been part of that agenda.

8. Conclusions and Recommendations

The outcomes of the PotsPan project led the project team to conclude that the work had confirmed the main benefits of administrative system digitisation:

Digitisation of the administrative system eliminated the need for the repeated entry of core data and ensured data consistency;

Digitisation made the entire administrative system available to programme managers at all times and from any location;

The system delivered significant operational efficiencies, mainly from the avoidance of physically transporting learner files between different geographical locations.

However, the requirement for physical signatures on audit documents compromised the full realisation of those benefits and the project team remained committed to advancing the debate and argument for full electronic signature recognition for EU funded programmes.

The overall conclusion was that the PotsPan project had achieved its primary goal of exploiting the benefits of the original Swani project and demonstrating its value and viability in the practical administration and management of the work-based learning programme in South West Wales. Furthermore, it was felt that the benefits were transferable to other programmes with similar needs and hence contributed positively to the Jisc e-Learning Programme Embedding Benefits objectives.

The recommendations arising from the project outcome included:

Noting that not only was a digitised administrative system beneficial from an efficiency point of view, but also from a cost effective viewpoint: the document management system used by the PotsPan being open source;

Noting also that there were open source electronic signature systems available that met standard international security and authentication requirements;

Recommending that the current EU debate about the acceptability of electronic signatures be concluded, not only for educational audit purposes, but for advancing the wider free market goals of the union.

9

JISC PotsPan Project Final Report

10

JISC PotsPan Project Final Report

References and Appendices

All the project documents and additional materials are referred to as links in the text of this report and/or are available on the project Wiki. For ease of access, the following links lead directly to the key project documents.

1. PotsPan project Wiki: http://potspan.pbworks.com 2. PotsPan project Blog: http://potspan.blogspot.co.uk 3. Project Management: http://potspan.pbworks.com/w/page/53357624/Project

%20Management 4. The Digitised Administration System:

http://potspan.pbworks.com/w/page/53357697/Digitised%20WBL%20Administration%20System

5. The Case Studies: http://potspan.pbworks.com/w/page/53357795/Project%20Deliverables

11