Smudie Project Report 3

The Academic Viewpoint

Introduction

This report covers the outcomes of the third phase of interviews carried out as part of the JISC Swansea Metropolitan University Data Integration Exercise (Smudie) Project. The focus was primarily on student information management by Academic Staff at the University.

Three members of staff from each of the four Faculties were interviewed and the sample included Programme Directors, Year Tutors and Module Tutors. Further interviews were also carried out with IS staff, ADQs and Faculty Secretaries, as well as with staff in the Finance Department and the Students Union.

The main message that came from the discussions was that information management in the Faculties was focussed on day to day course management issues and only periodically on summative assessment and statistics reporting. Communications with the students and their effective support was central to course team activity and for this reason local information recording systems and the Moodle online learning environment tended to play a much more prominent daily operational role than the institutional student records system.

A further factor that emerged was the variability in delivery between the curriculum areas and the effect that this had on student information management. It could also be seen that differences in reporting practice reflected differences in local management organisation within the Faculties. This ranged from student information management systems organised on a school-wide basis to systems devised and operated by individual academics. The formal process of reporting student academic progress, however, was periodically brought together through the entry of student assessment outcomes using the QLS V4 system and was reported and reviewed through the Faculty quality systems.

Attendance monitoring was also a major component of Faculty student information management activity. As detailed in other reports1, this is an area where attention is needed in terms of procedural and operational consistency across the University. It also has implications for the institutional reporting requirements and the way the University is viewed by external stakeholders, particularly UKBA, SLC and the way the institutional profile is reported by HESA.

This report summarises the systems in place in the Faculties for the purpose of aiding discussions about potential improvements. The fact that there are multiple systems within the institution is neither unusual for a large organisation nor, necessarily, ineffective. What may be undesirable is the existence of local information management systems that are unknown generally and hence out of institutional management control. The report concludes with some recommendations in this regard.

1 https://www.dropbox.com/s/cwzx991qap5s23z/Student%20Attendance%20Monitoring.docx

An Overview of the SMU Student Information Management System

The schematic diagram below is a development of the comprehensive picture produced by John Millward2. It shows the range of inputs and outputs to the student information management system, both internally and externally. On the right of the diagram is a representation of all the internal interactions and on the left are the external links. The overall message is that it is a complex system involving many stakeholders.

Important for the issues raised in this report are the components at the top and the bottom of the diagram. Firstly, academic staff have three different user interfaces with the student records system, which is reported as problematic. Secondly, the system communicates with other information management systems used by the University but, perhaps more importantly, does not communicate with a range of local databases, spreadsheets and records that are also key components of the information management processes.

2 https://www.dropbox.com/s/06uzb4tvrmna7y7/MIS%20Schematic.docx

Data entry into the system occurs through different interfaces and with different timing patterns largely aligned to the standard academic year. The timing patterns can be broadly described as:

1. Annual. Eg: UCAS applications, student enrolment, student registration, exam boards, HESA returns;

2. Periodic. Eg: student assessment recording, student attendance reporting; 3. Ongoing. Eg: student attendance monitoring, student progression monitoring, student

support records.

Much of the core student data is captured through the application and enrolment processes at the start of the year using customised system interfaces/processes. This is largely managed centrally in liaison with the Faculties. Periodic and ongoing student data is managed by programme teams in the Faculties and is recorded and stored locally in paper form and, typically, using individual or shared spreadsheets.

Processes and timings vary, but assessment results will be entered at appropriate points to the central student record system through the V4 interface. Mid-session and end of session Faculty quality procedures ensure the consistency and accuracy of this process and, as the primary performance indicator for the institution derives from the outcome, the quality system that supports it is taken very seriously.

Attendance monitoring does not have the same level of quality control or consistency. External reporting obligations vary from the regular and detailed information about overseas students required by UKBA to the more top level annual reporting to HESA. As a consequence, those programmes with a significant proportion of overseas students have more rigorous monitoring and reporting systems than those that do not.

Attendance monitoring is also used to inform the student support systems in the Faculties. Poor attendance triggers action to support students who have problems and, if not successful, leads to the formal processes for withdrawal, transfer or suspension of studies. There is inconsistency across the institution in the formality of this process and hence in the way data is captured. It should be noted however, that the interviews demonstrated that all academic staff are focussed on effective student support and always seek to find the best solution when problems arise. All that differs are the methods employed to achieve that outcome. The sharing and documenting of good practice would be of general benefit.

Information Systems in Academic Delivery and Assessment

There are four main information systems used by academic staff at SMU as part of programme delivery and assessment. These are:

1. The Faculty Query System (FQS): the original internally developed student record system at SMU, still used extensively by staff;

2. QLe: one of two user interfaces to the Agresso Student Management system introduced in 2010 and intended to replace FQS. This interface is read-only and provides access to student records;

3. QLS V4: the second interface to the Agresso Student Management system that academic staff use to populate the student records with assessment data;

4. Moodle: the Online Learning Environment adopted by SMU in 2011 as a replacement for the previous Blackboard system. Moodle is populated by core student data from QLe and provides teaching resources and communications functionality to support course delivery.

A fifth systems component is planned for online assessment submission. The Turnitin system will replace the current requirement for students to hand in assignments in the school office and to receive a receipt.

Summary of usage:

The Faculty Query System: FQS remains in use by many academic staff to access student and cohort information. It was originally expected to be replaced once the Agresso QLS system was in place, but usability issues with the new system has led to its retention and updating.

Key messages:

• FQS is used by many staff as their primary source of student information, particularly at the beginning of the year as it provides convenient access to core data including student photographs, and matrices of cohort information for checking correct registration on modules and similar details that are reported not easy to extract from QLe. FQS is populated by QLe when the students self-enrol and is used by many staff as their source of cohort information when creating class registers.

• Certain recent updates of FQS have not been universally welcomed. The two most quoted have been, firstly, the change from a name search to a student ID search process which staff find inconvenient because they are familiar with names but not with IDs. The second issue is with the decision to identify part-time students through the use of ‘year 9’ in the year of study data field. This means that their records do not include their actual year of study.

• This raises the issue of how system upgrades are agreed and implemented. The IS staff are seeking to optimise the system for the users and respond to requests for improvements. However, changes for one group of users may impact in an unforeseen way on the use by others. A review of change management processes may be needed here.

• Newer staff who joined SMU after the launch of QLe report using only the new system and not FQS.

QLe: This read only interface provides intranet access to the QLS student records system for academic staff. This includes application information from UCAS, student self-enrolment information and other data needed by academics and support staff preparing for new cohort delivery.

Key messages:

• There remains a view that FQS is a more user friendly interface to student cohort information than QLe, particularly when preparing registers and other administrative systems for course delivery. This is ironic as QLe populates FQS which effectively re-presents the same data.

• There also appears to be widespread confusion amongst academic staff why there are two different interfaces to QLS and that they need to be signed on to separately. The role of V4

in entering assessment data and preparing for exam boards is understood, but the purpose of a different read only interface to the same system is not.

• A common feature of academic programme management is that student information is managed locally most of the time and feeds into QLS only periodically. While FQS provides the day to day access to records needed by academics there is no incentive to adopt another system that is felt to provide a poorer service.

QLS V4: This is the main interface used by academic staff to enter student assessment outcomes. Data is entered at the conclusion of assessment periods which are typically twice per academic year but vary between curriculum areas. Academic staff enter the data individually, with help from the Faculty MIO or with help from the Faculty office.

Key messages:

• Because the system is only used by academic staff once or twice a year, they are unfamiliar with the menus and methods which make the process inefficient and frustrating (particularly part-time staff). Help from the MIOs and the Faculty office staff is often necessary and appreciated.

• Staff, particularly part-time staff, are not all aware of the training or support available from IS or the MIOs. Assistance is often sought from immediate colleagues.

• The interface is reported to be non-intuitive, but this may be due to infrequent usage. A common comment about usability, however, was that assessment data entry was frustratingly slow with a wait before the next entry could be made. This appeared to depend on the volume of system traffic at the time.

• Typically, academic staff keep their own spreadsheets of student assessment results as they can’t view a summary for the whole cohort from QLS. This was reported to be particularly the case when students were studying on a portfolio of modules drawn from a number of different programmes. In one curriculum area the tutor spreadsheet data was reported to be used for the external examiner pack rather than a QLS report.

• Multiple grade averaging and applying resit mark limits were identified as examples of procedural issues that needed to be resolved.

• There was a bottleneck leading up to exam boards when last minute assessment outcomes needed to be entered onto the system. This appeared to be more a management than a systems issue.

Moodle: the open source Moodle VLE was launched as the institutional online learning environment in 2011, replacing the commercial Blackboard application. Adoption by academic staff has been positive, not the least because of the support provided by the IS team and the e-Learning support officer. Moodle is populated with student information from QLS when the students have self-enrolled.

Key messages:

• Moodle is used to communicate with students for personal and course information. Email & forums (the latter to communicate with the whole group) are used and it encourages the students to use the university email system. For a number of courses it is the main method of out of class communications and is very good for peripatetic and part-time students.

• Moodle is used as teaching resource repository and to send materials to students. Student work can be submitted through Moodle. The intention is to use Turnitin for the future submission of all assignments and have receipts issued online, thus replacing the current Faculty office submission process.

• Moodle is also used for course announcements and students can use it to check on changes in daily course activity such a staff absence or room change for teaching sessions. It provides a calendar for course planning and can also contribute to the social aspect of cohort activity through appropriate communications.

• The availability of teaching resources online adds significantly to the choices available to students when they engage with their learning. Although it is not designed to replace formal taught classes, it can provide an effective backup if necessary. It is also very efficient for students during revision with all relevant resources assembled in the same place.

Information Systems in Attendance Monitoring and Reporting

Three reports have been produced concerning student attendance monitoring and reporting at SMU3,4,5. The overall conclusions were that student attendance monitoring was currently an institutionally unmanaged process and that locally managed systems were in place at Faculty, School, Programme and tutor levels. This applied at all stages in the process from the creation of attendance registers, through the completion of attendance recording, the storing and archiving of attendance records and the reporting of attendance to external bodies.

Typically the system uses paper based recording in the classroom and usually, but not always, by students individually signing next to their names on a circulated list. Other methods are also used, including the tutor completing a traditional roll-call or simply noting who is present and completing the form independently. These latter methods are only practicable for small class sizes.

The system employed also depends on the curriculum area, the session format and the attendance pattern. Under any circumstances where the students in a cohort are not carrying out the same activity in the same place at the same time, appropriately customised recording procedures need to be devised.

The actual registers used by tutors to record attendance are created at various levels by programme directors, year tutors or individual module tutors depending on the local management arrangements in place. Whoever creates the register will typically interrogate either FQS or QLe for details of the enrolled students as indicated in the diagram:

3 https://www.dropbox.com/s/hleeeijkgon3k3t/A%20Brief%20Report%20on%20Student%20Attendance%20Monitoring.docx 4 https://www.dropbox.com/s/cwzx991qap5s23z/Student%20Attendance%20Monitoring.docx 5 https://www.dropbox.com/s/zc7xufftogffquy/Update%20on%20the%20Student%20Non-attendance%20Review.docx

These paper records are then typically, but not always, transcribed onto local spreadsheets which are themselves sometimes transcribed onto a master programme or school spreadsheet. Where this does happen the spreadsheets are available as shared documents on the staff intranet and are designed to flag up problems which can then be acted upon.

In addition to these largely similar attendance monitoring practices, there are examples of individual variations where, for example, there are no paper records and the attendance is entered directly onto spreadsheets in the classroom or where absence rather than attendance is recorded.

All programme teams use the attendance monitoring information, together with academic progress information, in their support activities for students. They follow the institutional procedures activated by successive non-attendance that, if not resolved, lead to the formal processes of withdrawal, transfer or suspension of studies.

An important factor for the university, when instigating change to create a solution for problems students have with their study programmes, is to demonstrate that:

1. The needs and aspirations of the students are paramount and that the institution will always look for the best solution for the student;

2. The first choice solution will always be to guide the students to more suitable programme arrangements before the less desirable options of suspension of studies or withdrawal.

In that way the institution is demonstrating its commitment to finding pathways to student success and value for money both as a public sector body and, increasingly, as a student funded institution. This is important for the institutional public profile, future marketing and recruitment.

The need to establish a consistent and accurate system of attendance monitoring, particularly in this era of increased student fees and institutional service provision accountability, is recognised by SMU management. Trials have been carried out in the use of student proximity card attendance recording and the use of CELCAT timetabling software. The conclusions from that trial showed significant potential benefits, but that there are logistical, cost and procedural issues to be dealt with if an institution-wide implementation is to be successful.

There is a genuine concern expressed by academic staff that a technology based solution would impact on the need for flexible and adaptable curriculum delivery. This is particularly relevant as blended learning support systems evolve and student choice in engagement develops. A repeated comment was that they feared that an imposed IT solution might compromise academic quality and innovation.

Conclusions

The main conclusion that can be drawn from this study is that the management of student information at the university operates at different levels within the faculty structures and with different levels of management planning and control.

This is entirely expected as academic staff with different levels of management responsibility have different roles to play in the information management process.

• Primary data is gathered by the module tutor about student progress and attainment and this is used for subject level student feedback and support. Local records are kept of attendance, assessment outcomes and other relevant information such as mitigating circumstances;

• Programme Directors and Year Tutors access summative data, typically in spreadsheet form, drawn from individual module records. Often, but not always, these are available to the course team through shared spreadsheets on the staff intranet. This facilitates team discussions about individual student progress and decisions about any actions to be taken;

• ADQs, MIOs and Faculty Secretaries are all involved with the programme teams ensuring that assessment outcomes are entered onto the student records system and that the information is accurate and complete. It is at this level that consistent quality procedures are assured across the university in preparation for Exam Boards.

There are different categories of student data recorded that are used for different purposes by different staff in the institution. These include:

• Core Student Data: largely drawn from the student application and enrolment process. This triggers student registration, the generation of student cards, email and Moodle accounts and the population of FQS and QLS. It is also used by Finance to set up SLC and fees arrangements, The Exams Officer and Student Support Services to make provision for special needs, the International Office to liaise with UKBA etc.

• Student Academic Data: used by module tutors to provide academic support; by programme teams to report on student attainment; by the quality system to judge programme and faculty academic performance and to plan quality improvement; to report externally on institutional performance.

• Student Attendance Data: used by module and year tutors to monitor day to day progress and allow early identification of individual problems and take action; by the international office to report to UKBA; by finance to report to SLC; by information services to report to HESA.

It is clear that such a multi-faceted, multi-purpose, multi-stakeholder system is unlikely to be adequately supported by a one size fits all management information solution. The way the system is profiled here suggests a modular approach with shared data fields, which may well be possible using the Agresso QLS package with appropriate add-ons such as CelCat for timetabling and attendance monitoring, Moodle for learner support etc.

However, that is not the way it is working at the moment. It is also not obvious that the flexibility and adaptability needed for curriculum variety and innovation would be possible using these tools. It may be, but we don’t really know.

It is proposed, therefore, that the final phase of the Smudie project should carry out the task that was always in the plan: to create a ‘to be’ model for student information management. It is suggested that this should be strategic and process based rather than technology specific.

The objective would be to address the inconsistencies and variable management practices identified in this study, whilst also specifying a system that the stakeholders at each level agree meets their needs. Such a specification could be applicable in the design of whatever technology system is preferred by the newly merged institution.

Tony Toole August 2012