Visit Report on the University of Central Lancashire (UCLan) medical school

This visit is part of the new schools quality assurance annual cycle.

Our visits check that organisations are complying with the standards and requirements as set out in Promoting Excellence: Standards for medical education and training.

Summary

Education provider University of Central Lancashire (UCLan) medical school

Sites visited UCLan campus; Royal Blackburn Hospital

Programme MBBS

Date of visit

26th January 2017

2nd & 11th May 2017 (OSCEs)

7th June 2017 (Progression Examination Board)

8th June 2017

Key Findings

Over the 2016/17 academic year we visited the University of Central Lancashire (UCLan) medical school (the School) five times as part of our multi-year quality assurance review. We will continue the rolling cycle of annual quality assurance visits to the School following the first cohort of students through to graduation and their first year of practice.

During our visits we met with year one and two students and a range of staff from different teams across the School. We observed the May Objective Structured Clinical Examinations (OSCEs) and the Progression Board, as well as visiting the Royal Blackburn Hospital where we met with

2016/17 academic year

2

several Trust staff members who will be involved in the delivery of clinical placements.

In January, we felt that the School had not made adequate improvements to its assessments, with poor question stems and significant changes made to the assessments. As a result, we set two interim requirements (detailed below) for UCLan to work with its contingency School (Liverpool Medical School), as well as conducting an external review of the School’s assessments as a whole.

Whilst there continued to be a number of students who failed their summative assessments, we noted improvements to the content and management of the assessment load when we visited the School in June. Other concerns which had arisen in January, such as Phase 2 accommodation, also showed improvements. We will continue to monitor these areas closely over future visit cycles, and encourage the School to continue their successful collaboration with Liverpool.

Update on open requirements and recommendations

Open requirements Update Report paragraph

1 The School must ensure that the learning outcomes in the curriculum align with those of the contingency school, Liverpool, by 1 December 2014.

This requirement has been partially met. We heard that the learning outcomes for Phase 2 programme have now been aligned to Liverpool Medical School. We have received documentation on this subject, but will explore mapping in more detail over the future visit cycles.

95

2 The UCLan MBBS curriculum must be mapped in detail to the Liverpool MBBS curriculum to ensure no students are placed at a disadvantage for future assessments. Were there to be any delays or significant

This requirement has been partially met. We heard that the UCLan MBBS curriculum has now been mapped in detail to the Liverpool MBBS curriculum for Phase 2 of the programme. We have received documentation on this subject, but

95

3

problems with the MBBS, the UCLan students would be required to take the Liverpool assessments in order to complete their studies.

will explore mapping in more detail over the future visit cycles.

3 Given the relatively small size of the School and the committee structures, the School must give appropriate consideration to the security of students’ personal sensitive information. The School must ensure the committee structure excludes those formally involved in student support and progress decisions from making formal fitness to practise decisions.

This requirement has been met. The Fitness to Practise procedures specify that any member of the Fitness to Practise panel should have had no direct interest or prior involvement in the student’s case. The School also continues to take steps to protect student anonymity.

58

4 The School must include all known risks in its risk register so that the GMC can be satisfied that they are being identified and managed quickly and effectively.

This requirement has been met. The risk register now includes operational risks (including those pertaining to assessments), and we expect the School to continue to update the risk register as new risks arise.

29

5 The School must be clear how students move from student support to sanction. It is not clear what the thresholds are for placing a student into the fitness to practise process or how decisions are made to enter them into the fitness to practise process.

The requirement has been met. The School has put additional guidance in place and should continue to work with students to raise awareness.

83

6 The School must provide clear information for students about Educational Performance Measure scores and other criteria used for the UK foundation programme rankings.

This requirement has been met. The School has a clear process for calculating the score, and relevant information is included in student handbooks.

84

7 The School must review the guidance provided to students

This requirement has been met. All of the students we spoke to were

50

4

and educators about the professionalism and reporting system in order to improve understanding of the purposes of the process.

aware of the reporting process and knew how to raise concerns.

8 The School must develop a clear mechanism to assess student professionalism that is separate from fitness to practise.

This requirement has been partially met. The School has well developed plans for the assessment of professionalism, which will be piloted next year. We will explore these in detail at future visits.

53

9 The School must review its curriculum and assessment content and attached guidance in order to improve student preparedness for summative assessment and programme progression.

This requirement has not been met. Students continue to report that lecture content is inconsistent and is not always aligned with their assessments.

100; 123

10 The School must ensure that it has adequate numbers of experienced and trained assessment staff to:

• ensure the quality of item or station writing.

• ensure the quality of resit assessments.

• create detailed mark schemes that promote consistency of marking for short answer questions and essay questions.

• develop and administer standard setting procedures that are appropriate for the context of each summative assessment.

• prepare and interpret psychometric analyses for

This requirement has been partially met. Whilst the School ensures that staff are trained on how to deliver assessments and offers opportunities to attend external courses, a business case has been submitted for further assessment administrative staff. We will monitor to ensure that it is sufficient as the School continues to grow.

8; 89

5

each summative assessment event, and use this to quality manage item/station banks.

improve the School’s ability to write, mark and standard set assessments.

11 The School must work with Liverpool School of Medicine, as its contingency school, to incorporate questions from Liverpool’s item bank in to UCLan’s MCQ summative assessments for Years 1 and 2 in 2016/17. The questions from Liverpool’s item bank should make up 10% of UCLan’s written examinations, and should be inserted in amongst UCLan’s questions and standard set independently by UCLan.

This requirement was set following our visit in January and has been met. The School incorporated items from Liverpool Medical School’s question bank in their multiple choice question summative assessments. The School standard set the summative assessments independently, and also performed a standard setting exercise with Liverpool following the visit in January 2017.

112

12 The School must commission an independent review of its assessment processes, including standard setting and the performance and outcomes of the 2016 summer written examinations. The GMC will provide a list of names for the School to select from. If the School wishes to use an alternative expert, they must offer an appropriate rationale and discuss with the GMC. This must be completed before the summer diet of summative assessments in order that appropriate changes can be made.

This requirement was set following our visit in January and has been met. The School has commissioned an independent review of its assessment processes, with feedback from a range of external personnel. We saw evidence of reports from examiners as well as plans for further reviews.

113

6

Open recommendations Update Report paragraph

1 The School should ensure that the formative and summative assessments are close enough in format and scope, so that they do not impact students’ preparedness for what is expected of them.

This recommendation has been met. Students were satisfied that their summative assessments matched the formative assessments in scope and format.

122

2 The School should review the information available to students about various aspects of the programme and ensure that student understanding improves.

This recommendation has not been met. We continued to hear of areas where the guidance available to students did not appear to provide adequate information about key areas of the programme (such as Phase 2 accommodation and the USMLE).

76-77

3 The School should take steps to standardise the duration and depth of detail of class based lectures. In addition, the School should ensure that students receive lecture slides within a specified, standardised time frame.

This recommendation has been partially met. We heard that lectures have been uploaded on time, and that this is audited by theme leads. However, we continued to hear reports throughout our visit cycle of overrunning lectures which did not cover the intended content within the timeframe.

100

4 The School should take steps to improve the standardisation of students’ experience of primary care placements, by making expectations and learning outcomes explicit to both students and educators.

This recommendation has not been met. The School has taken positive steps to communicate with educators and students, but we still heard evidence of differing experiences in which student expectations were not met in both community and primary care placements. Students were concerned that this disparity in experience would affect their ability to meet their learning outcomes.

103

5 The School should review its This recommendation has now 117

7

policy for releasing OSCE station titles 24 hours in advance to ensure students allocated to earlier circuits are not disadvantaged compared to students on later circuits. In particular, it must ensure that students are aware that the policy is planned to be changed for the clinical years of the programme.

been met. The School has continued to expand its physical space to reduce the need for quarantine, although the practice of releasing station titles will continue for Phase 1 students. Students had, however, been made aware that titles would not be released from Phase 2.

6 The School should ensure that all equipment and simulated patients are standardised for each OSCE station to avoid unnecessary concerns from students.

This recommendation has partially been met. Some students were concerned that their simulated patients differed and that this could affect their mark. In addition, we found examples of where simulated patients did not appear to be fully briefed on their station content.

118

7 The School should review the marking sheets provided to OSCE examiners. This will avoid the risk of error due to transcription of marks from paper to an online system.

This recommendation has been met. Examiners are now provided with tablets on which to record marks, with back up paper mark sheets should the tablets fail.

18

8 The School should review whether it has sufficient in- house expertise required for assessment. The School should provide a written response to the GMC as to the outcome of this review and any actions to be taken.

This recommendation was set following our visit in January and has been met. The GMC received the School’s letter, detailing the actions it has taken. A business case has also been submitted for additional assessment support.

124

8

Areas that are working well

We note areas where we have found that not only our standards are met, but they are well embedded in the organisation.

Theme Areas that are working well Report paragraph

111

Theme 1: Learning environment and culture (R1.18)

The School appeared to have good resources for the OSCE in terms of their technical and personnel support.

16

2 Theme 1: Learning environment and culture (R1.19); Theme 5: Developing and implementing curricula and assessments (R5.1, R5.2, R5.4)

The preparations for Phase 2 are well advanced in terms of faculty, placements and the curriculum. The enthusiasm of the Phase 2 module leads we met with was evident, and we felt that they were well involved with curriculum development.

19; 97; 104

3 Theme 2: Educational governance and leadership (R2.20)

The School has taken steps to develop its widening participation plans with additional outreach work within the local community. We heard of two fully-funded widening participation places for this year, with plans for further expansion. We look forward to seeing this develop.

65

4 Theme 2: Educational governance and leadership (R2.18)

The School has engaged with a wide range of internal and external stakeholders to develop its Fitness to Practise agenda. For example, the School hosted a Fitness to Practise meeting with the medical schools within the region. This has led to regional threshold standard setting and cross-faculty panel development.

57

5 Theme 4: Supporting educators (R4.1)

The School supports the development of its staff by investing and allowing them to gain expertise in education,

89

9

teaching and assessment.

6 Theme 5: Developing and implementing curricula and assessments (R5.3)

The plans for the Health and Society module, specifically the third sector placements, are promising.

99

Requirements

We set requirements where we have found that our standards are not being met. Each requirement is:

targeted

outlines which part of the standard is not being met

mapped to evidence gathered during the visit.

We will monitor each organisation’s response and will expect evidence that progress is being made.

Theme Requirements Report paragraph

1 Theme 3: Supporting learners (R3.7); Theme 5: Developing and implementing curricula and assessments (R5.3)

We welcome the review into the student selected components over the summer. In particular, the School must look to standardise the guidance available on this module for tutors and students, and allow an equitable access to topics of students’ choice.

78; 101

2 Theme 5: Developing and implementing curricula and assessments (R5.4)

The School must review its inter-professional learning sessions to ensure that they adequately enhance students’ learning. We hope to see students given the opportunity to work and learn with and from other health and social care professionals and/or students to support multidisciplinary working.

106

10

3 Theme 5: Developing and implementing curricula and assessments (R5.6, R5.7)

The School must continue to work with its contingency School (Liverpool Medical School), and internally, in order to identify reasons for the continued high failure rate; to improve the summative examinations; to ensure assessment content reflects taught content; and to ensure there is an increased likelihood of students meeting the annual progression requirements (above what was observed for 2016-2017). This collaboration must include a wider review of the School’s curriculum, learning and teaching methods, and learning outcomes to ensure an internal alignment between teaching, learning and blueprinting of assessment content.

114; 123

Recommendations

We set recommendations where we have found areas for improvement related to our standards. They highlight areas an organisation should address to improve, in line with best practice.

Theme Recommendation Report paragraph

1 Theme 2: Educational governance and leadership (R2.12)

The School should review the discretionary powers of the progression board to award the option of resitting a year, and formalise this process.

47

11

Findings The findings below reflect evidence gathered in advance of and during our visit, mapped to our standards.

Please note that not every requirement within Promoting Excellence is addressed. We report on ‘exceptions’, e.g. where things are working particularly well or where there is a risk that standards may not be met.

Theme 1: Learning environment and culture

Standards S1.1 The learning environment is safe for patients and supportive for learners and educators. The culture is caring, compassionate and provides a good standard of care and experience for patients, carers and families. S1.2 The learning environment and organisational culture value and support education and training so that learners are able to demonstrate what is expected in Good medical practice and to achieve the learning outcomes required by their curriculum.

Raising concerns (R1.1)

1 We were pleased to hear from the clinical supervisors, who will be working with students on clinical placements in the next academic year, about their desire to create a culture where students can raise concerns. In addition, all students we spoke to in both January and June told us that they feel they would be able to do so. We look forward to seeing this develop next year as students begin their Phase 2 hospital placements.

Dealing with concerns (R1.2); Educational and clinical governance (R1.6)

2 In our previous report, we noted that the students we spoke to did not seem clear on how they should report a patient safety concern on placement. However, when we reviewed this area during the 2016/17 academic year we were pleased to hear that all students were now aware of the various routes through which they can raise concerns. They would do so through raising a Structured Event Reporting Form (SERF), or talk to the practice manager, their Academic Advisors or the Head of School. The School had also given students business cards with contact details of various School staff should they need assistance or have concerns whilst on placement.

3 During our meetings with placement staff, we heard that the School has developed a patient safety reporting process for students to follow whilst on hospital placements in Phase 2. On identifying a potential concern, students are required to complete a reporting form or raise a concern with the School, after which an investigation is launched and appropriate support for the student is put in place. The issue is then addressed through appropriate channels, and fed back to the student and the School,

12

as well as being reviewed by the Rapid Response Medicine (RRM) committee. This process will be included in student handbooks and the Trust inductions, and emphasises that the results of any concerns raised are fed back to the student who raised it. We will review the effectiveness of this process over future visit cycles.

Supporting duty of candour (R1.4)

4 We heard in January that the duty of candour is taught formally in Year 2. However, it was encouraging to note that, when asked, all students from both year groups were aware of the concept and could describe how it was relevant to practice.

Seeking and responding to feedback (R1.5)

5 Students continued to praise the School’s responsiveness to their feedback, and told us that they had seen positive changes as a result of the School acting on their comments in a timely manner. Actions taken are fed back through the course representatives or group discussions. Students were also aware of a ‘You Said, We Did’ document which demonstrates to students how the School are responding to their feedback. One example given by students to illustrate this was that some students were sent on a placement which closed after 11:00. After feeding this back, the practice is no longer used.

6 Students also told us that the School had an open door policy, and that this meant they felt able to informally feedback to staff at any point. In addition, we heard from quality management staff that the School provides various opportunities and ways for students to submit feedback, such as through online submissions on Blackboard, via text messages, a feedback box on site, and an end of year evaluation form.

7 In January, we heard in our meeting with the senior management team that the timing of the 2015/16 end of year survey meant that student feedback focused on the June 2016 summative assessments and gave little information about the rest of the programme. The School plans to change the timing of the end of year survey in order to capture student perceptions of the whole programme. We will review the process of delivering the survey and its results with the School during our next visit cycle.

Appropriate capacity for clinical supervision (R1.7)

8 The School is continuing to build on its work to ensure that staffing levels are adequate. In January, we heard that the School had recruited several more members of staff, including administrative staff, and had made joint appointments with East Lancashire Hospitals NHS Trust (ELHT). In June, we heard from the senior management team that many additional staff continue to be recruited, including the key post of Deputy Head of School. We were also pleased to hear that a business case has been submitted for additional assessment administration support. See open requirement 10.

13

9 We did hear, however, that some Year 1 students felt that there were not a sufficient number of science laboratory demonstrators to adequately run effective laboratory sessions. This led to longer, less focused sessions with less support than other learning opportunities. We will review this area during our next visit cycle to clarify whether students’ concerns have been resolved.

10 During our meeting with the module leads, we heard that foundation year two doctors will be able to sign off student Direct Observation of Procedural Skills (DOPS) whilst on placement at ELHT. These doctors have been given training by the Clinical Activity Support Team and provided with an assessor handbook which includes detailed outlines of what each DOPS should demonstrate; online resources and support will also be made available. The names of the trained foundation doctors will be held on a register so that the module leads can better monitor who is assessing students and how regularly. We will revisit this area over future visit cycles.

Appropriate level of clinical supervision (R1.8)

11 All students we spoke to felt that they had been appropriately supervised on placement. We will continue to explore this as students start their Phase 2 hospital placements.

Appropriate responsibilities for patient care (R1.9)

12 It was encouraging to hear that both Year 1 and 2 students were aware of what tasks and procedures they are permitted to perform on placement, and felt that they would be able to decline to carry out procedures and explain to supervisors what their level of competence is. This was evidenced in our June visit, when we heard that a student had been asked to perform a clinical task above their competence level, but had felt able to say that they had not learned the procedure.

Identifying learners at different stages (R1.10)

13 During our placements meetings in June, we heard that there will be different cohorts of UCLan medical students at the same placement at the same time. We understand that cohorts will have set days on clinical placement so as to make it easier for supervisors to identify students. In addition, staff at ELHT showed us the School’s uniform which will clearly identify them as medical students from UCLan.

Induction (R1.13)

14 We noted that the School has planned a thorough induction process for Phase 2. We heard that Year 2 students moving into Year 3 had already received a preliminary induction which included their Phase 2 handbooks and an introduction to Mini-Clinical Evaluation Exercises (miniCEX) and DOPS. We heard in our meetings with the senior management team that at the beginning of Year 3, there will be three sets of induction: a UCLan based induction to cover curriculum and UCLan processes; the

14

main Trust induction covering Trust processes, and various health and safety areas; and a local induction in specific Trust departments which will cover timetables, supervision arrangements and facilities.

15 The students we spoke to during our June visit told us that their placement inductions this year had helped them to feel well-prepared for their placements. Year 1 students told us that the School had provided a good level of detail in advance of their first GP placements. This included a lecture about expectations, as well as information on dress code, learning objectives and travel reimbursement. Year 2 students praised Royal Blackburn Hospital (RBH) for its comprehensive introduction to the site; however, students who had started placements at Blackpool Victoria Hospital (BVH) felt that the Trust was under prepared.

Adequate time and resources for assessment (R1.18)

16 The Objective Structured Clinical Examinations (OSCEs) took place in the School’s new clinical skills facility, which we felt gave a good representation of a hospital environment. The OSCE was also well supported by information system technicians, and School administrative staff were available to provide support to both examiners and students. See area working well 1.

17 Although we felt that the School’s new facilities were a positive development in assessing students, we noted during our OSCE observation that when students had completed the task in one station, they were clearly able to hear students and patients in adjacent stations. Students had been provided with station titles 24 hours in advance so would not have gained much advantage from this, but we were concerned that this should be addressed so that in Phase 2 (when students are not aware of station content) the School can ensure fairness. Noise levels are a common problem with OSCE assessments and difficult to manage, but the School may benefit from considering this when designing the examination to minimise noise distraction between stations as much as possible.

18 We had previously set a requirement for the School to review the marking sheet used for practical assessments, and we were pleased to see that the School now provide OSCE examiners with tablet computers. During both OSCE observations, the electronic marking system failed before and during the assessment. The School had prepared for this by providing stations with back up paper marking sheets, and told examiners that marks would be transferred at a later stage. It was encouraging to see that the examiners adapted quickly and the change of marking method did not have any impact on the students, although the School should investigate the reasons for tablet failure and take steps to avoid this in future. See open recommendation 7.

15

Capacity, resources and facilities (R1.19); Accessible technology enhanced and simulation-based learning (R1.20)

19 During the 2015/16 visit cycle, we raised concerns about the School’s available facilities as the student cohort grows. We were reassured to hear in our meetings this academic year that the School’s plans to expand the facilities for students for September 2017 are on track; these include new laboratories and a human anatomy and learning resource centre. Construction on the School’s new clinical skills’ facilities has also been completed and these are used regularly for teaching sessions and assessments; the space is also available for other healthcare students but we heard that it is used primarily by the School. In addition, we noted that further building work was underway to extend capacity by linking the two buildings currently used by the School, and were able to review plans. We will continue to monitor the School’s capacity over future visit cycles as student numbers grow. See area working well 2.

24 We reviewed resource maps in advance of our January visit, which demonstrated how the School has ensured there is adequate space for forthcoming cohorts of students. Quality management staff also told us that the School had analysed all three year cohorts’ facility and space requirements against one another for the 2017/18 academic year to ensure that there were sufficient resources and to address any overlaps or gaps.

25 In January, we heard in our meeting with the senior management team that there had been some initial problems with the new software used to upload lecture slides and recordings. All lectures should be uploaded within 48 hours of the session taking place, but the School received feedback that this was not the case. The School has now rectified the issues and will ensure that any new staff members have full training on the software facilities in order to reduce user errors. An audit has also been introduced to monitor this; the theme leads will check each week that lectures have been uploaded. We were pleased to hear from students in June that the issues had been resolved and they can now access lecture recordings promptly.

Supporting improvement (R1.22)

26 We heard in our curriculum meeting in January that quality improvement will be embedded within Student Selected Components (SSCs) in Year 3, which is likely to take the form of an audit for service improvement. There were some concerns raised regarding the viability and governance procedures for the audit process for students in secondary care, as students will choose the audit topic in the first placement, and then will have time set aside to go back into the placement to find the data required in the next rotation. We will monitor the implementation of this SSC during future visit cycles.

16

Theme 2: Education governance and leadership

Quality manage/control systems and processes (R2.1)

27 We were pleased to hear from quality management staff that the School has now implemented the new governance structure that was discussed during the June 2016 visit. In addition, the School has undertaken further streamlining activities such as amending the RRM membership; this now comprises the chairs of the subcommittees, Phase leads, the Strategic Projects Manager, the Head of School and senior administration officers.

28 The School will review the new governance structure on an ongoing basis, and has already made some changes (such as separating the assessment and evaluation sub-committees). The structure will be formally reviewed at the end of the 2016/17 academic year, but so far staff feel that there is less duplication of issues, discussions and staff than before, and that the flow between the levels of governance works well. For example, the Professor of Medical Education told us that the School of Medicine Academic Committee (MAC) will make recommendations to the sub-committees and monitor these. The governance structure will also be updated each year to capture any changes at a group level.

29 The Strategic Projects Manager told us that the programme level risk register has been mapped to Promoting Excellence. The risk register has also been updated to include both operational and strategic level concerns (including two assessment items), and these concerns are reviewed monthly. In addition, a School level risk register is now also under development; this will capture School wide, high level concerns that other Schools will also need to consider (such as regulatory compliance). The Strategic Projects Manager is responsible for monitoring the risk registers, which are reviewed monthly by the RRM, quarterly by the Vice Chancellor and separately during quarterly meetings with Liverpool. See open requirement 4.

30 We heard from the senior management team in June that there are two separate teams for Phases 1 and 2, each working solely on the development and operation of their specific Phase. This ensures that the School can plan staffing more effectively, and ring fence responsibilities so that capacity can be managed. The Phase 2 Lead

Standards S2.1 The educational governance system continuously improves the quality and outcomes of education and training by measuring performance against the standards, demonstrating accountability, and responding when standards are not being met. S2.2 The educational and clinical governance systems are integrated, allowing organisations to address concerns about patient safety, the standard of care, and the standard of education and training. S2.3 The educational governance system makes sure that education and training is fair and is based on principles of equality and diversity.

17

told us that there are monthly Phase 2 meetings with all clinical module leads and School staff.

31 In addition to the MBBS, we heard from the senior management team that the School now offers a PGDip Physician Associate Studies and BSc (Hons) Medical Sciences (Foundation Entry) across its campuses. A MSc Hospitalist Medicine and MPAS (Hons) Physicians Associate Studies will also be introduced for the 2017/18 academic year. Whilst each programme will have its own governance team, we noted that the Head of School of Medicine is ultimately responsible for these. We were concerned that the School is not yet sufficiently developed to provide adequate support (such as staffing and expertise) for multiple programmes, and this expansion could negatively impact on delivering the MBBS. We will continue to explore this area over future visit cycles.

Accountability for quality (R2.2)

32 The School continues to have a strong relationship with the central University which appears to be extremely supportive of the School’s development and expansion. We heard in our quality management meetings that the School’s needs are prioritised for timetabling.

33 To support the vertical flow of information, the Head of School sits on various Faculty level committees. We were told by quality management staff that this allows the School to share concerns and good practice.

Considering impact on learners of policies, systems, processes (R2.3)

34 Community Engagement and Service User Support (Comensus) users continue to take part in the admission process, as well as the teaching and assessment of students. Comensus users are also involved in various governance committees within the School. We also heard in our quality management meeting that students are invited to sub-committee meetings when necessary.

35 In January, we heard in our assessment meeting that student feedback is incorporated into the assessment design process and reviewed at various points. This is to be formalised as a step in the assessment design process where students can record feedback on Blackboard after sitting examinations. In addition, the Examinations Officer has regular meetings with students, some of which are initiated by the students themselves. We did not hear from any students who had fed into the design process, but all students told us that they felt able to discuss any concerns with the Examinations Officer. We will explore this step in more detail over future visit cycles.

Collecting, analysing and using data on quality and on equality and diversity (R2.5)

36 At the end of each admissions cycle, the School receives anonymised data on gender and disability from the central University. The School reviews this alongside other

18

equality and diversity (E&D) data; however, we heard from the E&D lead that the numbers are so small that it is inadvisable to draw any conclusions or use the data to make informed changes. Nevertheless, a high level E&D report is produced for the School each year.

Systems and processes to monitor quality on placements (R2.6)

37 The School’s process for using student evaluation to monitor placement quality appears comprehensive. In June, we heard in the placements meeting that information on the quality of placements is gathered through text feedback two weeks into the placement, through module and year end evaluation and via the suggestions box. Soft intelligence from Phase 2 will also be collected through the liaison officers (joint appointments between UCLan and ELHT) and Phase 2 staff.

38 The School continues to develop its partnerships with various trusts in the region in order to provide placements for students. We heard in our meetings with the senior management team that ELHT continues to be the School’s main clinical partner, and that a formal strategic alliance between the School and Trust is now in place with plans for a ten year growth period. In addition, Manchester Medical School will have withdrawn all students from the Trust by August 2017, providing further capacity for UCLan; we heard from the Director of Medical Education at ELHT that he believes this is a result of various factors such as tariff changes and the devolution of health and social care in Manchester. In January, we heard that the Trust is currently looking to offer placements to Lancaster Medical School, but are still confident that they will have capacity for the highest estimated student numbers from UCLan.

39 The School has also partnered with North West Boroughs Healthcare NHS Foundation Trust (formerly known as 5 Boroughs Partnership NHS Foundation Trust) for the 2017/18 academic year. It is the aim that the Trust will offer placements to all five cohorts but in small groups; we heard that the Trust is confident that it has sufficient capacity.

40 In January, we followed up on our concerns that were raised during our 2016 visits regarding the Bridgewater Community Healthcare NHS Foundation Trust (Bridgewater) placements. The School had met with the Trust to reiterate their expectations and to deliver additional training for their supervisors. We were told by the senior management team that Bridgewater does not have the capacity for all UCLan students undertaking their community placements. As such, some Year 1 and all Year 2 students will undertake these placements at BVH; BVH will also act as a contingency for Bridgewater should poor feedback continue. At present, only some students have rotated through BVH and there is a smaller quantity of student evaluation, so the School will monitor this area closely. The School will visit the trusts on a regular basis and monitor feedback in order to ensure an equitable and high standard of experience. In June, it was noted that Bridgewater has reached capacity for student community placements, and we will monitor this area further as the School moves into Phase 2.

19

41 The School has plans for a number of students to rotate through North Cumbria University Hospitals NHS Trust in the 2018 academic year; these students will be placed on site for the full year. Health Education England working across the North East (HEE NE) has expressed a wish to establish a formal relationship with the School in order to better monitor any placements at the Trust, and we look forward to exploring these plans over future visit cycles.

42 In January, we heard from the Head of Workplace Learning that there were no concerns regarding GP placement capacity for the two cohorts this academic year. The School has rewritten the GP service level agreement (SLA) and is in the process of reissuing these, but noted that all practices have SLAs.

43 Due to the wide geographical footprint, the School made us aware that it was becoming more complicated to monitor student allocation for GP placements. For example, some Year 2 students reported being placed in the same GP practice as in Year 1. As such, staff involved with managing placements told us that they are hoping to replicate the system currently used by the School of Nursing to assist with and monitor placement allocation. We look forward to a more formalized procedure of managing placements as the student cohorts grow.

Sharing and reporting information about quality of education and training (R2.8)

44 We felt that the School communicates proactively and positively with its local stakeholders across the wider community. The School has regular contact with key groups such as Health Education England working across the North West and other medical schools within the region. The School also continues to meet with Liverpool Medical School on a quarterly basis. In our meeting with the senior management team in January, we heard about plans for a conference for local education providers; this will help facilitate information sharing and develop collaborative resolutions to concerns.

Collecting, managing and sharing data with the GMC (R2.9)

45 Both before and during our visit in January, we were alerted to some significant student concerns which included issues with accommodation and changes to assessments. The senior management team did not raise these with us prior to our discussions with the students; however, when we highlighted these topics, we were advised that the School was aware of student concerns surrounding accommodation options for Phase 2 and we were able to explore these issues. Despite this, we felt that the School was then reticent to discuss the amendments they had made to the assessment load with us; we would have instead expected the School to have advised the GMC team of such concerns in a transparent way and share all relevant information.

20

Monitoring resources including teaching time in job plans (R2.10)

46 We were pleased to hear from quality management staff that there is now an active Planning and Strategy Group. This has been established to monitor resources and facility expansion, with frequent meetings between the School and the University taking place. The School also have a new resource co-ordinator in post that will be responsible for mapping resources on a School-wide level.

Managing progression with external input (R2.12)

47 During this visit cycle, we explored the School’s progression rules for those students who did not pass their resits. We understand that there is an option to re-sit the year or continue to a Bachelor of Medical Sciences degree, but we were told that this is at the progression board’s discretion. The School recognises the need to formalise this process, and plans to have this approved by the University Board at the earliest opportunity. We look forward to exploring the changes in the next visit cycle. See recommendation 1.

48 We observed the School’s Programme Board meeting in June, and were satisfied that this was conducted in an appropriate manner. A report of the summative assessments was presented to relevant staff members, with a considerable amount of detail included. All student information was recorded in an anonymous manner with the exception of any awards, which are published to all students. The School also clearly showed where students had been unsuccessful in passing assessments, and had recorded any extenuating circumstances.

49 The Head of School confirmed that permissive assessments, and therefore permissive progression, have now been removed from the programme. Students are allowed multiple attempts to satisfactorily complete their e-portfolio, but ultimately must pass before they can progress to the next academic year.

Managing concerns about a learner (R2.16)

50 We had previously set a requirement for the School to improve its guidance for staff and students to raise awareness of the reporting process. In our student support meeting in January, we heard that the School recognised that there were some negative connotations with SERF and is therefore now using the term ‘the reporting process’. The website and related guidance have been updated with the new terminology, and the School ran a session with the Year 1 students in order to address any negative preconceptions. The students themselves said that they had not completed any reporting forms, although they were aware that they were able to do so. See open requirement 7.

51 We did hear in January that the Year 2 students felt there was still considerable inconsistency between tutors regarding their reasons for completing the reporting forms. An example given was that a form was completed because a student was eating, but not when another student was chewing gum. Students also felt that tutors

21

should speak to them in advance of completing the reporting forms to explain why their behaviour was not appropriate. Nevertheless, students were clear on how to complete a form and were aware of the relevant processes. Additionally, in June we did not hear any further concerns from students regarding the reporting process.

52 We heard from staff involved with student support that the School has developed a spreadsheet in order to track and monitor the reporting forms, and ensure that all necessary actions are taken; raising a confidential report form is still an option. As of 2 May 2017, 76 reporting forms had been completed for Year 2 students (28 of which were positive and 38 were Level 1 professionalism) and 71 for Year 1 students (44 were positive and 21 were Level 1 professionalism). We will continue to monitor this area over future visit cycles.

53 We had also set a requirement for the School to strengthen their assessment of professionalism. Throughout this visit cycle, we were pleased to hear of the School’s plans to meet this requirement. Students will be judged on a range of equally weighted objective and subjective measures, including a portfolio professionalism scoring, multisource feedback and an objective structured clinical examination (OSCE) professionalism rating. The School will permit students to fail one area and progress, but any additional fails will require students to complete a self-awareness review (in Years 1 and 2) or repeat the failed elements (in Years 3 onwards). These measures will be piloted during the upcoming 2017/18 academic year, and we look forward to exploring this further. See open requirement 8.

54 We were pleased to hear from the Phase 2 Lead that the School has successfully appointed a Professionalism, Welfare and Safeguarding (PWS) tutor, who will be responsible for the welfare of Phase 2 students and act as a central triage point for all pastoral and disciplinary issues. The School hopes to introduce a level of continuity and a central process which should allow for collection of soft intelligence and prevent a repeated pattern of low level issues. The PWS Tutor will have access to the reporting form spreadsheet to better monitor and act on such low level concerns.

Sharing information of learners between organisations (R2.17)

55 The School continues to communicate with placement providers about any necessary information sharing on a personal, ad hoc basis, or through the e-portfolio. This is a result of the small number of students, and we heard from staff that they felt this process to be working effectively. Moving forward, the PWS tutor will work with the School, trusts and students to ensure Phase 2 handovers take place and information is up to date. The School will evaluate this model of using the PWS tutor to manage the transfer of information once students are on placement, but told us that it had been effective for other universities.

56 We also heard from staff involved with student support that the School has an Exposure Prone Procedures policy in place, and that the School shares occupational health clearance with placement providers. Should occupational health clearance not

22

be granted, then a separate conversation between the placement provider and the School will take place.

Requirements for provisional/full registration with the GMC (R2.18)

57 We were pleased to hear that the School has continued to host a regional Fitness to Practise panel with all regional medical schools (the latest was held in June 2017), where local medical schools take part in a joint panel threshold normalisation session. The Head of Welfare, Professionalism, Transition and Careers told us that UCLan’s processes appear to be well-aligned with other schools, and had received praise for its multi-professional perspective. As a result of the session in June 2016, we understand the Head of School will sit on the Manchester Fitness to Practise panel and a member of Manchester Medical School staff will provide training for the School. We look forward to hearing how these panels during future visits. See area working well 4.

58 We previously set a requirement for the School to ensure that appropriate consideration is given to the security of students’ information, especially when students move from support to disciplinary or Fitness to Practise proceedings. In addition to more general steps taken to protect student anonymity, the School’s guidance clearly shows that no staff member with prior knowledge or outside interest of an individual student can sit on their Fitness to Practise panel. See open requirement 3.

59 Staff involved with student support told us that the School had reviewed their Fitness to Practise policies in line with the new GMC guidance. At present, no Fitness to Practise cases have occurred, but cases in other Schools (which use similar policies) have been successfully upheld. The School is looking to train staff from across the Faculty in their processes, and are exploring sitting on panels in other schools, such as nursing, in order to build experience.

60 We heard from Year 1 students that they felt they had a good understanding of Fitness to Practise processes, and that they had received a talk from GMC staff which outlined the main concepts as well as seeing the latest GMC guidance. In addition, they had had sessions about acting professionally and what might comprise good or inappropriate behaviour.

Compliance with legislation (R2.19)

61 It is our understanding that the MAC will ensure that education and clinical training adheres to advice from the Department of Health, the General Medical Council and complies with all relevant legislation. Responsibility for meeting the Promoting Excellence standards is delegated from MAC to the various Heads of Phase and Leads within the School who have individual accountability for standards in their area.

23

Recruitment, selection and appointment of learners and educators (R2.20)

62 The School continues to refine its admissions process. In January, we heard in our quality management meeting that all applicants must achieve a minimum score of two in the transferrable skills section in order to progress to the multiple mini interview (MMI) stage. Most applicants are meeting this; the School told us that this prevents applicants from focussing solely on their personal statement. We heard from the Academic Lead for Admissions that the new number of MMI stations also seems to be working well. The School collects feedback from both applicants and assessors, which it will review at the end of this cycle and make further amendments to the process if necessary. The School also continues to gather data about entry routes for the MBBS.

63 We heard during the current visit cycle that the School is satisfied that it is working towards the admissions target for 2017/18 academic year, and have planned resources accordingly with clear contingency plans. Applicants’ UCAS data are monitored are monitored weekly so that the School has a better idea of conversion and capacity.

64 Quality management staff told us that the content of MMI stations changes each year, but the theme is consistent and there is a blueprint of each round to ensure that the MMI is mapped to Good Medical Practice. We also heard that two MMI stations are run by Comensus staff that are fully trained for their role; the Comensus group is also involved in the shortlisting process and review of personal statements. The School ensures that all staff receive training for the MMI and admissions processes every three years.

65 We heard from the University’s senior management team about their plans to develop their widening participation programme with additional outreach work within the local community. Primarily, this will take the form of a widening participation scholarship fully funded by the University and partner trusts to sponsor two local students through the programme; this includes the payment of tuition fees, the provision of placements and a maintenance grant equal to the loan which would have been provided by Student Finance England. During our June visit we heard that one has been recruited for the upcoming academic year. There are also plans for ten NHS commissioned MBBS places for regional students in the 2017 entry, which we will explore over the next visit cycle. Here, the University will apply a discount to the fees with UK tuition fees paid for by the students and placement fees by the NHS. We look forward to seeing how these plans develop, and how the School proposes to manage two cohorts of students with differing needs. See area working well 3.

24

Theme 3: Supporting learners

Standard S3.1 Learners receive educational and pastoral support to be able to demonstrate what is expected in Good medical practice and achieve the learning outcomes required by their curriculum.

Good Medical Practice and ethical concerns (R3.1)

66 Students told us that they continue to receive discussion-based lectures and small group sessions which focus on professionalism, and that they are able to reflect on this through their portfolios. We heard from curriculum staff that professionalism teaching has been reviewed so that it is an integral thread running throughout the programme. In addition, we heard that in Phase 2, case based learning professionalism will spiral around a practical case (such as clinical or legal) where patient safety is a core theme.

Learner's health and wellbeing; educational and pastoral support (R3.2); Access to educational supervision (R1.21); Support for learners in difficulties (R3.14)

67 Student support continues to be praised by students, in particular the Academic Advisors. We heard that students are always able to book appointments to see their Advisor, including ad hoc meetings for issues such as assessment feedback. For each meeting, students fill in a form that is sent to the Advisor in advance of the meeting in order to shape the discussion. This form is completed during the session and uploaded to the student’s e-portfolio, with any agreed actions followed up at the next meeting. Students stated that they could also discuss other issues in their meetings alongside this template. Quality management staff told us that the number of Academic Advisors has been increased to better manage the second cohort of students. All Academic Advisors must be available for two hours per week for students to book additional meetings, and the School has introduced a new booking system, ‘Starfish’, to make booking these meetings easier.

68 During our visits we noted a high level of educational support for the students such as group and one-to-one revision sessions. We heard in our assessment meeting that the School plans to provide the same level of support as the 2015/16 academic year for any student that has failed summative assessment. It is a University policy to provide this remediation, but remains optional for students.

69 Students told us that they are aware of how they could access support services at the central University; Year 1 students told us that they had received talks from the Pastoral Tutor and Student Union. In our student support meeting in June we heard that the University provides support for students involved with Fitness to Study proceedings, and that additional support is offered to students after the summative assessments in May.

25

70 The School continues to reimburse student travel costs to placement in Phase 1. Usually, students claim these costs back after purchasing tickets; students told us that they had been given reimbursement forms in their placement packs. However, the Placements Officer told us that the School buys some train tickets in advance for those students who cannot afford the upfront cost of travel.

71 In advance of our visit in January, we were made aware of concerns amongst the Year 2 students regarding their accommodation arrangements for the upcoming academic year. Following this up on the visit, we heard that the School had advised them to find accommodation in Blackburn or Burnley (where their clinical placements will take place in Years 3-5), a fact that was a surprise to many students. In addition, we heard that travel to placements will not be funded from Preston, although it will be funded from the East Lancs Hub and there is a free shuttle bus between the two sites. Students were concerned about the suitability of accommodation, especially the possibility of mixed sex bathrooms and lack of 24 hour security.

72 We followed this up with students at our visit in June, when we heard that students had received additional guidance but were still concerned about the suitability and availability of accommodation. Some students have opted to stay in hospital accommodation, but others did not feel comfortable staying in mixed-sex flats. Additional students told us that it was difficult to find affordable houses in Blackburn that met student expectations, despite having been sent a list of property agents in the area. The Head of Student Wellbeing told us that the Student Union would be able to offer support, and we also heard that the School had asked all students whether they had found suitable accommodation. We will review this area further over future visit cycles.

73 The students we spoke to did seem to understand their future pathways, but did have questions around their particular circumstances as international students. We understand that HEE NE will be visiting the school in the 2017/18 academic year to offer a careers’ presentation.

Undermining and bullying (R3.3)

74 We were pleased to hear that no students or staff members had experienced bullying or undermining behaviour. We will continue to review this area during future visit cycles, especially as students move into Phase 2.

Information on reasonable adjustments (R3.4)

75 It was encouraging to hear from students that the School makes it very clear that declaring a disability will not affect their progression, and that they feel supported to make a declaration. Some students also mentioned that they know how to apply for reasonable adjustments, and had received guidance through several avenues on how to do so. We heard from staff involved with student support that the University has

26

now formalised an ‘interruption to study’ policy, which usually applies for periods away from study of over a month.

Information about curriculum, assessment and clinical placements (R3.7)

76 Throughout the academic year we were concerned to hear of various issues that did not appear to be well communicated to students. For example, Year 2 students thought that they would be able to sit the United States Medical Licensing Examination (USMLE) before graduating, but have now been informed that this is not possible. UCLan must be listed on the World Directory of Medical Schools in order for students to take the assessment. This cannot take place until the School has gone through the multiyear quality assurance review for new medical schools, which follows the first cohort of students to the point of graduation. At this point, if successful, the institution will be added to the GMC list of bodies entitled to award a UK medical degree.

77 Students’ concerns regarding their accommodation in Phase 2 appeared to stem from a lack of clear guidance and information. Some students noted that the different placement sites were clearly stated on the UCAS website, but many were not aware and felt that UCLan had not reiterated this at the application or offer stage. We heard from the School that information is also stated on the School website, but that more work could be done to improve student understanding and manage expectations. See open recommendation 2.

78 We also found dissatisfaction amongst students about the guidance provided about SSCs. Students reported that the SSCs were advisor dependent, as different tutors provided conflicting guidance on how to complete them. We also heard that there was limited information at induction or in the course handbooks, which often provided conflicting advice given by supervisors. The Year 1 students told us that they are able to choose a topic of their choice, but felt that they had to select something in line with their advisor’s interest or they wouldn’t receive sufficient support or expertise. See requirement 1.

79 In addition, some students appeared dissatisfied about the number and quality of practice questions they had been given. We heard that they had requested practice questions in order to understand what was required of them in the assessments, but had only received a small number; often these were not sent in a timely manner. In addition, it was felt that those questions given did not reflect the summative assessments in difficulty or core knowledge.

80 During our visit in January, students commented that conflicting timetables on various platforms such as Blackboard and noticeboards existed, which meant that students were at times unclear when key lectures were taking place, or what materials they needed to prepare. Lectures could also change at late notice or the titles were sometimes missing from timetables altogether, again leaving students

27

feeling underprepared. However, we followed this up at our visit in June and were pleased to hear that this had been resolved.

81 Despite these areas of concerns, we noted some improvements from previous visit cycles. For example, students felt that their year handbooks were helpful and addressed key points. We also heard that learning outcomes for their placements were sent in a timely manner and were more focused than in the previous year. Students also told us that their pharmacology lecturers were particularly adept at highlighting the relevant learning objectives, and expressed a wish that this good practice could be followed by other lecturers.

82 We heard from curriculum staff that the School provides students with details of the various themes as well as specific information about the curriculum. The Year 1 and 2 placement handbooks have been reviewed, and some information regarding learning objectives have been revised. This will be reviewed again at the end of Phase 1, and we will review future handbooks over upcoming visit cycles.

83 A requirement had also previously been set for the School to ensure that the thresholds for how students enter the Fitness to Practise processes are clear, particularly how students move from support to sanction. Whilst the School has updated its guidance for staff, many students continued to appear unsure what the difference between Fitness to Study and Fitness to Practise would constitute. However, in January we heard from the Lead for E&D that a considerable amount of work had been undertaken by the School to improve student awareness, such as improving guidance and delivering sessions. The Head of Student Wellbeing also noted that the guidelines for each process are saved separately on Blackboard. See open requirement 5.

84 We had previously set a requirement for the School to improve student guidance on the educational performance measure (EPM). We heard from the Head of the School that the EPM is calculated as a mean of all assessments in Years 2-4. The School has reviewed other medical schools’ EPM calculations, and decided that using a mean allows the measurement to sample as much of the students’ work as possible. See open requirement 6.

85 During our OSCE observations, we noted that student briefings take place immediately before each assessment circuit. We observed two Year 2 briefings, and were pleased to see that the messages delivered were consistent and that the same presentation was shown. Students appeared to already be aware of the information provided, with the briefing serving as a final reminder of key points. In addition to information about the purpose of the OSCE and practical points of consideration, students were also reminded to report any illness during the assessment so that support could be provided. During the OSCE, students appeared clear on how they would rotate around stations. Two ‘room controllers’ were available during each circuit if students required assistance or were unsure which station was next.

28

Feedback on performance, development and progress (R3.13)

86 We heard from students that the School provides generic feedback to students such as average, rank, and score breakdown for various themes; students were also given RAG ratings before the Christmas break followed by more detailed feedback on their return. If students booked appointments with their Academic Advisors, they received more detailed feedback which identified any points for development; we heard that training is provided to Academic Advisors to ensure that this feedback is given in a standardised manner. Some students told us that they would have preferred more detailed written feedback, whilst others noted that they would like to review their exam papers with the marks. However, students also reflected that they received sufficient guidance from their meetings with the Academic Advisors.

Meeting the required learning outcomes (R3.15)

87 In January, it was noted that there was a small number of students who had ‘failed’ one or more of their Year 2 formative assessments. Within this group were students who had also failed their summative assessments in May 2016 but had all gone on to pass the subsequent resit examinations. The Examinations Officer confirmed during our June visit that this pattern had been repeated, and some students who had been required to resit in August 2016 would again need to retake certain assessments in August 2017. We were concerned that the remediation put in place for their Year 1 assessments had not adequately identified learning needs to ensure that students were able to progress in future stages of the programme. We anticipate that the School will monitor their progression closely and will revisit this at our visit cycles next year

Career support and advice (R3.16)

88 During our exam board observation, we heard that the School offers a range of exit awards depending on when the student exited the programme. For example, students exiting after the end of Year 2 could be offered the opportunity to transfer to BMedSci (with agreement from the Programme Board). Alternatively, students could leave the University with a Certificate of Higher Education or Diploma in Medical Studies depending on the credits obtained. We also heard that if a student whose progress, performance, health or conduct means they cannot continue on the course, there are no visa implications for them to receive exit qualifications or change courses.

29

Theme 4: Supporting Educators

Standards S4.1 Educators are selected, inducted, trained and appraised to reflect their education and training responsibilities.

S4.2 Educators receive the support, resources and time to meet their education and training responsibilities.

Induction, training, appraisal for educators (R4.1)

89 Training for School and Trust staff appears to be comprehensive. In the first instance, the School reviews what qualifications individual staff already hold to avoid duplication; the School then provides UCLan specific training such as the School ethos, curriculum and expectations in addition to training on assessments. The School also offers staff the opportunity to attend the International Advance Assessment course and various external conferences. In January, we heard in our quality management meeting that staff shadowing and peer observation is ongoing, and the Head of School told us that an annual conference for Trust staff will be provided which will act as refresher training. It is recognised that service provision may prevent some staff from attending these, so the School will provide training sessions at ELHT to ensure attendance. See open requirement 10 and area working well 5.

90 The Strategic Projects Manager also told us that there is a staff development group which is held twice each semester. This allows a shared discussion about individual and group training needs which staff can take forward. In addition, there are bi-monthly staff briefings to update staff on new staff and educational developments. The School continues to monitor staff through a range of performance indicators, such as student feedback and appraisal.

91 All module leads that we spoke to told us that they felt prepared for the arrival of students, and that they had received UCLan specific training. Those running the campus day will also receive additional training on areas such as teaching principles and the use of technology.

Time in job plans (R4.2)

92 We heard that the module leads feel that the Trust is very supportive in terms of providing adequate time for undergraduate education. The Director of Medical Education at ELHT told us that a paper on job planning had been agreed by the Trust Education Board. Some job plans have already been fully calculated, whilst for others, it was yet to be decided on whether they will be calculated weekly or annually.

30

Working with other educators (R4.5)

93 We heard in our placement meetings that the module leads based at ELHT regularly meet together as a team and share good practice and teaching sessions with primary care colleagues. We will review this area with primary care supervisors over future visit cycles.

Recognition of approval of educators (R4.6)

94 We heard in our meeting with the module leads that are all recognised undergraduate trainers. We will continue to review this area over future visit cycles as the supervisor body grows.

31

Theme 5: Developing and implementing curricula and assessments

GMC outcomes for graduates (R5.1)

95 We previously set a requirement for the School to ensure that the MBBS’s curriculum and learning outcomes are fully aligned with that of Liverpool Medical School. The senior management team told us in January that this has now been completed for Phase 2 of the programme; whilst we have now received documentation on this subject, we will explore mapping in more detail over the future visit cycle. In addition, we heard that the practical procedures included within Year 5 are aligned to the Foundation Programme. See open requirements 1 and 2.

96 We heard from curriculum staff that the Integrated Science and Clinical Medicine theme learning objectives have been reviewed over the academic year, with some moved between Years 1 and 2; the School hopes that this will help build the curriculum into Year 3. The School has used student feedback to help review the structure of the curriculum and identify where concepts need to be redistributed and developed.

Informing curricular development (R5.2)

97 We were pleased to hear in our meetings with placement staff that the Phase 2 curriculum has been developed with the clinical module leads based at ELHT. The enthusiasm of the module leads was evident; these individuals will have a key role in delivering the curriculum on placements, and will thus have a good understanding of the learning outcomes and expectations. See area working well 2.

Undergraduate curricular design (R5.3)

98 We heard in our January curriculum meeting that there will be an introductory week in advance of placements in Phase 2. Within each week there will be a ‘science day’ where students will apply the basic science concepts learned in Years 1 and 2 to clinical cases. In addition, case based learning sessions will encourage students to explain presentations on a scientific basis.

99 The plans we heard about for the Health and Society module, specifically the ‘third sector’ placements, are promising. These placements will focus on how various societal factors such as disability, poverty, and unemployment can underpin and affect access to healthcare. In June, we heard that the School has now identified the

Standard S5.1 Medical school curricula and assessments are developed and implemented so that medical students are able to achieve the learning outcomes required by graduates.

S5.2 Postgraduate curricula and assessments are implemented so that doctors in training are able to demonstrate what is expected in Good medical practice and to achieve the learning outcomes required by their curriculum.

32

providers for these placements and developed relevant SLAs, although work still needs to be completed to refine how students will meet their learning objectives. We look forward to exploring these placements in more detail over future visit cycles. See area working well 6.

100 We previously set a recommendation for the School to take steps to standardise the length and breadth of lectures. During our visits in the 2016/17 academic year, some subjects and lecturers (such as anatomy, physiology and public health) were commended by the students for explaining the topics well and therefore being easier to learn from. There were, however, also reports throughout our visit cycle of overrunning lectures which did not cover the required content in the timeframe. Students also reported variability in lectures, including issues surrounding lecture length and depth, and how well the lecturer identifies the core concepts. We heard from the School in our curriculum meeting in January that it is reluctant to distinguish between core and supplementary knowledge as they believe this leads to students focusing their learning too heavily on exam preparation. See open recommendation 3 and open requirement 9.

101 In addition to poor and conflicting guidance, students told us throughout our visits that they were dissatisfied with the options available for their SSC topics. We heard from Year 2 students that there were discrepancies between the choices open to students who were placed in GP practices on placements and those in secondary care, with more choice for students on placement in primary care. Year 1 students had also been allocated topics they were not interested in. We were told that this had been raised with the SSC lead, but that no changes were made. In our Quality Management meeting in June, we heard that the School is aware of students’ dissatisfaction, and we welcome the review of the SSCs over the summer. See requirement 1.

Undergraduate programmes and clinical placements (R5.4)

102 The School provides students with a comprehensive and appropriate educational induction. We heard that the School had introduced a two-day team building trip at the beginning of the academic year in Wales for Year 1 students and staff. Students were enthusiastic about their experience, noting that it was a positive introduction to the School and their peers. These students felt that the remainder of their induction was also helpful and comprehensive, with a number of workshops and tours of the area. In addition, a wide range of information was provided to students in advance of them starting at the School. The Year 2 students we spoke to were also satisfied that the induction they received at the beginning of the academic year was helpful with comprehensive information about how the year was structured. Students also had a session to reiterate key policies and highlight any changes.

103 We heard from students in June that most placements were very good and enabled the practical application of clinical skills learned in School. However, we did hear that some Year 1 community placements, in particular those based at Bridgewater, were

33

not prepared for the arrival of students and had not organised appropriate activities. We also heard from students in June that there are perceived inequalities between the GP placements: some students were felt to have significantly more opportunities than others on placement, such as more time spent talking with patients or increased responsibilities to practice procedures. Students were concerned that this disparity of experience would affect their ability to meet their learning outcomes. See open recommendation 4.

104 We heard in our meeting with curriculum staff that, during Phase 2, students will have two days of ringfenced clinical placements and one day in a GP practice; in Year 3 these clinical placements are split between four eight-week specialty blocks. The other two days will be split between SSC work, campus based teaching (including case based learning) and self-study. In June, placement staff told us that in Years 2, 3 and 4, students will have specific rotations on set days of the week, as opposed to two day blocks. The module leads told us that there are a large number of consultants for students to work with, and a huge breadth of work covered, so they did not feel that this placement structure would mean that students could miss a clinical procedure or teaching that may only happen on a set day of the week. We did, however, hear from some students in Year 2 who felt that attending a GP practice placement as a block, rather than one set day a week, was preferable as they felt they had been exposed to more learning experiences. We will revisit this area on future visit cycles. See area working well 2.

105 Staff involved with placement management told us that the School provides the Trusts with the learning outcomes but allows them to choose activities for students to meet these. There has been some degree of mapping to these learning outcomes, however, so that the School can have some assurance that they will be met. In addition, the School delivers training to the Trust in a standard way in order to promote parity of experience, and will closely monitor feedback from students.

106 We were concerned to hear that students in both Years 1 and 2 did not feel that their interprofessional learning sessions with other health care students were of particular value. It was acknowledged that this was in part due to a lack of participation from other groups – at times other groups of students failed to attend these sessions entirely – which left the medical students to complete the majority of the work. Year 1 students also felt that these sessions would be better structured as a case study, as often the sessions focused on professionalism rather than sharing and discussing the roles and views of the other professions. Curriculum staff told us in June that the interprofessional learning programme will be reviewed to improve student experience, and we will assess the results of this over future visit cycles. See requirement 2.

Assessing GMC outcomes for graduates (R5.5); Fair, reliable and valid assessments (R5.6)

107 As discussed under R3.15, we heard at our wrap up meeting in September 2016 that all students had successfully progressed to Year 2 despite the high failure rate after the first summative assessments earlier in the summer. As a result, we requested

34

documentation which set out the results and remediation available to students. We noted that the School had used slightly more than 20% of original questions in the resit assessments, and looked to have standard set appropriately. During our visit in June, assessment staff told us that they have revisited this area, and plan to use approximately 50% of original questions for the 2017 resit examinations.

108 We had previously been concerned that the School’s written assessments were not of an adequate quality. During our visit in January, we reviewed draft summative multiple choice question (MCQ) assessments, and were concerned that they had not sufficiently improved. In particular, the stems included irrelevant and potentially misleading information, and further work to refine the questions was deemed to be necessary. It was encouraging to see in June, however, that improvements had been made to the MCQ assessments by refining the question stems, and we look forward to seeing the School’s work to continue with these developments.

109 During our OSCE observation, we were pleased to see that each station task typically filled the allocated time, with no observed instances of students unable to complete the examination within time, although there were some instances of students completing the task very early. In addition, we felt that the stations tested a broad range of knowledge and skills, allowing students to demonstrate their ability to integrate basic and clinical science.

110 Assessment staff told us that the School’s strategy is to design the assessments to discriminate between students, with every question having a clinical action attached to it. All theme leads and lecturers contribute to the question bank with workshops held to help develop and draft assessment items. The School hopes that the assessment design process should become easier as the item bank grows and staff become more experienced. As an additional check each paper is also reviewed by external examiners.

Fair, reliable and valid assessments (R5.6)

111 As a result of our concerns in January that the quality and management of the School’s assessments had not adequately improved since our visit in June 2016, we formally wrote to the School to set interim requirements to be met before our visit in June. Firstly, we asked the School to incorporate questions from their contingency School (Liverpool Medical School) in the summative assessments in order to better identify any areas of concern. The requirement specified that questions from Liverpool’s question bank must make up 10% of each MCQ summative assessment, and the School should standard set these independently.

112 We heard during our examination board observation and assessment meeting in June that the School had incorporated a number of Liverpool questions so that they constituted 10% of the MCQ assessments. The School told us that UCLan students were approximately 10% less successful than Liverpool students when answering these questions, but that this difference is a result of curriculum alignment and subtle

35

differences in the style of the wording of questions between the two schools. The School standard set the summative assessments independently in the first instance, and also performed a standard setting exercise with Liverpool following the visit in January 2017, in which it was noted that the School set stricter standards than Liverpool. This experience was felt to be positive, and the School hopes the close collaboration will be maintained. We felt that this collaboration would be beneficial to the development of the School’s assessment design and delivery, and should continue. See open requirement 11.

113 Secondly, we asked the School to initiate an independent review of its assessments, including the standard setting process and item content. During our examination board observation and meeting with assessment staff in June we were pleased to hear that the School has approached a number of appropriate international external examiners. Some had yet to visit and provide feedback, but we were able to review the reports of those examiners who had already submitted their comments. These reports, alongside the independent review, had identified a number of positive aspects of the School’s assessments, as well as recommending areas for improvement. Many of these recommendations will be adopted by the School over the next diet of assessments, and we look forward to seeing how these improvements help develop the School’s assessment processes. See open requirement 12.

114 Despite the positive steps taken by the School to refine and improve the design and implementation of the assessments, we were concerned to learn that a high number of students did not successfully pass their summative assessments in May 2017 (over 20% in each cohort). We heard that amongst this group were students who had been required to resit in 2016; there were also an additional number of students who were yet to satisfactorily meet the requirements of their portfolio. As our review of the School’s MCQ assessment items showed that there had been improvements, we feel that the School must continue to work with Liverpool Medical School to conduct a wider review of its curriculum content and teaching to identify reasons for the continued high failure rate. See requirement 3.

115 At our visit in June, we discussed how the School had analysed the marking of the recent summer exams. The School utilises Maxinity software to analyse the performance of each item and the assessment as a whole. Whilst we commend the use of Maxinity for routine analysis of item performance, we encourage the School to invest in building the expertise of the assessment team to undertake independent analysis of the performance of each assessment and the cohort of students.

116 We felt that both Year 1 and Year 2 OSCEs were well organised by the School with a strict adherence to the agenda. Staff had a clear understanding of how the assessment would be run and their role within this. All stations were set up appropriately with the required equipment and station information. In addition, patients adhered to the script and station outline, and despite the physical differences

36

that are discussed below, there was clear consistency between the two patients playing each role in the two circuits that are run simultaneously.

117 Both Year 1 and 2 OSCEs took place over two days, with students rotating through six stations each day. The stations were ten minutes in length with a prior two minute reading period. The number of circuits run per day depends on the number of students in each cohort: there were four circuits for the Year 1 OSCE and three for Year 2. We had previously set a recommendation for the School to review its policy for releasing station titles 24 hours in advance of the assessment. During our visits, we heard from assessment staff that the School’s increase in space had reduced the need for quarantining students, and the practice of releasing titles would cease from Phase 2 (although would continue for Phase 1 students). The students we spoke to were aware of these changes, and told us that there would be a formative OSCE in January 2018 to allow them to practice under the new conditions. See open recommendation 5.

118 Whilst we identified many positive aspects of the OSCEs, students reiterated the concerns they raised with us in 2016 about the variation between stations due to differences between patients. We heard that some patients took a long time to remove clothing for a procedure whilst others were better prepared, which students felt was unfair. Other concerns included how hairy patients were or whether they were overweight or not. See open recommendation 6.

Mapping assessments against curricula (R5.7)

119 In January, students told us that the School would be removing the short answer question (SAQ) examination from the May 2017 summative assessments due to the poor performance during the formatives. We heard from the Year 2 students that they had been told by the School that there were a high number of failures, and that the examination covered content that had not been taught and questions where the language was confusing, which prevented them from demonstrating what they knew. As a result, we heard that there was inconsistency between students’ MCQ and SAQ scores.

120 In June, the Head of School told us that there were no current plans for future SAQ assessments, and students appeared to be aware of this decision. Instead, all students will undertake two MCQ papers of 80 questions each. The School believes that the MCQ examination will test the same knowledge as the SAQ assessment would have done, and will allow the School to present more questions to the students. The School also believes that MCQ assessments are easier to map to the curriculum.

121 Students also told us in January that the laboratory report had also been removed from the students’ assessment load. Some students raised concerns that there was now very little in the way of coursework, leading to a reliance on high stakes

37

examination which may not fully test the range of students’ skills. However, no further concerns were raised during our meetings in June.

122 We had previously set a recommendation for the School to ensure that the scope and format of the formative and summative assessments were close enough to improve student preparedness. The Year 1 students we spoke to found the formative assessments in December helpful in preparing them for their summative assessments later in the year, with useful feedback. They were confident that the formative examinations were a good indicator of future assessments; these students also felt that the assessments tested what they had been taught. See open recommendation 1.

123 Despite these improvements, some students reported poor alignment between their teaching and the assessment content. We heard that questions on specific illnesses were included in the MCQ examinations despite there being little time spent in lectures or teaching discussing these. These concerns may be in part caused by students’ concerns regarding the inconsistency between lecture content and depth as discussed in R5.3, and should be considered with the School’s wider review of the curriculum content, and teaching and blueprinting methods. See open requirement 9 and requirement 3.

Examiners and assessors (R5.8)

124 As part of our letter to the School in February 2017, we asked the School to provide a written response regarding their in-house assessment expertise. We received the School’s response in March 2017, which noted that additional assessment training (both internal and external courses) had been organised for all staff involved with assessing students. As discussed under R1.7, a business case has been submitted in order to increase the School’s assessment administration staffing, and these needs are now considered during meetings of the Planning and Strategy Group. See open recommendation 8.

125 We observed the examiner briefing in the morning of the Year 1 and Year 2 OSCEs. This appeared to serve as a reminder of key points included in the initial examiner training which had been circulated in advance of the assessment. We were pleased to see that the presentation included key information such as how examiners should complete the marking sheet, and basic ‘do’s and don’ts’. The School had also highlighted in the morning briefing prior to the Year 2 OSCE that patients were not to deviate from the character description or communicate with students outside the parameters of stations, and we saw no examples of this taking place. However, we noted that only the Year 1 examiner briefing included reminders about how inappropriate behaviour should be managed (although in the Year 2 briefing examiners were told how they could record instances of inappropriate practice).

126 Despite this preparation, during the Year 2 OSCE some volunteer patients did not seem to have been fully briefed about what the examinations entailed. We noted that

38

some seemed surprised and at times uncomfortable when asked to undress for examinations, and some did not feel able to continue after the first circuit; this could potentially have contributed to students’ comments about consistency. We heard from a Comensus user in June that, whilst the School provided a good level of detail for the station each patient was allocated to, during the OSCE there had been instances where they had to change to a different station and were thus less prepared. The Academic Lead for Clinical Skills told us that in future, volunteer patients would now receive information about all OSCE stations. We will monitor this measure over future visit cycles.

Reasonable adjustments in the assessment and delivery of curricula (R5.12)

127 During our OSCE observations we heard that no additional time in OSCE stations is awarded for students with reasonable adjustments. The OSCEs we observed comprised practical, simulated clinical encounters and did not require students to perform any reading or writing tasks (other than reading the brief station instructions) where students with conditions such as dyslexia may have been disadvantaged. As such, we were satisfied with the policy on reasonable adjustments for the OSCE.

39

Team leaders Professor Gill Doody

Professor Judy McKimm

Visitors

Professor Suzanne Chamberlain

Dr Tom Foley

Dr Russell Peek

Mr Thomas Shanahan

GMC staff

Ms Lucy Llewellyn

Ms Lindsay Bradley

Mr Chris Lawlor

Ms Jessica Lichtenstein