What we hope this will do for you…• Provide you with evidence of the impact the training
has on patient care and justify your investment by demonstrating the value of the course.
• Enable you to quickly and easily report this evidence to your stakeholders (Patients, Trust Board, various departments of your trust, LETB, CCG).
• Contribute to your compliance to the Education Outcomes Framework.
The HEE Mandate requires that there is a direct line of sight between education and training and the patient. 2
What we hope this will do for you…• Allow you to identify any issues with the training
whether they are local or course based.
• Allow you to compare the training before and after any changes are made.
• Instantly feedback to us so that we can make improvements/adjustments to the course.
3
The Surveys• On The Day survey• Post Course survey
• Both surveys were put together by a group made up of clinicians, L&D staff, and external survey experts.
• The questions are given weighted scores based on their importance and once collated give an overall score out of 100.
4
Results: On The Day Survey
5
Green Bars: ≥63Programme exceeding standards
Amber Bars: 46 - 62Programme only just meeting standards, or falling slightly
below standardsRed Bars: ≤45
Programme falling below standards
Data from 6 ALERT courses run between May 2014 to August 2014
100
25
25
25
25
86 /
23 /
22 /
22 /
19 /
Post Course survey
15
• Key to measure the actual impact of the course.
• Designed to gain honest answers from the respondent focussing on what they do rather than what they know.
• Questions are predominantly frequency based (Never / Sometimes Usually / Always).
• Each score indicates the return on investment as the number of pence in the pound. For example, a category score of 89 indicates that the trust is getting a return on investment of 89 pence in the pound for that category
Results: Post Course survey
16
100
31
44
25
Overall
Candidate
G ≥ 80A 61 - 79R ≤ 60
Programme only just meeting standards or falling slightly below standardsProgramme exceeding standards
Programme falling below standards
Collection Process• On The Day survey collected on the day of the
training course.• Post Course survey collected via e-mail
approximately 3 months after the training course.
• E-mail addresses of the trainee and their manager to be collected on the register and sent to PHT.
• Please promote the Post Course survey on the training day.
20
Collection process: On the Day Survey
21
+ Everyone can complete a survey
- Manual data entry required
PaperPaper
Manual Data EntryManual Data Entry
QR CodeQR
Code
Survey WebsiteSurvey Website
+ Data feeds directly into system
- Candidates need to have access to a smartphone / tablet
- QR code reader app required
- Possible difficulty typing correct web address
Web Address Web
Address
www…www…
::::::::::::::::::::::::::::::::::::::::
::::::::::::::::::::::::::::::::::::::::
- Accessible Wi-Fi or internet signal required
Option 1 Option 2
Collection process: Post Course Survey
24
Candidatee-mail
Candidatee-mail
PHT system
PHT system
Candidate’s Manager
Candidate’s Manager
Registration to include…Registration to include…
Link to Candidate survey sent to
candidate e-mail
Link to Candidate survey sent to
candidate e-mail
www…www…
Link to Manager survey sent to
manager e-mail
Link to Manager survey sent to
manager e-mail
www…www…
3 months later
?
26
Downloading the results
You can filter by date to report on a specific training day, a month,a quarter, or the year in full.
Using the Excel Templates
28
• We will send you 2 templates (1 for On the Day and 1 for Post Course) along with simple instructions on how to use them.
Basically select all, copy and paste into the template. Then Save As. Then Save As PDF.
Survey WebsiteSurvey
Website Excel ExportExcel ExportPHT
ExcelTemplate
PHTExcel
Template
PDF in printable
format
PDF in printable
format
Faculty / Peer Assessment
• This was one of the issues that we discovered and have subsequently resolved at PHT.
• This will be covered in Developing ALERT™ Faculty at 14:15.
31
Summary• Identified benefits of using this evaluation
process.
• Taken a look at example results from PHT.
• Mapped out the collection processes for both On The Day and Post Course.
• Talked about extracting the results.
• Shown what the print outs will look like.32