End of Alpha Report NHS App - WhatDoTheyKnow

End of Alpha Report

NHS AppMay 2018

2

Background

3

The NHS App programme seeks to empower patients by giving them access to core NHS services and visibility of their records. Key aims are to:

• Give access to digital services whenever, wherever and however users want to consume them

• Make it straightforward for local / regional providers, commissioners and 3rd parties to create and iterate high quality services

• Reduce administrative costs through increased use of digital channels

• With appropriate permissions, use patient record information to inform digital triage and promote approved digital apps from a competitive and vibrant marketplace

On 12th September 2017, the Secretary of State set out ‘eight challenges’ for NHS England, to be delivered in 2018 and met through online channels and apps. These eight challenges correspond to eight services that form the basis for scope of the NHS app

Context

The Secretary of State at the Health & Care Expo 2017 announcing the ‘8 challenges’, which would represent some world-firsts for the NHS

The 8 challenges:1. Symptom checking and triage2. Patient record access3. GP appointment booking4. Repeat prescriptions online5. Set or change data sharing preferences6. Set or change organ donation preferences7. Set or change end of life care choices8. Promote approved apps relevant to to the patient

Discovery Recap

• asdf4

The NHS App Discovery commenced in November 2017 and completed in January 2018

During Discovery we:

● Found that users found a two-app model confusing, and that a single app would be much better

● Found that users found apps convenient for accessing NHS services

● Identified an architecture that could be used to deliver the app

● Created an initial clickable prototype to start testing how user needs could be met

● Came up with a plan and a recommendation for commencing into Alpha

5

Alpha Objectives

Key Objectives for Alpha:

1. Develop a PoC application for GP Appointment Booking, deployed for Android/iOS/Web and integrated with one GP system.

2. Integration with Citizen ID (as-is sandpit)3. Embed current live Organ Donation registration

form into the app

4. Embed the current live NHS 111 online page into the app

5. Conduct further analysis on how End of Life Care can be implemented

The NHS App Alpha commenced at the end of January 2018, running until the end of April 2018

In parallel with work towards the key objectives, we also planned to:

1. Continue to engage principal GP system suppliers

2. Carry out user research & service design activities as per typical alpha

3. Identify and engage internal assurance teams

4. Support NHS England and NHS Digital colleagues on rollout planning, comms and engagement

5. Devise a plan, ramp up the team and scale our processes ready for Beta

Research and Design

6

7

In Alpha we interviewed and tested with a range of users to ensure that the app is suitable and accessible for all

This included 66 users with a range of the following:• Geographical location• Gender• Age (16-75)• Ethnicity• Socioeconomic group• Employment status• State of health (including long-term conditions)• Mobile device familiarity• Assisted digital support requirements & access needs

User research to date

8

Participant Demographics

Employment Status

Ethnicity

AgeSocioeconomic Status

9

1. Just gets stuckWill require Partial support

2. Self-skepticsWill require Partial support

3. Those with a barrierMay need Partial or Full support

4. Can't & won’tWill not seek support so we will promote the benefits of the online service

Assisted Digital

During our user research we identified four Assisted Digital user types that will require different levels of support:

10

Validated Personas

11

New Insights during Alpha

• A difficult registration journey is off-putting and will damage uptake

• People value the 111 Online function but often want simply to look

something up

• Health A-Z is the more popular symptom function but is difficult to use

on a mobile device

• People understand End of Life Care more than they understand

Advance Care Planning

• Once people saw the End of Life Care prototype they were more

supportive of it within the app (when in Discovery we only covered the

concept)

• When using an online service for prescriptions, people expect they will

be collecting the medication rather than the paper prescription

Key user insights

“If it's hard I won't use it. I'd give up. I'd call."

E - 20 - Healthy

“If you’re healthy you don’t want to think about it. But if you’re in that situation you want to know and to be able to make that choice.” J - 58 - Diabetes

“I don't want to answer a load of questions that aren't relevant! What if I know what I'm looking for?”

S - 39 - Anxiety

Insights from Discovery

• 2 app model - hard to understand and poor experience

• NHS brand is well - recognised & highly trusted. But this breeds high

expectation

• Appointment booking / Prescriptions / Symptom checking / and

Record Viewing are the most valuable services

• Security is important (lots of comparisons to online banking)

• App design and usability is preferable to current GP online services

The majority of users think that the app is a good idea: simple, quick, and all in one place

12

Key GP Practice findings

• A patient booking the wrong appointment with the wrong clinician is probably the greatest drain on resources

• Patients need more help to know if they need an appointment, and the next best action

• Current digital triage/screening solutions cannot replace the receptionist

• Digital processes empower patients and lead to better outcomes

• The app needs to add value to patients and GPs for clinicians to engage

• Workflow automation frees up time to be more patient focused

• Practices will struggle with significant increased demand for registrations, appointments & record access

• There is great enthusiasm for digital processes that relieve administrative burden

“There has to be a USP for the patient: ‘why would I have another app on my phone?’"

GP

“A little Asian lady who spoke no English loves it! [GP online services] All she has to do is click a button and her prescription is at the pharmacy.” GP

“It's not about stopping people from making appointments but saying 'is this the right appointment for you’?”

GP

13

• Regular usability testing of prototype and production system as

it becomes available (including tablet and web-app versions)

• Focus on design challenges around:

– Replication of receptionist-style triage for appointments

– Ensuring users understand which ‘Symptoms’ function is right

for their situation

– How consumed services fit into the app seamlessly

• Targeted sessions with people with assisted digital/access

needs

• Supporting plans for private beta participant recruitment (e.g.

design of sign-up form)

Research plan for Beta

14

Round 1

• Health A-Z added as an option to Symptom checker• Collaboration with 111 online and Health A-Z teams to define value

proposition and hierarchy of services• Disclaimer about sensitive information added before viewing medical record

Design & content iterations

Round 2

• Prompt to check symptoms added to start of appointment booking process

15

Round 3

• Improvements to design and structure of appointment viewing and booking• Design and content consistency across services: visual elements,

implementation of GOV.UK and NHS patterns, language and style guidelines


Round 4

• Repeat prescriptions design made consistent with appointments• EMIS API limitations reflected – allowing testing of realistic

scenarios

16

Work in progress

• Design of error and validation messages - consistent with GOV.UK and NHS guidelines

• Sitemap including all edge cases and error scenarios

• Content design collaboration with Citizen ID

• Research and analysis of user language and terminology


Citizen ID

• Two versions tested in collaboration with Citizen ID team

• Round 1: using linkage keys

• Round 2: online process overall, including ID verification

17


Home screen• Onboarding screens

design to be tested

Service design

18

19

Key performance indicators

Increase completion rate

Increase digital uptake

Reduce cost of service

Increase customer satisfaction

KPI Transaction MetricsAppointment booking, repeat prescriptions, etc

Referral Metrics111 online, Health A-Z, Organ donation, etc

As new capabilities (e.g. personalised apps library and interoperability with market services) are added to the app, additional KPIs will be required

§ Cost per referral (£)

§ Customer satisfaction through the app, vs existing channel (+/-), by referred service

Manage system demand

§ Completion rate through the app, vs existing channel (+/-), by referred service

§ Overall digital uptake (%), in each referred service

§ Cost per transaction (£), by transaction

§ Customer satisfaction, by transaction § GP practice satisfaction

§ Completion rate (%), by transaction

§ Digital uptake (%), by transaction

Platform Metricse.g. overall app ratings, installs etc.

§ App store rating

N/A

N/A

N/A

Note: additional KPIs will be explored as the business case and benefits are developed, for example:

DRAFT

Supported platforms

● Analysis work done to determine which OS/ Browser combinations we should support on mobile and desktop devices

● Based off NHS.UK traffic over the last 12 months (captured in Websense), GDS guidance in this area and details of vendor support policies and release plans

● This baseline position will cover over 99% of NHS.UK users and has informed our test approach

● We will be supporting the OS/Browsers documented above and testing on a representative subset of these

● Any issues reported on supported, but not tested, OS/Browser combinations will be resolved

● Major mobile OS release cycles mean we’ll keep the list under on-going review

■ Primarily exists for Windows Phone and Symbian users, although might conceivably be used by Android and iOS users who do not want to install the native app■ Chrome, Safari, Samsung Internet & Internet Explorer - latest versions (current and -1)

■ iOS version 9 (released September 2015) or newer, running a minimum of Safari9.0.3■ Next major OS release (expected Sept 2018)■ Both iPhones & iPads

AndroidNative app

■ Android version 5 (Lollipop released November 2014) or newer, running a minimum of Chrome 39 for Android■ Next major OS release (expected Oct 2018)■ Phones, Tablets & apps running natively on Chromebooks

Apple iOSNative app

■ Based on GDS guidance GDS guidance can be found herehttps://www.gov.uk/service-manual/technology/designing-for-different-browsers-and-devices

* bu partial support for IE8

Other mobileWeb interface

DesktopWeb interface

21

• The NHS app will launch in 2018, and will deliver a range of services including symptom checking and appointment booking.

• The NHS app will then be extended to present other apps from the NHS apps library (https://apps.beta.nhs.uk)

• In 2019, the app will be extended to act as a platform for other web-based services. The aims of this are:

– To stimulate innovation in the open market– To provide a positive user experience, by creating a

single point of access for mobile NHS services.• In order to address these aims, we envisage enabling access to

integrate into the app for the following uses:– Local and regional commissioners procuring

commercial services to appear in the app– Organisations in the open market wishing to access the

NHS app user base to promote their health-related digital service

• The purpose of this pack is to outline our intentions and seek feedback from market providers and NHS colleagues

Integrating services into the NHS App

Mockups are for discussion purposes only

Appointments Prescriptions

Quick links

Other services for you

In your area

VideoDocLeeds

Medicine Direct

DRAFT

https://apps.beta.nhs.uk/

22

NHS App – Service Blueprints

Our service blueprints allowed the project to study:

• The user journey across all services• The actors involved• The systems’ touch points• Policy/rules• Metrics and data flow• Stakeholders• Pain points• Opportunities

23

User Flows – CID without Linkage Keys

Working alongside Citizen ID, we studied all the system’s touchpoints around three options to help lift

the need to ask users for the GP system’s linkage key in order to create a better user experience.

24

Cross Cutting Journeys

These allowed to study the interaction between NHS app, NHS.uk and Citizen ID through cross-cutting scenarios thus exposing a plan for of action for sign-posting and promoting the app.

25

End to End Screen Flow

This flow allowed us to:• Catalogue all the screens as well as

understanding the need for new screens• Expose all the pain points from users and

clinicians• Expose all the needs from users and

clinicians• Align design, content and tech to the same

screen catalogue to promote traceability• Expose opportunities to ideate based on the

pain points encountered

Delivery, Engagement & Rollout

26

Delivery Roadmap

28

Assurance

• Work undertaken during Alpha to identify all the assurances and sign-offs required to launch the NHS App

• This includes technical architecture, information security, clinical safety, solutions assurance, information governance and security

• Named owners, budget allocation, ways of working requested and in train or in place already, for each stream

NHS App Rollout

End of Sep

2018

End of Dec

20182019

Private Beta

launch

Public Beta

launch

Private Beta (3 months) Public Beta (duration TBC)

Private Beta:

• Invite-only – requires only an email address from users.

• ~10 GP practices are being targeted to support this phase

(targeting high wifi uptake, high 111 online uptake, EMIS)

• Options are available for gathering email addresses from

participants electronically (e.g. text campaign and sign-up

form) or paper (e.g. standing at practices, or leaving forms

with practices)

• Projected number of participants is likely to be at least 2000

users, but will be confirmed in the coming weeks

Public Beta:

• Publicly available but access will be controlled in-app

• We will control access down to the individual practice

• Options exist for what we restrict during rollout: e.g.

everything, GP online services or logged-in services

• Some soft controls can be used to discourage users

from downloading unless it’s available in their area – e.g.

app name, app description, in-app guidance, social

media, possibly an in-app post-code lookup

BETA

Plan for Beta

30

Login/Register

Beta Epic Backlog

Symptom Checking

111 Online Service Initial Integration

EnhancementsHealth A-Z Integration

GP Appointment Booking

Repeat Prescriptions

Patient Record

Organ Donation

Organ Donation Initial

Integration

Data Sharing

Data Sharing Initial

Integration

App Readiness

Desktop experience

Terms & Conditions

During Private Beta

Restrict access during rollout

In-app feedback

Plus enhancements, platform work and non-functional requirements (security, accessibility, performance, monitoring, etc)

Initial Citizen ID Integration

(EMIS)

Citizen ID Integration

(TPP)

Citizen ID Integration (Microtest)

Citizen ID Integration

(Vision)

GP Appt Booking (EMIS)

GP Appt Booking (TPP)

GP Appt Booking

(Microtest)

GP Appt Booking (Vision)


(EMIS)


(TPP)


(Microtest)


(Vision)

View Summary Patient Record

(EMIS)


(TPP)


(Microtest)


(Vision)

Detailed Coded Record (EMIS)

Detailed Coded Record (TPP)

Detailed Coded Record

(Microtest)

Detailed Coded Record (Vision)

32

Beta Team Profile

Delivery Management

Security Specialist Quality

Assurance

Tech Lead

The team has grown during Alpha in preparation for Beta to fully support the build activity:

Service DesignDelivery & Technical Leadership

Scrum Team Web Ops

Non-Functional Test Team

x4

Quality AssuranceLead

Technical Architects

Web Ops Lead

Web Ops

Development&Quality Assurance

Product Management

User Research,Service Design & Content Design

Business Analysis & Service Design

Thank you

Documents

End of Alpha Report NHS App - WhatDoTheyKnow