National Efforts for Measuring EHR Usability and Adoption: Issues, Opportunities and Challenges

Andre W. Kushniruka,c,, Johanna Kaipiob, Marko Nieminenb, Christian Nohra

a School of Health Information Science, University of Victoria andrek@uvic.ca

b Department of Computer Science and Engineering, Aalto University, Finlandc Department of Development and Planning, Aalborg University, Denmark

Issues in Evaluating Electronic Record Systems Usability in Canada

Widespread adoption of electronic health records (EHRs) and related IT is well underway in Canada Provincial and federal programs fund up to

70% cost However:

– Evidence about usability issues from multiple types of users (e.g. physicians, nurses, pharmacists) needed

– Evidence from multiple locations and regions (different hospital settings and uses) needed

– Evidence about usability of different vendor systems needed

Need to share such information about system usability At local, regional, national and international

levels For improving system usability, selection,

procurement and regional deployment

IssuesApproaches to implementation and type of

systems implemented have varied greatlySimply deploying EHRs and health IT does not

guarantee that end users find them effective and usable

Can do small-scale usability studies, which are necessary and needed

However, need approaches to collect larger-scale data on use of specific EHRS and end user perception of their effectiveness and usability

Approach in Canada In Canada, to assess usability of EHRs in the large-scale, two

main approaches:

EHR Rating WebsitesProvincial websites that allow end users of EHRs (e.g.

physicians) to post their ratings of particular vendor products (i.e. EHRs), which can be reviewed by other physicians considering purchasing one of those systems

the National Physician Survey (NPS)a lengthy questionnaire sent out to all physicians, focused on a

wide range of aspects of medical practice in Canada, including use of EHRs and other technologies

National Physician Survey (NPS)Study Aims:

To characterize key aspects of physician experience in Canada (e.g. type of practice, type of patients etc.)

Includes scales about use of health information technology (HIT)

Methods:Questionnaire mailed or emailed out to Canadian

physiciansSample: Last version given mailed out (email and

regular mail) to all Canadian physicians 12,076 replied (response rate of 18%)

Examples of NPS Questions Related to HITThinking about your MAIN patient care setting,

which of these describes your record keeping system?PaperPaper-electronicElectronic only

Where do you access electronic records for your patient?

If you access electronic records in various locations, are the records electronically connected to allow for access to the same record from different locations?

LimitationsFew questions specifically about

usability/user experience in NPSNot offered every yearDoes not provide vendor specific information

about usability or adoptionNational survey needs to be expanded to

include greater emphasis on user experienceRecommendations made to government

EHR Rating Sites in CanadaEmergence of websites specifically about user

experience with specific vendor products and solutionsPart of provincial and regional programs to expand use and

adoption of EHRsAllows for entry of subjective ratings of specific vendor

products by physiciansCan guide physicians in buying (procuring) systems as they

can check the ratings made by their colleaguesOntario MD website is a good example

Allows physician users to rate EHRs (including rating of usability)

Physicians interested in buying and EHR can check the ratings made by other physicians of that system

Future DirectionsCanada needs a mechanism for end users to enter

information about usability and their experience in using specific health information systems

NPS should be expanded to include such questionsNeed to expand sites that allow for rating of

specific systems and to make the ratings richer (not just rating of overall usability on a 1-5 scale) – including rating of the dimensions of usability:E.g. scales for effectiveness, efficiency, learnability,

enjoyability and safety of systems (along with free text descriptions – i.e. more text box in the questionnaire)

Further Directions in Canada

Large-scale usability questionnaires could be complemented with qualitative usability studies conducted across regions in Canada

This could include collection of subjective user ratings and objective measures of usabilityUsing rapid low-cost usability engineering

processesCould publish information about vendor product

usability based on this

Additional IssuesIf usability (in addition to adoption) data can

be collected nationally, what can be done with it? Who will analyze it? Where will recommendations go to?