HEALTHCARE SOFTWARE TESTING STRATEGIES€¦ · When testing healthcare applications, we have to be very cognizant of the domain and speciﬁc application usage contexts, including

HEALTHCARE

SOFTWARE TESTING

STRATEGIES

Medical professionals, as well as patients, are using

healthcare software applications on mobile devices.

Portable access improves care and convenience. But the

increased complexity raises critical issues about maintaining

security, integration, and interoperability across platforms.

Laptops, smartphones, and tablets add entry points to the

mobile network that increase the risk of loss or unauthorized

access to Patient Health Information (PHI). Testing permutations

are increased several fold with new ways data is stored and

transmitted between various devices via Bluetooth, WIFI, and

cellular communications. Developing a test strategy to not only

cover functionality, but also non-functional requirements in

security, data integrity, display, and performance, requires

careful thought with a systematic approach.

requirements is not met, then it can’t satisfy the needs of a

healthcare application that’s HIPAA compliant. Mobile

security involves many more pieces than we are accustomed

to including in the network infrastructure, hardware, and

operating system. Additionally, with most healthcare

applications providing interoperability amongst medical

devices and information sources via HL7 messages, any

system sending and receiving messages needs to be added to

the test matrix.

Testing Patient Mobile Apps Security and Data Integrity

With the cost of healthcare skyrocketing, there is a drive

toward increased efficiency while raising the level of

patient care and striving for better outcomes. This can only lead

down the path to increasing the scope of software in automating

numerous procedures and processes in healthcare. The

healthcare IT market continues to grow1, supported by

expanding requirements for more sophisticated software across

all categories, including:

• Patient mobile apps

• Medical device software

• Electronic Health Record (EHR)

• Electronic Medication Administration Record (EMAR)

• Billing and claims processing

• Pharmacy Management and dispensing

• Physician Practice Management

There are many more types of software, including: healthcare

specific CRM, Health Information Systems, health analytics and

imaging systems. But the above represents the major categories.

When testing healthcare applications, we have to be very

cognizant of the domain and specific application usage contexts,

including government regulations and compliance issues, but

especially those regarding data security and privacy. In this white

paper, we discuss the major types of healthcare software and the

issues related to ensuring their quality.

11XBOSOFT.COM CONTACT US AT [email protected]

TESTING HEALTHCARE SOFTWARE

1 "Healthcare IT Market Size To Reach $104.5 Billion By 2020."

https://www.grandviewresearch.com/press-release/global-healthcare-it-market. Accessed 20 Mar. 2017.

2 "Testing your health care application for use on mobile devices."

http://searchsoftwarequality.techtarget.com/tip/Testing-your-health-care-applica. N.p., n.d. Web.

Data must be accurate and consistent, complete, timely,

interoperable, auditable and accessible.2 If any of these

https://xbosoft.com/

The way that your application is rendered probably has

the biggest impact on whether or not the application is

Healthcare apps on mobile devices must be reliable.

You can’t have an incident in the middle of collecting

patient data or doing a vital reading. Therefore, testing

functionality alone isn’t enough. We need to test in the field.

Simulating real conditions, such as syncing of healthcare data is

crucial when a practitioner is with a patient and needs to

upload data or receive it from another source. With a

sub-optimal connection, what happens? In a typical medical

building, signal strengths vary based on location, construction

materials as well as interference from other devices. Ensuring

the application can recover and adapt to suboptimal network

conditions is critical. Conflict resolution for data access is also

critical. What happens when the same patient record is

accessed from another location?

We all get frustrated if any of the apps on our mobile devices

don’t provide good performance. The frustration is exacerbated

with healthcare apps. Even though they may be more

complicated and involve many more moving pieces, user

expectations don’t take that into account. For testing, this

means having a coordinated performance testing strategy that

not only tests in the lab, but can prioritize and simulate

situations in the eld that would a ect performance. This means

you’ll need automated performance tests that simulate not only

real connectivity issues but also, real workfows.

Display

ultimately accepted as a replacement for whatever is currently

in place. When it comes to displaying on different devices,

you’ll have to ensure that you have tested all ‘supported

platforms’ covered. There are no excuses for field names not

displaying correctly or being cut off. Unnecessary scrolling

and clicking with a small screen won’t be tolerated. You’ll have

to make sure your application is optimized for a mobile device

and conduct usability assessments from the very beginning.

Reliability Performance



To continue to innovate and deliver more features while

keeping up with regulations and compliance, medical

device manufacturers are continuously developing devices

with the help of software. This process is similar to testing

mobile application software, except even more complex. Not

only are testers looking for defects, they must ensure that the

software meet the stringent requirements of the IEC 62304

standard, which applies to all software for medical devices. Thus

they must possess a thorough understanding of regulation and

compliance matters related to testing medical devices. Our

challenge is twofold; We have to test for explicit requirements

in data accuracy and reliability, while being aware of the

security concerns raised by the many new permutations of

connectivity. Therefore, testing medical devices requires a

thorough approach and strategy that covers many

requirements and is designed to reveal defects where they are

most likely to occur.

Testers need expertise and experience of the domain, device

design, and connected infrastructure to support their testing

in such areas as performance, interoperability, security, user

interface (UI), and reliability amongst others. You can’t

tolerate a reboot in the middle of surgery when a patient’s life

is at stake. Testing the UI is critical so that data is presented in

an understandable way to avoid misinterpretation and

confusion. Accurate and timely transmission of the data via

HL7 to various interfaces is imperative.

Behind all this we have increasingly sophisticated malware

trolling healthcare databases with the intent of stealing that

data. Frequent device connections, the transfer of data, and

even just being on the network exposes PHI vulnerabilities. To

thwart this threat, new software must allow medical devices to

integrate into hospital networks and various data sources

safely--and testers must be able to determine if the software

meets such a standard.

Testing Medical Device Software



Enhanced efficiency, cost containment, and improved

patient outcomes are the primary drivers for EHR software

implementation. Accuracy and completeness of workflows are

essential to accomplishing these objectives. For testing, you

should have established workflows or user stories where each

individual and departmental role is defined regarding patient

input, clinical documentation, billing, scheduling and patient

follow-up. Thorough testing is needed to ensure that all

processes work in the software including usability issues to

identify areas of confusion. Avoiding confusion is perhaps the

most overlooked element in testing healthcare software; it

requires an eye for detail, mapping the existing processes to the

software workflows. You want to find issues or breakdowns and

correct them before going live with real patients.

When testing, you’ll need to create many test patients to

practice every aspect of clinical documentation, billing, ordering

prescriptions, printing patient education handouts, scheduling,

etc. In doing so, you’ll need to create many user stories. For

example, a checkup for a patient with a broken arm, checkup

visit for a senior citizen, or surgery follow-up.

In your user stories, you’ll need to include every component of

documentation that would be required for that patient, from

patient personal data to medications, vital readings,

prescriptions and future appointments. For each role, step

through each step in seeing the patient. For example, the

receptionist will obtain identification and consent forms, and

input the information into the system. Next, the nurse will

examine the entered information and then collect vitals

including temperature, oxygen level, weight, height and blood

pressure. Physicians will do their physical exam and input their

data and diagnosis, followed by a plan of care and scheduling

for future appointments. By stepping through each of these

roles, you’ll be able to identify possible gaps in understanding or

workflow. The more variation in your user stories the better. You

can’t test all scenarios, so you’ll need a well-prioritized test plan.

Testing Electronic Health Record

Software (EHR)



Testing Electronic Medication Administration Record Software


Reduced medication errors and better outcomes are the primary

objectives of implementing Electronic Medication

Administration Record (EMAR) software. According to the FDA

Adverse Events Reporting System (FAERS), the number of

serious incidents and deaths continues to grow. EMAR systems

are being designed and implemented to alleviate this situation

by replacing paper and other non-digital records that are prone

to being incomplete, misread, or even misplaced. The goal is to

ensure that nurses can properly administer and track

medication. This also reduces the chances of ‘lost medicine’ and

provides relevant drug knowledge at their fingertips essential to

understanding drug interactions and effective integrated case

management. Again, documenting and understanding

workflows and processes are critical in testing EMAR software.

User stories need to step through each role. Their part in the

overall process of administering medication and interfaces with

other systems that dispense the medication needs to be

examined, including inventory tracking and knowledge bases

that keep up-to-date drug interaction information.

From FDA (https://www.fda.gov/Drugs/)

Medication Error-Related Deaths and Serious Incidents in the US



Electronic prescribing of controlled substances (EPCS) is

now permitted in all states, so we can expect it to gain

considerable momentum. Replacing paper prescriptions, EPCS

can reduce fraud/forgery and secure electronic records, while

improving safety and patient care. However, there are

challenges slowing down its adoption. Examples include

pharmacy and prescriber applications of EPCS that need to talk

to each other and ill-defined implementation standards. No

standards exist for interoperability other than HL7 for

information transfer. Thus is it necessary to test each system, its

behavior and its interaction with connected systems. This will

require attention to detail and an understanding of workflows

that cross system boundaries. Additionally, when it comes to

testing, the same issues that surround HL7 (see page 8) are

applicable specifically on HL7 compliance.

For physicians ‘writing’ prescriptions, proving identity is required

using 2-factor authentication (TFA) credentials. TFA credentials

can be implemented via tokens or biometrics such as fingerprint

scans. A physician might need a code and a fingerprint scan to

write an EPCS. As TFA may be implemented in a variety of ways,

a deep understanding of TFA implementation is essential to

develop scenarios that not only test if it works (positive), but also

to verify that it doesn’t work (negative testing).

Regulatory changes related to EPCS are becoming more

prevalent. Some states are requiring e-prescriptions for

opioids or other specific drug classes. They require pharmacies

and physicians to use only electronic prescriptions for

controlled substances. As these new regulations become

implemented in software, knowledge of domain business

rules will be critical to developing accurate user stories.

Testing Electronic Prescribing of

Controlled Substances (EPCS) Software



Government regulations apply to all of these software

types. These regulations can be assumed as stringent

For the healthcare industry in the U.S., the Health Insurance

Portability and Accountability Act of 1996, or HIPAA,

requirements, even if not explicitly stated, so it is critical for any

testing organization to both thoroughly understand these

regulations and ensure that the test strategy and test plan

accommodates them.

represents one of the main regulatory pieces of legislation. It

has two major components:

• Health insurance coverage is protected for workers and

their families when they change or lose their jobs.

• National standards are established for the Security and

Privacy of private health data while allowing the flow of

health information needed to provide and promote

high-quality healthcare, and to protect the public’s

health and well-being.

Therefore, from a software testing perspective regarding HIPAA,

we are concerned mostly with Security and Privacy of health data.

All healthcare applications must meet HIPAA standards; ensuring

that this happens can be very arduous, as data crosses many

disparate systems and follows a patient from system to system.

Data and information must be kept safe from unauthorized

access, and when it is accessed, it must be kept private to only

those given access. The data’s aggregation level must be

properly followed to prevent an individual’s data from exposure.

The five following areas must be covered when testing a software

product for HIPAA compliance3:

1. User Authentication – Using verification methods to

ensure that those logging on are who they say they are,

and to deny access to all others.

2. User Authorization – Authorizing access to

information is based on user role and patient limitations.

3. Audit Trail – All transactions and all attempts of

data access with a proper set of audit trail information

are recorded.

4. Data Transfers – Ensure data encryption at all transfer

points according to ANSI 5010.

5. Help Information – Help information on the correct

and incorrect uses of data.

There are many other specific regulations that may or may not

apply depending on your software product and its users. A few of

these requirements include3:

1. When a breach of security is discovered, notification to

the consumer affected must be within 60 days.

2. Rules for pre-existing conditions are numerous and

complicated. For example, the exclusion period for

pre-existing conditions can be 6, 12, or 18 months,

depending on the circumstances. These rules have many

variables and many test cases are needed to ensure

compliance. Having a thorough understanding of these

rules is paramount.

3. Rules concerning family members are also varied and

complicated. You’ll need to understand the coverage

variables for each type of situation.

TESTING FOR COMPLIANCE

Testing for HIPAA Compliance


3 Healthcare Software Testing Strategies for HIPAA ...(n.d.). Retrieved from

https://xbosoft.com/healthcare-software-testing-strategies-for-hipaa-compliance/


https://xbosoft.com/software-testing-services/security-testing/

Health Level Seven International (HL7) is a standards body

providing “a comprehensive framework and related

standards for the exchange, integration, sharing, and retrieval

of electronic health information”4 to support health services.

In order to test HL7 standards compliance, it’s important to

understand its functionality as well as its limitations. The HL7

framework is very flexible, which means that every single

implementation will be different. Two different environments

will have different data exchange rules. Therefore, testing HL7

interfaces has several associated challenges:

1. Each system needs to interpret the meaning of each

data item. This means you need to understand the

semantics of the data in different contexts and how

each system receives the data and deals with it.

2. Data items can be exchanged to achieve different

multiple goals in different contexts. Some fields may be

documented as optional but for specific workflows, there

may be data constraint rules. In some contexts, the rules

will be removed. This means you’ll have to analyze and

test each exchange on a case-by-case basis.

3. HL7 provides suggested values for some fields, but

you’ll need to understand how systems implement them

and the mapping between different systems.

4. Segments and fields require customization:

Field length, data types and other attributes can

be customized, which means you’ll have to

understand the data structure and differences

between interfacing systems.

HL7 interface testing typically includes:

1. Unit testing to confirm that HL7 messages sent and/or

received from each software application conform to the

HL7 interface specification.

2. Integration testing with user stories that ensure that

information flows correctly between applications.

3. System testing consisting of end-to-end scenario

testing to ensure all modules of all relevant applications

are able to correctly exchange data. This means you’ll

need to design scenarios that integrate EHRs, Billing

Systems, HIEs and other healthcare applications

requiring in-depth knowledge of each system.

Any and all testing needs to ensure the protection of privacy

regarding PHI. This means any information in a medical record

that can be used to identify a specific individual, including

information about health status, provision of health care, or

payment for health care.

Testing For HL7 Compliance

4 Health Level Seven International - Homepage. (n.d.). Retrieved from http://www.hl7.org/





Healthcare software continues to grow and encompasses

many different technologies and environments, expanding

from a typical hospital acute care setting to long-term care

facilities, home care, and more. As such, the drive is to allow the

flow of information to improve efficiencies and medication

decision making while improving outcomes. Achieving this

utopia, however, can involve over a dozen software components

that need to talk to and integrate with one another.

Testing all of these pieces and components needs to focus at

each level of aggregation. First, on each individual unit test,

then combining related units into a component test, and then

finally combining components into a system where complete

user scenarios can be executed from beginning to end. The

test plan should be composed of each function and feature

used in practice. This includes all scenarios: visit types, roles

and actions/information entered. Test cases should include

expected results as well as deliberate negative tests to quickly

detect defects and system logic deficiencies or errors.

Regarding regulations and compliance, it’s important for the

test strategy and plan to include which parts of the

regulations apply and to ensure that the right rules are

included in test case design to guarantee proper coverage.

This all requires very particular domain and software expertise

and experience--a body of knowledge that can only be

accumulated over time. This should be facilitated by

implementing a test case management system with

procedures within each scenario for tracking issues,

requirements, expected and unexpected results as well as

data needed for analysis of root causes. Without learning, and

managing the specific domain knowledge, it’s difficult to test

healthcare software and ensure its quality.

Lastly, in addition to knowledge of the regulatory

environment, domain knowledge, and understanding

interconnectivity complexities, we must always keep security

in mind. Testing for vulnerabilities should take place at all

phases of development to ensure PHI is protected to prevent

unauthorized access.

SUMMARY - ACCUMULATING DOMAIN KNOWLEDGE IS CRITICAL

https://xbosoft.com/test-strategy-test-plan/


ABOUT XBOSOFT

XBOSoft is a software testing and quality assurance company that offers services in web, mobile and desktop applications, as well

as broad domain experience with extended expertise in healthcare, finance, and software security. XBOSoft’s quality process

assessments and test methodologies speed products to market and improve clients’ software quality and performance throughout

the software’s life cycle.

With a proven track record working for Fortune 100 companies to small independent developers, XBOSoft today has over 100

employees in San Francisco and Beijing offices serving clients in the US and Europe. As the firm surpasses its tenth year of

continuous growth, XBOSoft proudly offers its full range of testing and quality assurance for both on and off-shore services.

PHONE: +1 703 995 9896 EMAIL: [email protected] XBOSOFT



tel:17039959896

Documents

HEALTHCARE SOFTWARE TESTING STRATEGIES€¦ · When testing healthcare applications, we have to be very cognizant of the domain and speciﬁc application usage contexts, including