3 Frequent Mistakes in Healthcare Data Analytics

© 2014 Health Catalystwww.healthcatalyst.comProprietary. Feel free to share but we would appreciate a Health Catalyst citation.


3 Frequent Mistakes in Healthcare Data AnalyticsBy John Wadsworth


Healthcare Data Analytics

Health systems and the healthcare industry in general are exploring the possibilities of healthcare data analytics.

Big Data, population health management, or accountable care are all hot topics, viewed by many as tenets for healthcare reform.

Underlying each of these themes is the concept of analytics. Without analytics, it is difficult (if not impossible) to manage population health effectively or determine how risk should be shared.

http://www.healthcatalyst.com/healthcare-analytics-adoption-model/


Healthcare Data Analytics

While healthcare analytics is critical, it is important to note that there is no such thing as a magic bullet for analytics. It is surprising how often healthcare organizations view analytics to cure all their woes.

Generally there are three common mistakes that consistently plague analytic endeavors.

Data Analytics Mistake 1

AnalyticsWhiplash


Coloring the Truth


Deceitful Visualizations


Analytics Whiplash

Every master fly fisherman knows this very important fact:

MISTAKES OF

HEALTHCAREDATA

ANALYTICS

1

River guides call it “Perfect presentation,” a technique that every good fisherman takes the time to master.

Far too many fishermen make the mistake of impatiently not giving fish sufficient time to study the fly. Hurriedly, they abandon what might be the perfect fishing-hole on a river bend.

A fish needs some time to study the fly. It must be confident that if it is going to make the effort to chase a fly, it will catch it. For the fish to get comfortable, a fisherman must flawlessly present the fly on the water.


Analytics Whiplash

Like the impatient fly fisherman many data analysts and BI developers often feel like they are being whiplashed from one analysis to the next. The cycle goes like this:

MISTAKES OF

HEALTHCAREDATA

ANALYTICS

1

Management engages a data analyst to begin studying a problem and just when the analysis is beginning to bear fruit, management wants to move on to the next problem.

It is frustrating for an analyst to feel leadership is impatiently casting to and fro without allowing the analyst enough time to firmly and fully define a problem that could yield a big catch in the way of process improvement.

Analyst begins to

grasp problem

Mgmt requests problem analysis

Analyst assigned to study problem

Analyst directed to

a new problem


Analytics Whiplash

Analytics projects are most successful when the analyst can follow an iterative process through cycles of analysis, measurement, adjustment, followed by more evaluation and readjustment to zero in on specific process improvements.

MISTAKES OF

HEALTHCAREDATA

ANALYTICS

1

Producing half-baked analyses will lead senior management and/or process owners to be uncertain about whether they can trust the information.

Adding more analysts may only compound the issue because the organization will have a larger capacity to generate even more incomplete analysis.


Analytics Whiplash

Prioritization from Leadership Is Key

Leadership needs to become proficient with prioritization. Not everything can be priority number one. Furthermore, analysts shouldn’t be put into a position of determining what comes first. That’s a function of leadership.

MISTAKES OF

HEALTHCAREDATA

ANALYTICS

1

If they haven’t already done so, senior management must collectively take a step back as a group, determine which projects have the highest value (as well as which can wait), and then commit to seeing the highest-value projects through to completion – even if a shiny new object comes along.

http://www.healthcatalyst.com/key-process-analysis-kpa/


Coloring the Truth

Which is more important — telling senior management what they want to hear or reporting bad news accurately?

In the short run, telling your senior leadership what you think they want to hear may seem like the easier path.

MISTAKES OF

HEALTHCAREDATA

ANALYTICS

2

However, it won’t help them make meaningful quality improvements within the hospital.

It will ultimately engender mistrust in IT and analytics.


Coloring the Truth

An example of this type of flawed thinking is illustrated below. Up until 2014, Acute care hospitals were required by the Centers for Medicare and Medicaid Services (CMS) to report on Central Line-Associated Bloodstream Infections (CLABSIs) acquired during a patient’s hospital stay.

Under that regulation, only ICUs within the hospitals were required to report CLABSI rates to CMS. As CLABSI rates influenced future reimbursement from CMS, hospitals kept a very close watch on those rates.

MISTAKES OF

HEALTHCAREDATA

ANALYTICS

2Example Scenario

http://www.healthcatalyst.com/hospital-data-warehouse-best-reports

http://www.healthcatalyst.com/reduce-penalties-hospital-acquired-condition-reduction-program-hac


Coloring the Truth

Imagine for a moment that you are an analyst tasked with outcomes reporting for CLABSI.

One day you discover that what you have been reporting as CLABSI incidence is limited only to the ICU, but you have now discovered an alarming number of CLABSIs occur outside of the ICU.

MISTAKES OF

HEALTHCAREDATA

ANALYTICS

2Example Scenario

In your mind, you begin to weigh the pros and cons of sharing this information.

You tell yourself there is no future financial penalty associated with these incidents (from a payer perspective).

You remind yourself that the hospital is not required to report on these newly identified cases.


Coloring the Truth

So, what do you do?

Do you go ahead and share this new information with senior leadership? You find yourself repeatedly asking yourself how you think they, the senior leadership, will respond to this kind of bad news. At length, you decide to let a sleeping dog lie.

MISTAKES OF

HEALTHCAREDATA

ANALYTICS

2Example Scenario

You figure that because there is no financial imperative and that leadership may not react well to the news, it’s just better that they don’t know.

“What they don’t know, won’t hurt ‘em” you tell yourself.


Coloring the Truth

Analysts Must Be Comfortable Sharing Bad News

This scenario has played out with many hospital systems. It’s symptomatic of a much bigger problem. In effect, is senior leadership unknowingly incentivizing analysts to lie to them?

MISTAKES OF

HEALTHCAREDATA

ANALYTICS

2

To incent analysts to act as real analysts, senior management has to be willing to hear the bad news with the good and include all the data.

If the goal is to use data analytics to become a high-performing organization, you need to take all of the data into consideration. It may cause uncomfortable or even painful moments early-on, but it will be much more effective in helping the organization become a high-performing, data-driven health system.


Deceitful visualizations

Politicians and the media often presents good data with deliberately misleading visualizations, thereby violating Edward Tufte’s six principles of graphical integrity.

This manipulates the public to see what they want them to see instead of what’s really there.

MISTAKES OF

HEALTHCAREDATA

ANALYTICS

3

http://stat.pugetsound.edu/courses/class13/dataVisualization.pdf





Deceitful visualizations

For example, the image below (courtesy of Political Math) shows the difference between an accurate scale and a misleading scale. The graph on the left is deceptive because it gives the impression that there was a large increase in deductibles over two years.

MISTAKES OF

HEALTHCAREDATA

ANALYTICS

3

The average family deductible increased 30%in two years, from $1,034 to $1,344. This effect is more pronounced for small firms, where PPO deductibles increased from $1,439 to $2,367.

- A rise of 64%

The average family deductible increased 30%in two years, from $1,034 to $1,344. This effect is more pronounced for small firms, where PPO deductibles increased from $1,439 to $2,367.

- A rise of 64%

The reality, shown on the right, is a more modest increase.

While both graphs show accurate information, the scale has been manipulated to give a different impression.

This practice is a disservice to the data and to the organization.

WRONG RIGHT

http://politicalmath.wordpress.com/2009/06/24/president-obama-i-fixed-your-graph-for-you/

http://politicalmath.wordpress.com/2009/06/24/president-obama-i-fixed-your-graph-for-you/


Empowered Healthcare Data Analysts Lead to Usable Healthcare Analytics

When data analysts are empowered to spend time with the data and be open and honest with senior leadership and use accurate, truthful visualizations — it can make the difference between failing and thriving in a value-based healthcare environment.


More about this topic

Healthcare Analytics Adoption ModelA framework and guide for assessing and implementing healthcare analytics in an organization

Using Healthcare Analytics for Improvement Projects: Where to Start Eric Just, Vice President, Technology

Getting the Most Out of Your Data Analysts Russ Staheli, Vice President, Analytics

The Best System for Healthcare Analytics Is Not a Point SolutionKen Trowbridge, Vice President

Advanced Healthcare Analytics Case Study: Improving Appendectomy CareA Success Story from Texas Children’s Hospital

http://www.healthcatalyst.com/healthcare-analytics-adoption-model/

http://www.healthcatalyst.com/quality-improvement-in-healthcare-start-with-healthcare-data

http://www.healthcatalyst.com/why-healthcare-data-warehouses-fail/

http://www.healthcatalyst.com/healthcare-analytics-best-practices

http://www.healthcatalyst.com/health-information-technology-point-solutions

http://www.healthcatalyst.com/success_stories/improve-appendectomy-care-outcomes-using-analytics-in-healthcare


For more information:

http://www.healthcatalyst.com/ebooks/healthcare-data-warehouse-ebook/

© 2013 Health Catalystwww.healthcatalyst.com

Other Clinical Quality Improvement Resources

Click to read additional information at www.healthcatalyst.com

John Wadsworth joined Health Catalyst in September 2011 as a senior data architect. Prior to Catalyst, he worked for Intermountain Healthcare and for ARUP Laboratories as a data architect. John has a Master of Science degree in biomedical informatics from the University of Utah, School of Medicine.

http://www.healthcatalyst.com/