AI/IoT/Analytics, Innovation/Emerging

Big Data Insights in Healthcare, Part II. A Perspective on Challenges to Adoption

Louis Frolio By Louis Frolio October 15, 2015

In Part I of this series I asserted that healthcare has the most to gain from the adoption of big data. Using a quote from the CEO of a leading healthcare technology company, I qualified the assertion:

“The end story is not the big data, or the analytics that comes from it. The real value of gleaning big data insights is to embed them in the workflow of the people who need them just-in-time to support actualizing the information.”

Traditional healthcare data collection and storage

Traditional healthcare data collection and storage


The number one cause of death in the United States is attributed to errors and mistakes of our healthcare system.

Putting this into perspective, the documented error rate of Intensive Care Units (ICU) is equivalent to the U.S. Postal Service losing 16,000 pieces of mail every hour of every day, 365 days per year.

Information technology and data are crucial components needed to reverse this situation. Moreover, they also provide the means to deliver a better experience of care, lower healthcare costs, and improve population health. This is achievable because technology, at scale, puts information in the hands of the people who need it just in time to make better informed data-driven decisions.

Before that can be fully realized, healthcare must first overcome several systemic challenges, such as:

  • Maturity – Out of the box, healthcare software does not natively “connect” to external healthcare systems.
  • Interoperability – There are many healthcare software vendors, each using a different standard to represent clinical data.
  • Siloed Data – When there is a transition of care from one unconnected practice to another the patient’s history becomes fragmented.
  • Organizational Buy-in – Healthcare organizations fear patient leakage will occur if they share their data.
  • Standard Nomenclature – Most EMR systems not equipped to provide codified clinical data that is actionable.

Before we dive in, I want to introduce a few relevant concepts.

The small practice is the backbone of American healthcare. Staffed by over 209,000 (2010) practicing primary care physicians, a Practice provides outpatient or ambulatory care. Types of primary care include family medicine, obstetrics, allergy, dermatology, plus many more.


Independent Hospitals are next up. Larger than a private practice, they offer more services and employ a much larger staff.


Practices and Independent Hospitals are generally limited in the services they provide. By joining a Hospital System, small practices and independent hospitals are able to leverage the services offered within a large system.


Large Healthcare Organizations (HCOs) – offering all services necessary for complete patient care – are the most complex of health systems.

HCOs of all sizes are like any other line of business. Patients are seen as customers and the HCO is constantly seeking growth by increasing its patient base. When it comes to choosing a provider, patients usually have some freedom of choice. If they are unhappy with a provider they can seek services from another.

Hospital systems are formed to increase the patient base, and to benefit from economies of scale. These systems make it easier to offer many services for the practices they own which helps ensure continuity of care for patients. Also, practices in the system make it easy for the system to monetize referral power.

It is here that the first challenge is encountered. Maturity in healthcare refers to the ability of healthcare systems to interchange patient data. Practices are the front line of support in healthcare: They just want to focus on providing excellent patient care and not wrestle with technology. As a small business, practices are finding it difficult to compete with larger healthcare networks.

To remain competitive, a practice will join a network which requires that they share information electronically within the network. This is referred to as being “connected.”

Getting “connected” usually requires a vendor professional services engagement that can cost anywhere from $2k-$8k; an expensive proposition to a small practice. The problem is compounded when the network has different hospitals using different EMR systems which can result in 2x or 3x the $2k-$8k engagement fee.

Enter the second challenge. Interoperability of healthcare software (PMS, EMR, EHR) is in its infancy regarding “connectivity”. Part of the 2009 American Recovery and Reinvestment Act included legislation for the Health Information Technology for Economic and Clinical Health (HITECH) Act. The HITECH Act, in conjunction with the Centers for Medicare & Medicaid Services (CMS), incentivizes – as part of Meaningful Use – providers to leverage electronic transmission (10%) for transitions of care and referrals, and submit a summary of care record to a different EHR.

Though most healthcare software today has the “plumbing” in place to enable “connectivity”, this does not mean that they can connect. This is because there are different standards for how healthcare software stores and exchanges its data. Every vendor will use a different standard. Some of the standards include: HL7 V3, HL7 V2, HL7 V2.3, and C32. For different systems to connect there needs to be a costly translation mechanism put into place.

The third challenge is siloed data. This is, in part, an artifact of the two challenges discussed above but results mostly from a lack of connectivity. During a transition of care, when a patient moves from one unconnected practice to another, their medical history undergoes fragmentation. As they move to yet another practice, the history becomes even more fragmented. This has a real impact on the ability of a physician to provide care to the patient.

A fourth challenge – organizations not buying into to the idea that they need to share their data – exacerbates the problem of siloed data. There is a fundamental financial disincentive for HCOs, especially hospitals, to share their data. Patients are seen as customers and HCOs fear that by sharing their data other HCOs will try to lure their patients away. Organizational Buy-in might be the most difficult challenge to mitigate.

The fifth challenge is related to software maturity. Most EMR vendors, including the most prolific, are still selling software built on a code base that is at least 10 years old. These systems don’t provide a means of capturing clinical documents and orders using a standard nomenclature. In fact, the most useful information in an EMR system is locked up in unstructured data.

These challenges, either entirely or in part, affect the gamut of HCOs. Most are trying to overcome these challenges, with the largest HCOs finding some success. I don’t expect that that these challenges will be eradicated overnight. Instead, I anticipate that small incremental changes will occur over time.

In part III of this series, I will discuss how the industry is addressing these challenges. I will also discuss how big data can be used by healthcare to deliver a better experience of care, lower costs for care, and improve the health of the population at large.

Regards, Louis.

Louis Frolio

About Louis Frolio

Read More

Share this Story
Join the Conversation

Our Team becomes stronger with every person who adds to the conversation. So please join the conversation. Comment on our posts and share!

Leave a Reply

Your email address will not be published. Required fields are marked *

3 thoughts on “Big Data Insights in Healthcare, Part II. A Perspective on Challenges to Adoption

  1. Hello Louis,

    The article is excellent and I would like to add two other hurdles that will need to be overcome.

    Legal liability issues. If a doctor takes action based on another doctor’s tests and evaluations, and later there are complications due to misinformation, who is liable?

    Generational issues. A number of doctors out there just aren’t technologically prepared and view the use of the electronic systems as problematic. There will be some adoption issues while there are still technology adverse doctors.

    While time may well cure one of these issues. The legal issues are likely to be very sticky.

    Rich Moore

    • Rich, thank you for sharing your insights on this important topic. My understanding is that the lack of “connectivity” is the biggest and most important challenge to overcome. I saw a recent survey that looked at the challenges of “achieving a fully connected healthcare environment” and the top three challenges were: Cost, Provider’s resistance to changing their practice styles, and Technological limitations.
      Your observation of providers being “risk averse” and “stubborn to change” is spot on.
      Regards, Louis.