Data Architect Shares 5 Lessons He Learned at a Major Healthcare Organization

April 1, 2022
Data Architect Shares the Lessons he's learnt in major healthcare orgs

Over the past seven years as Principal Data Architect for a multi-billion dollar health system in Southern California, I’ve learned many lessons about implementing and operationalizing data architecture and analytics product delivery in the modern healthcare setting.

I have fifteen years of Data Architecture, Engineering, and Analytics experience, with the last eight, focused on helping healthcare organizations in the provider & payer space. At Hakkoda, I’m the leader of our Data Engineering practice, a thought leader in our Healthcare practice, and helping our customers across government organizations, large health systems, and care management startups to architect and implement a modern data stack.

Here are my five biggest takeaways from my time at a major healthcare organization:

1. EHR data is just one piece of the data landscape

The move to major Electronic Health Records (EHRs) like Epic or Cerner will optimize workflows and consolidate data used for many purposes around the organization. This gives the impression that an EHR will solve most, if not all, of an organization’s current data problems.

While this may be true for many clinical and billing use cases, it will not solve many data challenges around:

  • Finance
  • Supply chain
  • Cost accounting
  • Staffing/HR
  • Marketing
  • Facilities management
  • Machine-generated data, such as audit logs, devices, and sensors
  • Unstructured data such as doctors’ notes, imaging files, emails, videos, etc.

Depending on your EHR implementation, the EHR might represent 60% of your overall data landscape, leaving 40% of the data needed to drive the organization still living outside the EHR. Certain types of data can be loaded into the EHR, such as patient satisfaction and external claims, but these are limited in size and scope.

Therefore, other business areas must devise a solution that will integrate EHR and non-EHR data—which, going forward, we’ll refer to as EHR Plus data.

Best Practice: An organization’s data strategy and architecture need to account for this EHR Plus data. By doing so, the organization’s strategy will be scalable and nimble enough to handle the variety, volume, and velocity of the requests for data they are asked to fulfill.

2. EHR software can be transformational to the organization, but analytics is just one focus of their designers

EHR companies are experts in producing software that can transform the organization towards delivering better patient care. For example, they have experience in clinical aspects, revenue cycle, patient experience, and population health. They produce analytics that integrate tightly with the EHR, but since analytics is not their sole focus their tooling may not always deliver the latest features. Or, perhaps upgrading to the latest version is an enterprise decision that may happen on a different timeline than that demanded by your analytics customers.

In this instance, your analytics team might be relying on out-of-date tooling. Some examples include a heavy reliance on databases hosted on-premise in SQL Server or Oracle, long-running overnight batch ETL processing, and extracts of data externally using SFTP. These introduce challenges around on-premise hardware sizing, resource contention, performance tuning, and waiting for batch loads to complete.

Modernizing the data stack and cloud migration is surely a large part of current and future EHR product roadmaps, but it’s worth considering the cost of waiting on future EHR releases versus the cost to build analytics solutions on a modern data stack in parallel to the efforts of your EHR vendor to deliver on immediate needs.

Best Practice: Adopt a modern data stack (such as Snowflake, Fivetran, Matillion, Sigma, DataRobot) and infrastructure (such as AWS, Azure, or GCP) for EHR Plus analytics and data science use cases. Have a set of easy-to-understand guiding principles and consider them for each use case. An example of a guiding principle is EHR First. Maximize your investment in your EHR, while also using the modern data stack for a comprehensive approach to data architecture and analytics.

3. Healthcare’s centralized data and analytics teams struggle to find and retain top talent

Analysis of EHR Plus data is currently being done by business analysts doing a large amount of data wrangling or engineering teams using siloed tools and processes because they can’t wait on IT to catch up and deliver for them.

Meanwhile, IT can’t keep pace because their team is stretched to its limit. Resource constraints, a lack of subject matter expertise, and the limitations around tooling described above make it difficult for IT to deliver rapidly on new requests from different areas of the business, each with its own unique needs.

Best Practice 1: Don’t try to be everything to everybody – Build a platform the business can depend on and empower them to use the platform to drive analysis and innovate. The tools and processes are centralized while the analysis and innovation are decentralized. As with good parenting, IT should only step in when business analysts are in danger (of producing incorrect data) or hopelessly stuck (unable to scale or deliver).

Best Practice 2: Select tools that highlight your existing talent and require skills that are easier to find in the market. Selecting Snowflake as a cloud data platform is a good example because of its heavy reliance on SQL skills. Other tools include low-code/no-code cloud ETL tools like Fivetran & Matillion and analytic tools like Sigma, which can analyze cloud data at scale with its spreadsheet-like interface.

4. Data teams are drinking from a firehose of urgent requests and don’t have enough time to prepare for the future

There is never a shortage of urgent requests for a Data & Analytics team supporting a healthcare organization.

An obvious example is assisting in the Pandemic Response, but there are many others (such as compliance requests, cyber security incidents, and new or updated governmental data needs). At the same time, Software as a service (SaaS) products are being added at a rapid pace to fulfill different gaps across the organization.

For Data & Analytics teams, this means new types of data sources and new integration use cases coming at an increasing pace. The team must account for this while also delivering on necessary urgent requests. This can make it difficult for them to prioritize the necessary time for research and development to make your data stack bulletproof today and into the future.

Best Practice: Implement a modern healthcare data stack that has the ability to solve today’s urgent needs and is already prepared to solve future needs, including the ability to:

  • Store & extract discrete data from unstructured data such as documents & images
  • Streaming use cases such as data from devices & monitors in a clinical setting
  • Sharing data with vendors and third parties
  • Developing machine learning models

5. Healthcare organizations want to be innovative with data and analytics, but they also want to stay aligned with their peers

Most healthcare organizations aspire to be a HIMSS AMAM stage 7 organization. They aspire to use data and analytics to predict and improve patient outcomes at the point of care. The reality is very few are doing this at scale.

Only four health systems achieved HIMSS AMAM Stage 7 in 2019.

Most are in some combination of levels 0 through 6 for many reasons, including those above.

Best Practice: To achieve HIMSS AMAM stage 7 status, the organization should achieve strong outcomes at levels 0-3 before progressing to higher levels. By adopting a modern data stack, the organization can create the momentum to advance to higher levels with:

  • A strong foundation of a robust data warehouse
  • The ability to deliver on a wide variety of reporting requests
  • Widespread adoption of data governance

To learn more about how your organization can take advantage of data and analytics innovations in healthcare, get in touch with a Hakkoda data architect.

Hakkōda 2024 Generative AI State of Data Report: 85% of Organizations Expect to Have Implemented Generative AI data tools by Year's End

Hakkōda 2024 Generative AI State...

Education, government and healthcare lag behind all other industries in Gen AI deployment.
Unlocking the Full Potential of Snowflake for SAP Data with Hakkōda

Unlocking the Full Potential of...

Although many organizations deploy tools alongside SAP to maintain enterprise functions, cross-functionality can be limited. Snowflake offers a platform where…
Summit 2024: Retail and CPG Data Insights and Trends from Modernization to AI

Summit 2024: Retail and CPG...

Explore some of the biggest retail and CPG data insights, themes, and takeaways from Data Cloud Summit 2024, including data…

Never miss an update​

Join our mailing list to stay updated with everything Hakkoda.

Ready to learn more?

Speak with one of our experts.