In the data engineering world, there’s a saying – “data’s data.”
As a data professional, when somebody asks you if you can do something with their specific type of data, you generally shrug and say, “Sure, data’s data.”
And generally, that’s true – most industries do pretty similar things. They order things, they sell things, they keep track of things, and they keep a balance. Those things can be products, money, customers, employees, etc. But all of their data is pretty similar, even across many different industries.
If you’re a data engineer, for most industries, you know you’re likely dealing with transactions of some type, master data which defines what or who is involved in the transaction, plus some reference data to help categorize these things, with transfer and storage patterns most are familiar with.
Data in Healthcare
But there are a couple of industries where “data’s data” doesn’t really apply, due to the complexity, variety, and urgency of what is being tracked and measured. A specialist’s knowledge is required, and can only be built over time in the trenches, due to the expansive and detailed nature of the data. Healthcare is one of those industries.
If you haven’t worked with healthcare data before, the first thing I would tell you is that the health of a person is not limited to a single transaction. So transactions, by their generally accepted definition, might exist, but they are rarely self-contained.
The Complexity of Data in Healthcare
For example, you might go visit your doctor and she might diagnose your problem immediately and prescribe a course of treatment that works so you never have to go back again. That’s the best-case scenario – one visit, one diagnosis, one prescription, done.
However, I think we all know that’s pretty rare. And, even if that is the case, the data involved in that one transaction also needs to be made available to the pharmacist so they can fulfill your prescription, so your system must interface with the pharmacy system, probably following HL7 and NCPDP protocols, which you’ll need to know about.
The coverage provider (aka insurer) needs to be informed so that they can pay, so your system should be able to exchange data with them using the X12N EDI standard, too. The insurer might also have additional documentation requirements of their own, like a PA, or prior authorization, for a prescription.
The diagnosis needs to conform to the ICD-10 standard in order for both of these systems to correctly interpret it, so your system must be up to date with those codes.
Also, you definitely want to see an accredited provider with prescribing privileges, so that data needs to be verified with the accrediting service as well.
All these systems need to reference SNOMED so that the records are correctly coded and the data can be used by an electronic health record. Oh, and the electronic health record is likely a custom system like EPIC that doesn’t follow standard relational data patterns so experience with EHR systems is required to understand how it’s storing data if you want to use it for reporting later. And much more. And that’s all for the easiest, most basic of visits.
Scratching beyond the surface of healthcare data
What if your doctor finds something that will take a long, or even permanent, course of treatment? Like an autoimmune disorder or degenerative nerve condition? The records for all of those visits need to be tied together coherently so your doctor can discover everything they need to know.
- What if the treatment leads to side effects that need to be tracked and managed?
- Who is checking to make sure multiple medications don’t interact or result in an adverse event for a patient?
- Is somebody checking to make sure schedule or resistance-invoking drugs, such as opioids or antibiotics, are being used responsibly?
- What if the condition results in hospitalization?
- Will the doctors at the ER or hospital be able to see what treatments are ongoing? Or, have been tried already?
- Will all of the systems involved in the patient’s care be able to speak to each other?
All of these examples are merely scratching the surface at a very basic level with healthcare data. This list goes on and on, with cascading levels of complexity and ever-expanding scope of considerations. Healthcare data is not “just” data, it’s more:
- Complicated
- Layered
- Regulated
- Granular
- Urgent
- Sensitive
- Life-or-death than almost any other type of data
Healthcare experience matters
This complexity and urgency have caused healthcare organizations to prioritize delivery over design. This results in some breathtaking data landscapes characterized by vast sprawl and invisible navigational hazards. Reporting can be slow due to duplicated or haphazardly organized structures. Results can be inconsistent or misleading. And, analysts can spend a lot of time finding the “right” sources and interpreting them correctly. Especially, if they don’t have previous experience with healthcare data or know to contextualize them properly.
Hakkoda has built a team of Snowflake experts with decades of experience in healthcare data. Let us help you get a handle on your data sprawl with a team of seasoned healthcare data veterans who are excited about the opportunities the Snowflake Healthcare and Life Sciences Data Cloud offers for driving greater value and insights from your healthcare data. To learn more, get in touch with a Hakkoda healthcare data expert.
If you’d like to join Hakkoda, check out our current open positions in the US and Costa Rica.
