Alex Borbón Porras
Operations Consultant Analyst at Hakkoda
Theodore Caulton
Data Engineer at Hakkoda
Murali Gandhirajan
Healthcare and Life Sciences Field CTO at Snowflake
Predictive models of healthcare provider resignation are a critical tool for hospitals today. Healthcare is a specialized field that requires highly competent, trained, and experienced staff to provide quality care to patients. That said, staff retention in the healthcare sector can be an ongoing challenge due to several factors.
In addition to my work as an Operations Consultant Analyst at Hakkoda, I’ve also worked weekends as an MD, where I have lived and witnessed how burnout and stress become common problems for healthcare professionals firsthand, especially during times of crisis like the COVID-19 pandemic. Being on the front lines at the height of the pandemic was an experience unlike any other, and the constant mix of fear, determination, and compassion formed a unique emotional landscape. The weight of responsibility pressed heavily on our shoulders as we cared for our first priority–our patients–but we also needed to mask how heavily it weighed on us in order to comfort patients who were often scared and isolated from their loved ones.
The PPE-clad interactions with fellow healthcare workers conveyed solidarity in the face of an invisible threat. It was a surreal blend of exhaustion and resilience and every moment carried the gravity of life and death. No matter how hard you studied and how many years of experience you have on your background, this was an experience no one was prepared to live for such a long period of time.
Medical and Nursing Staff Attrition: A Global Crisis
Medical and nursing staff often face overwhelming workloads and stressful working conditions. These workloads are taxing to the human body, and create a relentless and demanding retention problem in medical professions not just in Costa Rica, where I witnessed it up close, but globally. Each day feels like a marathon of high-stakes decisions, patient care, and the juggling of responsibilities. The weight of lives in your hands, compounded by the never-ending stream of paperwork, long shifts, and emotional strain, can lead to a profound sense of exhaustion and burnout very quickly and in front of everyone’s sight. Even with the deep sense of purpose and fulfillment you get from knowing you’re making a difference in people’s lives, these stressors can lead to long-term job dissatisfaction, depression, and, ultimately, resignation.
The demand for healthcare services, meanwhile, is continuous. This means hospitals must be prepared to cover shifts at all times, which translates to medical and nursing staff working long hours–sometimes without breaks–and dealing with a constant influx of patients, each requiring undivided attention. The challenge of balancing these emotional and physical demands can be overwhelming, and the responsibility to provide quality care under such circumstances can be daunting. Unexpected resignations can lead to staffing coverage issues, affecting the quality of care and increasing the workload for those who remain in the hospital.
To add to increasing demand for healthcare services and prevalence of clinician burnout, there are also challenges with academic institutions’ ability to accept more students in nursing school due to a shortage of teaching faculty. This slows down the supply of new talent into this space.
With this backdrop, healthcare organizations are now more than ever looking for innovative ways to address this crisis, including the use of data and AI to track employee attrition, proactively predict potential attrites, and design interventions to improve employee experience.
Nurse Attrition Streamlit and the Benefits of Predictive Models
This is where technologies like the Nurse Attrition Streamlit solution and the Snowflake Data Cloud come into play. Using data such as employee turnover rates, job satisfaction levels, distance to work from home, compensation, and other relevant burnout indicators, these models can predict when employees are most likely to consider quitting. This allows hospitals to take proactive steps to address issues before they become actual resignations.
The benefits of these predictive models are multiple. They help hospitals identify specific areas where intervention is needed, whether through improving working conditions, implementing staff support programs, or redistributing workload. This can increase job satisfaction and reduce the likelihood of resignations.
At this point it should be abundantly clear the human cost of high nursing attrition on both employees and residents. The problem of solving high attrition is made difficult by two factors. One: it can be difficult to establish which metrics are useful for uncovering a current problem or forecasting future issues. Two: stakeholders who are typically in charge of reducing attrition are not technical enough to do exploratory analysis based on their industry knowledge, and are thus at the mercy of a BI or Analyst team which may lack the same industry knowledge or be slow to deliver. For an issue like attrition which is quick to snowball, slow time-to-insight is unacceptable.
Improved Time-to-Insight with Hakkōda’s Nurse Attrition Streamlit
Striking a balance between technical knowledge and industry experience, Hakkoda has derived a solution that tackles both of the issues above by creating dynamic dashboards that enable faster time-to-insights by pairing a more traditional BI model with a cutting edge AI tool that enables non-technical stakeholders to ask key questions about data using human language.
The only way to get ahead of attrition is to understand the data and get to an employee before they leave. For hiring managers, key performance indicators need to be accessible and current at all times to allow for accurate decisions and real-time workload changes.
Hakkoda’s Nurse Attrition Streamlit solution facilitates this kind of quickturn decision-making by directly streaming all employee data straight from the Snowflake Data Cloud into Streamlit Cloud. This allows for analysts to build and display not only visuals, but complex models that can make accurate predictions about future attrition. Our solution aims to build and display metrics most important to tracking attrition and employee discontent, allowing stakeholders to quickly take action. By streaming data directly from Snowflake, stakeholders also have consistent access to up-to-date data as soon as it is made available.
Accessible Analytics for Non-Technical Stakeholders With Predictive Models and OpenAI
The above solution makes access to key metrics possible and enables stakeholders to make more informed decisions, but does not yet solve the second piece of our equation: enabling non-technical stakeholders to ask further questions about our data.
For technical analysts, it may seem simple to load a dataset and begin querying the data or building a complex model but this can seem like a monumental task, especially for employees who have the strongest grasp over the context of the data. What about hiring managers who need extra analysis or have specific questions about their employee data? Traditionally, hiring managers would need to contact their BI team, put in a ticket for a new feature, and wait several days for development, testing, and a push to production.
Hakkoda’s virtual business analyst integrated into our nurse attrition solution enables fast and secure answers to any question you might have about the data. OpenAI’s LLM model allows for non-technical stakeholders to ask complex questions about the data using human language, and our model intelligently responds with a concise, data-driven answer. For a problem like attrition, where time is of the essence, enabling stakeholders to quickly query data without additional in-house technical support, discover problems, and solve them before they lead to attrition is a game changer.
It is important to understand the importance of having both BI dashboards that display complex data science models and the features of a virtual business analyst. Chiefly, it is worth noting that data science models unearth underlying causal links between features in data in a way that LLMs cannot yet do. This makes them more time consuming than language-based queries, but nonetheless extremely important for fully understanding what the data actually says. Conversely, knowledgeable stakeholders may have exploratory questions about their data that need to be answered in real time: a scenario in which our virtual business analyst can be leveraged to complement the work done by data scientists.
Building Smarter Data Solutions with Hakkōda
At Hakkoda, we believe that data holds the power to solve humanity’s toughest challenges and make the world a better place. Our scalable data teams bring expertise from across the modern data stack together with deep industry experience in healthcare and other heavily regulated industries to design innovative data solutions that enable our clients to make informed decisions faster and improve outcomes for key stakeholders like medical professionals and their patients.
Ready to start your data innovation journey? Let’s talk.
