The Next Frontier: Implementing Image Classification on Snowflake

From detecting pneumonia to lung cancer and optimizing the diagnostic process, image classification models are revolutionizing how healthcare organizations address and overcome the most complex diagnostic challenges.
November 16, 2023
Hakkoda - Image Classification - Thumbnail
Dana George

Dana George

Lead Data Scientist at Hakkoda

Karuna Nadadur

Karuna Nadadur

Senior Data Cloud Architect at Snowflake

About 80% of patients’ health data is unstructured, making it a significant challenge to work with. Snowflake and Hakkoda have teamed up to bring you an end-to-end image classification solution that leverages accelerated compute power to perform image-based analysis to extract insights that you need from your data to better care for your patients. 

Snowflake’s solution, a PyTorch Model for Medical Image Classification, leverages a combination of Snowpark Container Services and the latest Snowflake ML Features such as Model Registry to classify image data and, for example, detect the presence of pneumonia on chest x-rays. Before this kind of model can be deployed for inference, however, it must be trained on customer images and tested for performance. Fortunately, with Hakkoda’s team of Snowflake experts, your internal data team will be provided with an end-to-end AIML solution with the support you need along the way—from model build, to model implementation and training, all the way to model impact tracking.

This blog will walk you through what this model does, how it does it, and how Hakkoda can support your implementation of this model or one just like it. 

The Image Classification Model

Healthcare organizations are looking to extract insights from a wide array of data to better understand patient health profiles and drive health outcomes. Snowflake’s Healthcare & Life Sciences Data Cloud supports multi-modal health data, enabling healthcare organizations to store, enrich, and derive insights from sources such as clinical documents and DICOM (Digital Imaging and Communications in Medicine standard) files. 

Snowflake has built a solution that uses deep learning to classify images by leveraging Snowflake’s latest ML features and capabilities. The solution enables the user to load DICOM image data, train the model using Pytorch within Snowpark Container Services, and easily deploy a model using Snowflake’s native model registry. To give an example in a clinical context, lung x-ray images can be classified to detect the presence of pneumonia.

The solution is trained on a PyTorch image classification model on GPUs through transfer learning from torchvision (Resnet, Vgg, etc.), and uses the newly announced Private Preview feature Snowpark Container Services in Snowflake. The solution then logs and manages the trained model in the Snowpark Model Registry, also in private preview, and deploys in Snowflake Warehouse (CPUs) or Snowflake Compute pools (CPU or GPU) for inference. Finally, the model can be run within Snowflake using a frontend app built using Streamlit in Snowflake (SiS).

So, once you have a model built, what’s next? How will you train the model on your organization’s medical images for your particular use case? How will you measure performance? How will this model be kept up to date? Where does the model live? What type of team do you need to manage the model? How will you make sure this model is actually helping patients?

How Hakkoda Can Help You Tackle Unstructured Data with Image Classification Solutions

The build and application of this type of solution is re-shaping the future of healthcare, but only for organizations that invest in the right talent, tools, and techniques to capitalize on these emerging opportunities. 

For healthcare organizations who still depend on legacy infrastructure and on-premise electronic medical record systems, leveraging Snowflake with machine learning begins with the migration of critical data and workloads into the cloud. Leveraging the flexibility, scalability, and cost-efficiency of cloud infrastructure allows healthcare organizations to deploy advanced ML capabilities and operationalize data in new ways to enhance patient care and drive operational efficiency. 

Using Hakkoda’s team of data experts on top of Snowflake’s Data Cloud, which brings together data warehouse and data science workloads, clinical teams see more value faster and ultimately save patient lives.

Healthcare organizations migrating to the cloud to support a more data-driven approach will also need to address and overcome challenges around data cleanliness, organization, and governance to achieve success with ML models like the solution above while also maintaining compliance with HIPAA and other data security/privacy regulations.


Of course, like many healthcare problems, when implemented in the real world, a custom solution will be needed. That’s the power of Snowflake and Hakkoda – we understand that a model cannot be one-size-fits-all. So, whether your goal is to detect lung cancer or concussions, we can support you. For a solution like this one, the process might look like the following:


  • Identify the problem and objective of the model and ensure that it is tuned to the patient population it will serve. 

Testing and Validation of Process

  • Replicate the Pytorch model in the new environment. Account for new dependencies, new data, new architecture, and access.
  • Validate the replicated model from start to finish.

Deploy to Baseline

  • Add and configure custom features.
  • Deploy the Pytorch model on the new data.
  • Explain the outcome.


  • Extract insights from the model while incorporating derived data into workflows and driving evidence based decision-making.


  • Continuously tune and monitor the model for alignment with operational and clinical objectives.

Hakkoda offers flexible teams, ramping up or down as your organization’s needs change and as projects progress from building to monitoring. This means that we can start with a larger team when hands on the ground are needed most, then scale down as time goes on to key players needed to implement a solution like this one.

Hakkoda offers all the skill sets your team would need—from data engineers to set up the data, to data scientists to build it, to industry experts and change management leads to take a solution like this one from a great idea to a fully integrated tool that can be used in clinician workflows.

Start Your ML Journey with Hakkoda

At Hakkoda, we’re on a mission to ignite the power of data and empower healthcare organizations to operate more efficiently while driving innovation with the most cutting-edge AI technologies. 

Hakkoda provides the solutions, experience, and technical expertise healthcare organizations need to migrate data into the Snowflake Data Cloud, automate data pipelines and workloads, overcome challenges around data cleanliness, organization, and governance, and open the path to AI-powered innovation.

Ready to learn more? Contact our data experts to discover how Hakkoda can help drive digital innovation at your healthcare organization.

FSI State of Data: Financial Services and Insurance Orgs are Falling Behind in Data Modernization. Here’s How They Can Catch Up.

FSI State of Data: Financial...

Hakkoda’s Financial Services and Insurance State of Data Report reveals key gaps in data strategy, barriers to data modernization, and…
Hakkōda 2024 Financial Services & Insurance State of Data Report: 97% Say Generative AI Matters for Success

Hakkōda 2024 Financial Services &...

Hakkoda releases its 2024 Financial Services & Insurance State of Data Report, surveying 145 director to CEO level data leaders…
Rainfall MDM

Rainfall MDM

The Sigma OCF Connector facilitates seamless integration between Alation's Data Catalog and Sigma, enabling comprehensive data discovery, governance, and search…

Never miss an update​

Join our mailing list to stay updated with everything Hakkoda.

Ready to learn more?

Speak with one of our experts.