Empowering AI-Driven Drug Discovery: Overcoming Challenges... Marius Tanawa Tsamo & Gustav Rasmussen

Empowering AI-Driven Drug Discovery: Overcoming Challenges and Scaling AI with Kubernetes

Speakers: Gustav Rasmussen, Tech Lead, Containerization Team, Novonordisk R&D; Marius Tanawa Tsamo, Senior Platform Engineer, Novonordisk R&D

Conference: KubeCon EU

YouTube: https://www.youtube.com/watch?v=FC5TAGsBbRQ

Overview

This presentation, delivered by Gustav Rasmussen and Marius Tanawa Tsamo from Novonordisk Research and Development, delves into the critical endeavor of extending Kubernetes with advanced AI capabilities to accelerate drug discovery. Novo Nordisk, a global pharmaceutical leader specializing in treatments for diabetes, obesity, and other serious chronic diseases, faces an exponential surge in data generation. In this context, AI is not merely an advantage but a "must-win battle" to efficiently process vast datasets, unlock new insights, and ultimately bring life-changing treatments to patients faster. The talk outlines a comprehensive strategy to integrate state-of-the-art AI infrastructure, leveraging a world-class supercomputer, into their R&D operations.

The core challenge lies in bridging the gap between highly specialized, resource-intensive AI models and the daily workflows of scientific researchers, who are domain experts rather than platform engineers. Rasmussen and Tsamo articulate how they are building a robust, scalable, and user-friendly platform atop Kubernetes to democratize access to powerful AI tools. This involves tackling complex issues such as managing massive container images, optimizing data access through concepts like data gravity, and harnessing the immense power of NVIDIA's cutting-edge GPU architectures, all while ensuring operational compliance and efficiency across a hybrid cloud environment.

The presentation provides a detailed technical blueprint for scaling AI, from foundational data principles to the intricacies of supercomputing hardware and advanced Kubernetes orchestration. It highlights how pharmaceutical companies can leverage modern cloud-native technologies to empower their researchers, accelerate the drug discovery pipeline, and ultimately transform healthcare outcomes. The speakers emphasize that effective AI adoption requires not just powerful hardware, but also a thoughtful approach to data management, platform design, and user experience.

Background

▶ Watch: Introduction to talk, speakers, and agenda (0:00)

Novo Nordisk operates within a rapidly evolving technological landscape where data is growing at an exponential rate, measured in zettabytes annually. This deluge of information presents both a challenge and an immense opportunity, particularly within pharmaceutical research and development. AI has emerged as a transformative force, capable of scanning and interpreting massive datasets at speeds and scales far beyond human capacity, excelling in areas like image recognition, language understanding, and reading comprehension. For Novo Nordisk, efficiently adopting and scaling AI across the entire company is deemed a strategic imperative.

The application of AI in research exists on a spectrum. At one end are "citizen AI" tools like ChatGPT or Co-pilot, widely available for common tasks such as text summarization or code generation. However, pharmaceutical research often deals with highly heterogeneous and unique data, necessitating more bespoke or custom AI models. These can range from knowledge graphs to advanced predictive models like AlphaFold, which can predict the three-dimensional structure of proteins from amino acid sequences. Each point on this spectrum introduces its own set of challenges related to hosting, connectivity, compute resources, and data management. AI's potential spans the entire pharmaceutical value chain, from optimizing communication with healthcare professionals and predicting stockouts, to detecting broken vials in manufacturing and optimizing clinical trials. The most exciting applications, however, are found in the discovery phase, where generative AI can scan vast numbers of potential targets, identifying leads with precise attributes for preclinical and clinical development.

To achieve scaled AI, Novo Nordisk employs a foundational equation: Business Value = Fair Data + Right Mindset + State-of-the-Art Infrastructure. Fair Data – meaning Findable, Accessible, Interoperable, and Reproducible – forms the bedrock. This is combined with cross-functional teams adopting an AI-first mindset, often working in rapid hackathons or sprints. The final component is providing the cutting-edge infrastructure necessary to support these efforts.

A significant challenge in this infrastructure lies with container images for Machine Learning (ML). These images can be exceptionally large, often exceeding 30GB, and sometimes even contain embedded data. Such large sizes introduce considerable latency during the container deployment process, from pod creation and node selection to image pulling. To mitigate this, Novo Nordisk first attempts to slim down image sizes. For persistent large images, their solution involves deploying Harbor as a proxy and cache, positioned geographically close to their compute resources, specifically the Geon cluster. This Harbor instance is integrated with JFrog Artifactory SAS, aligning with their corporate strategy of prioritizing Software-as-a-Service solutions.

Beyond size, the quality and correctness of data are paramount. The talk humorously references an early version of Google's Gemini model, which, trained on poor data (e.g., Reddit posts), suggested "glue" as the best pizza topping for texture. To avoid such "bizarre outcomes," Novo Nordisk emphasizes ensuring data correctness and quality, going beyond just the FAIR principles.

Another critical data aspect is data gravity, a concept coined by Dave McCroy. Similar to physical gravity, data exerts an intrinsic pull on related data, applications, and services. Embracing data gravity can simplify security and significantly boost performance by keeping compute close to the data it needs. To manage data globally for all users, Novo Nordisk employs a three-tier data strategy based on usage frequency:

• Hot tier: For data requiring immediate availability, utilizing Vea storage. This tier has the highest data gravity.
• Warm tier: For frequently accessed data, employing Cumulo. This data includes some redundancy and has a slightly smaller data gravity pool than the hot tier.
• Cold tier: For historical and compliance data, using an archival solution. This tier has the lowest data gravity and can be stored in fewer, more centralized locations, as frequent access is not required.

Key Findings

▶ Watch: AI spectrum: citizen AI to bespoke models like AlphaFold (2:40)

The talk highlights several pivotal findings in Novo Nordisk's journey to empower AI-driven drug discovery:

• AI as a Strategic Imperative: AI is not optional but a "must-win battle" for pharmaceutical companies like Novo Nordisk due to the exponential growth of scientific data and AI's superior capability in data processing and pattern recognition.
• Specialized AI for Research: While citizen AI tools are useful, bespoke and custom AI models (e.g., knowledge graphs, AlphaFold) are indispensable for handling the heterogeneous and complex data inherent in drug discovery.
• Infrastructure Challenges of Large Models: Large ML container images (often >30GB) and the principle of data gravity present significant infrastructure challenges, necessitating solutions like local image caching (Harbor) and tiered data storage.
• Geon Supercomputer as the Backbone: The Geon supercomputer in Denmark, ranked 21st globally and built as an NVIDIA DGX SuperPOD with approximately 200 H100 nodes, serves as the core high-performance compute engine for their AI initiatives.
• NVIDIA Hopper Architecture's Impact: Key features of the NVIDIA Hopper architecture (H100/H200 GPUs), specifically Multi-Instance GPU (MIG) for efficient inference and Scalable Interconnect (NVLink/NVswitch) for high-speed training, are critical enablers for next-generation AI workloads.
• Kubernetes for Abstraction and Orchestration: Kubernetes is the chosen platform to abstract the complexity of the underlying supercomputing hardware, providing a manageable and scalable environment for AI workloads.
• Workload Managers are Essential for HPC: Standard Kubernetes scheduling falls short for High-Performance Compute (HPC) workloads. Specialized workload managers (like Run:AI) are crucial for features such as multiple queuing systems, dynamic resource allocation, and optimizing GPU utilization.
• User-Centric Platform Design: Empowering researchers, who are domain experts and not platform engineers, requires a self-service, user-friendly interface that simplifies access to powerful AI tools and underlying infrastructure.
• Hybrid Cluster Architecture for Flexibility: A hybrid approach, combining bare metal, virtual machines, and various Kubernetes distributions (e.g., kubeadm, RKE, AKS, EKS), is necessary to support diverse development and production needs across different environments.

Technical Deep Dive

▶ Watch: Overcoming latency with large ML container images (4:45)

Novo Nordisk's strategy for scaling AI is deeply rooted in providing a robust yet user-friendly technical foundation for its researchers. The ideal user journey, exemplified by "Eda" the researcher, demands a platform that allows compliant authentication, integration of preferred tools like Jupyter notebooks, the ability to train and deploy Large Language Models (LLMs) on specific datasets, and real-time experiment tracking and monitoring. Crucially, Eda is a scientist, not a DevOps or platform engineer; therefore, the platform must seamlessly handle hardware display, smart resource allocation, and provide a self-service experience to accelerate innovation.

The cornerstone of this infrastructure is Geon, a powerful supercomputer located in Denmark. Ranked as the 21st largest supercomputer globally, Geon is an NVIDIA DGX SuperPOD designed specifically for AI at scale. It boasts approximately 200 H100 nodes, each equipped with eight GPUs and nearly 700GB of RAM. These nodes are interconnected with InfiniBand Quantum 2, a high-speed networking solution vital for distributed AI training. Novo Nordisk's immediate objective is to build an inference cluster directly adjacent to Geon, allowing researchers to access its immense computational power without dealing with the underlying complexities.

The NVIDIA Hopper architecture (H100 and H200 GPUs) that powers Geon offers three major features: Confidential Compute, Multi-Instance GPU (MIG), and Scalable Interconnect. While Confidential Compute was mentioned, the talk focused on MIG and Scalable Interconnect. MIG allows a single GPU to be securely partitioned into smaller, isolated GPU instances. This is particularly beneficial for inference workloads, enabling multiple users or applications to share a single physical GPU efficiently, addressing challenges associated with time-slicing and resource contention. The Scalable Interconnect, facilitated by technologies like NVLink and NVswitch, dramatically increases GPU-to-GPU communication speeds. This bypasses the traditional bottlenecks of PCI Express, enabling rapid data exchange between GPUs, even across different nodes, which is essential for large-scale distributed training.

To abstract this powerful hardware and provide a manageable environment for researchers, Novo Nordisk has chosen Kubernetes. They are building a Kubernetes cluster on H200 nodes and CPU nodes within the inference cluster. Key components of this Kubernetes deployment include:

• Harbor: As previously mentioned, Harbor acts as a crucial proxy and cache for large ML container images, positioned close to the compute resources to minimize latency.
• Workload Managers: Recognizing that native Kubernetes scheduling is insufficient for the demands of HPC and AI workloads, specialized workload managers are integrated. These provide advanced features like multiple queuing systems and more sophisticated resource allocation. The talk referenced the importance of such managers, drawing parallels to systems like Slurm for HPC.
• NVIDIA Operators: To fully leverage the NVIDIA hardware, the NVIDIA GPU Operator and NVIDIA Networking Operator are deployed. These operators automate the management of NVIDIA GPUs, networking components, and associated drivers within the Kubernetes cluster, including tasks like Node Feature Discovery to label nodes based on their hardware capabilities.

The overall architecture is a sophisticated hybrid cluster designed for flexibility and scale. The base layer consists of heterogeneous bare metal nodes, some optimized for GPU compute, others for CPU. Above this, a control plane, system tools, and various controllers run across a range of operating systems, including NVIDIA DGX Ubuntu and general Ubuntu for VMs. The Kubernetes layer itself is flexible, comprising combinations of container runtimes with kubeadm for custom deployments, or managed services like Rancher Kubernetes Engine (RKE), Azure Kubernetes Service (AKS), or AWS Elastic Kubernetes Service (EKS) for cloud-based development. On top of this, essential system tools like Harbor for container images and Argo CD for GitOps-driven deployments are hosted. This multi-layered approach ensures that researchers, who are domain experts rather than Kubernetes specialists, can interact with a simplified, self-service interface, allowing them to focus on their scientific endeavors while the platform handles the underlying complexity.

Demo / Proof of Concept

▶ Watch: Importance of data quality and the concept of data gravity (6:00)

The demonstration aimed to showcase how a researcher like Eda could interact with the powerful AI infrastructure without needing deep Kubernetes knowledge. The demo environment was an AKS cluster featuring two V100 GPU nodes (distinct from the H100/H200 nodes of Geon, likely for demo practicality) and several CPU nodes, all pre-configured and ready for workloads.

The initial part of the demo highlighted the underlying infrastructure. It confirmed that the NVIDIA Networking Operator was running, performing Node Feature Discovery to automatically label the nodes with their specific hardware capabilities, such as v100 for the GPU nodes. This automation is crucial for intelligent workload placement.

Next, the deployment of a Large Language Model (LLM), specifically Llama 3, was demonstrated. This was achieved using a Helm chart with predefined values, showcasing a common method for packaging and deploying applications in Kubernetes. The deployment created an endpoint, and the Llama 3 model was confirmed to be running, configured to utilize one GPU core. A simple query was then posed to Llama 3: "I want to deploy LLMs in Kubernetes using awesome platform. What are the options out there?" The model processed the query and returned an output, demonstrating its functionality.

The core of the demo, however, was the introduction of the user-friendly platform called Run:AI (an NVIDIA-acquired platform). This platform provides a graphical interface that abstracts Kubernetes complexities from the researcher. Upon authenticating, a user like Eda could see the available GPU resources (e.g., two GPUs on the node) and was empowered to allocate fractions of these GPUs to her workloads. The interface also displayed all running workloads, providing an overview of resource utilization.

Crucially, Run:AI demonstrated its workload manager capabilities. Instead of interacting with kubectl or managing Helm charts directly, Eda could deploy an LLM for inference with just a few clicks. She would select the desired workload (e.g., LLM inference) and choose her model, and the platform would handle the underlying deployment.

A significant point reiterated during the demo was the importance of image caching. While deploying Llama 3, it was observed that pulling the image without a local cache could take 20 minutes or more. This starkly illustrated the latency challenge associated with large ML images and reinforced the necessity of solutions like Harbor as a local proxy and cache, ensuring instant fetching of images and significantly accelerating the development and deployment cycles. This practical demonstration underscored how the platform directly addresses the technical challenges discussed earlier, empowering researchers with speed and simplicity.

Defensive Implications

▶ Watch: Handover to Marius: user journeys and Geon supercomputer (8:00)

While the talk primarily focuses on empowering AI-driven drug discovery through advanced infrastructure rather than traditional cybersecurity, several aspects of Novo Nordisk's approach inherently contribute to a robust and resilient operational posture, which can be considered "defensive" in the context of critical research infrastructure.

First, the emphasis on Fair Data principles (Findable, Accessible, Interoperable, Reproducible) combined with strict data quality and correctness checks serves as a fundamental defense against "garbage in, garbage out" scenarios. In drug discovery, flawed data can lead to erroneous scientific conclusions, wasted resources, and potentially dangerous outcomes. By ensuring data integrity and quality, the platform inherently defends the validity of research findings.

The strategic implementation of data gravity and the three-tier data strategy also has defensive implications. By keeping frequently accessed and critical data geographically close to the compute resources, Novo Nordisk minimizes data movement across networks. This not only boosts performance but also reduces the attack surface associated with data in transit, simplifying security controls and limiting potential exposure. The tiered approach ensures that sensitive or compliance-critical historical data is stored in highly controlled, archival solutions with reduced accessibility, aligning with data governance and regulatory defense.

The use of Harbor as a local image proxy and cache acts as a defense against network dependency and latency. In an environment where ML container images can be tens of gigabytes, a slow or unreliable network pull can halt research. Harbor ensures consistent, high-speed access to approved images, preventing operational delays and potential disruptions to critical workflows. It also provides a layer of control over which images are used, acting as a gatekeeper for approved software versions.

Furthermore, the adoption of workload managers like Run:AI on Kubernetes defends against inefficient resource utilization and operational complexity. Without such tools, researchers might struggle with resource contention, leading to suboptimal GPU usage or system instability. By abstracting the intricacies of Kubernetes and providing dynamic resource allocation, the platform defends researchers from infrastructure-related frustrations, allowing them to focus on science and minimizing the risk of misconfigurations or resource starvation that could impede progress.

Finally, the focus on compliant authentication and a user-friendly, self-service platform contributes to a strong security posture by reducing the need for researchers to interact directly with low-level infrastructure. This minimizes the risk of human error or misconfiguration, which are common vectors for security incidents. By providing controlled, authenticated access to powerful tools through a managed interface, Novo Nordisk defends its sensitive research environment from unauthorized access and ensures that operations align with corporate governance and regulatory requirements.

Key Takeaways

• AI is a Strategic Imperative for Pharma: Novo Nordisk views AI as critical for managing exponential data growth and accelerating drug discovery, requiring bespoke models and scalable infrastructure beyond general-purpose AI.
• Leveraging Supercomputing Power: The Geon Supercomputer, an NVIDIA DGX SuperPOD with ~200 H100 nodes and InfiniBand Quantum 2 interconnect, provides the foundational high-performance compute for their AI initiatives.
• Kubernetes as the Orchestration Layer: Kubernetes, augmented by NVIDIA GPU/Networking Operators and specialized workload managers like Run:AI, is crucial for abstracting the complexity of HPC hardware and providing a manageable platform for AI workloads.
• Addressing Infrastructure Bottlenecks: Challenges like large ML container images are mitigated through local caching with Harbor, while data gravity is managed via a three-tier storage strategy to optimize performance and security.
• Empowering Researchers with User-Friendly Platforms: Creating self-service, intuitive interfaces (e.g., Run:AI) is essential to enable domain experts, who are not platform engineers, to efficiently utilize powerful AI tools and accelerate innovation.
• Hybrid Cluster Architecture for Flexibility: A sophisticated hybrid infrastructure combining bare metal, VMs, and various Kubernetes distributions (e.g., kubeadm, RKE, AKS, EKS) provides the necessary flexibility for diverse development and production environments.

About the Speaker(s)

Gustav Rasmussen serves as the Tech Lead for the Containerization Team at Novonordisk Research and Development. His expertise lies in building and managing container-based platforms, crucial for supporting the company's advanced AI and ML initiatives.

Marius Tanawa Tsamo is a Senior Platform Engineer at Novonordisk Research and Development. He plays a key role in designing and implementing the technical infrastructure that empowers researchers, focusing on extending Kubernetes capabilities for AI workloads and optimizing user journeys.