Accelerating Science and Engineering With NVIDIA CUDA-X Libraries | NVIDIA GTC D.C.

Accelerating Science and Engineering With NVIDIA CUDA-X Libraries

Speakers: NVIDIA (NVIDIA)

Conference: NVIDIA GTC

YouTube: https://www.youtube.com/watch?v=jbpzSnQxC5c

Overview

This talk delves into CUDA-X, NVIDIA's comprehensive suite of libraries, frameworks, and AI models designed to accelerate every conceivable workload across diverse scientific and engineering domains. Building upon the foundational CUDA platform, which has married NVIDIA hardware to software for over two decades, CUDA-X represents the next logical evolution, aiming to empower millions of developers to build transformative applications. The presentation highlights NVIDIA's strategic shift from merely providing powerful GPUs to offering a rich software ecosystem that directly addresses complex computational challenges in fields ranging from industrial engineering and climate modeling to data science, computational biology, and quantum computing.

The core premise of CUDA-X is to move beyond accelerating single applications, instead developing reusable libraries and frameworks that can be integrated across a multitude of applications and workflows. This approach is crucial for unlocking orders of magnitude gains in speed and scale, ultimately closing the gap between high-fidelity simulation and real-time digital twins. The talk showcases specific CUDA-X components and their impact, demonstrating how NVIDIA is enabling partners and developers to accelerate scientific discovery, reshape industries, and push the boundaries of AI, simulation, and high-performance computing.

The importance of CUDA-X lies in its ability to democratize access to advanced GPU acceleration, abstracting away much of the underlying complexity. By providing pre-built, optimized solutions for common computational patterns – such as sparse matrix solvers, AI physics models, data frame operations, and quantum simulations – NVIDIA empowers a broader developer base to leverage its hardware without deep expertise in GPU programming. This strategy not only enhances performance for existing applications but also unlocks entirely new workflows, such as interactive real-time design exploration and hyper-local weather forecasting, driving innovation across critical sectors.

Background

▶ Watch: Introduction to CUDA-X vision and developer enablement (0:00)

The genesis of CUDA-X is rooted in NVIDIA's long-standing commitment to parallel computing and its foundational CUDA platform. Introduced over 20 years ago, CUDA revolutionized general-purpose GPU computing, providing a software layer that allowed developers to harness the parallel processing power of NVIDIA GPUs for tasks beyond graphics rendering. This innovation laid the groundwork for the modern AI and HPC revolution, enabling significant speedups for computationally intensive algorithms.

However, as the complexity and diversity of scientific and engineering workloads grew, a simple hardware-software interface like CUDA became insufficient to address the myriad challenges faced by developers across different domains. Many applications involve highly specialized computational patterns—from sparse linear algebra in structural analysis to complex fluid dynamics, large-scale optimization problems, and intricate molecular simulations. Accelerating these diverse workloads efficiently and effectively required more than just raw GPU compute; it demanded domain-specific software building blocks.

This need spurred the development of CUDA-X. The problem it addresses is multifold:

1. Developer Accessibility: Not all developers are GPU programming experts. CUDA-X aims to make GPU acceleration accessible through higher-level libraries and frameworks, allowing domain experts to focus on their science or engineering problem rather than low-level optimization.
2. Workload Diversity: A single GPU architecture needs to efficiently support vastly different computational paradigms. CUDA-X provides tailored solutions for these diverse needs, ensuring optimal performance for specific operations common in various fields.
3. Integration with Existing Workflows: Many scientific and engineering applications rely on established software ecosystems. CUDA-X libraries are designed to integrate seamlessly with popular frameworks and partner applications, easing adoption.
4. Performance and Efficiency: While GPUs offer raw power, achieving peak performance requires highly optimized algorithms. CUDA-X libraries are meticulously engineered to maximize throughput and minimize latency, often delivering orders of magnitude speedups compared to CPU-based solutions.
5. Rapid Innovation: By providing a rich set of accelerated primitives, CUDA-X enables faster prototyping and deployment of new algorithms and AI models, accelerating the pace of scientific discovery and technological advancement.

CUDA-X effectively acts as an abstraction layer built on top of CUDA, offering a collection of over 900 libraries and AI models. This ecosystem allows NVIDIA to reach a broader developer base across industries, enabling them to leverage the power of NVIDIA GPUs (such as the latest Blackwell architecture, which is 50,000 times faster than the first CUDA GPU) to tackle the most demanding computational challenges.

Key Findings

▶ Watch: Accelerating Industrial Engineering, CAE, and EDA workflows (2:50)

The talk presents several key findings and contributions from the CUDA-X ecosystem, demonstrating its profound impact across various scientific and engineering domains:

• Ubiquitous Acceleration Across Domains: CUDA-X encompasses over 900 libraries and AI models, providing domain-specific acceleration for a vast array of workloads including computer-aided engineering (CAE), computational lithography, quantum computing, genomics, climate and weather modeling, and data science. This breadth underscores NVIDIA's commitment to accelerating "every possible workload."

• Dramatic Performance Speedups: The presentation consistently highlights significant performance improvements. Examples include:
• Computational Lithography: A 20x speedup compared to state-of-the-art CPU-based solutions using cuLitho.
• Sparse Solvers (KUDSS): Customers report 5-25x faster performance on average for full applications, with individual solve steps showing even higher speedups (e.g., 80x in extreme cases) over CPU solvers.
• AI Physics Surrogate Models: Enable evaluation of 10,000-17,000 different designs per day for automotive aerodynamics, a monumental leap from 300/day with accelerated solvers, 1/day with simulation, or 1/week with physical wind tunnels.
• Weather Prediction (CoreDiff): Achieves 12x higher resolution (from 25 km to 2 km) in weather patterns, while being 1,000 times faster and 3,000 times more energy-efficient than traditional physical simulation methods.
• Data Preprocessing (cuDF): Demonstrates roughly a 50x performance improvement for data frame operations on the GPU compared to the CPU.
• Optimization (cuOpt): Consistently breaks records for various optimization problems, despite being less than a year old.

• Enabling Real-time Interactive Workflows: CUDA-X facilitates the creation of interactive digital twins and real-time design exploration. The Physics Nemo framework, combined with NVIDIA Omniverse and NIM (NVIDIA Inferencing Microservice), allows engineers to modify design parameters (e.g., wing geometry, angle of attack) and instantly visualize the aerodynamic effects, drastically accelerating the design cycle and enabling exploration of a much wider design space.

• Advancements in Climate and Weather Forecasting: Earth-2 and its core component, CoreDiff, represent a monumental leap in weather prediction. The ability to generate high-resolution forecasts much faster and more efficiently enables hyper-local predictions (down to tens of meters) and the detection of less likely but potentially impactful storm paths, offering critical advanced warnings.

• Revolutionizing Computational Biology: cuEquivariance introduces equivariant neural networks, which are crucial for molecular simulations in drug discovery, protein folding (used in AlphaFold and Allegro), and docking. These networks preserve geometric properties under transformations, leading to more accurate and efficient analysis of molecular interactions.

• Pioneering Hybrid Quantum-Classical Computing: NVIDIA is actively shaping the future of quantum computing with cuQuantum (for quantum simulation) and cuDAQ (a quantum programming platform). The announcement of NVQLink establishes a standardized, high-speed connection between NVIDIA AI supercomputers and quantum processing units (QPUs), reflecting NVIDIA's vision of a hybrid computing paradigm where classical and quantum systems work synergistically for error correction and complex applications.

• Strong Partner Ecosystem and Open-Source Contributions: NVIDIA emphasizes its collaborative approach, partnering with major Independent Software Vendors (ISVs) across all domains. Many CUDA-X components, like Physics Nemo and the Earth-2 blueprint for real-time CAE digital twins, are open-source, fostering broad adoption and community contributions.

These findings collectively underscore CUDA-X's role as a transformative platform, not just for accelerating existing computations but for enabling entirely new paradigms of scientific research, engineering design, and industrial innovation.

Technical Deep Dive

▶ Watch: Introducing KUDSS: NVIDIA's high-performance sparse matrix solver (4:45)

The CUDA-X platform is built on the philosophy of providing specialized, highly optimized libraries and frameworks that abstract the complexities of GPU programming, enabling developers to leverage NVIDIA hardware for a vast array of domain-specific workloads. This deep dive explores the technical underpinnings of key CUDA-X components across different sectors.

Industrial Engineering (CAE and EDA)

In industrial engineering, particularly Computer-Aided Engineering (CAE) and Electronic Design Automation (EDA), performance is critical for complex simulations.

• KUDSS (CUDA Direct Sparse Solvers): This library is a cornerstone for accelerating applications dealing with sparse matrices, which are prevalent in fields like fluid dynamics, structural analysis, electromagnetics, and circuit simulation. Sparse matrices are characterized by a large number of zero elements, making traditional dense matrix operations inefficient in terms of memory and compute. KUDSS optimizes the three primary steps of a sparse matrix solve: factorization, the solve step itself, and reordering. By efficiently handling these sparse structures on the GPU, KUDSS delivers substantial speedups, often 5-25x faster for full applications compared to CPU solvers. It's designed to be a direct sparse solver, providing high accuracy.
• AI Physics (Physics Nemo): This open-source software product provides a collection of AI physics models that act as surrogate models for traditional physics simulations. Instead of running computationally expensive simulations for every design iteration, engineers can train a neural network on a dataset of high-fidelity simulation results.
• Domino neural operator is a specific type of neural network within Physics Nemo used for tasks like automotive aerodynamics. It learns the underlying physics to predict outcomes much faster than traditional solvers.
• NVIDIA provides pre-initialized solvers, trained on general datasets (e.g., automotive data), which users can then fine-tune with their own proprietary data and geometries.
• This approach, combined with NVIDIA Omniverse and the NIM (NVIDIA Inferencing Microservice) for CFD, enables real-time interactive digital twins. Engineers can manipulate design parameters (e.g., wing geometry, angle of attack) and instantly see the simulated effects, drastically reducing design cycles and allowing for exploration of thousands of design variations per day.

Climate and Weather (Earth-2)

Earth-2 is NVIDIA's initiative to create a digital twin of the Earth, leveraging AI and physical simulations for climate and weather prediction.

• CoreDiff: This is NVIDIA's generative AI model trained on WRF numerical simulations (Weather Research and Forecasting model). CoreDiff excels at downscaling, taking coarse-resolution weather data (e.g., 25 km grid) and generating high-resolution predictions (e.g., 2 km grid) at an unprecedented speed. It's 1,000 times faster and 3,000 times more energy-efficient than traditional physical simulations. This allows for more precise forecasting, such as typhoon landfalls in Taiwan, and enables the prediction of hyper-local phenomena down to tens of meters, incorporating effects of city infrastructure.
• Palm (Purdue Atmospheric-Oceanic Large-Eddy Simulation Model): CoreDiff is also being trained using high-resolution data from models like Palm, which simulates atmospheric and oceanic boundary layers. When combined with weather simulation windfields, Palm can model air flow around buildings, predicting phenomena like downwash, where strong winds funnel to street level.
• The ability to run ensembles of predictions (multiple model runs with slightly varied initial conditions) much faster increases confidence in likely paths and allows for the detection of less likely but impactful storm trajectories.

Data Science

CUDA-X provides essential tools for accelerating the entire data science workflow, from data preparation to complex optimization.

• cuDF: This library provides a GPU-accelerated equivalent to popular CPU-based data frame libraries like Pandas and Polars. By keeping data frames on the GPU, cuDF significantly accelerates data preprocessing tasks such as slicing, filtering, and aggregation, delivering roughly a 50x performance improvement.
• cuOpt: A relatively new but impactful library for solving optimization problems. This includes classical challenges like the traveling salesperson problem, as well as complex real-world scenarios in resource allocation, risk management, inventory control, supply chain optimization, and pricing in finance. cuOpt consistently breaks performance records for these NP-hard problems.
• RAPIDS, cuNumeric, JAX, cuDNN: These are broader components of the CUDA-X data science ecosystem. RAPIDS is a suite of libraries for end-to-end data science pipelines on GPUs. cuNumeric provides GPU-accelerated numerical computing primitives. JAX is a high-performance numerical computing library with automatic differentiation. cuDNN (CUDA Deep Neural Network library) is one of the original CUDA-X libraries, providing highly optimized routines for deep learning operations, forming the bedrock for many AI frameworks.

Computational Biology

Life sciences, particularly drug discovery and genomics, were early adopters of GPU acceleration and continue to benefit from specialized CUDA-X libraries.

• BioNeMo: This is a class of AI models specifically designed for biological applications, enabling various transformations and analyses within biological workflows.
• cuEquivariance: This library is a critical new addition for molecular simulations. It enables the use of equivariant neural networks, which are neural networks whose outputs transform predictably when their inputs undergo certain transformations (e.g., rotations or translations). In molecular modeling, this means that if a molecule is rotated, the network's prediction of its properties or interactions will also rotate predictably, preserving the underlying physical geometry. This property is crucial for tasks like:
• Protein folding (e.g., in applications like AlphaFold and Allegro).
• Drug docking, where the goal is to find molecules that bind effectively to target proteins.
• Generating crystals for materials science (e.g., superconductors).

cuEquivariance ensures that these transformations maintain geometric consistency, leading to more accurate and physically sound predictions.

Quantum Computing

NVIDIA is positioning its platform to be central to the future of quantum computing, envisioning a hybrid quantum-classical paradigm.

• cuQuantum: This library is designed to accelerate every major quantum simulator. It provides highly optimized primitives for simulating quantum circuits on GPUs, enabling researchers to develop and test quantum algorithms before physical QPUs (Quantum Processing Units) are widely available or sufficiently powerful.
• cuDAQ: This is NVIDIA's programming model and platform for quantum computing. It is designed to be hardware-agnostic, allowing developers to write quantum algorithms that can run on various types of QPUs (superconducting, trapped ion, neutral atom) as well as on GPU-accelerated simulators. cuDAQ supports hybrid applications, where classical operations run on an accelerated supercomputer and quantum operations run on a QPU, with seamless interaction between the two.
• NVQLink: Announced at GTC, NVQLink is a standardized, high-speed interconnection technology designed to link NVIDIA AI supercomputers directly with QPUs. This physical connection is vital for the hybrid computing vision, enabling:
• Quantum error correction, where classical supercomputers perform complex calculations to correct errors inherent in noisy qubits, eventually leading to stable logical cubits.
• Hybrid algorithm execution, where the classical part of an algorithm can prepare inputs for the QPU, and the QPU's results can then be further processed by the classical system. NVIDIA believes that QPUs will sit alongside classical accelerated supercomputers, much like GPUs sit alongside CPUs, accelerating specific, quantum-advantageous workloads.

Experimental Setup & Results

▶ Watch: Interactive engineering design using AI surrogate models (7:25)

While the talk focuses on a broad overview of CUDA-X capabilities rather than a single, detailed experimental setup, it consistently presents performance benchmarks and real-world impact across various domains. The "experimental setup" implicitly refers to comparing CUDA-X accelerated workflows against state-of-the-art CPU-based or traditional GPU-accelerated methods, often in collaboration with NVIDIA's industry partners.

• Computational Lithography:
• Baseline: State-of-the-art computational lithography on a modern CPU.
• Acceleration: Using cuLitho on NVIDIA GPUs.
• Result: A 20x performance speedup. This allows for faster design and manufacturing of semiconductor chips.

• Sparse Solvers (KUDSS):
• Baseline: CPU-based sparse solvers within existing scientific and engineering applications.
• Acceleration: Integration of KUDSS into applications.
• Result: While individual solve steps can show extreme speedups (e.g., 80x), customers report 5 to 25x faster performance on average for full applications. This has led to strong adoption by key software vendors, even though KUDSS is still a relatively young product (not yet at 1.0 release).

• AI Physics Surrogate Models (Physics Nemo/Domino):
• Context: Automotive aerodynamics design.
• Baselines:
• Physical wind tunnels: ~1 design evaluation per week.
• Traditional simulations: ~1 design evaluation per day.
• Accelerated solvers (presumably GPU-accelerated but not AI-driven): ~300 design evaluations per day.
• Acceleration: Using AI physics surrogate models (e.g., Domino neural operator in Physics Nemo), trained on a dataset of nearly 2,000 real simulations.
• Result: One partner reported evaluating over 17,000 different models per day. This massive speedup enables engineers to interact in real-time with designs, explore a much wider range of possibilities, and significantly accelerate the product development cycle.

• Climate and Weather Prediction (Earth-2/CoreDiff):
• Baseline: Traditional physical simulation methods for weather forecasting (e.g., WRF numerical simulations).
• Acceleration: CoreDiff (generative AI model) trained on WRF data, potentially combined with high-resolution data from models like Palm.
• Metrics: Resolution, speed, energy efficiency.
• Result: CoreDiff generates weather patterns at 12 times higher resolution (from 25 km to 2 km). It performs this task 1,000 times faster and is 3,000 times more energy-efficient than traditional methods. This enables more precise typhoon forecasting and hyper-local predictions (down to tens of meters). The talk also highlighted the ability to detect less likely but critical storm paths that traditional models might miss.

• Data Preprocessing (cuDF):
• Baseline: Data frame operations using CPU-based libraries like Pandas or Polars.
• Acceleration: Using cuDF on NVIDIA GPUs.
• Result: Roughly a 50x performance improvement for data preprocessing tasks.

• Optimization (cuOpt):
• Context: Various optimization problems (e.g., traveling salesperson, resource allocation).
• Result: cuOpt is noted for consistently "breaking records" in solving these problems, indicating superior performance compared to existing solutions. Specific numerical benchmarks for cuOpt were not detailed in the transcript, but the claim of record-breaking performance implies significant speedups.

• Quantum Computing:
• Context: Algorithmic development, QPU design, data generation for quantum-inspired neural networks, quantum error correction.
• Result: NVIDIA GPUs are already being used to achieve significant speedups in these areas today, enabling the development of algorithms and the design of better qubits even as physical QPUs mature. Specific numbers were not provided, but the statement indicates a measurable performance advantage.

Across all these examples, the consistent message is that CUDA-X components provide substantial, often orders-of-magnitude, performance improvements and efficiency gains, fundamentally changing what is possible in these domains.

Practical Implications

▶ Watch: Physics Nemo: Open-source package for AI physics models (8:05)

The widespread adoption and continuous development of NVIDIA CUDA-X Libraries carry profound practical implications for a broad spectrum of professionals, from individual practitioners to large-scale infrastructure teams.

For practitioners and engineers, the most immediate benefit is a dramatic acceleration of their workflows. In Computer-Aided Engineering (CAE), the ability to leverage AI physics models like those in Physics Nemo means engineers can iterate on thousands of designs per day rather than just a few. This enables real-time interaction with digital twins, fostering unprecedented design exploration and optimization. For instance, an automotive engineer can instantly see the aerodynamic impact of a wing geometry change, leading to faster innovation and better products. The interactive nature reduces the need for expensive physical prototypes or lengthy high-fidelity simulations for initial exploration, reserving them only for the most promising designs.

Infrastructure teams and model deployers benefit from the optimized and standardized nature of CUDA-X components. Deploying AI models for inference, such as the NIM (NVIDIA Inferencing Microservice) for CFD, becomes more streamlined and efficient. The underlying libraries ensure that applications run optimally on NVIDIA GPU hardware, maximizing resource utilization in data centers and cloud environments. The commitment to an open ecosystem, with tools like Physics Nemo being open-source and blueprints available on GitHub, also empowers teams to customize and integrate these technologies into their existing infrastructure.

For model builders and data scientists, CUDA-X offers powerful tools that simplify and accelerate complex tasks. cuDF allows data scientists to perform data preprocessing directly on the GPU, achieving 50x speedups and eliminating the costly data transfer bottlenecks between CPU and GPU memory. cuOpt provides a robust solution for a wide array of optimization problems, from supply chain logistics to financial modeling, enabling faster and more accurate decision-making. In computational biology, cuEquivariance is a game-changer for developing equivariant neural networks essential for drug discovery and materials science, ensuring physical consistency in molecular transformations.

The impact on climate and weather forecasting is particularly vital for public safety and economic planning. The Earth-2 initiative, powered by CoreDiff, enables hyper-local, high-resolution predictions that can provide earlier and more precise warnings for extreme weather events. This capability has critical implications for emergency services, agriculture, insurance companies, and even military operations, allowing for better preparedness and mitigation strategies. The ability to predict less likely but high-impact storm paths adds a crucial layer of resilience.

In the nascent but rapidly evolving field of quantum computing, CUDA-X provides a strategic pathway forward. cuDAQ and cuQuantum offer a stable platform for developing quantum algorithms and simulating quantum circuits today, preparing developers for the future. The announcement of NVQLink signals a clear vision for hybrid quantum-classical computing, allowing QPUs to be seamlessly integrated with powerful AI supercomputers. This means that as quantum hardware matures, the software ecosystem will already be in place to tackle complex problems like quantum error correction and hybrid applications, accelerating the journey towards useful quantum advantage.

Tradeoffs and Limitations:

While the benefits are substantial, there are practical considerations.

1. Data Requirements: AI physics models and surrogate models require large datasets of high-fidelity simulations for training. Generating this initial data can be computationally intensive, though it's a one-time cost that pays off in subsequent inference speed.
2. Integration Complexity: While CUDA-X simplifies GPU programming, integrating these libraries into complex, legacy codebases or highly specialized scientific applications still requires development effort and expertise. NVIDIA addresses this by working closely with ISVs.
3. Hardware Dependency: The performance gains are intrinsically linked to NVIDIA's GPU hardware. Organizations committed to CUDA-X will likely standardize on NVIDIA's platform.
4. Specialized Knowledge: While CUDA-X abstracts low-level details, understanding the underlying algorithms (e.g., sparse matrix properties, neural operator architectures, equivariant principles) remains crucial for effective model design and application.

Overall, CUDA-X represents a significant step in democratizing and accelerating advanced computing, empowering a wide range of professionals to tackle previously intractable problems and drive innovation across science and engineering.

Key Takeaways

• CUDA-X as a Universal Accelerator: CUDA-X is NVIDIA's expansive ecosystem of over 900 libraries, frameworks, and AI models, built on the foundational CUDA platform, designed to accelerate virtually every scientific and engineering workload, enabling a broad developer base.
• Orders of Magnitude Performance Gains: The platform delivers unprecedented speedups across diverse domains, including 20x for computational lithography, 5-25x for sparse solvers (KUDSS), 1000x faster and 3000x more energy-efficient weather prediction (CoreDiff), and 50x for GPU-accelerated data frames (cuDF).
• Enabling Real-time Digital Twins and Interactive Design: Through AI physics models in Physics Nemo and integration with NVIDIA Omniverse and NIM, CUDA-X facilitates interactive, real-time exploration of design spaces, dramatically accelerating engineering cycles (e.g., evaluating over 17,000 automotive designs per day).
• Transformative Impact on Climate and Biology: Earth-2 with CoreDiff provides hyper-local, high-resolution weather forecasting, enhancing preparedness for extreme events. cuEquivariance revolutionizes computational biology by enabling equivariant neural networks for accurate molecular simulations critical to drug discovery and protein folding.
• Pioneering Hybrid Quantum-Classical Computing: NVIDIA is actively shaping the future of quantum computing with cuQuantum for simulation and cuDAQ as a programming platform. The NVQLink interconnect standardizes the connection between AI supercomputers and QPUs, fostering a hybrid approach for quantum error correction and advanced applications.
• Open Ecosystem and Partner Collaboration: NVIDIA emphasizes an open-source approach for many CUDA-X components (e.g., Physics Nemo, Earth-2 blueprint) and collaborates extensively with Independent Software Vendors (ISVs), national labs, and QPU builders to ensure broad adoption and integration across industries.

About the Speaker(s)

This detailed technical article is based on a presentation given by a representative from NVIDIA at the NVIDIA GTC conference. While specific individual speakers were not named in the transcript, the talk reflects NVIDIA's overarching vision and strategic direction as a leading provider of accelerated computing platforms. NVIDIA, renowned for its pioneering work in GPU technology and the CUDA platform, is dedicated to empowering developers and researchers across the globe. Through the CUDA-X ecosystem, NVIDIA continues to innovate at the intersection of hardware and software, driving advancements in AI, high-performance computing, and simulation across a vast array of scientific and engineering disciplines. The presentation serves as a testament to NVIDIA's commitment to building comprehensive tools and fostering an open ecosystem that accelerates discovery and innovation.