Live at NVIDIA GTC with Acquired

Live at NVIDIA GTC with Acquired

Speakers: Ben Gilbert, David Rosenthal (Acquired), NVIDIA

Conference: NVIDIA GTC

YouTube: https://www.youtube.com/watch?v=pgLdJq9FRBQ

Overview

This article delves into the "Live at NVIDIA GTC with Acquired" pre-keynote show, a unique retrospective and forward-looking discussion held live from the SAP Center in San Jose. Hosted by Ben Gilbert and David Rosenthal of the popular Acquired podcast, this special event brought together a stellar lineup of industry titans, including NVIDIA CEO Jensen Huang, former Intel CEO Pat Gelsinger, Dell CEO Michael Dell, and leaders from TSMC, Foxconn, Cadence, ServiceNow, and Sequoia Capital. The talk offers an "outside-in" perspective on NVIDIA's three distinct epochs: the genesis in graphics and gaming, the pivotal shift to accelerated computing, and its current foundational role in the burgeoning AI era.

The core of this discussion revolves around NVIDIA's improbable journey from a fledgling graphics card company on the brink of bankruptcy to a $3 trillion market cap powerhouse at the heart of the AI revolution. It highlights the visionary leadership of Jensen Huang, his willingness to make audacious, company-betting decisions, and NVIDIA's relentless focus on building a comprehensive platform encompassing hardware, software, and a robust developer ecosystem. The dialogue underscores why NVIDIA's technology, particularly its GPUs and the CUDA programming model, became uniquely suited to power the most demanding workloads, from scientific supercomputing to today's large language models, fundamentally reshaping the landscape of technology and industry.

Background

▶ Watch: NVIDIA GTC pregame show introduction (0:00)

NVIDIA's story begins in the turbulent 1990s, a highly competitive landscape for 3D graphics. The company's initial offering, the NV1 graphics card, was a bold but ultimately misaligned attempt to innovate using quadrilaterals instead of the industry-standard triangles and polygons. This early misstep, coupled with the reluctance of software developers to adopt their approach, nearly led to bankruptcy. However, NVIDIA persevered, eventually dominating the graphics card era by adapting to market demands and forging critical partnerships, notably with TSMC starting around 1997. This period, characterized by intense competition and financial volatility, instilled a deep-seated drive for survival and innovation that would define the company's future.

The pivotal turning point arrived in the early 2000s with NVIDIA's audacious bet on general-purpose computing on GPUs (GPGPU) and the introduction of CUDA in 2007. At a time when GPUs were exclusively for graphics, Jensen Huang envisioned repurposing their massive parallel processing capabilities for broader computational tasks. This was a highly contrarian view, met with skepticism from established players like Intel, who saw the CPU as the undisputed "king of the hill." NVIDIA invested heavily in CUDA for over a decade, reportedly losing money, but laying the groundwork for a future that few could foresee. This era saw GPUs first adopted by academics and scientists for supercomputing, demonstrating their potential for highly parallelizable workloads.

The advent of the AI era, particularly milestones like AlexNet (2012), AlphaGo (2016), the Google Transformer paper (2017), and the ChatGPT moment (2022), proved to be the ultimate validation of NVIDIA's long-term vision. The inherent parallelism of deep learning algorithms and matrix multiplication, which are central to AI, found a perfect match in the GPU architecture. NVIDIA's foresight in integrating Tensor Cores—specialized hardware accelerators for AI computations—into its Volta (V100) and later Turing (RTX 2080s) GPUs, even at the risk of cannibalizing valuable chip real estate, cemented its indispensable role as the foundational technology provider for the AI revolution. This journey highlights a consistent pattern of identifying future workloads, making significant long-term investments, and building comprehensive platforms rather than just discrete chips.

Key Findings

▶ Watch: NVIDIA's unique founder-led status and growth (4:00)

The discussion illuminated several key findings regarding NVIDIA's strategic trajectory and its impact on the technology landscape:

1. Visionary Leadership and Market Creation: Jensen Huang's consistent ability to foresee and bet on markets that didn't yet exist – from 3D graphics to GPGPU to AI – was a recurring theme. His tenacity and long-term vision allowed NVIDIA to invest in foundational technologies like CUDA for over a decade before they became profitable, ultimately creating entirely new industries.
2. The Power of Platform over Chip: A crucial insight was that NVIDIA's enduring moat isn't merely superior chip design but its comprehensive platform approach, integrating hardware, software (CUDA, optimized frameworks like TensorFlow and PyTorch), and a vast developer ecosystem. This "data center is the computer" philosophy makes it incredibly challenging for competitors focusing solely on silicon to catch up.
3. Adaptability and Relentless Innovation: NVIDIA is not a static entity; it continuously adapts and innovates its architectures. As Cadence CEO Anarrude Devon noted, NVIDIA's yearly cadence of innovation, adding not just compute but architectural advancements, is remarkable. This dynamic evolution ensures their GPUs remain cutting-edge for emerging workloads.
4. CPU-GPU Co-design is Critical: For complex and diverse workloads, the future lies in tightly integrated CPU-GPU architectures, as exemplified by Grace Blackwell and Grace Hopper. The ability to share memory seamlessly and optimize communication between these processing units is becoming paramount, especially as AI workloads become more sophisticated and less "embarrassingly parallel."
5. Memory Bandwidth as a Bottleneck: A significant revelation from supercomputing experts was that 70-80% of high-performance computing workloads are bandwidth-bound, not compute-bound. Even Transformer models, central to modern AI, spend approximately 50% of their time accessing memory. This underscores NVIDIA's strategic focus on high-bandwidth memory (HBM) and efficient memory hierarchies.
6. AI's Transformative Business Impact: Beyond technological prowess, the talk showcased the profound practical implications of AI. Enterprises like ServiceNow are leveraging NVIDIA's stack to build AI operating systems for business transformation, projected to unlock a $20 trillion addressable market and generate $4 trillion in operational expenditure (opex) savings over the next five years. This signifies a shift from mere efficiency gains to fundamental re-imagination of business processes.

Technical Deep Dive

▶ Watch: Overview of NVIDIA's three historical epochs and guests (5:00)

NVIDIA's technical journey, as chronicled in the discussion, reveals a strategic evolution from specialized graphics rendering to a general-purpose computing paradigm, culminating in its current dominance in AI.

The story begins with the NV1 graphics card, NVIDIA's first commercial product. Its unique approach of rendering 3D graphics using quadrilaterals (four-sided polygons) rather than the industry-standard triangles was technically innovative but ultimately failed due to a lack of software ecosystem support. This early lesson in ecosystem dependence would profoundly shape NVIDIA's future.

The true technical inflection point arrived with the development of CUDA (Compute Unified Device Architecture) in 2007. CUDA transformed the GPU from a fixed-function graphics accelerator into a programmable parallel processor. This was a radical departure, enabling developers to write general-purpose code for GPUs using C, C++, and Fortran, effectively treating the GPU as a highly parallel SIMD (Single Instruction, Multiple Data) machine. Early applications in supercomputing, such as the Sububame supercomputer, demonstrated CUDA's power. Dr. Satoshi Matsuoka recounted how a 128-GPU cluster in 2007 ran a protein docking application four times faster than an IBM Blue Gene supercomputer at a fraction of the cost (1/20th). This showcased the GPU's inherent advantage for data-parallel tasks. The subsequent introduction of Windows Direct Compute in 2009 further legitimized GPGPU, making it easier for a wider range of scientific and engineering applications to leverage GPU acceleration.

The AI era brought new demands, which NVIDIA met with further architectural innovations. The introduction of Tensor Cores in the Volta V100 GPU (around 2016) was a game-changer. These specialized processing units are highly optimized for matrix multiplication and tensor manipulation, the fundamental operations of deep learning. This hardware acceleration significantly boosted the performance of AI training and inference. Gavin Baker highlighted a "bet the company" moment when NVIDIA integrated Tensor Cores into its Turing (RTX 2080s) consumer GPUs, initially for features like DLSS (Deep Learning Super Sampling). This move, which dedicated valuable chip real estate away from traditional graphics, was a bold strategic gamble that made AI development accessible to a broader developer base.

Modern AI workloads, particularly large language models based on the Transformer architecture, are not solely compute-bound. As Dr. Matsuoka pointed out, 70-80% of supercomputing workloads are bandwidth-bound, and Transformers are approximately 50% memory-bound. This critical insight drives NVIDIA's focus on advanced memory technologies like HBM (High Bandwidth Memory) and sophisticated memory hierarchies. The latest NVIDIA architectures, such as Grace Blackwell and Grace Hopper, exemplify this understanding by integrating powerful CPUs (Grace) with high-performance GPUs (Blackwell/Hopper). This CPU-GPU co-design ensures efficient data movement and shared memory access, addressing the increasing complexity and heterogeneity of AI workloads.

Looking ahead, Pat Gelsinger emphasized the need for 10,000x greater efficiency in AI inference, suggesting that future architectures will focus on in-memory computing and specialized low-radix math. He also posited that quantum computing (QPUs), potentially silicon-based, will emerge as a critical component of the future data center, handling problems that are inherently quantum in nature and currently intractable for classical computers. This vision suggests a future data center that is a hybrid of traditional CPUs, GPU-accelerated training systems, inference-optimized chips, and QPUs, each tailored to specific workload characteristics.

Experimental Setup & Results

▶ Watch: First panel guests introduced, NV1 graphics card presented (7:00)

This conference talk was primarily a retrospective and forward-looking discussion, featuring expert insights and historical accounts rather than presenting new experimental data from a specific setup. Consequently, a traditional "Experimental Setup" section is not applicable. However, the discussion provided numerous compelling "results" and benchmarks that underscore NVIDIA's impact and the efficacy of its strategic decisions.

Key "results" highlighted include:

• Early GPGPU Performance: Dr. Satoshi Matsuoka detailed how the Sububame supercomputer, utilizing 128 CUDA-powered GPUs in 2007, achieved a 4x speedup on a protein docking application compared to an IBM Blue Gene supercomputer, at roughly 1/20th of the cost. This demonstrated an early, significant performance-per-dollar advantage for GPU-accelerated computing.
• NVIDIA Stock Performance: CJ Muse provided a historical context of NVIDIA's stock price, adjusted for splits, illustrating the market's gradual appreciation of its AI bets:
• AlexNet (2012): $0.33
• AlphaGo (2016): $2.00
• Google Transformer Paper (2017): $5.00
• ChatGPT Moment (Nov 2022): $16.00
• NVIDIA's $4 Billion Guidance Beat: $40.00

A $1,000 investment at IPO (1999) would be worth $5 million today, showcasing extreme long-term growth.

• Dell's AI Infrastructure Growth: Michael Dell reported that Dell delivered over $10 billion in accelerated AI servers in the past year (2023), including more than 2,200 Dell AI factories to enterprise customers. Dell's infrastructure business grew 29% to approximately $45 billion last year, highlighting the massive demand for integrated AI solutions.
• Foxconn's Automation Levels: Chairman Younglu noted that Foxconn's manufacturing facilities, which produce NVIDIA's advanced computing platforms like DGX systems, have achieved 60-80% automation levels. He attributed the ability to further enhance this automation, especially for "dull, dirty, and dangerous" tasks, to the capabilities of generative AI.
• ServiceNow's Market Opportunity: Bill McDermott projected that the addressable market for AI-driven business transformation, facilitated by platforms built on NVIDIA's stack, is $20 trillion over the next five years. He also estimated a $4 trillion reduction in operational expenditures (opex) for enterprises adopting AI during the same period, demonstrating tangible financial outcomes.
• Workload Characteristics: Dr. Matsuoka's observation that 70-80% of supercomputing workloads are bandwidth-bound and that Transformer models are 50% memory-bound provides critical data influencing future hardware and system design, emphasizing the importance of memory architecture over raw compute in many scenarios.

These "results," while not from controlled experiments, serve as powerful evidence of NVIDIA's technological impact and the real-world economic and performance benefits derived from its innovations in accelerated computing and AI.

Practical Implications

▶ Watch: Expert explains NVIDIA's first product, the NV1 (8:00)

The discussions at GTC highlighted profound practical implications for various stakeholders in the AI/ML and broader tech ecosystem:

• For Practitioners (Developers, Data Scientists):
• CUDA Proficiency is Key: The enduring relevance of CUDA means that understanding parallel programming models is crucial for optimizing workloads on NVIDIA GPUs.
• Workload Characterization: Practitioners must recognize that not all AI workloads are "embarrassingly parallel" or solely compute-bound. Many are memory-bound, especially in complex models like Transformers. This necessitates careful consideration of data movement, memory hierarchy, and communication patterns in application design.
• Evolving Architectures: Keep abreast of advancements like Grace Blackwell and the integration of CPUs and GPUs. Future models may require hybrid programming approaches that leverage the strengths of both.
• Inference Optimization: As AI moves from training to widespread deployment, specialized inference architectures will emerge. Practitioners should prepare for frameworks and tools optimized for low-precision math and in-memory computing to achieve significant cost and energy efficiency.

• For Infrastructure Teams (IT, Cloud Ops):
• System-Level Thinking: The "data center is the computer" paradigm means infrastructure teams need to move beyond managing individual components to deploying and optimizing integrated AI factories. This includes not just GPUs, but also high-speed networking (InfiniBand, NVLink), storage, and comprehensive management software.
• Partnerships are Crucial: Dell's role in providing complete, supported AI factory solutions (e.g., Dell AI factories) underscores the value of partnerships for enterprise deployment. Infra teams will rely on vendors who can deliver integrated, reliable, and scalable systems with associated services.
• Memory and Interconnect Focus: Prioritize systems with high-bandwidth memory (HBM) and robust interconnects, as these are increasingly critical bottlenecks for AI workloads.
• Future-Proofing for Quantum: While early, the emergence of quantum processing units (QPUs) means infra teams should monitor hybrid classical-quantum computing models for specialized problem domains.

• For Model Builders (AI Researchers, Architects):
• Beyond Brute Force: As AI models mature, the initial "brute force" approach (dense matrix multiplies) will evolve. Model builders will need to design algorithms that are more efficient in terms of memory access, sparsity, and latency partitioning.
• CPU-GPU Synergy: Leverage the combined power of Grace Blackwell-like architectures. Understanding how to partition workloads between CPUs (for conditional logic, data handling) and GPUs (for parallel compute) will be vital.
• Low-Precision and In-Memory Compute: For inference, explore techniques using 4-bit precision or even unary computing, and consider how models can benefit from emerging in-memory computing architectures to achieve 10,000x efficiency gains.

• For Deployers (Business Leaders, CIOs):
• AI as an Operating System: Enterprises should view AI as a foundational technology for business transformation, not just a tool for marginal efficiency gains. Platforms like ServiceNow are positioned to be the "AI operating system" for the enterprise, integrating with various LLMs and automating workflows across all departments.
• Massive ROI Potential: The projected $20 trillion addressable market and $4 trillion opex savings highlight the immense financial incentive for aggressive AI adoption.
• Reimagining Processes: AI enables a fundamental rethinking of business processes. Examples from banking (personalized customer interaction), healthcare (tonality analysis, automated claims), manufacturing (worker productivity, supply chain resilience), and engineering (reducing setup work) demonstrate the breadth of impact.
• Tradeoffs and Limitations: While powerful, GPUs can be "10,000x too expensive" for widespread inference deployment. Deployers must balance performance needs with cost, considering specialized inference chips or optimized software stacks for scale.

Key Takeaways

• NVIDIA's success stems from Jensen Huang's long-term vision and willingness to make company-betting investments in emerging technologies, notably CUDA and Tensor Cores, years before market validation.
• NVIDIA's competitive advantage lies in its comprehensive platform strategy, integrating hardware, software, and an extensive developer ecosystem, rather than just superior chip performance.
• Modern AI workloads, particularly Transformers, are increasingly memory bandwidth-bound, driving the need for advanced memory technologies like HBM and tight CPU-GPU co-design in architectures like Grace Blackwell.
• AI is a transformative force for enterprises, offering trillions of dollars in new market opportunities and operational cost savings, fundamentally changing business processes across all industries.
• The future of computing will likely be a hybrid environment, combining traditional CPUs, GPU-accelerated systems for training, specialized chips for inference optimization, and emerging quantum processing units (QPUs) for specific problem sets.
• Industry partnerships, such as those with TSMC, Dell, and Foxconn, are critical for NVIDIA's ability to scale production and deliver integrated, end-to-end AI solutions to a global customer base.

About the Speaker(s)

The "Live at NVIDIA GTC with Acquired" event was hosted by Ben Gilbert and David Rosenthal, the dynamic duo behind the highly acclaimed Acquired podcast. Known for their meticulous, in-depth historical analyses of iconic companies, Gilbert and Rosenthal brought their signature "outside-in" perspective to NVIDIA's journey. Their podcast episodes often involve hundreds of hours of research and extensive editing, a stark contrast to the live, guest-heavy format of this GTC pre-show. Their expertise lies in dissecting the strategic decisions, turning points, and market dynamics that shape technology giants.

The event featured a remarkable roster of industry leaders as guests, providing diverse insights into NVIDIA's impact:

• Jensen Huang: Founder and CEO of NVIDIA, described as a visionary leader who has bet the company multiple times and shaped the future of computing.
• Pat Gelsinger: Former CEO of Intel, offering a perspective from NVIDIA's long-time competitor and insights into the evolution of processor architectures.
• Michael Dell: Founder and CEO of Dell Technologies, a decades-long partner of NVIDIA, discussing the evolution of PC and data center infrastructure.
• Rick Sai: Former CEO of TSMC and current CEO of MediaTek, providing insights into NVIDIA's crucial chip manufacturing partnerships.
• CJ Muse: A veteran investor who tracked NVIDIA's public market journey, detailing its stock performance relative to key AI milestones.
• John Penny: Longtime industry analyst, offering historical context on the early graphics card wars.
• Dr. Satoshi Matsuoka: A pioneering supercomputing expert who led the development of the first CUDA-powered supercomputer, Sububame.
• Dr. Anarrude Devon: CEO of Cadence, a leader in chip design software, discussing the role of GPUs in accelerating electronic design automation.
• Bill McDermott: Chairman and CEO of ServiceNow, elaborating on AI's transformative impact on enterprise software.
• Younglu: CEO of Foxconn, the world's largest technology manufacturer, detailing their role in assembling NVIDIA's advanced systems and leveraging AI in manufacturing.
• Alfred Lynn: Partner at Sequoia Capital, an early NVIDIA investor, discussing venture capital's perspective on accelerating technological change.
• Gavin Baker: An investor recognized for his early conviction in NVIDIA's AI potential, sharing insights into strategic decisions like the integration of Tensor Cores.