The Advent of Confidential AI

The Advent of Confidential AI

Speaker: Raghu Yeluri — Fellow and Lead Architect, Confidential AI, Intel Corporation

Conference: [un]prompted 2026 — The AI Security Practitioner Conference

Date: March 3–4, 2026, San Francisco

Reading time: ~8 minutes

Note: This session did not have a publicly available YouTube recording. Coverage is based on the conference slides and transcript.

TL;DR

AI models and training data are exposed to cloud administrators, rogue insiders, and co-tenants in ways that most practitioners don't fully account for. Intel's Raghu Yeluri presents Confidential AI — hardware-enforced Trusted Execution Environments that encrypt AI workloads even from the cloud provider itself — and walks through three concrete deployments, including multi-hospital federated learning that complies with HIPAA without any party ever seeing another's data.

Introduction

When Raghu Yeluri asked the [un]prompted 2026 audience how many of them had heard of confidential computing, a reasonable number of hands went up. When he asked specifically about confidential AI, fewer did. His opening joke: "The joke about confidential AI is it's so confidential it stays in this room."

That light touch belied the weight of the problem he was addressing. When an organization runs AI workloads in the cloud — training a model on proprietary data, running inference on sensitive inputs, building RAG pipelines over internal documents — the cloud provider's administrators can, in principle, snapshot that virtual machine and walk away with everything. Encryption at rest and in transit is well understood, but encryption of data in use — while it is actively being processed — has not been the default. Confidential computing changes that.

Yeluri brings unusual authority to this topic. As a Fellow and Lead Architect for Confidential AI at Intel Corporation, he sits at the convergence of hardware security primitives and the AI deployment stack. His perspective spans both "security for AI" and "AI for security," and his talk at [un]prompted focused squarely on the former: how to protect AI workloads from threats that no amount of software-level defense can address.

What Confidential Computing Is (and Why It Matters for AI)

Confidential computing rests on three properties:

1. Data-in-use protection: Applications and workloads run inside hardware-protected environments called Trusted Execution Environments (TEEs). Everything executing inside is encrypted and under the control of the workload owner — not the cloud provider, not the infrastructure administrator, not co-tenants.

1. Ownership isolation: The key controlling access to the TEE lives in the CPU, not in any software layer that an administrator can access. A rogue administrator who takes a virtual machine snapshot gets only ciphertext.

1. Cryptographic attestation: Before any secret, key, or sensitive data is released to a workload, you verify cryptographically that the workload is running in a genuine confidential computing environment — correctly patched hardware, legitimate manufacturer, expected software stack. Attestation answers "how do you know for sure?" when a cloud provider claims you're in a confidential instance.

For AI workloads specifically, this protection extends to models (protecting intellectual property from theft or tampering), training data (protecting sensitive inputs during fine-tuning), prompts and context windows (protecting user queries in inference), and RAG pipeline content (protecting the knowledge base itself).

Yeluri framed this against an increasingly relevant regulatory backdrop. The EU AI Act (Article 15) mandates that high-risk AI systems "shall be resilient against attempts by unauthorised third parties to alter their use, outputs or performance." The U.S. Executive Order on Safe, Secure, and Trustworthy AI calls for agencies to use privacy-enhancing technologies where appropriate. DORA requires financial entities to maintain confidentiality of data "at rest, in use, or in transit." Confidential computing directly addresses the "in use" gap that neither traditional encryption nor access controls fill.

Intel TDX: Lift-and-Shift Confidential Computing

Intel's first approach to TEEs was SGX (Software Guard Extensions), which offered very strong security guarantees but required developers to redesign applications specifically for enclave execution. "SGX was not for the faint-hearted," Yeluri acknowledged. The security boundary was tight — no access except through the CPU — but the engineering burden was high.

The industry's demand is for lift-and-shift: take existing containers and workloads, move them into a confidential computing environment with minimal friction. That is Intel TDX — Trusted Domain Extensions.

With TDX, a cloud customer goes to Azure, GCP, IBM, or Oracle and selects a confidential compute instance instead of a standard one. Same number of vCPUs, same RAM, NVIDIA GPU attachments supported. The management plane, control plane, and operational model are identical to regular VMs. The workload owner gets full protection in use without changing a line of code.

Azure has moved TDX from preview to general availability. GCP, IBM Cloud, Oracle Cloud Infrastructure, Alibaba Cloud, and ByteDance (Volcano Cloud) all offer TDX instances across multiple Intel CPU generations (4th and 5th Gen Xeon). OS support includes Ubuntu 22.04+, RHEL 9.2+, SUSE SLES 15+, and VMware with vSphere 9.0+.

Two Confidential AI Models: CPU-Only and Accelerated

For AI-specific workloads, Yeluri described two deployment models:

CPU-only confidential AI: Suitable for most inference workloads and training with models under 10 billion parameters. Intel AMX (Advanced Matrix Extensions) acceleration provides reasonable inference throughput. Simple, mature, available today.

Accelerated confidential AI (CPU + GPU): Required for larger models. Both Intel CPUs and NVIDIA GPUs have their own confidential computing capabilities. The challenge is extending the TEE across the CPU-GPU boundary:

• Near term (bounce buffer): The CPU writes encrypted data to a shared buffer; the GPU picks it up, decrypts inside using a shared secret, processes, and returns encrypted results. A software-based approach available now with H100 and Blackwell GPUs.

• Longer term (TDX Connect): Hardware-level PCIe encryption where CPUs and GPUs negotiate trust and exchange data through encrypted PCIe lanes. Single logical TEE across CPUs and GPUs. Expected 1-2 years out, offering both full protection and performance parity with non-confidential GPU computing.

Attestation: Without It, Confidential Computing Means Nothing

Yeluri was emphatic on this point: attestation is not optional. A cloud provider can tell you that you're running in a confidential compute instance — but how do you verify that claim? Attestation provides a cryptographic proof, signed by the hardware manufacturer, that:

• The hardware is genuine Intel, AMD, or NVIDIA silicon
• The firmware is correctly patched and unmodified
• The hypervisor and guest OS match expected measurements
• The specific workload container you deployed is what's actually running

Intel Trust Authority is Intel's independent attestation service — a third party that is separate from the cloud infrastructure provider. This separation enforces the four-eyes principle (also called separation of duties): the entity providing the infrastructure should not be the same entity certifying that infrastructure as trustworthy. A single API covers unified attestation across Intel CPUs (TDX) and NVIDIA GPUs (Confidential Computing GPUs) — whether that's one H100, one Blackwell, or eight Blackwells on a single system.

Intel Trust Authority operates as a SaaS service and is available for on-premises deployments. It works across Google Cloud, Azure, IBM, Alibaba, and on-premises environments through a single API — so application owners don't write separate attestation workflows for each deployment target.

Three Deployments: From Data Clean Rooms to Agentic AI

Yeluri walked through three concrete examples that progress from available-today to where the industry is heading.

1. Google Confidential Space

Google Confidential Space is essentially a data clean room for AI on Google Cloud. A user takes their containers, runs them in a TDX-protected VM, and everything — the guest OS and the application container — is measured and made available for verification. When the container launches, the Confidential Space environment attests to Intel Trust Authority. Trust Authority verifies policy compliance and issues an attestation token. That token is presented to data sources (in Google Cloud, AWS, or on-premises), access control systems verify it, and only upon successful verification is data released to the workload. The workload owner and the CPU are the only parties who see what's inside. Google Confidential Space is in production today and deployable with a few clicks.

2. Multi-Party Medical AI Fine-Tuning (Thales CipherTrust)

This example addresses a real HIPAA compliance problem: a medical imaging AI model needs to be fine-tuned on patient data from multiple hospitals to meaningfully improve diagnostic efficacy. A single hospital's dataset is insufficient. But HIPAA makes sharing patient data between hospitals — or even with the model owner — untenable without extensive agreements and risk exposure.

The solution using Thales CipherTrust and Intel TDX:

• Each hospital provisions an encryption key in the CipherTrust key management platform and encrypts their dataset with that key.
• The model owner also encrypts their model with a CipherTrust key.
• All encrypted material is brought into a confidential computing environment.
• CipherTrust communicates with Intel Trust Authority, verifies the environment's attestation, and releases decryption keys into the TEE per each party's individual policy settings.
• Inside the CPU+GPU TEE: datasets decrypt, the model fine-tunes on all combined data, training completes, datasets are deleted, and the model is re-encrypted and returned to the model owner.

The outcome: model fine-tuned on multi-hospital data; model owner's IP never exposed; no hospital's patient data visible to any other hospital, to Intel, or to Google. Meaningfully improved diagnostic efficacy. Full HIPAA compliance.

3. Agentic AI and Confidential Computing

Yeluri described the agentic AI architecture as a reasoning engine (large planning LLM), domain-specific models for specialized tasks, RAG pipelines, and a cluster of MCP servers talking to various tools. His prediction: enterprises will converge on many small, highly efficient domain models rather than one large model doing everything.

In this architecture, MCP servers become a primary confidential computing target. There are several scoping variants depending on the threat model:

• Run just the MCP server in a TEE
• Run the MCP server together with its associated tools in the same TEE
• Run domain models and RAG systems alongside MCP servers in the same TEE

COSI (Coalition for Secure AI) — which includes major security companies, cloud providers, and hardware providers — recently published an MCP security positioning paper that addresses where confidential computing fits in this stack.

Notable Quotes

"Confidential computing eliminates the threat that a cloud operator could have a rogue administrator who takes a snapshot of your virtual machine and walks away with all your secrets." (from transcript)

"Without attestation, confidential computing means nothing. Your cloud provider can tell you you're running in a confidential compute instance, but how do you know for sure?" (from transcript)

"The result is a model fine-tuned on multiple hospitals' data, with the model owner's IP never exposed, no hospital's patient data ever visible to any other hospital or to Intel or Google, and meaningfully improved diagnostic efficacy." (from transcript)

"In my view, enterprises are going to end up with many localized domain models that are highly efficient for their specific tasks rather than one large model doing everything." (from transcript)

Key Takeaways

• Confidential computing protects AI workloads from privileged attackers — rogue cloud admins, malicious co-tenants, privilege escalation exploits, and zero-days targeting the management stack. These are threat categories that software-layer defenses cannot address.
• Intel TDX enables lift-and-shift: existing VM images and containers run in a confidential computing environment with no code changes, available today across Azure, GCP, IBM, Oracle, Alibaba, and ByteDance.
• Attestation is non-negotiable: cryptographic proof that you are in a genuine, correctly configured TEE — from an independent third party like Intel Trust Authority — is what separates real confidential computing from marketing claims.
• Multi-party AI workflows (like federated fine-tuning across hospitals) become feasible under confidential computing, where no party ever sees another's raw data but all parties' data contributes to shared models.
• Agentic AI will increasingly require TEE-protected MCP servers as enterprises deploy small domain models at scale. The architecture is defined — MCP servers in TEEs, optionally with domain models and RAG — and the industry is moving toward it.

Slides Reference

Slides are available as 2026-04-04-D1-S2-13-31-The-Advent-of-Confidential-AI-pdf.pdf. Key slide topics include:

• Confidential AI architecture diagram: TEE with hardware-based protection, untrusted software layer separation
• Two models of Confidential AI: CPU-based (inference, <10B params) vs. Accelerated with Intel TDX Connect or bounce buffer
• Regulatory alignment: EU AI Act Article 15, U.S. Executive Order Section 2(f), DORA Chapter 2
• Intel TDX availability matrix across cloud providers and CPU generations
• Intel Trust Authority architecture: independent attestation, unified API across CPUs and NVIDIA GPUs
• Thales E2EDP multi-party medical AI fine-tuning solution diagram
• Agentic AI + confidential computing architecture: MCP servers in TEE with scoping variants