Resilience in the Uncharted AI Landscape

Resilience in the Uncharted AI Landscape

Speaker: Ranita Bhattacharyya

Conference: BSidesSF 2025 — April 26-27, 2025, San Francisco

YouTube: Watch on YouTube

Reading time: ~7 minutes

TL;DR

Building resilient AI systems is not a post-deployment afterthought — it requires hundreds of small, deliberate decisions made from the earliest design stages. Ranita Bhattacharyya frames resilience through three pillars (robustness, recovery, and plasticity) and walks through a concrete agentic AI product example to show exactly what those decisions look like in practice.

Introduction

The word "resilience" gets a fresh coat of paint every time a major technology shift arrives. With AI now woven into production systems at scale, organizations face a question that sounds deceptively familiar: how quickly can you get back up, dust yourself off, and be ready for the next fight?

Bhattacharyya, a security GRC and product management leader, argues that the answer is largely determined before any incident ever occurs. The CrowdStrike outage of July 2024 is her opening salvo: airlines like Delta grounded over 1,200 flights while competitors like United and Allegiant recovered quickly. The difference was not the threat itself — it was design decisions made long before the event. The same principle now applies to AI, where agentic pipelines, LLM integrations, and autonomous workflows introduce new failure surfaces that old playbooks only partially address.

The Three Pillars: Robustness, Resilience, and Plasticity

▶ Watch: Defining the three pillars (1:30)

Drawing on a research framework, Bhattacharyya breaks resilience into three interlocking concepts:

• Robustness — the ability to withstand impactful events. In practical terms, this maps to foundational cybersecurity: secure-by-design practices, defense in depth, zero trust, policy adherence, and a culture of accountability.
• Resilience — the ability to recover and return to steady state during an impacting event. This translates to business continuity and disaster recovery.
• Plasticity — the ability to incorporate lessons learned back into the steady state after recovery. This is process improvement: the organization that comes out the other side is structurally better than the one that went in.

She pairs these with a dual philosophical tension. Hans Hofmann's artist maxim — "eliminate the unnecessary so the necessary may shine" — is in direct dialogue with Einstein's scientist caution: "everything should be made as simple as possible, but not simpler." Experience, Bhattacharyya says, is what teaches you where that line sits.

Resilience by Design: A Worked Example

▶ Watch: The Food & Stuff agentic product walkthrough (8:00)

Rather than staying abstract, Bhattacharyya builds a tangible example: a fictional grocery chain called "Food & Stuff" that deploys a GenAI chat feature for customer Ron Swanson to plan dinner parties. That feature is one node in a larger agentic workflow — shopping list generation, purchasing, order fulfillment via Instacart and DoorDash integrations, insights generation, and targeted advertising. The chain illustrates how quickly AI product complexity compounds.

She then walks through eight decision categories where resilience either gets baked in or left out:

1. Product architecture — How much autonomy is acceptable? What does the blast radius of each agent look like? These questions define which parts of the system need the most resilience investment.
2. Build vs. buy — A grocery chain probably cannot justify building everything from scratch. But vendor choices determine who owns recovery responsibility — and that must be contractually defined.
3. Infrastructure design — The example goes with a GCP backend using microservices, Kubernetes (GKE), and Terraform for infrastructure-as-code. Each agent gets its own container; the orchestration workflow is itself a microservice. These choices buy visibility, isolation, and reproducibility.
4. AI model development — Dialogflow, Gemini, GPT, Vertex AI, and a custom-trained ML model for Food & Stuff's own inventory knowledge base all feed the product. The guiding principle: build the best product the budget allows, because security wraps around it.
5. Deployment pipeline — Git source control, Jenkins or GitHub Actions for CI/CD, unit and integration tests, Docker images on Docker Hub, deployment to Kubernetes. Security and reproducibility, not convenience, drive decisions here.
6. Observability — ELK Stack (Logstash, Elasticsearch, Kibana), Prometheus, Grafana, and a SIEM consuming VPC flow logs, application logs, and AI platform logs. SAST/DAST, CSPM, SSPM, and vulnerability scanning tools round out the stack.
7. Security, privacy, and compliance — Food & Stuff collects dietary restrictions, geographical data, and demographic information. If operating in California, CCPA is not optional — and compliance decisions made late are expensive.
8. Backup and storage architecture — Does the contingency environment need to be a full, AI-enabled replica, or is a heavily populated FAQ database sufficient for a limited outage window? Honest answers to questions like this save significant cost.

Cost Optimization and the Role of Regulators

▶ Watch: Cost trade-offs and KPIs (22:00)

Security practitioners rarely control their own budgets, and Bhattacharyya does not pretend otherwise. She introduces a framework for prioritizing resilience spending: quantify the ROI, match multi-region/multi-zone requirements to actual business exposure, and plan for scenarios proportionate to the organization's risk profile. A grocery chain whose primary revenue comes from foot traffic does not need the same uptime posture as a streaming platform that lost $65–100 million across Facebook and WhatsApp during a six-hour outage in 2021.

A recurring theme is that regulators, often framed as adversaries, are actually forcing functions for prioritization. When compliance requirements are mandatory, the decision-making process is simplified. The CISO who invokes regulatory necessity to justify a security investment is using the regulator as an ally, not fighting bureaucracy.

Incident Response and Recovery Playbooks for AI Systems

▶ Watch: Recovery process walkthrough (28:00)

Bhattacharyya traces a realistic incident: anomalous traffic at a Food & Stuff endpoint with off-geography characteristics and syntactically unusual queries — a DDoS-style prompt injection. The response sequence flows from detection and alerting, through tiger team assembly, to breach assessment, restore-and-recover, and finally the plasticity phase.

Her recovery checklist has five categories:

1. System integrity validation — every agent, the model, and the data must be individually verified.
2. Gradual traffic reintroduction — load balancers configured to allow only a percentage of traffic, with dashboards from both platform engineering and security glued to behavioral metrics before incrementally increasing load.
3. Agent synchronization — the full agentic workflow is tested end-to-end: health of dependencies, decision engine logic, stress tests, and edge case runs through Vertex AI pipelines.
4. Security and compliance restoration — rolling back to the last known good state is the default move, but it means lost patches, lost scan data, and elevated access that was never revoked. Everything needs to be re-scanned and re-patched. If the company is public, SEC reporting may be triggered.
5. Communication and customer outreach — internal runbooks, PR playbooks, Slack templates, and external customer notifications must all be pre-built. Drafting them during an incident is too late.

Proving Resilience: Certification Scope

The final section addresses a question customers and regulators increasingly ask: how do you prove this actually works? Bhattacharyya's answer is to align all the design decisions already made to a certification framework — SOC 2, ISO 27001, or similar — and define scope clearly. The scope question has three components: does the organization provide tooling and environment only, does it also own data and storage, and does it own query development and AI/ML models? Once that boundary is drawn, what remains is identifying which obligations fall to the vendor and which fall to the customer.

Notable Quotes

"Resilient by design starts at the start — when your product was but a doodle on the back of a napkin."

"Compliance is your friend. Regulations can be friendly. Use that to prioritize this effort."

"The tools change, the challenges change, but ultimately the approach, the goals, the ethos remains the same. Good resilience is based on thorough risk assessment and practiced plans."

Key Takeaways

• Start at the doodle stage. Hundreds of micro-decisions about architecture, vendors, infrastructure, and pipelines accumulate into resilience or its absence. There is no shortcut to inserting these concerns later.
• The CrowdStrike lesson is transferable. The difference between Delta and United in July 2024 was design decisions made long before the incident. The same is now true for AI workloads.
• Vendors are partners in resilience, but only if the contracts say so. Build-vs-buy decisions determine who owns recovery. If the contract does not specify it, the answer defaults to the customer.
• Regulators prioritize your backlog. Mandatory compliance requirements cut through cost debates. Use them accordingly.
• Recovery is not one action. Rolling back to a last known good state is the right first move, but it empties the security toolbox — scans, patches, and access revocations must all be repeated before traffic is restored.