Building Secure Agentic Systems: Lessons from Daily-Driver Agents

Building Secure Agentic Systems: Lessons from Daily-Driver Agents

Speaker: Brooks McMillin, AI Security Researcher & Infrastructure Security Engineer, Dropbox

Conference: [un]prompted 2026 — The AI Security Practitioner Conference

Date: March 4, 2026, San Francisco

Watch: YouTube — Full Talk

Reading time: ~9 min

TL;DR

Brooks McMillin has built a personal ecosystem of 19 AI agents running 73 MCP tools — and has been breaking it, learning from those failures, and hardening it in real time. His [un]prompted talk is a practitioner-level account of what actually goes wrong when you run agents at meaningful scale, and what security controls work in practice: capability bounding, memory isolation, context-aware trimming, and granular observability.

Introduction

Most AI security talks describe threats in the abstract. Brooks McMillin's talk at [un]prompted 2026 was something different: a live dissection of a production system he built himself, runs every day, and has been actively breaking and fixing over time.

McMillin is an infrastructure cloud security engineer at Dropbox, but the agents he described are personal projects — open-sourced on his GitHub, not polished products. "Everything I'm going to reference is open source on my GitHub," he told the audience. "Not open source in the sense that you can just download it and run it — more like, the code is there if you want to figure out how to do it yourself. It does all run for me, so good luck."

That honesty runs through the entire talk. The lessons McMillin shares are not theoretical. They come from running a system that, at the time of the conference, comprised 19 agents (13 interactive, plus service agents running on cron), 73 MCP tools on a local stdio server, and multiple access interfaces including CLI, web UI, voice, and a personal Slack integration. He had updated the numbers the night before and joked they were probably already out of date.

What makes this talk unusually valuable for security practitioners is the failure-first framing. McMillin explicitly walks through what he did wrong, what broke, and how he fixed it — a pattern that rarely makes it into formal security presentations.

▶ Watch: The system overview (00:00)

Defense Layer 1: Capability Bounding and Permission Control

The first and most foundational control McMillin put in place was capability bounding — a deny-by-default permission system where each agent only has access to the tools it genuinely needs.

The principle he articulates is clean: "If you can't enumerate what an agent CAN do, you can't reason about what it SHOULDN'T do." Each agent in his system has both an explicit allowlist of tools and a permissions level for each of those tools (READ, WRITE, SEND, ADMIN). Unknown tools default to ADMIN — which maps to deny — by default.

His permission matrix, as described in the slides, looks roughly like this:

| Agent | Allowed Tools | Permissions |

|---|---|---|

| Task Manager | web, memory, Slack, email | READ, WRITE, SEND |

| Security Researcher | web, memory, RAG search | READ only |

| Email Intake | email read, email send | READ, SEND |

| Chatbot | all 50 tools | Full (general purpose) |

The key design principle: "New tools are locked down by default. You opt IN to access, not out." As McMillin adds tools to the ever-expanding MCP server, they are not automatically available to any agent — a developer has to explicitly grant access.

During the live demo, McMillin showed what this looks like in practice: the chatbot agent running with read and send permissions but no memory write access, being asked to save a memory, and receiving a denial. Critically, the error message is verbose enough that the agent itself understands what happened and can communicate it — it doesn't just silently fail.

This approach also has a direct functional benefit. With 73 MCP tools available, injecting all of them into every agent's context wastes tokens and gives agents a large amount of irrelevant information. Scoping down tool access keeps agents focused on their actual mission.

▶ Watch: Capability bounding demo (04:00)

The Memory Failure Story: When Agents Share a Brain

The most vivid failure story McMillin shared involves what happens when multiple agents share the same memory store without any isolation.

His initial approach was simple: create a PostgreSQL table, write memories to it, done. It worked — agents did remember things. But it produced a cascade of strange functional problems. The clearest one: he had built a "business agent" and asked it to browse his GitHub and figure out how he could generate $1,000 a month. Every time he subsequently asked his task manager agent to break down tasks for him, it would drag that $1,000 goal into the conversation, because both agents were pulling from the same memory store.

The security implication goes further than functional annoyance. As he noted in the talk: if an attacker could poison the email intake agent's memory — which processes untrusted input — they could potentially influence the responses of all other agents drawing from that same pool.

The fix was a proper memory namespace isolation system. Each agent now has its own identity, and that identity maps to a memory namespace. The identity is derived from the Python class name of the agent subclass — crucially, the class name is passed to the MCP server at startup, not by the LLM at runtime. "LLMs have no control over what their namespace key is — they can't decide they're somebody else and grab another agent's memories."

The architecture also now distinguishes between memories that get auto-injected at context startup and memories that require an explicit search call. Injecting all memories at startup was filling context windows rapidly and hurting performance. Now only high-importance memories (filtered by an importance score) get auto-injected; everything else requires a deliberate retrieval.

▶ Watch: Memory isolation — before and after (10:00)

Prompt Injection Detection: Not Plug-and-Play

McMillin deployed an LLM firewall for prompt injection detection — and ran into the same problem many practitioners face: out-of-the-box configurations are too aggressive.

The firewall was silently blocking legitimate tasks he had queued. He didn't discover this until reviewing firewall logs two weeks later. One of the blocked items: a task to "add a server time field (ISO 8601 format) in every API response." That is a real task McMillin wanted done. Another blocked item: "What are the top prompt injection techniques?" — a query from his security researcher agent doing exactly what it is supposed to do.

The lesson is not that prompt injection detection is useless — it is that the sensitivity settings need to be tuned per-agent type. A security researcher agent is inherently going to be asking about attack techniques and injection patterns; its firewall config needs to be either very relaxed or absent entirely for those query types. An email intake agent processing untrusted external input, on the other hand, should have the strictest settings available.

McMillin also articulated an important availability preference: "If the firewall API is down, log a warning and continue — availability over perfect security." This reflects the reality of running a system you depend on daily. A firewall that takes the entire agent offline when it fails is worse than no firewall at all.

▶ Watch: Prompt injection detection in practice (14:01)

Context-Aware Trimming: Keeping Security Events Alive

One of the more novel defensive ideas McMillin presented addresses a subtle attack vector: context window trimming.

As LLM conversations grow, older messages get dropped to stay within the context limit. An attacker who understands this could space their attack attempts 200,000 tokens apart — executing the first attempt, waiting for it to be trimmed from the context, then retrying. The agent would have no memory of the previous attempt or the fact that it had been blocked.

McMillin's solution is context-aware trimming: tagging certain messages — blocked SSRF attempts, permission denials, prompt injection flags — as security events that are "pinned" and survive trimming regardless of how old they are. As the slides put it: "Without this, an attacker waits for context trimming, then retries the same attack. Agent has no memory of the previous attempt."

The implementation tags messages at the time they are generated, and the trimming logic preserves tagged messages unconditionally while pruning general conversation. It is not a complete solution — as contexts get very long and agents run for extended durations, there are still edge cases — but it meaningfully raises the cost of a retry-based attack.

▶ Watch: Context-aware trimming explained (14:01)

Observability: The Insight That Paid for Itself

Observability was the area where McMillin found the most immediate return on investment. Turning on cost tracking in week one of his observability setup revealed a 10x token waste problem.

The root cause was twofold: injecting all memories on every turn (including low-importance ones) was burning tokens unnecessarily, and his coding agents were spinning up all five review sub-agents every time they ran because that's what the LLM decided would be helpful when he built the feature. Running all five review agents adds approximately 250,000 tokens per invocation — a significant cost when triggered unnecessarily.

His current setup uses Langfuse for per-turn traces (with tool call spans, token counts, and latency), Grafana dashboards for per-agent cost tracking and budget alerts, and a security audit trail that logs permission denials and SSRF blocks with full context and agent ID. His stated benchmark: "If someone asks what your agent did at 3am, you can answer with exact tool calls, inputs, and security events."

This level of observability also caught a specific category of mistake: accidentally wiring up Claude Opus where he had intended to use Claude Sonnet — a costly error that only became visible once he had traces detailed enough to show which model was being called per turn.

▶ Watch: Observability and cost tracking (16:01)

The LLM Chat Room: A Preview of Mistakes

In a section McMillin explicitly framed as "a preview of mistakes I'm definitely going to make," he described an MCP relay — essentially a chat room for LLMs. The system lets two Claude Code sessions communicate with each other: one debugging a client, one with access to server-side code, chatting back and forth to solve a problem collaboratively.

He acknowledged it directly: "This is absolutely not a secure setup right now. We're injecting context from one LLM to another, capabilities are not being scoped down." It runs on his home lab, it is not production-ready, and he is working on adding Redis persistence and scoping down inter-agent capabilities.

The value of including this in the talk is the honest modeling of what an iterative security approach looks like: you build something useful, you flag its vulnerabilities openly, and you work toward hardening it over time rather than holding it back until it's perfect.

▶ Watch: The LLM chat room (06:00)

Notable Quotes

"If you can't enumerate what an agent CAN do, you can't reason about what it SHOULDN'T do." — ▶ 04:00

"New tools are locked down by default. You opt IN to access, not out." — ▶ 04:00

"10x token waste found in week one of cost tracking." — ▶ 16:01

"If someone asks what your agent did at 3am, you can answer with exact tool calls, inputs, and security events." — ▶ 16:01

Key Takeaways

• Capability bounding is both a security and a functionality improvement. Restricting each agent to only the tools it needs reduces its attack surface and keeps its context focused — both of which make it more effective.
• Memory isolation is not optional in multi-agent systems. Without namespace separation, memory becomes a cross-contamination vector that can be exploited by poisoning any agent that processes untrusted input.
• Prompt injection detection requires per-agent tuning. Out-of-the-box configurations will block legitimate queries. A security researcher agent and an email intake agent need radically different sensitivity settings.
• Pin security events in context to resist retry-based attacks. An attacker who understands context trimming can space attacks to avoid detection. Tagging and preserving security events defeats this approach.
• Observability pays for itself immediately. Cost tracking and per-agent traces reveal waste and misconfigurations that compound quickly at scale, and enable the kind of security audit trail that autonomous agents require.

Slides Reference

Slides are available from the conference as 2026-04-04-D2-S1-11-43-Building-Secure-Agentic-Systems-Lesson.pdf. Key slides cover the full system architecture (14 agents, 50 tools, one security architecture), the capability bounding and permission matrix, the memory isolation failure story, context-aware trimming design, and the observability stack (Langfuse + Grafana). Source code available at github.com/brooksmcmillin/agents.