The Dangers of Serverless & IAM Proliferation
Sam Thorogood (Senior Consultant)
purplecon 2024 · Day 1 · Main Room
Overview
In his thought-provoking talk at purplecon, "The Dangers of Serverless & IAM Proliferation," Sam Thorogood, a Senior Consultant, challenged the conventional wisdom surrounding modern cloud architectures. Thorogood argued that while cloud providers present serverless and their diverse primitives as the "well-lit path" to efficient development, this often leads to an alarming increase in architectural complexity. This complexity, he posited, not only hinders understanding and maintainability but also significantly expands the security attack surface, particularly through the unchecked proliferation of IAM (Identity and Access Management) roles.

Key moments
- 0:00 Introduction and the overwhelming AWS diagram
- 2:00 The true complexity of serverless solutions
- 3:20 A simpler, more tractable service design
- 4:30 Embracing the Grug Brain Developer philosophy
- 8:00 Avoiding the 'Lego trap' of cloud primitives
- 11:00 Practical 'do's and don'ts' for cloud services
- 13:00 IAM proliferation: The pointy end of complexity
- 14:00 Final takeaway: Don't follow patterns blindly
The Dangers of Serverless & IAM Proliferation
Speakers: Sam Thorogood, Senior Consultant
Conference: purplecon
YouTube: https://www.youtube.com/watch?v=IsHNK0m7D2U
Overview
In his thought-provoking talk at purplecon, "The Dangers of Serverless & IAM Proliferation," Sam Thorogood, a Senior Consultant, challenged the conventional wisdom surrounding modern cloud architectures. Thorogood argued that while cloud providers present serverless and their diverse primitives as the "well-lit path" to efficient development, this often leads to an alarming increase in architectural complexity. This complexity, he posited, not only hinders understanding and maintainability but also significantly expands the security attack surface, particularly through the unchecked proliferation of IAM (Identity and Access Management) roles.
Thorogood's presentation served as a critical call to action for developers and security professionals alike, urging them to question blindly adopted "best practices." Drawing from his extensive experience, including a tenure at Google, he highlighted how the pursuit of perceived scalability and cloud-nativeness can inadvertently lead to over-engineered systems that are difficult to secure and costly to operate. The talk emphasized the importance of architectural simplicity as a foundational security principle, advocating for a more deliberate and critical approach to leveraging cloud services.
This article delves into Thorogood's arguments, examining the pitfalls of excessive serverless adoption, the specific challenges posed by IAM proliferation, and practical advice for building more secure and manageable cloud-native applications. It aims to distill the core message: that true innovation lies not in assembling an ever-growing array of complex primitives, but in crafting elegant, understandable solutions that prioritize clarity and robust security.
Background
▶ Watch: Introduction and the overwhelming AWS diagram (0:00)
The genesis of Thorogood's talk lies in a fundamental dilemma faced by modern software developers: the overwhelming array of choices and prescriptive "best practices" offered by major cloud providers. He vividly illustrated this with a common scenario: a developer tasked with building a simple video processing service—taking user-uploaded videos, resizing them, and storing them for later access. What should be a straightforward architectural problem often devolves into a labyrinthine solution.
Thorogood presented an actual diagram from a major cloud provider for such a service, describing it as "6,000 lines of configuration" and an incomprehensible "maze" for even experienced developers, let alone novices. This complexity is often a direct result of following the "well-lit path"—a term he borrowed from his time in Google's Developer Relations. This path, he explained, often means developers are "banging something into Google, going to the first page that has the content, and copy-pasting it into your code." This uncritical adoption of complex, provider-recommended architectures, often fueled by the promise of infinite scalability and reduced operational overhead, creates systems that are far more intricate than necessary.
The speaker attributed this phenomenon partly to a developer's innate desire to solve puzzles, a "dopamine hit" derived from assembling various cloud "primitives" like building blocks. However, he cautioned that this approach often leads to developers "not seeing the forest for the trees," blindly believing that the only way to build software is through these specific, often over-engineered, tools pushed by providers. These serverless building blocks, while powerful, are frequently oversold and pitched at a level inappropriate for most small to medium-sized companies or even many enterprise use cases. They are designed to avoid "running a real computer," yet they introduce a different, often more insidious, form of complexity.
Thorogood championed a philosophy inspired by "the grug brain developer," advocating for simplicity and avoiding unnecessary complexity. He stressed that the core job of a software engineer is to take complicated concepts and turn them into "magic" for the end-user, abstracting away internal complexity. However, he observed that the industry often fails to apply this principle to its own internal systems, resulting in bloated, poorly abstracted architectures that make understanding and securing them incredibly difficult.
Key Findings
▶ Watch: A simpler, more tractable service design (3:20)
Sam Thorogood's talk elucidated several critical findings regarding the modern cloud development landscape:
- Complexity as the Apex Predator: The most significant finding was that complexity, particularly the kind introduced by over-reliance on serverless architectures and numerous cloud primitives, is the primary adversary of good software design and, crucially, security. He demonstrated how a seemingly simple task like video processing could balloon into a system involving "12 different services," leading to a diagram so intricate it becomes indecipherable. This excessive complexity makes it nearly impossible to grasp the system's full attack surface or identify potential vulnerabilities.
- IAM Proliferation as a Direct Consequence: A central and alarming finding was the direct link between serverless architectures and IAM proliferation. As developers adopt more granular serverless components, each often requires its own distinct set of permissions and IAM roles. This leads to an unmanageable explosion of roles and policies, creating a "pointy end of that complexity." Thorogood recounted instances in his consulting work where companies literally "ran out of IAM roles" in their AWS accounts, a situation he described as "totally bonkers," yet a stark indicator of the problem's severity. Such proliferation significantly increases the risk of misconfigured permissions, leading to privilege escalation or unauthorized access.
- The Deceptive "Well-Lit Path": Cloud providers often present their complex, multi-service reference architectures as "best practices" or "well-lit paths." Thorogood argued that this is often "disingenuous," leading developers to blindly adopt solutions that are far more intricate than their needs dictate. This uncritical adoption results in systems that scale bills more effectively than they solve problems, without a clear understanding of the underlying trade-offs.
- Poor Abstraction in Cloud Primitives: The talk highlighted a failing in how cloud providers design their primitives. Instead of offering clear, high-level abstractions, many services are presented as low-level building blocks that, when combined, lack a coherent overarching structure. This absence of clear "silos" or logical groupings makes it difficult to reason about the system holistically, further contributing to complexity.
- Simplicity Enhances Security: Counter-intuitively for some, Thorogood asserted that having "simpler bits of software makes it easier to understand your vulnerabilities." Fewer, well-understood components mean a smaller, more comprehensible attack surface. This contrasts sharply with the "more pieces, more surface area" reality of overly complex serverless deployments.
Technical Deep Dive
▶ Watch: Avoiding the 'Lego trap' of cloud primitives (8:00)
Thorogood's technical deep dive centered on contrasting a pragmatic, simple architecture with the overly complex serverless paradigms often pushed by cloud providers. He began by sketching a conceptually simple video processing service: a user uploads a video, a server processes it (potentially with a background task for longer operations), the processed video is stored, and then served back to the user. This "pretty tractable diagram," as he described it, stands in stark contrast to the "6,000 lines of configuration" AWS diagram for the same task, which he characterized as a "blob of stuff" without clear abstractions.
The core issue, Thorogood explained, is how cloud providers package and market their "primitives"—services like AWS Lambda, DynamoDB, or Google Compute Engine. These are presented as essential building blocks for modern applications, collectively forming the serverless ecosystem. While acknowledging their utility, he stressed that these primitives are often "oversold" and "pitched at a level that is not really appropriate for most of us." The implicit message from providers is that using these tools is the only way to avoid the perceived horrors of "running a real computer."
Thorogood then explored the two extremes of infrastructure management. On one end, running your own hardware (as was common in the 90s) necessitates managing everything: firewalls, SSL certificates, IP addresses, process isolation, and container management. This is a "bad time" for most organizations. On the other end is the blind adoption of every cloud primitive, leading to the aforementioned complexity.
He advocated for a "middle ground," where organizations can leverage some cloud primitives strategically without going "full red pill or blue pill." This approach involves using managed services for undifferentiated heavy lifting—tasks that are standardized and have "only really one right way to do." Examples include:
- Managed SSL Certificate Generation: Instead of generating SSL certificates with tools like Let's Encrypt, use a provider that handles this seamlessly.
- DoS Protection: Offload Distributed Denial of Service (DoS) mitigation to cloud-managed services.
- IP Address Management: Let the cloud provider manage network infrastructure and IP addresses.
Conversely, Thorogood provided specific "do's and don'ts" for core application architecture:
- Do Serve HTTP: The web is built on HTTP. Applications should communicate using standard HTTP protocols, avoiding proprietary or overly abstracted invocation methods that hinder interoperability and create vendor lock-in.
- Do Get Things Talking: A significant portion of cloud provider offerings boils down to enabling inter-service communication. He referenced a blog post by Tailscale, which highlights how simplifying network topology (e.g., creating a secure "same LAN" environment for services) can eliminate the need for complex database logins or extensive certificate management between internal components.
- Do Prevent Unauthorized Access: For authentication and authorization, stick to established standards like OAuth and JWT (JSON Web Tokens). Do not "reinvent the wheel" in this critical security domain.
- Don't Over-Engineer for Scale: While scalability is important, Thorogood cautioned against preemptively designing for "infinite users" with complex, expensive solutions. Many scaling problems can be addressed with "low-tech solutions" when they arise, and premature optimization often leads to unnecessary complexity and cost.
Crucially, Thorogood revisited IAM proliferation. He explained that each serverless component, such as a Lambda function or a DynamoDB table, typically requires its own specific IAM role with fine-grained permissions. While granular permissions are desirable in principle, the sheer number of distinct components in a complex serverless architecture leads to an unmanageable explosion of roles and policies. This makes it incredibly difficult to audit, understand, and secure the overall permission model, creating a vast attack surface where a single misconfigured role can lead to significant compromise. The anecdote of companies "running out of IAM roles" underscored the practical, and alarming, extent of this issue.
Demo / Proof of Concept
▶ Watch: Practical 'do's and don'ts' for cloud services (11:00)
Sam Thorogood's talk did not feature a live code demonstration or a specific technical proof of concept in the traditional sense. Instead, the "demo" was conceptual, using a common scenario – building a video processing and serving website – to illustrate the core problem of architectural complexity.
He presented and analyzed an actual, highly intricate diagram from a major cloud provider that purported to show a "best practice" architecture for such a service. This diagram, described as "6,000 lines of configuration" and a "maze," served as the primary visual "proof" of the over-engineering and poor abstraction he was critiquing. By contrasting this convoluted diagram with a simple, more traditional three-box architecture (user -> server -> storage), Thorogood effectively demonstrated the philosophical and practical differences in design approaches without needing to execute live code. The power of his presentation lay in this direct, visual comparison of complexity versus simplicity for a common use case.
Defensive Implications
▶ Watch: Final takeaway: Don't follow patterns blindly (14:00)
The defensive implications derived from Sam Thorogood's talk are profound and call for a fundamental shift in how security professionals and developers approach cloud architecture.
- Prioritize Architectural Simplicity: The foremost defensive strategy is to actively combat complexity. Defenders should advocate for and design systems with fewer, well-understood components. A smaller, more comprehensible attack surface is inherently easier to secure, monitor, and audit. This means questioning every new service or primitive introduced into an architecture and evaluating its necessity against the added complexity and potential security burden.
- Scrutinize Cloud Provider "Best Practices": Security teams must move beyond blindly accepting cloud provider reference architectures as gospel. These "well-lit paths" should be critically evaluated for unnecessary complexity, potential vendor lock-in, and the actual security benefits versus the increased management overhead. Instead of defaulting to the most feature-rich or serverless-heavy solution, teams should prioritize the simplest architecture that meets functional and security requirements.
- Implement Robust IAM Hygiene: Addressing IAM proliferation is critical. Security teams need to establish stringent policies for IAM role creation, ensuring that every role has a clearly defined purpose, minimal necessary permissions (least privilege), and a documented owner. Automated tools for auditing IAM policies, identifying over-privileged roles, and detecting unused roles become essential. The goal is to prevent the "running out of IAM roles" scenario by actively managing and consolidating permissions where appropriate, rather than allowing them to proliferate unchecked.
- Simplify Inter-Service Communication Security: Thorogood's point about "getting things talking" highlights an opportunity to simplify network security. Instead of relying on complex firewall rules, security groups, and individual service authentication for every internal connection, defenders can explore solutions like secure mesh networks (e.g., using Tailscale or similar zero-trust networking solutions). These can create a trusted, encrypted "same LAN" environment for internal services, significantly reducing the complexity of securing east-west traffic and simplifying authentication between internal components.
- Leverage Managed Services Strategically: Defenders should encourage the use of managed cloud services for undifferentiated heavy lifting—tasks that are non-core to the business and have a clear, secure "one right way" of being done. This includes services for SSL certificate management, DoS protection, and core networking infrastructure. Offloading these responsibilities to the cloud provider can free up security teams to focus on application-specific vulnerabilities and business logic, where their expertise is most critical. However, this strategic adoption must not extend to abstracting away core application logic into an unmanageable array of primitives.
- Foster a Culture of Questioning: Ultimately, a strong defensive posture requires a culture where developers and security professionals are empowered to question architectural choices, even those presented as industry standards. This involves asking "why" a particular tool or pattern is being used, rather than just "how" to implement it. By understanding the trade-offs and potential security implications of each architectural decision, teams can build more resilient and defensible systems from the ground up.
Key Takeaways
- Complexity is the Enemy of Security: Over-engineered architectures, especially those resulting from blind adoption of numerous serverless primitives, significantly increase the attack surface and make vulnerabilities harder to identify and mitigate.
- IAM Proliferation is a Critical Threat: The granular nature of serverless components often leads to an unmanageable explosion of IAM roles and permissions, heightening the risk of misconfiguration, privilege escalation, and unauthorized access.
- Question the "Well-Lit Path": Cloud provider "best practices" and reference architectures should be critically evaluated for unnecessary complexity and vendor lock-in, rather than being adopted without scrutiny.
- Embrace the "Grug Brain Developer" Philosophy: Prioritize simplicity, clear abstractions, and understandable architectures over perceived "cloud-nativeness" or infinite scalability for most use cases.
- Strategic Use of Managed Services: Leverage managed cloud services for undifferentiated heavy lifting (e.g., SSL, DoS protection, IP management) to offload operational burden, but maintain simple, coherent application architectures.
- Understand the "Why," Not Just the "How": Developers and security professionals must understand the contextual reasons and trade-offs behind architectural choices, not just how to implement a given cloud tool or pattern.
About the Speaker(s)
Sam Thorogood is a Senior Consultant, currently "fun employed," who specializes in advising organizations on their technical strategies. His talks often draw from his extensive experience, including a significant tenure at Google, where he worked in Developer Relations. Thorogood frames his presentations as insights he wished he had known earlier in his career, sharing valuable lessons learned about software development and architectural philosophy. He is known for his analytical approach to modern challenges in technology and can be found online under the handle "samthor."
Reviews
Dr. Zero (Offensive Security Researcher) — SOLID
A reasonable rant against serverless complexity and IAM sprawl, but it's philosophy dressed as findings. Thorogood names the problem clearly—cloud providers push over-engineered reference architectures, developers cargo-cult them, IAM roles explode—but the 'technical deep dive' is just architecture diagrams and vibes. No exploitation, no metrics, no novel insight.
Heather Calloway (CISO) — SOLID
A useful corrective for teams drowning in serverless sprawl, but stays at the philosophy level. I'd hand this to engineering leadership as a conversation starter, not a playbook.