Threat Modeling Stories From the Front Line

David Johansson (Synopsis)

Swiss Cyber Storm 2019 · Day 1 · Main

Overview

In this insightful talk from Swiss Cyber Storm, David Johansson, a seasoned Security Consultant at Synopsis, delves into the critical but often misunderstood practice of threat modeling. Johansson shares compelling real-world anecdotes from his "front line" experience, illustrating how even seemingly robust security controls can fail when their design, integration, or deployment is flawed. The presentation serves as a stark reminder that security is not merely about implementing individual controls, but understanding how they interact within a broader system and its operational environment.

Watch on YouTube

Visual summary for Threat Modeling Stories From the Front Line by David Johansson
Visual summary for Threat Modeling Stories From the Front Line by David Johansson

Key moments

  1. 2:00 What is threat modeling and common design flaws
  2. 4:15 Case 1: Ineffective integration of multiple security controls
  3. 8:30 Case 2: Bypassing step-up authentication by mixing sessions
  4. 16:00 Case 3: Insecure password storage with unintended side-effects
  5. 21:30 Case 4: One-time password replay due to synchronization issues
  6. 24:30 Summary: The importance of combining theory and practice
  7. 27:30 Q&A: Simplifying design to reduce attack surface

Threat Modeling Stories From the Front Line

Speakers: David Johansson, Security Consultant, Synopsis

Conference: Swiss Cyber Storm

YouTube: https://www.youtube.com/watch?v=HmapajsX7w0

Overview

In this insightful talk from Swiss Cyber Storm, David Johansson, a seasoned Security Consultant at Synopsis, delves into the critical but often misunderstood practice of threat modeling. Johansson shares compelling real-world anecdotes from his "front line" experience, illustrating how even seemingly robust security controls can fail when their design, integration, or deployment is flawed. The presentation serves as a stark reminder that security is not merely about implementing individual controls, but understanding how they interact within a broader system and its operational environment.

Johansson’s core message emphasizes that a superficial "checkbox" approach to security is insufficient. Instead, he advocates for a deep, analytical dive into the actual implementation and deployment context of security features. Through a series of practical examples, he uncovers common design flaws that lead to significant vulnerabilities, ranging from misconfigured authentication mechanisms to unintended side-effects of security "fixes." This talk is essential for anyone involved in software development, architecture, or security, highlighting the necessity of rigorous threat modeling to bridge the gap between theoretical security and practical resilience.

The talk underscores why comprehensive threat modeling matters: it reveals weaknesses in the design phase, long before code is written or systems are deployed. By challenging assumptions and scrutinizing the interplay of components, organizations can proactively identify and mitigate risks that might otherwise go unnoticed until exploited. Johansson's stories are not just cautionary tales but actionable lessons, urging practitioners to adopt a more holistic and inquisitive approach to securing their systems.

Background

▶ Watch: What is threat modeling and common design flaws (2:00)

Threat modeling is a structured process used to identify potential security threats, vulnerabilities, and risks in a system's design. As Johansson explains, it typically involves first modeling the system to understand its components and data flows, then analyzing that model to identify design flaws, and finally determining appropriate mitigations for identified risks. While often compared to other security activities like penetration testing, threat modeling focuses on "breaking the design" rather than just breaking the implemented code. It's a proactive technique aimed at preventing vulnerabilities from ever making it into production.

The fundamental problem that threat modeling addresses is the inherent complexity of modern software systems and the tendency for security controls to be implemented in isolation without a holistic view of their interactions. Developers and architects often select security controls based on best practices or compliance requirements, but they may overlook critical details about how these controls integrate with one another, how they handle edge cases, or how they behave in specific deployment scenarios. This often leads to a false sense of security, where a system appears to have all the necessary security features on paper, but harbors exploitable design flaws in practice.

Johansson illustrates this concept with a poignant image of a gate on a road that can be easily bypassed by driving around it. This analogy perfectly encapsulates the common scenario in software where a strong security control in one place is rendered ineffective by an overlooked bypass elsewhere. Similarly, using a control for the wrong purpose, such as relying on encryption alone for integrity protection, demonstrates a misunderstanding of the control's capabilities and limitations. These scenarios highlight why a deeper, design-level analysis is crucial, moving beyond the mere presence of security features to scrutinize their effectiveness in context. The challenges arise from assumptions, lack of cross-functional understanding, and the dynamic nature of system evolution and deployment.

Key Findings

▶ Watch: Case 2: Bypassing step-up authentication by mixing sessions (8:30)

David Johansson's talk presents several key findings that challenge conventional wisdom about security implementation, derived from real-world threat modeling engagements:

  • Security Controls Are Not Enough in Isolation: A critical finding is that simply having "good" security controls (e.g., mutual authentication, XML digital signatures, secure logging) does not guarantee a secure system. The effectiveness of these controls hinges entirely on how they are integrated and how they correlate information across different layers and components. A lack of correlation can allow an attacker to bypass intended protections by mixing and matching identities or attributes from different control mechanisms.
  • Deep Dive into Implementation is Essential: Superficial checks or "checkbox security" are insufficient. Threat modeling must involve a deep dive into the actual implementation details of security controls, not just their theoretical design. High-level designs can hide significant flaws in how controls manage state, handle user sessions, or interact with other system components, as demonstrated by the step-up authentication bypass.
  • Fixing One Problem Can Create Another: Security fixes, even well-intentioned ones, can inadvertently introduce new vulnerabilities or shift existing risks to less protected areas. This "whack-a-mole" effect often occurs when the broader system context, including deployment and operational details, is not fully re-evaluated after a change. Iterative threat modeling is crucial to assess the implications of any significant modification to security-critical components.
  • Deployment Environment Matters Critically: The operational environment and deployment architecture can fundamentally alter the security posture of a system. Assumptions made during design might not hold true in a distributed, load-balanced, or high-availability setup. Network topology, server configurations, and inter-instance synchronization can introduce new attack vectors, as vividly illustrated by the one-time password replay vulnerability.
  • Complexity is the Enemy of Security: While not explicitly a "finding" from a specific story, Johansson's talk implicitly and explicitly (in the Q&A) highlights that increased system complexity often leads to more opportunities for design flaws and vulnerabilities. Adding more security controls without simplifying the underlying design can paradoxically increase the attack surface and make thorough security analysis more difficult.
  • Authenticated Data is Paramount for Security Decisions: Any security decision, especially those related to identity, authorization, or logging, must be based on authenticated and trusted data sources. Relying on user-supplied or unverified data for critical security functions is a common design flaw that attackers can easily exploit to spoof identities or tamper with audit trails.

These findings collectively underscore the necessity of a holistic, iterative, and deeply analytical approach to threat modeling, moving beyond theoretical best practices to confront the realities of complex system design and deployment.

Technical Deep Dive

▶ Watch: Case 3: Insecure password storage with unintended side-effects (16:00)

Johansson illustrates his key findings with four distinct stories, each revealing a common design flaw exposed through threat modeling:

1. Security Controls Not Integrated: The B2B Web Service

This first example involved a business-to-business (B2B) application with stringent security requirements for client identification and message tracking. The system employed multiple, seemingly robust security controls:

  • Transport Layer Security: Clients needed a client certificate for mutual authentication to connect to the reverse proxy, ensuring identity at the network level.
  • Message Layer Security: Messages were protected using Web Service Security (WS-Security) standards, incorporating XML digital signatures to verify the sender's identity and message integrity.
  • Secure Logging: Audit logs were not only stored but also shipped to a secure log server, encrypted and signed to prevent tampering.

The critical design flaw emerged when questioning the correlation between these controls. The architects had implemented individual controls but failed to integrate them:

  • There was no check to ensure that the identity from the client certificate (transport layer) matched the identity from the XML digital signature (message layer).
  • Crucially, the identity used for logging and audit trails was extracted from a custom header field within the message, not from the authenticated client certificate or the verified XML signature.

Attack Scenario: An attacker could connect to the system using their own legitimate client certificate (e.g., Client A). They could then sign a message with a different client's certificate (e.g., Client B) using XML digital signatures. Finally, they could insert a third, arbitrary identity (e.g., Client C) into the custom header field that the system used for logging. The system would process the request because both the transport-layer authentication and message-layer signature would pass individually. However, the audit logs would falsely record the transaction as being performed by "Client C," making it impossible to accurately trace the true sender or trust the audit trail. This demonstrated that good controls are ineffective if they don't work in concert and rely on authenticated data.

2. Step-Up Authentication Bypass: The Trading System

This scenario involved a mobile trading system where users initially logged in with a simple password. For sensitive operations, such as a large financial transaction (e.g., selling 10,000 units of Acme Corp), the system required step-up authentication (e.g., a One-Time Password (OTP) from an SMS or authenticator app) to gain higher assurance of the user's identity.

The system's flow was:

  1. User logs in, obtains Session 1.
  2. User initiates a high-risk transaction.
  3. The backend challenges the user for step-up authentication.
  4. User provides OTP, which is validated and marked as used.
  5. The system issues a separate, higher-assurance Session 2.
  6. The transaction is executed using Session 2.

The design flaw was that the system **did not verify that Session 1 and Session 2 belonged to the *same user***. Each session was checked independently by different components (potentially in a microservices architecture).

Attack Scenario: An attacker could:

  1. Create their own account, log in, perform a step-up, and obtain their own Session 1 and Session 2.
  2. Compromise a victim's simple password, log in as the victim, and obtain the victim's Session 1.
  3. The attacker then crafts a request that combines the victim's Session 1 with their own (attacker's) Session 2.
  4. When this request reaches the system, the component checking Session 1 sees a valid session for the victim. The component checking Session 2 sees a valid, stepped-up session (from the attacker). Since no overall correlation check was performed, the system concluded that the victim had successfully performed step-up authentication and executed the high-value transaction on the victim's behalf. This highlights the importance of analyzing the state model and ensuring identity correlation across different session states.

3. Insecure Side-Effects of Secure Password Storage: The Retail Company

Initially, a retail company stored user passwords in plaintext in its database – a severe vulnerability. The security department later mandated a fix, implementing modern password storage best practices:

  • Using bcrypt for hashing.
  • Employing an appropriate number of rounds.
  • Generating unique salts for each password.

The IT security department was satisfied, believing the plaintext password problem was solved. However, the fraud department relied on access to plaintext passwords for their legacy fraud detection algorithms. When their algorithms broke, they requested a solution.

The developers, trying to be helpful and avoid storing plaintext passwords in the database, devised a "temporary" fix:

  • When a user logged in, their plaintext password was stored in the session in memory for a short period.
  • This allowed the fraud engine to retrieve the password from the session for its algorithms.

Design Flaw and Deployment Issue: The assumption that the password was temporarily in memory was flawed due to the actual deployment environment. Being a large company, they used multiple web application servers in a web farm, maintaining state across these servers via a session database server. Consequently, the "temporary in-memory" password was actually persisted to disk in the session database server, unencrypted (though possibly hex-encoded), effectively moving plaintext passwords from a protected datacenter database to a less protected server in the DMZ. This was a classic case of fixing one problem while inadvertently creating a new, potentially worse, vulnerability by failing to re-evaluate the threat model in the context of the deployment environment.

4. One-Time Passwords in Parallel Universes: Remote Access

This final example involved a multinational company providing remote access to its workers, requiring strong authentication: normal domain credentials combined with a One-Time Password (OTP), such as an RSA SecurID token. A key security feature was that once an OTP was used, it would be marked as invalid to prevent replay attacks.

To handle a large user base globally, the company deployed a high-availability solution with multiple server instances, load-balancing incoming requests.

Design Flaw: The multiple instances lacked appropriate synchronization for the OTP state. While an OTP was marked as "used" on the specific instance that processed the initial login, this state change was not immediately or reliably propagated across all other load-balanced instances.

Attack Scenario: An attacker could capture a legitimate user's OTP during a login attempt. Within a short time window (before synchronization occurred), the attacker could attempt to replay the captured OTP. Due to the load-balancing, there was approximately a 50% chance that the replay request would hit a different server instance that had not yet received the "OTP used" update. This instance would then validate the "unused" OTP, granting the attacker unauthorized access. This highlights the critical need to consider the deployment environment, especially distributed architectures, and ensure robust state synchronization for security-sensitive data.

Demo / Proof of Concept

▶ Watch: Summary: The importance of combining theory and practice (24:30)

While the talk primarily focuses on design flaws uncovered through analytical threat modeling, David Johansson explicitly mentions "hands-on testing" for the "One-Time Passwords in Parallel Universes" example.

Specifically, he states: "in this case and we actually did some hands-on testing as well to confirm this we were able to in about nearly fifty percent of the cases to within a time window replay this and it was actually accepted by the server." This confirms that the vulnerability related to the lack of OTP synchronization across load-balanced instances was not merely theoretical but was actively demonstrated and verified through a Proof of Concept (PoC). The success rate of approximately 50% underscores the practical exploitability of this design flaw in a real-world, distributed environment.

For the other examples, the flaws were described as being identified through the analytical process of threat modeling, rather than live demonstrations or PoCs.

Defensive Implications

▶ Watch: Q&A: Simplifying design to reduce attack surface (27:30)

The stories shared by David Johansson offer crucial lessons for defenders, emphasizing a shift from a superficial to a deeply analytical approach to security:

  • Prioritize Integration and Correlation of Controls: Do not treat security controls in isolation. Defenders must ensure that authentication mechanisms across different layers (e.g., transport, message, session) are properly integrated and that identities are correlated. Any critical security decision, especially for logging and authorization, must be based on data derived from authenticated sources, not user-supplied inputs.
  • Demand Deep Implementation Details: Move beyond "checkbox security." When evaluating security features, ask probing questions about their specific implementation. Understand how sessions are managed, how state changes are handled, and what assumptions are made. This includes scrutinizing state models and component interactions, as high-level designs can mask critical vulnerabilities.
  • Adopt Iterative Threat Modeling: Security is not a one-time activity. Any significant change to an application, especially concerning security controls, necessitates revisiting and iterating on the threat model. This ensures that new features or fixes do not inadvertently introduce new risks or shift existing vulnerabilities to less protected areas.
  • Always Consider the Deployment Environment: The theoretical design of a system can be drastically different from its practical security posture in a live deployment. Factors like load balancing, high availability, distributed architectures, and network segmentation (e.g., DMZ) introduce unique challenges. Defenders must explicitly incorporate the deployment environment into their threat models to identify environment-specific risks.
  • Ensure Robust State Synchronization in Distributed Systems: For security-critical state (like used OTPs, session validity, or authorization tokens) in distributed or load-balanced environments, robust and timely synchronization mechanisms are paramount. A lack of synchronization can lead to replay attacks or inconsistent security decisions across instances, creating exploitable windows for attackers.
  • Strive for Simplicity in Design: As highlighted in the Q&A, complexity is often the enemy of security. Overly complex designs with many interconnected controls are harder to analyze, verify, and secure. Defenders should advocate for simpler, more elegant architectural solutions that minimize attack surface and reduce the potential for hidden flaws.
  • Foster Cross-Functional Collaboration: Effective threat modeling and defense require collaboration between security professionals, developers, and operations teams. Security teams need to understand how developers implement features and how operations deploys and manages systems. Conversely, development and operations teams need to understand security principles and potential attack vectors to build and maintain secure systems. This combined perspective helps bridge the gap between theoretical knowledge and practical application.

By adopting these defensive implications, organizations can move towards building more resilient and truly secure systems, rather than merely implementing a facade of security controls.

Key Takeaways

  • Integration over Isolation: The effectiveness of security controls depends entirely on their proper integration and correlation across different system layers and components, not just their individual strength.
  • Question Everything: Always delve into the specifics of how security controls are implemented and how they manage state; never take their theoretical presence or high-level design at face value.
  • Iterate Threat Models: Any significant change or "fix" to a security-critical component demands a re-evaluation of the threat model to identify unintended side-effects or new vulnerabilities.
  • Deployment is Security-Critical: The actual deployment environment, including load balancers, distributed architectures, and network zones, has profound implications for a system's security and must be part of the threat model.
  • Simplify for Security: Complexity often introduces vulnerabilities; simpler designs are inherently easier to analyze, verify, and secure.
  • Collaborate Across Disciplines: Effective security requires close collaboration between security experts, developers, and operations teams to ensure a comprehensive understanding of both design and implementation realities.

About the Speaker(s)

David Johansson is a Security Consultant at Synopsis, a prominent American company known for its work in software integrity and security. Johansson brings a wealth of experience from various roles within the IT security industry, having served as a developer, architect, security consultant, and security tester. His background includes working to develop secure software, as well as helping companies enhance their security by identifying defects through methods like source code review, penetration testing, and threat modeling. In his current role at Synopsis, he leverages this extensive hands-on experience to guide organizations in understanding and mitigating design-level security flaws, particularly through the practice of threat modeling.

Reviews

Dr. Zero (Offensive Security Researcher) — SOLID

Johansson walks through four architectural threat modeling engagements from his time at Synopsys. It's standard application security consulting fare. We're looking at session correlation failures, OTP race conditions, and trust boundary violations, but grounded in actual deployment realities rather than academic theory.

Heather Calloway (CISO) — STRONG ACCEPT

A sharp, pragmatic look at how perfectly good security controls fail at the integration layer. Johansson proves that buying best-of-breed tools means nothing if your architecture allows attackers to slip through the seams between them.

→ Top-rated talks at Swiss Cyber Storm 2019

All talks from Swiss Cyber Storm 2019