Panel: Peak Performance Under Pressure: Building Cross-Functional Resilience in Incident Response

Peak Performance Under Pressure: Building Cross-Functional Resilience in Incident Response

Speakers: Merisa Lee (Former, Okta); Brooke Pearson (Alphasense); Melanie Ensign (Detection Engineering / Incident Command, Discernible)

Conference: FIRSTCON

YouTube: https://www.youtube.com/watch?v=9BC7BoX-Rzk

Overview

This panel discussion, "Peak Performance Under Pressure: Building Cross-Functional Resilience in Incident Response," delivered at FIRSTCON, brought together seasoned incident response (IR) professionals Merisa Lee, Brooke Pearson, and Melanie Ensign. Drawing extensively from their shared foundational experiences at Uber over a decade ago, the speakers explored the critical importance of integrating incident response protocols seamlessly into daily operational workflows. Their core premise is that the closer an organization's IR processes mirror its day-to-day activities, the more effectively it practices for real-world incidents, thereby fostering a culture of continuous readiness rather than reactive panic.

The talk delves into the multifaceted challenges of managing security incidents, emphasizing that true resilience extends far beyond technical prowess. It highlights the indispensable role of robust cross-functional coordination, clear communication, and proactive relationship building with both internal and external stakeholders. The panelists, representing diverse functions within incident response—from detection engineering and incident command to technical program management and communications—offered unique perspectives on how to navigate the complex landscape of security breaches while maintaining organizational composure and trust.

The discussion is particularly relevant in today's dynamic threat environment, where the speed and sophistication of attacks demand not only rapid technical remediation but also agile organizational responses. By focusing on human factors, cultural integration, and strategic preparedness, the speakers provided a compelling blueprint for how organizations can move beyond rigid playbooks to cultivate a truly resilient and adaptable incident response capability, ensuring that teams perform optimally even under the most intense pressure.

Background

▶ Watch: Introduction: Incident response mirrors daily operations (0:00)

The panelists' shared history at Uber served as a foundational backdrop for their discussion, shaping much of their philosophy regarding cross-functional incident response. At Uber, they collaboratively defined and refined many of the practices they advocate, recognizing early on that an effective IR strategy requires more than just technical expertise; it demands a holistic, integrated approach across various organizational functions. This experience underscored the critical insight that incident response should not be an isolated, emergency-only function but rather an extension of daily operations.

A key structural innovation at Uber, highlighted by Merisa Lee, was the deliberate separation of Incident Command (IC) from Detection Engineering. While these roles could overlap, their distinct specialties allowed engineers to focus on their strengths. Merisa noted that "engineers aren't one-size-fits-all," observing that excellent coders might struggle with incident coordination, and vice-versa. A dedicated IC team, she argued, prevents engineers from struggling during incidents, as running an incident requires a different mindset and skillset than writing code 90% of the time. This specialized approach ensured that individuals with natural aptitudes for coordination, communication, and strategic thinking were at the helm of incident management, allowing detection engineers to concentrate on identifying threats and developing countermeasures.

The panel also addressed the pervasive issue of incident response playbooks that exist in shared folders but are rarely consulted or followed during actual crises. Brooke Pearson articulated the common pitfall where organizations inadvertently signal panic to external stakeholders through rushed, ill-considered communications. This often stems from an internal lack of coordination and a physiological response to stress, leading individuals to act outside their trained roles or established approval processes. The speakers stressed that such impulsive actions, like hastily posting statements on social media, can undermine trust and signal a disconnect between leadership and the security team. This problem exists because many organizations fail to build the necessary "muscle memory" and cross-functional relationships before an incident occurs, leaving teams unprepared for the cognitive load and emotional pressure of a real breach. Melanie Ensign further reinforced this by drawing an analogy to scuba diving's task loading concept: every new skill or unfamiliar situation adds cognitive burden, making it harder for inexperienced individuals to perform under stress.

Key Findings

▶ Watch: How organizations inadvertently signal panic externally (5:00)

The panel presented several key findings crucial for building effective cross-functional incident response:

• Specialized Incident Response Roles: Merisa Lee emphasized the benefits of separating Incident Command (IC) from Detection Engineering. This allows individuals to specialize, with some excelling at incident coordination and communication, and others at technical analysis and code development. This specialization prevents "struggle" during incidents, as the IC role demands distinct communication and coordination skills often not found in pure coding roles.
• Communication as a Core IR Skill: All panelists underscored that strong communication, both internal and external, is paramount. Merisa highlighted that engineers who write amazing code might struggle with presenting to a CISO or the board, and these communication skills are vital for bringing calm to a crisis. Brooke warned against "half-cocked responses" to external stakeholders, advocating for trained communicators to handle public statements to avoid signaling panic.
• The Power of "Muscle Memory" and Chaos Engineering: Melanie Ensign introduced the concept of "task loading" from scuba diving to explain why daily practice is essential. Merisa described incorporating "chaos" into tabletop exercises, such as simulating key personnel being unavailable (e.g., "you're gone... somebody else needs to step up"). Regular, unexpected scenarios, even quick walk-throughs in team meetings, keep minds engaged and build the necessary familiarity for real incidents. Writing post-mortems routinely also serves as practice.
• Judicious Use of AI and Automation: While acknowledging AI's utility for crunching vast amounts of data and identifying subtle signals (e.g., "needles in a haystack"), the panel cautioned against over-reliance. Brooke and Merisa highlighted AI's value in note-taking during incidents, freeing junior team members to participate and learn rather than just transcribe. However, they warned against using AI for critical tasks like writing full post-mortems or making decisions, as it can mask gaps in human judgment and potentially introduce vulnerabilities if used for code generation without proper oversight.
• Graduated Responses and Clear Role Definition: Brooke Pearson advocated for graduated responses and clear role definition using frameworks like RACI (Responsible, Accountable, Consulted, Informed). She stressed the importance of letting experts drive the response within their domain (e.g., lawyers for legal risk, communicators for external messaging), stopping "pontificating" from those outside their expertise. This clarity prevents anxiety-driven overreach and ensures efficient decision-making.
• Proactive Relationship Building: A central theme was the necessity of building cross-functional relationships before an incident. Melanie's adage, "in order to call a friend, you have to have one," encapsulates this. Merisa included "relationship matrixes" in performance reviews for incident commanders, requiring them to build rapport with comms, legal, and other key stakeholders. These pre-established trusts ensure smoother collaboration during high-stress events, as teams are already "on the same page about what we care about."
• Quantifying Security Value and Education: Brooke highlighted that security teams are often perceived as "cost centers" or "decelerants." To counter this, security leaders must quantify the value they bring, demonstrating how they save the company money through incident prevention and response. Merisa suggested TLP Red briefings (with legal sign-off) to educate internal stakeholders across the company on the impact and cost of incidents, fostering a better understanding of security's role.
• Blameless Post-Mortems and Accountability: For effective learning, post-mortems must be blameless, focusing on systemic improvements rather than finger-pointing. Brooke suggested a technique where participants write down what went well, "meh," and bad, then upvote to prioritize discussion. Accountability for findings is crucial; Merisa and Brooke suggested integrating action items into performance reviews and roadmaps, tracked via tools like Jira, to ensure they are addressed and not left to "die forever."
• Adaptability and Ownership: The Uber principle of "act like an owner" proved universally applicable. Merisa noted that this "radical ownership of outcomes" allows teams to fill gaps when resources are scarce. While advocating for flexibility over rigid adherence to runbooks, she cautioned against allowing VPs or CISOs to take over incident command if they are disconnected from the operational realities.

Technical Deep Dive

▶ Watch: Pros and cons of AI/automation in incident response (12:30)

While the talk was not a traditional "technical deep dive" into code, exploits, or network protocols, it extensively covered the technical and operational architecture necessary for effective incident response. The panelists emphasized that technical tools and processes must support human capabilities, not replace them, and that foundational technical hygiene is paramount.

A core tenet discussed was the strategic separation of Incident Command (IC) from Detection Engineering. This architectural decision, pioneered at Uber, acknowledges that the skill sets for these roles are distinct. Detection engineers are primarily focused on writing code for security tools, developing detections, and understanding Indicators of Compromise (IoCs) and Tactics, Techniques, and Procedures (TTPs). In contrast, Incident Commanders manage the overall response, coordinating diverse teams, communicating status, and making high-level strategic decisions. This "split" allows engineers to optimize for their strengths, preventing the cognitive overhead that arises when a coding-focused engineer is suddenly thrust into a high-pressure leadership and communication role during an incident. The implication is that security teams should be structured to allow for this specialization, rather than expecting every engineer to be a "full-stack" IR professional.

From a tooling perspective, Merisa Lee highlighted a critical, yet often overlooked, technical foundation: logging standards. She stated, "ensure that you have a logging standard and that all your logs at least fit that standard, whatever tool that you use." This is because the "biggest issue" during an incident is trying to parse disparate, non-standardized logs, which significantly hampers timely analysis. This points to the need for robust log aggregation, normalization, and standardization frameworks (e.g., using a common schema like Elastic Common Schema or OpenTelemetry) across an organization's infrastructure. Without this, even the most sophisticated Security Information and Event Management (SIEM) or Extended Detection and Response (XDR) solutions will struggle to provide actionable intelligence.

The panel also touched upon the practical application of tools for cross-functional collaboration. While acknowledging the lack of a "perfect incident response tool," the speakers emphasized using existing, familiar platforms. Brooke Pearson noted that her communications team is "forced" to learn and use the technical teams' tools, such as Jira, for tracking issues. This not only builds relationships but also ensures that during an incident, "people aren't struggling with a tool they're unfamiliar with." This highlights the importance of tool interoperability and familiarity across all stakeholders, rather than introducing bespoke, complex IR platforms that require steep learning curves under pressure. For real-time communication, Slack was mentioned as a common platform for running incident drills, leveraging its ubiquity within many organizations.

Finally, the discussion implicitly underscored the technical infrastructure required for effective post-incident analysis and continuous improvement. The ability to "write up reports" and "keep our post-mortem findings and actually look at them routinely, quarterly at least" necessitates systems for incident ticketing, vulnerability management, and action item tracking. Tools like Jira, when integrated into performance reviews and roadmaps, provide the technical backbone for ensuring that lessons learned translate into tangible security improvements, thus closing the loop on the incident response lifecycle.

Demo / Proof of Concept

▶ Watch: Clear roles and expert-driven decisions in incident response (17:30)

This panel discussion did not include a live technical demonstration or proof of concept. The content focused instead on strategic, organizational, and human-centric aspects of incident response.

Defensive Implications

▶ Watch: Practical tips for blameless post-mortems and action tracking (39:00)

The insights shared by the panelists offer several critical defensive implications for security teams aiming to build robust and resilient incident response capabilities:

1. Prioritize Cross-Functional Relationship Building: Proactively establish strong relationships with legal, communications, marketing, product, and other non-technical teams before an incident occurs. This means regular engagement, understanding their perspectives, and building mutual trust. As Melanie Ensign put it, "in order to call a friend, you have to have one."
2. Integrate IR into Daily Operations: Design incident response protocols to mirror daily operational workflows as much as possible. This fosters "muscle memory" and reduces the cognitive load during actual incidents. Regular, low-stakes practice, like weekly Slack drills or mini-scenarios in team meetings, is crucial.
3. Invest in Communication Skills for IR Roles: Recognize that strong communication is as vital as technical expertise for incident commanders and other IR leads. Develop and train individuals in presenting, coordinating, and managing internal and external messaging effectively. Consider communication skills as a performance metric for IR team members.
4. Implement "Chaos Engineering" in Tabletop Exercises: Move beyond predictable tabletop scenarios. Introduce unexpected variables, such as key personnel unavailability, to simulate real-world chaos and test team adaptability. This helps identify single points of failure and forces broader team members to step up.
5. Leverage AI Judiciously: Utilize AI for its strengths, such as data crunching, signal correlation, and automated note-taking during incidents, which frees human analysts for higher-level tasks. However, avoid over-reliance on AI for critical decision-making or generating sensitive reports, always maintaining human oversight and judgment to catch errors or "wrong" content.
6. Standardize Logging Practices: Establish and enforce comprehensive logging standards across all systems and applications. Ensure that logs are consistently formatted, aggregated, and easily parsable. This foundational technical hygiene is paramount for efficient threat detection and incident investigation.
7. Adopt Graduated Response Frameworks: Implement clear, tiered incident response plans that define escalation triggers and corresponding roles and responsibilities (e.g., using RACI matrices). Empower domain experts (e.g., legal counsel, PR specialists) to lead within their areas of expertise during incidents, preventing non-experts from "pontificating" and causing delays.
8. Conduct Blameless Post-Mortems with Accountability: Foster a culture of learning by conducting blameless post-mortems that focus on systemic improvements rather than individual blame. Crucially, integrate action items from post-mortems into team roadmaps and individual performance reviews, using tools like Jira to track progress and ensure findings are addressed.
9. Quantify and Communicate Security Value: Proactively articulate the financial and reputational value that security brings to the organization. Share insights into the cost of incidents and the savings achieved through security measures (e.g., via TLP Red briefings) to counter the perception of security as merely a "cost center" or "decelerant."
10. Use Existing Collaboration Tools Effectively: Leverage familiar tools like Slack for real-time communication and Jira for issue tracking across technical and non-technical teams. Ensure all stakeholders are proficient in these tools to minimize friction during high-stress situations.

Key Takeaways

• Incident Response is a Daily Practice: Integrate IR protocols into routine operations to build "muscle memory" and reduce cognitive load during actual incidents, making preparedness a continuous state, not a reactive sprint.
• Cross-Functional Collaboration is Paramount: Proactively build trusted relationships with legal, communications, and other business units before an incident, ensuring seamless coordination and shared understanding during crises.
• Communication Skills are Critical: Effective incident response demands strong communication, both internal and external. Prioritize training and staffing for roles that can clearly articulate complex technical issues to diverse audiences, from engineers to board members.
• AI Augments, Not Replaces, Human Judgment: Utilize AI for data crunching and administrative tasks like note-taking, but maintain human oversight for critical decision-making, post-mortem analysis, and ethical considerations to prevent over-reliance and mask human judgment gaps.
• Blameless Post-Mortems Drive Continuous Improvement: Conduct post-mortems that focus on systemic learning rather than individual blame. Crucially, embed action items into roadmaps and performance reviews to ensure accountability and prevent recurring issues.
• Act Like an Owner and Be Flexible: Cultivate a culture of "radical ownership" where team members proactively address issues regardless of their title. While processes are important, adaptability and flexibility are essential when facing unforeseen challenges in a crisis.

About the Speaker(s)

Merisa Lee is a seasoned professional in the cybersecurity domain, with a background spanning incident command and detection engineering. Her experience includes a significant tenure at Uber, where she helped shape foundational philosophies for cross-functional incident response. She has since applied and adapted these principles at subsequent organizations, including Okta, focusing on effective incident management and team structuring, particularly advocating for dedicated incident command teams.

Brooke Pearson brings expertise in Technical Program Management (TPM) and communications to the incident response discussion. Her career includes time at Uber, where she developed her understanding of cross-functional coordination. Brooke has also held roles in policy and communications, currently working with Alphasense, and emphasizes the critical role of clear, strategic communication with external stakeholders and the importance of defining roles within incident response to prevent panic and ensure expert-driven decisions.

Melanie Ensign is a distinguished figure in detection engineering and incident command, currently contributing her expertise to Discernible. Like her co-panelists, Melanie's foundational experience at Uber profoundly influenced her approach to incident response. She is known for her analytical insights into human behavior under pressure, drawing analogies from her experience as a rescue scuba diver to explain concepts like "task loading" in cybersecurity incidents. Melanie champions proactive relationship building and the strategic integration of incident response into daily operations.