Arcana: A Unified Framework for Incident Response Documentation and Knowledge Management

Arcana: A Unified Framework for Incident Response Documentation and Knowledge Management

Speakers: Vishal Thakur (Senior Incident Responder, Atlassian); Jayden Vo (Senior Incident Responder, Atlassian)

Conference: FIRSTCON

YouTube: https://www.youtube.com/watch?v=IXqvoZik2e8

Overview

Incident Response (IR) teams globally grapple with a pervasive, yet often improperly addressed, challenge: documentation. All too frequently, critical IR documents are outdated, scattered across disparate systems, missing entirely, or plagued by inconsistent formatting and terminology. This disarray leads to significant operational inefficiencies, compliance risks, and analyst burnout, particularly impacting new team members who struggle to navigate an unstructured knowledge base. This talk, "Arcana: A Unified Framework for Incident Response Documentation and Knowledge Management," presented by Vishal Thakur and Jayden Vo, Senior Incident Responders at Atlassian, introduces a novel, open-source framework designed to standardize, structure, and maintain IR documentation.

Arcana proposes a systematic approach, borrowing principles from the software engineering world, specifically documentation as code. By treating IR documentation with the same rigor as source code—enforcing structure, versioning, peer review, and clear ownership—Arcana aims to transform chaotic documentation into a reliable, actionable asset. The framework categorizes IR knowledge into five distinct document types, each with a defined purpose and structure, and provides mechanisms for their consistent management. The ultimate goal is to foster a community where standardized IR documentation can be shared and collaboratively improved, elevating the collective defensive posture of the security industry.

Background

▶ Watch: Introduction and the IR documentation problem (0:00)

The speakers vividly describe a common plight within IR teams: the elusive "perfect documentation" that, in reality, rarely exists. Years of experience across various security operations centers (SOCs) and incident response teams revealed a consistent pattern: documentation is often nonexistent, severely outdated, or simply incorrect. This problem is exacerbated by the fast-paced, chaotic, and knowledge-heavy nature of incident response. When systems are "on fire," responders need immediate access to accurate, actionable guidance, yet often find themselves without it. The consequences of poor documentation are far-reaching, incurring significant costs:

• Operational Inefficiencies: Responders waste precious time searching for information, performing incorrect actions, or starting investigations at the wrong level, slowing down Mean Time To Respond (MTTR).
• Reduced Efficiency: With leaner teams and tighter budgets, every minute counts. Inefficient processes due to poor documentation directly impact a team's ability to operate effectively.
• Compliance Risks: Inconsistent or unapproved procedures can lead to non-compliance with regulatory requirements, potentially resulting in fines or reputational damage.
• Burnout: The repetitive stress of rediscovering solutions, dealing with unclear processes, and constantly reinventing the wheel contributes significantly to analyst burnout.

The root cause, as identified by the speakers, is a tendency to treat IR documentation as a "build once, hope for the best" endeavor—static, optional, and often written post-incident in a hurried, inconsistent manner. This leads to several critical issues:

• Documentation Sprawl: Documents become fragmented and scattered across different platforms (wikis, shared drives, local machines), leading to broken links and an inability to find relevant information quickly.
• Consistency Gap: A lack of industry-wide or even team-wide standards for naming conventions, structure, and vocabulary creates confusion. What one analyst calls a "playbook," another might consider a "runbook," making collaboration and knowledge transfer difficult, especially for new analysts.
• Tribal Knowledge: Over-reliance on individual experts ("ask Bob") means critical knowledge is not codified, becoming a single point of failure when that individual is unavailable.

Recognizing these systemic issues, the Atlassian team, composed of incident responders from various big tech companies, sought inspiration from the engineering world. Engineers have long addressed similar challenges through documentation as code, where documentation is managed with the same rigor as source code. This includes using defined templates, version control, and mandatory peer reviews, ensuring accuracy, maintainability, and traceability of changes. Arcana applies these proven engineering principles to the unique demands of incident response, aiming to bring structure, versioning, peer review, and clear ownership to IR documentation.

Key Findings

▶ Watch: Arcana's five key document types explained (10:40)

The core contribution of the Arcana framework is its comprehensive and unified approach to standardizing Incident Response documentation. The key findings and contributions presented in the talk can be summarized as follows:

1. Definition of Standardized Document Types: Arcana defines five distinct, interconnected types of IR documentation, each serving a specific purpose and level of detail:

• Incident Response Plan (IRP): The overarching strategy, lifecycle, and roles for the entire IR team.
• Playbooks: High-level guides for specific incident types (e.g., ransomware, BEC), outlining response tasks and linking to more granular documents.
• Runbooks: Detailed, step-by-step instructions (ideally around 10 steps) for executing a single, specific task within an incident (e.g., isolating a host, resetting credentials).
• Standard Operating Procedures (SOPs): Approved, cross-functional processes and procedures that define how teams engage with each other during an incident, requiring explicit agreement and approval from all involved parties.
• Knowledge Base (KB) Articles: Deep-dive technical articles, how-to guides, tutorials, or post-incident reviews (PIRs) that provide detailed context and institutional knowledge referenced by other document types.

1. Establishment of Core Principles for Document Quality: To ensure reliability and trustworthiness, Arcana mandates four fundamental principles for every document:

• Owned: Every document must have an accountable owner responsible for its maintenance and accuracy.
• Reviewed: Documents must undergo peer review and approval before going live, with a defined cadence for periodic re-review to ensure continued accuracy.
• Versioned: Changes are tracked with version numbers or change summaries, providing a clear history of modifications.
• Status Tracked: Each document has a clear status (e.g., draft, published, retired) so responders know its current usability and reliability.

1. Adaptable Implementation Strategies: The framework offers two primary implementation paths:

• Documentation as Code (Git-based): For engineering-first organizations, leveraging Markdown for content, Git (GitHub, Bitbucket, GitLab) for version control, and Pull Requests (PRs) with platform-specific rule sets (e.g., GitHub Rule Sets, Bitbucket Branch Restrictions) for peer review and approval.
• Page-based (Manual): For smaller teams or those not ready for a full "docs-as-code" setup, utilizing familiar tools like Confluence, Notion, or Google Docs, with a manual Document Control Table at the start of each document to track ownership, status, review dates, and version changes.

1. Emphasis on Interconnectivity and Flow: Arcana stresses that these document types are not isolated but form a cohesive system. Playbooks link to relevant Runbooks, SOPs, and KB articles. Runbooks and SOPs, in turn, can link to KB articles for deeper technical context. This interconnectedness ensures a logical flow of information, guiding responders seamlessly from high-level strategy to granular execution.

1. Audit-Ready by Design: By embedding principles of ownership, review, versioning, and status tracking, Arcana inherently makes documentation audit-ready. Auditors can easily verify processes, ownership, and change histories without the team scrambling for disparate information.

1. Open-Source Contribution Model: The framework is open-sourced, encouraging community contributions of generic playbooks, runbooks, and SOPs. This collaborative approach aims to build a shared, robust knowledge base for the entire IR community, reducing the burden on individual teams and fostering collective defense.

In essence, Arcana provides a much-needed blueprint for transforming IR documentation from a neglected chore into a powerful, reliable, and continuously improving asset that enhances efficiency, reduces MTTR, and eliminates tribal knowledge.

Technical Deep Dive

▶ Watch: Arcana's core principles and implementation methods (14:00)

Arcana's technical strength lies in its structured approach, defining both the types of documentation and the principles governing their lifecycle, all while offering flexible implementation options.

The Five Pillars of Arcana Documentation

1. Incident Response Plan (IRP): This is the foundational document, setting the strategic direction for the IR team. It outlines the overall IR strategy, the team's lifecycle, and roles. Its structure includes:

• Purpose and Scope: Defining what the IRP covers and who it applies to.
• Team Structure: An organizational chart of the IR or security team.
• Roles and Responsibilities: Detailed descriptions of who does what during incident preparation and response (e.g., Incident Commander, Legal, Comms, Forensic Analysts).
• Incident Life Cycle: The phases of incident response (e.g., detection, investigation, containment, eradication, recovery) and high-level activities within each.
• Severity Levels: Clear definitions for incident severities (e.g., SE0, SE1, SE2, SE3) and corresponding response expectations.
• Communication and Notification: Protocols for internal and external communication based on incident thresholds.
• Review and Maintenance: How often the IRP itself is updated.

1. Playbooks: These are high-level guides tailored to specific types of incidents. A playbook acts as a master orchestration document, directing responders through the appropriate sequence of actions for a given incident. Examples include playbooks for ransomware, Business Email Compromise (BEC), or credential-based incidents.

• Document Control Table (for page-based): Owner, version, status, next review date, approval, change summary.
• Purpose and Scope: What the playbook covers.
• Detection Triggers: What events initiate this playbook.
• Roles and Responsibilities: Who is involved and their duties.
• Initial Actions: First steps upon detection (e.g., validate EDR alert).
• Investigation and Analysis: Steps for deep-diving, often linking to specific Runbooks.
• Containment, Eradication, and Recovery: Steps and referenced Runbooks for these phases (e.g., "Isolate Workstation" Runbook).
• Communication and Escalation: Thresholds and procedures for internal/external comms and escalations (e.g., to CEO, CTO, CISO for SE0 incidents).
• Post-Incident Response Activities: Actions after incident closure (e.g., verify system restoration, conduct PIR).

1. Runbooks: These are granular, step-by-step guides designed for speed during live incidents. Arcana recommends limiting runbooks to approximately 10 steps to ensure clarity and conciseness. They execute a single, specific task, such as "Isolate a Host," "Investigate a Host," or "Reset Staff Credentials."

• Naming Convention: RB-{Incident Phase}-{Numbering}-{Description} (e.g., RB-Containment-01-IsolateWorkstation).
• Document Control Table.
• Prerequisites: Required access, tools, or prior steps.
• Step-by-Step Instructions: The core actions to perform.
• Post-Action Section: Steps after completing the main instructions (e.g., record containment time, verify effectiveness).

1. Standard Operating Procedures (SOPs): SOPs define and standardize team and cross-organizational processes. Crucially, all teams involved in the process must approve an SOP before it goes live, ensuring buy-in and preventing pushback during an incident. Examples include "Performing Customer External Notification" or "Paging Teams During Incidents."

• Document Control Table.
• Purpose and Scope.
• Definitions: Essential for cross-team alignment.
• Roles and Responsibilities: Who is involved and their specific duties.
• Procedure: The actual steps of the process.
• Compliance and Auditability: How the process is ensured to be followed and audited.
• Communication and Escalation: How teams communicate during the process and escalate deviations.
• References and Dependencies.
• Review and Maintenance.

1. Knowledge Base (KB) Articles: These are deep-dive resources for institutional knowledge. They can include "how-to" guides, technical write-ups, tutorials, or detailed analysis of specific malware on different operating systems. Post-incident reviews (PIRs) can also reside here.

• Document Control Table.
• Purpose and Scope.
• Definitions.
• Intro, Main Content, Conclusion, References.

Core Principles and Enforcement

Arcana's principles ensure documentation reliability:

• Owned: Accountability for maintenance.
• Reviewed: Peer approval and regular review cadence.
• Versioned: Change tracking for transparency.
• Status Tracked: Clear state (draft, published, retired) for usability.

These principles are enforced through two main implementation strategies:

1. Documentation as Code (Git-based Workflow): This is the ideal for engineering-mature organizations.

• Markdown: Documents are written in Markdown, a universal, plain-text format, making them easy to write, read, and version control.
• Git: Git (e.g., GitHub, Bitbucket, GitLab) provides robust version control. Every change is tracked, allowing for rollbacks and historical analysis.
• Pull Requests (PRs): Peer review is enforced via PRs. Changes must be proposed, reviewed by one or more colleagues, and approved before being merged into the main branch.
• Platform Rule Sets: Platforms like GitHub offer Rule Sets, Bitbucket has Branch Restrictions, and GitLab has Merge Request Approval Rules to programmatically enforce review requirements (e.g., requiring N approvals, specific reviewers).
• External Ticketing: For ownership and review cadence, an external system like Jira can link to documents, tracking current owners and next review dates.
• Benefits: All changes are transparent, reviewed, and trackable, significantly enhancing document quality and trust.

1. Page-based Tools (Manual Workflow): For smaller teams or those without a robust Git workflow, Arcana can still be implemented using tools like Confluence, Notion, or Google Docs.

• Document Control Table: A manual table at the beginning of each document records the name, version, owner, status, next review date, approval, and change summary.
• Trade-off: This method requires manual oversight to ensure the table is updated and documents are reviewed on schedule, lacking the automated enforcement of Git.

Interconnectivity and Flow

A key architectural aspect of Arcana is the deliberate linking between document types. Playbooks are designed to reference specific Runbooks, SOPs, and KB articles at appropriate stages. For example, a "Malware Playbook" might instruct a responder to "Go to RB-Containment-01-IsolateWorkstation" or "Consult KB-Malware-GhidraAnalysis" for a deep dive. This hierarchical linking ensures that responders are always guided to the right level of detail without getting lost in excessive information within a single document.

Demo / Proof of Concept

▶ Watch: Arcana is open source: GitHub repo and contributions (30:00)

The practical application of Arcana was illustrated through a hypothetical incident workflow and the announcement of an open-source GitHub repository.

Incident Workflow Example

The speakers walked through a scenario triggered by an EDR alert for malware execution to demonstrate how Arcana's interconnected documents guide a responder:

1. Start with the Malware Playbook: This high-level document would be the first point of reference.
2. Triage Runbook: The playbook would direct to a Triage Runbook (e.g., RB-Triage-01-ValidateEDRAlert) to confirm the alert's validity. If a true positive, proceed.
3. Log Acquisition Runbook: Next, an Acquisition Runbook (e.g., RB-Investigation-02-HostLogAcquisition) would detail steps for collecting relevant logs from the compromised host.
4. Malware Analysis Runbook & KB Article: A Malware Analysis Runbook (e.g., RB-Investigation-03-MalwareAnalysis) would outline the high-level steps for analysis. This runbook, in turn, might link to a Knowledge Base Article (e.g., KB-Malware-GhidraAnalysis) for detailed instructions on using specific tools like Ghidra for reverse engineering.
5. Further Investigation Runbooks: The playbook could then lead to other runbooks like a Host Investigation Runbook or a Malware Persistence Hunt Runbook to identify TTPs across the fleet.
6. SOPs for Cross-Functional Engagement: If the malware execution led to sensitive data loss, an SOP (e.g., SOP-CustomerNotification-LegalEngagement) would define the process for engaging the legal team and performing external customer notifications. Another SOP might detail procedures for paging external teams during critical incidents.
7. Containment Runbooks: For containment, two distinct runbooks might be invoked: RB-Containment-01-IsolateWorkstation (potentially linking to a KB if the isolation technique is complex) and RB-Containment-02-RotateStaffCredentials if credentials were compromised.
8. Recovery Runbook: Finally, a Recovery Runbook (e.g., RB-Recovery-01-RebuildHost) would guide the host restoration process.

This example clearly demonstrates Arcana's ability to provide a structured, navigable path through complex incident response, ensuring all necessary steps are documented and accessible.

Open-Source GitHub Repository

The most significant proof of concept is the immediate release of the Arcana GitHub repository (accessible via QR code presented at the end of the talk). This repository serves as a starter kit and a community hub.

• Current Contents: At launch, the repo includes 19 Playbooks, approximately 90 Runbooks, and several SOPs.
• Purpose: These templates are designed to be generalized and easily adaptable. While users will need to tweak them for their specific organizational context and tools, they provide a strong foundation.
• Call for Contributions: The speakers strongly emphasized that Arcana is a community project. They encourage defenders to fork the repo, strip away business-specific details from their existing documentation, and contribute generic playbooks, runbooks, and SOPs.
• Contribution Process: The process is straightforward: fork the repo, add content following the provided templates, and submit a pull request (PR) for review and merge. Contributors are encouraged to add their names for recognition.
• Roadmap: Future plans for the project include:
• Community Onboarding: Fostering widespread adoption and contribution.
• Doc Tool Chain Integration: Automating the conversion of Markdown files from Git into publishable formats for platforms like Confluence via CI/CD pipelines (e.g., GitHub Actions, Bitbucket Pipelines).
• Standardized Templates: Continuously refining and expanding the set of available templates.
• Visual Navigator: Developing a tool to visually map incident workflows and dependencies, enhancing navigability.

The open-source nature and the immediate availability of a comprehensive starter kit underscore Arcana's commitment to empowering the IR community with practical, shared resources.

Defensive Implications

▶ Watch: Q&A: Enabling programmatic/agent-based incident response (34:50)

The Arcana framework offers profound defensive implications for incident response teams, moving them from reactive chaos to proactive, structured, and continuously improving operations.

1. Reduce Mean Time To Respond (MTTR): By standardizing documentation and making it easily discoverable, analysts spend less time hunting for the right document and more time on the actual incident. Clear, step-by-step runbooks and playbooks enable faster, more accurate decision-making and execution, directly impacting MTTR.

1. Eliminate Tribal Knowledge: Arcana explicitly addresses the "ask Bob" problem. By mandating ownership, review, and documentation of all critical processes, institutional knowledge is codified and retained. This ensures business continuity, reduces reliance on individual experts, and allows any analyst, regardless of experience level, to follow established procedures.

1. Improve Consistency and Quality: The framework enforces consistency in document structure, naming conventions, and vocabulary across the team and potentially the industry. This reduces confusion, streamlines onboarding for new analysts, and ensures that every responder follows the same, approved process for a given incident type or task. The peer review mechanism further guarantees the accuracy and quality of the documentation.

1. Enhance Audit Readiness: Compliance is baked into Arcana's core principles. Every document has clear ownership, approval dates, review schedules, and version history. When auditors come knocking, teams can readily provide comprehensive, verifiable documentation without scrambling, demonstrating due diligence and adherence to established procedures.

1. Scale Incident Response Capabilities: With standardized and easily maintainable documentation, IR teams can scale their operations more effectively. New team members can quickly become productive, and processes can be consistently applied across a growing organization. This reduces "documentation debt," allowing teams to focus on evolving threats rather than struggling with internal knowledge management.

1. Facilitate Automation and Programmability: The structured and consistent nature of Arcana's runbooks, especially when implemented as "documentation as code," lays a foundation for future automation. The speakers hinted at this potential, noting that historically, runbooks contained literal commands. Moving forward, human-readable instructions in standardized runbooks can be interpreted by agents or AI-driven systems, enabling programmatic response activities. This allows for increased efficiency and reduced human error in routine tasks.

1. Foster Community Collaboration: The open-source nature of Arcana is a call to action for the entire IR community. By contributing generic playbooks, runbooks, and SOPs, organizations can collectively build a robust, shared knowledge base. This collaboration elevates the defensive posture of all participating teams, allowing them to learn from each other's experiences and best practices, and reducing the individual burden of creating every piece of documentation from scratch. Defenders can help each other, improving overall industry resilience.

Adopting Arcana principles means investing in a more mature, resilient, and efficient incident response function. It shifts the paradigm from documentation as an afterthought to documentation as a strategic, continuously improving asset.

Key Takeaways

• Standardization is Paramount: The lack of standardized IR documentation leads to significant operational inefficiencies, compliance risks, and analyst burnout. Arcana provides a unified framework to address this by defining clear document types and consistent structures.
• Embrace Documentation as Code: Borrowing from software engineering, treating IR documentation with the rigor of source code—using Markdown, Git for version control, and Pull Requests for peer review—ensures accuracy, maintainability, and transparency.
• Arcana's Five Document Types: The framework categorizes IR knowledge into IRP, Playbooks, Runbooks, SOPs, and Knowledge Base Articles, creating an interconnected system that guides responders from high-level strategy to granular execution.
• Enforce Core Principles: Every document must be Owned, regularly Reviewed, clearly Versioned, and have its Status Tracked to ensure trustworthiness and reliability during critical incidents.
• Community Contribution is Key: Arcana is an open-source project encouraging collective intelligence. Contributing generic documentation helps build a shared knowledge base, reducing the burden on individual teams and strengthening the entire IR community.
• Tangible Benefits for Defenders: Adopting Arcana reduces Mean Time To Respond (MTTR), eliminates tribal knowledge, improves consistency, ensures audit readiness, and lays the groundwork for future automation in incident response.

About the Speaker(s)

Jayden Vo is a Senior Incident Responder at Atlassian, where he has been contributing to the company's security efforts for the past five years. His experience in the field highlighted the critical need for improved documentation, which ultimately led to the development of the Arcana framework.

Vishal Thakur, also a Senior Incident Responder at Atlassian, brings a wealth of experience from various big tech companies, including Salesforce and Amazon AWS. Based in Sydney, Vishal, affectionately known as Vish, shares Jayden's passion for standardizing incident response processes. He is also a co-founder of the newly established Insider Threat SIG at FIRST, demonstrating his commitment to collaborative security initiatives within the industry. Together, they leveraged their collective experience to create Arcana, aiming to solve a common problem for IR teams globally.