Ransomware protection with immutable AWS Backup - it's complicated ...

Ransomware Protection with Immutable AWS Backup — It's Complicated...

Speakers: Paul Schwarzenberger, Head of Assurance, Searchlight Cyber; Kurtis Mash, Cloud Security Architect, The National Archives

Conference: fwd:cloudsec Europe 2025

YouTube: https://www.youtube.com/watch?v=9OY6LtamqRQ

Overview

Paul Schwarzenberger and Kurtis Mash presented the National Archives UK's journey to implement a centralized, immutable AWS Backup solution to protect their digital archives — which include records ranging from the Doomsday Book to the Magna Carta — against ransomware attacks. The talk candidly exposed the significant complexity hidden beneath the seemingly simple task of "just turning on AWS Backup," covering KMS key type trade-offs, vault lock modes, cross-account backup architecture, service-specific feature gaps, and cost considerations. The National Archives has open-sourced their entire Terraform module for centralized immutable AWS Backup, providing the community with a production-tested implementation that handles the many sharp edges they discovered.

Background

▶ Watch: National Archives' four requirements: immutability, centralization, data prot... (3:30)

Ransomware has become one of the most prevalent and damaging cyber threats facing organizations globally. The British Library suffered a ransomware attack that disrupted operations for months. Marks & Spencer experienced a similar attack affecting online sales and in-store systems. According to ransomware.live, there were 288 distinct ransomware attacker groups and 5,360 victims recorded as of the day before the talk — likely an undercount since many organizations do not report incidents.

The National Archives UK serves as the official archivist and publisher for the UK government, hosting both physical records (the Doomsday Book, the Magna Carta, the Coronation Roll of King Charles) and digital records transferred from government departments, parliamentary records, public inquiries, and other public bodies. Safeguarding digital assets against ransomware is a critical mission requirement.

The organization defined four key requirements for their backup solution: immutability (backups cannot be tampered with or deleted — not by malicious actors, not by administrators, not even by organization-level privileged users), centralization (avoiding the risk of backup loss if a workload account is compromised, closed, or suspended, and providing a consistent recovery approach across teams), data protection (centralized backups create a honeypot that must be secured as rigorously as source systems), and managed service (preferring to spend engineering time on archiving rather than operating backup infrastructure).

Key Findings

▶ Watch: Vault lock compliance mode: even AWS Support cannot override until expiry (8:00)

AWS Backup is not a simple "turn it on" solution. While AWS Backup provides a single API across many services for managing backups, the implementation reveals significant complexity in three areas: key management, cross-account architecture, and service-specific feature support.

KMS key types create a fundamental trade-off for cross-account backup immutability. AWS offers three types of KMS keys: AWS-managed keys (exist in your account, can't modify policy or delete, but cannot be used for cross-account operations), customer-managed keys (fully customer-controlled, support cross-account operations, but can be deleted with a 7-day waiting period), and AWS-owned keys (newer type, can't be deleted, support cross-account operations, but exist in an AWS-owned account). For cross-account backup copying, customer-managed keys are required — but these keys can be deleted, rendering the encrypted backups permanently inaccessible even if the backup data itself is immutable. This creates a gap: compliance-mode vault lock prevents backup deletion, but it cannot prevent KMS key deletion.

Logically air-gapped vaults partially solve the key problem. AWS recently released logically air-gapped vaults that enforce compliance mode and use AWS-owned keys, eliminating the KMS key deletion risk. However, not all resource types support this vault type, requiring a multi-vault architecture for complete coverage.

AWS Organizations backup policies deploy plans but not prerequisites. Backup policies (the Organizations-level equivalent of backup plans) can deploy backup configurations across member accounts, but they expect IAM roles and backup vaults to already exist in each account. There is no native mechanism within AWS Backup to deploy these prerequisites, requiring separate infrastructure deployment via CloudFormation stack sets or similar tools.

Service-specific limitations break the centralized model. Not all AWS services support cross-account backup copying. RDS multi-AZ database clusters can be snapshotted but snapshots cannot be copied across accounts, forcing organizations to accept the backup remaining in the workload account or engineer around the limitation.

Technical Deep Dive

▶ Watch: RDS multi-AZ clusters can't copy snapshots cross-account — breaking centraliz... (10:00)

The National Archives' architecture uses a dedicated backup delegate account within their AWS Organization to hold centralized backups and manage deployment. The solution deploys via Terraform to the central account and uses CloudFormation stack sets to push resources (IAM roles, backup vaults, and supporting infrastructure) into workload accounts. AWS Organizations backup policies then deploy backup plans from the management account.

The backup flow for each resource type involves multiple stages. First, a backup is taken to a local vault within the workload account. Then the backup is copied cross-account to the central backup account. For resource types that support logically air-gapped vaults, the copy goes directly to an air-gapped vault with compliance-mode lock and AWS-owned key encryption. For resource types that do not support air-gapped vaults, the backup first copies to an intermediate vault, re-encrypting with a customer-managed key (required for cross-account transfer), and then an EventBridge listener triggers a second copy into a standard immutable vault encrypted with an AWS-managed key — eliminating the customer-managed key deletion risk.

Vault lock operates in two modes: governance mode (backups cannot be deleted unless a user has a specific permission to remove the lock) and compliance mode (backups cannot be deleted until the lock expires — not by customers, not by AWS, not even via AWS Support). The National Archives uses compliance mode for their central vaults, providing the strongest immutability guarantee. Even an attacker with organization administrator privileges cannot modify a compliance-mode vault lock.

Backup plans are configured for daily, weekly, and monthly schedules, with tag-based selection to back up specific resources rather than everything. The solution includes intelligent assignment of backup types based on the AWS service being backed up.

For restoration, the central account shares backup vaults back to workload accounts via AWS Resource Access Manager (RAM), allowing teams to restore from centralized backups directly into their accounts using a pre-deployed IAM role.

Demo / Proof of Concept

▶ Watch: Live demo: walking through production backup vaults and compliance-locked con... (16:00)

Schwarzenberger demonstrated the live AWS environment, showing a test organization with approximately 10 accounts including a dedicated centralized backup account. He walked through the workload account configuration (showcasing AWS's new account color feature for production identification), the deployed backup vaults and plans pushed via Organizations backup policies, the central account's compliance-mode locked vaults, and initiated a live S3 bucket restore operation through the RAM-shared vault. The demonstration showed the end-to-end flow from centralized backup storage to workload account restoration.

The solution is published as an open-source Terraform module at the National Archives GitHub repository (nationalarchives/terraform-aws-immutable-backup) with accompanying documentation on GitHub Pages covering deployment options, configuration, prerequisites, and architecture diagrams.

Defensive Implications

▶ Watch: Live restore initiated from RAM-shared vault back to workload account (18:00)

Organizations implementing AWS Backup for ransomware protection should:

• Use a dedicated central backup account separate from workload accounts to prevent backup loss if a workload account is compromised, closed, or suspended.
• Understand the KMS key deletion risk when using compliance-mode vault lock with customer-managed keys. A compliance-locked backup is useless if the encryption key is deleted. Use logically air-gapped vaults (with AWS-owned keys) where supported, and re-encrypt to AWS-managed keys for other resource types.
• Deploy backup prerequisites via stack sets since AWS Organizations backup policies do not provision IAM roles and vaults in member accounts.
• Enable compliance-mode vault lock on centralized backup vaults for maximum immutability — even organization administrators and AWS Support cannot override it.
• Review backup costs carefully and align plans to recovery objectives. Be aware that S3 backup incurs per-object API costs (a million-object bucket means a million GET requests) and that Glacier-stored objects require retrieval charges.
• Test restore procedures regularly — automated restore testing on a quarterly cadence ensures that backups are functional and that application teams understand the restore workflow.
• Evaluate service-specific limitations (e.g., RDS multi-AZ clusters cannot copy snapshots cross-account) and implement compensating controls where centralized backup is not feasible.

Key Takeaways

• AWS Backup immutability is far more complex than "just turn it on" — KMS key types, vault lock modes, and cross-account architecture all introduce sharp edges.
• Compliance-mode vault lock is the strongest immutability guarantee in AWS — even org admins and AWS Support cannot override it until expiry.
• Customer-managed KMS keys can be deleted in 7 days, potentially rendering compliance-locked backups permanently inaccessible. Use AWS-owned or AWS-managed keys where possible.
• Logically air-gapped vaults solve the key management problem but are not yet supported for all resource types.
• The National Archives has open-sourced their complete centralized immutable backup solution as a Terraform module, available for community use and contribution.
• Automated restore testing and backup auditing are critical next steps that many organizations neglect.

About the Speaker(s)

Paul Schwarzenberger is the Head of Assurance at Searchlight Cyber, with extensive experience as a cloud security architect across multiple industries. He is the creator of the OWASP Domain Project and a frequent speaker at cloud security conferences, including the first fwd:cloudsec event. Kurtis Mash is a Lead Technical Architect at the National Archives UK, where he leads work on cyber security and cloud architecture in AWS. Together they developed and open-sourced the centralized immutable backup solution presented in this talk.