Ransomware and Backups: A Multi-Layered Defense Strategy

Ransomware and Backups: A Multi-Layered Defense Strategy

Speakers: Amol Sarwate

Conference: BSidesSF 2024

YouTube: https://www.youtube.com/watch?v=HnCGTVyRKTU

Overview

Amol Sarwate, Cyber Resilience Leadership at Veritas Technologies, presented a critical talk at BSidesSF 2024 titled "Ransomware and Backups: A Multi-Layered Defense Strategy." The presentation underscored the paradigm shift in the primary purpose of data backups, moving from traditional disaster recovery and accidental deletion scenarios to a crucial defense mechanism against the pervasive threat of ransomware. Sarwate emphasized that a successful defense against modern ransomware attacks necessitates a multi-layered defense strategy, a concept he reiterated throughout his talk.

The core of Sarwate's discussion revolved around equipping backup administrators with actionable strategies to combat ransomware across its entire lifecycle: before an attack (prevention), during its dormant phase within a network (threat hunting), and during active data encryption or exfiltration (detonation). By drawing parallels with established network and endpoint security practices, he outlined how backup teams can proactively secure data, detect stealthy threats within backup repositories, and ensure effective recovery. This talk is vital for any organization grappling with the escalating threat of ransomware, offering practical guidance to fortify their most critical last line of defense: their backups.

Background

▶ Watch: Shift in Backup Purpose: From disaster recovery to primary ransomware recovery. (1:00)

Historically, data backups served as a safeguard against unforeseen events such as natural disasters, hardware failures, data corruption, or human error like accidental file deletion. However, as Sarwate highlighted, the landscape has dramatically shifted. Today, the predominant reason for performing backups is to enable recovery from a ransomware attack. This evolution reflects the increasing sophistication and prevalence of ransomware, which has become a primary concern for organizations across all sectors.

Sarwate presented compelling statistics from the FBI's most recent crime report, indicating a 74% increase in money lost due to ransomware in 2023, alongside an 18% increase in reported ransomware complaints. This disparity suggests that ransomware gangs are increasingly targeting more lucrative victims, leading to higher financial losses per incident. The report identified healthcare and public health as the most affected sectors, followed by critical manufacturing and government entities. Locally, California's data security breach website revealed a 41% increase in breaches in the state, with a concerning 38% increase in repeated breaches on the same businesses and a 45% decrease in the time between first and second breaches. One organization was even breached 11 times in 2023, underscoring the persistent vulnerability and the attackers' efficiency.

Despite decades of awareness, ransomware continues to thrive. Sarwate attributed this success to the diverse and constantly evolving techniques and tactics employed by attackers. He referenced the MITRE ATT&CK Matrix as a comprehensive framework for classifying these tactics, which range from initial access and lateral movement to credential theft and defense evasion. While xDR vendors typically focus on detecting these tactics in live environments, Sarwate's talk uniquely focused on how backup administrators can leverage their unique position and data to contribute to the overall defense, specifically by integrating backup strategies into a multi-layered security approach that addresses prevention, dormant threat detection, and active attack response.

Key Findings

▶ Watch: Ransomware Landscape & Impact: FBI report on 74% increase in money lost to ra... (4:00)

Amol Sarwate's presentation distilled the complex challenge of ransomware defense into a structured, multi-layered strategy specifically tailored for backup administrators. His key findings and contributions can be categorized into three distinct phases of a ransomware attack, each requiring specific actions from backup teams:

1. Prevention (Pre-Attack Phase): Sarwate emphasized that proactive planning is paramount. This involves clearly defining Recovery Point Objectives (RPO) and Recovery Time Objectives (RTO), classifying data criticality, and implementing robust contingency plans based on established frameworks like NIST 800-53. A cornerstone of this phase is adopting a resilient backup strategy, such as the 3-2-1 rule (three copies of data, on two different media, with one copy offsite), and critically, rigorously testing recovery processes, including bulk recovery scenarios.

1. Dormant Phase (Threat Hunting in Backups): This phase addresses the stealthy presence of malware within a network before it detonates. Sarwate highlighted that attackers increasingly target backup infrastructure itself, viewing it as a blueprint for an organization's "crown jewels." Key findings here include:

• Monitoring Backup Software for Anomalies: Implementing User Behavior Analytics (UBAA) on backup software to detect unusual activities (e.g., unauthorized user additions, disabling backups).
• Leveraging Backups as a Honeypot: Strategically deploying intentionally misconfigured backup servers to lure and monitor attackers.
• Scanning Backup Data for Malware: Utilizing traditional malware scanners on decrypted backup data, acknowledging potential performance limitations.
• Hunting for Indicators of Compromise (IoCs): Applying techniques like SHA-256 or Context Triggered Piecewise Hashing (CTPH) to identify known malware hashes within backup data, drawing from sources like FBI and CISA advisories or open-source threat intelligence feeds.
• Employing Yara Rules: Using the Yara language to create and apply rules that detect malicious patterns, anti-debugging techniques, exploit kits, specific CVEs, or web shells within backup files, even for dormant or non-executable malware.

1. Detonation Phase (Active Attack Detection and Recovery): Even when ransomware is actively encrypting or exfiltrating data, backup administrators can play a crucial role. Sarwate's findings here focus on detecting anomalies in the data being backed up:

• Observing Data Entropy Changes: Monitoring the randomness of data. A rapid increase in entropy can indicate active encryption by ransomware, allowing for the suspension of backups to prevent overwriting clean data with corrupted versions.
• Tracking Deduplication Ratio Changes: Significant, sudden drops in deduplication efficiency can also signal widespread data modification or encryption, serving as another indicator of an active attack.
• Analyzing File Attributes and Access Patterns: Dramatic shifts in file attributes or access patterns compared to a baseline can also point to malicious activity. These anomalies aid in quickly identifying "good" versus "bad" data during recovery.

In essence, Sarwate's core finding is that backups are not merely a post-incident recovery tool but an integral component of a proactive, multi-stage defense strategy, capable of both preventing and detecting ransomware at various points in its attack chain.

Technical Deep Dive

▶ Watch: Ransomware Targeting Backups: Attackers actively seeking to corrupt or use ba... (16:00)

Sarwate's presentation provided a comprehensive technical deep dive into how backup administrators can implement a multi-layered defense against ransomware, categorizing actions into prevention, dormant threat detection, and detonation response phases.

Prevention Phase: Building a Resilient Foundation

The initial phase focuses on establishing robust backup and recovery strategies before an attack occurs.

• Defining RPO and RTO: Organizations must clearly define their Recovery Point Objective (RPO), which is the maximum acceptable amount of data loss measured in time (e.g., 1 hour of data), and their Recovery Time Objective (RTO), the maximum acceptable downtime after an incident. These metrics guide backup frequency and recovery infrastructure design. Sarwate stressed the importance of realistic RPO/RTO values tailored to the organization's specific data and operational needs.
• Data Classification: Before implementing any backup strategy, data must be classified based on its criticality, sensitivity, and regulatory requirements. This helps prioritize what data needs to be backed up most frequently and with the highest level of protection.
• Contingency Planning: Adopting established frameworks like NIST 800-53 for contingency planning is crucial. This involves identifying critical systems, comparing backup types (e.g., full, incremental, differential), and selecting appropriate backup solutions. NIST provides guidance on using technologies like WORM (Write Once, Read Many) storage for immutability, encryption for data at rest and in transit, and cloud backups for offsite storage.
• Backup Strategy (3-2-1 Rule and Variants): The 3-2-1 backup strategy is a widely recommended best practice:
• Three copies of your data: The primary data and two backups.
• Two different media types: Storing copies on different types of storage (e.g., disk and tape, or disk and cloud) reduces the risk of a single point of failure.
• One copy offsite: Traditionally, this meant a physical location. With cloud adoption, there's a debate on whether cloud storage truly counts as "offsite" if it's accessible from the internal network.

Sarwate also mentioned variants like 3-2-1-1 (with the additional '1' for a disconnected, air-gapped copy) and 3-2-1-1-0 (where the '0' signifies no errors during backup), or 4-3-2, indicating the evolving nature of these strategies.

• Testing Recovery: This was highlighted as Sarwate's "personal favorite" and the most critical step. No amount of planning or sophisticated software is useful without rigorous testing of recovery processes. This includes not just single workload recovery but also bulk recovery scenarios to simulate real-world post-ransomware situations. The goal is to ensure that when an actual recovery is needed, the process works flawlessly.

Dormant Phase: Threat Hunting in Backups

During the dormant phase, malware has infiltrated the network but has not yet detonated. Attackers are in stealth mode, seeking to gain privileges, move laterally, evade defenses, and increasingly, target backup infrastructure itself.

• Targeting Backups: Attackers understand that robust backups can negate their leverage. They actively seek to corrupt, delete, or disable backups. Backup data can also serve as a "blueprint" for attackers, revealing an organization's most critical assets (crown jewels) that are being backed up.
• Indicator of Attack (IoA) on Backup Software: Implementing User Behavior Analytics (UBAA) on backup software logs can detect anomalous activities. This involves monitoring access patterns, configuration changes (e.g., disabling backups), or user additions that deviate from established baselines. Such analysis can reveal compromised credentials or insider threats.
• Backup Software as a Honeypot: Deploying a honeypot – an intentionally vulnerable or misconfigured backup server – can lure attackers. By placing "breadcrumbs" (e.g., fake credentials, misleading network paths) in Active Directory or other systems, defenders can guide attackers to the honeypot, where their activities can be monitored and analyzed without risking production systems.
• Scanning Backup Data with Malware Scanners: If backup data is stored in an accessible format (not encrypted or heavily deduplicated), traditional malware scanners can be used. However, this process can be slow given the volume of data, and often requires decrypting and rehydrating data, which can be resource-intensive.
• Hunting for Indicators of Compromise (IoCs): More advanced threat hunting involves looking for specific Indicators of Compromise (IoCs) within backup data. This includes:
• Hashing: Using cryptographic hashes like SHA-256 or Context Triggered Piecewise Hashing (CTPH) to identify known malicious files or nearly identical variants. CTPH is particularly useful for identifying malware that has undergone minor modifications.
• IoC Sources: IoCs are readily available from sources like FBI and CISA advisories, numerous open-source threat intelligence feeds (e.g., AlienVault OTX, MISP), and commercial threat intelligence vendors.
• Yara Rules: Yara is a powerful pattern matching language used by security researchers and threat hunters to identify malware families based on textual or binary patterns in files. Sarwate highlighted its utility for scanning backup data for dormant malware. Examples of what Yara rules can detect include:
• Anti-debugging techniques: Many malware samples check if they are running in a sandbox or debugger environment and will not execute their malicious payload if detected. Yara rules can identify binaries containing such techniques, which are typically absent in legitimate software.
• Exploit kits: Reusable libraries or components commonly found in malware to exploit specific vulnerabilities.
• Specific CVEs or vulnerabilities: Patterns associated with known exploits.
• Web shells: Malicious scripts uploaded to web servers to gain remote access.

Numerous open-source Yara rule repositories exist on platforms like GitHub, providing a rich resource for threat hunting.

Detonation Phase: Active Attack Detection

Even when ransomware is actively encrypting data, backup systems can provide crucial insights to mitigate damage and aid recovery.

• Observing Data Entropy: Entropy measures the randomness of data. Low entropy indicates predictable patterns (e.g., text files, databases), while high entropy suggests randomness (e.g., encrypted data, compressed files). Ransomware encryption dramatically increases the entropy of files. By establishing a baseline entropy for backed-up data and monitoring for rapid, anomalous changes in entropy, backup systems can detect active encryption. This can trigger automated actions, such as stopping backups of the affected data to prevent overwriting clean, older backups with newly encrypted, unusable versions.
• Deduplication Ratio Changes: Deduplication stores only unique data blocks, with references to existing blocks. When ransomware encrypts files, it fundamentally changes the data at a block level, making previously identical blocks unique. This causes a significant drop in the deduplication ratio. Monitoring this ratio can serve as another strong indicator of widespread data modification by ransomware.
• File Attributes and Access Patterns: Anomalous changes in file attributes (e.g., file extensions, modification dates) or access patterns (e.g., a sudden surge in write operations to many files) can also signal an active attack. These indicators help backup administrators identify corrupted data and pinpoint the last known good backup for recovery.

Recovery Phase: The Ultimate Test

The final phase involves restoring data and systems. Sarwate emphasized that a well-executed recovery hinges entirely on the thoroughness of the rehearsals conducted during the prevention phase. If RTOs and RPOs were realistic and recovery tests (especially bulk recovery) were successful, the actual recovery process should be significantly smoother, allowing organizations to quickly return to normal operations.

Demo / Proof of Concept

▶ Watch: Dormant Phase Defense: UBA on backup software, honeypots, scanning backups wi... (18:00)

Amol Sarwate's presentation focused on theoretical frameworks, statistical data, and strategic recommendations rather than a live demonstration or proof of concept. While he referenced a "30-60-90 day checklist" for recovery, this was presented as a resource rather than an interactive demonstration of a tool or technique. The talk provided conceptual and technical guidance on how various security tools and methodologies (like Yara rules, UBAA, entropy monitoring) could be applied, but did not include a practical, real-time demonstration of these in action.

Defensive Implications

▶ Watch: Importance of Recovery Rehearsals: Emphasizing testing recovery in bulk for s... (30:00)

The insights shared by Amol Sarwate offer critical defensive implications for organizations facing the persistent threat of ransomware. The overarching message is that backups are not merely a reactive recovery mechanism but a proactive, multi-layered defense component that must be integrated into the broader security strategy.

1. Elevate Backup Administration: Backup administrators are no longer just data custodians; they are frontline defenders. Their role needs to evolve to include threat hunting, anomaly detection, and active participation in incident response planning.
2. Proactive Planning is Non-Negotiable: Organizations must invest significant effort in the prevention phase. This includes defining realistic RPO/RTOs, classifying data, implementing robust 3-2-1 (or enhanced) backup strategies, and, most importantly, regularly and thoroughly testing recovery processes, especially for bulk data. A well-tested recovery plan is the ultimate insurance policy.
3. Secure the Backup Infrastructure Itself: Attackers actively target backup systems. Defenders must apply the same security rigor to backup software and infrastructure as they do to production systems. This means implementing User Behavior Analytics (UBAA) on backup applications to detect suspicious activity, considering honeypots to trap and monitor attackers, and ensuring backup credentials are highly protected.
4. Leverage Backups for Threat Intelligence: Backup data is a rich source of information for threat hunting. Defenders should explore capabilities to scan backup data for malware and hunt for IoCs using techniques like SHA-256, CTPH, and Yara rules. This allows for the detection of dormant malware that might have bypassed live network and endpoint defenses. Integrating open-source and commercial threat intelligence feeds into backup scanning processes is crucial.
5. Implement Anomaly Detection on Backup Streams: During an active attack, rapid changes in data characteristics can be detected by backup systems. Monitoring data entropy and deduplication ratios for sudden, anomalous shifts can indicate active encryption. This allows defenders to automatically pause or stop backups of corrupted data, preserving clean recovery points and preventing the overwriting of good data.
6. Embrace Immutability and Air-Gapping: While not explicitly detailed as a new finding, the discussion around the 3-2-1-1 strategy implicitly supports the need for immutable storage (WORM) and air-gapped or disconnected copies to ensure that at least one version of the backup remains untouched by even the most sophisticated ransomware.
7. Continuous Improvement: The statistics on repeated breaches and decreasing time between attacks highlight that ransomware is an ongoing, evolving threat. Defensive strategies, including backup and recovery plans, must be continuously reviewed, updated, and adapted to new attack techniques. The 30-60-90 day checklist mentioned by Sarwate serves as a valuable framework for this continuous improvement.

By adopting these defensive implications, organizations can transform their backup strategy from a mere safety net into a formidable, multi-layered defense against ransomware, significantly improving their resilience and ability to recover.

Key Takeaways

• Backups are now primarily for ransomware recovery: The purpose of backups has fundamentally shifted from general disaster recovery to a critical defense against ransomware attacks.
• Multi-layered defense is essential: A single solution is insufficient; organizations need a comprehensive strategy addressing prevention, dormant threat detection, and active attack response across their backup infrastructure.
• Proactive planning and testing are paramount: Defining RPO/RTO, classifying data, implementing robust 3-2-1 (or enhanced) backup strategies, and rigorously testing recovery (especially bulk recovery) are non-negotiable.
• Backup systems are targets and threat intelligence sources: Attackers actively target backup software. Defenders should use UBAA, honeypots, and threat hunting techniques (IoCs, Yara rules) on backup data to detect dormant malware and compromised credentials.
• Anomaly detection aids in active attack mitigation: Monitoring data entropy, deduplication ratios, and file attribute changes in backup streams can detect active ransomware encryption, allowing for the suspension of backups to preserve clean recovery points.
• Continuous adaptation is crucial: Given the evolving nature of ransomware, backup and recovery strategies must be regularly reviewed, updated, and tested to maintain effectiveness.

About the Speaker(s)

Amol Sarwate is the Cyber Resilience Leadership at Veritas Technologies. He presented his insights on ransomware and backup strategies at BSidesSF 2024. His expertise lies in cyber resilience, focusing on how organizations can leverage backup and recovery mechanisms as a core component of their defense against modern cyber threats like ransomware.