Advancing Event Reconstruction in Network Forensics: Extending and Evaluating SMB Command Fingerprinting

Advancing Event Reconstruction in Network Forensics: Extending and Evaluating SMB Command Fingerprinting

Speakers: Jan-Niclas Hilgert, Doctoral researcher, Fraunhofer FKIE

Conference: DFDS

YouTube: https://www.youtube.com/watch?v=lWTRdZRD4uE

Overview

In the realm of digital forensics, reconstructing user actions from network traffic presents a significant challenge, especially as file systems and access methods evolve. This talk, presented by Jan-Niclas Hilgert at DFDS, introduces and thoroughly evaluates an advanced technique called SMB Command Fingerprinting (SCF). The core objective is to move beyond simply identifying file transfers within network captures to accurately reconstruct the specific user interactions and API calls that generated the network activity. This work builds upon an initial proof of concept, significantly extending the algorithm, developing new rule sets for diverse applications, and performing a large-scale evaluation involving thousands of operations.

The research is critical for enhancing network forensics and incident response. Traditional file system analysis, while foundational, often struggles with contemporary distributed and remote file access scenarios. By analyzing Server Message Block (SMB) traffic, which is ubiquitous in Windows environments and increasingly seen in cross-platform setups, investigators can gain granular insights into user intent—such as creating, modifying, deleting, or moving files and directories—even when direct host access is limited or the file system state is volatile. This talk not only demonstrates the efficacy of SCF but also highlights its potential to bridge gaps in traditional forensic methodologies, offering a powerful tool for event reconstruction.

Background

▶ Watch: Introduction: Contemporary file system forensics challenges (0:00)

Jan-Niclas Hilgert's research is rooted in the evolving landscape of file system forensics. As a doctoral researcher, his PhD focuses on contemporary file system forensic analysis, a field that critically examines modern file systems and their implications for investigations, building upon the foundational work laid out by Brian Carrier two decades ago. Hilgert points out that much has changed since 2005, with new file systems like ZFS and Btrfs becoming prevalent, and traditional models struggling to keep pace. His work seeks to identify new features and characteristics of these systems that could enrich forensic analysis.

A significant challenge in modern forensics is the prevalence of files transferred over networks, often via protocols like Server Message Block (SMB). While network captures (pcaps) contain a wealth of information, extracting meaningful user interactions beyond raw file data has historically been a labor-intensive and often incomplete process. Tools like Wireshark can show files transferred, but they typically lack the context of why or how those files were accessed or manipulated by a user. Hilgert's initial work, presented at APEC and later extended for DFDS, aimed to address this by exploring how network traffic, particularly SMB, could enhance traditional file system forensics. The initial paper introduced the concept of reconstructing user interactions, specifically focusing on a small proof-of-concept evaluation involving roughly 20 command-line operations on an SMB share. This demonstrated the potential of identifying specific commands from SMB packet sequences.

However, the limited scope of the initial proof of concept highlighted the need for a more robust and scalable solution. The problem remained: how to move from a raw packet capture to a detailed understanding of the user-performed commands and events. This talk details the advancements made to overcome these limitations, specifically by extending the core SMB Command Fingerprinting (SCF) algorithm, developing new rule sets for a broader range of applications, and conducting a significantly larger-scale evaluation to validate its effectiveness across various scenarios.

Key Findings

▶ Watch: Introducing SMB Command Fingerprinting (SCF) methodology (4:10)

The central finding of this research is the successful development and evaluation of SMB Command Fingerprinting (SCF) as a highly effective method for reconstructing specific user interactions from network traffic. The talk demonstrates that by meticulously analyzing sequences of SMB commands and their internal attributes, it is possible to infer the original API calls or user actions with remarkable accuracy.

Key findings include:

• High Event Reconstruction Recall: The SCF approach achieved an impressive 99.4% recall for reconstructing file system operations performed using the Windows command-line utility. This indicates that the vast majority of user actions can be correctly identified from SMB network traces.
• Discrimination through Command Sequences and Attributes: The research confirms that distinguishing between different user interactions relies on two critical factors: the sequence of SMB commands generated by an API call (e.g., Create, QueryInfo, SetInfo, Close for SetFileAttributes) and the internal attributes within those commands (e.g., FileBasicInfo versus FileDispositionInfo within a SetInfo command). This granular analysis is crucial for overcoming ambiguities where command sequences alone might be identical for different actions.
• Scalability and Robustness: The evaluation moved beyond a small-scale proof of concept (20 commands) to a large dataset comprising over 12,700 operations, generating 320,000 SMB packets for just the command-line utility alone. This extensive testing confirms the scalability and robustness of the SCF methodology.
• Cross-Application Analysis Insights: While highly effective, the study revealed nuances in cross-application identification. For closely related Windows applications like PowerShell and the command-line utility, certain operations (e.g., file/directory rename and move) generated identical SMB command sequences. This meant that while the event type (e.g., "rename") could be accurately identified (high event recall), attributing it to a specific application (lower app recall) was sometimes challenging.
• Operating System and Application Specificity: The SMB client running on Linux (SMBclient) showed no interference with rules designed for Windows applications. This demonstrates that SCF can effectively distinguish between different operating systems and applications, enhancing attribution capabilities in diverse network environments.
• Unexpected SMB Protocol Behavior: The research uncovered interesting SMB behavior, such as the observation that deleting a directory with subdirectories and files on a Windows SMB share results in a distinct delete command being sent for each individual sub-item, rather than a single command for the parent directory. This detailed understanding of protocol implementation is vital for accurate reconstruction.
• False Positive Considerations: The evaluation highlighted that very short rules, especially those consisting of a single SMB command (e.g., for listing directory contents), can lead to a significant number of false positives (e.g., 5,000 for PowerShell in one instance). This suggests a need for careful rule crafting and potential exclusion of such low-fidelity rules in certain analytical contexts.

In summary, the key findings underscore that SMB Command Fingerprinting is a powerful and validated technique for deep event reconstruction in network forensics, offering unprecedented visibility into user actions on remote file shares.

Technical Deep Dive

▶ Watch: Generating unique SCF rules from SMB command sequences (6:00)

The technical foundation of this work lies in the observation that specific Windows API calls translate into unique sequences of SMB commands when interacting with network shares. For instance, performing a SetFileAttributes API call on an SMB share results in a sequence of Create, QueryInfo, SetInfo, and Close SMB commands. Conversely, a GetFileAttributes call might only involve Create and Close. This variation in command sequences forms the first layer of identification.

However, command sequences alone are often insufficient. The talk highlights a crucial detail: some API calls, such as Delete File and Set File Attributes, can generate identical SMB command sequences. The key to disambiguation lies within the internal attributes of these SMB commands. For example, a SetInfo command might contain FileBasicInfo for setting attributes, whereas a delete operation would involve FileDispositionInfo within its SetInfo command. By examining both the sequence and these specific internal attributes, a much higher fidelity of event reconstruction can be achieved.

To facilitate this process, Hilgert introduced SMB Command Fingerprinting (SCF), an approach conceptually similar to network fingerprinting techniques like JA3 or JA4. SCF generates an MD5 hash for each individual SMB command. This hash incorporates parameters crucial for identification, such as the command type (e.g., Create, Write, SetInfo) and relevant static attributes. Importantly, dynamic attributes like file paths, which would vary with each operation and prevent consistent fingerprinting, are explicitly excluded from the hashing process. This results in a unique MD5 hash for a specific type of SMB command with its characteristic attributes.

These individual SMB command hashes are then combined into a sequence to form an SCF rule. An SCF rule represents the complete signature of a particular user interaction or API call. For example, a rule for "create file" would be a specific sequence of hashes corresponding to the Create, SetInfo, and Close commands with their relevant static attributes. These manually crafted rule sets are the backbone of the system, enabling the automated identification of user interactions from raw network traffic.

The DFDS paper specifically detailed several improvements to the initial SCF algorithm:

1. Increased Flexibility: The algorithm was enhanced to allow for skipping or excluding certain hashes within rules, accommodating variations in SMB command sequences that might not be critical for identification.
2. Improved Matching Logic: The underlying mechanism for matching observed SMB command sequences against the predefined SCF rules was refined for greater accuracy and efficiency.

The creation of new rule sets involved a meticulous manual process. Researchers performed specific file operations (e.g., read, write, create, move, delete files and directories) using various applications, including the Windows command-line utility (cmd.exe), PowerShell, and the Linux smbclient. For each operation, they captured the resulting SMB traffic, analyzed the generated SMB command sequences and their internal attributes, and then translated these into SCF rules. This labor-intensive process was crucial for building a comprehensive and accurate set of fingerprints for different applications and actions.

For evaluation, a scalable framework was developed to generate a robust dataset. This framework ensured a ground truth for every operation performed, critical for validating the accuracy of the reconstruction. The data generation process involved five phases:

1. Creating files and directories.
2. Exploring (listing contents).
3. Modifying (renaming, moving).
4. Deleting files and directories.
5. A Combined phase incorporating all supported commands.

For the command-line utility alone, this framework generated approximately 12,700 operations, which translated into over 320,000 raw SMB packets. Analyzing this volume of data manually would be virtually impossible, underscoring the necessity of an automated fingerprinting approach.

The evaluation measured recall, defined as the number of correctly reconstructed operations divided by the total number of performed operations. In cross-application scenarios, two types of recall were distinguished:

• App Recall: Identified the correct event and the correct application that performed it.
• Event Recall: Identified only the correct event, irrespective of the application.

This distinction was important for understanding the limitations of attribution, particularly when closely related applications (like PowerShell and cmd.exe) produced identical SMB command sequences for certain actions (e.g., move/rename operations). Despite these nuances, the ability to reconstruct the type of event remained consistently high, demonstrating the power of SCF in providing deep forensic insights.

Demo / Proof of Concept

▶ Watch: Cross-application evaluation: Distinguishing applications and limitations (12:00)

While the talk did not feature a live, interactive demo in the traditional sense, the "proof of concept" was manifested through the extensive evaluation framework and the resulting large-scale data sets generated to rigorously test the SMB Command Fingerprinting (SCF) methodology. The initial paper had presented a "very very small proof of concept" involving only 20 commands, primarily to establish the viability of the core idea. The work presented at DFDS significantly extended this proof of concept into a robust, scalable evaluation designed to validate SCF's effectiveness across a much broader and more realistic range of scenarios.

The demonstration of the methodology's capabilities was presented through the statistical analysis of the generated data sets. This included:

• Dataset Generation: The creation of a controlled environment where specific file operations (e.g., reading, writing, creating, moving, deleting files and directories) were performed using various applications such as the Windows command-line utility, PowerShell, and the Linux smbclient. This process generated a comprehensive ground truth for every single operation.
• Scale of Operations: The framework successfully generated approximately 12,700 operations for the command-line utility alone, which resulted in a massive capture of 320,000 SMB packets. This scale itself serves as a testament to the practical applicability of the framework for large-scale testing and its necessity given the volume of network traffic in real-world environments.
• Observing SMB Behavior: During the evaluation, the researchers observed and documented specific SMB protocol behaviors. A notable example highlighted in the talk was the finding that when a user deletes a directory containing subdirectories and files via the command-line utility on a Windows SMB share, the network traffic does not show a single delete command for the parent directory. Instead, the Windows operating system iteratively sends a distinct delete command for each individual subfile and subdirectory within the target directory. This detailed, practical observation, derived directly from their evaluation setup, showcases the kind of granular insights SCF can provide that would be extremely difficult to ascertain from high-level logs or manual packet analysis.
• Performance Metrics: The ultimate "proof" was in the numbers: the high recall rates (e.g., 99.4% for single-application reconstruction) and the detailed analysis of cross-application interference. These metrics served as the quantitative demonstration of SCF's effectiveness and its ability to accurately reconstruct user events from complex network captures.

Thus, the demonstration for this talk was not a live interaction but a thorough, data-driven validation of the SCF technique, showcasing its ability to provide deep forensic insights into user actions on SMB shares under realistic operational loads.

Defensive Implications

▶ Watch: Future work: Expanding SCF to new protocols and automation (14:30)

The advancements in SMB Command Fingerprinting (SCF) offer significant defensive implications for cybersecurity professionals, particularly in incident response, threat hunting, and proactive security monitoring.

1. Enhanced Incident Response and Event Reconstruction: The most direct implication is the ability to reconstruct precise user interactions from network captures. Instead of merely seeing that a file was transferred, defenders can now accurately determine how it was interacted with—whether it was created, modified, deleted, renamed, or moved. This is invaluable during a breach investigation to understand the full scope of an attacker's actions, including lateral movement, data staging, or unauthorized configuration changes on network shares. For example, identifying a sequence of "create directory," "write file," then "delete file" could indicate data exfiltration or a malware droppper, providing far richer context than just "SMB traffic detected."
2. Improved Attribution and Threat Intelligence: SCF can often attribute actions to specific applications or even operating systems. The finding that smbclient on Linux shows no interference with Windows-based rules means defenders can potentially distinguish between actions originating from a compromised Windows host versus a Linux-based attacker tool. This level of attribution helps in building a clearer picture of the attack chain and identifying the tools and techniques used by adversaries.
3. Behavioral Anomaly Detection: By establishing baselines of "normal" SMB command fingerprints for typical user activities and applications, security teams can develop rules or machine learning models to detect anomalous sequences. An unexpected series of file creation and deletion commands, or a specific sequence of SetInfo operations, could signal suspicious activity indicative of malware, ransomware, or an insider threat.
4. Forensic Value in Ephemeral Environments: In cloud environments or virtualized infrastructure where host-based logs might be volatile or difficult to access, network captures often serve as the most persistent and complete record. SCF provides a method to extract high-fidelity forensic evidence from these network traces, offering insights that might otherwise be lost.
5. Understanding Attacker Tradecraft: The detailed insights into SMB protocol behavior, such as the iterative deletion of sub-items within a directory, can inform defensive strategies. Knowing these low-level details allows defenders to better understand how attacker tools might interact with network shares and to craft more precise detection signatures.
6. Challenges and Future Considerations for Defenders:

• Encrypted SMB Traffic: The talk explicitly mentions encrypted SMB traffic as a future challenge. As more organizations implement SMB encryption, the visibility gained by SCF will diminish unless decryption capabilities are integrated or alternative analysis methods are developed.
• Automated Rule Generation: Manual rule creation is laborious. For broad adoption, automated generation of SCF rules for new applications and protocol versions is crucial. Defenders would benefit from tools that can automatically generate and update these fingerprints.
• False Positives: The observation about short, single-command rules leading to false positives (e.g., for directory listings) highlights the need for careful tuning and context-aware analysis. Defenders must be aware that not all identified sequences are inherently malicious and may require further correlation with other security telemetry.
• Integration with Existing Tools: For SCF to be widely adopted, it needs to be integrated into existing network forensic tools (e.g., Zeek, Suricata, Wireshark plugins) or specialized security information and event management (SIEM) systems to enable real-time or near real-time analysis of SMB traffic.

In essence, SCF equips defenders with a powerful new lens to examine network traffic, transforming raw packets into actionable intelligence about user intent and application-level actions on network file shares, thereby significantly bolstering their ability to detect, investigate, and respond to cyber threats.

Key Takeaways

• SMB Command Fingerprinting (SCF) is a validated and highly effective technique for reconstructing granular user interactions from network traffic on SMB shares, moving beyond simple file transfer identification.
• Accurate event reconstruction relies on both the sequence of SMB commands and their internal attributes, enabling differentiation between actions that might otherwise appear similar at a high level.
• The methodology demonstrates high recall (e.g., 99.4%) in large-scale evaluations, confirming its robustness and scalability for real-world forensic investigations involving thousands of operations and hundreds of thousands of SMB packets.
• While event types are reliably identified across applications, attributing actions to specific applications can be challenging for closely related operating system utilities (e.g., PowerShell vs. CMD) due to identical SMB command sequences for certain operations.
• SCF provides critical insights for defensive security, enhancing incident response by detailing attacker actions, enabling behavioral anomaly detection, and improving attribution in complex network environments.
• Future work is focused on addressing encrypted SMB traffic, automating rule generation, and extending the fingerprinting approach to other network protocols like NFS, further broadening its applicability and utility in digital forensics.

About the Speaker(s)

Jan-Niclas Hilgert is a dedicated Doctoral researcher at Fraunhofer FKIE in Germany, where he conducts cutting-edge research in digital forensics. Concurrently, he is a PhD student at the University of Bonn, focusing his doctoral studies on the critical area of contemporary file system forensic analysis. His PhD work aims to update and extend the foundational knowledge in file system forensics, addressing modern file systems (such as ZFS and Btrfs) and the challenges they pose for investigations, building upon the legacy of influential works like Brian Carrier's 2005 book. Hilgert is a familiar face in the digital forensics community, having been a regular attendee of DFWs conferences since EU 2017. His research interests prominently feature enhancing file system forensics through network traffic analysis and event reconstruction. Notably, at the time of this talk, he had recently submitted his PhD thesis and was preparing for his defense in June, marking a significant milestone in his academic and research career.