Unmasking Supply Chain Attacks via Application Behaviour

Unmasking Supply Chain Attacks via Application Behaviour

Speakers: Dagmawi Mulugeta and Colin Estep — Netskope Threat Labs

Conference: Black Hat USA 2025 — August 6-7, 2025, Mandalay Bay, Las Vegas

YouTube: https://www.youtube.com/watch?v=UGB5W-yJCrQ

Reading time: ~9 minutes

Type: Briefing

TL;DR

Researchers at Netskope Threat Labs spent two years building BEAM (Behavioral Evaluation of Application Metrics), an open-source tool that profiles the network behavior of native desktop applications and flags anomalous traffic indicative of a supply-chain compromise. Trained on 56 billion HTTP transactions from 2,000 organizations, BEAM uses per-application XGBoost models with SHAP explainability to detect when a trusted app — such as Spotify or Box — is communicating with attacker-controlled infrastructure. A red-team/blue-team exercise confirmed it catches real-world attacks with up to 99% confidence.

Introduction

The SolarWinds supply chain compromise was the catalyst for this research. Colin Estep, a seven-year veteran of Netskope Threat Labs, recalled that SolarWinds customers were often mature organizations — with SOC analysts, vulnerability scanning, and penetration testing programs — yet the intrusion had a dwell time of more than a year. Every conventional defensive layer was in place; none detected the SUNBURST implant communicating out through what appeared to be a legitimate SolarWinds Orion update.

"There was a dwell time of more than a year for this," Dagmawi Mulugeta told the audience. "It struck me how organizations that did all these things we're supposed to be doing still got hit."

BEAM represents a fundamentally different detection philosophy: rather than looking for known-bad indicators, it models what normal looks like for each application individually, then flags deviations. It works entirely from network traffic captures, sits out-of-band, and requires no endpoint agent.

The Detection Approach: Profile, Compare, Explain

The research follows a three-phase methodology: attribution, profiling, and detection.

▶ Watch: Three-Phase Approach (10:01)

Phase 1 – Attribution: Before modeling an application, traffic must be attributed to it. BEAM primarily leverages HTTP User-Agent strings, which are surprisingly reliable: data from 2,000 organizations represents real users whose behavior is not tainted by manual User-Agent modification. BEAM uses a combination of Python parsing libraries and an embedded Llama model (with optional Gemini API integration) to normalize free-form User-Agent strings to structured (application name, version) tuples.

Phase 2 – Profiling: With 56 billion HTTP request-response pairs (transactions) across 4.2 million devices running 1.5 million distinct native applications, the team identified 185 behavioral features to capture the "essence" of how an application normally communicates. These include:

• Request and response latency distributions (min, max, median, standard deviation)
• Inter-request timing patterns
• URL entropy (Shannon entropy of path components and query strings)
• HTTP method and status code distributions
• File types and transfer sizes
• Destination hostnames and whether they are related to known application infrastructure

▶ Watch: Feature Engineering (14:02)

The choice of features was directly motivated by known attacks: domain anomalies mirror SUNBURST, repository-access anomalies mirror the 3CX attack, and unexpected outbound data transfer mirrors MOVEit.

Phase 3 – Detection: An initial random-forest multi-class experiment with 20 applications and injected malware traffic validated the approach: applications were highly separable from each other even at small data volumes. For production BEAM, the team switched to one XGBoost model per application, trained on 500,000 observations each. Predictions are explained using SHAP (SHapley Additive exPlanations) values, which show which features drove the anomaly score and in which direction.

Red Team vs. Blue Team: The Spotify Test

To validate BEAM against a realistic adversarial scenario, the team ran an internal red team/blue team exercise. They told their red teamer, Mohanraj, to pick any application from the monitored set, compromise it, and establish C2 communication — without disclosing which app, which C2, or any operational details.

▶ Watch: Red Team Exercise (20:03)

Mohanraj chose Spotify. His technique: inject a malicious Spotify User-Agent into a process communicating with GitHub Codespaces (super-duper-chains-github.dev), giving him remote code execution on the victim machine while masking C2 traffic as developer activity. "If you're just looking at this traffic in aggregate, it probably wouldn't raise any red flags," Estep noted — a Spotify process talking to GitHub Codespaces looks like a developer listening to music.

BEAM flagged the traffic as anomalous with 94% confidence. The SHAP analysis identified the contributing factors:

• Request timing was off: the millisecond latency distribution did not match Spotify's normal profile.
• Client-byte volumes (sum, average, minimum) were inconsistent with what Spotify normally sends.
• Transaction count was low — Mohanraj deliberately tried to stay "low and slow," which partially suppressed one signal — but the other features overrode this.

The red teamer's reaction to seeing the detection result, Mulugeta said, "made us giggle like a couple of schoolgirls."

Live Demo: Box Compromise Detection

A second scenario demonstrated BEAM in its current open-source form, using a HAR (HTTP Archive) file as input.

▶ Watch: BEAM Tool Demo (26:04)

BEAM parsed 300 network transactions from the HAR file, automatically identified seven applications in the traffic, found it had a model for Box, and ran detection. It flagged a "critical compromise" — Box communicating with dagmawi.io — with 99% confidence. The top four contributing SHAP factors:

1. URL entropy: Shannon entropy of the URLs was inconsistent with Box's normal path structure (Box URLs contain file identifiers, chunk markers, etc.).
2. Server bytes: Volume of data returned by the server did not match Box's distribution.
3. Request latency: Timing was not in line with normal Box behavior.
4. Key hostnames: The destination hostname was not only absent from Box's known-good host list, it was also unrelated to any Box infrastructure — flagging it as especially anomalous.

BEAM's output shows the top few indicators in the terminal; a full SHAP waterfall plot is also saved to disk for deeper analysis.

Building Custom Models for Your Own Applications

BEAM ships with pre-trained models for eight common enterprise applications (including Box and Spotify) — deployable immediately with no learning period. For other applications, it includes an unsupervised training pipeline requiring only a PCAP or HAR traffic capture with at least 100 transactions.

▶ Watch: Custom Model Training (32:05)

The unsupervised ensemble consists of an isolation forest, a one-class SVM, and an autoencoder. A live demo showed training a Notion model from a HAR file captured with ProxyMan, then running detection over the same file — producing a clean "no anomalies" result. The same workflow supports any application accessible via a proxy.

The team acknowledged open challenges:

• High-entropy applications (browsers, general-purpose runtimes) produce noisy behavioral profiles that make anomaly detection harder.
• Attribution accuracy remains a work in progress when User-Agent strings are absent, stripped, or modified by an attacker.
• Bespoke model quality from the unsupervised path is less robust than the production XGBoost models trained on half a million observations.

Community contributions are welcomed; BEAM is available at github.com/Netskope as of the date of this talk.

Notable Quotes

"The SolarWinds compromise struck me how organizations that did all these things we're supposed to be doing still got hit."

— Dagmawi Mulugeta ▶ 00:00

"If you're just looking at this traffic in aggregate, it probably wouldn't raise any red flags because it's just like, oh, maybe it's just developer traffic."

— Colin Estep ▶ 22:03

"Supply chain compromises are too terrifying to leave to one type of solution."

— Colin Estep ▶ 36:05

"Every time we do that demo, it kinda gives me a little bit of chills."

— Dagmawi Mulugeta ▶ 26:04

Key Takeaways

• Application-level behavioral profiling is a viable supply chain detection strategy: By modeling each app's network fingerprint independently, BEAM narrows the problem to detecting deviations in known-good behavior rather than searching for known-bad patterns.
• 185 features capture meaningful application identity: URL entropy, request timing, client/server byte distributions, and destination hostnames reliably distinguish applications and expose anomalous communications.
• SHAP explainability turns ML predictions into actionable alerts: Rather than opaque confidence scores, defenders receive ranked, human-readable reasons why traffic was flagged.
• BEAM is open source and deployable today: Pre-trained models cover eight common enterprise applications; the unsupervised training pipeline extends coverage to any application with a traffic capture.
• Out-of-band, proxy-independent deployment: BEAM analyzes existing PCAP or HAR files; it does not need to be in the network path or integrated with any specific NDR/EDR product.

Slides PDF not available for this talk.