KnowHow: Automatically Applying High-Level CTI Knowledge for Interpretable and Accurate Provenance Analysis

KnowHow: Automatically Applying High-Level CTI Knowledge for Interpretable and Accurate Provenance Analysis

Speakers: Yuhan Meng

Conference: NDSS Symposium

YouTube: https://www.youtube.com/watch?v=6kOxaUDRz9s

Overview

Advanced Persistent Threat (APT) attacks remain one of the most critical challenges facing governments and enterprises, distinguished by their advanced, stealthy, and persistent characteristics. While provenance-based detection systems that construct directed attributed graphs of system entities have shown promise, existing knowledge-driven approaches are limited by their reliance on manually crafted, instance-level rules. This talk introduces KnowHow, a system that bridges the fundamental semantic gap between high-level Cyber Threat Intelligence (CTI) reports and low-level system provenance data by automatically mapping abstract threat descriptions to concrete system events.

The core innovation is the General Indicator of Compromise (GIOC), a structured triple decomposition of CTI knowledge into conductor-action-target format that aligns naturally with system-level entities and behaviors. KnowHow achieves near-perfect graph-level accuracy while reducing false positives by up to 90% at the node level, and it has been validated in real-world deployment across 180 endpoints.

Background

▶ Watch: Introduction: APT challenges and provenance-based detection (0:00)

APT attacks differ fundamentally from traditional malware through three key characteristics: they are advanced in technique, stealthy in execution, and persistent in their presence within target environments. These properties make them extremely difficult to detect using conventional defense tools like signature-based antivirus or simple anomaly detection.

Provenance-based detection systems have emerged as a promising approach. These systems construct directed attributed graphs where nodes represent system entities (processes, files, network connections) and edges represent interactions between them. This graph structure captures causal context and interdependencies that are critical for detecting multi-stage attack campaigns. Among these systems, knowledge-driven approaches are widely deployed in practice due to their efficiency and interpretability. They incorporate attack knowledge -- primarily crafted by security experts from CTI reports -- to detect known malicious patterns in provenance graphs.

However, existing knowledge-driven systems suffer from a critical limitation: they rely on two types of CTI knowledge with very different automation profiles. Low-level knowledge such as Indicators of Compromise (IOCs) -- file hashes, IP addresses, domain names -- can be automatically extracted and matched using tools like Extractor and TTP Drill. But IOCs are inherently time-sensitive and frequently fail to match zero-day techniques. High-level knowledge describes abstract adversary behaviors, such as "carefully scan the browser login directory," which provides greater generalizability but cannot be directly mapped to low-level system events due to the fundamental semantic gap between natural language descriptions and system call logs.

Key Findings

▶ Watch: Key insight: GIOC structured triples bridge the semantic gap (4:00)

The research yields several significant findings about CTI-driven detection:

• Near-perfect graph-level accuracy: KnowHow detects all attacks with zero false positives at the graph level, meaning every APT campaign in the evaluation dataset was correctly identified without any false alarm campaigns.
• 90% false positive reduction at node level: Compared to existing approaches, KnowHow reduces node-level false positives by up to 90% while maintaining recall, addressing the alert fatigue problem that plagues security operations.
• 87% technique labeling accuracy: KnowHow accurately labels 87% of attack actions, with over 500 actions uniquely identified by GIOCs, accounting for more than 80% of all identified actions. Only around 100 of those actions could have been detected by concrete IOCs alone.
• Scalability validated in production: The system successfully monitors over 180 host endpoints in real-world enterprise deployment, demonstrating that the approach scales beyond academic benchmarks.
• Robustness against evasion: KnowHow detects all attacks regardless of inserted mimicry graph data ratios and maintains stable performance even with incomplete attack sequences, only experiencing slight degradation when attack integrity drops below 80%.
• Search optimization reduces computation by 17%: The two-stage clustering approach using mean shift algorithm on GIOC embedding vectors reduces unnecessary comparisons by over 17%, enabling real-time detection throughput.

Technical Deep Dive

▶ Watch: Embedding-based ProofQ matching and two-stage search optimization (8:00)

The semantic gap between high-level CTI and low-level provenance data is the central technical challenge. The key insight is that high-level CTI descriptions can be decomposed into a subject-verb-object semantic structure that maps directly to provenance graph elements. KnowHow introduces the General Indicator of Compromise (GIOC) -- a structured triple consisting of:

1. Conductor: The attack entity performing the action (maps to process entities in provenance data)
2. Action: The adversary behavior being performed (maps to system call types)
3. Target: The entity being acted upon (maps to file paths, network endpoints, or other system resources)

Unlike standard subject-verb-object triples that only retain core words, GIOCs incorporate modifiers and subordinate clauses associated with each element, capturing richer behavioral context. For example, "carefully scan the browser login directory" produces a GIOC where the conductor maps to a process, the action "scan" corresponds to "read" system calls, and "browser login directory" maps to concrete file paths like Firefox resource directories.

The extraction pipeline identifies attack-relevant concepts by checking whether nouns satisfy specific criteria for conductors or targets, then incorporates their modifiers and subordinate clauses to form complete GIOC components.

For matching GIOCs against runtime events, KnowHow employs an embedding-based query operation (ProofQ). The process involves three steps: event semantic lifting (converting system events into natural language templates), embedding both the lifted event and the GIOC into vector space, and cosine similarity computation between the vectors. Events exceeding a predefined threshold are flagged as potentially malicious.

To make ProofQ scalable for real-time detection, a two-stage search optimization is employed. In an offline stage, all GIOCs are clustered using the mean shift algorithm on their embedding vectors, processing subjects, verbs, and objects separately. At runtime, each event field is first matched to its nearest cluster, and only GIOCs within that cluster are compared, reducing unnecessary computations by over 17%.

The CTI Knowledge Database (CKD) stores processed intelligence, where each entry (termed an ATI entry) represents one TTP and stores corresponding CTI reports and related GIOCs. The attack reasoning model based on a relaxed APT lifecycle reconstructs attack chains and filters false positives by enforcing two principles: attacks must include an initial compromise and at least one following step, and attack stages must follow a valid temporal sequence.

Demo / Proof of Concept

▶ Watch: Evaluation results: near-perfect accuracy and 90% false positive reduction (10:00)

KnowHow was validated across multiple evaluation dimensions. In closed-world evaluation, the system achieved near-perfect graph-level accuracy across benchmark datasets. For open-world evaluation, the researchers constructed a new dataset called NewlySeen using newly disclosed CVEs to validate performance against previously unseen attack techniques. KnowHow successfully detected two unseen multi-stage APT attacks in this dataset and generated human-readable reports, demonstrating scalability to novel threats.

The system was also tested against mimicry attacks, where mimicry graph data was generated following previous research methodology and inserted into original datasets. KnowHow detected all attacks regardless of the inserted ratio of mimicry graph data. Performance remained stable even with incomplete attack sequences, only experiencing slight degradation when attack integrity ratios dropped below 80%.

In real-world enterprise deployment, KnowHow successfully monitored over 180 endpoints, demonstrating that the approach translates from academic benchmarks to operational security environments.

Defensive Implications

▶ Watch: Q&A: discussion of mimicry dataset and relaxed lifecycle model (14:00)

KnowHow addresses a critical operational challenge for security operations centers: the overwhelming volume of false positive alerts generated by existing knowledge-driven detection systems. By moving from instance-level IOC matching to behavior-level GIOC matching, defenders gain several advantages:

• Reduced alert fatigue: The 90% reduction in false positives at the node level means analysts can focus on genuine threats rather than chasing phantom alerts.
• Automated CTI operationalization: High-level CTI reports that previously required manual expert translation into detection rules can now be automatically converted into actionable detection logic through GIOC extraction.
• Scalable knowledge base expansion: Because GIOCs capture behavioral patterns rather than specific artifacts, the detection knowledge scales across thousands of CTI reports without requiring expert rule-writing for each one.
• Interpretable detection: Unlike black-box ML approaches, KnowHow's detection decisions can be traced back to specific CTI reports and GIOCs, providing analysts with context for understanding why an alert was generated.
• Zero-day resilience: Because GIOCs describe behaviors rather than specific indicators, the system can detect novel attack techniques that use the same behavioral patterns described in existing CTI reports, even when specific file hashes or IP addresses have changed.

Key Takeaways

• The semantic gap between high-level CTI knowledge and low-level system events is the primary barrier preventing fully automated, extensible knowledge-driven APT detection
• General Indicators of Compromise (GIOCs) provide a structured conductor-action-target decomposition that bridges this gap by aligning CTI semantics with provenance graph elements
• Embedding-based matching with cosine similarity enables fuzzy semantic matching between abstract CTI descriptions and concrete system events
• KnowHow achieves near-perfect graph-level detection with zero false positives and reduces node-level false positives by 90%
• Real-world deployment across 180 endpoints validates that the approach scales beyond academic benchmarks
• The system is robust against mimicry attacks and maintains stable performance with incomplete attack sequences

About the Speaker(s)

The presentation was delivered by a co-author from Peking University on behalf of the first author, Yuhan Meng, who could not attend due to visa issues. The research was conducted in collaboration with San Jose State University and Southeast University. The work represents a cross-institutional effort combining expertise in provenance analysis, natural language processing, and cyber threat intelligence.