Filtering the Noise: Crafting an Actionable Threat Feed from Community-Shared Intelligence

Filtering the Noise: Crafting an Actionable Threat Feed from Community-Shared Intelligence

Speakers: JJ Josing, MISP Wrangler, Retail & Hospitality ISAC

Conference: FIRST CTI Conference

YouTube: https://www.youtube.com/watch?v=mo9W9XsytpE

Overview

In the realm of cybersecurity, the sheer volume of threat intelligence can be overwhelming, often leading to analyst fatigue, delayed responses, and reduced trust in shared data. This talk, "Filtering the Noise: Crafting an Actionable Threat Feed from Community-Shared Intelligence," presented by J.J. Josing, a MISP Wrangler at the Retail & Hospitality ISAC (RH-ISAC), addresses this critical challenge head-on. Josing outlines a scalable, automated methodology designed to transform raw, high-volume indicators of compromise (IOCs) into high-fidelity, actionable intelligence.

The core premise of the presentation is that intelligence is only valuable when it is operational. Josing emphasizes the need to move beyond simply collecting indicators to actively vetting and enriching them, ensuring that security teams can confidently take defensive actions. The RH-ISAC's approach, leveraging open-source tooling like MISP and a custom Python library, offers a practical blueprint for organizations and intelligence sharing communities to operationalize threat feeds, foster proactive defense, and build greater trust among members.

This article delves into the RH-ISAC's innovative framework, exploring its foundational principles, technical architecture, and the demonstrable benefits it delivers. From tackling the inherent noise in community-shared intelligence to empowering defenders with low-regret, high-confidence indicators, Josing's insights provide a compelling case for a more intelligent and efficient approach to threat intelligence consumption and utilization.

Background

▶ Watch: Introduction and Talk Goals (0:00)

The landscape of threat intelligence sharing, particularly within Information Sharing and Analysis Centers (ISACs), presents a paradox: an abundance of data often translates into a scarcity of actionable insight. ISACs, formed under a 1998 US presidential directive, serve as vital hubs for critical infrastructure owners and operators to exchange threat information, aiming to protect organizations, personnel, and customers. Their mission involves collecting, analyzing, and disseminating actionable threat intelligence. A cornerstone tool for many ISACs and intelligence communities is MISP (Malware Information Sharing Platform), an open-source threat intelligence platform renowned for fostering the sharing of structured information and its robust API and Python library.

However, the reality of shared intelligence feeds is fraught with challenges. As Josing highlights, these feeds typically contain a large volume of indicators, often arriving with uneven quality. Many indicators, while technically accurate (e.g., a hash for cmd.exe found in a malware's process chain), are operationally harmful if actioned blindly, leading to excessive noise and potential false positives. This lack of confidence, context, or relevance to specific industries contributes significantly to analyst fatigue, where the sheer volume of triage becomes a bottleneck, leading to burnout. This fatigue erodes trust in shared intelligence, resulting in indicators being ignored and, critically, a delayed response to emerging threats. The goal, therefore, is twofold: prevent low-fidelity indicators from entering the operational pipeline and establish a scalable, automated enrichment and vetting process to produce truly actionable intelligence.

The consequences of low-quality feeds are severe, including noisy enforcement, ignored true positives, reduced trust in data sources, and a reactive rather than proactive incident response posture. To counteract this, Josing introduces the low-regret methodology pioneered by the Johns Hopkins University Applied Physics Lab (JHU APL). This approach prioritizes speed and automation over absolute certainty, focusing on identifying defensible actions with minimal operational risk, even under uncertainty. A core insight is that waiting for complete determination of maliciousness often negates the intelligence's value. JHU APL's research found that nearly one-third of all threat intelligence can be automatically evaluated as low-regret, meaning it is extremely unlikely to disrupt operations if blocked. Conversely, high-regret indicators (e.g., legitimate O365 infrastructure IPs) carry significant operational risk if actioned. The RH-ISAC's framework builds upon this methodology, aiming to identify and action low-regret indicators at scale, thereby shifting the focus from whether to take an automated action to when.

Key Findings

▶ Watch: Low-Regret Methodology for Actionable Intel (4:49)

The RH-ISAC's journey into automated threat intelligence vetting has yielded several significant findings, demonstrating a practical and effective model for intelligence sharing communities.

Firstly, the talk conclusively shows that automated enrichment and vetting can achieve high rates of actionable intelligence. By applying their methodology, the RH-ISAC achieved a vetted indicator rate of approximately 45-48% across quarters in 2025, significantly exceeding the Johns Hopkins University Applied Physics Lab's research finding that roughly one-third of intelligence can be classified as low-regret. This demonstrates that a well-tuned automated process can extract a substantial portion of immediately actionable indicators from community feeds.

Secondly, the introduction and successful implementation of the Python Open Threat Intelligence (py-oti) library stands out as a major contribution. This open-source, modular API framework standardizes the output from dozens of different threat intelligence and reputation APIs. py-oti abstracts away the complexities of interacting with various vendor-specific APIs, providing a unified, machine-readable format. Its lightweight design (only requests, AIODNS, and disposable-email-domains as dependencies) makes it highly accessible, even for those without deep developer expertise.

Thirdly, the development and application of a custom enrichment taxonomy, PA ODi, is crucial. This taxonomy translates the raw outputs of various enrichment lookups into a standardized, machine-readable format compatible with MISP's classification system (namespace:predicate=value). This tagging system provides essential context, moving beyond mere lists of IOCs to explain why an indicator is deemed malicious or benign, thereby preserving valuable information for analysts and reducing the need for manual inference.

Fourthly, Josing emphasizes that consistent vetting builds trust, reduces manual review, and preserves context. By transparently documenting and applying their vetting logic, the RH-ISAC has cultivated confidence among its 350+ member companies, leading to over 139 members integrating their tools with the community MISP for automated ingestion. This increased trust has also spurred nine members to automate their intelligence sharing, pushing indicators directly from their SOAR platforms or MISP instances, significantly improving timeliness.

Finally, the presentation highlights the importance of strategic tuning and continuous improvement. Josing recommends setting minimum enrichment sources (at least two per indicator type) to avoid single points of failure, defining clear thresholds for confidence and reputation, and maintaining allow lists for benign files or legitimate infrastructure. The process is described as a "living process," requiring ongoing feedback and adjustments, rather than a one-time configuration. This iterative approach ensures the threat feed remains relevant, accurate, and highly trusted.

Technical Deep Dive

▶ Watch: Identifying False Positives with Services (8:15)

The RH-ISAC's framework for crafting actionable threat intelligence is built upon a sophisticated yet accessible technical architecture, primarily leveraging MISP and the custom py-oti library.

At the heart of the automated enrichment process is the Python Open Threat Intelligence (py-oti) library. This open-source project, available on GitHub, serves as a universal translator for external threat intelligence APIs. It provides standardized wrappers for approximately three dozen different services, covering a broad range of indicator types (IPs, domains, hashes, emails). The primary objective of py-oti is to normalize all API outputs into a consistent, machine-readable format, significantly reducing development time for integrating diverse intelligence sources. Its lightweight nature, with minimal dependencies (requests, AIODNS, and disposable-email-domains), ensures ease of deployment and use. For example, checking Google Safe Browsing is as simple as module = GoogleSafeBrowsing(); result = module.check_indicator(url);.

The RH-ISAC specifically utilizes a combination of these services for different indicator types. Key services include:

• AbuseIPDB: For abuse confidence scores on IP addresses.
• Barracuda Central Reputation Blacklist: To identify domains on known blacklists.
• Google Safe Browsing: To detect social engineering threats or malicious URLs.
• Greynoise Riot: A critical tool for identifying potential false positives related to IP addresses. It indicates if an IP belongs to cloud infrastructure and is controlled by the cloud provider (trust level one), suggesting it should not be automatically actioned (e.g., legitimate O365 IPs). Trust level two indicates a cloud IP leased for a VPS, which might warrant further investigation.
• Circle Hash Lookup: Essential for vetting file hashes. This service provides a software registry to identify benign files commonly found in operating systems like Windows and Linux distributions, helping to filter out noisy indicators such as cmd.exe or PowerShell hashes.
• Cisco Umbrella Investigate: Used for domain risk scoring and reputation lookups. This is one of the few commercial tools the RH-ISAC pays for, demonstrating its value in their process.
• Spamhaus: For identifying domains on phishing or spam blacklists.
• VirusTotal: Utilized for various lookups, including identifying if a domain was registered within the last 30 days (a common indicator of malicious intent) and detecting if a file hash is associated with a known software distributor.

Once enrichment lookups are performed, the results are translated into a standardized format using the PA ODi taxonomy. This custom MISP taxonomy (namespace:predicate=value) provides granular, machine-readable context. For instance, an IP with a high abuse confidence score from AbuseIPDB would be tagged priority:abuse-IPDB="high". If it's on Barracuda's blacklist, priority:barracuda-reputation="blacklist" is applied. A file hash identified as a known distributor by VirusTotal receives priority:virustotal-known-distributor="true". This contextual tagging is crucial because it moves beyond a simple "bad/good" disposition, explaining why an indicator is categorized as such, preserving valuable information for analysts.

The RH-ISAC's workflow integrates these components seamlessly:

1. Collection: Intelligence from various sources (Slack, MISP, etc.) is gathered by the team without imposing sharing restrictions on members.
2. Normalization and Ingestion: All collected intelligence is pulled into MISP, where it's normalized for subsequent enrichment and vetting. Published events within MISP trigger the automated enrichment process.
3. Automated Enrichment & Vetting:

• For a malicious domain, py-oti queries services like VirusTotal (for new registration data), Cisco Umbrella (for risk score), and Spamhaus (for blacklists). If the domain is new (<30 days), has a medium-high risk score (e.g., umbrella-investigate:risk="medium-high"), and is on a phishing blacklist (spamhaus:fish="true"), these tags are applied. If any of the defined vetting criteria are met, the indicator is marked with the RH-ISAC:vetted tag.
• False positives are explicitly handled. For benign file hashes like cmd.exe, Circle Hash Lookup might assign a hash-lookup:trust="medium" score, and VirusTotal might flag it as a virustotal:known-distributor="true". For legitimate cloud infrastructure IPs (e.g., Outlook), Greynoise Riot tags them as greynoise:riot="trust-level-1". Commonly abused legitimate services like Amazon SES or SendGrid are tagged with abuse:legit="sendgrid", indicating their dual-use nature without necessarily marking them as outright malicious in all contexts.

1. Manual Review (Human-in-the-Loop): After automated enrichment, the event is temporarily unpublished. The RH-ISAC team reviews the applied tags and vetting decisions to ensure accuracy and prevent automation errors before publishing the event for member consumption. This crucial step maintains quality control.
2. Dissemination: Vetted indicators are disseminated to members directly through the MISP API or MISP synchronization, allowing members to integrate them into their security tools.

The vetting logic is defined programmatically. For a domain, the logic might be: "If risk score > 50 OR on any reputation blacklist OR Google Safe Browsing identifies it as a threat OR registered < 30 days, THEN mark as vetted." If none of these conditions are met, it is not vetted, but all enrichment tags are still preserved. This ensures that even non-vetted indicators retain valuable context.

To optimize API usage and reduce costs, the RH-ISAC implements a Redis cache database. Enrichment lookup results are stored for approximately 72 hours. If an indicator is shared again within this window, the cached results are used, conserving API quotas, particularly for freemium services.

Finally, Josing recommends leveraging external projects like the Lolbas Project (Living Off the Land Binaries and Scripts) and the Living Off Trusted Sites Project to add further context. While indicators related to these projects might not always warrant immediate blocking, knowing that a file or URL is part of a "living off the land" technique is invaluable for an analyst during an investigation. This reinforces the principle that tags should add context, not just a binary disposition.

Demo / Proof of Concept

▶ Watch: Automated Vetting Logic and Decision Tree (17:00)

While the talk did not feature a live, interactive "demo" in the traditional sense, J.J. Josing effectively demonstrated the practical application and efficacy of the RH-ISAC's automated threat intelligence pipeline through detailed explanations of the workflow, illustrative examples of indicator enrichment, and the tangible outcomes achieved. The entire process, from indicator ingestion to member consumption, serves as a robust proof of concept for the presented methodology.

The demonstration of the py-oti library's simplicity, with its three-line code examples for checking services like Google Safe Browsing, showcased its accessibility for a wide range of users, not just "hardcore developers." Josing walked through the RH-ISAC's community MISP instance workflow, detailing how various sources feed into MISP, how the intelligence is normalized, and then enriched and vetted before dissemination.

Key elements of the "demo" included:

• Illustrative Tagging Examples: Josing presented specific scenarios, such as a malicious domain being enriched with tags like virustotal:new-domain="true", umbrella-investigate:risk="medium-high", and spamhaus:fish="true". This visually conveyed how raw indicators are transformed into contextualized intelligence within MISP.
• False Positive Handling: The presentation explicitly "demonstrated" how the system identifies and tags benign file hashes (hash-lookup:trust="medium", virustotal:known-distributor="true") and legitimate cloud infrastructure IPs (greynoise:riot="trust-level-1"). This critical aspect highlighted the system's ability to prevent noisy or harmful enforcement actions.
• MISP Interface Screenshots: Although not live, the speaker showed how the enriched and vetted indicators appear within MISP, displaying a list of domains alongside their applied tags (e.g., Spamhaus phishing, new domain, medium-high risk score) and the crucial RH-ISAC:vetted tag. This provided a clear visual representation of the end product consumed by members.
• Statistical Validation: The presented statistics—a 45-48% vetted indicator rate from approximately 9,000 indicators per quarter—served as empirical proof of the system's effectiveness and scalability, validating the low-regret methodology in a real-world ISAC environment.

In essence, the "Demo / Proof of Concept" was woven into the fabric of the technical deep dive, showcasing the operationalization of the framework and its tangible results rather than a step-by-step live tool demonstration. It underscored that the RH-ISAC's automated pipeline is not merely theoretical but a fully functioning system delivering high-fidelity, actionable threat intelligence to its diverse membership.

Defensive Implications

▶ Watch: Q&A: Vetted Indicator Percentage and Process (28:00)

The RH-ISAC's automated threat intelligence framework carries profound implications for defensive cybersecurity strategies, offering a pathway to more efficient, proactive, and trustworthy security operations.

Firstly, this approach enables a crucial shift from reactive to proactive defense. By consistently vetting and enriching indicators, organizations can ingest high-fidelity intelligence with confidence, allowing them to block threats before they manifest as incidents. Josing shared anecdotes of members' email gateways blocking phishing campaigns because they had ingested vetted indicators shared just days earlier, demonstrating the immediate, tangible benefits of this proactive stance.

Secondly, the automation significantly reduces analyst fatigue and triage time. By offloading the repetitive, tier-one lookups (e.g., checking IP reputation, domain registration dates) to automated systems, security analysts are freed from mundane tasks. This allows them to focus on deeper investigations, complex threat analysis, and more meaningful collaboration within sharing communities, ultimately leading to greater job satisfaction and preventing burnout. The RH-ISAC's 45-48% vetting rate means nearly half of all incoming indicators can be actioned without manual intervention.

Thirdly, the consistent and transparent vetting process builds immense trust in shared intelligence. When members know that the indicators they receive have undergone rigorous, documented scrutiny, they are far more likely to integrate and action them. This increased confidence is evidenced by the 139 RH-ISAC members who have integrated their tools with the community MISP to pull down vetted indicators, and the nine members who have fully automated their intelligence sharing. This trust is foundational for effective community-driven defense.

Fourthly, the framework provides scalability for handling the ever-increasing volume of threat data. As threat feeds grow, manual processing becomes unsustainable. Automation allows ISACs and organizations to process thousands of indicators per quarter without a proportional increase in human resources, ensuring that intelligence remains timely and relevant.

Fifthly, the emphasis on contextual tagging ensures that even non-vetted or potentially benign indicators retain valuable information. Defenders can use tags like greynoise:riot="trust-level-1" or abuse:legit="sendgrid" to understand why an indicator might be noisy or dual-use, allowing for nuanced decision-making rather than blanket blocking. This prevents operational disruptions from legitimate services.

Finally, the methodology fosters customization and continuous improvement. Organizations can tune the vetting requirements to their specific risk appetite, starting with strict thresholds (e.g., only blocking indicators with a "high" risk score) and gradually loosening them over time. The recommendation to deploy integrations in "audit mode" initially allows defenders to observe potential impacts before enforcing blocks. The transparency of the documented enrichment and vetting process encourages feedback from members, creating a living system that adapts to evolving threats and organizational needs. This collaborative refinement ensures the defensive posture remains robust and relevant.

Key Takeaways

• Automated Enrichment is Essential: Transforming raw, high-volume indicators of compromise into high-fidelity, actionable intelligence requires robust, automated enrichment and vetting processes to combat analyst fatigue and enable proactive defense.
• Leverage Open-Source Tools: Platforms like MISP, coupled with custom open-source libraries like py-oti, provide a scalable and cost-effective solution for standardizing and processing threat intelligence from diverse sources without relying heavily on commercial tools.
• Embrace Low-Regret Methodology: Prioritizing speed and automation for indicators with minimal operational risk allows organizations to confidently block a significant portion of threats (e.g., RH-ISAC's 45-48% vetted rate) while carefully managing high-regret indicators.
• Context is King, Tags are Key: Beyond simple malicious/benign flags, detailed contextual tagging using machine-readable taxonomies (like PA ODi) is crucial for analysts to understand why an indicator is flagged, preserving valuable information and reducing manual inference.
• Build Trust Through Transparency and Tuning: A transparent, well-documented vetting process, combined with continuous tuning based on feedback, is vital for building confidence in shared intelligence and fostering greater adoption and automated actioning among community members.
• Start Small and Iterate: Organizations should begin by defining clear logic and thresholds, potentially starting with stricter requirements or audit modes, and then iteratively loosen them based on observed results and ongoing feedback, treating the process as a living system.

About the Speaker(s)

J.J. Josing is a MISP Wrangler at the Retail & Hospitality ISAC (RH-ISAC). His professional tradecraft heavily relies on open-source tooling, automation, and scalable intelligence sharing solutions. In his role, J.J. supports member adoption of the RH-ISAC's community MISP instance, troubleshoots issues, and ensures the smooth operation of their shared intelligence pipeline. His day-to-day responsibilities involve deep diving into emerging threats and operationalizing intelligence through MISP integrations and enrichment pipelines. J.J. hails from Milwaukee, Wisconsin, where he describes himself as a deep-fried cheese curd enthusiast and a professional milk drinker. Outside of CTI, he enjoys lifting weights, playing music, and wandering in the woods.