Why does intelligence work still suffer from a noise problem? The technology to tackle the ‘big data’ challenge exists, and in some places it’s working. But most tools in our field still haven’t solved it.
Instead, analysts are stuck with tools that force an impossible tradeoff: systems designed to find specific needles in haystacks—targeted threat identification when you know exactly what you’re looking for—work well for identifying known threats that might pose great risk, but potentially leave you blind to emerging threats you haven’t anticipated. Meanwhile, tools built for broad awareness bury critical signals under mountains of noise. This fundamental limitation has shaped how we build tools, allocate resources, and measure success for decades. But it really doesn’t have to!
The last six months have made this problem impossible to ignore. We’ve pivoted from the Ukrainian drone attack on Russian airbases to 12 days of direct hostilities between Israel and Iran, then back to the situation in Gaza, across to Venezuela and drug interdictions in the Caribbean, over to the Thailand-Cambodia border, and back to the Middle East with Israel’s strike on Hamas in Qatar. As I write, Europe is experiencing Russian incursions in Polish, Romanian and Estonian airspace, Sudan and DRC are simmering, and countless other developing situations demand attention.
That’s just six months of geopolitical whiplash and global crisis hopping and I know I’m not even capturing it all. No analyst, no team, no tool can meaningfully cover it all without help. Which is exactly the point.
This constant pivoting between regions and crisis types leaves analysts exhausted and stakeholders confused. How do you know what to focus on when everything feels urgent? How do you maintain situational awareness across such a broad threat landscape without drowning in information overload?
AI should be eliminating this choice entirely, yet most intelligence tools are making it worse. We’re still building systems optimized for finding needles in haystacks, when what analysts really need is effective horizon scanning, and indicators and warning (I&W) capabilities that separate signal from noise at scale.
The Barrier to Entry Problem
There’s a fundamental barrier to entry problem in intelligence work. If you don’t have a baseline understanding of what matters, it’s nearly impossible to know what you should be paying attention to. That’s the heart of effective I&W work – looking for trigger events happening around you and understanding how they’re changing.
For analytical minds, this requires significant brain power that tooling should be handling. For non-analysts, it creates an almost insurmountable barrier to entry.
Consider the difference between having a specific question – like needing a travel risk assessment for India – versus trying to understand what’s happening when you don’t necessarily have a question to answer. The first is directed and answerable. The second requires horizon scanning capabilities that can surface what you should be paying attention to, even when you don’t know what to ask.
Current tools claim to solve this through alerts and notifications, but most analysts report living out of their inboxes, inundated with updates that lack context or prioritization. The signal-to-noise ratio remains unacceptably high because organizations are still trying to solve for specific and individual threats instead of addressing the broader challenge of trend identification and landscape monitoring.
Why the Industry Accepts Subpar Outcomes
There’s clear demand for needle-hunting because the fear of missing something is real. Budgets get cut, credibility gets questioned, and careers end when threats that were detectable aren’t detected. This creates incentives to focus on measurable threat identification rather than the harder problem of horizon scanning.
But I would argue this focus has led the industry to accept subpar outcomes. Analysts receive monotonous updates without understanding which ones matter. False positives aren’t necessarily the problem: it’s the volume of irrelevant information. How do you know which updates to a situation deserve follow-up? How do you distinguish between routine developments and genuine escalations?
Most tools, especially ones utilizing AI, haven’t invested the brain power to solve horizon scanning effectively. Many lack the historical, human-driven analytical data needed to derive trustworthy insights about what constitutes meaningful change versus noise. Without understanding analyst tradecraft and workflow requirements, these systems generate alerts without context and updates without prioritization.
When You Don’t Know What to Ask
Effective horizon scanning is really about knowing the right questions to ask and signals to look for. Analysts often have intuitive concerns that they struggle to articulate: “Something feels off about this situation, but I can’t pinpoint what.” Or they need help exploring related patterns: “If this is happening here, what should I be watching for elsewhere?”
The most valuable AI systems would help analysts formulate these questions based on available analytical content. Instead of just responding to queries, AI could provide what analysts should be asking for based on developing situations, historical patterns, and expert analysis.
This becomes even more critical for intelligence-adjacent personas who need to consume intelligence but often don’t have the analytical training, by design. Regional security managers, crisis coordinators, executive protection teams, and corporate communications staff all need threat insights but don’t necessarily have the tradecraft to sort through this chaff effectively, nevermind its not built into their daily responsibilities.
Reactive and Proactive Intelligence
Effective AI for intelligence work requires both these reactive and proactive capabilities. Analysts need the ability to ask specific questions and explore developing situations on their own terms. But they also need systems that can surface the developments they should be paying attention to, even when they haven’t thought to ask.
This dual approach recognizes that analysts sometimes have directed inquiries like “What’s the security situation for our facilities in City X?” while other times they need help identifying emerging patterns they might miss. The latter requires AI that can assess accelerating news stories, evaluate analytical assessments, continuously monitor geopolitical developments, and determine urgency based on scenario analysis and predetermined triggers.
The key insight is that these aren’t competing approaches. They’re absolutely complementary. An analyst might start by exploring a specific concern through targeted questions, then discover related patterns through AI-surfaced trends they hadn’t considered. Or they might begin with AI-identified escalating situations and then drill down with specific queries to understand implications unique to their needs.
This creates a workflow where AI handles the cognitive burden of constant monitoring and pattern recognition, while preserving analyst agency in determining what matters for their specific context. Instead of forcing analysts to choose between overwhelming information streams or restrictive alert parameters, the system adapts to both directed inquiry and continuous awareness needs.
Building this capability requires AI grounded in analytical expertise that can distinguish between routine developments and meaningful escalations. It means understanding not just what’s happening, but what questions those developments should prompt, and which scenarios warrant immediate attention versus longer-term monitoring.
Imagining What Could Be
The technical challenge is significant but absolutely solvable. Instead of receiving seventeen separate alerts about the same developing situation, analysts should see one consolidated update that shows them how the story is evolving and why it matters now. Better yet, imagine consolidated updates centered around when a significant development happens in a story. And then being able to decide what your thresholds are—do you want to see more or fewer updates on that story? On a global scale, that story may be small but for your equities, it’s huge and you want more updates versus fewer.
These systems need to provide context about why particular trends matter and suggest related content based on human expertise. When an analyst sees that protests are spreading across three cities, the system should be able to surface whether this follows historical patterns that led to government crackdowns, economic instability, or resolution through dialogue.
Most importantly, these systems need to integrate with actual workflows rather than creating additional information streams to monitor. Analysts and intelligence consumers alike need tools that reduce cognitive load.
This means moving beyond simple alert systems toward interfaces that can help users explore developing situations, understand implications, and identify blind spots. The goal isn’t just to provide information but to support better decision-making through uncertainty.
Signal Through the Noise
The future of I&W in intelligence lies in eliminating the false choice between targeted threat detection and noise tolerance. AI should enable both precision threat detection and effective horizon scanning, giving analysts time back to think strategically while making intelligence insights accessible to everyone who needs them.
This requires AI systems built with analyst tradecraft in mind, grounded in human analytical insights, and designed for actual intelligence workflows rather than impressive demonstrations. Technology exists to solve this problem. What’s missing is the recognition that analysts can’t sustain this level of geopolitical whiplash indefinitely.
We can’t keep accepting that more alerts equal better intelligence, or that analysts can meaningfully process the constant pivots from Ukrainian drones to Iranian missiles to Caribbean drug interdictions to whatever crisis emerges next. The question isn’t whether AI can help the analyst because it can and it does, within reason. But it remains to be seen if we’re willing to admit we deserve better, as the most prevalent approach cracks under the weight of global chaos.
