Security Experts:

The Three Pillars of Hypothesis-Driven Threat Defense

 There are two possible outcomes: if the result confirms the hypothesis, then you've made a measurement. If the result is contrary to the hypothesis, then you've made a discovery. - Enrico Fermi

Threat hunting is the act of proactively identifying and counteracting an adversary that has gained a foothold in an organizations environment. This contrasts with Threat Detection, that focuses on identifying a threat by approaches such as signature-based detection or the correlation of system events to identify indicators of compromise (IOCs). A threat in this context is an adversary that has the intent, capability and opportunity to do harm.  Threat hunting is a complimentary process to threat detection and is based on the recognition that threat detection has limitations and relies on four prerequisites (or hypotheses):

· IoC’s are predictable

· IoC’s are logical and quantifiable

· IoC’s are static and not permutable

· IoC’s are visible and discoverable

These four prerequisites are not always met. and in some cases, none of them are. Even if the first three are fulfilled, the greatest challenge is that the last one often is not. Monitoring every possible system, network or user can be challenging. More importantly, it can be expensive in time and monetary terms. The resulting volume of alerts, events and false positives creates its own set of problems. Even if threat detection is successful, the associated alert may be missed or only noticed a long time after the act. Statistics of threat actor dwell times bear this out again and again.


If you are inclined towards Zen, you can ask “If there is an IoC is on a network and no one is monitoring, is it still a threat?”. Judging by the frequency of high profile breaches, the answer is a resounding Yes.

Threat actors are also aware of these factors and change their tactics, techniques and procedures accordingly to eliminate as many of the prerequisites as possible. This is the basis of natural selection in cyber security, and the root cause of the ongoing arms race between hackers and cyber security professionals.

Threat Hunting seeks to redress the weaknesses inherent in threat detection. Obviously if threat actors are already present in an internal network, threat detection has already failed or at least was not responded to until well after the initial exploitation. If it is the former, the only way to identify if you have been breached is by changing from waiting for your detection technology to point out a threat to manually investigating whether there are indicators that the detection technology has missed. If it is the latter, threat hunting will focus on assessing the scope and spread of a compromise with the aim of rooting out any foothold that an attacker has gained.

3 Styles of Threat Hunting

There are three primary styles of Threat Hunting:

IOC-driven Threat HuntingKnown indicators of compromise have been detected and are used to identify related IoC’s and TTP’s to permit the search and analysis of their presence in the environment.

Analytics-driven Threat HuntingAdvanced analytics, machine learning and behavioural analysis technologies have identified anomalous or suspicious activity that warrants further investigation. To an extent, behavioural analysis technologies have already automated some aspects of traditional threat hunting, but have inherited similar limitations as threat detection, and also created some of their own.

Hypothesis-driven Threat HuntingAn initial hypothesis or assumption that a specific threat or threat actor may have successfully compromised the environment. Related TTP’s and IoC’s that are associated with the threat can be extrapolated to search for these and to analyse them.

An astute observer will notice that the first two styles rely on the successful detection and discovery of at least one IoC or observable. As such, these are primarily used to verify if a breach has occurred, and to assess the scope and spread of a threat.

Hypothesising Threats

Hypothesis-driven threat hunting does not rely on a prior detection. It will assume that a threat or threat actor was not detected, but may have targeted an organization anyway. What is less obvious is the relation between hypothesis-driven threat hunting, threat assessments and threat simulations.

Threat AssessmentsIn a threat assessment, potential and relevant threats and threat actors that may target the organization are identified, often by utilizing threat intelligence. These can then be included in the risk management process to determine what security precautions must be deployed to safeguard against potential threats.

Threat SimulationsIn a threat simulation, the TTP’s of threats or threat actors are correlated against existing security technologies, people and processes to assess how these would hold up in the case of an attack. Even better is when they are practically tested. A true penetration test or ethical hack, or what mature organizations call red-teaming, is an example of this approach.

Hypothesis-driven Threat Defense

One definition of Hypothesis is “a supposition or proposed explanation made based on limited evidence as a starting point for further investigation.” Limited evidence for cyber security professionals is the key word here. Threat Detection technologies provide limited evidence – they may discover some of the IoC’s, but may not provide a clear assessment of how advanced or widespread a compromise is. They may not successfully detect a threat at all. Threat Intelligence provides a lot of evidence about what is out there in theory, but is limited in determining who will actually target you. Prevention technology prevents against many types of known attacks, but we do not have sufficient evidence that they will prevent the next attack to face us.

Hypothesis-driven threat defense integrates these assumptions into a single framework and is used as the basis for a risk management program. Threat assessments, threat simulations and hypothesis-driven threat hunting are the three pillars of hypothesis-driven security and the triumvirate of successful threat mitigation. Taken together they consider who may target you, whether they successfully could, and whether they already have.

To succeed in an era of rapidly evolving threats and emerging technology and the resulting shortfall of actionable evidence and lack of certainty, hypothesizing is the closest we can get to prediction.

view counter
Oliver Rochford is the Vice President of Security Evangelism at DFLabs. Oliver is a recognized expert on threat and vulnerability management as well as cyber security monitoring and operations management. He previously worked as research director at Gartner. He has worked as a security practitioner and white hat hacker for Tenable Network Security®, HP Enterprise Security Services, Verizon Business, Secunia® (now Flexera Software), Qualys®, and Integralis (now part of NTT Com Security).