Security Experts:

Collective Intelligence: Realities and Hardships of Crowdsourced Threat Intel

Enterprise security teams need to move from the consumption of crowdsourced threat intelligence (CTI) to an additional mode of contribution

Cybersecurity has an information sharing problem. We are a community with grand ideas around the concept of crowdsourced threat intelligence (CTI), but with little history or previous successes that show CTI as a viable idea. In this context, the crowdsourced data could be from the many open-source projects or the third-party vendors who provide threat intel. It is essentially the desire to aggregate valuable information sources and provide value back to those sources. The concept of crowdsourcing is not new to enterprise security teams - we have all been sharing our intel for a while, mostly in small private friendly circles through email, instant message, and other messaging platforms. 

As technology has advanced and converged to support sharing in more recent times, there has been a stronger desire to leverage these new technical capabilities for greater sharing at higher volumes and faster speeds. The number of attacks we see daily has increased, and our need to keep up has done so as well. Today, though, CTI sharing has yet to evolve into the utopia we have wanted it to be. We see the way our adversaries share information for profit, destruction, and other nefarious deeds. Their success only makes our desire to share greater because we want to defend ourselves better and together. 

Challenges of Defending Together

While the idea of CTI has the good intention of a cybersecurity community defending itself “together,” there are still numerous challenges we must overcome for it to become more real. I would sum these challenges up with the four comparisons in the graphic below:
crowdsourced threat intel

The lack of quantitative indicators is not the challenge of leveraging threat intel anymore. We are in a day and age where the ability to inundate security teams with high-volume threat intel is common. What is missing from this equation is uncovering quality or getting signals from all the noise. Changing our focus from quantity to quality is not an easy move. Moving towards quality requires a mature team with the technical ability to know how to mine data instead of generically applying indicators that come inbound. 

We also have many security teams improperly consuming indicators without context. Context refers to the valuable pieces of data surrounding an indicator. This is not something a crowdsourced solution or even a vendor can provide. Each company creates context as to what an indicator means to them. A single indicator, let alone all the aggregate indicators, can tell something entirely different to a specific business vertical, let alone a specific company. Injecting context into CTI is foundational for making it have worth in its proper use. In a CTI based model, it’s additionally essential when contributing to provide context as well. 

An additional qualifier that affects many security teams is the legal challenge behind data sharing. Not surprisingly, there might exist the capability and desire to share information and accomplish the above challenges inside an enterprise security team. However, many times there also exists a legal hurdle about the sensitivity or risk associated with doing so. This challenge can spin off into many different threads depending on the business vertical, legal team, data types, etc. Still, permission to share data for many is strictly forbidden in many companies. Legal challenges behind data sharing can be frustrating because our adversary has no such limitations, and watching them do it so successfully proves the value while leaving us wanting. 

Lastly, to do CTI correctly, enterprise security teams need to move from the consumption of CTI to an additional mode of contribution. This is the most important part of the ‘crowdsourced’ equation! Over the years of attempting to do crowdsourcing, many companies consumed while not giving back. It’s hard to build a community and protect one another if only a handful of companies contribute to the community. Contributing back typically requires tooling and scripting skills to pull data from internal systems and post to the CTI provider. People with these skills have been uncommon on enterprise security teams because it usually resides more on software engineering teams. More mature enterprise security teams are hiring for this skill set because they value the expertise required to build such pipelines. 

Look out soon for Part 2 of this column, where I’ll summarize the customer challenges inside the enterprise security world and offer suggestions on making CTI easier for everyone.
view counter
Rob Fry is Chief Technology Officer at Armorblox, an email and cloud office security company. He is an accomplished engineering and technology leader with twenty-five years of demonstrated career success with startups and Fortune 500 companies. Prior to Armorblox, Rob was CTO at JASK, where he led the company to the successful acquisition and IPO by SumoLogic. He also did stints at Netflix and Yahoo. Rob enjoys mentoring college students, working with universities on research, and guiding technology companies and startups through their growth and innovation phases as an active investor, advisor, and participant of boards.