Threat Intelligence Comes of Age

Several moves to update and modernize threat intelligence tactics have popped into the market this year, aiming to make it easier for companies to understand what they’re up against when it comes to hackers, the Dark Web, fraud and other nefarious cyber-challenges.

Threat intelligence security refers to an organization’s ability to analyze and understand information related to all types of cyberattacks. The push to implement this has grown significantly amidst a global shift in organized crime. Similar to the early 20th century mob rings, modern day cybercriminals are rapidly banding together in efficient and complex networks to launch more sophisticated, higher-ROI attacks, while data breaches have become almost ubiquitous.

A Growing Arena

Given that it takes enterprises almost half a year on average to uncover a new threat (and once discovered, it can take another week’s time to investigate and resolve it), the market for threat intelligence security is booming, according to Gartner (News - Alert) Research. As a result, by 2019, 60 percent of digital business infrastructure will include reliance on threat intelligence feeds as a functional requirement to ensure operational resiliency.

MarketsandMarkets echoes the trend, and forecasts that the global threat intelligence security market will grow from $300.3 billion in 2015 to $586.1 billion by 2020, at a compound annual growth rate (CAGR) of 14.3 percent during the forecast period. In the current scenario, North America is expected to be the largest market on the basis of spending and adoption of the threat intelligence security market. One of the major factors driving this is the rising number of sophisticated cyberattacks, which have increased 48 percent during the last five years.

According to Enterprise Strategy Group’s (News - Alert) (ESG) Threat Intelligence Survey, nearly three-quarters (72 percent) of participants responded that spending on their organization’s threat intelligence program will increase significantly or somewhat in the next 12 to 18 months.

Participants also responded that 72 percent of their organizations plan to collect and analyze significantly or somewhat more internal threat intelligence over the next 12 to 24 months—while 55 percent of their organizations plan to collect and analyze significantly or somewhat more external threat intelligence in that time.

Better Information Feeds

While threat intelligence and information-sharing on cybersecurity is an increasing focus for the security community, vertical markets, SMBs and the federal government, there is much debate about how to best architect the systems for doing so. The traditional way to gather and share threat intelligence has been a convoluted, non-automated collation of disparate information sources—and this remains a major hurdle.

It has meant that even when threat intelligence technology is in place, companies can still be at risk because these solutions often operate in silos, incorrectly flagging a benign event as a threat through a lack of contextualization, which can lead to unacceptable levels of false alarms. Automated anomaly threat detection technologies that don’t address the need for specialist analyst intervention can simply add further to the flood of information to be investigated.

The ESG survey in fact found that the top challenges for implementing information-sharing include: threat intelligence collected and analyzed by different individuals/tools, making it difficult to get a holistic picture of internal and external threats (32 percent); organizations inadvertently blocking legitimate traffic as a result of a problem with threat intelligence collection/analysis (32 percent); threat intelligence collection and analysis workflow process and integration problems (31 percent); and threat intelligence that isn’t always as timely or actionable as respondents need it to be (28 percent).

As a result, while IT personnel overwhelmingly (94 percent) believe that it’s highly or somewhat valuable to share threat intelligence, only 37 percent of ESG respondents’ organizations regularly share internally driven threat intelligence with other organizations or industry Information Sharing and Analysis Centers (ISACs).

“There is clearly an understood value in leveraging threat data, but organizations are finding it difficult to collect, analyze and pinpoint critical threats,” said Jon Oltsik, ESG senior principal analyst. “According to our research, automation is needed for organizations to wade through the mass of alerts they receive, and standards are needed for the secure sharing of threat intelligence.”

New Directions

Several security companies are moving to eliminate the issues around effective threat intelligence and information-sharing.

For instance, Terbium Labs, a security software company that proactively searches for stolen data on the Dark Web using a patented, privacy-protected, data fingerprinting technique, is using the MapR Distribution as the big data platform for Matchlight, a new data intelligence system.

Matchlight continuously crawls the Internet, including the Dark Web, where veiled criminal activity often takes place. The average data breach takes more than 200 days to discover, giving adversaries months or even years to exploit a security incident. With Matchlight, identification of stolen data takes just minutes.

Relying on the MapR Distribution, the new Matchlight system registers digital fingerprints of data, which range from valuable source code to corporate documents, and searches for stolen data by comparing them to data gathered across the Internet. There are currently more than 350 billion data fingerprints in its database, which continues to grow by ten to fifteen billion every day. Operating on all types of digital assets, the Matchlight system is able to discover unexpected appearances of sensitive information, alerting companies immediately and automatically to potential data breaches. For example, the system was able to identify as many as 30,000 newly stolen credit cards and 6,000 newly compromised email addresses for sale on the Dark Web in a single day.

“We want to shut down the market for stolen data by reducing the time to detect a breach and thereby minimize the damage,” said Danny Rogers, CEO and Co-founder of Terbium Labs. 

SolarWinds (News - Alert) is working on automation as well. For instance, it recently added a threat intelligence feed to SolarWinds Log & Event Manager, security information and event management (SIEM) product designed for resource-constrained IT organizations. It offers perks like automatically tagging events to ensure that suspicious activity can be identified by simply running a report or search. And, by analyzing and comparing activity against a list of known malicious threats compiled by third-party security research teams, IT security pros can identify known, proven threats and limit the impact of cyberattacks.

“In a new security reality where most security IT pros have to assume the worst—a breach has already occurred – it is imperative to have constant visibility into known threats in order to quickly detect security issues and limit the loss associated with a data breach,” said Nikki Jennings, group vice president, product strategy at SolarWinds. “With added threat intelligence, SolarWinds Log & Event Manager now enables IT security pros to take immediate action if a threat is detected and proactively monitor for additional vulnerabilities in their environment.”

Social Approaches

Crowdsourcing is a new idea in threat intelligence as well, with both AlienVault and IBM (News - Alert) embracing social approaches to the task.

AlienVault’s Open Threat Exchange (OTX) 2.0 acts as a platform for collaborative cyber-defense. It’s a crowd-sourced threat intelligence-sharing system, with members so far contributing 1 million threat indicators per day. This iteration represents a big shift from the traditional contribution-based model for sharing threat intelligence, and the company said that it hopes that building OTX 2.0 on a foundation of a social networking architecture will allow the OTX community—which has 26,000 participants—to actively discuss, explore, validate and share the latest threat data, trends, techniques and research.

Users can import and export indicators of compromise for security tools via the open API, as well as collaborate with researchers and other members of the security community. Users can also create or subscribe to an existing “Pulse (News - Alert),” an analysis of a particular threat that provides a summary of the impact, as well as get a view into the software targeted and related indicators of compromise used to detect threats.

“When we first released Open Threat Exchange, our goal was to deliver an open threat intelligence-sharing network that put effective security measures within the reach of all organizations,” said Barmak Meftah, president and CEO of AlienVault. “As others in the industry have developed threat intelligence offerings that support that vision, our goal for OTX 2.0 is to move the needle on how threat intelligence data is shared, making it more collaborative and engaging in order to build a stronger security community working in unison to stop cyberattacks.”

Not to be outdone, IBM earlier this year made a move to open up more than two decades worth of cyber-threat intelligence via a new data-sharing exchange that is modeled off of social networking.

The IBM X-Force Exchange provides additional context on an indicator that has been brought to a user’s attention, whether from a security tool or another user, helps the user make a decision on how to further use that information. Extending this to action naturally leads to programmatic access and application programming interface integration, which helps organizations make better and quicker decisions.

“You can think of the X-Force Exchange as a Pinterest for security analysts, allowing them to build collections of data and engage with others,” a spokesperson noted. “Currently, security analysts often use Word documents or spreadsheets to do this type of work. IBM is bringing them a digital platform for better organizing intelligence.”

The most common indicators include IP addresses, domain names, URLs, registry settings, email addresses, HTTP user agents, file hashes and file names, IBM noted. There is depth of information associated with each of these, such as the historical context, as well as the pivoting between them to allow for the real understanding of how they relate to each other in order to gain insights on tactics and techniques.

All of this is moving threat intelligence into a central arena for organizations, and it should be a big area of investment in security going forward.

“The idea around sharing threat intelligence among organizations is rapidly gaining traction,” said Anne Bonaparte, CEO of Vorstack, which sponsored the ESG survey. “To achieve this, organizations need a holistic picture of internal and external threats for the enterprise, and the ability to share threat intelligence among organizations in a manner that is secure, anonymous, non-attributed and standards-based.”