Integrating Web Intelligence Into Cyber Ops

7/27/2019 Integrating Web Intelligence Into Cyber Ops

http://slidepdf.com/reader/full/integrating-web-intelligence-into-cyber-ops 1/8

WhitePaper:

LeveragingWebIntelligenceto

EnhanceCyberSecurity

October2013

Inside:

•NewcontextonWebIntelligence

• Theneedforexternaldatainenterprisecontext

•Makingbetteruseofwebintelligence



2

CTOlabs.com

Web Intelligence: A new category of actionable information

Web Intelligence is the parsing of millions of sources of Internet connected information in a way

that is useful to decision-making. It enables the harnessing of the global information grid and

adds predictive power to functions such as strategy development, investment decisions and risk

assessment/mitigation.

This paper, sponsored by Recorded Future, examines this new category of Web Intelligence in

a cyber defense context and provides information you can use in deciding the best ways to

integrate Web Intelligence into enterprise cyber security operations.

Our Insights into Web Intelligence:

The lead author of this paper led some of the first contributions of all source intelligence to cyber

defense in the US Department of Defense and has been an active contributor to the cyber

security and technology communities for two decades. For the last four years, the research

team at CTOlabs.com has been contributing to studies and analysis and community events on

cyber security operations, security technology and analytical tools. We interact with the

community through our blog and newsletters, including daily and weekly newsletters tracking

cyber security and analytical tools.

We leveraged our background in cyber intelligence and technology in producing this

assessment. We also checked our assumptions by asking for inputs from a range of enterprise

CISOs in the financial, manufacturing and retail sectors.

Web Intelligence and Cyber Security

Web Intelligence can significantly enhance enterprise cyber security operations. In a cyber

context, web intelligence is being used to track vulnerabilities being discussed in hacker

channels and exploited in successful attacks. Web intelligence also portrays information on the

nature of malicious code and its mitigation strategies. Further, it is a means of tracking the

technologies and tactics being employed by attackers, as well as the proven best practicesbeing applied to mitigate threats. It is in this last category of information that web intelligence is

making its most unique contributions to cyber defense. Web intelligence is bringing new insights

into the identity, motivation and intentions of threat actors, and it is doing so in ways that can

contribute to predictions of future behavior.

Since Web Intelligence can provide enhanced information on threat intentions it enables a

shifting of cyber defense to more proactive strategies. For example, information on past

behaviors of cyber actors associated with real-world events can lead to predictions on future

behaviors associated with coming events. This can lead to predictions of when to expect DDoS



3

WebIntelligenceforCyberSecurityOperations attacks or when to expect more focused phishing attacks. In some cases it can also lead to

predictions on the nature of the deceptive content that can be used in phishing attacks. With

more precise insights, action can be taken to mitigate threats before they strike.

Web intelligence also makes critically important contributions to the issue of assessing who is

attacking and why. More refined assessments on this critical element can contribute toassessments of an adversary’s next step. Web intelligence can help defenders assess whether

an attack is hactivism or something more sinister. It can also help in assessments of whether or

not others will be targets – in particular business partners such as suppliers or customers - and

if a more collective defense will need to be mounted.

Web Intelligence from Recorded Future

Recorded Future is a web intelligence company. Their mission is to harness open web sources

that publish open information on the web for analysis. They create insight in support of

government missions and business decisions.

Recorded Future and their Temporal Analytics™ Engine organize web information for analysis

to yield new insights. Recorded Future specializes in analyzing human writing to detect events,

actions and descriptions of actions and then place this information in a time-based (temporal)

context. These timelines and topics can be aggregated and correlated to ensure information onthe same event can be viewed by multiple angles. This enables analysis in the light of all

related information, including historical information.

Recorded Future ingests, in real time, over 300,000 real time sources, performing over 50

extractions per second and building a deep history at the same time. They have already

amassed a fact based of over 5 billion facts in multiple languages including English, Chinese,

Russian, Arabic, Farsi, Spanish, and French.



4

CTOlabs.com

Background: The Roots of Web Intelligence

The origins of web intelligence for cyber security can be traced to the beginning of organized

enterprise cyber security activities that began after the famous Morris Worm of November 1988.

In the worm’s aftermath, responders noted shortcomings in their ability to know information from

outside their organizations. Since then:

- Most major organizations have established dedicated efforts to stay informed on

external threats.

- There has been an explosion in original content publicly available on the web,

including blogging, niche publications, social media, but also vast stores of

commercial data that were once locked away and inaccessible to others.

- Increasingly, both threat actors and defenders are openly sharing valuable

information on open source web channels, making totally new sources of information

available

The Use of Web Intelligence For Cyber Security Today

CISOs who leverage Web Intelligence for Cyber Defense are finding far more utility technical

feeds of vulnerabilities and attack signatures. Advanced streams of information on adversaries

and their intentions, correlated and assessed, can now be provided in a context ready for use by

enterprise cyber security teams

Most CISOs we spoke with are in the process of enhancing their ability to use web intelligence,

and we believe this will be a high growth segment of the security technology portfolio in all major

enterprises.

Web Intelligence can contribute to dedicated cyber security efforts by parsing and correlating

millions of data sources

relevant to computer security.

Succinct articulations of threat

actors, their capability, history

and intentions can be

presented along withdynamically updated

information on vulnerabilities

and methods required to

mitigate vulnerabilities. This can

all be presented in conjunction

with dynamically updated

information on international and



5

WebIntelligenceforCyberSecurityOperations regional events that may trigger cyber security events. This automated extraction and

presentation of knowledge is already contributing to the situational awareness of several global

industries and is now available for general use by cyber defenders everywhere.

Web Intelligence and Enterprise Security Management Suites

Recorded Future provides a means of interacting directly with data and analysis on global

events, including cyber security focused information. However, the capability can be even more

impactful when considered in the light of existing enterprise capabilities. We believe most

enterprises will want to find the optimal connection between their existing security information

management systems and Recorded Future. Fortunately, modern security solutions provide

data integration APIs to get data in and out. The following provides some context on how

Recorded Future fits in the context of major security suites:

Tool Capability Web Intelligence Integration

HP-ArcSight Focused on logs and events butconnectors to Autonomy andHadoop show potential for futureall source capabilities

Information from Recorded Future can be easilymoved to ArcSight and feeds from ArcSight can bemoved back. This later path is being used byenterprises to establish an “analytical SIEM” that isstrong at correlating SIEM incidents with other threatfeeds (including Malware IPs, Vulnerabilities, threatintel etc). This can help rapidly prioritize eventresponse.

McAfee ESM DPI and log data. Databasemonitors. No all sourcecapabilities.

McAfee has always stressed interoperability in their solutions and the ESM architecture allows easyimport and export of data. However, we have noexamples of the use of ESM as an analytical SIEM

or in support of one.

Splunk New release provides speed andscale and ability to add externalthreat feeds, showing potentialfor integrating Web Intelligence.Dashboarding capabilitiesimportant.

Splunk has had strong import and export capabilitiesin place since their first offering, and these can beautomated as desired. The dashboardingcapabilities of Splunk can be used as all sourcedisplays of information, potentially includinginteractive connections to Recorded Future.

RSA

NetWitness

Leveraged for log based andnetwork data analysis

The powerful tools for analysis of ongoing and pastevents leverage very large datastores and aredesigned to provide analysts easy ways to exportdata and analysis. This enables the use of data fromNetWitness to power “analytical SIEM” applications.This can be a powerful contribution to forensic

analysis.IBM-Q1

QRadar

Log and event management withbehavior analysis. Netflow dataa strength.

We are not aware of a smooth way to moveinformation out of the Q1 Radar architecture,however exports based on user-selected criteria canbe done. No indications of all source capabilities infuture roadmap.

Sensage A purpose-built “big data” SIEMtool. Ability to take data feedsand integrate other informationshows promise.

Unique clustered columnar database is not designedfor use by other systems, but exports of selectedinformation can be made.



6

CTOlabs.com

Most enterprises are also leveraging link analysis and related investigative tools, including IBM’s

Analyst Notebook (which is ubiquitous), and the rapidly proliferating Maltego. Some use the

advanced capabilities of Palantir. Users of current versions of these systems can rapidly and

easily move information to and from advanced web intelligence platforms like Recorded Future.

A User Look at Web Intelligence

Web Intelligence feeds can be presented in interactive interfaces that enable rapid assessment

of dynamic information. Interfaces of Recorded Future offer analysts a means interacting with

data and forming hypotheses and conclusions quickly. Analysts are presented with polished and

sophisticated ways to interact with large stores of correlated and assessed information.

Recorded Future also enables direct access to specialized modeling and visualization of events

in time and over geography, while still enabling drill-down into sources of any data.

The Cyber Intelligence Application on the Recorded Future Enterprise Platform is delivered via

software as a service. This simple account-based access to the platform gives access to the full

power of Recorded Future’s understanding of Internet connected information



7

WebIntelligenceforCyberSecurityOperations

Optimizing the use of Recorded Future for Web Intelligence

in Support of Cyber Operations

The new field of Web Intelligence is already providing actionable information relevant to cyber security professionals. Recorded Future provides the only automated solution in this space that

is capable of ingesting, in real time, the right security related information from the Internet. Their

fast and valuable information feeds fill a gap.

Our recommendations:

1. Establish your enterprise vision for the use of Web Intelligence in support of your

security posture.

2. Launch a proof of concept leveraging Recorded Future’s Software as a Service cyber

intelligence application. This application enables rapid delivery of capability that can put

Web Intelligence to use in your enterprise almost instantly. During the proof of concept

formulate evaluations on criteria like:

a. Ability to meet your vision for web intelligence support to cyber operations

b. Ability to leverage the full spectrum of intelligence information from the Internet

and your internal sources

c. Ability to enable shared situational awareness across all levels of your

organization

d. Ability to drive proactive mitigation of threats.



More Reading

For more federal technology and policy issues visit:

• CTOvision.com- A blog for enterprise technologists with a special focus on Big Data.

• CTOlabs.com - A reference for research and reporting on all IT issues.

• FedCyber.com – Focused on federal cyber security

• J.mp/ctonews - Sign up for technology newsletters including the Security Technology Weekly.

About the Author Bob Gourley has been active in the cyber defense community since 1998, specializing in intelligence

support to cyber operations. He is CTO and founder of Crucial Point LLC and editor and chief of

CTOvision.com He is a former federal CTO. His career included service in operational intelligence centers

around the globe where his focus was operational all source intelligence analysis. He was the first

director of intelligence at DoD’s Joint Task Force for Computer Network Defense, served as director of

technology for a division of Northrop Grumman and spent three years as the CTO of the Defense

Intelligence Agency. Bob serves on numerous government and industry advisory boards. Contact Bob [email protected]

For More Information

If you have questions or would like to discuss this report, please contact me. As an advocate for better IT use

in enterprises I am committed to keeping this dialogue up open on technologies, processes and best practices

that will keep us all continually improving our capabilities and ability to support organizational missions.

CTOlabs.com

Documents

Integrating Web Intelligence Into Cyber Ops