This Conference brought to you bywww.ttcus.com

Linkedin/Group:Technology Training Corporation

Technology Training Corporation

@Techtrain

Corporationwww.ttcus.com

U S A I t lli d S it C d

UNCLASSIFIEDUNCLASSIFIED

U.S. Army Intelligence and Security Command

Army Intelligence and “Big Data” 29 S t 2015

INSCOM … the Army’s Force for Dominant Intelligence

29 Sept 2015

UNCLASSIFIEDUNCLASSIFIED

Why? Big DataUNCLASSIFIED

All Sensors need Processing, Exploitation and Dissemination (PED)

UNCLASSIFIED3

Why? New Army Operating ConceptUNCLASSIFIED

4UNCLASSIFIED

Unified Cloud Data (UCD): A Joint Interagency Effort Aimed at Defining the

Unified Cloud Data (UCD)UNCLASSIFIED

• Unified Cloud Data (UCD): A Joint, Interagency Effort – Aimed at Defining the Service Technical Approach to the ICITE Framework and Data … While Solving the Global Processing, Exploitation & Dissemination (PED) Migration Challenges

• UCD: Our “Big Data” Reference Implementation of a Unified Data Layer• All Data Work Regardless of Type of Intelligence and Security Level• All Data Work, Regardless of Type of Intelligence and Security Level• All Analytics + Enrichment Processes Run against UCD: Write Once, Use Often• All Indexes Make UCD Data Discoverable to Analysts• Security Is “Baked In” – Security Markings Are Integrated at the Source Level,

Event / Document / Entity level, Down to Individual Attributes for an Entity

• UCD Pilot – Army’s Instantiation, Led by INSCOM, Assessed by Soldiers in Live Environment – To Inform Army Programs of Record• Army Mission Command and Army Cyber Pilots show Benefit of Unified Data

for Operations-Intel Convergence for the Army Operational Concept (AOC)

5UNCLASSIFIED

Extending Cloud Advanced AnalyticsExtending Cloud Advanced AnalyticsUNCLASSIFIED

6UNCLASSIFIED

Open Source Information Alerts AnalystsUNCLASSIFIED

• Use Open Information – Pilot New Analyticsp y

• Train Analysts on Sources and Capabilities• Sources Change; Terms Evolve• Stay Engaged: Frequent Changes • Share Information, including Concepts of Operation

P t i t C t t f Wh t’ K• Put into Context of What’s Known • Mature Techniques to Verify and Understand:

Who, What, Why??

• Certify for Policy Compliance• Certify for Policy Compliance• Reinforce Mission and Legal Authorities• Do the Right Analysis, the Right Way

• Leverage the Enterprise:A l ti C biliti• Analytic Capabilities

• Data Approach: Pay for Data Once, Use Many Ways• Enterprise Impact: Data Retention, Storage,

Correlation, Cyber Security…

Big Data: U f l f Ti i C i

7UNCLASSIFIED

Useful for Tipping + Cueing, But Has Risks

The Enterprise is the FoundationUNCLASSIFIED

Foundation Layer BackboneLayer Backbone

8UNCLASSIFIED

Unified Cloud Data (UCD) – Partners & PilotsUNCLASSIFIED

9UNCLASSIFIED

Unified Cloud Data (UCD) – Partners & PilotsUNCLASSIFIED

• Demonstrated Value:

ldi d i+ Soldiers used UCD + Live Data+ Saved Mission Command  7+ months to pilot Ops/Intel Convergence, DTRA to pilot g , pConstellation on UCD baseline

+ Remote Mgt + Puppet deployment lets 1 SysAdminmanage multiple UCD sites

10UNCLASSIFIED

manage multiple UCD sites+ Piloted AWS GovCloud for Integration: Saved $

Implementing Unified Cloud Data (UCD)Implementing Unified Cloud Data (UCD)UNCLASSIFIED

11UNCLASSIFIED

Assessed UCD + Value of ‘Big Data’ Analytics to Inform Future Requirements

Good Feedback from Functional AssessmentUNCLASSIFIED

Good Initial Feedback: Users Want More Access + More Data

Assessed UCD + Value of Big Data Analytics to Inform Future Requirements

Improved Analyst Usability:• Ease of Use: Easy Multi-INT exploitation of Unified Data with Widgets + Workflows

• Rapid Mastery: After 3 days training, Soldiers could use UCD for mission threads

• Speed of Analysis: Soldiers used UCD to do Country Study in 30 min (1/3 time)

• Fast Data Access: Facial Recognition in seconds against 100Ks of records

Operations-Intelligence Convergence:• Improved Situational Awareness: Blue Force + Red Data in Common Operating Picture

• Pre-Deployment Checks: Soldiers easily checked New Area: Know What’s Known

Enterprise Efficiencies and Security:• Built-In Support: Self-Configurable Dashboards + Workflows Soldiers can share

• Info Sharing: Built-in Reports/Report Creation (no support needed), Coalition Info Sharing

• Enterprise Operations: Remote Admin by Fewer System Admins support multiple sites

• Improved Security: Cell-level Security, Thin Client/PKI, Separation of Roles

12UNCLASSIFIED

UCD Support of Intel FunctionsUCD Support of Intel Functions

Separate Data from Analytics

UNCLASSIFIED

Sensors

Separate Data from Analytics

Security:P

UCD handles

Data Access:User

Authorizations

Sensors • Provenance• Security Labels • Metadata Tagging• Extract Entities + Geo/Temporal Attributes• Metrics• more

many kinds of data

Data Access:Match User

Roles/Authorizations against Data Security

Data Ingestion

Data from Many Sources/Types

• Images• Audio• Video

Velocity +

Content

Authorizations

Real Time

Community Partners• Video

• Messages• Public Info• Mission

Command• Etc. Analytics

Update

Real-TimeCell-Level Security

Analyst’s Conclusions

Indexes Enrich Data

• Correlate All Data• Context-Based Data Navigation

• Analysts Enrich Correlated Data:+ Know What’s Known Now+ Helps Analysts “Connect the Dots”

• Supports ‘Big Data’ Analytics

13

• Map Reduce Analytics Enable Data Sharing

Context Based Data Navigation+ Beneficial to All Domains:+ Operations-Intel Convergence+ Medical Support+ Logistics Support

• Supports ‘Big Data’ Analytics and Multi-Discipline Fusion for AOC environments: A2AD, Megacities, Ad Hoc Response…

UNCLASSIFIED

UCD Support of Intel FunctionsUCD Support of Intel FunctionsUNCLASSIFIED

UCD handles Separating Data from

Apps lets Analytics U S D t

many kinds of data

Use Same Data: Read Once, See Data

Many Ways

UNCLASSIFIED

UCD Support of Intel FunctionsUCD Support of Intel FunctionsUNCLASSIFIED

UCD handles Separating Data from

Apps lets Analytics U S D t

Counter-Insider Threat

Security checks

many kinds of data

Use Same Data: Read Once, See Data

Many Ways

Security checks

UNCLASSIFIED

UCD Support of Intel FunctionsUCD Support of Intel FunctionsUNCLASSIFIED

UCD handles Separating Data from

Apps lets Analytics U S D t

Counter-Insider Threat

Security checks

many kinds of data

Use Same Data: Read Once, See Data

Many Ways

Security checks

UCD implements full CRUD functionality: Analysts can

Create, Read, Update, Delete

UNCLASSIFIED

UCD Support of Intel FunctionsUCD Support of Intel FunctionsUNCLASSIFIED

UCD handles Separating Data from

Apps lets Analytics U S D t

Counter-Insider Threat

Security checks

many kinds of data

Use Same Data: Read Once, See Data

Many Ways

Security checks

UCD implements full CRUD functionality: Analysts can

Create, Read, Update, Delete

UCD Unifies Data

UNCLASSIFIED

UCD Support of Intel FunctionsUCD Support of Intel FunctionsUNCLASSIFIED

UCD handles Separating Data from

Apps lets Analytics U S D t

Counter-Insider Threat

Security checks

many kinds of data

Use Same Data: Read Once, See Data

Many Ways

Security checks

UCD implements full CRUD functionality: Analysts can

Create, Read, Update, Delete

UCD Unifies Data

Fine-Grained Data Security Markings are stored in Accumulo

UNCLASSIFIED

UCD Support of Intel FunctionsUCD Support of Intel FunctionsUNCLASSIFIED

UCD handles Separating Data from

Apps lets Analytics U S D t

Counter-Insider Threat

Security checks

many kinds of data

Use Same Data: Read Once, See Data

Many Ways

Security checks

UCD implements full CRUD functionality: Analysts can

Create, Read, Update, Delete

UCD Unifies Data

Fi G i d D tFine-Grained Data Security Markings are stored in Accumulo

UNCLASSIFIED

Fast GPU-based Geospatialand Temporal indexing

UCD Support of Intel FunctionsUCD Support of Intel FunctionsUNCLASSIFIED

UCD handles Separating Data from

Apps lets Analytics U S D t

Counter-Insider Threat

Security checks

many kinds of data

Use Same Data: Read Once, See Data

Many Ways

Security checks

UCD implements full CRUD functionality: Analysts can

Create, Read, Update, Delete

UCD Unifies Data

Fi G i d DIntegrated Capabilities

Fine-Grained Data Security Markings are stored in Accumulo

g p

such as Facial Recognition

UNCLASSIFIED

Fast GPU-based Geospatialand Temporal indexing

UCD Support of Intel FunctionsUCD Support of Intel FunctionsUNCLASSIFIED

UCD handles Separating Data from

Apps lets Analytics U S D t

Counter-Insider Threat

Security checks

many kinds of data

Use Same Data: Read Once, See Data

Many Ways

Security checks

UCD implements full CRUD functionality: Analysts can

Create, Read, Update, Delete

UCD Unifies Data

Integrated Capabilities Fine-Grained Data

Security Markings are stored in Accumulo

g p

such as Facial Recognition

Can Make Workflows –

UNCLASSIFIED

Fast GPU-based Geospatialand Temporal indexing

Can Make Workflows for Analyst Tasks + for

MapReduce Jobs

Conform to the Enterprise

Constraints We Must HandleConstraints We Must HandleReduce Costs

UNCLASSIFIED

Conform to the Enterprise Reduce Costs• Software Licenses• Physical Footprint• IT Supportpp• Leverage Open Source / Other

Software

Increase SecurityIncrease Security• Provenance: Track Every Interaction• PKI and Thin Client• Bastion Node Construct

Expect Change• New Kinds of Data: OSINT/Social Media

• Remote Administration: Separate System Administration from the Data

Exploit Exponential • New Kinds of Data: OSINT/Social Media• New User Needs: Heat Map• New Capabilities from Partners: WAMI

Track Extraction from NGA

Increase in Data

22

• New Technology Components: GPU Processors for Geospatial Data

UNCLASSIFIED

Protect the Data:

UCD Lessons LearnedUCD Lessons LearnedUNCLASSIFIED

Protect the Data: Build on a secure Cloud architecture with cell/object-level security, and extend security down to the weapon system to be able to Counter Insider Threat (with PKI, provenance etc.).

"Big Data” Strategy with Unified Data: Capture and triage vast, increasing amounts of data of all types, from many sources, with automated "Big Data" p g , g yp , y , ganalytics. Analytics from different providers should leverage the same correlated data: write once, use many ways.

Enterprise-Scale Remote Administration: Leverage automated build, delivery, install, configuration management, system administration, and monitoring to make operational use simple and scalable to the Enterprise level: automate routine tasks so humans focus on problems.

Leverage Prior Capabilities with Enterprise Platform: Break apart legacy stovepipe capabilities from battlefield-proven Quick Reaction Capabilities (QRCs); re-host unique components on IC ITE security architecture to ensure needed capabilities endure for the future.

Transform Capabilities Acquisition: To benefit from the innovation & speed of new capabilities, at long-term greatly reduced cost, incentivize organizations and individuals to use Open Source software, Agile business models of vendors & gov’t organizations.

Encourage and Reward Partnering: Strong INSCOM + Mission Command partnership integrated INSCOM UCD software onto MC tactical cloud servers: at NIE 15 1 showed Intel/Ops Convergence situational awareness DIL operations Bde TOC server consolidationNIE 15.1, showed Intel/Ops Convergence, situational awareness, DIL operations, Bde TOC server consolidation …

Future Ready for Capabilities Integration: Future analytic capabilities that exploit Unified Data can be rapidly integrated and (done right) can inherit security.

Need New Categories of Innovative Capability: Cyber Security and Info Assurance require new Enterprise analytics, to understand threats, attacks, system health –but we also need Innovative Capabilities + Processes to demonstrate Info Assurance (IA) and Capabilities Security.

23UNCLASSIFIED

U S A I t lli d S it C d

UNCLASSIFIEDUNCLASSIFIED

U.S. Army Intelligence and Security Command

Questions

INSCOM … the Army’s Force for Dominant Intelligence UNCLASSIFIEDUNCLASSIFIED 24