US Army: poindexter-information-technologies-national

8/14/2019 US Army: poindexter-information-technologies-national

http://slidepdf.com/reader/full/us-army-poindexter-information-technologies-national 1/20

Information, Technologies &

National Securityfor the 21st Century

Neal Pollard LectureGeorgetown University14 November 2007

Dr. John M. Poindexter

[email protected]



Agenda

• National Security Process

– A systems view

– Homeland Security is sub-set of National Security – Counter Terrorism & Proliferation of WMD

• Need for Technology to Help

– Problems of scale, complexity (bordering on chaos),multivariate

• Memory Bases and Data Bases

– Human Analogy• Nature of Intelligence

• Discussion

Technology needed to manage data explosion…



Problem: Drowning in Ocean of Data

(just before he drowns)



Biggest Problem

• One can never have too much data.

• But the question is: Do you have the right tools?

• To find the relevant data in the context of interest.

• To quickly focus on this data with laser-like

precision.• Without being swamped by the tidal wave.

• Before proceeding let’s look at the NationalSecurity process

How to make sense and understand what the data means…



Context: National Security Process

Data −> Information => Knowledge π> Options ε> Action

Where operators (functions) are:

−> Analysis = selects data in context to produce information

=> SenseMaking = understanding what the information means

π> PathFinding = deciding what to do about it in policy contextε> Execution = “operational forces” carry out decisions taken

Iteration = many steps are often repeated

• Simplified but basic non-linear process that is essential to understand.• Analysis is an over-used term.• This provides a working definition of Sensemaking and Pathfinding.

• Process carried out in a collaborative environment with relevant agencies.• Great deal of confusion amongst the terms data, information and knowledge.• “Operational Forces” – military, diplomatic, economic, law enforcement, etc.• This process is applicable in many other domains.

• Business, Financial, Medicine, Social Decisions to name a few.

Involves All of National Security Community Not Just Intelligence…



Limitations of Data Bases & Searching

• Relational Data Bases – Primarily designed to answer static queries from structured data – in records and fields – Typical in the accounting, logistics, personnel areas – Tables are designed to answer the queries promptly – They take a big performance hit when fields relevant to a new query are in different tables – Table joining takes a lot of time – Unstructured text presents complications

• All words are not indexed – just those deemed significant by somebody in some general context

– Great way to store information, but not a good way to find relevant information in response to changing queries

• Traditional Search – Google is good example, but there are many others – When you submit a Google query it goes against a prepared linear index

– Lists of documents are returned – not entities (people, places, and things) – Again not all words in the documents are indexed – Difficult to focus in on relevant documents when context is complicated – Following one dimensional links in documents is a laborious process

• Situation – Enormous amounts of data being collected and stored in relational data bases

– But can’t find data of interest or it takes too long – Hard to find patterns – Dr. Tom Fingar, DDNI(Analysis), has said:

“It isn’t intelligence until it has been processed through the brain of an analyst. It’s

just data. And we are awash in data.

We don’t have enough analytical brains to meet all of the challenges. We have to

rely on technology.”

Traditional ways of finding information in data…



Mammal Analogy

• Early mammals started sniffing around in the dark, so they neededa memory-map* based on olfaction to remember where things wereand predict where they might be.

• We have surely advanced beyond only our olfaction sensors, butthe process is instructional.

• Need memory-map of patterns (associations) and means for

exploring this map through: – Navigating (including precise fixes as well as dead-reckoning)

– Hunting (when looking for a specific target)

– Foraging (when targets are sparse and hard to find)

– Browsing (when high concentration of relevant data)• Scale of today’s problem is enormous.

– Most of scale is noise – uninteresting data

We should learn from our past about finding things…

*For macro theory on the way human brain works:On Intelligence, Jeff Hawkins with Sandra Blakeslee



Solution: Associative Memory

• Humans remember and predict things through patterns of association.• Associative Memory problems in the past have been of scale & performance

– Must manage large N x N matrices for anything but toy problems. – Graffiti uses something on the order of N = 100 (Jeff Hawkins story).

– Can now manage on the order of N = 10 million. – And have on the order of 10 million entities with 107 x 107 memories. – Near instantaneous response to queries independent of memory base size

• Because N can now be so large, do not have to resort to dimensionality reductionfor complex problems. – Remember all associations from every observation.

– Do not throw away any information. – The more information the better. – However typical problem in this area is large number of entities with sparse data on

each buried in lots of noise which kills conventional approaches.

– Associative memory approach squeezes out every last ounce of information to provideintelligence.

• Have achieved very high ingestion rates.

• The memory base technology is a truly disruptive technology

Note: I’m a founding member of the Board of Directors of Saffron Technology,a producer of associative memory technology. www.saffrontech.com

Design influenced by the way we think the brain works…



Entity Memory NAttribute 1 1Attribute 2 1 2Attribute 3 1 4 1…………… 0 0 1 0Attribute n 0 9 0 0 1

A t t r i b u t e 1

A t t r i b u t e 2

A t t r i b u t e 3

… … … … …

A t t r i b u t e n

Saffron MemoryAgent N

(Actually attribute/value pairs)(Attributes may also be entities)

Characteristics :

• N can be of order 10 millions *

• n can be of order 10 millions *

• Can manage terabytes of data• Data can be distributed

• Highly parallel processing for ingestion

• Incremental real-time update

• Fraction of second query times

• Functions:

• Observe

• Imagine

• Explain

• Categorize

• Analogues

• PredictNote: 10M is not limit, but just what has been

demonstrated with quad processor

*

Construct of a Memory BaseSimply a co-occurrence matrix, but very large number and size…

Entities are people, places or things.



Knowledge from Text

Unstructured Data

Russian President

Vladimir Putin

praised Vladimir

Ustinov for his efforts

in cracking down oncorporate fraud.

attribute:corporate

Position:President

Country:Russia

Middle:Vladimirovich

Person:Vladimir Putin

attribute:Russian

attribute:praised

attribute:efforts

attribute:cracking

Person:Vladimir Ustinov

attribute:fraud

a t t r i b u t e : c o r p o r a t e

P o s i t i o n : P r e s i d e n t

C o u n t r y : R u s s i a

M i d d l e : V l a d i m i r o v i c h

a t t r i b u t e : R u s s i a n

a t t r i b u t e : p r a i s e d

a t t r i b u

t e : e f f o r t s

a t t r i b u t e : c r a c k i n g

P e r s o n : V l a d i m i r U s t i n o v

a t t r i b

u t e : f r a u d

1

1

1

1

0

1

1

0

1

1

1

1

1

0

1

1

0

1

1

1

0

1

1

0

1

1

0

1

11

0

1

1

0

0

1

1

0

0

1

0

1

1 10

Position:Prosecutor General

attribute:corporate

Country:Russia

Middle:Vasilievich

Person:Vladimir Ustinov

attribute:Russian

attribute:praised

attribute:efforts

attribute:cracking

Person:Vladimir Putin

attribute:fraud

a t t r i b u t e : c

o r p o r a t e

P o s i t i o n : P r o s e c u t o r

G e n e r a l

C o u n t r y : R u s s i a

M i d d l e : V a s i l i e v i c h

a t t r i b u t e : R u s s i a n

a t t r i b u t e

: p r a i s e d

a t t r i b u t e : e f f o r t s

a t t r i b u t e : c r a c k i n g

P e r s o n : V l a d i m

i r P u t i n

a t t r i b u

t e : f r a u d

1

1

1

1

0

1

1

0

1

1

1

1

1

0

1

1

0

1

1

1

0

1

1

0

1

1

0

1

11

0

1

1

0

0

1

1

0

0

1

0

1

1 10

E x t r a c t i o n

Simple concept, but the complication is doing this at scale withterabytes of data, millions of entities, and millions of attributes.

Simple example of very small portion of much larger matrices…



Real-time Context at Scale

A typical network diagram only has labels at nodes – not memorieswhich enable detailed context.

• Every entity

• Every link

• Every context• Every snippet

Not a simple social network diagram…



An Army of Personal Assistants

What issimilar?

What ismissing?

What willhappen?

What isrelated?

DistributedData Bases

Memory Base

Able to answer human-like questions…

Each memory agent remembers all of the associations it has observed for that entity.



Humans Need AssistanceFor analysis, sensemaking, pathfinding and decision making…

Data −> Information => Knowledge π> Options ε> Action

• Memory Base assists human brain

• Able to ask it questions as if human• It never forgets

• Scales to enormous values• Immediate response to queries

• Entity created for each user

• These are shareable



Nature of Intelligence

• There is no such thing as absolute intelligence.

– We are never 100% certain.

• It is all relative. – Because of uncertainty and ambiguity in the data.

• Often the information assembled is open to multiple

interpretations. – Thus need competitive analyses with different hypotheses.

• In the end the decision maker must make a judgment call

– But that needs to be based on the best possible analyses,sensemaking and pathfinding.

Thus need information sharing and collaborationamongst the various components of theNational Security community.

Especially in counter-terrorism & counter-proliferation…



Sharing & Collaboration Are Essential

CollaborationSharing

Analysis SenseMaking

Pathfinding

In the context of uncertainty of available data…

• Humans are essential• To do heavy lifting of thinking• Multiple viewpoints needed

• Multiple functions

• No magic crystal ball• But they need assistance• To provide more thinking time

Need a way to visualize the stories being created for deep, rapid understanding.



Presenting Analysis to Others

• Gelertner’s Mirror Worlds and “Top Sight” – An accurate, dynamic model of real world

• Google Earth is coming close – As starting point

• Morph from one view to another – Need to define new relevant dimensions

• Narratives for analysis and deep understanding• A President’s time is precious

– Future Presidents will have grown upwith technology

– She is…

Visualization, drill-down and narratives are essential…



Visualization TechnologyGeo-temporal display of documents by SSS Research…

Note: www.sss-research.com [email protected]

• One example• Web app

• Thin client• WYSIWIS• Same info – • Various views

http://www.sss-research.com/

mailto:[email protected]

mailto:[email protected]

http://www.sss-research.com/



What’s Happening in ODNI

• Developing and implementing A-Space – For Analysis Space

• Environment for new tools and technology – That are ready today

• Emphasis on more, faster, better analyses

• In an information sharing & collaborative framework

• It will be cross agency – Multiple agencies are important to get different viewpoints

• IARPA is beginning a program called A-Space-X – To provide future technology

• Concept needs to be expanded to include more than justthe intelligence components of National Security.

To improve analysis, sensemaking, sharing & collaboration…

B k



Backups

Leaving CD with this presentation.

National Security Organization of Executive Branch



National Security Organization of Executive Branch

The President

Director NationalIntelligence

(DNI)

National SecurityAdvisor(NSA)

Director CentralIntelligence Agency

(DCIA)

Vice President

Secretary ofDefense(SecDef)

Secretary ofState

(SecState)

AttorneyGeneral

Secretary ofHomelandSecurity

DNI

Staff

National CTCenter(NCTC)

National CPCenter(NCPC)

National

IntelligenceCouncil (NIC)

NSC StaffDirectorate ofIntelligence

(DDI)

NationalClandestine

Service (DDO)

Chairman JCS

(CJCS)

CombatantCommanders

Defense IntelAgency(DIA)

Military Departments

Bureau of Intel& Research

(INR)

Bureaus

FBI Office ofIntelligence

(FBII)

Office ofInfo Analysis

(OIA)

Open SourceExploitation

Center

Statutory NSCStatutory NSC AdvisorsIntelligenceAnalysis/Sensemaking/PathfindingCoordination (honest broker)

National SecurityAgency

(NSA)

National GeospatialIntelligence Agency

(NGA)

NationalReconnaissance

Office (NRO)

Coast Guard

IntelligenceDirectorate (CGI)

Note: Principal components

Not all agencies are listed.Does not address otherbranches of government.

Note analysis/sensemaking/pathfinding are distributed…

Documents

US Army: poindexter-information-technologies-national