Fusion Based Knowledge for the Objective Force: A Science and Technology Objective Presented August...

Fusion Based Knowledge for the Objective Force: A Science and Technology Objective

Presented August 20, 2003By Joan E. Forester

Workshop on Satellite Data Applications and Information Extraction

*ASB Study - Knowledge Management and Information Assurance, dated 09/01

Army Science Board* Estimates of Technology Readiness for Select Fields

Technology Readiness Levels

Enabling Technologies 2004 2008 Commercial

Aided ATR 3 3 2Smart Portals to push pull 6 9 9Mobile Wireless (pagers, PDA) 6 9 7Malicious Mobile Code 1 2 3Visualization - Presentation 4 7 6Data Extraction 6 8 8Virtual environment 3 6 6Automatic routers, priorities 5 8 5

Data fusion, information fusionData fusion, information fusion 22 33Secure Intelligent Agents 2 5 7Encryption and authentication 4 7 6Exploitation Algorithms and assist 2 2 2RTIC 5 8Future Internet 6 9 9Individual Soldier Tech. 4 8 5Collaboration Technologies 6 9 8Sync Distributed Secure Data base 4 7 5Secure Access Technology Biometrics 3 8 5Translingual language transcription 4 6 7Soldier Education 6 8 7Associates 6 7 5Next Generation Internet 6 9 9

Level 0: Sensor-level target identification- Processing raw data near the sensor

Level 1: Where is the enemy? (Multi-sensor correlation)- Multi INT Correlation for highly detailed Enemy Situation

----------------------------------------------------------------------------

Level 2: What is the enemy doing?Level 2: What is the enemy doing?- Aggregation for COPAggregation for COP- Interpreting activities in contextInterpreting activities in context- Develop hypotheses about current ECOADevelop hypotheses about current ECOA

Level 3: What are the enemy’s goals?- Future ECOA’s- Predict Intent and Strategy

Level 4: How should we respond?− How do we redirect the ISR system to get better SU?

Joint Directors of Laboratories (JDL) Fusion Levels

DATA FUSION PROCESSING

Where What When Who Why How

physical objects

events

environment & enemy tactics

friendly vulnerabilities & mission

effectiveness

enemy doctrine objectives & capability

individual organizations

specific aggregated

local global

options needs

local global

battle theatre

How well

DynamicTactical

Targeting

DynamicData

Exchange

Level 1: Object Refinement

Level 2: Situation Refinement

Level 3: Global Threat Refinement

Level 4: Performance Refinement

ENABLING TECHNOLOGIES

CoABSDAMLRKF

CPOF

EvidenceExtraction

& LinkDetection

Adv.ISR

Mgmt

resource management

local global

Battle Assessment &

Data Dissemination

(SUO-SAA)

DARPA Programs Related to Levels 2 & 3 Fusion

Ref: DARPA IXO Information Fusion Workshop, final briefing, 28 Feb 2002

Why We Need Fusion

Information volume exceeds war-fighter capabilities to develop situational understanding required for planning and acting within the adversary’s decision cycle

# full time Analysts, Latency forEchelon # Msg’s per hour* w/ workstations Level III Fusion

Legacy Division 400-600 15 1 Hr

Future UA Bde 17,000** 0-6 (TBD) NRT (req)

Future UA Bn 4,000** Zero NRT (req)

Future UA Company 1,200** Zero NRT (req)

* Current and estimated bottom-up sensor feeds; Top-down feed is much larger** (Date) Sensor briefing from CG, USAIC&FH to Dir, UAMBL / MAPEX indicates an order of magnitude increase

PLT COP

UE

C o m p l e x i t y

6K+ Reports/Hour

Plus…Information from echelons above UA

Report count based on DCGS-A MAPEX

results using Caspian Sea Scenario

Report count based on DCGS-A MAPEX

results using Caspian Sea Scenario

Bde COP

Bn COP

18K+ Reports/Hour

56K+ Reports/Hour

170K+ Reports/Hour

Co COP

Reports Without Fusion

Reports generated from FCS EO/IR and COMINT Sensors only.Add MASINT sensors and reporting at UA goes to @ 600K/hour.

Mr. Hayward’s Brief, Force Operating Capability (FOC) S&T Assessment Review

FCS C4ISR Software Brick

Network Foundation – (e.g, LAN, Hardware Device Drivers )

Operating System

Operating System Abstraction Services

Distribution Middleware Services

SOS Framework ServicesSecurityServicesSecurityServices

Network CentricServices

Network CentricServices

Agent FrameworkServices

Agent FrameworkServices

SystemServicesSystemServices

WebServices

WebServices

Inference EngineServices

Inference EngineServices

COTSNDI

SOS Knowledge Management Services

SOS/Domain Application Programming Services (API’s. Applets/Servlets, …)

Vehicle Applications Mission Applications DoD Enterprise Applications

Administration Applications

Warfighter Machine Interface (WMI)

COTSNDI

DistributedFrameworkDistributedFramework

System,Fault Mgt, Health Monitoring

System,Fault Mgt, Health Monitoring

LogicalDatabaseLogical

DatabaseStorage

& RetrievalStorage

& RetrievalSecurityServicesSecurityServices

RoutingServicesRoutingServices

VirtualMemoryVirtual

Memory Run-TimeRun-Time ThreadsThreads SocketsSocketsProcessProcess Dynamic Linking

Dynamic Linking

Select IO Comp

Select IO Comp

Memory MappingMemory Mapping

Virtual MemoryVirtual Memory CommunicationsCommunications Process/ThreadsProcess/Threads

Info Access& Control Services

Info Access& Control Services

InteroperabilityServices

InteroperabilityServices

InformationDiscoveryServices

InformationDiscoveryServices

User ProfileServices

User ProfileServices

Information Dissemination

Services

Information Dissemination

Services

TacticalConfiguration

Services

TacticalConfiguration

ServicesCommonSupportServices

CommonSupportServices

Wea

pon

Sys

Wea

pon

Sys

Airs

pace

Mgt

Airs

pace

Mgt

Info

rmat

ion

War

fare

In

form

atio

n W

arfa

re

Com

m S

yste

ms

Com

m S

yste

ms

C2C2

“Fusion” is a component of the C2 Application Software

Shull FCS LSI Concept Brief at MAPEX

UA (I)UA (I)

UA (X)

UA (II)

Theater/Natio

nal (IBS/G

BS)

UE (XX/XXX)

RJ

U2R

JSTARS

UE

UA (I)UA (I)

UA (X)

UA (II)

Theater

Right Information…Right Time…to the Point of DecisionRight Information…Right Time…to the Point of Decision

ACS ACS

MC2A

PREDATORTUAV

GLOBAL HAWK

ALLIED/

COALITION

The Objective Force Sensor Grid

SPACE SYSTEMS

Direct Linkage(s) to CDR, Staff & Shooters

Joint ISR toFCS Via DCGS-A

Interdependent, Multi-Echelon, Cross-BOS, Net-Centric

PROPHET

2R

Presented by Col Ron Nelson

11 Dec 02

DCGS-A

FBKOF: Overcoming Information Overload

BARRIERS

• Limited computational modelsLimited computational models• Knowledge is METT-TC dependentKnowledge is METT-TC dependent• COTS knowledge acquisition technology too COTS knowledge acquisition technology too

slowslow• Information sources poorly integratedInformation sources poorly integrated• Knowledge discovery tool limitedKnowledge discovery tool limited

APPROACH

• Cognitive engineering and user-centered Cognitive engineering and user-centered designdesign

• Apply Blackboard architecture, diverse Apply Blackboard architecture, diverse knowledge representation and inferencing, knowledge representation and inferencing, approximation techniques, and “to each his approximation techniques, and “to each his own” cooperative human-machine problem own” cooperative human-machine problem solvingsolving

• Exploit DARPA rapid knowledge formation Exploit DARPA rapid knowledge formation technologies to develop knowledge-technologies to develop knowledge-intensive reasoning for interpretationintensive reasoning for interpretation

• Leverage Semantic Web techniques for Leverage Semantic Web techniques for source integration.source integration.

• Integrate and tailor COTS tools for directed Integrate and tailor COTS tools for directed knowledge discoveryknowledge discovery

DELIVERABLES

• SW for knowledge generation and SW for knowledge generation and explanation to answer PIR’s in a timely explanation to answer PIR’s in a timely mannermanner

• Ontology based information agents for Ontology based information agents for objective force systemsobjective force systems

• User-directed knowledge discovery toolsUser-directed knowledge discovery tools• Modeling and simulation toolsModeling and simulation tools

Schedule

Tasks

•

Knowledge Infrastructure Development

•

•

•

Modeling and Simulation Support•

C4I experiments and evaluations•

Transitions Decision Points

FY05

CECOM

ARL 2.1 2.1 1.0

FY06 FY07FY04

Baseline / Assess Knowledge tools and Fusion Algorithms

FY03

2.1

1.7 2.2 3.0 4.0 4.0

•

Knowledge Acquisition

3 4 5

42

2.1

Mining-Component Development 3 4

3

2 3 4

TOTAL $24.3

Notional Blackboard Architecture for Fusion Subsystem

CONTROL

Answers to PIRs

COAs and COA Fragments

Relations betweenobjects (commandhierarchy, behavioral)Events & Activities

Objects(equipment andplatform-levelentities)

Plans KS

:

• Doctrine•

• History

•

Terrain & Weather

Activities KS

:•

Force Structure• Commo Patterns

• Tactics

• Terrain & Weather

Sensor-Data Fusion KS:

• Platform & Equipment Classification and Movement Attributes

• Terrain & Weather

KnowledgeSourcesBlackboard

Levels ofAnalysis

GOALS

• Minimize burden on user

– Automate well-structured problems

– Support ill-structured problems

• Interface tuned to the task and to the user• Task centered, not tool centered• Support information push and pull• Support collaboration• Accommodate multi-modal data types• Visualization tools to support understanding• Smarter integration of sources

– Limit the number of required retrievals (bandwidth)

– Minimize exploration after retrieval (time constrained)

– Automate and personalize the process

Providing a Knowledge Environment (Agents and Ontologies)

DBMS

Interface

OLAP

Fusion

Search Engine

Data-base

Data-baseData-

base

Interface

Know-ledge Base

Web

Why Agents?

What are they? (ATL)

• The concept of software agents represents a new way of applying artificial intelligence techniques such as machine reasoning and learning.

• Software agents are computer programs designed to operate in a manner analogous to human agents. Human agents, such as real-estate agents, carry out tasks on your behalf using expertise you may not have. Software agents carry out information processing functions in the same manner.

• Agents can be thought of, in software engineering terms, as a step beyond the objects of object-oriented programming. Whereas objects are passive entities that must be invoked to execute, agents use AI mechanisms such as machine reasoning to actively operate as autonomous entities.

• Research has shown greatest utility in multi-agent applications is information mgmt.

How do they help?

• Huge problem broken into small components• Much can be handled in parallel rather than serially• Reflect changes in priorities without coding changes• Technology is coming of age• Many web applications [6, 9]: mediator, personal assistant

Active, persistent sw components that perceive, reason, act and communicate

-- Huhns

Agent Functionality

• Filter (ATL)• Monitors (ATL)• Alert (ATL)• Retrieve – pull (ATL)• Disseminate – push (ATL)• Adapt to the user priority (CTA, OH S U)• Adapt to the environmental changes (CTA, OH S U)• Mediate across legacy systems (UMD)• Intruder detection (HPC, UMINN)• Policy enforcement (CTA, U W Fl)

PROBLEMS• Agents new, few success stories and limited developmental environments• Present complex parallel processing paradigm• Issues of teaming, security, mobility, efficiency• Establishing optimum ontology size/approach• Integrating ontologies across heterogeneous sources (single, multiple, hybrid)

Why Ontology-Based?

DCGS-A Data StoreSingle-INTs

COMINT ELINTMASINT ImageryImages/ Video/ Audio

MTI HUMINTOther Multimedia

Open Source

MIDB Blue

Asset Mgmt

Weather

CCIR/IR/

OPLANs

Terrain

Targets

Alert/SearchCriteria

External COPs(above/below/beside)

COPCOP

COP COPCOP

Brigade level

All Source Fusion(ASFDB)

Units Pieces of EquipmentFacilities Events

Individuals OrganizationsAnd their interrelationships

Mediator

Agent

Agent Commo Module

Reasoner

Prioritzer

Ontology

Agent

• Information heterogeneous (type, syntax, semantics)

• Heterogeneity of semantics results in conflicts (naming, scaling, confounding)

• Ontologies explicitly describe information sources • Identify and share formal descriptions of domain-relevant concepts• Identify classes of objects and organized them hierarchically• Characterize classes by the properties they share• Identify important relationships between classes

Fusion

• On Line Analytical Processing (OLAP) emerged in the early 90’s (Inmon, Codd)• Multi-dimensional data structure• Better (more flexibly) address decision process (forecasting, time-series

analysis, link analysis)• More natural & efficient storage and retrieval mechanism• Provides a mechanism for accommodating time and space • Flexible graphical interface• Commercial Product• Natural Transition to Data Mining

PROBLEMS

• Representation of space and time• Complexity of user interface• Inefficiency of algorithms

Providing User-Directed Knowledge Discovery Tools

Partners and Leveraged Programs

• RDECOM(provisional) RDEC I2WD• Army G2 (Woodson/ ISR Working Group)• Huachuca (Schlabach – Cahill)• BAH (Brown - Army MI SME)• ADA CTA (U W Fl, UMD, SA Tech)• ARMY HPC Program (UMINN, Data mining)• ARL CENTERS OF EXCELLENCE (CAU, Data mining)• PENN State (Yen, Teaming Agents)• C2CUT and Warrior’s Edge• DARPA: Taylor (RKF, Staff Officer in a Box); Alex Kott

(AIM); Burke (DAML, CoABS Grid)• ENDORSEMENTS: BCBL-H; BCBL-L; PM DCGS-A, PM

IE, PM FCS

FY03 : (1) Conduct cognitive engineering with SME to identify users’ goals, tasks and info requirements (1) Conduct cognitive engineering with SME to identify users’ goals, tasks and info requirements most germane to the Intel BOS in support of higher level fusion -- identify candidate tasks to focus on in most germane to the Intel BOS in support of higher level fusion -- identify candidate tasks to focus on in FY04. (2) Develop initial human-machine and software-level evaluation plan for fusion. Design and FY04. (2) Develop initial human-machine and software-level evaluation plan for fusion. Design and conduct pilot experiment for fusion. (3) Develop a small prototype Knowledge Environment (KE) that uses conduct pilot experiment for fusion. (3) Develop a small prototype Knowledge Environment (KE) that uses agent techniques to access the two highest priority data sources. This will establish a baseline system on agent techniques to access the two highest priority data sources. This will establish a baseline system on which to build in out years, demonstrate our initial concept of the use of ontologies by the KE agent which to build in out years, demonstrate our initial concept of the use of ontologies by the KE agent communities, and provide a mechanism for integrating CECOM’s fusion modules. (3) Conduct an internal communities, and provide a mechanism for integrating CECOM’s fusion modules. (3) Conduct an internal demonstration of the baseline system to support refinement of the HCI/KE concepts.demonstration of the baseline system to support refinement of the HCI/KE concepts.

FY04 : (1) Integrate two more data sources into the baseline system to assess the extensibility of the infrastructure and provide the CECOM fusion module access to a greater variety of data sources. (2) Develop and populate a prototype multi-dimensional data structure for user directed data mining or knowledge discovery (KD). This will allow us to explore the use of user-in-the-loop fusion tools to supplement CECOM automated fusion techniques. (3) Conduct an internal joint CECOM/ARL demonstration to refine the HCI and KE concepts.

FY05 : (1) Modify the KE system architecture, based on the FY04 evaluation and integrate 5th data/information source. (2) Jointly demonstrate to DCGS-A and user communities the integration of CECOM’s fusion algorithms, the user-directed KD tools and 5 data sources. This provides a formal review for the targeted transition system developers (FCS/DCGS-A) of the refined approach at a point when all the required components are in place.

FY06 : (1) Finalize user-directed mining scripts and system architecture, based on FY05 evaluation. The goal will be to simplify access to the KD tools. (3) Develop information agents to support I2WD fusion task. These agents will be directed toward increasing the efficiency and effectiveness of information push/pull. (2) Internally demonstrate automated cross-source integration using the enhanced agent environment and work with CECOM to evaluate and enhance the system’s functionality.

FY07 : (1) Finalize system development, based on FY06 evaluation. (2) Jointly conduct the final system demonstration and evaluation to support system transition to FCS LSI contractor, PM-CGS, and PM-IF.

Status

(1)(1) User involvementUser involvement

(2)(2) Working with ATL’s EMAA to establish Working with ATL’s EMAA to establish mediators and baseline architecture. mediators and baseline architecture. Warrior’s Edge (WE) link may increase the Warrior’s Edge (WE) link may increase the size and scope of this effort. ATL also size and scope of this effort. ATL also working a portion of CECOM’s fusion effort.working a portion of CECOM’s fusion effort.

(3)(3) Internal demo scheduled for late September. Internal demo scheduled for late September. More visible demo may occur with WE. More visible demo may occur with WE.

(+) CAU/UMINN Date Mining demo(+) CAU/UMINN Date Mining demo

Conclusion

• Goal: Facilitate quick war fighting decisions that fully leverage the huge volumes of information that the UA will receive. – RDECOM I2WD user-centered fusion system design (architecture,

inferencing techniques, algorithms, representations, and HCI)– ARL knowledge management infrastructure– ARL user-directed knowledge discovery tools

• Proposed relatively modest software readiness levels, due to difficulty of the task, but driving to get a transition:– PM DCGS-A demonstration in 05, with a transition decision point in 07 – PM IE demonstration in 05, transition decision point in 07– Demonstration to FCS LSI 05, AMSAA transition decision point in 07

• Data mining resources far exceed initial expectations.

• First year of agents development will receive a boost from related ARL programs (C2CUT, Warrior’s Edge)

• Strong support from user community