11:00 Self-Introductions11:15 Report on ontology-based data integration work in DCGS-A--- Goals and methodology--- Practical experience and results so far--- Risks and problems12:15 Lunch (Brown Bag Lunch?)13:00 Discussion of NSA work and goals (to be expanded)--- ontology alignment / management--- OMaaS (Object Management as a Service)14:00-15:30 Discussion of how collaborative ontology effort (e.g. between I2WD and NSA) will work in practice--- how to ensure consistency (architectural and content)--- how to address governance--- business development and work flow

DSC Ontology Work

1. Goals and methodology2. Practical experience and results so far3. Challenges

02/05/13

Goal: To realize Horizontal Integration(HI) of intelligence data

HI =Def. the ability to exploit multiple data sources as if they are one Problem: the data coming onstream are out of our

control Any strategy for HI must be agile in the sense that

it can be quickly extended to new zones of emerging data according to need

Ontology can provide the needed agility and (incremental approach to) comprehensiveness

The Business Case

• Huge resources are wasted as multiple different agencies create lexicons, glossaries, data models, messaging and exchange standards with the same or closely overlapping coverage.

• Additional resources are wasted in creating mappings between these artifacts, and in maintaining them in light of new needs and challenges. These mappings always fail.

• A sensible solution must incorporate an evolutionary process which will ensure that artifacts used to manage data converge over time.

Methodology: Create an agile strategy for building ontologies within a Shared Semantic Resource (SSR) and apply and extend these ontologies

to annotate new source data as they come onstream

⁻ Strategy pioneered in biomedical and other scientific fields: leaves data as they are, and incrementally tags data sources with terms from a growing, consistent, non-redundant set of ontologies

⁻ Problem: Given the immense and growing variety of data sources, the development methodology must be applied by multiple different groups

⁻ How to manage collaboration? This is what this meeting is about

Current State2010-2011

– Architectural implementation (DRIF) to create the Dataspace (a cloud of intelligence data) = lossless representation of sources with their native semantics

– Initiated Semantic Enhancement (SE) = using ontologies to annotate these native semantics

– Demonstration of use of SE to index and query the content of the Dataspace

2012– Created methodology and architecture for ontology development– Initiated Shared Semantic Resource (SSR); created suite of prototype ontologies

enabling SE also outside the Dataspace– CUBRC: demonstrated ability to leverage SE for analytics – Priming potential users of SE (in DoD CIO, NGA, JIEDDO, TRADOC …)

2013-– Milportal– Event Reporting Application– Initial negotiations with other agencies and groups

Ontologies We Have (Feb. 2013)• Physical Artifact, including Infrastructure, Facility,

Vehicle, Weapon

• Information Artifact, including Report, Image, Map

• Event, including Military, Criminal, Economic, Political, Religious, Social

• Human Physical Characteristics• Agent, including Person, Organization, Social Network

• Geospatial • Time

MilPortalhttp://milportal.ncor.buffalo.edu

slide from Margaret Storey

Next step

slide from Margaret Storey

Work plans 1: For DSC Cloud (on-going)

• Perform needs analysis: Review DCGS-A Logical Data Model and schemas of other DSCG-A data sets; in each case, examine content and establish what terms are needed to ensure sufficient coverage for SE;

• Where these terms already exist within the SSR, check that definitions exist and that these definitions are adequate

• Where the terms do not exist within the SSR, create new terms (or new ontologies), with appropriate definitions as necessary to fill gaps.

• Use the terms in 2. and 3. to annotate the corresponding entries in the data models to effect horizontal integration;

• With each new expansion in scope of DSGS-A data sets, iterate the above as needed.

• In addition, we are engaging in documentation of the methodology as here described, and in dissemination and training.

Work plans 2: For interagency collaboration

Step 1: Initiating interagency collaboration in the service of horizontal integration of intelligence dataIdentify candidate teams / agenciesEstablish collaboration with one or more specific teams

– Formulate and ratify agreements as appropriate– Create work plan and identify funding needs– Perform risk assessment

Step 2: Establishment of the inter-agency ontology development process Examples of types of work to be performed would include: • Create governance infrastructure• Establish needed technological support• Implement workflow• Conduct training in methodology where needed • Explore opportunities for inter-agency HI of data • Begin application to relevant agency data models of the SE strategy• Dissemination of results in a form which will allow improved systems to perform

enhanced analytics exploiting semantic interoperability.

The SSR methodology and governance is neutral as to how SSR-annotations are used• Currently SE of DSC enhancement only integration

through improved indexing/search capability• On CUBRC project, we have much more, including

ontology-based reasoning. • In the future we will have for DSC enhancement

applied – to multiple models such as LDM– to unstructured text

• see the various methodology documents provided so far

Event Reporting Application (EvR)

• Aggregates content from various report generating applications (CPOF, TIGR, TIGR with MAPHT extension)

• The underlying data model contains nearly 500 terms (e.g. ReportName, EventName, DeclassificationEvent)

• The semantics of the data model seems similar to that of a relational model

EvR Data Model as Relational Database• Hierarchies of events and units are referenced by EventType and UnitEchelon

columns, but this alone provides no capability for traversing these hierarchies• A report is related to both a report name and a report unit by the inclusion of

appropriately named columns, but what is the difference between these relationships?

• Our approach shows how to express the difference, e.g. between report-to-event and event-to unit relationships

Report

ReportId

ReportName

ReportUnit

ContainedEventIds

…

Event

ID

UnitIdentificationCode

Location

EventType

…

UnitInformation

UIC

UnitName

UnitAffiliation

UnitEchelon

…

Enhancing the vocabulary of the EvR Data Model

• Semantic enhancement (SE) amounts to associating a database field to a whole knowledge system enabling machines to process data … – “vertically” e.g. by traversing the echelon command hierarchy

and – “horizontally” e.g. by following specified relations between

units and events. • SE separates semantics from structure, reducing

maintenance costs as source databases no longer have to be modified each time we improve our understanding of reality

• The ontology tags move with the data …

Enhancing the vocabulary of the EvR Data Model

Progress to Date• Two techniques of SE are available

– Partial enhancement (now being used in the DSC) associating ontology label terms with EvR terms

– Full enhancement (not yet implemented in the DSC) aligning terms from the EvR to assertions using terms and relations from SSR ontologies

• At present the current ontologies… – provide 70% coverage for partial enhancement– provide 38% coverage for full enhancement– plan for extending the ontologies to raise the coverage for the

EvR to 90% and 60% respectively by end of February

Enhancing the vocabulary of the EvR Data Model

Event Reporting Term

Partial Enhancement via Ontology Label

Full Enhancement via Ontology Assertion(s)

Enhanced EvR Data as a Graph

Act of Reporting

Event

EventID

Report

ReportName

Report ID

Event Type

Unit Identification

Code

Unit

Command Unit

Service

designatespa

rt_of

describes

designates participates_in

agent_in

contro

ls

affiliated_w

ith

has_output

is_subcla

ss_of

UnitName

Report Unit

designates

designates

designates

shows how full annotation provides the semantics missing from the EvR relational model described above

Act of Reporting

Event

EventID

Report

ReportName

Report ID

Event Type

Unit Identification

Code

Unit

Command Unit

Service

designatespa

rt_of

describes

designates participates_in

agent_in

contro

ls

affiliated_w

ith

has_output

is_subcla

ss_of

UnitName

Report Unit

designates

designates

designates

relationships between • event type and echelon • report name and reporting unit

are made distinct and machine processable

Challenges • Challenges to Horizontal Integration in general

– Too many lexicons – The scope of the domain: signal, sensor, image, … intelligence

about the whole world• Our solution

– Incremental extraction and sanitization, and creation of content– Distributed collaborative development – Strong methodology

• Challenges for our solution– Governance and management of ontology development to

ensure consistent evolution– Lack of expertise

• For ontology development and management• For annotation

– Success will breed failure

We have these precursor ontologies

• The prototype ontologies in the existing SE (Malyuta-Salmen) helped indexing

• The ontologies we now have are much better and more than these prototypes

• How should we implement them re Event Reporting Ontology, global graph, etc. …?

• Next steps with I2WD

Our ideas are being heard

• 2 classes of collaborator: observers and partners

• companies such as DataTactics are realizing that in case of success this provides huge potential business benefits.

• (DT invited our team to talk at the Ontology Summit they have called on Feb. 12, devoted to the development of shared semantics)

Peter Morosoff and Bill Mandrick

• How to create a strategy?