80
An Ontology-Centric Approach to Sensor-Mission Assignment Mario Gomez Matthew P. Johnson + Geeth de Mel Wamberto Vasconcelos Christopher Gibson Amotz Bar-Noy + Konrad Borowiecki Tom La Porta Hosam Rowaihy Gavin Pearson § and Tien Pham EKAW 2008 † University of Aberdeen, UK + City University of New York, USA Pennsylvania State University, USA § Defence Science and Technology Laboratory, UK ¶ Army Research Laboratory, USA IBM, UK Diego Pizzocaro Alun Preece ‡ Cardiff University , UK

An Ontology-Centric Approach to Sensor-Mission Assignment

Embed Size (px)

DESCRIPTION

Paper presentation at EKAW 2008, Aci-Trezza, Sicily (Italy)
03/10/08

Citation preview

Page 1: An Ontology-Centric Approach to Sensor-Mission Assignment

An Ontology-Centric Approach to Sensor-Mission Assignment

Mario Gomez † Matthew P. Johnson + Geeth de Mel † Wamberto Vasconcelos † Christopher Gibson ♦

Amotz Bar-Noy + Konrad Borowiecki ‡ Tom La Porta∗ Hosam Rowaihy ∗ Gavin Pearson § and Tien Pham ¶

EKAW 2008

† University of Aberdeen, UK

+ City University of New York, USA

∗ Pennsylvania State University, USA

§ Defence Science and Technology Laboratory, UK

¶ Army Research Laboratory, USA ♦IBM, UK

Diego Pizzocaro ‡

Alun Preece ‡

‡ Cardiff University, UK

Page 2: An Ontology-Centric Approach to Sensor-Mission Assignment

An Ontology-Centric Approach to Sensor-Mission Assignment

Mario Gomez † Matthew P. Johnson + Geeth de Mel † Wamberto Vasconcelos † Christopher Gibson ♦

Amotz Bar-Noy + Konrad Borowiecki ‡ Tom La Porta∗ Hosam Rowaihy ∗ Gavin Pearson § and Tien Pham ¶

EKAW 2008

† University of Aberdeen, UK

+ City University of New York, USA

∗ Pennsylvania State University, USA

§ Defence Science and Technology Laboratory, UK

¶ Army Research Laboratory, USA ♦IBM, UK

Diego Pizzocaro ‡

Alun Preece ‡

‡ Cardiff University, UK

Page 3: An Ontology-Centric Approach to Sensor-Mission Assignment

3. Sensor Deployment and Delivery

Outline

Sensor-Mission Assignment problem

Conclusion & Future

1. Sensor-Task Fitting

2. Sensor-Task Allocation

Page 4: An Ontology-Centric Approach to Sensor-Mission Assignment

Why so many authors?

• The heterogeneity of the problem requires a modular approach.

• We present a conceptual system architecture:every author developed interlinked modules.

• The architecture is centered around an ontology.

Mario Gomez † Alun Preece ‡ Matthew P. Johnson + Geeth de Mel † Wamberto Vasconcelos † Christopher Gibson ♦

Amotz Bar-Noy + Konrad Borowiecki ‡ Tom La Porta∗ Diego Pizzocaro ‡ Hosam Rowaihy ∗ Gavin Pearson § and Tien Pham ¶

Page 5: An Ontology-Centric Approach to Sensor-Mission Assignment

Why so many authors?

• The heterogeneity of the problem requires a modular approach.

• We present a conceptual system architecture:every author developed interlinked modules.

• The architecture is centered around an ontology.

Mario Gomez † Alun Preece ‡ Matthew P. Johnson + Geeth de Mel † Wamberto Vasconcelos † Christopher Gibson ♦

Amotz Bar-Noy + Konrad Borowiecki ‡ Tom La Porta∗ Diego Pizzocaro ‡ Hosam Rowaihy ∗ Gavin Pearson § and Tien Pham ¶____________________

= 13

Page 6: An Ontology-Centric Approach to Sensor-Mission Assignment

Sensors (or Sensing asset)

Simple sensors

Platforms

Page 7: An Ontology-Centric Approach to Sensor-Mission Assignment

Sensors (or Sensing asset)

Simple sensors

Platforms

Page 8: An Ontology-Centric Approach to Sensor-Mission Assignment

Sensors (or Sensing asset)

Missions

Simple sensors

Platforms

composed by different tasks:

e.g. Search-&-Rescue mission

TASK 1

Injured people to identify

TASK 4

Area Surveillance

(possible threats)

TASK 3

Area Surveillance

(possible threats)

TASK 2Injured

people to identify

Page 9: An Ontology-Centric Approach to Sensor-Mission Assignment

Main problem:

Sensor-Mission Assignment

is the problem of assigning sensing assets to missions to cover the information needs of

individual tasks in each mission.

Page 10: An Ontology-Centric Approach to Sensor-Mission Assignment

Scenario

Page 11: An Ontology-Centric Approach to Sensor-Mission Assignment

• A network of heterogeneous sensing assets:

- Support multiple missions to be accomplished simultaneously.

Scenario

TASK 8

Detect suspiciousvehicles

TASK 6

Discover mines

TASK 4

Area Surveillance

(possible threats)

TASK 2Injured people to identify

TASK 1

Injured people to identify

TASK 5

AudioIntelligence

TASK 3

Area Surveillance

(possible threats)

TASK 7Aerial

Surveillance

Page 12: An Ontology-Centric Approach to Sensor-Mission Assignment

• A network of heterogeneous sensing assets:

- Support multiple missions to be accomplished simultaneously.

- Sensing assets are scarce and in high demand.

Scenario

TASK 8

Detect suspiciousvehicles

TASK 6

Discover mines

TASK 4

Area Surveillance

(possible threats)

TASK 2Injured people to identify

TASK 1

Injured people to identify

TASK 5

AudioIntelligence

TASK 3

Area Surveillance

(possible threats)

TASK 7Aerial

Surveillance

Page 13: An Ontology-Centric Approach to Sensor-Mission Assignment

• A network of heterogeneous sensing assets:

- Support multiple missions to be accomplished simultaneously.

- Sensing assets are scarce and in high demand.

- Highly dynamic (sensing asset failures, change of plan)

Scenario

TASK 8

Detect suspiciousvehicles

TASK 6

Discover mines

TASK 4

Area Surveillance

(possible threats)

TASK 2Injured people to identify

TASK 1

Injured people to identify

TASK 5

AudioIntelligence

TASK 3

Area Surveillance

(possible threats)

TASK 7Aerial

Surveillance

Page 14: An Ontology-Centric Approach to Sensor-Mission Assignment

Subproblems• We identify 3 subproblems to solve the SENSOR-MISSION ASSIGNMENT:

Page 15: An Ontology-Centric Approach to Sensor-Mission Assignment

1. SENSOR-TASK FITTING

Tasks have different information requirements:

we need to figure out which types of sensing assets are suitable.

Subproblems• We identify 3 subproblems to solve the SENSOR-MISSION ASSIGNMENT:

Page 16: An Ontology-Centric Approach to Sensor-Mission Assignment

1. SENSOR-TASK FITTING

Tasks have different information requirements:

we need to figure out which types of sensing assets are suitable.

2. SENSOR-TASK ALLOCATION

Tasks may compete for the exclusive usage of a sensing asset:

we need a scheme to assign individual assets to tasks.

Subproblems• We identify 3 subproblems to solve the SENSOR-MISSION ASSIGNMENT:

Page 17: An Ontology-Centric Approach to Sensor-Mission Assignment

1. SENSOR-TASK FITTING

Tasks have different information requirements:

we need to figure out which types of sensing assets are suitable.

2. SENSOR-TASK ALLOCATION

Tasks may compete for the exclusive usage of a sensing asset:

we need a scheme to assign individual assets to tasks.

3. SENSOR DEPLOYMENT & DELIVERY

Sensing assets must be dynamically configured and deployed,

ensuring the right information is efficiently delivered to the user:

we need a sensor infrastructure to mediate with physical-world sensors.

Subproblems• We identify 3 subproblems to solve the SENSOR-MISSION ASSIGNMENT:

Page 18: An Ontology-Centric Approach to Sensor-Mission Assignment

• A Semantic Web approach:

The Core is a set of interlinked ontologies describing Missions/Tasks and

Sensors/Platforms (building on pre-existing ontologies and schemas wherever possible).

Conceptual architecture

Deployment

Deployment

& Tasking

ISR assets available

Fitting

Reasoner

Allocation

algorithms

Monitoring

Day & night video

surveillance Locate snipers

in the wood

Requirements

- Capabilities

- QoI

- Environment

Assets

- Type

- Location

- Status

Ranking feasible

solutions

Near-optimal

allocation

A BD

E

C

Sensor

Infrastructure

("Fabric")

S1

S2

S3

S4

Planning

Logistics

Dynamic events

Update status

Page 19: An Ontology-Centric Approach to Sensor-Mission Assignment

• A Semantic Web approach:

The Core is a set of interlinked ontologies describing Missions/Tasks and

Sensors/Platforms (building on pre-existing ontologies and schemas wherever possible).

Conceptual architecture

Deployment

Deployment

& Tasking

ISR assets available

Fitting

Reasoner

Allocation

algorithms

Monitoring

Day & night video

surveillance Locate snipers

in the wood

Requirements

- Capabilities

- QoI

- Environment

Assets

- Type

- Location

- Status

Ranking feasible

solutions

Near-optimal

allocation

A BD

E

C

Sensor

Infrastructure

("Fabric")

S1

S2

S3

S4

Planning

Logistics

Dynamic events

Update status

SENSOR-TASK

FITTING

Page 20: An Ontology-Centric Approach to Sensor-Mission Assignment

• A Semantic Web approach:

The Core is a set of interlinked ontologies describing Missions/Tasks and

Sensors/Platforms (building on pre-existing ontologies and schemas wherever possible).

Conceptual architecture

Deployment

Deployment

& Tasking

ISR assets available

Fitting

Reasoner

Allocation

algorithms

Monitoring

Day & night video

surveillance Locate snipers

in the wood

Requirements

- Capabilities

- QoI

- Environment

Assets

- Type

- Location

- Status

Ranking feasible

solutions

Near-optimal

allocation

A BD

E

C

Sensor

Infrastructure

("Fabric")

S1

S2

S3

S4

Planning

Logistics

Dynamic events

Update status

SENSOR-TASK

FITTING

SENSOR-TASK

ALLOCATION

Page 21: An Ontology-Centric Approach to Sensor-Mission Assignment

• A Semantic Web approach:

The Core is a set of interlinked ontologies describing Missions/Tasks and

Sensors/Platforms (building on pre-existing ontologies and schemas wherever possible).

Conceptual architecture

Deployment

Deployment

& Tasking

ISR assets available

Fitting

Reasoner

Allocation

algorithms

Monitoring

Day & night video

surveillance Locate snipers

in the wood

Requirements

- Capabilities

- QoI

- Environment

Assets

- Type

- Location

- Status

Ranking feasible

solutions

Near-optimal

allocation

A BD

E

C

Sensor

Infrastructure

("Fabric")

S1

S2

S3

S4

Planning

Logistics

Dynamic events

Update status

SENSOR-TASK

FITTING

SENSOR-TASK

ALLOCATION

SENSOR

DEPLOYMENT

AND DELIVERY

Page 22: An Ontology-Centric Approach to Sensor-Mission Assignment

• A Semantic Web approach:

The Core is a set of interlinked ontologies describing Missions/Tasks and

Sensors/Platforms (building on pre-existing ontologies and schemas wherever possible).

Conceptual architecture

Deployment

Deployment

& Tasking

ISR assets available

Fitting

Reasoner

Allocation

algorithms

Monitoring

Day & night video

surveillance Locate snipers

in the wood

Requirements

- Capabilities

- QoI

- Environment

Assets

- Type

- Location

- Status

Ranking feasible

solutions

Near-optimal

allocation

A BD

E

C

Sensor

Infrastructure

("Fabric")

S1

S2

S3

S4

Planning

Logistics

Dynamic events

Update status

SENSOR-TASK

FITTING

SENSOR-TASK

ALLOCATION

SENSOR

DEPLOYMENT

AND DELIVERY

Page 23: An Ontology-Centric Approach to Sensor-Mission Assignment

Sensor-Task Fittingdecides which collections of asset types

fit the information requirements of each task.

TASK

Area video Surveillance

(possible threats)

SENSING ASSET TYPES

Page 24: An Ontology-Centric Approach to Sensor-Mission Assignment

Sensor-Task Fittingdecides which collections of asset types

fit the information requirements of each task.

TASK

Area video Surveillance

(possible threats)

SENSING ASSET TYPES

Page 25: An Ontology-Centric Approach to Sensor-Mission Assignment

Sensor-task fitting

• Semantic matchmaking to evaluate the fitness-for-purpose of collections of asset

types to the information requirements of a task.

Methodology

Page 26: An Ontology-Centric Approach to Sensor-Mission Assignment

Sensor-task fitting

• Semantic matchmaking to evaluate the fitness-for-purpose of collections of asset

types to the information requirements of a task.

Methodology

• We use ontologies for:

Page 27: An Ontology-Centric Approach to Sensor-Mission Assignment

Sensor-task fitting

• Semantic matchmaking to evaluate the fitness-for-purpose of collections of asset

types to the information requirements of a task.

Methodology

• We use ontologies for:

Specify the information requirements of a task.

Specify the sensing capabilities provided by assets.

ISR ontologies: tasks, sensors, etc

Page 28: An Ontology-Centric Approach to Sensor-Mission Assignment

Sensor-task fitting

• Semantic matchmaking to evaluate the fitness-for-purpose of collections of asset

types to the information requirements of a task.

Methodology

• We use ontologies for:

Specify the information requirements of a task.

Specify the sensing capabilities provided by assets.

Compare the two of them.

ISR ontologies: tasks, sensors, etc

MMF ontology

Page 29: An Ontology-Centric Approach to Sensor-Mission Assignment

Sensor-task fitting

• Semantic matchmaking to evaluate the fitness-for-purpose of collections of asset

types to the information requirements of a task.

Methodology

• We use ontologies for:

Specify the information requirements of a task.

Specify the sensing capabilities provided by assets.

Compare the two of them.

ISR ontologies: tasks, sensors, etc

MMF ontology

Page 30: An Ontology-Centric Approach to Sensor-Mission Assignment

• Based on the pre-existent Missions and Means Framework (MMF)

‣ MMF is an informal framework to help human planners determine the capabilities

required to accomplish a military mission. (developed by US ARL)

MMF ontology

Page 31: An Ontology-Centric Approach to Sensor-Mission Assignment

• Based on the pre-existent Missions and Means Framework (MMF)

‣ MMF is an informal framework to help human planners determine the capabilities

required to accomplish a military mission. (developed by US ARL)

MMF ontology

Expert’s knowledge

Page 32: An Ontology-Centric Approach to Sensor-Mission Assignment

• Based on the pre-existent Missions and Means Framework (MMF)

‣ MMF is an informal framework to help human planners determine the capabilities

required to accomplish a military mission. (developed by US ARL)

MMF ontology

Expert’s knowledge

Mission

Page 33: An Ontology-Centric Approach to Sensor-Mission Assignment

• Based on the pre-existent Missions and Means Framework (MMF)

‣ MMF is an informal framework to help human planners determine the capabilities

required to accomplish a military mission. (developed by US ARL)

MMF ontology

Expert’s knowledge

Mission

OperationOperation

Page 34: An Ontology-Centric Approach to Sensor-Mission Assignment

• Based on the pre-existent Missions and Means Framework (MMF)

‣ MMF is an informal framework to help human planners determine the capabilities

required to accomplish a military mission. (developed by US ARL)

MMF ontology

Expert’s knowledge

Mission

OperationOperation

TaskTask

Page 35: An Ontology-Centric Approach to Sensor-Mission Assignment

• Based on the pre-existent Missions and Means Framework (MMF)

‣ MMF is an informal framework to help human planners determine the capabilities

required to accomplish a military mission. (developed by US ARL)

MMF ontology

Expert’s knowledge

Mission

OperationOperation

TaskTask

ComponentComponent

Page 36: An Ontology-Centric Approach to Sensor-Mission Assignment

• Based on the pre-existent Missions and Means Framework (MMF)

‣ MMF is an informal framework to help human planners determine the capabilities

required to accomplish a military mission. (developed by US ARL)

MMF ontology

Expert’s knowledge

Mission

OperationOperation

TaskTask

ComponentComponent

SystemSystem

Page 37: An Ontology-Centric Approach to Sensor-Mission Assignment

• Based on the pre-existent Missions and Means Framework (MMF)

‣ MMF is an informal framework to help human planners determine the capabilities

required to accomplish a military mission. (developed by US ARL)

MMF ontology

Expert’s knowledge

Mission

OperationOperation

TaskTask

Platform

ComponentComponent

SystemSystem

Page 38: An Ontology-Centric Approach to Sensor-Mission Assignment

• Based on the pre-existent Missions and Means Framework (MMF)

‣ MMF is an informal framework to help human planners determine the capabilities

required to accomplish a military mission. (developed by US ARL)

MMF ontology

Expert’s knowledge

Mission

OperationOperation

TaskTask CapabilityCapability

Platform

ComponentComponent

SystemSystem

Page 39: An Ontology-Centric Approach to Sensor-Mission Assignment

• Based on the pre-existent Missions and Means Framework (MMF)

‣ MMF is an informal framework to help human planners determine the capabilities

required to accomplish a military mission. (developed by US ARL)

MMF ontology

Expert’s knowledge

Mission

OperationOperation

TaskTask CapabilityCapability

Platform

ComponentComponent

SystemSystem

Page 40: An Ontology-Centric Approach to Sensor-Mission Assignment

• Based on the pre-existent Missions and Means Framework (MMF)

‣ MMF is an informal framework to help human planners determine the capabilities

required to accomplish a military mission. (developed by US ARL)

MMF ontology

Expert’s knowledge

Mission

OperationOperation

TaskTask CapabilityCapability

Platform

ComponentComponent

SystemSystem

Capabilitiesrequired

Page 41: An Ontology-Centric Approach to Sensor-Mission Assignment

• Based on the pre-existent Missions and Means Framework (MMF)

‣ MMF is an informal framework to help human planners determine the capabilities

required to accomplish a military mission. (developed by US ARL)

MMF ontology

Expert’s knowledge

Mission

OperationOperation

TaskTask CapabilityCapability

Platform

ComponentComponent

SystemSystem

Capabilitiesrequired

• In the military domain very precise definitions of capabilities required:

• ISR requirements (Intelligence - Surveillance - Reconnaissance)

Page 42: An Ontology-Centric Approach to Sensor-Mission Assignment

MMF ontology (contd)

• Why develop an MMF ontology?

✓ Formally define relationship: Task’s ISR reqs Sensor/platform’s ISR capabilities.

✓ Directly use it inside the SENSOR-TASK FITTING reasoner.

Page 43: An Ontology-Centric Approach to Sensor-Mission Assignment

• Main concepts and relations in MMF ontology (implemented in OWL DL):

MMF ontology (contd)

• Why develop an MMF ontology?

✓ Formally define relationship: Task’s ISR reqs Sensor/platform’s ISR capabilities.

✓ Directly use it inside the SENSOR-TASK FITTING reasoner.

Task Capability

Operation

Mission

Asset

Platform System

Sensor

comprises toAccomplish

comprises toAccomplish

toPerform

is-a

is-a

is-a

mounts

attachedTo

requires

providesallocatedTo

interferesWith

entails

Page 44: An Ontology-Centric Approach to Sensor-Mission Assignment

ISR ontologies

Sensor-task fitting

MMF ontology

• To describe instances of the MMF ontology concepts:

we need domain specific ontologies.

• Previous attempts to conceptualize ISR domain: mixed dimensions.

ISR ontologies: tasks, sensors, etc

Page 45: An Ontology-Centric Approach to Sensor-Mission Assignment

ISR ontologies

Sensor-task fitting

MMF ontology

• To describe instances of the MMF ontology concepts:

we need domain specific ontologies.

• Previous attempts to conceptualize ISR domain: mixed dimensions.

ISR ontologies: tasks, sensors, etc

Page 46: An Ontology-Centric Approach to Sensor-Mission Assignment

ISR ontologies

Sensor-task fitting

MMF ontology

• To describe instances of the MMF ontology concepts:

we need domain specific ontologies.

• Previous attempts to conceptualize ISR domain: mixed dimensions.

ISR ontologies: tasks, sensors, etc

Page 47: An Ontology-Centric Approach to Sensor-Mission Assignment

ISR ontologies

Sensor-task fitting

MMF ontology

• To describe instances of the MMF ontology concepts:

we need domain specific ontologies.

• Previous attempts to conceptualize ISR domain: mixed dimensions.

ISR ontologies: tasks, sensors, etc

Example: UAV (Unmanned Aerial Vehicle) classification mixes

- size (Micro vs Mini)

- performance (Medium altitude vs High altitude)

- task type (Maritime reconnaissance, Wide area surveillance, ...)

Page 48: An Ontology-Centric Approach to Sensor-Mission Assignment

ISR ontologies (contd)• We use OWL DL to conceptualize the ISR domain based

on pre-existent ontologies and schemas:

‣ Define new concepts by specifying property restrictions and relations on existing concepts:

– Aircraft ! (Platform " # hasRealm.Atmosphere)– UnmannedVehicle ! (Platform " # hasQuality.Without-crew-mobility)– UAV ! (Aircraft " UnmannedVehicle)

Page 49: An Ontology-Centric Approach to Sensor-Mission Assignment

ISR ontologies (contd)

In order to deal with the challenges introduced above, we propose a compositionaland multidimensional approach to conceptualize the ISR domain. Such an approach iswell suited to Description Logics (DL) [7] languages such as OWL DL. One of the mostpowerful features of DLs is their ability to define classes in terms of sufficient and nec-essary conditions. New concepts can be defined by specifying property restrictions andrelations on existing concepts. As an example, consider the following DL definitionsfor some concepts relating to aerial sensor platforms:

– Aircraft ! (Platform " # hasRealm.Atmosphere)– UnmannedVehicle ! (Platform " # hasQuality.Without-crew-mobility)– UAV ! (Aircraft " UnmannedVehicle)– CombatUAV ! (UAV " # providesCapability.Firepower)– MALE ! (UAV " # providesCapability.MediumAltitude " # providesCapabil-

ity.LongEndurance)– EnduranceUAV $ ¬ (SmallUAV % TacticalUAV).

By using OWL DL, we can apply off-the-self reasoners to infer new classificationsbased on concept definitions, which is very appropriate because different classificationsare useful for different purposes. For example at some point one might be interested inselecting a platform in terms of the type of tasks that it can perform (reconnaissance,surveillance, battle damage assessment, etc), but in other circumstances one might beinterested in selecting a platform according to its takeoff and landing capabilities (cata-pult, runaway, VTOL, etc). Some of these dimensions are:

– For platforms: mobility, realm, performance (range, endurance, altitude, speed,etc.), application or mission type (surveillance, reconnaissance, target acquisition,etc.), firepower, landing and takeoff, communications, vulnerability and survivabil-ity, availability.

– For sensors: phenomena detected (type and spectrum), performance (resolution,sample rate, etc), weather/terrain/contamination influence, vulnerability, interfer-ences with other sensors.

(a) Platforms (b) Sensors (c) ISR tasks (d) Intelligence

Fig. 5: Sample of concept taxonomies relevant to the ISR domain

Figure 5 shows a sample of some of the taxonomies we have developed for the ISRontology. Platforms (a) and sensors (b) are used to characterize ISR assets; the formerdraws on concepts from OntoSensor and CIMA, the latter from the MMI Platforms

• We use OWL DL to conceptualize the ISR domain based

on pre-existent ontologies and schemas:

‣ Define new concepts by specifying property restrictions and relations on existing concepts:

• A reasoner can infer different classifications (to deal with multi-dimensions):

– Aircraft ! (Platform " # hasRealm.Atmosphere)– UnmannedVehicle ! (Platform " # hasQuality.Without-crew-mobility)– UAV ! (Aircraft " UnmannedVehicle)

In order to deal with the challenges introduced above, we propose a compositionaland multidimensional approach to conceptualize the ISR domain. Such an approach iswell suited to Description Logics (DL) [7] languages such as OWL DL. One of the mostpowerful features of DLs is their ability to define classes in terms of sufficient and nec-essary conditions. New concepts can be defined by specifying property restrictions andrelations on existing concepts. As an example, consider the following DL definitionsfor some concepts relating to aerial sensor platforms:

– Aircraft ! (Platform " # hasRealm.Atmosphere)– UnmannedVehicle ! (Platform " # hasQuality.Without-crew-mobility)– UAV ! (Aircraft " UnmannedVehicle)– CombatUAV ! (UAV " # providesCapability.Firepower)– MALE ! (UAV " # providesCapability.MediumAltitude " # providesCapabil-

ity.LongEndurance)– EnduranceUAV $ ¬ (SmallUAV % TacticalUAV).

By using OWL DL, we can apply off-the-self reasoners to infer new classificationsbased on concept definitions, which is very appropriate because different classificationsare useful for different purposes. For example at some point one might be interested inselecting a platform in terms of the type of tasks that it can perform (reconnaissance,surveillance, battle damage assessment, etc), but in other circumstances one might beinterested in selecting a platform according to its takeoff and landing capabilities (cata-pult, runaway, VTOL, etc). Some of these dimensions are:

– For platforms: mobility, realm, performance (range, endurance, altitude, speed,etc.), application or mission type (surveillance, reconnaissance, target acquisition,etc.), firepower, landing and takeoff, communications, vulnerability and survivabil-ity, availability.

– For sensors: phenomena detected (type and spectrum), performance (resolution,sample rate, etc), weather/terrain/contamination influence, vulnerability, interfer-ences with other sensors.

(a) Platforms (b) Sensors (c) ISR tasks (d) Intelligence

Fig. 5: Sample of concept taxonomies relevant to the ISR domain

Figure 5 shows a sample of some of the taxonomies we have developed for the ISRontology. Platforms (a) and sensors (b) are used to characterize ISR assets; the formerdraws on concepts from OntoSensor and CIMA, the latter from the MMI Platforms

using task type using platform features

Select platform

Page 50: An Ontology-Centric Approach to Sensor-Mission Assignment

Sensor-Task Fitting

Sensor-task fitting

MMF ontology

ISR ontologies: tasks, sensors, etc

Page 51: An Ontology-Centric Approach to Sensor-Mission Assignment

Sensor-Task Fitting

Sensor-task fitting

MMF ontology

• Given a task T, with a set of ISR requirements: RT = {R1, R2, ...}

ISR ontologies: tasks, sensors, etc

Page 52: An Ontology-Centric Approach to Sensor-Mission Assignment

Sensor-Task Fitting

Sensor-task fitting

MMF ontology

• Given a task T, with a set of ISR requirements: RT = {R1, R2, ...}

• A valid solution is

- a (PC) Platform Configuration = (P , S) where P is a single platform type ,

S = {S1, ... , Sm} is a set of sensor types mountable on P (at the same time).

- The aggregate capabilities of a PC have to semantically match RT

ISR ontologies: tasks, sensors, etc

Page 53: An Ontology-Centric Approach to Sensor-Mission Assignment

Sensor-Task Fitting

Sensor-task fitting

MMF ontology

• Given a task T, with a set of ISR requirements: RT = {R1, R2, ...}

• A valid solution is

- a (PC) Platform Configuration = (P , S) where P is a single platform type ,

S = {S1, ... , Sm} is a set of sensor types mountable on P (at the same time).

- The aggregate capabilities of a PC have to semantically match RT

• There can be more than one valid solution:

- Set of valid solutions = { PC1 , PC2 , ... }

ISR ontologies: tasks, sensors, etc

Page 54: An Ontology-Centric Approach to Sensor-Mission Assignment

Sensor-Task Fitting (contd)

• Three interlinked semantic matching relations:

Task Platform

Sensor

Page 55: An Ontology-Centric Approach to Sensor-Mission Assignment

Sensor-Task Fitting (contd)

• Three interlinked semantic matching relations:

Task Platform

Sensor

match(T,S)

Page 56: An Ontology-Centric Approach to Sensor-Mission Assignment

Sensor-Task Fitting (contd)

• Three interlinked semantic matching relations:

Task Platform

Sensor

match(T,P)

match(T,S)

Page 57: An Ontology-Centric Approach to Sensor-Mission Assignment

Sensor-Task Fitting (contd)

• Three interlinked semantic matching relations:

Task Platform

Sensor

match(T,P)

match(T,S) match(P,S)

Page 58: An Ontology-Centric Approach to Sensor-Mission Assignment

Sensor-Task Fitting (contd)

• Three interlinked semantic matching relations:

Task Platform

Sensor

match(T,P)

match(T,S) match(P,S)

Ontologies

Page 59: An Ontology-Centric Approach to Sensor-Mission Assignment

Sensor-Task Fitting (contd)

• Three interlinked semantic matching relations:

Task Platform

Sensor

match(T,P)

match(T,S) match(P,S)

Ontologies

• All matching relations have to be satisfied for a valid solution.

Page 60: An Ontology-Centric Approach to Sensor-Mission Assignment

Deployment

Deployment

& Tasking

ISR assets available

Fitting

Reasoner

Allocation

algorithms

Monitoring

Day & night video

surveillance Locate snipers

in the wood

Requirements

- Capabilities

- QoI

- Environment

Assets

- Type

- Location

- Status

Ranking feasible

solutions

Near-optimal

allocation

A BD

E

C

Sensor

Infrastructure

("Fabric")

S1

S2

S3

S4

Planning

Logistics

Dynamic events

Update status

SENSOR-TASK

FITTING

SENSOR-TASK

ALLOCATION

SENSOR

DEPLOYMENT

AND DELIVERY

Page 61: An Ontology-Centric Approach to Sensor-Mission Assignment

Deployment

Deployment

& Tasking

ISR assets available

Fitting

Reasoner

Allocation

algorithms

Monitoring

Day & night video

surveillance Locate snipers

in the wood

Requirements

- Capabilities

- QoI

- Environment

Assets

- Type

- Location

- Status

Ranking feasible

solutions

Near-optimal

allocation

A BD

E

C

Sensor

Infrastructure

("Fabric")

S1

S2

S3

S4

Planning

Logistics

Dynamic events

Update status

SENSOR-TASK

FITTING

SENSOR-TASK

ALLOCATION

SENSOR

DEPLOYMENT

AND DELIVERY

Page 62: An Ontology-Centric Approach to Sensor-Mission Assignment

Sensor-Task Allocationassign individual asset instances

to competing tasks.

XTask 1

XTask 3

XTask 2

Page 63: An Ontology-Centric Approach to Sensor-Mission Assignment

Sensor-Task Allocationassign individual asset instances

to competing tasks.

XTask 1

XTask 3

XTask 2

Page 64: An Ontology-Centric Approach to Sensor-Mission Assignment

• This allocation problem can be represented as a bipartite graph:

‣ Vertex sets:Assets {Ai}, Task {Tj}

‣ For each task (Tj): dj = utility demand pj = profit

‣ For each asset-task pair: eij = utility that Ai could contribute to Tj

• Goal: an allocation of asset that maximizes the total profit of the satisfied tasks.

• A task is satisfied when:

• This is an NP-Complete problem!

• We developed centralized and distributed algorithms to find an approximate solution.

Allocation model

of the satisfied tasks, where!(Ai,Tj)!F eij ! dj

A1

A2

A3

A4

T1

T2

Assets

Tasks

e11

(d1, p1)

(d2, p2)

e12

Page 65: An Ontology-Centric Approach to Sensor-Mission Assignment

Quantitative Vs Qualitative

• It needs a numerical utility for each (task, asset).

• Usually utility value includes only QUANTITATIVE measures of sensing capabilities.

• Example: Utility is a function of the distance Dij between assset and task

• In our approach we include also QUALITATIVE measures of capabilities:

• Two results:

• Reduce the solution search space (and therefore the complexity).

• Better solution quality.

eij =

!"

#

11+D2

ij/cif Dij ! SR

0 otherwise

eij = 0 if there is no match between RT and CA

Page 66: An Ontology-Centric Approach to Sensor-Mission Assignment

More flexibility

• Another advantage of using semantic matchmaking:

The fitting output is a set of possible asset types which we can use for each task.

• If a certain asset type becomes unavailablethe task can still be accomplished by usign alternative asset types.

XTask 1

XTask 3

XTask 2

Page 67: An Ontology-Centric Approach to Sensor-Mission Assignment

More flexibility

• Another advantage of using semantic matchmaking:

The fitting output is a set of possible asset types which we can use for each task.

• If a certain asset type becomes unavailablethe task can still be accomplished by usign alternative asset types.

XTask 1

XTask 3

XTask 2

Page 68: An Ontology-Centric Approach to Sensor-Mission Assignment

More flexibility

• Another advantage of using semantic matchmaking:

The fitting output is a set of possible asset types which we can use for each task.

• If a certain asset type becomes unavailablethe task can still be accomplished by usign alternative asset types.

XTask 1

XTask 3

XTask 2

‣ Matches both ISR requirements of Task 3 and of Task 2

‣ ... but Task 3 is critical

Page 69: An Ontology-Centric Approach to Sensor-Mission Assignment

More flexibility

• Another advantage of using semantic matchmaking:

The fitting output is a set of possible asset types which we can use for each task.

• If a certain asset type becomes unavailablethe task can still be accomplished by usign alternative asset types.

XTask 1

XTask 3

XTask 2

‣ Matches both ISR requirements of Task 3 and of Task 2

‣ ... but Task 3 is critical

Page 70: An Ontology-Centric Approach to Sensor-Mission Assignment

Deployment

Deployment

& Tasking

ISR assets available

Fitting

Reasoner

Allocation

algorithms

Monitoring

Day & night video

surveillance Locate snipers

in the wood

Requirements

- Capabilities

- QoI

- Environment

Assets

- Type

- Location

- Status

Ranking feasible

solutions

Near-optimal

allocation

A BD

E

C

Sensor

Infrastructure

("Fabric")

S1

S2

S3

S4

Planning

Logistics

Dynamic events

Update status

SENSOR-TASK

FITTING

SENSOR-TASK

ALLOCATION

SENSOR

DEPLOYMENT

AND DELIVERY

Page 71: An Ontology-Centric Approach to Sensor-Mission Assignment

Deployment

Deployment

& Tasking

ISR assets available

Fitting

Reasoner

Allocation

algorithms

Monitoring

Day & night video

surveillance Locate snipers

in the wood

Requirements

- Capabilities

- QoI

- Environment

Assets

- Type

- Location

- Status

Ranking feasible

solutions

Near-optimal

allocation

A BD

E

C

Sensor

Infrastructure

("Fabric")

S1

S2

S3

S4

Planning

Logistics

Dynamic events

Update status

SENSOR-TASK

FITTING

SENSOR-TASK

ALLOCATION

SENSOR

DEPLOYMENT

AND DELIVERY

Page 72: An Ontology-Centric Approach to Sensor-Mission Assignment

Sensor deployment & delivery

configuring and deploying sensing assets, ensuring the right information is efficiently delivered

to the user.

assumption that the degree d of the problem instance (the maximum number of assetswithin range of any task) is limited, however, a simple greedy algorithm can guaranteea d-approximate solution. An alternative, knapsack-like formulation, SUM, has beendescribed in [13].

5 Fabric for sensor deployment and deliveryWe are currently working on the problem of deploying selected sensor instances on thenetwork, so that they operate as a coherent system that can deliver the required infor-mation to users. Our main focus at present is to interface the fitting and allocation com-ponents with a particular sensor infrastructure — the Sensor Fabric — a prototype im-plementation of which has been built using commercial off-the-shelf components [14].While the main aim of the fabric is to research and apply algorithms for the retrievaland dissemination of task-specific information across sensor networks, it provides threemain components that complement the fitting and allocation work previous described:1. Sensor Catalogue: provides a global inventory/registry of sensor instances, describ-

ing all known assets and their availability. It manages the visibility of, and accessto, sensor data feeds. The Sensor Catalogue is used to select the sensors required tofulfill the requirements of a specific task.

2. Topology Manager: manages the network topology and inter-node communication,describing all known network nodes and their availability. The Topology Manageris used for node location and routing information.

3. Fabric Manager: responsible for the control (configuration and sensor-mission as-signment) and monitoring of individual sensors and intermediate nodes, and es-tablishing the communication channels between them. The Fabric Manager alsoprovides a container for running in-network information fusion and filtering algo-rithms, and registering them as assets with the Sensor Catalogue.Figure 9 sketches the Sensor Fabric

architecture and the three main servicesprovided: the Sensor Catalogue (SC), theTopology Manager (TM), and the Fab-ric Manager (FM). There is one instanceeach of the Sensor Catalogue and Topol-ogy Manager per Sensor Fabric, and oneinstance of the Fabric Manager per node.Sensor nodes publish data locally to atopic in a global topic name space, andsimilarly consumers subscribe locally tothe same global topic name. The Fab-ric establishes the communication chan-

A

B E

FC

SC TM

FM FM

FM

FM

D

FM

S1

S2

S3

S4

S5

S6

Fig. 9: Overview of the Fabric

nel between the two automatically, and ensures that published messages are deliveredcorrectly. In this example data published from sensors S1, S2, and S3 on node E isaccessed by the client on node A as if it is local. The Fabric transparently delivers thedata across the network between the nodes, E to B to A.

We have defined interfaces that allow the fitting and allocation components to callon the Sensor Catalogue to obtain several kinds of information, including: availableasset types (for fitting), available asset instances (for allocation), and current status ofassets (obtained from the Fabric Manager and providing, for example, data to be used as

Page 73: An Ontology-Centric Approach to Sensor-Mission Assignment

• We need a sensor infrastructure:

• Mediate between real world assets and fitting/allocation modules

• Providing an high level interface to

configure and deploy assets & ensure efficient information delivery.

• “Sensor Fabric” a prototype implementation:

Sensor fabric

A

B E

FC

SC TM

FM FM

FM

FM

D

FM

S1

S2

S3

S4

S5

S6

Page 74: An Ontology-Centric Approach to Sensor-Mission Assignment

• We need a sensor infrastructure:

• Mediate between real world assets and fitting/allocation modules

• Providing an high level interface to

configure and deploy assets & ensure efficient information delivery.

• “Sensor Fabric” a prototype implementation:

• Sensor Catalogue:

- Asset instances and their ISR capabilities.

(supporting the SENSOR TASK FITTING)

Sensor fabric

A

B E

FC

SC TM

FM FM

FM

FM

D

FM

S1

S2

S3

S4

S5

S6

Page 75: An Ontology-Centric Approach to Sensor-Mission Assignment

• We need a sensor infrastructure:

• Mediate between real world assets and fitting/allocation modules

• Providing an high level interface to

configure and deploy assets & ensure efficient information delivery.

• “Sensor Fabric” a prototype implementation:

• Sensor Catalogue:

- Asset instances and their ISR capabilities.

(supporting the SENSOR TASK FITTING)

• Topology Manager:

- Network topology.

Sensor fabric

A

B E

FC

SC TM

FM FM

FM

FM

D

FM

S1

S2

S3

S4

S5

S6

Page 76: An Ontology-Centric Approach to Sensor-Mission Assignment

• We need a sensor infrastructure:

• Mediate between real world assets and fitting/allocation modules

• Providing an high level interface to

configure and deploy assets & ensure efficient information delivery.

• “Sensor Fabric” a prototype implementation:

• Sensor Catalogue:

- Asset instances and their ISR capabilities.

(supporting the SENSOR TASK FITTING)

• Topology Manager:

- Network topology.

• Fabric Manager:

- Allocate asset instances to tasks (SENSOR-TASK ALLOCATION)

- Establish communication channel (DELIVERY)

Sensor fabric

A

B E

FC

SC TM

FM FM

FM

FM

D

FM

S1

S2

S3

S4

S5

S6

Page 77: An Ontology-Centric Approach to Sensor-Mission Assignment

• Evaluation: we have implemented a first prototype of a Top-to-Bottom application

for Sensor-Mission Assignment.

Conclusion & Futurepart of the computation of a utility value). Then, an interface to the Topology Managerallows us to specify how a set of selected instances needs to be configured to operate asa coherent system to deliver data to a user.

Fig. 10: Tool for integrating the reasoner and the Sensor Fabric

Figure 10 is a screenshot of a software prototype we are developing to enable theintegration of the services provided by the reasoner and those provided by the SensorFabric. This screenshot shows the main user interface, which enables the specificationof tasks, and supports users in the allocation of appropriate assets for their their tasks.Each task is characterised by qualitative requirements used by the reasoner to discoverplatforms that are fit-for-purpose. In addition, each task can define a geographic area ofinterest where some information has to be collected. The left part of the interface showsa map provided by Google Map Web services8, where the user can specify the area ofinterest (overlapping rectangle) for a particular task. After selecting some requirementsfor the task, the user can retrieve the assets available that satisfy them, together withinformation on their availability and geographic location. In addition, the location ofthe assets is depicted on the map. In the example, we can see that user needs ImageryIntelligence (IMINT), which can be provided by three assets: one UAV of class Predator(P1) and two UAVs of class PredatorB (P1 and P5). In addition, we can see that oneof the PredatorB platforms is not available, because it has crashed previously (P8).Therefore, in this case the user can select either P1 or P5 to accomplish the task.Multiple tasks can be defined, and then the assets committed to one task will not beavailable to other tasks. The next step is to integrate the allocation algorithms so as tomaximize the global profit from multiple tasks.

6 Discussion and ConclusionAt present, we have initial implementations of the complete set of ontologies describedin Section 2, a prototype of the semantic reasoner that performs the sensor-task fitting,

8 http://code.google.com/apis/maps/

• Positive feedback from US and UK users:

‣ liked the use of MMF and previous ISR ontologies/schemas.

‣ liked the multi-dimensional approach.

Page 78: An Ontology-Centric Approach to Sensor-Mission Assignment

• Evaluation: we have implemented a first prototype of a Top-to-Bottom application

for Sensor-Mission Assignment.

Conclusion & Futurepart of the computation of a utility value). Then, an interface to the Topology Managerallows us to specify how a set of selected instances needs to be configured to operate asa coherent system to deliver data to a user.

Fig. 10: Tool for integrating the reasoner and the Sensor Fabric

Figure 10 is a screenshot of a software prototype we are developing to enable theintegration of the services provided by the reasoner and those provided by the SensorFabric. This screenshot shows the main user interface, which enables the specificationof tasks, and supports users in the allocation of appropriate assets for their their tasks.Each task is characterised by qualitative requirements used by the reasoner to discoverplatforms that are fit-for-purpose. In addition, each task can define a geographic area ofinterest where some information has to be collected. The left part of the interface showsa map provided by Google Map Web services8, where the user can specify the area ofinterest (overlapping rectangle) for a particular task. After selecting some requirementsfor the task, the user can retrieve the assets available that satisfy them, together withinformation on their availability and geographic location. In addition, the location ofthe assets is depicted on the map. In the example, we can see that user needs ImageryIntelligence (IMINT), which can be provided by three assets: one UAV of class Predator(P1) and two UAVs of class PredatorB (P1 and P5). In addition, we can see that oneof the PredatorB platforms is not available, because it has crashed previously (P8).Therefore, in this case the user can select either P1 or P5 to accomplish the task.Multiple tasks can be defined, and then the assets committed to one task will not beavailable to other tasks. The next step is to integrate the allocation algorithms so as tomaximize the global profit from multiple tasks.

6 Discussion and ConclusionAt present, we have initial implementations of the complete set of ontologies describedin Section 2, a prototype of the semantic reasoner that performs the sensor-task fitting,

8 http://code.google.com/apis/maps/

• Positive feedback from US and UK users:

‣ liked the use of MMF and previous ISR ontologies/schemas.

‣ liked the multi-dimensional approach.

• MAIN MESSAGE:

The Ontology centric approach is crucial

- to cope with the heterogeneity of the sensor network and of tasks,

- to provide flexibility.

Page 79: An Ontology-Centric Approach to Sensor-Mission Assignment

• Evaluation: we have implemented a first prototype of a Top-to-Bottom application

for Sensor-Mission Assignment.

Conclusion & Futurepart of the computation of a utility value). Then, an interface to the Topology Managerallows us to specify how a set of selected instances needs to be configured to operate asa coherent system to deliver data to a user.

Fig. 10: Tool for integrating the reasoner and the Sensor Fabric

Figure 10 is a screenshot of a software prototype we are developing to enable theintegration of the services provided by the reasoner and those provided by the SensorFabric. This screenshot shows the main user interface, which enables the specificationof tasks, and supports users in the allocation of appropriate assets for their their tasks.Each task is characterised by qualitative requirements used by the reasoner to discoverplatforms that are fit-for-purpose. In addition, each task can define a geographic area ofinterest where some information has to be collected. The left part of the interface showsa map provided by Google Map Web services8, where the user can specify the area ofinterest (overlapping rectangle) for a particular task. After selecting some requirementsfor the task, the user can retrieve the assets available that satisfy them, together withinformation on their availability and geographic location. In addition, the location ofthe assets is depicted on the map. In the example, we can see that user needs ImageryIntelligence (IMINT), which can be provided by three assets: one UAV of class Predator(P1) and two UAVs of class PredatorB (P1 and P5). In addition, we can see that oneof the PredatorB platforms is not available, because it has crashed previously (P8).Therefore, in this case the user can select either P1 or P5 to accomplish the task.Multiple tasks can be defined, and then the assets committed to one task will not beavailable to other tasks. The next step is to integrate the allocation algorithms so as tomaximize the global profit from multiple tasks.

6 Discussion and ConclusionAt present, we have initial implementations of the complete set of ontologies describedin Section 2, a prototype of the semantic reasoner that performs the sensor-task fitting,

8 http://code.google.com/apis/maps/

• Future:

• Continue to develop each module and

• to study interactions between them.

• Positive feedback from US and UK users:

‣ liked the use of MMF and previous ISR ontologies/schemas.

‣ liked the multi-dimensional approach.

• MAIN MESSAGE:

The Ontology centric approach is crucial

- to cope with the heterogeneity of the sensor network and of tasks,

- to provide flexibility.

Page 80: An Ontology-Centric Approach to Sensor-Mission Assignment

Thanks for listening