04/10/2006

1

A System Theoretical Approach to Situation Awareness and its application

A Holistic View of Purposeful Elements

Charlie LagervikPer Gustavsson

Cambridge, September 27, 2006

Student

04/10/2006

2

Objectives

• Address Situational Awareness

• Concept of Situational Awareness

• Machine human context

• Example of our usage

• Our continued work

• A glimpse of ideas in building Situation Awareness

04/10/2006

3

Outline

• Why want a theoretical framework for Situational Awareness?

• Theory:• Ackoff - Adaptive Management• Bedny & Meister - Orientational Activity

• Lagervik conceptual design for the common part of SA, • Architecture that uses the idea:• An Implementation

• Costal Control Application

• Summarization• Future Work• Questions

04/10/2006

4

Transformation in defense operations Not in paper

04/10/2006

5

Requirements

• Net-Centric

• Adaptive

• Multi User

• Multi Role

• Information Fusion

• Knowledge

• Interoperability

• Decision Support

• Temporal

• Networked

Not in paper

04/10/2006

6

I-Terminal

I-Terminal

I-Terminal

I-Terminal

I-Terminal

I-Terminal

EU

OwnOwn

CivilAuthority

CivilAuthority

Service

Own

I-Terminal

I-Terminal

Net-Centric Technical Networks enables Human Networks

Not in paper

04/10/2006

7

DecideDecide

OrientOrient ActAct

ObserveObserve

DecideDecide

OrientOrient ActAct

ObserveObserve

DecideDecide

OrientOrient

ObserveObserve

ActAct

DecideDecide

OrientOrient

ObserveObserve

ActAct

Decide

Orient

Observe

Act

Decide

Orient

Observe

Act

Decide

Orient Act

Observe

Decide

Orient

Observe

Act

Decide

Orient

Observe

Act

Decide

Orient

Observe

Act

Decide

Orient

Observe

Act

Decide

Orient Act

Observe

Decide

Orient

Observe

Act

Decide

Orient

Observe

Act

Decide

Orient

Observe

Act

Decide

Orient

Observe

Act

Decide

Orient Act

Observe

Decide

Orient

Observe

Act

Decide

Orient

Observe

Act

Decide

Orient

Observe

Act

Decide

Orient

Observe

Act

0

1

2

3

4

0

1

2

3

4

0

1

2

3

4Data

KnowledgeInformation

Data

KnowledgeInformation

Data

KnowledgeInformation

Joint / Coalition / Interservice

Asymmetric – Constantly change only stable state

Not in paper

04/10/2006

8

Background

• Our interest comes from • Want to supply commander with Intention Awareness

• Need a system that can execute multi-hypothesis of COA from previous, present and predicted states/situations

• Need models that are adaptive, goal oriented

• Models need internal representation of state, mission, vision, goal

• Models need to be able to exchange information

• Models need to have some internal representation of the environment

• Need an architecture

• Realized that such representation of situation also is needed for operational systems.

• Thought that if we have such representation present in the M&S domain the same representation can be used in the operational domain.

Not in paper

04/10/2006

9

Definitions

Decision-Making with Act• A decision is made to make something occur that should not else

occur or to make something not to occur that should else occur• Expected result in a certain time intervall• Based on assumptions, information, feelings• Conscious vs. unconscious

Situational Awareness• The conscious and unconscious dynamic processes to achieve a

reflection of the systems purpose and form, function, system state and future system states

Intention Awareness• The sub part of Situation Awareness that express the path between

current state and goal state for own and opponents’.

04/10/2006

10

The change of change

• Constant change makes it harder to make decisions• Uncertainty in what to look for• Past information that were valid then is not guaranteed for decision-making

now

Myth: The more information the decision maker has, the better decisionis made .

Information Overload• Filtering • Limited time -> Filtering delegated to other people • Limited time -> computer filtration• But computer has no understanding for the situation of the data or the

situation of the decision-making

04/10/2006

11

JDL-Model (altered)

Signal

Object

Situation

Intention

Control

Human

information fusion, data mining and data warehousing

Database

Queries

Human

CategorizationIdentificationGroupingAggregationSummarizationCorrelation…

04/10/2006

12

Understanding / Human

Patterns / Computer Based

Signal

t+0, position ± offsett+1, position ± offset

…t+n, position ± offset

t+n, Position ± offset

Object

SituationObjA

t+0, pos ± oft+1, pos ± of

…t+n, pos ± of

ObjBt+0, pos ± oft+1, pos ± of

…t+n, pos ± of

ObjCt+0, pos ± oft+1, pos ± of

…t+n, pos ± of

IntentionObjA

t+0, pos ± oft+1, pos ± of

…t+n, pos ± of

ObjBt+0, pos ± oft+1, pos ± of

…t+n, pos ± of

ObjCt+0, pos ± oft+1, pos ± of

…t+n, pos ± of

Human

04/10/2006

13

ManualSemiAuto

Semi

Increased information uncertainty

Increased scenario complexity

Manual

Manual

Orient

Observe

Situation Awareness

Semi

04/10/2006

14

ManualAuto

Increased information uncertainty

Manual

Orient

Observe

Situation Awareness

Semi

Increased scenario complexity

ManualSemiAuto

Manual

Semi

04/10/2006

15

Automated

• Computers will be in the position of deciding what to look for. • Need understanding of the nature of the decision and what will effect the

decision.

• Computer must be able to reason about the situation.

• Reasoning systems: Goal seeking systems. • The properties of the elements are monitored and are used to understand

the goal of the element. The understanding about how the different properties of the environment and other elements in the system affect the goal of the element is the main concern

04/10/2006

16

Social Purposeful System

• Humans are non-deterministic systems

• A system that at a first glace seems to be totally mechanical and deterministic often involves humans as decision makers, for a part of the system or for the whole system.

• So to understand the system the goal of the human has to be determined and evaluated in the process.

• A social system should be described as a social system with models indented for this description.

04/10/2006

17

How to describe Situation Awareness

• System Theory• Adaptive-Learning (Ackoff )

• intended for describing the internal process for an organization to reach SA

• Orientational-Activity (Bedny & Meister)• intended for describing the internal process for a person to reach SA

• Even though the difference in fields there are common aspect according to SA that can be compared.

• The comparison is focusing on these parts that are common for both of the theories (the intersection of the theories).

04/10/2006

18

(D)Decision-making

subsystem

(F)Diagnosis and

prescription

(E)Memory and comparator

(G)Symptom and

presymptom analyzer

(C)Information subsystem

(B) Environment

(A)Organization

(1)Data

(4)Inquire

(2) Information

(3) Requests

(5)Instructions

(6) Decision record

(7) Instructions on information required

(8)Assumption -

relevantinformation

andactual

performance

(9) Deviants

(10a) Change

(10d) Change

(10c) Change

(16) Change

(11) Performanceindicators

(12) Symptoms andpresymptoms

(10b) Change

(13) Threats andopportunities

(14) Diagnostic andprescriptive record

(15)Instructions on

informationrequired

(17) Threats andopportunities

Theory:Adaptive ManagementOrientational ActivityConceptual SA design

Architecture:ImplementationSummarization

Adaptive-learning management system, after Ackoff

04/10/2006

19

Theory:Adaptive ManagementOrientational ActivityConceptual SA design

Architecture:ImplementationSummarization

Past experience Criteria ofevaluation

Making a decsion andperformance

Orienting a explorativeaction

Meaning of inputinformation Evaluative and inducing components of

motivation

Sense(significance) Motivation

Subjectively relevant tasksconditions

Image-goal

Conceptual model

Operativeimage

Situationawareness

1

2

3

4

5

678

functional model of orientational activity, after Bedny & Meister

04/10/2006

20

Orientationalactivity

(Bedny & Meister)

Management information

(Ackoff)

Conceptualsystem

Meaning of input information Information subsystem Data gateway

Evaluative and inducingcomponents of

motivationMemory and comparator Evaluative system

Conceptual model, pastexperience Memory and comparator Memory

Image-goal (Decision-makingsubsystem) Goal

Making a decision and performance

Decision-makingsubsystem Perform

Subjectively relevant tasks conditions

Symptom and presymptomanalyzer, Diagnosis and

prescriptionDiagnostic

Theory:Adaptive ManagementOrientational ActivityConceptual SA design

04/10/2006

21

Systemof

Interest

Perform

GoalDataGateway

EvaluateSystem

Diagnostic

Environment

Memory

Data

Data

Info req.

04/10/2006

22

Perform

GoalDataGateway

EvaluateSystem

Diagnostic

Memory

Data

Info req.Perform

Goal DataGateway

EvaluateSystem

Diagnostic

EnvironmentMemory

Data

Info req.

Connection between two systems

• Interchange of data, information, awareness, knowledge, wisdom

Orient

Decide

Act

Observe

Systemin

Focus

Systemin

Focus

Other

Orient

Decide

Act

Observe

04/10/2006

23

Information SharingInteroperability

Network / Connectivity

Simulation /Implementation

Modeling / Abstraction

Data

Information

Knowledge

Awareness

Wisdom

Level 0No Interoperability

Level 1Technical Interoperability

Level 2Syntactic Interoperability

Level 3Semantic Interoperability

Level 4Pragmatic Interoperability

Level 5Dynamic Interoperability

Level 6Conceptual Interoperability

04/10/2006

24

Multi-Hypothesis Intention Simulation Agent Architecture

Database

*ML

Doctri

ne Protocol

Representation

*MLDoctri

ne Protocol

Representation

n

fe

n

f e

n

f

e

n

f e

e nf

f

f

f f

ff

Queries

ECA-Rules Object Hypothesis

Situation Hypothesis

Signal Hypothesis

Intention HypothesisK-10d

K-10hK

K+10hK+10d

Fusion that are derived from doctrinal patternsUser/Agent state that this doctrinal pattern existsUser/Agent “discover” new patterns

New patternsSimulation

Simulation

refinementrefinement

Hypothesis

Sensor

04/10/2006

25

Onotlogy (other representation of SA)

Agent

hasInput

Information ControlCommandcommands

ObjectPropertyDatatypePropertySubClass/Property

Simulated NewFusion

controls

agentName

hasOutput

Data

name

value

hasLiaison

04/10/2006

26

Agent

Information

Control

Command

Simulated NewFusion

Data

04/10/2006

27

Costal Control

Radar Directionfinder TKM BD

OpenSIS

AIS UW C2 Sim

04/10/2006

28

Future Interest

• Memory: shared memory between SA component and decision-maker. • Representation of information, LCIM , images, C-BML

• Unconscious process, the N component finding new relations, goals, similarities etc

• Requiring new Data/Information. Dynamic and Conceptual Interoperability

• Curiosity. The computer should not stop to execute just because there are no instructions to execute.

• Motivation and sense. Motivation is regulating the energy of achieving the goal, and sense is responsible for determine the priority (significance) of the goal for the system.

• Sensor feedback. the importance and how the data is used and how the sensors should be design to support the given feedback of the gathered data.• also need sensor system with ability to make use of such information

04/10/2006

29

In Focus

• Representation• Conceptual representation of the belief process (OWL-S, UML)

• Semantic, Pragmatic representation C-BML• Web services, grammar, ontology for orders and reports

• Syntactic, Semantic representation JC3IEDM, EXDL

• Implement more fusion algorithms

• Experimentation

• Qualitative VV&A

04/10/2006

30

Summarization

• The Lagervik System Theoretical Approach to Situation Awareness helps us to think

• Developing• Information fusion framework (MuHISA)• Costal application

• Future• Representation• More fusion algorithms• Qualitative VV&A

04/10/2006

31

Questions?Charlie.Lagervik & Per.M.Gustavsson

Enviroment

DataGateway

Evaulation

Memory

Goal

Perform Diagnostic

Systemof

interest

Data

Data

IntepretationReflection

Deviants

Critera of evalation

Pastexperience

Information

Definitions

T &O

Input ongoal

Motivation

Goal

Prefomance indicators/data (external)

InquireInquire

Diagnostic and prescriptive recordDerivants/data(internal)

Instruction on information required

Request of information

InformationRequest information

Information

Criteria of evaluation

Xxxx Data

Function block Association Change Instruction System

Xxxx

04/10/2006

32

www.saabgroup.com

04/10/2006

33

Ontology

Service

Capability

hasCapabilityCapabilityLevel

PersonnelInformation ControlLiaisonEconomyTrainingInfrastructureCommandLogisticImpact Delivery

capabilityName

capabilityLevelName

capabilityLevelDescription

hasCapabilityLevel

hasRecursiveCapability

ObjectPropertyDatatypePropertySubClass/Property

Land Sea Air Info