1

SHARED SITUATION AWARENESS FORNETWORKED DISMOUNTED SOLDIERS

Kevin Barry and *Craig StonekingLockheed Martin Advanced Technology Laboratories

Cherry Hill, NJ 08002{kbarry, cstoneki}@atl.lmco.com

Cyndi CarpenterU.S. Army, RDECOM CERDEC

Fort Monmouth, NJCyndi.Carpenter@us.army.mil

Karen GoselinVitronics Inc., an MTC Technologies Company

Eatontown, NJ 07724Karen.Goselin@us.army.mil

ABSTRACT

The effectiveness of the network-edge dismountedsoldier is maximized when the soldier acts as both aconsumer and producer of Situation Awareness (SA) datawithin the larger tactical network. Providing this degreeof information sharing in a tactical military environmentpresents unique technical challenges. This paper willdiscuss the design of a prototype demonstration system,jointly planned by the Army’s Electronics ResearchDevelopment and Engineering Center (CERDEC) andLockheed Martin’s Advanced Technology Laboratories(ATL), that integrates CERDEC’s C2MINCS tacticalinformation sharing framework, and ATL’s data fusionand Grapevine intelligent data dissemination technologiesto more fully address the problem of bringing sharedsituation awareness to a team of networked dismountedsoldiers.

1. INTRODUCTION

The effectiveness of the network-edge dismountedsoldier is maximized when the soldier acts as both aconsumer and producer of Situation Awareness (SA) datawithin the larger tactical network. As force-multiplyingtechnologies increase the capabilities of the individual, adismounted soldier may increasingly encounter a view ofthe battlefield that is unique compared to that of otherteam members. This is particularly true in an urbanenvironment, where the view of the battlefield maychange greatly with just the turn of a corner. However, fora team to perform effectively, each individual needs tohave awareness of all areas of the battlefield that arerelevant to the individual and the team’s security andmission objectives. With network connectivity extendingto the front-line dismounted soldier, the opportunity existsfor the individual soldier to benefit from informationavailable from the tactical network and also to act as asensor, providing information back to the network andbenefiting teammates and echelons above.

Providing this degree of information sharing presentsunique technical challenges. The tactical military operatesin an environment that is hostile to computing platformsand networks. With the new wealth of data available, thewarfighter is in danger of suffering information overloadwithout some intelligent, automated means of selecting,managing and presenting the information relevant to theindividual. In order for the warfighter to be an effectiveprovider as well as consumer of SA information, aconvenient, intuitive and effective interface is required.

Recent work by the Army’s CommunicationsElectronics Research Development and EngineeringCenter (CERDEC) has resulted in a demonstrationprototype system that provides an information sharingframework in which soldiers, carrying Personal DigitalAssistant computing devices can share situation aware-ness data with both peers and echelons above. Indepen-dently, Lockheed Martin Advanced TechnologyLaboratories (ATL) has for several years been maturingsensor data fusion and intelligent information dissemina-tion technologies that support situation awareness intactical environments. This paper will discuss the designof a prototype demonstration system, jointly planned byCERDEC and ATL, that integrates the capabilities fromboth organizations to address the problem of bringingshared situation awareness to a team of networkeddismounted soldiers. After discussing the operationalcontext of the planned system, the paper will discuss theindividual constituent technologies, followed by adescription of the planned demonstration system as anintegrated whole.

2. OPERATIONAL CONTEXT

The operational context for the planned system,illustrated in Fig. 1, includes a small team of dismountedsoldiers, accompanied by a Multi-purpose Utility/Logistics Equipment (MULE) vehicle. Dismountedsoldiers carry handheld devices that provide a personaltactical situation display, data entry capability, limited

Report Documentation Page Form ApprovedOMB No. 0704-0188

Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering andmaintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information,including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, ArlingtonVA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if itdoes not display a currently valid OMB control number.

1. REPORT DATE 00 DEC 2004

2. REPORT TYPE N/A

3. DATES COVERED -

4. TITLE AND SUBTITLE Shared Situation Awareness For Networked Dismounted Soldiers

5a. CONTRACT NUMBER

5b. GRANT NUMBER

5c. PROGRAM ELEMENT NUMBER

6. AUTHOR(S) 5d. PROJECT NUMBER

5e. TASK NUMBER

5f. WORK UNIT NUMBER

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) Lockheed Martin Advanced Technology Laboratories Cherry Hill, NJ08002; U.S. Army, RDECOM CERDEC Fort Monmouth, NJ

8. PERFORMING ORGANIZATIONREPORT NUMBER

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR’S ACRONYM(S)

11. SPONSOR/MONITOR’S REPORT NUMBER(S)

12. DISTRIBUTION/AVAILABILITY STATEMENT Approved for public release, distribution unlimited

13. SUPPLEMENTARY NOTES See also ADM001736, Proceedings for the Army Science Conference (24th) Held on 29 November - 2December 2005 in Orlando, Florida. , The original document contains color images.

14. ABSTRACT

15. SUBJECT TERMS

16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF ABSTRACT

UU

18. NUMBEROF PAGES

6

19a. NAME OFRESPONSIBLE PERSON

a. REPORT unclassified

b. ABSTRACT unclassified

c. THIS PAGE unclassified

Standard Form 298 (Rev. 8-98) Prescribed by ANSI Std Z39-18

2

FBCB2

Fig. 1. The context for the demonstration system includesa small team of PDA-equipped dismounted soldiers,

networked with a MULE.

computational power, and local network connectivity withthe MULE and other team members. The MULE providesgreater computational resources and network connectivityto higher-echelons. Through the MULE, the team hasaccess to the larger Tactical Internet and associatedinformation systems such as the Force XX1 BattleCommand Brigade and Below (FBCB2) system.

3. C2MINCS

The underlying information sharing framework andthe soldier interface are provided by the CERDEC-developed Command and Control Mobile Intelligent Net-Centric Computer System (C2MINCS). The C2MINCSapplication, shown in Fig. 2, provides real-time tacticalsituational data to the dismounted soldier, sending andreceiving red and blue force entities through the MULE toand from upper echelons, connected to BCB2. Standardmapping products, provided by the National Imagery andMapping Agency (NIMA) and MILSTD 2525B icons areused to display all situational data. Users can perform allcustomary map display functionality, including pan,zoom, and view icon details. This allows a commonoperational picture to be displayed at each node on thenetwork. Dismounted soldiers report their position andhave the ability to send Spot Reports. C2MINCS alsoprovides a Peer-to-Peer Whiteboard, or “John Madden”capability. It allows team members to collaborate informulation of on-the-move COA plans, and to shareideas remotely.

C2MINCS has been designed to exploit commercialsoftware technology appropriate for emerging handheldcomputing hardware. This design is based uponrequirements for an open distributed systems softwareframework for information sharing, and an extensibleapplication for dismounted warfighter tactical use. The

Fig. 2. The handheldC2MINCS interface allows soldiersto share real-time situation awareness, including “John

Madden” whiteboard drawings.

present design configuration matches the representativeoperational context of the Future Force in which mobilehandheld devices interoperate with the MULE andFBCB2.

Figure 3 depicts the C2MINCS tactical informationsharing architecture. The architecture follows a service-based, publish-subscribe paradigm with “push-based”content to support an open framework. Each subsystem,or service, encapsulates its responsibilities to provideautonomous functionality, producing a flexible design toallow extensibility in both features and implementation.C2MINCS is component driven, where each component isindependent and modularized. All data exchange usesW3C compliant XML. A standard parser is utilized toparse, read and create XML messages.

MULEFBCB2

FBCB2 Adapter

PASSServer

Red and BlueForce Positions

FBCB2Data

FBCB2 Data

Spots

Track

s

Tracks

Spots

Tracks

Spots

Spots

Tracks

Spots &Drawings

MobileDevice

C2SVR

C2MINCS

C2SVR

PassAdapter

C2MINCS

MobileDevice

Fig. 3. C2MINCS Architecture diagram.

3

A CORBA compliant server, residing on the MobileDevice (C2SVR), is responsible for sending and receivingmessages to and from the MULE. All client componentcommunication is accomplished using the C2SVR. Itmaintains the master subscriber list for all componentsresiding on the client and forwards messagesappropriately. The C2SVR is the sole client subscriber tothe PASS Server residing on the MULE.

The MULE contains a web-based PASS Server tomaintain subscribers and forward messages published byclients. Adapters are used to translate one message formatand connectivity into another, allowing multiple non-homogeneous systems to communicate with minimalimpact to each system.

The PASS Adapter is implemented as a CORBAServer to all Mobile Devices, maintaining a master list ofdevices connected to it. It implements a web-basedconnection to the PASS Server. This allows multipleadapters to be executing, minimizing a single point offailure and maximizing scalability.

The FBCB2 adapter implements a web-basedconnection to the PASS Server to send and receivemessages from the mobile devices. It converts XML datamessages to VMF messages and vice versa.

C2MINCS software is initially targeted for use onruggedized Personal Digital Assistants (PDAs) inconjunction with a Linux-based MULE-mountedcomputing platform. However, through DII-COEcompliance and an architectural paradigm emphasizingmodularity, extensibility and platform independence,C2MINCS software is intended for operation on a varietyof operating systems and hardware platforms, both legacyand future, and permit all or portions of its functionalityto be embedded in other systems.

4. MULTI-SENSOR DATA FUSION

The task of battlefield situation assessment requiresthe ability to take reports from a variety of sensors,(RADAR, Infrared, IFF, Spot Reports, etc.), and combinethem into a single composite view of the position andidentification of all of the targets, (tanks, aircraft, airdefense unites, etc.), within the battlespace. To illustrate,consider the situation depicted in Fig. 4. Figure 4 presentsa hypothetical battlefield situation display onto whichreports from two different sensors, S1 and S2, are plotted.Due to errors inherent in the sensor measurements, theplotted position of each sensor report really onlyrepresents the center of an ellipse that defines a region inwhich the actual position of the target is expected to lie,with some high confidence.

X

XO

O

PossibleRotorcraft

PossibleSAM

Legend:X - Report from sensor S1O - Report from sensor S2

Possible Rotorcraft

PossibleSAM

Fig. 4. Battlefield situation display with plots, (includingerror ellipses), from two sensors.

Given that the actual positions of the targets may besignificantly displaced from the sensor-reported positions,and that some targets may be invisible to some sensors, itmay be reasonable to interpret the sensor reports torepresent two, three, or four actual targets on thebattlefield. The problem of sensor data fusion is to choosethe best interpretation of the collection of available sensorreports.

In the case of our example it is apparent from therelationships between the error ellipses that the bestinterpretation of the scene is likely to be as depicted inFig. 5. Notice that the two center-most sensor reports, ortracks, were interpreted to represent a single actual target.By comparing the expected-error regions of the twosensor tracks, the expected error in the position of theresultant “fused” track has been greatly reduced. Noticealso that the corroboration between the two sensors, in theclassification of the target, results in a higher confidencein the classification of the fused track [Hofmann, 1997].The result of sensor data fusion is a single, de-clutteredrepresentation of the battlefield with every known targetplotted only once, with higher accuracy than could beachieved with any single sensor.

X

XO

O

XO

HighProbability

SAMLegend:X - Report from sensor S1O - Report from sensor S2 - Fused TrackOX

PossibleRotorcraft

Possible Rotorcraft

Fig. 5. The result of fusion is a de-cluttered display withhigher confidence in position and identification of targets

on the battlefield.

4

In such a simple case, it is not difficult for a human tointerpret the scene mentally, without the aid ofautomation. However, as the numbers of sensors andtargets increase to realistic values, the complexity of theproblem quickly increases to a level that requiresautomation.

From 1993 to 1999, ATL participated in the Army’sRotorcraft Pilot's Associate (RPA) Advanced TechnologyDemonstration program, sponsored by the Army AviationTechnology directorate (AATD) [Malkoff et al., 1996].ATL developed the multi-sensor Data Fusion system toprovide a fused track picture to the RPA pilot and theRPA decision aiding systems onboard AH-64/Dhelicopters. In the RPA Data Fusion system, datarepresenting as many as 200 battlefield entities, from 14different types of onboard and offboard sensors, iscorrelated and fused in real time into a consolidatedpicture of the battlespace. The RPA system, includingATL’s Data Fusion system, was successfully flightdemonstrated in 1999. Since then, ATL’s Data Fusionsystem has been applied to a number of additionalprograms including the Army’s AirborneManned/Unmanned Systems Technology –Demonstration (AMUST-D) [Jameson et al., 2002]program and the Survivability Planner Associate Rerouter(SPAR) program.

5. GRAPEVINE INTELLIGENT INFORMATIONDISSEMINATION

The Grapevine architecture [Jameson, 2001] wasoriginally developed by ATL for use on DARPA's SmallUnit Operations (SUO) program, and has been extendedas part of many other Army programs such DismountedWarrior C4I STO, and AMUST-D. It makes maximumuse of bandwidth for information sharing by providingeach node with a description of the information needs ofits peers, so that each node can selectively transmit onlythose bits of information that it understands to be of realvalue to its neighbors. By sharing relevant sensor data,each participant can build a common tactical picture thatis consistent between participants, and is as complete asthe sum of all participants’ information sources can makeit.

The implementation of the Grapevine architecture(Fig. 6) builds upon our previous work combining multi-sensor Data Fusion with intelligent agent software. Eachnode in the architecture contains a Data Fusion processthat fuses locally obtained data (from local sensors anddata sources) and data received from other peer nodes.The Grapevine Manager at each node manages theinterchange of data with peer nodes. For each peer node,it contains a Grapevine proxy agent that represents the

Grapevine Node 3

Grapevine Manager

Data Fusion

Grapevine Proxy Agent 1

Grapevine Proxy Agent 2

Grapevine Node 2

Grapevine Manager

Data Fusion

Grapevine Proxy Agent 1

Grapevine Proxy Agent 3

Grapevine Node 1

Grapevine Manager

Data Fusion

Grapevine Proxy Agent 2

Grapevine Proxy Agent 3

Peer-to-peer configuration

Needs-basedData Exchange

Fig. 6. Grapevine data dissemination architecture.

5

information needs and capabilities of that peer node. Asthe sensors or other sources on the platform generate localinformation, each grapevine agent evaluates thatinformation against the needs of the peer platform itrepresents for factors such as:• Sensor type: Data from remote sensors is sent only if

the recipient does not already have access to that data.• Mission Context: For example, the peer platform’s

mission may or may not require the knowledge offriendly tracks.

• Location: The peer platform may only need informationwithin a geographic or temporal/geographic radius.

• Coverage: The peer platform may need informationfrom outside its own field of view.

In addition, the Grapevine agents are aware of theprocessing and bandwidth limitations of the peer nodesand communication links. Data identified as relevant to apeer node based on the above criteria may be down-sampled or prioritized to meet resource limitations. EachGrapevine agent propagates the needed information to thepeer platform it represents, providing an intelligent pushof data through the network.

At the same time, the Grapevine Manager has arepresentation of the local platform’s information needsand capabilities, expressed in terms of available sensorsand data sources, mission, location, and sensor coverage.A Sentinel Agent within the Grapevine Manager monitorsthe local fused picture to identify information needs notmet by the local picture. Based on this, it sends outupdated configuration data for the local platform to theGrapevine Manager on peer platforms. This is used toupdate the Grapevine Agents on the peer platforms thatrepresent the local platform. This propagation ofinformation needs effects an intelligent pull of data tomeet the changing information needs of the localplatform.

There are several distinctive features of theGrapevine Architecture. First, it is a peer-to-peer archi-

tecture. Propagation of data occurs between peer nodes inthe network. This architecture is easily extended toaccommodate a hierarchical C2 control structure. Second,propagation is needs based—peer-to-peer data propaga-tion includes only data known to be of use to the recipientnode, thus limiting the required processing andbandwidth. Third, the architecture is extensible. It canaccommodate the addition of peer nodes merely byreconfiguring nearby nodes to reflect the addition of thenew nodes. Fourth, it is survivable—there is no singlepoint of failure. Since, in general, each node will havemultiple peers, data can spontaneously reroute aroundmissing nodes, and thus the loss of any single node willonly result in the loss of the data sources local to thatnode.

The result of this capability is to permit, in the face ofstringent bandwidth and processing constraints, thecreation of a User Defined Operating Picture (UDOP)across all participating platforms. The UDOP is a sharedpicture of the battlefield, with all participants having aconsistent view of the world, and each participant seeingthat portion of the picture as it is relevant to their needs.In the case of infinite processing and bandwidthcapabilities, this can scale to become a true CommonOperational Picture, with all participants seeing the samecomplete picture. In the case of significant limitations onthe ability to exchange and process information, as is thecase now and for the near future, the intelligentdissemination capability of the Grapevine ensures that allparticipants receive the most relevant information.

6. THE INTEGRATED DEMONSTRATIONCONCEPT OF OPERATIONS

Figure 7 depicts the architecture of the plannedsystem. It results from integration of ATL’s sensor datafusion and Grapevine intelligent information dissemina-tion technologies into CERDEC’s C2MINCS tacticalinformation sharing framework.

In the integrated system, the MULE receivessituation awareness data from the higher-echelon FBCB2system, and spot report data, entered by soldiers usingMobile Devices. On the MULE, FBCB2 reports and spotreports are fused to provide a clear, complete, andcoherent operational picture. From the MULE, a tailoredview of the total operational picture is disseminated toeach individual team member’s Mobile Device.Communications bandwidth is conserved through the useof the Grapevine to prioritize and tailor the disseminationof data. This provides each individual with an operationalpicture that is consistent with the team, but most relevantto the individual. In the event of loss of communicationsto the MULE, the team members can maintain a level ofsituation awareness by sharing spot reports and other datadirectly between their Mobile Devices, in a peer-to-peerconfiguration.

The architecture allows future extensibility to allowincorporation of additional information sources andconfiguration to adapt to and exploit the computingenvironment and capabilities of new and varied hostplatforms.

7. SUMMARY AND FUTURE CONSIDERATIONS

This system provides a testbed for exploring thetactical payoff of enabling tactical SA and C2 decisionmaking through the adaptation of emerging lightweightmobile computing technologies. Several enhancements

6

MULEFBCB2

FBCB2 Adapter

PASSServer

Red and BlueForce Positions

FBCB2Data

FBCB2 & SpotReports

Spots

Track

s

Tracks

Spots

Tracks

Spots

Spots

Tracks

Spots &Drawings

MobileDevice

C2SVR

C2MINCS

C2SVR

PassAdapter

C2MINCS

MobileDevice

FusionAdapter

FusionFused Tracks

FBCB2& Spot

FusedTracks

FBCB2 DataGrapevine

Fused Tracks

Prioritized Tracks

Grapevine Grapevine

Prioritized

Spots Priorit

ized

Spots

Spo

tsSpots

Fig. 7. Illustrates the component architecture of the integrated demonstration system.

have been identified for future implementation at both thearchitectural level and the application level. Examples ofenhancements under consideration include securitymeasures such as role based access control via biometricauthentication and incorporation of multimodal interfacesvia speech and/or gesture. SA richness and accuracy willbe enhanced with integration of additional data sources.Decision aiding will be enhanced by reachbackcapabilities into additional information systems. C2Decision Tools that provide theory, models, and tools toassist commanders and teams to adapt to complexsituations across the full spectrum of Stability andSupport Operations (SASO) environments will beresearched to add functionality to the C2MINCS.

REFERENCES

Hofmann, M., 1997: Multi-Sensor Track Classification inRotorcraft Pilot's Associate Data Fusion, Proc.American Helicopter Society 53rd Annual Forum,Virginia Beach, VA.

Jameson, S.M., 2001: Architectures for DistributedInformation Fusion to Support Situation Awarenesson the Digital Battlefield, Proc. 4th InternationalConference on Data Fusion, Montreal, Canada.

Jameson, S.M. and Stoneking, C., 2002: Army AviationSituational Awareness through Intelligent Agent-Based Discovery, Propagation, and Fusion ofInformation, Proc. American Helicopter SocietyForum 58th Annual Forum, Montreal, Canada.

Malkoff, D. and Pawlowski, A., 1996: RPA Data Fusion,9th National Symposium on Sensor Fusion, InfraredInformation Analysis Center, 1, 23-36.