Copyright 2007 Northrop Grumman Corporation Proprietary Level 1 Copyright 2007 Northrop Grumman Corporation 1 Northrop Grumman Private/Proprietary Level

Copyright 2007 Northrop Grumman Corporation 1 Northrop Grumman Private/Proprietary Level I

Pythagoras - an Agent-based DistillationA Presentation to ABSVal Workshop

25 March 2008

Edd Bitinas, Chief Modeler – 703-968-1196Brittlea Sheldon, Analyst – 703-968-1137Northrop Grumman Corporation

NORTHROP GRUMMAN PRIVATE/PROPRIETARY LEVEL 1


Overview

• Introduction To Pythagoras– What Is Significance Of Pythagoras And Why We

Developed It– How Pythagoras Works– What We Have Done To Improve Pythagoras– What Our Clients And Other Users Have Done With It


The Pythagoras Environment

Physics

Leadership

Human Factors

• Morale• Training• Confidence• Fatigue• Fear• Comradeship• Political Leanings• Anger

• Decision Making• Intelligence• Communication• Inspiration

• Influencers• Sensors• Platforms• Resources• Terrain• Weather

The Three Regions Are Tightly Linked


Types of Models/Simulations

0

5

10

15

20

25

30

35

40

Y

-5 0 5 10 15 20 25 30 35F1

Overlay Y's

Y F2 F3

Overlay Plot

Software Objects That: – Are Capable of Choice– Have Autonomous Behaviors and Are

Distinct from the Background or Environment and Each Other

– Perform Actions Like:• Sense• Order• Communicate• Move• Influence

PythagorasPythagoras

• Physics-Based– Verifiable through Experimentation– May Not Account for All Interactions

• Example: Weather Models– Outcomes Could Be Calculated, but Math Is Too Hard

• Probability-Based– Includes Some Physics– Account for Some/Most Random Effects– Range of Outcomes Could Be Calculated (See Above)

• Agent-Based– Decision-making Entities – Includes Probabilities– Outcomes Not Predictable


A Brief History of Pythagoras• July, 2001 – Northrop Grumman Wins USMC Contract To Support

Project Albert via Archimedes Development and Modification• September, 2001 – Northrop Grumman States ‘It Can’t Be Done.’

Project Albert Leadership States ‘We Need an Agent-Based Tool That:

– Uses Fuzzy Logic– Can Run Both in a Batch Mode at Maui High Performance Computing

Center (MHPCC) and from a GUI on a PC– Is Easy Enough That a Non-Computer Science Major Can Use It.’

• Pythagoras Development Timeline– April 2002 – Pythagoras v1.0 Is Delivered: Reproduces Archimedes

Functionality– June 2003 – Pythagoras v1.6.3 Is Delivered: Added Complete Behavior

Change Triggers– September 2005 – Pythagoras v1.9.0 Is Delivered: Influencers Replace

Weapons– September 2006 – Pythagoras v1.10.5 Is Delivered: Adds

Communications Devices – April 2008 – Pythagoras v2.0.0 is Delivered – Upgrade to JAVA 1.5,

JAXB 2.0, Add More Attributes, More...


What is Pythagoras?• Pythagoras Is an Agent-Based Simulation System That Puts

the Complexity of Creating Behaviors in the Hands of the Analyst, Not in the Software

• It Offers the Analyst the Ability To:– Mirror Actual Groups, Including Their Characteristics, Beliefs,

and Standards of Behavior– Examine Outcomes Based on Behavior– Data Farm Over a Trade Space Using High Performance

Computer Clusters• Execute a Range of Runs• Similar to Sensitivity Analysis• Design of Experiments Can Reduce Runs and Preserve Interactions

– Assess Impact of:• Uncertainty in Inputs

– Some Idea of the Minimum-To-Maximum Range• Unclear Interactions Among Inputs

• User Interface Is Easy To Understand and Manipulate– The Structure Allows the Functionality To Evolve as New

Scenario or Operational Requirements Are Identified


Pythagoras Software Components

GUI

Scenario

Engine

MovementOutput

MOEOutput

XML File

Legend

JAVA Software

Text File

CLI


How Does Pythagoras Work?• Soft Decision Rules

– A Way To Capture Variability • Agent’s Decisions

– Movement – Influencing • Generic Attributes

– Characteristics To Measure– Controls On Behavior

• Dynamic Affiliation– Same Unit – Same Side– Enemy – Neutral

• Rule-Based Influencers– Lethal – Non-Lethal

• Triggers– Change Behavior Rules When Triggered

• MOEs• Sensors (Three Bands)• Comms (Three Channels)• Terrain Features

– Mobility – Protection – Height– Concealment (Three Bands)


• Affiliation/Uniform (Shades of Red, Green, Blue)• Vulnerabilities (Multiplier on Lethality and Affiliation Changes)• Up to 10 Optional Attributes• Attribute Vulnerability (Multiplier on Influencers Affecting Attributes)• Leadership (Charismatic or Hierarchal)• Obedience – Repeating Orders Results in Nagging• Knowledge Retention• Communications Effectiveness• Marksmanship – Strength of the Ability To Influence• Height and Altitude• Detectability (Multiplier on Sensor Detection)• Vulnerability (Multiplier on Lethality)• Speed, Speed Variability• Movement Selection Mode (Mode 1, 2, 3 or 4)• Has Weapon Type (Up to Ten)• Has Sensor Type (Up to Ten)• Has Comm Type (Up to Ten)• Next Waypoint• Waypoint Proximity (How Close Is Close Enough?)• Target Value• Hold-Fire Desire• Target Persistence• Fuel Tank Capacity, Fuel Initial Load, Usage per Step• Resource X,Y,Z Capacity, Resource X,Y,Z Initial Load, Usage per Step

Agent Characterization

Italics Indicate Possible Soft Decision Rule Variable


Agent Time CycleDie

Clear Knowledge

Sense Environment

Communicate Knowledge

Choose Leader

Re-Supply (Resources)

Change Behavior (if Trigger Tripped)

Check if at Waypoint

Influence

Reset Influencers &Pick Targets

Set Speed

Move (if Can)

Change Color/Attributes (if Necessary)

Draw GUI Info

Record Info

Self Evaluation

Pick Direction

Load Balance (Resources)


Time Series MOEs• MOEs Recorded Over Time Include:

– Agent Red, Green, Blue – Agent Attributes– Agent Behavior– Agent Fuel, Resource X, Resource Y and Resource Z– # Detected Units (Friend, Enemy, Neutral)– # Known Units (Friend, Enemy, Neutral)– Distance to Final Objective– Killed By– Kills Scored– Shots Fired– Survival (Alive, Dead, Injured)

• Recording Methods Include:– Individual Agent– Entire Class Average

• Frequency Options:– Not Collected– End of Run– Beginning and End of Run– Beginning, End and Every x Time Steps


End of Run MOEs• Initial, Minimum, Maximum and Final Number of Agents

Near the Final Objective– This Can Measure How Agents Force the Enemy To “Run

Away”• Initial, Minimum, Maximum and Final Average Number of

Unit, Friendly, Enemy, Neutral and Unknown• Initial, Minimum, Maximum and Final Value of Red, Green,

Blue, Attributes 1-10 • Initial, Minimum, Maximum, Final and Used Amount of

Resource X, Resource Y and Resource Z, Fuel• Number of Survivors by Class• Number of Casualties by Class

– Both Injured and Dead• Number of Shots (by Influencer) and Effects


New in Pythagoras 2.0.0

• Modernization– Java 1.5– JAXB 2.0

• 4000 X 4000 Pixel Playbox• Multi-Dimension Visualization

– Map Location, Color, Health, Attributes and Resources to x, y, r, g, b, and Transparency

• Ten Generic Attributes – Changed By: - Changer Types

• Weapons • Incremental• Communications • Absolute• Events • Relative• Terrain • Multiplier

– Can Be Normalized


Pythagoras 2.0.0 (cont.)

• Agents Carry Up To Ten Each Of– Sensors – Weapons– Communications Devices

• Agents May Now Shoot At Multiple Targets Per Time Step

Status: Delivered To OAD & NPS, April 2008


How Has Pythagoras Been Used?• Terrorist Development (Sandia Labs)

– Conceptual Model of Human Factors: Explored Terrorist Recruiting• Population Dynamics (USMC)

– Disaster Relief and Counter-Insurgency: Examined Impact of MAGTF Commander’s Alternatives on Insurgency

• MAGTF Optical Requirements (Night Vision Lab/Ft. Belvoir)– Peacekeeping at Night – Comparison of Alternative Optical Technologies/Suites

• Historical Analysis (USNA)– Battle of Midway– Battle of Ia Drang, Viet Nam

• Thermobaric Weapons Assessment (MCCDC)– Urban Environment

• Shallow Water Obstacle Clearing (MCWL)– Use of JDAM and Robotics

• Homeland Defense (Northrop Grumman)– Pre-Proposal Analysis of Critical Node Defense from WMD

• Environmental Concerns (MITRE)– Spread of Hemlock Woolly Adelgid

• User Community– NPS – SOCOM – USNA – OSD – USMC/MCCDC – TRAC/Monterey– Northrop Grumman – USMA – Many Others


Summary

• Government-Owned, Open Source• Wide Variety of Potential Applications• Able To Deal with Uncertainty• Scenarios Are Expandable as New/More Data

Become Available• Not Limited to One Theory/Approach

– Compare Theories with the Same Software• No ‘Scripts’

– Group Behavior Emerges from Individual Behaviors• Scenarios Can Be Built In Hours

– Complexity Can Be Added As Needed


Backups


What Is Data Farming?

• Similar To Sensitivity Analysis• Multiple Simultaneous Variations In Inputs• Stochastic Scenarios May Require Tens Of

Thousands Of Runs– Use Super Computer Clusters

• Design Of Experiments Can Reduce The Number– Near Orthogonal Latin Hypercube,– D-Optimal, Others

• Identify Correlations And Trends• Outliers Also Provide Insight

– Why Was It Different?– Can That Difference Be Exploited?


Sample Animation Output

Convoy of Vehicles

ImprovisedExplosiveDevices

Detonator


What’s Next?

• Event-Driven Engine Option– Current Version Is Time-Stepped

• Arc/Node Terrain– Network-Oriented Terrain Option Would Provide New

Capabilities• Configurable GUI

– Start Simple, Add Capability as Needed• Additional Functionality

– Imperfect Knowledge About Affiliation– More Fire And Movement Desires– “OR” Options for Attributes and Affiliations– Moving and Shooting Detection– Explicit Initial Locations– Many Others...– Application-Driven Upgrades


Soft Rules

• An Attempt To Turn English Language Orders Into Numbers– Example: “Don’t Communicate Until The Enemy Gets Close

Enough” • How Close Is Close?

• People Interpret And Apply Orders Differently– Example: A Team Agrees That ‘Close’ Is 100 Meters.

• Some Start Talking At 115 Meters; Others Wait Until 90 Meters• People Interpret And Respond To Input Differently

– Example: MAGTF Optical Scenario • Hungry Villagers Move Toward Marines, Terrorists Move Away

From Marines• Soft Rules Attempt To Capture This Variability

– Each Agent Selects His Own Thresholds From An Input Range– Random Distribution May Cause Emergent Behavior

• Expect The Unexpected


Example of Individualized Numbers

Homogenous Populations React AlikeHeterogeneous Populations May React Differently or at Different Times

Frac

tion

of T

otal

0

0.1

0.2

0.3

0.4

0.5

0.6

0 2 4 6 8 10

FirmSoftLoose

Common Decision Threshold: Mean = 5Firm/Low Individuality: Range = 1Soft/Medium Individuality: Range = 2Loose/High Individuality: Range = 4


Agent Decision Variables

INFLUENCER/WEAPON:• Hold Influence Unless Being Influenced/Actively Influence• Highest Effectiveness/Lowest EffectivenessMOVEMENT:• Toward (if > d) or Away From (if < d) Closest Leader • Toward (if > d) or Away From (if < d) Closest Unit Member/Friendly • Toward Furthest Unit Member/Friendly (if > d) – Group Cohesion• Toward Next Waypoint (if > d)• Toward (if > d) or Away From Nearest Enemy (if < d)• Toward Small Number of Enemy (if # Enemy or Ratio within d < e) – Advance• Away From Large Number of Enemy (if # Enemy or Ratio within d > e) – Withdraw• Toward Objective• Toward Injured Friend – Restore or Heal• Toward Friend (if > d) Needing Fuel, Resource X, Resource Y, Resource Z • Toward Friend (if > d) Supplying Fuel, Resource X, Resource Y, Resource Z • At Random – Crowds• Continue Same Direction• Stay in Place if Desire To Move Is Weak – SlothTERRAIN:• Avoid Bad Terrain if (Low Mobility or Poor Concealment or No Defense)• Prefer Good Terrain if (Azimuth, High Mobility, Good Concealment, or Protection)

d – Distance, e – Enemy Count All Breakpoints Are Soft


Movement Implementation Alternatives

• Each Agent Calculates the Movement Direction for Each Non-Zero Desire

• They Are Then Combined Either by:– Using Vector Algebra, with the Desires as a Weight, or– Picking the ‘Strongest’ One (the Highest Desire), or– Averaging the Top Two, or– Using Monte Carlo, Weighted by Desire

• Thus, Movement Directions Are Calculated Based on the Current Situation, Not Scripted in Advance


Sensor Modeling• Each Terrain Grid Point (1 x 1) Has a Concealment Value for

Each of Three User-Defined Bands• For Each Grid Box Traversed by the Line of Sight (Rather

Than Proportional), the Concealment Value Is Multiplied– V = V * (1 - C)

• Each Agent Pair Draws a Random Number for Each Band at the Beginning of the Run– Determines the Range at which Detection Occurs from Input

Pd Curves• Detectability Is Held Constant during the Run

– Modified by Current Concealment • Sensors Are Either Personal (Owner Only) or Broadcast

(Owner and All Friends within Range)• Target Location Error (TLE)

– Only Used for Communicating Sensed Agents’ Locations


Sidedness• Three Primary Alternatives (Represented by RGB)• Agents Could Have Multiple Sympathies

– User Decides Which Colors To Use for Color-Distance Calculations (1, 2 or 3 Dimensions)

• Affiliation Changes Add or Subtract Color– Example: Shot At = +.01 Green, -.01 Blue, -.01 Red– Distances and Values Are All Soft Rules

• Behavior Change Triggers Occur When Sidedness Thresholds Are Exceeded– Example: Green > .5 or Red < .5, Read New Behavior =

Surrender• Alpha (Transparency) of 0.0 Means Agent is Dead


Example Two-Dimensional Affiliation

0 2550

255B

lue-

ness

Green-ness

MeMy UnitFriendlyNeutralEnemy


Attributes

• Ten Alternatives: 1-10• Similar to Sidedness Except “Affiliation” Is

Optional• Attribute Change Rules Add To or Subtract From

an Attribute– Example: Shot At = +.01 Att[0], -.01 Att[1], -.03 Att[6].– Distances and Values Are All Soft Rules

• Behavior Change Triggers Occur When Attribute Thresholds Are Exceeded– Example: Att[4] > .5 or Att[6] < .5– Read New Behavior = Follow Leader or Become Suicide

Bomber


Side or Attribute Change Events• Know About Leader

– Leader from Same Unit Found Out About• Don’t Know About Leader

– No Leader from Same Unit Found Out About• See Leader• Too Many Enemies• Too Few Friends• Outnumbering by a Ratio• Outnumbered by a Ratio• Being Influenced

– One Change for Each Occurrence• Influencing• At Waypoint• At Objective• All Is Well

– Used To Restore Original Conditions– Example: No Leader for a While, Then Leader Is Reacquired


Influencer TypesTwo General Types (Up To Ten Influencers Per Agent)

– Directed Target• Requires Line of Sight (LOS)• P(hit) and P(influence) Are a Function

of Range• Examples: Leaflets, Food• Only Affects Target

– Area Effect• No LOS Required• Accuracy (Hit Vs. Aim) a Function of

Range• P(influence) a Function of Distance

from Hit Point• Examples: Bull Horns• Affects All Agents within Radius• User Chooses Effect

– Carlton Damage Function Effect (Degrades with Distance), or

– Cookie Cutter Effect (Constant within Radius)

Non-Lethal Effects• Suppression

– Requires a Hit by Directed Target– Requires Possible Influence by

Area Effect– Causes Agent To Stop All Activity

for Input Time• No Movement• No Sensing/Broadcast• No Influencing

• Restore/Resurrect– Remove Damage (e.g., Medicine)

• Paint Balls– Change the Color/Attribute of

Targeted Agent• Absolute, Incremental, or Relative to

Agent Firing the Weapon• May Cause Triggers To Execute


Abstract Terrain• Fixed Box Size of Up To 4000 X 4000 Pixels

– User Decides What a Pixel Equates To• Terrain Attributes

– Mobility (Affects Agent Speed: 0-100% = Impassable to No Effect)– Concealment (Affects Agent Detection: 0-100% = No Effect to

Hidden)• Up To Three Detection Bands

– Protection (Affects Agent Vulnerability: 0-100% = No Effect to Invulnerable)

– Terrain Feature Height (Enables Looking Down From Above)• Additional, User-Defined Polygons Can Be Overlaid

– Feature Floor & Ceiling (Enables Clouds)• Terrain Can Change Agent’s Attributes (v2.0)


Triggers• Events That Cause an Agent To Change Behavior:

– Loss of Leader (No Leader Within d)– Red, Green, or Blue Becoming Greater or Less Than v– Attributes Becoming Greater or Less Than v– Resource X, Resource Y, or Resource Z Becoming Greater or Less

Than v%– Being Shot At– Detecting an Enemy– Detecting a Friend – Arriving at a Waypoint or Objective– Friendly Casualties (Fewer Than x% Known Remaining)– Fuel Becoming Greater or Less Than v%– Time Step (Absolute and Relative)

• Trigger Action Reads a New, Complete, Explicitly Named Behavior Template (Including Setting New Triggers)

• Delay Before Action (Future Capability)– Simulates Communication/Spin-up Time

x– Force Strength Threshold, d – Distance Threshold, v –Value


Activities When A Trigger Fires• Triggers Are User-Prioritized

– If More Than One Fires, Only One Is Acted Upon• Each Trigger Is Associated with a Named Behavior

– Example: Loss of Leader = ‘Surrender’• When Fired, a Trigger Reads in New Agent Behavior

– Movement Desires & Terrain Preferences– Changes to Other Attributes (Speed, Altitude, Vulnerability,

Leadership, etc.)– New Waypoints– New Current Color or Change Color by Actions– New Current Attribute or Change Attribute by Actions– New Weapon, Sensor, Comm Possessions– New Engagement Desires (Hold-Fire, Weapon Selection Logic, Target

Persistence)– New Fuel, Resource X, Resource Y, or Resource Z, Settings– New Behavior Change Triggers (and Priorities)

• No Limit To Number of New Behavior Changes (One Per Time Step)

• Records Time Step, Agent Number and Class


Example Behavior Tree

Initial: PatrolIf See EnemyIf Lose LeaderIf Take CasualtiesIf Influenced

Surrender

AttackIf Lose LeaderIf Take Casualties

Take CoverIf Lose LeaderIf Take Casualties

WithdrawIf Lose Leader


BACKUP


PythagorasUSAF

February 2008Edmund J. BitinasDonna MiddletonJohn McGillvray

Pythagoras: Helping the World Look at Problems

from the Right Angle

Documents

Copyright 2007 Northrop Grumman Corporation Proprietary Level 1 Copyright 2007 Northrop Grumman Corporation 1 Northrop Grumman Private/Proprietary Level