Presented to: SOA Brown Bag #12 By: The SWIM Governance Team & Steve Thomas, iTKO Representative Date: January 11, 2012 Federal Aviation Administration

Presented to: SOA Brown Bag #12

By: The SWIM Governance Team &

Steve Thomas, iTKO Representative

Date: January 11, 2012

Federal AviationAdministration

System Wide Information Management (SWIM)

iTKO LISA

2Federal AviationAdministration

SOA Brown Bag #12: iTKO LISA

January 11, 2012

• The product is the entire brand and customer experience

• Service-oriented products are delivered late, over budget, and with questionable quality… WHY?

Innovate Or Die



January 11, 2012

Mainframe Client/Server(including packaged

apps like SAP, Siebel, Oracle…)

Composite(Custom Applications,

SOA, Cloud)

1980 … 1985 … 1990 … 1995 … 2000 … 2005 … 2010 … 2015

What iTKO Does

Founded in 1999

LISA Software Suite:– Virtualization, Automated Testing, Pathfinder,

Cloud DevTest Environments

Optimize the Application Lifecycle



January 11, 2012

UATPerformance

TestIntegration

Test

Many Others…

New Product Offering

New Partner Support

Summer Promotions

80+ Fixes

Business Requirements

1000s of Technical Requirements

• Parallel Development Challenges (chicken & egg)• Every team needs their own environment but shares only one

• Testing can’t be automated due to tech & environment issues

• Developers’ stubs wreak havoc downstream

2M+ Hours Later…

30+ InterdependentDevelopment Teams

Design, Develop

System Test

•Manual testing rules because automation has failed, labor and time costs are way too high

•MTTR of issues is too high and growing-“100 person conference calls”

•Performance testing often squeezed or meaningless because it’s not production scale

•Most of their defects, even requirements and design, are discovered here

•Integration testing waits until every system is ready–this wrecks the schedule

•One shared environment, very often down, the “manufacturing line” stops constantly

•Defect discovery very difficult, 80% of the interfaces are machine interfaces

•Manual testing rules because automation has failed, labor and time costs are way too high

Reality: Major Releases 3+ Times Per Yearplus monthly minors, weekly “content” changes and urgent BreakFixResults: costs explode, quality suffers, time to market increases



January 11, 2012

Composite Applications

More Labor Required

Missed Deadlines

Production Failures

Increased Infrastructure Cost

Growing Pressures Of Application Development



January 11, 2012

Gain transparency and traceability across complex, distributed application

architectures

Presentation Layer

Service Layer

IntegrationLayer

Data/Applications

Other UI

CORBA J2EEWeb Service BPM

ESB Web Service RMI Custom

LegacyDatabase File SystemPackaged App

Web 2.0

Identify bottlenecks and defects at every level

Message traffic performance through the hardware

iTKO is a solution for an IT problem that CA has not previously addressed



January 11, 2012

RMI Objects

MessagingService

BI Tools

Business Rules

FileSystem

ESB

WebInterface

ContentDatabase

DataWarehouse

FinancialsMainframe

Web/WAPInterface

EJB

VirtualService

RoutingService

SOAP

CRM App

Products

CollaborationApp Portal

HelpEngine

BPMSLegacyInfrastructure

Internal

External CloudPartners

# of Interconnected Components

# of Inter-dependent Teams

# of Heterogeneous Technologies

Rate of Change

Today’s Environment



January 11, 2012

Root Causes Of Application Development Problems



January 11, 2012

SYSTEMUNAVAILABLE

ACCESS FEES

INCOMPLETEDEVELOPMENT

“I can’t do anything until I have everything…and I never have everything!”

INVALID DATA

Problem: Constraints



January 11, 2012

24/7 availability Saved $30M Reduced Timeline by 50%

Solution: Service Virtualization



January 11, 2012

80% of all new applications are composite and highly integrated

Agile has replaced waterfall development methodologies

Most new application development is self-service and customer-facing making complexity exponential

Customers expect Facebook and Google quality

RMI ObjectsMessaging

Service

BI Tools

Business Rules

FileSystem

ESB

WebInterface

ContentDatabase

DataWarehouse

FinancialsMainframe

Web/WAPInterface

EJB

VirtualService

RoutingService

SOAP

CRM App

Products


HelpEngine

BPMS

InfrastructureInternal


Modern Application Development Architecture

70% of the information developers need is outside of their control when they need it

Limited predictability on how it’s going to perform and where it’s going to break

Traditional products and methodologies don’t work in this new architecture



January 11, 2012

iTKO invented, patented and released Service Virtualization in 2007

We clone the behavior of constrained systems

We solved the 70% availability issue

Proven value with 100+ F500 customers

Mature 4th generation product virtualizes everything

Service Virtualization

RMI Objects

MessagingService

BI Tools

Business Rules

FileSystem

ESB

WebInterface

ContentDatabase

DataWarehouse

FinancialsMainframe

Web/WAPInterface

EJB

VirtualService

RoutingService

SOAP

CRM App

Products


HelpEngine

BPMS





January 11, 2012

LISA was built specifically for this architecture to solve the predictability issue

Continuous Validation of performance and usability

Increase collaboration through automation

Applications are more predictable

Fixing problems in production is 10x the cost

Enables enterprise agile

RMI ObjectsMessaging

Service

BI Tools

Business Rules

FileSystem

ESB

WebInterface

ContentDatabase

DataWarehouse

FinancialsMainframe

Web/WAPInterface

EJB

VirtualService

RoutingService

SOAP

CRM App

Products


HelpEngine

BPMS



Built For Modern Architecture



January 11, 2012

This Is What LISA AddressesATM (TFM) Automation

AFSS / FSS

Flight Service Automation

USNO(Herndon VA)

Aeronautical Automation

PRODUCTS

FIDSL

ACARS VDL-2

ACARS VDL-2

Air Surveillance Facilities Anchorage

ATC Comm (Digital)

ATC En Route Automation

Non-Euro Fac

Euro Fac

ATM CommsCCS

HQ Washington DCNFDC (Airspace Lab)

NASR

USNS

ETMS-HS/ CDM-HS (VNTSC) (Cambridge MA)

GUS

ATCT (FCT)

Ocean (DOTS)

ATM Automation

ETMS

FTI

OCEAN (New York NY & Oakland CA)

ATC CommATC Automation

ATC Comm (Digital)

ARTCC/ OCEAN (Anchorage AK)ATC Ocean Automation (& Domestic)

ATC Comm

ATCT Weather (WSP at airports w/ o TDWR)

ATC Comm (Digital)

Runway Lights

ATM Comm (TMU)

TRACON

ATC Data Link

ATC Comm (Data)

ATC Comm (Digital)ATC Comm (Digital)

STARS

CARTS

NAVAIDFacilities

RCE

43 44

Terminal Surveillance

ATCTWind Shear (8-

12 mi out)

En Route Surveillance

KG-84Mode 4KIRB-1

Wx,NOTAMS,

SUA

RCE

DOASIS

A

B

C

1 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45

DE

FG

H

I

J

K

L

M

N

O

P

Q

R

S

T

W

X

YZ

AA

ABAC

AD

AE

AF

E

F

H

IJ

K

L

M

N

O

P

Q

R

S

T

U

V

W

XY

Z

AA

ABAC

AD

AE

AF

1 2 3 4 5 6 7 8 9 10 12 13 14 15 16 17 18 19 20 21 22 23 24 26 27 28 29 30 3132 33 34 35 36 37 38 39 40 41 42

46

AH

ENET SWITCH

Via NADIN

WME

LLWAS-RS

TDWR

SAWS

ASOS & AWOS

RVR

DASI

AG

CCSP(ACARS)

2

NNCC(Salt Lake City UT

& Atlanta GA)

WMSCR

METAR (SAO) &SPECI (ADAS)

A

Visual NAVAIDsALSF-2 VASI

MALSR

PAPI

REIL

ODALS

SALSLDIN

ALS

SSALS SSALR

MALS

25

G

Wx ServerRouter

CISCOSYSTEMS

CISCOSYSTEMS

RL/ REL

RAIL

RWCLL

TDZL

DSP

ATP

ACP

ITWS

FDIO RCU

EFSTS

FSP

11

DUATS

Surveillance Facility

RTR Facility

RCE

RTR FacilitiesRCE

RCO FacilitiesRCE

FTI

B

AIS (Salt Lake City, UT)

AISDB Server

Secondary

AIS-RDB

AIS SD

AISLAN

Web Server

C

TMA-SC

TMALAN

TMASwitch

RA/ TS

ADIF / ESB Video Sw

itch

Via OP-IP

FAATSAT

VOICE(UHF/VHF)

HWDS &

UWDS

UHS/ VHS

CCSP (ACARS) (ARINC)

VDL-2

RCL

AG

M1FCAWP

M1FCWS

Flight DataServer

GW

HWDS & UWDS

NOTAMS, WX

IGWDS

21

AH

4645

V

UHF/ VHF

ACARS (aircraft)

Aeronautical Automation

LLWAS-NE

NDPSurveillance Net

P/ O OP-IP NDP AMICC

NDPADS

HOSTRDP FDP

HNLURET

HADDS

CP

DSP

ECG Primary

ATC Weather (En Route - CWSU)

WARP

ATC Comm (Voice)

VTABS

BUEC

ADASITWS AWIPS

RDWARPERIDS

ATC

ATC

ATC

VSCS

ATC

ATC

ATC

ATC

ATC

ATC

ATC

ATC

ATC

RCAGFacilities

ARSRFacilities

RCE

RCE

ASR (TRAD, TSEC)

CUSTOMSOCEAN (ODAP)

TRACON, ATCT, CERAP, DoD (FDAT)

Adjacent ARTCC, TRACON/ ATCT (IDAT)

ARSR (RDAT, BDAT)

RAPPI

ARSR (IP)

OP-IP

FLIGHT &WEATHER DATA

DSR & CRMR

ERIDS

ERIDS

DSR R

DSR D

DSR-A

FSP

ALDARS

FTIFDIO

CCU ACESNDP Comm

(Digital)

AGW ARSR

ATC Navigation

NAVMNFTI

(FDIO)

IDS-4(Kansas City,

Denver)

ITWSITWS PG

TCWP

Comm or Router

ITWSLAN

RouterCISCOSYSTEMS

CISCOSYSTEMS

CIWS

MIGFATDWR

RPG

TWIP

LLWAS

Algorithms:PrecipStormMicroburstGustRBDT

Algorithms:ATISWindTornadoLightning

ANICS

STARS CGW

SIU(RGW / CGW)RDPCDR

ARTS GW

AGW

DBRITEDBRITE ScannerLBP

RDP

RGW

SMP

ATC Comm (Voice)

LRR

FMAFDAD

ACD

RADS DEDS RDM PRMTCW

ATC

ATC

ATC

ATC

ATC

ATC

STARSLAN

FMAPRM

ARTS-IIIELAN

FDIO

Via TML

LLWASTDWRWSP

RBDT

ACE-IDS Control Cabinet

IDS-4

ACE-IDS

ACE-IDS LAN

FSP

Automation

ECG

Cha

nnel

A

LAN

NASLAN

WSP RDP ACUDFWSPServer

WSPLAN

TRACON (ITWS)

From TDWR/LLWAS

From NEXRAD (WSR-88D)

NAVAIDS (above)

WSP

ETMS

ETMSLAN

TMUServer

ETMS SDITWS

FSM

STA

RS

C-A

RTS

FSPTDU TCP

IDS-4

FSP

RBDT FDIO

ATC Comm - Air

ATC

ATC

ATC

ATC

ATC

ATC

ATC Comm - Ground

AMASSw/ RWSL ASDE-3 (40)

ASDE-X (27)w/ RWSL

TFM Sup

ATCT (AWIS)

ARTCC (ADAS)

TRACON (ITWS )

ATC Terminal Automation

NAVMN

FDIO RCUEFSTS

ATC Terminal Automation

RWSL

Airport Air Surveillance

Airport Ground SurveillanceRWSL

RWSL

SMS(Louisville)

SMS / SMA

SMS / SMALAN

Server

SMS

SMS / SMA

AOCs

SMS

SMS / SMA

SMS / SMALAN

Server

SMA

ETMSTMUServer

ETMSFSM

ATM (TMU) TFM TFM Sup

ITWS

SMS

ETMSLAN

FSM

Int’nl Nodes (AFTN)

Other ARTCCs NCRCC

DSR

Bac

kup

LAN

DSR

Prim

ary

LAN

UR

ET

LAN

URETServer

ICE

NORAD

NFDC(NOP Extractor)

ATCSCC (FBWTG)

NEXRAD (WSR-88D)

ATCSCC (CSS)

FTI

FTI

FTI

EHIServer

NASServer

TRACON(ACE IDS)

NADIN (MSN)

CCSPCERAP / HCF

LAW ENFORCE

AFTNMBO

WMSCRATCSCC Fuel & Email

ASOS

TRACON (ETMS)

TRACON (ACE ICS)

ATCT RVR (above)

RVR (ATCT)

RVR DPU

ATCT RVR (above)

TRACON Weather

AWIS

NIMS OCC

ARTCC Equipment

Monitor AutomationSOC RMM

S

MPS

NIMS

EMMDT

NIMSLAN

CommServer

RMDB

NIMSRouter

CISCOSYSTEMS

CISCOSYSTEMS

Via IP

Monitor Automation SOC

RMMS

MPS

NIMS

EMMDT

NIMSLAN

CommServer

RMDB

NIMSRouter

CISCOSYSTEMS

CISCOSYSTEMS

Firewall

NIMS OCC

TRACON Equipment

Via IP

Monitor AutomationSOC

RMMS

MPS

NIMS

EMMDT

NIMSLAN

CommServer

RMDB

NIMSRouter

CISCOSYSTEMS

CISCOSYSTEMS

ETMS

ETMSLAN

TMUServer

ATM Comm (TMU)

ETMS SDITWS

ATM Automation (TMU)TFM TFM TFM Sup

FSMWSD

WSP

NIMS OCC

ATCT Equipment

Firewall

Via IP

RMMSMPS

Via NADIN

NIMS OCC

Via IP

NIMS OCC

ETMS HS (via Internet)

ETMS HS (via OP-IP)

FTI

Other MPS

From ASOS/AWOS

From ARTCC (ADAS)

From ARTCC (WARP)

ARTCC (ITWS SD)

From ASR-9

ATCSCC (ITWS NFU)

ARTCC (TMA)

FTIFTI

From ATCT (WSP)

TRACON (ITWS)

From ASOS/AWOS

ETMS-HS

AWIPS (NWS)

NADIN PSN

From TRACON (ITWS)

TRACON (ITWS)

FTI

TRACON (IDS-4)

USNO (USNS)

AFSS (M1FC)

AFSS / NNCC / ARTCC (M1FC)

ATCSCC (AIS)

ARTCC (ADAS)

ARTCC (WARP)

TRACON (ITWS)

ASOS/AWOS

ARTCC (TMA)

ETMS HS (ETMS Users)

FTI

Remote ATCT

or

NADINPSN MSN

FDIO(Store & Fwd)

NOTAMSWXFlight Plans

CDM WS (ASDI Non-Euro)

ETMS LAN

RMTDB

ASDIRouter

CISCOSYSTEMS

CISCOSYSTEMS

CISCOSYSTEMS

NORADSwitch

CDMSwitch

ETMSServerRouter

CISCOSYSTEMS

CISCOSYSTEMS

CISCOSYSTEMS

InterfaceServer

CommServer

OAGDB

NPBKDB

IP / HTTP

CCFP

CDM WS (ASDI Euro)

From NWS (AWC)

Firewall

Firewall

NORAD From OAG (Chicago)

Firewall

AWNFileserver

WSI

Firewall

RouterCISCOSYSTEMS

CISCOSYSTEMS

Firewall

IngestRouter

CISCOSYSTEMS

CISCOSYSTEMS

FTPRouter

CISCOSYSTEMS

CISCOSYSTEMS

Firewall

AOC NET (AFD)

CCSPCDM NET

FSDPS

FTI

ATOP

ATM Comm (TMU)

FAATSATHWDS

& UWDS

ATC Weather(Ocean - CWSU)

WARP

VTABS

BUEC

ADAS

AWIPS RD

WARP

VSCS

ALDARS

Via WMSCR

AutomationETMSTMU

Server

ETMSFSM

ATM TFM

TFM

TFM Sup

ETMSLAN

ATCSCC (FBWTG)

NEXRAD (WSR-88D)

ATCSCC (CCS)

NASServer

HFROAFTN (AIDC)CCSP (SITA)

WMSCR

NIMS OCC

NWS (AWIPS)

To/ From ARTCC (WARP)

From ASOS/AWOS

ASD

ADS-BRouter

CISCOSYSTEMS

CISCOSYSTEMS

ADS-B

WINSServer

Modeling data

ASD

FTI

Voice Comms

TRACON (ITWS)

ATCSCC (DOTS)

R-Posit

D-Posit

ECS

FDPSFDPSServer

SDPSServer

ATO

P LA

N

CWP

SDPS

ATOP

ATOP

ARSR

ASR

NADIN (PSN)

ETMS HS

MEARTS

FDP2000

Monitor AutomationSOC

Ocean Equipment

NIMS RMMS

MPSEM

MDT

NIMSLAN

CommServer

RMDB

NIMSRouter

CISCOSYSTEMS

CISCOSYSTEMS

NADIN (PSN)

Oce

an

LAN

DS

LAN

DS

LAN

Via IP

Firewall

ATC Fairbanks

TRACON

ARTCC (FDIO)

FSP

ATC Comm (Voice)

CISCOSYSTEMS

CISCOSYSTEMS

VaisalaNLDN

FSP

MEARTS

OFDPS

DS

LAN

DS

LAN


WMSCRFTI

NADIN (PSN)

TDLSPDC

D-ATISFDIO

Emulator

TRACON & ARTCC

TCS

DVRS

TCS

DVRS

ATC

ATC

ATC

AFSSVSPBX

DVRSICSS

TCS

DVRS

ARTCC, AFSS, ATCSCC

Surveillance Facility

RCAG, RCOFacilities

RCE

FTI

RCE

ARTCC

OCEANFDIO

ATC

ATC

FTI

ETMS

ETMSLAN

NASServer

ETMS SD

ATM Auto-mation (TMU)

FSM

TFM TFM Sup

SMS

ATC Ocean Automation

ATOP


WMSCR

ADS-BRouter

CISCOSYSTEMS

CISCOSYSTEMS

ADS-B

FTIECS

FDPS

FDPSServer

SDPSServer

ATO

P LA

N

CWP

SDPS

ARSR

ASR

NADIN (PSN)

ATM Comm(TMU)

ATC Weather(use ARTCC Systems)

DOTSDOTSLAN

DOTS

DOTS Proc

TPD

TMD

DOTSRouter

DOTS DOTS

DOTS Proc TPD

TMDARTCC WARP WINS

USAF USN

AT

CCom

m

ATC

ATC

DS

LAN

DO

TS

LAN

ATM ETMS -TMU(use ARTCC Systems)

AT

C C

omm

(Voi

ce)

(use

AR

TCC S

yste

ms)

DVRS

Surveillance

TMUServer

FTI

ETMS HS

FTI

FSMGDP

Airlines (Sched Chgs)

Airlines (ASD)

ASD

GDP Proc

FSM Proc

ETMSFSM

ETMSLAN

Server

TMA

TMA

TMA

TMA

OISOIS Proc

OIS

POETPOET Proc

POET

OISPOET

DOTSRouter

CISCOSYSTEMS

CISCOSYSTEMS

DOTSDOTSDOTS

Proc

TPD

TMD

DO

TS

LAN

DOTSRouter

CISCOSYSTEMS

CISCOSYSTEMS

VaisalaNLDN

WMSCR

WMSCR

NADIN

WMSCR

ITWS / ADASTRACON (ITWS)

FTIInternet

FTI

TRACON

ATCT

ARTCC

Wx Graphics

AFSSD

ARTCC (FSDPS)

NNCC (AWP)

Via NADIN

NWS (GOES)

DoD

NWS / NOAA

NWS

NWSTG

NOAA

AWIPS

ATCSCC (FBWTG)

AWN

ARTCC(CWSU AWIPS RD)

EMC(Camp Springs MD)

Wx Proc

Gridded Wx

NWS NCEPs

· CCFP· NCWF· SIGMETS· AIRMETS· METARS· PIREPS (MDCRS)· CIP / FIP· GTG

· ETA· RUC· AVN

AWC(Kansas City MO)

ETMS HS

AWNFileserver

Via NADIN

Via NADIN

DOTS (ATCSCC)

Wx Graphics, Winds Aloft, Temperature

ETA, RUC, AVN

CCSP (Aircraft)

Internet Users

To CCSP (Aircraft)

PIREPS (MDCRS), METARS / SPECI

SIGMETS, AIRMETS

CCFP, GTG,

NCWF, CIP, FIP

TMUs (ETMS)

Flight PlansAFSS(OASIS, M1FC,

DUATS)

NADIN

Fi rewal l

SwitchingEquipment

Voice CommTrunks

AFSS (ICSS)

MBOs

ATCT (TCS)

TRACON (ICSS)

ARTCCs (VSCS)

ATCSCC (PSTN)

CCSP (VDL) (ARINC, SITA)

NADIN (PSN)

Firewall

Radio NAVAIDS

RMVC RMMS

Firewall

NIMS NOCCMonitor

AutomationNIMS

NIMS OCCMonitor Automation

OCC

SOCs:ARTCC, TRACON, ATCT, AFSS, etc.

RMMSMPS

NIMS

EMMDT

NIMSLAN

CommServer

RMDB

NIMSRouter

CISCOSYSTEMS

CISCOSYSTEMS

From Radars, NAVAIDs, etc.

NACO (Silver Spring MD)Aeronautical Automation (East) Publications

A/ FD NACO Charts

Radar Video Maps

AVNISServer

DoD PublicFAA

RAPT

Jeppesen Sanderson, Inc.

Rockwell Collins, Inc.Universal Avionics Systems Corp.

eNOTAMS

DINSDBNOTAMS

DB

ARTCC (ERIDS)

DoD(NIPRNET)

FPO (IAPA)

WMSCR

NADIN

Internet

NFDC (TFR)

USNS

USNSLAN

eNOTAMServer

Firewall

NOTAMServer

Firewall

DNSServer

DoD(DINS)

Internet

International

CCSP

TL

NFDCLAN

NASRServer

NASRDB

USNO (USNS)

NASR Proc

TFR Builder

eNASRServer

NTAP

NASR

TFR

NFDCD

TFR Repository

TRACON (CARTS, STARS)

NOPExtractor

NOP Archive Proc

MBOsForeign ATCs

ARTCCs NNCC (AWP)

NFDC

NADIN (PSN)

NADIN (PSN)

NADIN (MSN)

DAICCEPIC

NORAD

Firewall

FTI(OASIS WAN)

OCEAN

FTI

FTI

FTI

FTI

FTI

FTI

FTI

FTI

FTI FTI

FTI

FTI

NADIN (PSN)

FTI

FTI

FTI

FTI

ARTCC (ECG)

FTI

FTI FTI

FTI

FTI

FTI

FTI

FTI

FTI

FTI

FTI

FTI

FTI

FTI

FTI

FTI

FTI

FTI

FTI

ATCSCC (AIS)

NGA

ARTCC (FDIO / ECG)

AIS

AISATCSCC

(AIS)

AIS

ATCSCC (AIS)

ARTCC (excludes Anchorage AK)

ATCSCC (Herndon VA)

Firewall FTIARTCC

(Radio NAVAIDs)

ILS-II/ III

DME VOR

MLS

NDB

TACAN

FTI

TRACON, ATCT, AFSS

TRACON, ATCT Visual NAVAIDs

CISCOSYSTEMS

CISCOSYSTEMS

MPSServer

FTI

FTI

USCGLORAN-C

NIMSLAN

CommServer

RMDB

EMMDT

RCE

VaisalaNLDN

VaisalaNLDN

WMS (San Diego CA& Herndon VA)

WAAS

FMS GPS/ WAAS

GEO

GPS

GPS/ WAASFMS

HWDS &

UWDS

NEXRAD (WSR-88D)

GEO

FTI

FTI

FTI

FTI

FTI

FTI

FTI

FTI

FTI

FTI

RCE

LRR

ARTCC (ECG)

FTI

LL(CIWS)

FTI

WRS

FTI

FTI

FTI

FTI

WAAS

FTI

FTI

U

LL(CIWS) CIWS

DAFIF

DVRS

DVRS

USNS

NNCC (USNS - Sec)

USNO (USNS)

ATCSCC (AIS Pri)

FTI

FTI

WAAS

CERAP (Honolulu HI, Guam GQ, San Juan RQ)& HCF

HGW

TRACON / ATCT

Firewall

DoD(DINS)

Fire

wal

l

TRACON (ITWS PG)

ARTCC (ADAS)

TRACON (IDS-4)

ATCT (IDS-4)

AFSS (OASIS)

FAA

ARTCC

WMSCR

AIS

NADIN

eAISServer

DoD (NIRPNET)

NFDC (NASR)

AISLAN

AISServer

AISDB

Firewall Firewall

NEXRAD (WSR-88D)

ATCSCC Weather

FBWTGITWS NFU

AVN

, ETA,

RU

C

FAAT-SATNWS

(NWSTG)

WARP

WARP

WINSServer

ITWS

HWDS &

UWDS

CWIS

NODA

ARTCC (SAFA)

NASR

WINSServer

ARTCC (FDIO)

STARSRouter

CISCOSYSTEMS

CISCOSYSTEMS

ARTSRouter

CISCOSYSTEMS

CISCOSYSTEMS

ARTCC (ADAS)

ATC

DVRS

ATM Comm (TMU)

FTI

EVCSPBXICSS

STVS

ETVSRDVS

EVCSETRICSS

STVS

ETVSRDVS

EVCSETRICSS

ETVSRDVS

EVCS

EVCS

EVCSETR

RDVSPSTN

EVCS

STVSRDVS

EVCSPSTN

ERMS

ERMS

FTI

DoD

Cmd & Control

CERAP OCEAN

RCAG

, RCO

RCAG

, RCO

RCAG

, RCO

ADL

LOC

FTI FTI

FTI

FTI

NASSI DS

RVR DB

NWS

Flight Procedures (Ok City OK)Aeronautical Automation (West)

AVNIS

DBAVNIS DB

Server(read/ write)

Firewall

IAPA

TMARouter

CISCOSYSTEMS

CISCOSYSTEMS

TMA-SC

TFMLAN

TMASwitch

RA/ TS

ADIF / ESBVideo Sw

itch WS TMARouter

CISCOSYSTEMS

CISCOSYSTEMS

Via HID to OP-IP

HIDRouter

CISCOSYSTEMS

CISCOSYSTEMS

FTI

ANICSANICS

NNCC (ANICS)

ANICS

Via NNCC

FTI

TSD

TSD

TSD

WAAS

WAAS

IPDS IFPAIRNAV OE-IFR APTS

AIPA

FOMS

AVNISDB (read)

Flt Proc IS East:AirspaceEn RouteARINCGPS productCharting

Flt Proc West:AirspaceEn RouteARINCGPS productCharting

· Airport· Runway· Navaid

FTI

FTI

FTI

NDDF

PANS-OPS(ICAO)

TERPS

PRODUCTS

NAS web sites:Internet: http://www.nas-architecture.faa.gov/home.cfm

Intranet: http://cats-i.faa.gov/nas/

ATC Navigation

NAVMN

FTI

ARSR-1ARSR-2ARSR-3ARSR-4

FPS-20/ 60

ATCBI-4ATCBI-5ATCBI-6Mode S

ARSRASRFPS

ATCBIMode S

ADS

ModemCD-2 Modem

CV-4400 / ASIS

ModemModemModem

Digitizer

ASR-9ASR-8ASR-7

ATCBI-4/ 5 ATCBI-4/ 5 ASR-11 (MSSR)

ADS-B (Memphis/ Louisville)ASR-9

AMASS ASDE-3ASDE-X

Mode-C ADS-B

Mode-C ADS-B

FTI

TSD

GPS

TMA TSD

SAFA

FTI

TMA

SMR

TSDOISDSP

RCE

RADS TDWRVR

DBRITER-ACD

AWN Users

AWIPS

TMA

September 2006

Non-FAA FacilitySystem A System B

FAA Facility

ATC Weather

ATC NavigationSystem A System B

ATC CommunicationSystem BSystem A

System A System B

ATC SurveillanceSystem BSystem A

ATC AutomationOperations

Aeronautical Monitoring

Legend

System Component

Simplified NAS WSI

FISDL

FISDL

NNCC (AWP)

ORS

IOE/AAA

FCC

ORS

NFDD

TIS

FTI

TPP

DAICD

DACS DOF

ARTCC (FSDPS)

FTI

ECG Backup ECGM&C

Firewall

ECG

Mai

n-te

nanc

e LAN

BIP-B

EBUSBIP-A

EBUS

PIP-B

PIP-A

ECG

Cha

nnel

B

LAN

EBUSM&C

Obstruction Evaluator

IOE/ AAARepository NGS

NGA

External Agencies

SAMSWS

SAMSWS

SAMSSAMS Proc

DoD (MAMS)

ARTCC (SAMS WS)

TRACON (SAMS WS)

CERAP (SAMS WS)

ATCT (SAMS WS)

FTI

ATCSCC (SAMS)AVN (NASE)

FTI

NASE

ARTCC (ACES)

Web

FTI

Off-line

SAMSWS

ATCSCC (SAMS) FT

I

FTI

SAMSWS

ATCSCC (SAMS)

FTI

SAMSWS

ATCSCC (SAMS)

FTI

FTI

FAA Orginazations

FTI

NFDC (USNSR)

NNCC (AWP)

TARGETS

SDAT

TRACON (CARTS, STARS)AFIS

FMSFAA Flt Acft

FTI

RNAV Pro

NACO (Chart CDs)

FTI

FTI

EURO (BADA)

FTI

NNCC (AWP)

FTI

Via NADIN PSN

Via NADIN PSN

FTI

Power, HVAC

HVACEngine

Generator

CPP

KW

ACEP,

CPDS,

Alt Pwr

DC

System

UPSLPGBS

FAA Facilities

Mode STIS

Mode STIS

Mode S



January 11, 2012

“My team of 50 is now a team of 250 in just 5 yearsdue to more components and releases.”

Problem: Manual Labor



January 11, 2012

Achieve and maintain 85%-95% coverage after 1st test cycle.

Solution: Automation



January 11, 2012

ChallengesManual testing incurring delays and costsNo traceability or transparency across tiersSMEs required to manually validate results

Manual UI Testing

Manual Validation Before After

Automate end-to-end coverage to

find bugs “between the

boxes”

Results with LISA95% reduction in per-test costsRegression testing reduced from 18 weeks of labor to 1 day

Validate With Complete Automated Testing (National Cable Service Provider)



January 11, 2012

“We spend far more time figuring out where the bug is than we do fixing the bug”

Problem: Complexity



January 11, 2012

Reduced “non-reproducible” defect closure by > 66%

DEVELOPER TESTER

“Alt-Click!”

Solution: Collaboration



January 11, 2012

UPSETCUSTOMERS

SERVICEDOWN

PRODUCTIONISSUES

“Too many things go into production without adequate testing”

Problem: Change And Unintended Consequences



January 11, 2012

Reduced problems delivered to QA to <1% and took 5 months off integration release cycle

Solution: Continuous Validation



January 11, 2012

Value Analysis: Ask Yourself… • How many requirements are

tested manually vs. automated?

• What % of QA’s reported defects are rejected as "not reproducible“ by Dev?

• Time spent “proving” an issue that's been rejected as not reproducible?

• % of resolved cases from Development re-opened by QA as still failing?

• Avg. Dev & QA time recoding and retesting each unresolved issue?

• The big one: % of Total Test time in the next release spent retesting what you are testing in the current release?

Typical Results: Cost Savings > 25% | Releases > 20% Faster

“90% manual”

“At least 35%”

“3-4 hours

each”

“25% or more”

“4-5 hours

avg.”

“80%+”



January 11, 2012

Reduced infrastructure cost

Parallel development and testing

Increased quality and performance

Shorter release times

Value delivered with LISA



January 11, 2012

Questions and Comments

Documents

Presented to: SOA Brown Bag #12 By: The SWIM Governance Team & Steve Thomas, iTKO Representative Date: January 11, 2012 Federal Aviation Administration