Fleet Numerical Meteorology and Oceanography Center (FNMOC)

COPC Site UpdateNovember 14, 2007

Captain John G. Kusters, USNFleet Numerical Meteorology and Oceanography Center

7 Grace Hopper AveMonterey, CA 93943

John.Kusters@navy.mil(831) 656-4327

• Operational Models and Applications Operational Models and Applications • Information AssuranceInformation Assurance• Automation InitiativesAutomation Initiatives• Computer SystemsComputer Systems• Satellite ProductsSatellite Products• NPOESSNPOESS• MILCONMILCON• SummarySummary

OutlineOutline

NOGAPSNOGAPS

Aerosol Modeling

MesoscaleModels

EnsembleModels

Ocean Models

Tropical Cyclone Forecasts

Optimum TrackShip Routing

Automated High Seas / Wind Warnings

Ballistic Wind Computations

Electro-Optical Forecasts

Aircraft Routing

Ocean AcousticForecasting

Long-Range Planning

Ice ForecastsTarget Weapon

Systems

Visibility/DustForecasts

Search and Rescue

Models and ApplicationsModels and Applications

Operational ModelsOperational Models• World-class operational and information-assured models are

critical to our Nation’s defense– At core of Fleet Numerical reachback/automation efforts

• Navy Operational Global Atmospheric Prediction System (NOGAPS)– Premier global model for maritime environment

• Tailored for Navy/DOD missions• Drives Navy ocean models

– Supports the Ensemble Forecast System (EFS)

• Coupled Ocean/Atmosphere Mesoscale Prediction System (COAMPS)– Regional model for high-resolution support to Naval operations– Coupled with littoral ocean models– Re-locatable in minutes for on-demand operations support– Classification levels up to TS/SCI

• Direct Feedback Connection with Fleet Users– Fleet Numerical provides enhanced model verification plots– Fleet users provide objective feedback

• Model and/or post-processing adjustments• Rigorous but rapid Configuration Management process• Long-term model improvements based on Fleet needs• Continuous model improvement adds skill to the critical

72 – 120 hour period (resource protection, sortie decisions)• Meets weekly• “Team Monterey” effort

– Collocation with Naval Research Lab

– Rapid and successful transition of R&D Ops– Naval Postgraduate School Collaboration

Continuous Model Improvement Process

Operational Models Improvement ProcessOperational Models Improvement Process

Model Upgrades Last 6 months Model Upgrades Last 6 months• NOGAPS/NAVDAS/NAVDAS-AR

– AVHRR cloud-track polar winds (20071114)

– Tuning variational analysis scale factors (20071002). Improved short term tropical forecast performance

– NAVDAS-AR (4DVAR) running in NRL alpha utilizing CRTM and METOP AMSU radiances (20070901)

– NAVDAS - CRTM installed under FNMOC CM

– NOGAPS - Stopped reading AFWA snow fields and reverted to model snow due to overestimation (2007062212).

– NAVDAS - Assimilating 3-hourly feature track wind files from UW CIMSS for Meteosat-9. It was previously 6-hourly (April)

– NAVDAS - EUMETSAT Meteosat-9 replaced Meteosat-8 atmospheric motion vector (AMV) winds (20070411)

Model Upgrades Last 6 months• COAMPS

– All COAMPS areas converted from MVOI to NAVDAS-3DVAR– Running COAMPS areas operationally on IBM AIX to take load off

of SGI systems (20071017)– COAMPS verification graphics package promoted to FNMOC OPS

(20070804)– Significant COAMPS update to improve height bias, stratus

prediction, and tropical cyclone intensity (20070801). Surface fluxes were increased thus impacting NCOM during VS-07.

– METOP-A ATOVS temperature retrievals assimilated by COAMPS/NAVDAS (20070808)

– COAMPS-OS operational in the SCIF on AMS2 Origin 3800 hardware (200706)

– COAMPS Europe resolution change from 81/27 to 54/18 km (20070516)

– Completed COAMPS SWA dust source file updates and added capability for dust forecasts to WPAC area (20070522)

NAVDAS-ARNavy Atmospheric Variational Data Assimilation

System –Accelerated Representer

NAVDAS-ARNavy Atmospheric Variational Data Assimilation

System –Accelerated Representer

• 4D-VAR data assimilation system• Basis for future fully interactive data

assimilation/forecast system supporting adaptive observing

• Computationally efficient Accelerated Representer numerical solution

• The means by which NOGAPS and COAMPS assimilate data in the NPOESS era

Model Upgrades Next 6 Months Model Upgrades Next 6 Months

• NOGAPS/NAVDAS/NAVDAS-AR– Assimilate Korean AMDAR winds when quality improvements made

(Q1FY08)

– FNMOC Beta NAVDAS-AR (Q1FY08)

– COSMIC/GPS data assimilation (Q2FY08)

– Assimilate AIRS radiances (Q2FY08)

– FNMOC Beta on new A2 Linux cluster (Q3FY08)

– FNMOC Ops on new A2 Linux cluster (Q3FY08)

• COAMPS– COAMPS-OS (NAVDAS, classified obs, RUC) (Q2FY08)

– COAMPS-OS (coupled NCOM, WW3, SWAN) upgrade (Q3FY08)

– COAMPS-OS merges with AMS COAMPS and becomes COAMPS on A2-0 (Q4FY08)

• Booze-Allen-Hamilton contracted to facilitate transition of ODAA to Naval Network Warfare Command (NNWC)– ATO in progress including robust Cross Domain Solution

• Defense Information Systems Agency (DISA) Enhanced Compliance Review– Annual inspection to evaluate classified network (SIPR), unclassified

network (NIPR) and traditional security posture

– Over 99% IAVM compliant on evaluated items

• High Performance Computing Modernization Officer (HPCMO) Technical and Administrative Reviews– Annual inspection to evaluate HPCMO HPC connections to the

network

Information Assurance (IA) Environment

Information Assurance (IA) Environment

Automation Initiatives

•AOTSR

•CAAPS

•EVIS

• Ship routing has traditionally been a labor intensive process that requires forecasters to examine each ship route and corresponding model/forecast information

• Significant aspects of maritime routing to optimize heavy weather avoidance can be automated

– If 70% of oceans/skies are calm, those forecasts are automatically transmitted

– As operating thresholds are approached, ship routes are flagged for forecaster review

– Builds on progress automating high winds/seas warnings for the Pacific and Indian Ocean, which have increasing forecaster efficiency by as much as 90%

• Existing resources are being leveraged to develop prototype of Automated OTSR that incorporates a fuel efficiency option

– Ship Tracking and Routing System (STARS) automation and integration (Mr. Henry Chen)

Automated OptimumTrack Ship Routing (AOTSR)

Automated OptimumTrack Ship Routing (AOTSR)

Centralized Atmospheric Analysis and Prediction System

• CAAPS is a relocatable, higher-resolution modeling application of COAMPS– User configurable areas and model resolutions

– Linux cluster based

• Rapid area implementation permits repositioning in real-time (20 min) to support critical Naval operations

• Incorporates a Vapor-Liquid-Solid (VLS) Track Dispersion Model for WMD plume forecasts

• Multiple CAAPS areas currently running to support classified Naval and Joint operations

CAAPS can provide on-demand support for a transitingStrike Group within 20 minutes

CAAPSCAAPS

• Automates the manual process of generating impacts graphics (stoplight charts)

• Ties mission critical impacts to highest resolution models available (COAMPS/CAAPS)

• Thresholds custom based (XML) or tied to reference pubs (RP-1, IWEDA, JTS)

• Tailors impacts to mission area, geographic area, or theater requirements

• Allows for forecaster modification of results or threshold inputs

EnvironmentalVisualization (EVIS)

EnvironmentalVisualization (EVIS)

• Primary Oceanographic Prediction System (POPS)– Principal operational platforms– Run models and applications at the UNCLAS, SECRET and

TS/SCI Levels– Analysis and Modeling Subsystem (AMS) – Models– Applications Transactions and Observations Subsystem (ATOS) -

SOA– A2 (AMS + ATOS)

• DoD High Performance Computing Modernization Program (HPCMP) funded Distributed Center System (DC3)

• Remote Operations at the HPCMP Major Shared Resource Center (MSRC) located at NAVO

HPC SystemsHPC Systems

*Ranked in the top 5-10% of supercomputer sites worldwide in terms of overall computing power (~30 TFLOPS peak processing power; i.e., ~30 trillion calculations per second) The FNMOC Computer systems are also linked directly to ~150 TB of disk space and ~160 TB of tape archive space.

NAME TYPE #CPUs MEMORY (GB)

PEAK SPEED (TFLOPS)

OS

AMS2 SGI ORIGIN 3800 128 128 0.2 TRIX

AMS3 SGI ORIGIN 3900 256 256 0.7 TRIX

AMS4 SGI ORIGIN 3900 256 256 0.7 TRIX

AMS5 IBM p655+ 288 567 1.8 AIX

DC3 IBM p655+ 256 512 1.9 AIX

ATOS2 IBM 1350s/x440s/x345s 438 645 2.2 Linux

CAAPS IBM e1350s 148 272 0.7 Linux

A2-0 Linux Networx Custom Supersystem

928 1800 10.0 Linux

A2-1 Linux Networx Custom Supersystem

1168 1170 12.0 Linux

TOTAL 3794 6637 ~30

HPC PortfolioHPC Portfolio

POPS2-U SYSTEM ARCHITECTURE

UnclassifiedLAN

Unclassified ATOS2

Classified ATOS2

ClassifiedLAN

Origin300012PTRIX

amsfs1

Single Level Unclass

Single Level Class

CXFS Filesystems (15 TB)

L8 L4 L0

Origin300012PTRIX

amsfs2

Fibre Switch

SIPRNETINTEL LINKDATMS-UDREN

COM SERVERS

APPLICATION SERVERS APPLICATION SERVERS

COM SERVERS

UnclassifiedLAN

Unclassified ATOS2

Classified ATOS2

ClassifiedLAN

Origin300012PTRIX

amsfs1

Single Level Unclass

Single Level Class

CXFS Filesystems (15 TB)

L8 L4 L0

CXFS Filesystems (15 TB)

L8 L4 L0

Origin300012PTRIX

amsfs2

Fibre Switch Fibre Switch

SIPRNETINTEL LINKDATMS-UDREN

COM SERVERS

APPLICATION SERVERS APPLICATION SERVERS

COM SERVERS

Near-lineStorage

IBMP655288P

576GBAIX

HPCMPAMS5/DC2/AMS6

Origin3900 256P

256GBTRIX

AMS4

Origin3900 256P

256GBTRIX

AMS4

NIPRNET

Origin

3900

256P

256GB

TRIX

HPCMP

AMS3

21 TB

Origin3900 256P

256GBTRIX

AMS3

Origin3900 256P

256GBTRIX

AMS3

SDREN

SDREN Host

LNXI228

NODES1824GBSUSE

A2-0

LNXI228

NODES1824GBSUSE

A2-0

LNXI146 NODES2200GBSUSE

A2-1

MLS

SECRET

UNCLASSIFIED

• Technology advances open the opportunity for combining AMS and ATOS functionality by utilizing specialized computing– Emergence of Linux based HPC

– Convergence of server and HPC processors

• Efficiency for reachback/on-demand modeling– Shared file system

– Shared databases

– Lower latency for on-demand model response

• Long term cost efficiency of Linux vice proprietary operating systems

• Cost savings by converging to a single operating system across FNMOC

POPS Architecture StrategyPOPS Architecture Strategy

Specialized ComputingSpecialized Computing• Specialized computing which complements high-capacity

systems by tuning hardware to applications will:– Enable transition to an easily scalable Linux environment

– SE Linux to replace TRIX and Trusted Solaris• Aligned with NSA investments for an MLS solution

– Triple peak speed from 7.33 to ~ 29.56 TFLOPS

– Triple memory from 2,645 to ~ 7,005 GB

– Double number of CPUs from 2,026 to ~ 4,234

• Combining this with a CAAPS or EVIS interface provides full capability that combines all layers of Battlespace On Demand:– Environmental data

– Sensor/weapon performance

– Commander/operator decision

A2 Computer SystemsA2 Computer Systems

• A2-0 - Linux Networx Custom Supersystem– 232 nodes with 2 Intel Xeon 5100-Series “Woodcrest” dual-core processors

per node (928 core processors total)– 1.86 TB memory (8 GB per node)– 10 TFLOPS peak speed – 35 TB disk space with 2 GB per second throughput– Double Data Rate Infiniband interconnect at 20 Gb per second– 40 Gb per second connection to external Cisco network– 4 Clearspeed floating point accelerator cards

• A2-1 - Linux Networx Custom Supersystem– 146 nodes with 2 Intel Xeon 5300-Series “Clovertown” quad-core

processors per node (1168 core processors total)– 1.17 TB memory (8 GB per node)– 12 TFLOPS peak speed– 40 TB disk space with 2 GB per second throughput– Double Data Rate Infiniband interconnect at 20 Gb per second– 40 Gb per second connection to external Cisco network– 2 GRU floating point accelerator cards

A2 Hardware SpecificationsA2 Hardware Specifications

Satellite ProductsSatellite Products

• Tools: – MET7 dust enhancement

loops– SeaWIFS true color/dust

enhancement– Feature tracked winds– MODIS dust

enhancement• Products:

– Low clouds over snow – Low clouds at night – Convective cloud top

heights– Fire and smoke

enhancement

Satellite Focus Webpage (SatFocus)Satellite Focus Webpage (SatFocus)

• Satellite data Satellite data

fusion pagefusion page

• Multi-sensor Multi-sensor

images images

• Operational Operational

Unclassified / Unclassified /

Classified Classified

webweb

• Available to Available to

the public the public

Tropical Cyclone Webpage (TCWeb)Tropical Cyclone Webpage (TCWeb)

Scatterometer WindsScatterometer Winds

•Need ASCAT from METOP via NESDIS ODAA

•Central site for scatterometer data distribution to the Navy

•Used for:

- Assimilation into models

-Near real-time displays for operational forecasters

ERS and QuikScat Scatterometer, SSMI Wind and Rain, FNMOC Directional Ambiguity Removal

NPOESS Processing Architecture

NPOESS Processing Architecture

FNMOC IDPS(Standard RDRs, TDRs, SDRs, EDRs)

FNMOC Satellite Processing System

(Navy Unique EDRs)

Data Quality / MonitoringAlternate Communications Interface

(ACI) SAN

Assimilation into NOGAPS and COAMPS via

NAVDAS-AR

METOC Portal/Web Services

Products to Users via the Global Information Grid (GIG)

NPOESS WAN

Cross Point ~ 700,000 obs

•Number of asynoptic observations will increase by a factor of ~10 in the NPOESS era (left panel below).

•FNMOC is well-positioned to fully exploit this data via NAVDAS-AR because:

–4D-VAR technique is inherently well-suited for getting the most out of asynoptic data

–NAVDAS-AR processing scales with model grid resolution, but is relatively insensitive to the number of observations assimilated (right panel below).

•Number of asynoptic observations will increase by a factor of ~10 in the NPOESS era (left panel below).

•FNMOC is well-positioned to fully exploit this data via NAVDAS-AR because:

–4D-VAR technique is inherently well-suited for getting the most out of asynoptic data

–NAVDAS-AR processing scales with model grid resolution, but is relatively insensitive to the number of observations assimilated (right panel below).

NPOESS and NAVDAS-ARNPOESS and NAVDAS-AR

• $7.3M project with groundbreaking planned for July 2007 and completion by June 2008

• NPOESS IDPS - Additional Computer Room

• SCIF improvements / enhancements

• Watch Floor modernization

• New Operational Briefing Area adjacent to Watch Floor

• 9400 sq. ft. - new construction

• 4800 sq. ft. - rehabilitation of existing spaces

MILCONMILCON

• Fleet Numerical is an operational command, with direct support relationships and connectivity to Fleet and Joint Forces

• Fleet Numerical is continually improving its core Numerical Modeling Capability and provides the only DoD Information-Assured Global Atmospheric Model

• “Team Monterey” is a unique partnership that focuses expertise to provide operational support like nowhere else

SummarySummary

“Supercomputing Excellence for Fleet Safety

And Information Superiority”

Questions?

FNMOC

BACKUPS

Tropical 200/850 Wind Mean RMSVE (day 3)

OCTOBER 2007

WPAC (Jan-Sep 2007)

NOGAPS TC Track Performance

West Pacific

2007 NOGAPS track error beats the NOGAPS 2006 at days 3-5. However, 2007 sample size much smaller (less statistical significance)

NOGAPSNOGAPSGFDNGFDNNCEP GFSNCEP GFSUK METUK METJTWC ForecastJTWC ForecastConsensusConsensus

2006

Models ImprovementModels Improvement

2007