Embed Size (px)
Citation preview
Stan Posey, HPC Program Manager, ESM Domain, NVIDIA (HQ), Santa Clara, CA, USA
GPU Progress in Atmospheric Sciences
2
3X HPC Developers
2016 2014
400,000
120,000
55,000
25x Deep Learning Developers
•Higher Ed 35% •Software 19% •Internet 15% •Auto 10% •Government 5% •Medical 4% •Finance 4% •Manufacturing 4%
2016 2014
2,200
NVIDIA GPU Growth from Advances in HPC and AI
GPUs Power World’s Leading Data Centers for HPC and AI:
3
NOAA To Improve NWP and Climate Research with GPUs
Develop global model with 3km resolution, five-fold increase from today’s resolution
NWP Model: FV3/GFS (also climate research)
HPC System Deployments for GPU-Based NWP
Cray CS-Storm, 760 x P100, 8 GPUs per node
MeteoSwiss Deploys World’s 1st Operational NWP on GPUs
2-3x higher resolution for daily forecasts
14x more simulation with ensemble approach for medium-range forecasts
NWP Model: COSMO
Cray CS-Storm, 192 x K80, 8 GPUs per node
4
NGGPS final selection: FV3
NGGPS Models with GPU developments:
NIM
MPAS
NEPTUNE/NUMA
FV3
NOAA NGGPS Launched Several GPU Collaborations
From: Next Generation HPC and Forecast Model Application Readiness at NCEP
-by John Michalakes, NOAA NCEP; AMS, Phoenix, AZ, Jan 2015
NGGPS NH Model Dycore Candidates (5)
5
COSMO 7 (6.6 KM) 3 per day, 3 day forecast
COSMO 2 (2.2 KM) 8 per day, 24 hr forecast
IFS from ECMWF 2 per day, 10 day forecast
COSMO E (2.2 KM) 2 per day, 5 day forecast
COSMO 1 (1.1 KM) 8 per day, 24 hr forecast
IFS from ECMWF 2 per day, 10 day forecast
MeteoSwiss COSMO NWP
Configurations During 2016
With GPUs
MeteoSwiss COSMO NWP
Configurations Since 2008
Before GPUs
“New configurations of higher resolution and ensemble predictions possible owing to the performance-per-energy gains from GPUs” –X. Lapillonne , MeteoSwiss; EGU Assembly, Apr 2015
MeteoSwiss and Operational COSMO NWP on GPUs
6
Large Scale Climate ~1km with COSMO and P100
Source: PASC 2017, Lugano, CH, Jun 2017; Contact Hannes Vogt, CSCS, [email protected]
Strong Scaling to 4888 x P100 GPUs
Piz Daint #3 Top500
25.3 PetaFLOPS
5320 x P100 GPUs
- Oliver Fuhrer, et al, MeteoSwiss
3.7km GPU
3.7km CPU
Higher
Is
Better
19km GPU
19km CPU 1.9km
GPU
.93km
GPU
7
DOE CORAL Systems with Volta and NVLink
LLNL Sierra 150PF in 2018 ORNL Summit 200PF in 2018
CAAR support from IBM and NVIDIA
~1/4x
~29x
~1.7x
27,600 GPUs
8
ACME: Accelerated Climate Modeling for Energy First fully accelerated climate model (GPU and MIC)
Consolidation of DOE ESM projects from 7 into 1 DOE Labs: Argonne, LANL, LBL, LLNL, ORNL, PNNL, Sandia
Towards NH global Atm 12 km, Ocn 15 km, 80 year
ACME component models and GPU progress Atm – ACME-Atmosphere (NCAR CAM-SE fork)
Dycore now in trunk, CAM physics started with OpenACC
Ocn – MPAS-O (LANL) LANL team at ORNL OpenACC Hackathon during 2015
Others – published OpenACC progress Sea-Ice – ACME-CICE (LANL) Land – CLM (ORNL, NCAR) Cloud Superparameterization – SAM (SBU, CSU) Land-Ice – PISCEES (Multi-lab – LLNL, Sandia)
DOE ACME GPU-Accelerated Coupled Climate Model
9
V100 (2017) P100 (2016) K40 (2014)
Double Precision TFlop/s 7.5 5.3 1.4
Single Precision TFlop/s 15.0 10.6 4.3
Half Precision TFlop/s 120 (DL) 21.2 n/a
Memory Bandwidth
(GB/s) 990 720 288
Memory Size 16GB 16GB 12GB
Interconnect NVLink: Up to 300 GB/s
PCIe: 32 GB/s
NVLink: 160 GB/s
PCIe: 32 GB/s PCIe: 16 GB/s
Power 300W 300W 235W
New NVIDIA Volta – GPU Feature Comparisons
Volta Availability DGX-1: Q3 2017; OEM : Q4 2017
1.42x
1.42x
1.25x
3.8x
2.5x
2.5x
1.33x 1.00x
1.00x
~6x
10
SENA – NOAA funding for accelerator development of WRF, NGGPS (FV3), GFDL climate, NMMB
ESCAPE – ECMWF-led EUC Horizon 2020 program for IFS; NVIDIA 1 of 11 funded partners
ACME – US DOE accelerated climate model: CAM-SE, MPAS-O, CICE, CLM, SAM, PISCEES, others
AIMES – Govt’s from DE, FR, and JP for HPC (and GPU) developments of ICON, DYNAMICO, NICAM
SIParCS – NCAR academia funding for HPC (and GPU) developments of MPAS, CESM, DART, Fields
AOLI – US DoD accelerator development of operational models HYCOM, NUMA, CICE, RRTMG
GridTools – Swiss gov funding MCH/CSCS/ETH for accelerator-based DSL in COSMO, ICON, others
GPU Funded-Development Growing for ESM
NOTE: Follow each program LINK for details; Programs listed from top-down in rough order of newest to oldest start date
HPC Programs with Funding Specifically Targeted for GPU Development of Various ESMs
11
Organization Location Model GPU Approach
ORNL, SNL US ACME-Atmosphere OpenACC (migration from CUDA-F)
ORNL, PNNL, UCI, SBU US SAM OpenACC
NCAR; THU US CAM-SE OpenACC (migration from CUDA-F)
NCAR, KISTI US MPAS-A OpenACC
NOAA GFDL, ESRL US FV3/GFS OpenACC
NASA GSFC US GEOS-5 OpenACC (migration from CUDA-F)
US Naval Res Lab, NPS US NUMA/NEPTUNE DSL – dycore only
ECMWF UK IFS Libs + OpenACC
MetOffice , STFC UK UM/GungHo OpenACC back-end to PSyKAI
DWD, MPI-M, CSCS DE, CH ICON DSL – dycore, OpenACC – physics
JAMSTEC, UoT, RIKEN JP NICAM OpenACC, DSL
NCAR; TQI/SSEC US WRF-ARW (i) OpenACC, (ii) CUDA
DWD, MCH, CSCS DE, CH COSMO DSL – dycore, OpenACC – physics
Bull, MF FR HARMONIE OpenACC
TiTech JP ASUCA Hybrid-Fortran, OpenACC
NVIDIA Collaborations with Atmospheric Models
Global
Regional
12
Internet Services Medicine Media & Entertainment Security & Defense Autonomous Machines
Cancer cell detection
Diabetic grading
Drug discovery
Pedestrian detection
Lane tracking
Recognize traffic signs
Face recognition
Video surveillance
Cyber security
Video captioning
Content based search
Real time translation
Image/Video classification
Speech recognition
Natural language processing
AI and Deep Learning Expanding Across All Domains
13
HPC + AI for Weather and Climate Applications
AI in Weather
Applications
Challenges for
HPC and AI
in Weather
and Climate
NERSC (on CPUs)
Yandex + Start-ups
NOAA, MCH, others
NCAR, KISTI, others
14
NWP Nowcasting Systems Apply Deep Learning
15
NVIDIA Feature on ClimaCell Nowcasting – 17 Oct 17
https://blogs.nvidia.com/blog/2017/10/17/nowcasting/
16
JAMSTEC Deep Learning for TC Prediction
17
NVIDIA Feature on KISTI NWP Research – 24 Oct 17
https://blogs.nvidia.com/blog/2017/10/24/how-ai-could-help-people-dodge-monster-storms/
KISTI use numerical models WRF and MPAS,
and deep learning to predict typhoon tracks
18
NOAA Neural Network Study on Thompson MP
https://ams.confex.com/ams/97Annual/webprogram/Paper310969.html
19
Background
Unexpected fog can cause an airport to cancel or
delay flights, sometimes having global effects on
flight planning.
Challenge
While the weather forecasting model at MeteoSwiss
work at a 2km x 2km resolution, runways at Zurich
airport is less than 2km. So human forecasters sift
through huge simulated data with 40 parameters, like
wind, pressure, temperature, to predict visibility at
the airport.
Solution
MeteoSwiss is investigating the use of deep learning to
forecast type of fog and visibility at sub-km scale at
Zurich airport.
MCH Use of DL Models for Fog Forecast at Airport
20
NCAR AI Research to Improve Weather Forecasting
NCAR Application of Generative Adversarial Networks (GANs) on Understanding Sea Level Pressure
• Input data from 4096 x NOAA GEFS model outputs of pressure forecasts to train the GAN model
• Results can demonstrate model produces “realistic” pressure fields after 100 epochs of training
Courtesy Dr. David John Gagne, NCAR, Jul 2017
