Upload
insidehpc
View
150
Download
1
Tags:
Embed Size (px)
Citation preview
Intel Confidential, Copyright © 2015, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others. Optimization Notice
Wolfgang Gentzsch & Burak Yenier
ISC Frankfurt, July 16, 2015
Docker Workshop
Containers for Scientific & Engineering Applications
2012: UberCloud started
its free voluntary Cloud Experiments
HPC as a Service, on demand, in a team experiment
to explore the end-to-end process
using cloud computing resources
as a service, on demand, at your fingertips
and learning how to resolve the roadblocks.
TODAY: 172 experiments, 2500 companies, 72 countries, 50 case studies
Supported by our sponsor INTEL
Major Cloud Roadblocks
Supply Cloud providers ISVs Consultants Trainers
Demand Engineers Scientists Data analysts Experts
.
.
.
.
.
Complexity
Data Transfer
Security Licensing
Uncertain Cost
Roadblocks
2015: The UberCloud Marketplace
Supply Cloud providers ISVs Consultants Trainers …
Demand Engineers Scientists Data analysts Experts
Marketplace
And HPC Docker
UberCloud application containers
Ready-to-execute packages of software, designed to deliver the tools that an engineer needs
Ready to execute, in an instant. No need to install software, deal with complex OS commands, or configure.
The ISV or Open Source tools are pre-installed, configured, and tested, and are running on bare metal, without loss of performance.
Based on open Linux container technology
Enhanced for engineering & scientific applications
ANSYS, CD-adapco, OpenFOAM, Gromacs, Scilab, and more
Virtual Machines vs Docker
VM: Each virtualized application includes not only the application and the necessary binaries and libraries, but also an entire guest operating system.
Docker Engine container comprises application and its dependencies. Isolated process in user space on host operating system, sharing kernel with other containers. Enjoys resource isolation and allocation benefits of VMs but more portable and efficient.
9
Builder
Launcher
Controller ISV Data Tools
Stackable units with tools (ex: encryption), ISV application codes (ex: OpenFOAM). Just add your own codes and data.
Run anywhere with UberCloud Run Time. Scale up or down the compute power as needed.
Collect granular usage data, logs. Monitor, alert, report.
Any Workstation Any Cluster Any Cloud
Run Time Run Time Run Time
Containers: Build once, run anywhere
Builder
Launcher
Controller ISV Data Tools
Stackable units with tools (ex: encryption), ISV application codes (ex: OpenFOAM). Just add your own codes and data.
Run anywhere with UberCloud Run Time. Scale up or down the compute power as needed.
Collect granular usage data, logs. Monitor, alert, report.
Any Workstation Any Cluster Any Cloud
Run Time Run Time Run Time
Containers: Build once, run anywhere
Builder
Launcher
Controller ISV Data Tools
Stackable units with tools (ex: encryption), ISV application codes (ex: OpenFOAM). Just add your own codes and data.
Run anywhere with UberCloud Run Time. Scale up or down the compute power as needed.
Collect granular usage data, logs. Monitor, alert, report.
Any Workstation Any Cluster Any Cloud
Run Time Run Time Run Time
Containers: Build once, run anywhere
Builder
Launcher
Controller ISV Data Tools
Stackable units with tools (ex: encryption), ISV application codes (ex: OpenFOAM). Just add your own codes and data.
Run anywhere with UberCloud Run Time. Scale up or down the compute power as needed.
Collect granular usage data, logs. Monitor, alert, report.
Any Workstation Any Cluster Any Cloud
Run Time Run Time Run Time
Containers: Build once, run anywhere
Containers remove Portability & Standard related challenges. By integrating 3rd party tools we tackle the following:
UberCloud multi-container environment
Docker Runtime Docker Runtime
Docker Runtime
. . . .
NFS
Host Host Host
Container Container Container
MPI
Ve
rtic
al s
cali
ng
Horizontal scaling
Dramatically simplifying Packaging, Porting, Access, Use, Scaling, with Low Overhead
OpenFOAM Test: Bare Metal vs Docker
Performance Test Results demonstrating comparison between the same OpenFOAM run (with 1.4 million cells) repeated on bare metal and in a Docker container.
Dockerized Technical Computing Ecosystem
Experts CFDsupport
CAEtech Dacolt
Any Inhouse &
Open Source
ISV ANSYS
CD-adapco OpenFOAM
Resource AWS
Azure LLNL
UberCloud multi-container environment
Intel MPI libraries pre-setup for multi-node high performance
Intel compilers achieve architecture specific high performance
Networking between containers
Setting up shared file system access (NFS)
Secure communication (ssh) between containers
Benefits from single-node transferred to multi-node containers:
No setup of the environment
Portability and packaging
Easy access and use
Low overhead
+
Hot Spot Question for Jérôme Petazzoni
Docker recently announced that it acquired SocketPlane to help with container networks. I am very interested in some more background and details and how this could support HPC multi-container approach.
There are multiple container implementations, Docker, CoreOS and now a collective effort, runC. Can you talk about what this all means for the community.
Virtualization has been around for a long time, why re-invent the wheel?
In HPC clusters shared file systems are quite commonly used. Can you comment on how to secure a Docker installation in conjunction with a shared file system. Wouldn't the shared file system be vulnerable when Docker is in the mix?
ANSYS Container Demo
ANSYS multi-physics (5 s/w) running inside Docker container
Pre- and post-processing in the cloud works the same way that it would work
on the engineer’s desktop
User experience in the cloud matches that on the desktop
GPU support for high-resolution remote visualization
Automated monitoring keeps you connected, under control
4-min demo: https://goo.gl/YIrTr4
6-min demo: http://goo.gl/pDXiTq
Containers as basic technology for the UberCloud SaaS Factory
UberCloud SaaS Factory
Software SaaS
Commercial ISV Open Source In-house
Public Cloud Private Cloud On-Premise
$$ $
We turn high-TCO, complex software into user-friendly pay-per-use services:
Development Services (Docker) --------------------
Deployment & Management Services
------------------ Marketplace
Providers Consumers
$
UberCloud SaaS Factory
Intel Confidential, Copyright © 2015, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others. Optimization Notice
Thank You !
Please register at
http://www.TheUberCloud.com
It’s free