15
Digishape: "Heel AHN2 is inmiddels ingeladen in de Microsoft Azure cloud“ Deltares Software Days 2017, june 15 th 2017 Gerben de Boer, Van Oord [email protected] Bas Hoonhout, Deltares et al.

DSD-NL 2017 Digishape project: "Heel AHN2 is inmiddels ingeladen in de Microsoft Azure cloud" - De Boer, Hoonhout

Embed Size (px)

Citation preview

Page 1: DSD-NL 2017 Digishape project: "Heel AHN2 is inmiddels ingeladen in de Microsoft Azure cloud" - De Boer, Hoonhout

Digishape: "Heel AHN2 is inmiddelsingeladen in de Microsoft Azure cloud“

Deltares Software Days 2017, june 15th 2017

Gerben de Boer, Van Oord

[email protected]

Bas Hoonhout, Deltares

et al.

Page 2: DSD-NL 2017 Digishape project: "Heel AHN2 is inmiddels ingeladen in de Microsoft Azure cloud" - De Boer, Hoonhout

HQ fsteamdelft.nl Tu Delft lab: cables, composites, tires, 3D printer

2

Page 3: DSD-NL 2017 Digishape project: "Heel AHN2 is inmiddels ingeladen in de Microsoft Azure cloud" - De Boer, Hoonhout

MVRDV HQ Rotterdam: people, ideas, paper, books, foam maquettes

Page 4: DSD-NL 2017 Digishape project: "Heel AHN2 is inmiddels ingeladen in de Microsoft Azure cloud" - De Boer, Hoonhout

HQ OpenEarth: monthly docker sprintsession Microsoft NL, Schiphol

4

Van Oord, Deltares, Tu Delft, KNMI, NLeSC, Sogeti, Microsoft, …

Page 5: DSD-NL 2017 Digishape project: "Heel AHN2 is inmiddels ingeladen in de Microsoft Azure cloud" - De Boer, Hoonhout

HQ OpenEarth: cloud: docker, postgres, python, apache, linux

5

Page 6: DSD-NL 2017 Digishape project: "Heel AHN2 is inmiddels ingeladen in de Microsoft Azure cloud" - De Boer, Hoonhout

Web services: everything central in cloud except …. the data

Duplicate storageDuplicate band width (egress)

Page 7: DSD-NL 2017 Digishape project: "Heel AHN2 is inmiddels ingeladen in de Microsoft Azure cloud" - De Boer, Hoonhout

The solution is to make central datalake per provider: cached copies

bit like bittorrent: every user becomes an extra node to spread the data

Page 8: DSD-NL 2017 Digishape project: "Heel AHN2 is inmiddels ingeladen in de Microsoft Azure cloud" - De Boer, Hoonhout

Cloud providers already offer some data as cached copies

Page 9: DSD-NL 2017 Digishape project: "Heel AHN2 is inmiddels ingeladen in de Microsoft Azure cloud" - De Boer, Hoonhout

Dutch coastal data not available as cached copy: link to 100s of tiffs

Page 10: DSD-NL 2017 Digishape project: "Heel AHN2 is inmiddels ingeladen in de Microsoft Azure cloud" - De Boer, Hoonhout

Getting al AHN into Azure takes 30 mins with 100 docker nodes !

Page 11: DSD-NL 2017 Digishape project: "Heel AHN2 is inmiddels ingeladen in de Microsoft Azure cloud" - De Boer, Hoonhout

iPython Notebook as-a-service running on AHN cache

Page 12: DSD-NL 2017 Digishape project: "Heel AHN2 is inmiddels ingeladen in de Microsoft Azure cloud" - De Boer, Hoonhout

Why Azure despite hurdles? Compliance assurances: 3rd party checks

12

Un

ited

Sta

tes

Ind

ust

ryR

eg

ion

al

ISO27001

PCI DSSLevel 1

SOC 1Type 2

SOC 2Type 2

ISO27018

Cloud ControlsMatrix v3.0.1

Content Delivery andSecurity Association

SharedAssessments

SOC 3 MPAA

SingaporeMTCS Level 3

United KingdomG-Cloud

Australian SignalsDirectorate

New ZealandGCIO

China GB18030

European UnionModel Clauses

Argentine DataProtection Act 25.326

ChinaTRUCS

CanadianPrivacy Laws

FISMAHIPAA /HITECH

FIPS140-2

DISALevel 2

(DIACAP)

FERPAFedRAMPJAB

P-ATO

21 CFRPart 11

IRS1075

Section508

VPATs

NIST800-171

MARS-E GxP DIACAP

Japan MyNumber Act

JapanFinancial Services

ENISAIAF

Cloud SecurityMark Gold

SpainENS

FACT EU-USPrivacy Shield

ISO/IEC27017

China Multi LayerProtection Scheme

ISO 22301

Germany ITGrundschutz

workbook

Page 13: DSD-NL 2017 Digishape project: "Heel AHN2 is inmiddels ingeladen in de Microsoft Azure cloud" - De Boer, Hoonhout

It can work: CoVadem: Van Oord hosted shared notebook in Azure

https://publicwiki.deltares.nl/display/TKIP/DEL036+-+CoVadem+pilot+Waal+Home

Page 14: DSD-NL 2017 Digishape project: "Heel AHN2 is inmiddels ingeladen in de Microsoft Azure cloud" - De Boer, Hoonhout

DigiShape use cases: Flood safety, North Sea workability, …

Cloud:

bring model tosensitive data,

instead ofdata to model.

https://repository.tudelft.nl/islandora/object/uuid%3Aad4432da-86d6-4208-b177-69c8ef576221?collection=education https://repository.tudelft.nl/islandora/object/uuid%3A620bc4f5-d1ba-42b3-b4a7-5cd389d4e331?collection=education

Page 15: DSD-NL 2017 Digishape project: "Heel AHN2 is inmiddels ingeladen in de Microsoft Azure cloud" - De Boer, Hoonhout

• Docker spints session every month

• https://github.com/openearth-stack

• Van Oord, Tu Delft, Deltares, MictosoftNLeSC, KNMI

• Sign-up: [email protected]

OpenEarth Docker Azure DigiShape

15

Technical level

• DigiShape kern groep every month

• https://www.digihape.nl

• IHM, RWS, Deltares, Topsector Water,Maris Van Oord, KNMI, Van Oord,S&T corp., WUR, Mictosoft, HKV, SMEs

• Sign-up: [email protected]

Organizational level