24
EC-Earth and ESGF Kim Serradell Pierre-Antoine Bretonniere (Computational Earth Sciences, BSC) Prashanth Dwarakanath (Linkoping University/NSC) 03/12/2017 7th Annual ESGF F2F 2017 A R

EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic

  • Upload
    others

  • View
    4

  • Download
    0

Embed Size (px)

Citation preview

Page 1: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic

EC-Earth andESGF

KimSerradellPierre-Antoine Bretonniere

(Computational Earth Sciences,BSC)Prashanth Dwarakanath

(Linkoping University/NSC)

03/12/2017 7thAnnual ESGFF2F2017

AR

Page 2: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic

Outline

• BarcelonaSupercomputingCenter• EC-Earthclimatemodel• BSCExperience• LIU• EC-EarthandESGF

Page 3: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic

BarcelonaSupercomputing CenterCentroNacionaldeSupercomputación

SpanishGovernment 60%

CatalonianGovernment 30%

Univ.Politècnica deCatalunya (UPC)10%

BSC-CNSisaconsortiumthatincludes

BSC-CNSobjectives

SupercomputingservicestoSpanishandEUresearchers

R&DinComputer,Life,Earthand

EngineeringSciences

PhDprogram,technologytransfer,publicengagement

Page 4: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic

EarthSciencesDepartmentWhat

Environmental forecasting

HowDevelop a capability to model air quality processes from urban to global and the impacts on weather, health and ecosystems

Implement climate prediction system for subseasonal-to-decadal climate prediction

Develop user-oriented services that favour both technology transfer and adaptation

Use cutting-edge HPC and Big Data technologies for the efficiency and user-friendliness of Earth system models

Why

Our strength …… research …… operations …… services …… high resolution …

Earth system services

Atmospheric composition

Computational Earth sciences

Climate prediction

Page 5: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic

EC-Earth

Page 6: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic

EC-EarthModelEC-EARTH coupled model

• TheIntegratedForecastingSystem(IFS)asatmospheremodel

• TheNucleusforEuropeanModellingoftheOcean(NEMO)asoceanmodel

• TheOASIS3-MCT coupler

• TheLouvain-la-Neuve sea-IceModel3(LIM3)asseaicemodel

Page 7: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic

Communitydrivendevelopment

Schematic outline ofthe EC-Earth consortium andits members asofMay 2014.

Page 8: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic

PRIMAVERAH2020

• a EuropeanUnionHorizon2020 project,• anewgenerationofadvancedandwell-evaluatedhigh-resolutionglobalclimatemodels,

• simulationsandpredictionsofregionalclimatewithunprecedentedfidelity,

• forthebenefitofgovernments,businessandsocietyingeneral.

Oceansurfacecurrentsfrom

HadGEM3-basedglobalcoupled(atmosphere-ocean/sea-ice)modelsatthree

differentresolutions- (left)25km-1/12degree,(middle)60km-1/4degree,(right)130km-1degree(courtesyof

MalcolmRoberts)

Page 9: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic

BSCExperience

Page 10: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic

BSCDataNode

• ESGFTier2“data”node:publishingBSCandAEMETPRIMAVERAandCMIP6data

• IndexnodeatSTFC(partnerinvolvedinPRIMAVERA)

• PRIMAVERAStream1datastilluploadedtoJasminbutpublishedfromBSCnode

• PRIMAVERAStream2datawillbeonlystoredandpublishedfromBSC

• Estimateof~600TBofPRIMAVERAdata

Page 11: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic

BSCDataNode

Indexnode Datanode

Page 12: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic

BSCDataNode

Page 13: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic

BSCcmorization

• Part ofPRIMAVERAproject

• ece2cmor3 is trying tostandardize the cmorization within EC-Earth

• Developed by asetofpartners (KNMI/e-Science CenterandBSC)

• But each partner cmorizes its own data(lots ofcoordination needed)• online(on HPCasthe simulation runs toonly storecmorized data)• offline(adding timetopublish)

Page 14: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic

BSCDataNode(installer)

• March2017:

• startedinstallingversion2.3topublishCMIP5/SPECSdataastest

=>impossibleduetodependenciespointingtoupdatedversionsofpackagesbreakingcompatibilityoftheoverallinstallation

• August2017:

• Successfullyranversion2.5oftheinstaller,evenifstillunderdevelopment

Page 15: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic

BSCDataNode(publisher)

Oncewehadthenodeupandrunning,updating:

• updatingtheesg.[primavera-cmip6].ini

• esgcet_models_table.txt

• certificatesandPIDslandingpages

PRIMAVERAspecific

EC-Earthspecific

BSCspecific

Page 16: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic

BSCrecommendations

• Freshstartinstallationtesting

• Semanticversioningof• dependencies(gitsubmodules?)• releases

• Allowuserstoinstalldependenciesbyhandwithintheinstallerrun

• Identifywho(personorinstitution)isresponsibleforwhat(PIDs,installation,publication,...)

• Wiki/documentationfor“beginners”• Howto’s?• Howtodeployaproductionandatestingserver

Page 17: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic

LIU

Page 18: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic

LiU on ESGFandCDNOT

• LiU operatesaTier-1ESGFnode,offeringallESGFservices,includingindexpeeringtootherdatanodes,fundedbySMHIandIS-ENESprojects.

• HostsCORDEX,CMIP5,SPECS,andCLIPCprojects,fromSMHI.• ManagesattributeservicesforCORDEXdataaccess.• [email protected],forusersregisteredtoaccessCORDEXdataonESGF.

• LiU ManagesoneoftheESGFFederationCAsandhelpeddevelopCApolicyforESGF.

• Prashanth Dwarakanath (LiU)co-leadstheESGFInstallationWorkingTeam,andisarepresentativeforEC-EARTHinCDNOT.

• KaiLu(LiU)isanactivecontributortousersupportonESGFUsersandIWTmailinglists.

Page 19: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic

Ongoing andfuture activities on CDNOT

• LiU hascontributed toachieving better documentation byauthoring manuals for datanode administrators,andlookstotake an activeroleindocumenting procedures for CMIP6datapublication andnode operations.

• Hasconducted trainingworkshopsandcode sprints andisworking on developing asuiteoftools toreduceuncertaintieswith infrastructure setups for workshopsandcollaborativedevelopment events.

• Actively working on setting upofan Incident-responseteam,andtodisseminate traininganddocumentation toensureadoption ofsecurity best-practices.

Page 20: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic

EC-EarthandESGF

Page 21: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic

Previousexperiences

• CMIP5andSPECS• CentralizedatBADC

• Onlyareducedsetofpartnerswasinvolvedinthepublic• DataNodeinLIU• Partnerswithout datanodehadto:

• Upload lotsofdata• Needingassistance

Page 22: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic

Futureexperiences

• CMIP6(PRIMAVERA)• Moredata

• spatialresolutionincreasing• spatialdisseminationincreasing(morepartners)

• Morepartners• moreESGF/EC-Earthdatanodes• more“ESGFbeginners”

• Howtodealwithsuchaproject?• attheEC-Earthlevel• attheESGFlevel

Page 23: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic

EC-Earth’scontributionstoCMIP6Institute Contact Contributions HPCplatform ESGFnode

1.AEMET J.A.ParodiPerdomo

DECK(EC-Earth3-CC) CrayXC30@ECMWF+BULL@AEMET

?(BSC?)

2.BSC/IC3 F. Doblas-Reyes DECK,DCPP,HighResMIP (?) Marenostrum@BSC BSC

3.CNR J.vonHardenberg DECK (EC-Earth-CC,EC-Earth-CC-LR) CNR

4.DMI S. Yang DECK,DCPP,GeoMIP,HighResMIP, ISMIP6,LS3MIP,SenarioMIP

CrayXC30@DMI/IMO DMI

5.ENER A.Alessandri LS3MIP CrayXC30@ECMWF

6.FMI H.Korhonen AeroChemMIP CrayXC30@FMI FMI/UHEL

7.IPMA P.Viterbo ??

8. KIT A.Arneth C4MIP,LUMIP

9.KNMI R.Bintanja DECK,AeroChemMIP,CFMIP(?),HighResMIP,SenarioMIP

CrayXC30@ECMWF+BULL@KNMI

KNMI

10.MetEirre/ICHEC

R.McGrath DECK,HighResMIP,SenarioMIP CrayXC30@ECMWF+CrayXC40@PRACE

ICHEC

11.LU P. Miller C4MIP,LUMIP

12.MISU L.Brodeau DECK

13.NGSU Q.Zhang PMIP NSC

14.SMHI K.Wyser DECK,C4MIP,HighResMIP,C4MIP,SenarioMIP

LIU

15.UHEL R.Makkonen AeroChemMIP CrayXC30@CSC FMI/UHEL

Page 24: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic

Thank you

[email protected]