Joint agINFRA and SCI-BUS Workshop & agINFRA Open Stakeholder Day, 30 May 2013 - SZTAKI, Hungary

FP 7-INFRASTRUCTURES

Grid and Cloud Infrastructures for Agriculture

Antun BalazInstitute of Physics Belgrade, Serbia

Joint agINFRA and SCI-BUS workshop &agINFRA Open Stakeholder Day

30 May 2013, SZTAKI, Budapest, Hungary

Joint agINFRA and SCI-BUS Workshop & agINFRA Open Stakeholder Day, 30 May 2013 - SZTAKI, Hungary

FP 7-INFRASTRUCTURES

Overview

• agINFRA infrastructure• Hardware resources• Grid resources• Scientific Gateways• agINFRA RESTful gateway• Key results

Joint agINFRA and SCI-BUS Workshop & agINFRA Open Stakeholder Day, 30 May 2013 - SZTAKI, Hungary

FP 7-INFRASTRUCTURES

agINFRA Infrastructure

• Technically divided into three main layers:– Hardware resources (Grid and Cloud)– Scientific Gateways (Catania SG, WS-PGRADE SG, RESTful gateway)– Integrates services (Organic.Edunet, AGRIS, CIARD RING, AgLR-TF)

Joint agINFRA and SCI-BUS Workshop & agINFRA Open Stakeholder Day, 30 May 2013 - SZTAKI, Hungary

FP 7-INFRASTRUCTURES

Hardware resources• Resource centers

– PARADOX cluster – 88 x (2 x 4-core Xeon E5345@2.33 GHz) = 704 CPU (1 GB of RAM / core)50 TB of disk space, Gigabit Ethernet

– Roma Tre cluster – 350 CPUs; 100TB– Catania cluster – 800 CPUs; 700 TB – SZTAKI cluster – 8 CPUs– PARADOX ugrade – 106 x (2 x 8-core Xeon E5-2670@2.6 GHz)

= 1696 CPU (2 GB of RAM / core)100 TB of disk space, Infiniband

• Virtualized resources– SZTAKI Desktop Grid

90 000 volunteer resources; 3 Tflops.– EDGeS@home - 15 000 CPUs

Joint agINFRA and SCI-BUS Workshop & agINFRA Open Stakeholder Day, 30 May 2013 - SZTAKI, Hungary

FP 7-INFRASTRUCTURES

Grid resources [1/2]

• agINFRA Virtual Organization: vo.aginfra.euhttps://voms.ipb.ac.rs:8443/voms/vo.aginfra.eu https://voms.ct.infn.it:8443/voms/vo.aginfra.eu

• MyProxy service (PX) – myproxy.ipb.ac.rs

• Computing Elements (CREAM)– cream.ipb.ac.rs:8443/cream-pbs-aginfra – 704 CPUs– ce-01.roma3.infn.it:8443/cream-pbs-grid – 328 CPUs– ce-02.roma3.infn.it:8443/cream-pbs-grid – 208 CPUs– grid012.ct.infn.it:8443/cream-lsf-aginfra – 20 CPUs

Joint agINFRA and SCI-BUS Workshop & agINFRA Open Stakeholder Day, 30 May 2013 - SZTAKI, Hungary

FP 7-INFRASTRUCTURES

Grid resources [2/2]• Logical File Catalogue (LFC)

– lfc.ipb.ac.rs• Storage Elements (SE) – 850 TB

– dpm.ipb.ac.rs– prod-se-02.ct.infn.it– storm-01.roma3.infn.it

• Metadata Catalogue (AMGA)– amga.ct.infn.it

• Workload Management System (WMS)– wms.ipb.ac.rs– wms-demo.ipb.ac.rs– wms-aegis.ipb.ac.rs

• Logging and Bookkeeping (LB)– wms.ipb.ac.rs– wms-demo.ipb.ac.rs– wms-aegis.ipb.ac.rs

• Information System (BDII)– bdii.ipb.ac.rs

Joint agINFRA and SCI-BUS Workshop & agINFRA Open Stakeholder Day, 30 May 2013 - SZTAKI, Hungary

FP 7-INFRASTRUCTURES

Scientific Gateways

• Catania Scientific Gatewayhttps://aginfra-sg.ct.infn.it/

• gUSE/WS-PGRADE Portal (workflows)http://aginfra-portal.lpds.sztaki.hu/liferay-portal-6.0.5/

Joint agINFRA and SCI-BUS Workshop & agINFRA Open Stakeholder Day, 30 May 2013 - SZTAKI, Hungary

FP 7-INFRASTRUCTURES

agINFRA RESTful gateway• Cataloging

– User’s configurations of Grid-ported applications– Datasets’ metadata information and locations– Arbitrary user-defined metadata information– Querying of these information

• Off-line processing– Collect new datasets – Transform existing datasets– Produce new datasets’ metadata information

• Data management– Automatic replication of datasets– Ensures existence of the same dataset on different storage systems– Exposure through the HTTP protocol

Joint agINFRA and SCI-BUS Workshop & agINFRA Open Stakeholder Day, 30 May 2013 - SZTAKI, Hungary

FP 7-INFRASTRUCTURES

Key results• Design, specification and deployment of the agINFRA infrastructure• Establishment and operation of agINFRA Grid infrastructure (VO:

vo.aginfra.eu, VOMS, BDII, LFC, WMS/LB, PX, AMGA, DPMs, CREAMs)• Establishment and deployment of Cloud infrastructure (CLEVER and

Okeanos)• Deployment of agINFRA Scientific Gateways (Catania and gUSE/WS-

PGRADE SGs)• Development of agINFRA RESTful gateway• Development and deployment of data infrastructure

– dataset exposure layer (RESTful gateway)– data management layer (Grid RESTful API, HTTP exposure)– data transformation layer (agDCtoLOM, agLOMtoAK, agLOMtoRDF)– data processing layer (agDataHarvest, agMetaExtract,

agRecommender)