Global Access to Resources Using Architectureagenda.ct.infn.it/event/1109/contribution/6/material/slides/0.pdf · English ‐a total of 110 language pairs. ... Statistical machine

12/18/2014

1

1

Global Access to Resources Using Distributed Architecture

Asvija BCentre for Development of Advanced Computing (C-DAC),

Bangalore, INDIA

2

GARUDA ‐Motivation

Sharing of high‐end computational resources with the larger scientific and engineering community across the country.

Creating a Collaborative Framework for solving applications which are interdisciplinary requiring experts from multiple domains and distributed locations

Providing Universal access to resources

12/18/2014

2

3

Scenario

GG-BLR GG-CHE GG-HYD TF BLR TF PUNEIITD YUVAPRL

Climate Modeling

Disaster Manageme

nt

Bio Informatics

CFDCrypt analysis

How to Make Super Computers available to these users

3

4

GARUDA

75+ Partners

1700+ Certificates

6000 CPUs 550 TFs

220TB Storage

EGI, CHAIN‐REDS, caBig

NKN

India’s National Grid Computing Initiative

12/18/2014

3

5

GARUDA Backbone

InternetAccessGeneric

User

Local UserLAN

CC--DAC, PuneDAC, Pune

Partner Partner without resourceswithout resources

Access Terminals

PartnerPartnerwith resourceswith resources

LAN

Telescope

Local UserAccess Terminal

AccessTerminal

Access Terminal

CC--DAC, DAC, BangaloreBangalore

LAN Local User

Access Terminal

HeadNode

DatabaseDatabase

HeadNodeHeadNode

DatabaseDatabaseDatabaseDatabase

HeadNode

HeadNode

HeadNode

HeadNode

DatabaseDatabase

HeadNodeHeadNode

DatabaseDatabaseDatabaseDatabase HeadNode

HeadNode

Storage

Compute Nodes

Port

Switch

HeadNode

DatabaseDatabase

HeadNodeHeadNode


HeadNodeHeadNode

HeadNodeGrid

Headnode

Grid Headnode

Port

Switch

Port Switch

Submit Node/

Grid Head node

PortSwitch

M P L S AccessM P L S Access

Submit Node/

Grid Head node

InternetAccess

InternetAccessGeneric

User

Local UserLAN

CC--DAC, PuneDAC, Pune

Partner Partner without resourceswithout resources

Access Terminals

PartnerPartnerwith resourceswith resources

LAN

TelescopeTelescope

Local UserAccess Terminal

AccessTerminal

Access Terminal

CC--DAC, DAC, BangaloreBangalore

LAN Local User

Access Terminal

HeadNode

DatabaseDatabase

HeadNodeHeadNode


HeadNodeHeadNodeHeadNode


HeadNodeHeadNode

HeadNode

DatabaseDatabase

HeadNodeHeadNode

DatabaseDatabaseDatabaseDatabase HeadNodeHeadNodeHeadNode


StorageStorage

Compute Nodes

Port

Switch

HeadNode

DatabaseDatabase

HeadNodeHeadNode


HeadNodeHeadNode

HeadNodeHeadNodeHeadNodeGrid

Headnode

Grid Headnode

Port

Switch

Port Switch

Submit Node/

Grid Head node

Submit Node/

Grid Head node

PortSwitch

M P L S AccessM P L S AccessM P L S AccessM P L S Access

Submit Node/

Grid Head node

NKN

6

High Level Architecture

Computing Resources and Virtual Organizations

Research Organizations

Educational institutions Computing Centers

WSRF+GT4 + other Services +Cloud S/w]

NKN

Grid-Enabled Applications

Grid PSE

Virtualization support

Workflow tool

Job Scheduler

Grid Security and High-Performance Grid Networking

Non‐Research Organizations

Data G

rid

Resou

rce Enab

ler & M

onitorin

g

CDAC Resource centers

Access PortalCLI

Visualization

Federated

Information Server

Grid ProgrammingEnvironment

Grid ApplicationsSecurity

Resource Management

User Environments Middleware

Grid Programming & Development Environment -MpichG2 -Compiler Service

12/18/2014

4

7

Grid Middleware

Middleware is a layer between grid applications and low level functionality of grid.

Grid Middleware is a layer of S/W to enforce:

Scalability, Transparency, Heterogeneity, Fault Tolerance and Security of the Grid below the application layers.

It provides an uniform interface to the Grid users and handle all the complexity generated due to heterogeneous systems.

8

Components of Grid Middleware

12/18/2014

5

9

Popular Middleware

Globus – Globus Alliance

GridBus – University of Melbourne

UNICORE ‐ Uniform Interface to Computing Resource

gLite – CERN / EGEE

Legion – (Avaki ‐ Corporate Distributor)

Alchemi – (.NET Grid Computing Framework)

10

12/18/2014

6

11

– Indian Grid Certification Authority located at C-DAC, Knowledge Park, Bangalore, India.

– IGCA is the accredited member of APGridPMA.

– Issues X.509 Certificates to support the secure environment in Grid. (for GARUDA, institutes that do research in grid from India and foreign institutes that collaborate with GARUDA).

IGCA

12

IGTF

The International Grid Trust Federation (IGTF) is a body to establish common policies and guidelines between its Policy Management Authorities (PMAs) members.

http://www.igtf.net/

12/18/2014

7

13

Virtual User Community in GarudaGroup Name Description

Bioinformatics application of statistics and computer science to the molecular biology

ClimateModelling Deals with the dynamics of the climate system.

OSDD Community dedicated to develop drugs for tropical infectious diseases like malaria, tuberculosis

GeoPhysics Study related to physics of the Earth and its environment in space

CAE usage of computer software to solve engineering problems

IndianHeritage Focused on technology products for preserving & processing Heritage texts

HealthInformatics

Focused on utilizing compute power for health informatics

MaterialScience interdisciplinary field applying the properties of matter to science and engineering

Euindia The vision of a worldwide Grid for Research by both Europe and India

ToolsDeveloper Forum to communicate and collaborate on developing Garuda Tools

14

GARUDA Tools & Services

12/18/2014

8

15GBC-2013- Pandharpur

Access Methods

Two ways of Accessing Garuda Resources

Command Line Interface

Web Interface


Command Line Interface

Provides Command Line Access to Garuda Resources

Requirements Valid IGCA Certificate User is required to have Garuda User Account

Facilities Certificate and Proxy Management Single Sign‐On to all Garuda Resources Job Submission, Monitoring and Management Data transfer and Management Querying Resource Information Application Compilation Facility Accomplishing Resource Reservation

Useful for Advanced Users

12/18/2014

9


Web Interface Provides Garuda Access using Web Browser Requirements

Valid IGCA Certificate Garuda User Account

Provides easy and intuitive interfaces for Certificate and Proxy Management Job Submission, Monitoring and Management Data transfer and Management Querying Resource Information Carrying out Resource Reservation

For Novice users Web Interfaces

GARUDA Access Portal Problem Solving Environment


GARUDA Access Portal

12/18/2014

10


• Job Submission

• Resubmission of jobs and checking the job status using Mobile phone

• Browse Available resources

• Download any files from Grid Head Node

• Provision to View Status, Output, Error Files

• Portal.garudaindia.in

GARUDA Access Portal ‐ Features


GARUDA Job Submission

Supports sequential and parallel Jobs

The user can specify the required environment variables

Uses the scheduler for submission

The files can be taken from Data Grid, the submit node, or the local machine

12/18/2014

11

21

• The status of submitted jobs can be viewed at any point of time

• The output and error files can be viewed or downloaded

• Cancel Jobs

GARUDA Access Portal ‐ Job Information


Problem Solving Environment

12/18/2014

12


Protein Structure Prediction

Protein Structure Prediction (PSP) based on evolutionary computing (Genetic Algorithms) method.

Functional modules of the PSP application implemented as grid services on Garuda

Released PSE‐PSP v1.0

Accessible from Garuda Portal

24GBC-2013- Pandharpur 24

Utility Services

12/18/2014

13


Login Service

The Login service provides an abstract mechanism for an application to securely verify the user credentials.

Login service can be integrated to any application portals in GARUDA.


Login Service

Login Service

12/18/2014

14


Grid Support

To log your support request

Go to http://gridsupport.garudaindia.in

Login with your GARUDA Account.

Raise ticket in appropriate queue.

To raise a problem ticket

Select the queue

Click new ticket

Support request can also be sent via e‐mail


Garuda RT (Request Tracker) Login page

Raising a ticket

12/18/2014

15


List of support queues in Garuda

Sl. No. Project Name Support queue E-mail

1 Any grid related problem GDeployment [email protected]

2 Grid Portal Portal [email protected]

3 Service Oriented Architecture

SOA [email protected]

4 GARUDAVOMS Voms [email protected]

5 Grid IDE Tool GridIDE [email protected]

6 OSDD OSDD [email protected]

7 Garuda Network Related Netops [email protected]

8 GARUDA SRM GDS [email protected]

9 Not sure about the queue Other [email protected]

30

Applications on Garuda

12/18/2014

16

31

Grid Enabled Applications

• Objectives– Enable applications of

national importance – Deliverables

– Grid enablement of illustrative applications • Disaster

Management• Bioinformatics

– Other Applications• Geomatics• Earthquake

Research • Weather and Climate

Modeling• Evolutionary

Computing

32

Bioinformatics : Open Source Drug Discovery

• OSDD users given access to Garuda through OSDD VO

• HPC clusters to run drug discovery problems

• Users connected through both NKN and Internet

• Galaxy Workflow for genomics proteomics applications

• Distributed job execution through Gridway

Project Team : OSSD community

12/18/2014

17

33

Winglet Application

Designing of the winglet to reduce the drag and hence improve the fuel efficiency.

Winglet application simulation has been grid enabled to achieve remarkable improvements in the simulation time

In collaboration with Zeus Numerix

34

Disaster Management - SAC DMSAR System

DMSAR scans the terrain and capture the data in 10,5,1m resolution modes The min amount of raw data captured in a single run is 9GB

12/18/2014

18

35

SAC DMSAR System Cont ..

One flight schedule will have ‘n’ no of runs Flight duration – 3Hrs Speed 450km/hr Data collection time – 1 to 1 ½ hrsAmount of data collected per flight = 50 – 80 GB

36

Grid Enabled DMSAR System

Raw Data Transfer

Disaster

Slide courtesy : Mr. Manavalan

12/18/2014

19

37

Raw data Transfer

Collecting Data & Recording

Raw data Transmission to the Grid Head Node

38

12/18/2014

20

39

40

NLP on Garuda Centre for Indian Language Technology ‐ IIT Bombay

Services Available on Megha Cloud resources

Wordnets for Indian Languages

Hindi

http://megha.garudaindia.in/iitb‐nlp/hindiwn/

Marathi

http://megha.garudaindia.in/iitb‐nlp/marathiwn/

Sanskrit

http://megha.garudaindia.in/iitb‐nlp/sanskritwn/

12/18/2014

21

41

NLP ‐ Other services Sata‐Anuvadak

Broad coverage statistical machine translation system for Indian languages.

Translation among 10 major Indian languages and English ‐ a total of 110 language pairs.

Currently adapted for tourism and health domains.

Sata‐Lipyaantaran

Statistical machine transliteration system for Indian languages.

4217/11/2010 Indo-Russian Workshop

Collaborations Space Application Centre, Ahmedabad

Disaster Management Grid Middleware for Satellite Grids

Indian Institute of Science, Bangalore Climate Modeling

Madras Institute of Technology, Chennai Grid Middleware for Semantic Grid Services

Indian Institute of Technology, Mumbai Computational Fluid Dynamics Package on PARAM Padma

University of Pune Quantum Chemistry, Materials Modeling, Bioinformatics INDMOL Molecular Package developed

EU-India Grid BARC, C-DAC, ERNET, Saha Institute, TIFR, Univ. Pune Consortium GARR, ICTP, INFN (Italy) Cambridge University (UK)

12/18/2014

22

43

EU‐Indiagrid Project Goals

To support the interconnection and interoperability of the prominent European Grid infrastructure (EGEE) with the Indian Grid infrastructure (Garuda National Grid Initiative, DAE‐Grid) for the benefit of eScience applications;

To identify and aggregate research, scientific and industrial communities which may benefit from the use of Grid technology resulting in an eScienceNetwork Community

44

PARTICIPANTSEUROPE Italian National Institute of Nuclear Physics, INFN (project

coordinator), Metaware SpA, Italian Academic and Research Network (GARR) Cambridge University

INTERNATIONAL Abdus Salam International Centre for Theoretical Physics (ICTP) INDIA Indian Education and Research Network (ERNET), University of Pune, SAHA Institute of Nuclear Physics, Kolkata, Centre for Development of Advanced Computing (C‐DAC), Bhabha Atomic Research Centre, Mumbai TATA Institute for Fundamental Research (Mumbai)

National Centre for Biological Sciences (Bangalore)

12/18/2014

23

45

caBIG‐GARUDA

Collaboration between cancer research grid from NCI (National Cancer Institute, USA) and GARUDA.

Achieve interoperability among technologies of both the grids.

To provision tools available on caGrid to Indian oncologists.

As POC, caTissue tool has been setup.

46

CHAIN‐REDS

12/18/2014

24

47

CHAIN‐REDS

CHAIN‐REDS is funded by the European Commission aiming at promoting and supporting technological and scientific collaboration across different e‐Infrastructures established and operated in various continents.

http://www.chain‐project.eu/

4817/11/2010 Indo-Russian Workshop

International Visibility

12/18/2014

25

49

50

Thrust application areas

Addressing National Challenges in

Agriculture

Energy

Biodiversity

12/18/2014

26

51

Agriculture Challenges

Increase Crop productivity,

Water management & Drought prediction

Less use of pesticide

Developing new seeds with inbuilt immune for diseases

Technical Solutions

Building Mathematical models for Drought prediction

Creating ontological databases

Analytics for crop tolerance

towards drought, water, pesticides etc.,

52

Biodiversity

Challenges

Understanding the ecological relationship among species

Modeling of Migration and extinctions

Habitat changes due to climate change and natural calamities

Conservation and sustainability of Bio‐resources

Technical Solution

Tools for efficient access and analyses of data

Image courtesy : NBA India

12/18/2014

27

53

Energy Sector Challenges

Demand & supply mismatch

Quality of generated power

Continuous power supply to critical unit

Avoid power loss

Transmission & theft

Technical Solutions

Modeling of weather & integrate with Power system

Load flow analysis using Analytics system to predict

Forecasting of trends and demands

Preempting catastrophe

54

Thank You !!

Documents

Global Access to Resources Using Architectureagenda.ct.infn.it/event/1109/contribution/6/material/slides/0.pdf · English ‐a total of 110 language pairs. ... Statistical machine