Grid Portal Development for Sensing Data  Retrieval and Processing

Diego Arias, Mariana Mendoza, Fernando Cintron, Kennie Cruz, and Wilson Rivera

Parallel and Distributed Computing LaboratoryUniversity of Puerto Rico at Mayaguez

GCE06-SC06 – Tampa, FLNovember 12-13, 2006

Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 2

Agenda

Introduction CASA: Collaborative and Adaptive Sensing of the Atmosphere

– NSF: EEC-031374; UMass, UPRM, OU, CSU

WALSAIP: Wide Area Large Scale Automated Information Processing– NSF: CISE-CNS-0424546; UPRM

Remarks and conclusions

Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 3

Introduction: The PDC Laboratory

Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 4

Introduction: PDCLab Research

Optimized e-Resource Administration

Automated Composition Signal Processing Operators

Adaptive Provisioning and Orchestration of Grid Services

PDC Portal

WALSAIPPortal

Information Dispersal STB Portal

CenSSIS Portal

Semi-Analytical Inversion Model

Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 5

Introduction: PDCLab Grid Testbed

Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 6

CentOS 4.2 Globus Toolkit 4.0.1.

– OpenPBS

– Torque

– PosgreSQL

– Apache Ant version 1.6.5

– Java SDK version 1.5

– Jakarta Tomcat version 5.5.9.

Introduction: PDCLab Grid Testbed

Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 7

Introduction: PDCLab Grid Testbed

Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 8

Agenda

Introduction CASA: Collaborative and Adaptive Sensing of the Atmosphere

– NSF: EEC-031374; UMass, UPRM, OU, CSU

WALSAIP: Wide Area Large Scale Automated Information Processing– NSF: CISE-CNS-0424546; UPRM

Remarks and conclusions

Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 9

10,000 ft

tornado

wind

earth surface

snow3.

05 k

m

0 40 80 120 160 200 240RANGE (km)

Horz. Scale: 1” = 50 kmVert. Scale: 1” -=- 2 km

5.4

km

1 km

2 km

4 km

gap

CASA: Collaborative and Adaptive Sensing

gap - earth curvature prevents 72% of the troposphere below 1 km from being observed.

Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 10

Academic Institutions/Researchers

Government

Media

Service/Request

Data

Service/RequestData

CASA: Student Testbed (STB)

Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 11

CASA: STB Grid Portal Interface

Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 12

CASA: STB Grid Portal Interface

Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 13

CASA: Information Dispersal Algorithm

IDA - Dispersal Operation

0

100

200

300

400

500

600

700

800

900

1000

1100

1200

1300

1400

1500

0 100 200 300 400 500 600 700 800 900 1000

Data Size (MB)

Ela

ps

ed

Tim

e (

Se

c)

IDA-OLD

8MMA

8MAP

8FRL

16MMA

16MAP

16FRL

IDA - Dispersal Operation

0

100

200

300

400

500

600

700

800

900

1000

0 100 200 300 400 500 600 700 800 900 1000

Data Size (MB)

Ela

ps

ed

Tim

e (

Se

c)

8MMA

8MAP

8FRL

16MMA

16MAP

16FRL

Data File F

6

2

31

7 1112

10

5

9

8

4

IDA (12,4)

Data Retrieval

Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 14

CASA: Data Management Portlets

Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 15

CASA: Weather Information Portlets

Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 16

CASA: Job Submission

Client

GridPortal

Resources

GSI

JOB MANAGER

GateKeeper

Resource Manager

Process

Process

Process

1 2 3 4 5 6 7 8 9 10

98 98 98 97 98 97 9794 93 93

2 2 1 1 1 1 1 0 0 00

10

20

30

40

50

60

70

80

90

100

CP

U U

sag

e (

%)

Netcdf to JPG - CPU Usage

Grid Server

Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 17

Agenda

Introduction CASA: Collaborative and Adaptive Sensing of the Atmosphere

– NSF: EEC-031374; UMass, UPRM, OU, CSU

WALSAIP: Wide Area Large Scale Automated Information Processing– NSF: CISE-CNS-0424546; UPRM

Remarks and conclusions

Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 18

WALSAIP: Signal Based Information Processing System

Physical Environment

Physical Environment

Observables

Signals

Data

Sensors Effectors

SignalProcessing

Information Processing

InformationKnowledgeProcessing

Knowledge

Decision System

Intelligence

Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 19

SAS: Sensor Array Structures

DRS: Data Representation System

CSPS: Computational Signal Processing System

SDP: Signal Data Post-processing

SCS: Signal Conditioning System

RDS: Raw Data Server

CDS: Computed Data Server

IRS: Information Rendering System

DRSSCSSAS

RDS CDS

CSPS SDP

IRS

IRS

IRS

Pre-processing StagePost-processing Stage

Processing Stage

SAS

INTERNET

WALSAIP: Conceptual Framework Design

Net-CentricMIMO

Services

Net-CentricMIMO

Services

DRSSCSSAS

CSPS

SDP

IRS

IRS

IRS

SDP

IRS

IRS

IRS

DRS

SCS

SAS

RDS DBMS

RDS DBMS

CDS DBMS

CIP over a Cyberspace Infrastructure

CSPS

CIP: Computational and Information Processing

Net-CentricMIMO

Services

Net-CentricMIMO

Services

DRSSCSSAS

CSPS

SDP

IRS

IRS

IRS

SDP

IRS

IRS

IRS

DRS

SCS

SAS

RDS DBMS

RDS DBMS

CDS DBMS

CIP over a Cyberspace Infrastructure

CSPS

CIP: Computational and Information Processing

Net-CentricMIMO

Services

Net-CentricMIMO

Services

DRSSCSSAS

CSPS

SDP

IRS

IRS

IRS

SDP

IRS

IRS

IRS

DRS

SCS

SAS

RDS DBMS

RDS DBMS

CDS DBMS

CIP over a Cyberspace Infrastructure

CSPS

CIP: Computational and Information Processing

Net-CentricMIMO

Services

Net-CentricMIMO

Services

DRSSCSSAS

CSPS

SDP

IRS

IRS

IRS

SDP

IRS

IRS

IRS

DRS

SCS

SAS

RDS DBMS

RDS DBMS

CDS DBMS

CIP over a Cyberspace Infrastructure

CSPS

CIP: Computational and Information Processing

Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 24

WALSAIP’s Jobos Bay NERR Environmental Surveillance and Monitoring Testbed

@ Jobos Bay NERR @ UPRM

INTERNETINTERNET

RF-Link

MEP-510

MEP-410

RS-232Base Station

MBR410

Local Host

Data Base on Host

Database Querying and Processing

Engine

Sensing-Data

SensorNode1

SensorNode5

Cooked-data

1-Jun022006-21:02:00-humidity-data.txt

1-Jun022006-21:02:00-humidity-meta.txt

Remote User

Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 25

WALSAIP: Grid Portal Interface

Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 26

WALSAIP: Grid Portal Interface

Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 27

WALSAIP: Grid Portal Interface

Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 28

Agenda

Introduction CASA: Collaborative and Adaptive Sensing of the Atmosphere

– NSF: EEC-031374; UMass, UPRM, OU, CSU

WALSAIP: Wide Area Large Scale Automated Information Processing– NSF: CISE-CNS-0424546; UPRM

Remarks and conclusions

Wilson Rivera, Parallel and Distributed Computing Laboratory @ UPRM 29

Remarks and Conclusions

Grid portals for large scale applications – Sensing of the Atmosphere (CASA)

– Environmental Surveillance and Monitoring (WALSAIP)

Grid services provided via portlets– Distributed data management

– Information dispersal

– Weather services

– Multiple job submission

– Service orchestration

Future work – Remote instrumentation via grid portals

– Integration to community grid infrastructures (Teragrid, etc….)