41
Grid Technologies in Disaster Management & Infrastructure Protection Geoffrey Fox Computer Science, Informatics, Physics Pervasive Technology Laboratories Indiana University Bloomington IN 47404 [email protected] http://grids.ucs.indiana.edu/ptliupages/presentations/CIGr idGDINmar28-04.ppt http:// www.infomall.org http://www.grid2002.org

Grid Technologies in Disaster Management

Embed Size (px)

Citation preview

Page 1: Grid Technologies in Disaster Management

Grid Technologies in Disaster Management & Infrastructure

Protection Geoffrey Fox

Computer Science, Informatics, PhysicsPervasive Technology Laboratories

Indiana University Bloomington IN 47404

[email protected]://grids.ucs.indiana.edu/ptliupages/presentations/CIGridGDINmar28-04.ppt

http://www.infomall.orghttp://www.grid2002.org

Page 2: Grid Technologies in Disaster Management

Introduction• Grid is NOT a mesh in this talk!!!!• The Grid represents “Internet Scale Distributed Computing”

or the “Managed Internet”• Major commercial and research Initiative – IBM HP Oracle

Sun SGI Platform• We describe how one can use familiar system of systems

(Grid of Grids) language for critical infrastructure and emergency response (command and control) Grids

• We give infrastructure examples from flood and earthquake cases with HPC simulations linked to other Grid resources

• Discuss GIS and Collaboration Grids

Page 3: Grid Technologies in Disaster Management

e-Business e-Science and the Grid e-Business captures an emerging view of corporations as

dynamic virtual organizations linking employees, customers and stakeholders across the world.

e-Science is the similar vision for scientific research with international participation in large accelerators, satellites or distributed gene analyses.

The Grid or CyberInfrastructure integrates the best of the Web, Agents, traditional enterprise software, high performance computing and Peer-to-peer systems to provide the information technology e-infrastructure for e-moreorlessanything.

A deluge of data of unprecedented and inevitable size must be managed and understood.

People, computers, data and instruments must be linked. On demand assignment of HPC resources, experts,

computers, networks and storage resources

QuickTime™ and a decompressor

are needed to see this picture.

QuickTime™ and a decompressor

are needed to see this picture.

IMAGING INSTRUMENTS

COMPUTATIONALRESOURCES

LARGE-SCALE DATABASES

DATA ACQUISITION ,ANALYSIS

ADVANCEDVISUALIZATION

Page 4: Grid Technologies in Disaster Management

e-Defense and e-Crisis Grids support Command and Control and provide Global

Situational Awareness • Link commanders and frontline troops to themselves and to archival and

real-time data; link to what-if simulations

• Dynamic heterogeneous wired and wireless networks

• Security and fault tolerance essential

System of Systems; Grid of Grids• The command and information infrastructure of each ship is a Grid; each

fleet is linked together by a Grid; the President is informed by and informs the national defense Grid

Crisis Management and Response enabled by a Grid linking sensors, disaster managers, and first responders with decision support• Grid supports On-demand HPC Simulations

Define and Build DoD relevant Services – Collaboration, Sensors, GIS, Database etc.

Page 5: Grid Technologies in Disaster Management

Raw (HPC) Resources

Middleware

Database

PortalServices

SystemServices

SystemServices

SystemServices

Application Service

Libraries

UserServices

“Core”Grid

Typical Grid Architecture

Application Service

Application Service

Re-use

Re-use

ApplicationCustomization

Each service should beable to run independently on separate machines

Service OrientedArchitectures scalebetter than traditionalDistributed objects

Page 6: Grid Technologies in Disaster Management

A typical Web or Grid Service In principle, services can be in any language (Fortran .. Java ..

Perl .. Python) and the interfaces can be method calls, Java RMI Messages, CGI Web invocations, totally compiled away (inlining)

The simplest implementations involve XML messages (SOAP) and programs written in net friendly languages like Java and Python

PaymentCredit Card

WarehouseShippingcontrol

WSDL interfaces

WSDL interfaces

Security CatalogPortalService

Web Services

Web Services

Page 7: Grid Technologies in Disaster Management

Composing Systems• We are familiar with a hierarchy

Lines of Code Methods Objects Programs Packages

• Grids extend the software hierarchy to larger distributed systems and link it to a corresponding resource hierarchy

Overlayand ComposeGrids of Grids

Methods Services Functional Grids

CPUs Clusters ComputeResource Grids

MPPs

DatabasesFederatedDatabases

Sensor Sensor Nets

DataResource Grids

Page 8: Grid Technologies in Disaster Management

Information Grid

Enterprise Grid

Compute Grid

Campus Grid

R2R1

Teacher

Students

Dynamic light-weight Peer-to-peerCollaborative Training Grid

Composing anOverlay Grid

Page 9: Grid Technologies in Disaster Management

Education Grid

Inservice TeachersPreservice TeachersSchool of Education Teacher Educator

Grids

Informal Education(Museum)

Grid

Student/Parent …Community Grid

Science GridsCI Grids

Typical Science GridService such as ResearchDatabase or simulation

Transformed by Grid Filterto form suitable for education

Learning Managementor LMS Grid

Publisher Grid

Campus orEnterprise

AdministrativeGrid

Education as a Grid of Grids

DigitalLibrary

Grid

Page 10: Grid Technologies in Disaster Management

Solid Earth Research Virtual Solid Earth Research Virtual Observatory (SERVO)Observatory (SERVO)

NASA funded: JPL, UC Davis, UC Irvine, USC, Brown, IndianaWeb/Grid-services and portlet based Problem Solving Environment (PSE)Couples data with simulation, pattern recognition software, and visualizationEnable investigators to seamlessly merge multiple data sets and models, and

create new queries.

Data• Spaced-based observational data• Ground-based sensor data (GPS, seismicity)• Simulation data• Published/historical fault measurements

Analysis Software• Earthquake fault• Lithospheric modeling• Pattern recognition software

International Version iSERVO• Australia China and Japan as an APEC (Asia-Pacific Economic

Cooperation) Initiative – next meeting Beijing July 2004

Page 11: Grid Technologies in Disaster Management

Database Database

ResearchSimulations

Analysis and VisualizationPortal

RepositoriesFederated Databases

Data Filter

Services

Crisis Data

Streaming Data

Sensors

?DiscoveryServices

SERVOGridResearch Crisis

CustomizationServices

From Research

to CI/Crisis

Cell PhoneMobileNetwork

Geoscience Research andCrisis Grids

OtherInterdependentCI and CrisisGrids

GIS

Page 12: Grid Technologies in Disaster Management

Database Database

Analysis and VisualizationPortal

RepositoriesFederated Databases

Data Filter

Services

Field Trip DataStreaming Data

Sensors

?DiscoveryServices

SERVOGrid

ResearchSimulations

Research Education

CustomizationServices

From Research

to Education

EducationGrid ComputerFarm

Geoscience Research andEducation Grids

GIS

Page 13: Grid Technologies in Disaster Management

HPCSimulation

DataFilter

Data FilterD

ata

Filt

er

Data

Filter

Data

Filter

Distributed Filters massage dataFor simulation

Other

Grid

and W

eb

Servi

ces

AnalysisControl

Visualize

SERVOGrid (Complexity) Computing Model

Grid

OGSA-DAIGrid Services

This Type of Gridintegrates with

Parallel computingMultiple HPC

facilities but only use one at a time

Many simultaneous data sources and

sinks

Grid Data Assimilation

Page 14: Grid Technologies in Disaster Management

Crisis and CI Grids• We first form Critical Infrastructure CI Grids from the

“atomic” functional and resource Grids and Services– This gives water (flood), gas, postal, agricultural etc. Grids

• Then we customize, compose and overlay CI with other Grids (such as weather, census data) for – Public Health

– Emergency Response (Command and Control) or Crisis Grids

– Infrastructure Planning (e.g. transportation planning)

– Education (schools) and Training (of managers and first responders)

– Interdependency analyses between the different infrastructures

• Note need to link to “Cell Phone Grid” for first responders with services that reduce image sizes etc.

Page 15: Grid Technologies in Disaster Management

What are we trying to do• Center for Disaster and Critical Infrastructure Informatics

– IUPUI, Indiana, Purdue, openGIS Consortium project to build a generic Crisis/CI Grid infrastructure with a FloodGrid as initial prototype

• SERVOGrid (Solid Earth Research Virtual Observatory) building Grid to support Earthquake Forecasting

• GlobalMMCS is an open-source Audio-Video conferencing and collaboration system built totally on Web Services and spanning Access Grid, H323, SIP and WebCAM’s.

• NaradaBrokering is a Grid message system supporting reliable message delivery, software multicast, firewall friendliness, security. (Grid as managed web)

• Hope to partner with NISAC (National Infrastructure Simulation and Analysis Center) on core CIGrids

Page 16: Grid Technologies in Disaster Management

Minicomputer

Firewall

ComputerServer

PDA

Modem

Laptop computerWorkstationPeers

Peers

Audio/VideoConferencing Client

Audio/VideoConferencing Client

NaradaBrokering BrokerNetwork

NaradaBrokering

Queues

Web Service A

Web Service B

Stream

Reliable Routedsoftware multi-castSecure filtered Message delivery

Page 17: Grid Technologies in Disaster Management

CI/Crisis Grid of Grids Services in a CI or Crisis Grid fall into three classes 1) Those that special to Critical Infrastructure or

Crises such as biological sensor nets or evacuation planning service

2) Those that are important but can be taken from other Grids such as GIS or collaboration and security

3) Those that come from other Grids and are refactored for crises• The simulation is reduced in size• The public health database interface is simplified

GridResource

Filter CI/Crisis Grid View

of generic Grid Resource

CI/Crisis Grid

Page 18: Grid Technologies in Disaster Management

Gas Servicesand Filters

Security Workflow MessagingNotification

Metadata Data Access/StorageRegistry

Physical Network

Compute

Sensors Visualization

Collaboration

openGISServices

Flood Servicesand Filters

Flood CIGrid Gas CIGrid… Electricity CIGrid …

Critical Infrastructure CIGrids

Page 19: Grid Technologies in Disaster Management

openGIS Grid Semantics• Note GIS (Geographical Information System) Grid at heart of all

these Grids• Geography Markup Language (GML) is an XML encoding for the

specification of the geometry and properties of geographic features. GML utilizes the OpenGIS Abstract Specification geometry model which has been harmonized with the ISO geospatial geometry model. – We are building CI specific ontologies in terms of GML to define

faults, satellites etc.– http://ripvanwinkle.ucs.indiana.edu:4780/examples/download/schema/

• Styled Layer Descriptor (SLD) specifies the format of a map-styling language for portraying the output of Web Map Servers, Web Feature Servers and Web Coverage Servers etc. SLD will enable different communities in the Emergency Response area to develop a set of customized portrayal rules that best fit their mission requirements.– This becomes the specification of portals to different composite Grids

• Sensor Markup Language (SensorML) defines the information model for discovering, querying and controlling Web-resident sensors.

• Observations & Measurements (O&M) defines the information model for observations that are returned from the CrisisGrid sensors.

Page 20: Grid Technologies in Disaster Management

GIS Grid Services I• Web Feature Service (WFS) supports the query and discovery of

geographic features delivering GML representations of simple geospatial features in response to queries from HTTP clients. WFS can access geographic features including critical infrastructure features, incident locations, and flood-related geographic features including inundation areas, watershed boundaries, and demographic feature.

• Web Coverage Service (WCS) supports the query and discovery of digital geospatial information such as digital elevation models, imagery, orthophotography, weather coverages (such as predicted rainfall, air pressure, wind speed and direction), and any other space-varying flood-related phenomena.

• Web Map Service (WMS) uses a SLD portrayal to generate "pictures" of georeferenced feature or coverage data. WMS will provide a means to portray geographic information independent of the underlying data model (WFS or WCS).

• Coverage Portrayal Service (CPS) defines a standard interface for producing visual pictures from coverage data typically accessed via WCS with a SLD portrayal.

Page 21: Grid Technologies in Disaster Management

GIS Grid Services II• Web Terrain Service (WTS) augments WMS with advanced

visualization including 3D terrains. • Catalog Service - Web Profile (CS-W) is a catalog service

that will be built on a general Grid metadata service• Sensor Collection Service (SCS) fetches observations from

a sensor or group of sensors and will be integrated with research on Grid sensor services

• Sensor Planning Service (SPS) assists in 'collection feasibility plans' and to process collection requests for a sensor or group of sensors.

• Web Notification Service (WNS) will be replaced by standard Grid notification service

Page 22: Grid Technologies in Disaster Management

Grid Portals Portals need to provide services for security,

customization, layout, rendering Jetspeed and GridSphere are two well known portals

• Commercial systems like IBM WebSphere similar These Portals are very important as they encourage

“component” model for user interfaces and so this fits service model so every service can be packaged with its (document fragment) user interface

Portlets are good as they support they mixing and matching of services for a particular composed/overlay Grid to allow this to be easily reflected in user interface• New JSR168 portlet Java Standard

NSF Middleware Initiative NMI funding collection of open source Grid portlets

Page 23: Grid Technologies in Disaster Management

OGCEOGCEConsortium

The OGCE Computing Grid Portal

• Provides Portlets for– Management of user proxy

certificates– Remote file Management via

Grid FTP– News/Message systems

• for collaborations– Grid Event/Logging service– Access to OGSA services – Access to directory services– Specialized Application Factory

access• Distributed Workflow

– Access to Metadata Index tools• User searchable index

– Real Time Collaboration• Audio/Video Conferencing

Download from open source http://www.ogce.org

Page 24: Grid Technologies in Disaster Management

OGCEOGCEConsortium

Example Capability: File Management

• Grid FTP portlet– Allow User to manage remote file spaces– Uses stored proxy for

authentication– Upload and download files– Third party file transfer

• Request that GridFTP server A send a file to GridFTP server B

• Does not involve traffic through portal server

Portal Server

User

GridFTPServer A GridFTP

Server B

GridFTPService

1 of many Portlets

Jetspeed

Page 25: Grid Technologies in Disaster Management

Watershed Entry Page: Select a County

FloodGridPortal

Page 26: Grid Technologies in Disaster Management

Watershed Boundaries in Black

Open source GIS MapServer presented as a simple portlet

Page 27: Grid Technologies in Disaster Management

Data Download Portal

This is where you can download ArcView compatible data files for your watershed. Some of this gets hooked to the runoff model in our connected crisis grid model.

Page 28: Grid Technologies in Disaster Management

Flood Grid

DataArchives

DataArchives

RunoffModel

RunoffModel

FlowModel

FlowModel

FlowModel

GIS Grid Services Link Distributed

Data and Applications

SOAP MessagesAnd Events

DataArchives

DataArchives

RunoffModel

RunoffModel

FlowModel

FlowModel

FlowModel

GIS Grid Services Link Distributed

Data and Applications

SOAP MessagesAnd Events

Parallel Simulations

Sensors

GIS Data Archives provide terrain, elevation, and land use data

Grid WorkflowLinks Data, Sensors, Model

Page 29: Grid Technologies in Disaster Management

SERVOGrid HPC ApplicationsSERVOGrid HPC Applications Codes range from simple “rough estimate” codes to parallel, Codes range from simple “rough estimate” codes to parallel,

high performance applications.high performance applications.• DislocDisloc: handles multiple arbitrarily dipping dislocations (faults) in an : handles multiple arbitrarily dipping dislocations (faults) in an

elastic half-space.elastic half-space.• SimplexSimplex: inverts surface geodetic displacements for fault : inverts surface geodetic displacements for fault

parameters using simulated annealing downhill residual parameters using simulated annealing downhill residual minimization. minimization.

• GeoFESTGeoFEST: Three-dimensional viscoelastic finite element model for : Three-dimensional viscoelastic finite element model for calculating nodal displacements and tractions. Allows for realistic calculating nodal displacements and tractions. Allows for realistic fault geometry and characteristics, material properties, and body fault geometry and characteristics, material properties, and body forces. forces.

• VirtualVirtual CaliforniaCalifornia: Program to simulate interactions between vertical : Program to simulate interactions between vertical strike-slip faults using an elastic layer over a viscoelastic half-space strike-slip faults using an elastic layer over a viscoelastic half-space

• RDAHMMRDAHMM: Time series analysis program based on Hidden Markov : Time series analysis program based on Hidden Markov Modeling. Produces feature vectors and probabilities for Modeling. Produces feature vectors and probabilities for transitioning from one class to another. transitioning from one class to another.

• PARKPARK: Boundary element program to calculate fault slip velocity : Boundary element program to calculate fault slip velocity history based on fault frictional properties; a model for unstable slip history based on fault frictional properties; a model for unstable slip on a single earthquake fault.on a single earthquake fault.

• PDPCPDPC: Phase Dynamics Probability Change: Phase Dynamics Probability Change Preprocessors, mesh generatorsPreprocessors, mesh generators Visualization tools: Visualization tools: RIVARIVA, , GMTGMT

Page 30: Grid Technologies in Disaster Management

QuakeSim Portal ShotsQuakeSim Portal ShotsSERVOGrid linking HPC, Data and Visualization

SAR Data

Page 31: Grid Technologies in Disaster Management

Run Finley – Simulation from Australia

Page 32: Grid Technologies in Disaster Management

Global-MMCS 2.0 XGSP MCU We are building an open source protocol independent Web

Service “MCU” which will scale to an arbitrary number of users and provide integrated thousands of simultaneous users collaboration services.

We will deploy it globally with first release end of May 2004. The function of A/V media server will be distributed using

NaradaBrokering architecture.• Media Servers mix and convert A/V streams

Open XGSP MCU based on the following open source projects• openh323 is basis of H323 Gateway

• NIST SIP stack is basis of SIP Gateway

• NaradaBrokering is open source messaging from Indiana

• Java Media Framework basis of Media Servers

Page 33: Grid Technologies in Disaster Management

XGSP Web Service MCU Architecture

SIP H323 Access Grid Native XGSPAdmire

Gateways convert to uniform XGSP Messaging

High Performance (RTP)and XML/SOAP and ..

Media ServersFilters

Session ServerXGSP-based Control

NaradaBrokeringAll Messaging

Use Multiple Media servers to scale to many codecs and manyversions of audio/video mixing

NB Scales asdistributed

WebServices

NaradaBrokering

Page 34: Grid Technologies in Disaster Management

A/V Collaboration Systems GlobalMMCS Grid federates existing A/V Protocols H323

H.323 is defined as an umbrella standard specifying the components to be used within an H.323-based environment.

SIPThe Session Initiation Protocol (SIP) defines how to establish, maintain and terminate Internet sessions including multimedia conferences

Access Grid enhanced Mbone A/V tools ( VIC, RAT ) Internet 2 network ( Multicast support )

Page 35: Grid Technologies in Disaster Management

Grid Services in GlobalMMCS I Audio Mixing

The audio mixer creates a mixed audio stream from all the audio streams in the session

Video Mixing Video mixing makes the unicast users watch the pictures of

multiple participants in a meeting through one video stream Video Thumbnail visualize the VS set in the session, embedded into the control panel

of each endpoint, which Image grabbers capture video streams and save them as static JPEG

files. All the media processing components can be distributed among

the pool of the media servers connected to NaradaBrokering infrastructures.

This generalizes to a HPC farm of “stream servers” doing image processing etc.• Perhaps need 100 node cluster to support 10,000 simultaneous A/V

streams

Page 36: Grid Technologies in Disaster Management

H.323, SIP Gateway Servers, A/V Session Server H.323 and SIP gateway transform their protocol

specific messages into XGSP signaling messages so that H.323 and SIP A/V endpoints could communicate with the XGSP A/V session server

The session server implements session management logics • creating/destroying A/V sessions

• allowing endpoints to join/leave session

• Allowing users to make audio/video selection, managing A/V application components

Page 37: Grid Technologies in Disaster Management

0

10

20

30

40

50

60

0 200 400 600 800 1000 1200 1400 1600 1800 2000

De

lay

(Mill

ise

con

ds)

Packet Number

Average delays per packet for 50 video-clients NaradaBrokering Avg=2.23 ms, JMF Avg=3.08 ms

NaradaBrokering-RTP JMF-RTP

Page 38: Grid Technologies in Disaster Management

Polycom, Access Grid and RealVideo views of multiple streams using

GlobalMMCSA/V Web Service

Page 39: Grid Technologies in Disaster Management

Unicast AG Portlet

Page 40: Grid Technologies in Disaster Management

WSDisplay

WSViewer

WS Display

WS Viewer

Master

WSDisplay

WS Viewer

NaradaBrokeringMessage Bus

Collaboration as a WSSet up Session with GlobalMMCS

defining participants

Application orContent source

WSDL

Web Service

F

I

U

O

F

I

R

O

Shared Output Port Collaboration

OtherParticipants

Text ChatWhiteboardGIS Services etc. areAUTOMATICALLYCollaborative ifBuilt as a Grid Services

Page 41: Grid Technologies in Disaster Management

Integration of PDA, Cell phone and Desktop Grid Access