Upload
elgin
View
37
Download
0
Tags:
Embed Size (px)
DESCRIPTION
The MAGIC Project Final Demonstration and Briefing. Ira Richer 29 July 1999. MAGIC Scenario. Datasets. Globe San Francisco Bay area Menlo Park Ft. Irwin Camp Pendleton (low resolution) Camp Pendleton (high resolution) Laguna Beach Yosemite Colorado Lawrence , KS Ohio - PowerPoint PPT Presentation
Citation preview
Corporation For NationalResearch Initiatives
The MAGIC ProjectFinal Demonstration and Briefing
Ira Richer
29 July 1999
DARPADARPA
Corporation For NationalResearch Initiatives
MAGIC Scenario
1 m aerial terrain10/95
10 m satellite terrain“yesterday”
1 km satellite weather“today”
10 m aerial troop movement“now”
10 m satellite terrain1/96
Corporation For NationalResearch Initiatives
Datasets
Globe– San Francisco Bay area
Menlo Park
– Ft. Irwin– Camp Pendleton (low resolution)
Camp Pendleton (high resolution)– Laguna Beach
– Yosemite– Colorado– Lawrence, KS– Ohio– Washington, DC– Ft. Benning
Corporation For NationalResearch Initiatives
Background
The MAGIC scenario is based on applications such as mission planning and crisis management
These applications need to:– locate, process, and display real-time and/or archived datasets
on-the-fly– fuse multiple types of data (e.g., weather, terrain)– be able to transfer large volumes of data for interactive
visualization– support a range of user capabilities (e.g., command center,
mobile user)
MAGIC focuses on a terrain visualization application that uses geo-referenced datasets
Corporation For NationalResearch Initiatives
Objectives
To develop an information system that provides application users with fast, interactive access to very large volumes of remote data distributed over a wide geographic area
To demonstrate the above capability in the context of a 3-D terrain visualization application
Corporation For NationalResearch Initiatives
MAGIC Participants
Corp. for National Research Initiatives USGS/EROS Data Center Lawrence Berkeley National Laboratory Sprint SRI International University of Kansas
Corporation For NationalResearch Initiatives
Distinguishing Characteristics
Distributed information system, designed from the outset to work over a network– adaptation to network conditions and to system load
No limit on size of viewable datasets No restrictions on location of datasets Fusion of multiple datasets Near real-time visualization of terrain data
Corporation For NationalResearch Initiatives
MAGIC Components
High-speed internetwork Distributed parallel storage system (DPSS) Software for terrain dataset preparation 3-D interactive terrain visualization
application: TerraVision
Corporation For NationalResearch Initiatives
DPSS
DPSS
DPSS
DPSS
Generic Configuration
High-speedinternetwork
Repository
Processor
Processor
User
User
Repository
Corporation For NationalResearch Initiatives
7172
7374
70 4331
8788
4445
4632
3334
3536
86
2524
System Operation
D1 D2
733335
45
SERVER 2Server 1
D1 D2
7143 34
SERVER 1
Internetwork
User
7143 33 34 35 73 45
Tiles located onDPSS server/disk
S1D1
S1D2
S1D1
S2D1
S2D2
S2D2
S2D1
Tiles onpath of travel
Tiles retrievedand transmitted
Corporation For NationalResearch Initiatives
Today’s Demonstration and Briefing
3-D, interactive visualization of large volumes of remote data
Platform scalability On-demand data preparation and 3-D
flyover Fusion of multiple types of data from
multiple sources DPSS fault tolerance
Corporation For NationalResearch Initiatives
Today’s Configuration
DARPA ISI-E
LBNL
ESnet GPN
NREN
ATDnet
Abilene
EDC
NASAGoddard
KU
DPSS
DPSS
DPSSDPSS
Corporation For NationalResearch Initiatives
Dataset Characteristics
Resolution (m) Size (MB)
Globe ~1000 4,400San Francisco Bay area 25 670Menlo Park 1 680Ft. Irwin 1 330Camp Pendleton 30 44Camp Pendleton 1 12,300Laguna Beach 1 140Yosemite 1 2,100Colorado 30 200Lawrence, KS 1 680Ohio 30 1,000Washington, DC 1 8,600Ft. Benning 0 .15 380
Corporation For NationalResearch Initiatives
Data Flow (1)
DARPA ISI-E
LBNL
GPN
NREN
ATDnet
DPSS
DPSS
Abilene
EDC
DPSS
NASAGoddard
KUDPSS
ESnet
Corporation For NationalResearch Initiatives
Dataset Characteristics
Resolution (m) Size (MB)
Globe ~1000 4,400San Francisco Bay area 25 670Menlo Park 1 680Ft. Irwin 1 330Camp Pendleton 30 44Camp Pendleton 1 12,300Laguna Beach 1 140Yosemite 1 2,100Colorado 30 200Lawrence, KS 1 680Ohio 30 1,000Washington, DC 1 8,600Ft. Benning 0 .15 380
Corporation For NationalResearch Initiatives
Platform Scalability
SGI Octane (~$50k)– OC-3, IRIX, DPSS– 60 frame/sec, ~80Mb/s
Dual-processor PC with graphics accel. (~$8k)– Fast Ethernet, Windows NT, HTTP– 30 frame/sec, ~0.5Mb/s (Internet-limited)
Dual-processor PC (~$4k)– OC-3, Linux, DPSS– 10 frame/sec, ~10Mb/s
Laptop (~$3k)– Fast Ethernet, Windows 98, HTTP– ~4 frame/sec, ~0.5Mb/s (Internet-limited)
Corporation For NationalResearch Initiatives
TerraVision II
Composite datasets: data from multiple sources, at different resolutions
VRML overlays: wind vectors, buildings, roads; viewable with standard browser– Wind-vector dataset hosted at NAVO
Co-registered maps for navigation Pre-defined flight paths for operations
support, testing
Corporation For NationalResearch Initiatives
C lient A pp lica tion
P ara lle lD isks
D P S S S erve r
P ara lle lD isks
D P S S S erve r
P ara lle lD isks
D P S S S erve r
D PSS M aster
da ta b locks
da ta b locks
da ta b locksLog ica l B lock
R eques ts
log ica l to phys ica lb lock lookup
access con tro l load ba lanc ing
P hys ica l B lockR eques ts
DPSS Cache Architecture
Corporation For NationalResearch Initiatives
DPSS Costs (Jan. ‘99)
Throughput Capacity Cost(Mb/s) (GB) servers disks ($k)
80 33 1 2 6
400 165 5 10 32
400 1000 5 64 80
800 1000 10 64 100
Configuration
Corporation For NationalResearch Initiatives
Monitoring and measurement
Real-time traffic monitor DPSS system monitor NetLogger
– Logs events at critical points of distributed applications– Provides complete view of system operation– Visualization tools allow for real-time monitoring
Agent-based management of measurement tools
Corporation For NationalResearch Initiatives
NetLogger Instrumentation
ClientApplication
Shared Memory Cache
BlockRequestThread
DiskRead
Thread
DiskRead
Thread
DiskRead
Thread
DiskRead
Thread
DPSS Master
from o ther D P S Sservers
*
= monitoring point
DPSS Data Server BlockW riterThread
to o ther c lien ts
Disk Disk DiskDisk
*
**
*
**
*
*
*
**
Corporation For NationalResearch Initiatives
DPSS Capabilities
Load balancing– Dataset must be replicated – Tile requests sent to server that will provide the best
performance, based on current server load and network conditions
Automatic TCP tuning– Window size determined by throughput and delay
information
Fault tolerance– Automatic detection of failed server, switchover to
alternate servers, and reconnection after recovery
Corporation For NationalResearch Initiatives
Data Flow (2)
DARPA ISI-E
LBNL
ESnet GPN
NREN
ATDnet
DPSS
DPSS
Abilene
EDC
DPSS
NASAGoddard
KUDPSS
Corporation For NationalResearch Initiatives
Technology Transfer and “Spinoffs”
DPSS– Earth Systems Grid (climate modeling)
– Kaiser Permanente project
– Combustion Corridor (diesel modeling)
– Matisse (MEMS/Supernet)
NetLogger– Being used by many network and grid projects
Corporation For NationalResearch Initiatives
Technology Transfer and “Spinoffs” (concluded)
USGS archive – Web-based access to digital earth-science data
– Gateway to the Earth: Ohio Pilot (automatic dissemination of Landsat imagery)
TerraVision– BADD (Dissemination of VRML terrain datasets to warfighter)
– Digital Earth (fast indexing of geo-data, multimodal capabilities, and education applications)
– Netscape plug-in