Upload
lyle
View
56
Download
3
Embed Size (px)
DESCRIPTION
High-Performance Computing for Processing Earth Observation Data By Dr Ashok Kaushal Senior Divisional Director Enterprise Geospatial & Defense Solutions Rolta India Limited [email protected]. Innovative Technologies for Insightful Impact. Agenda. Trends Needs Process Automation - PowerPoint PPT Presentation
Citation preview
High-Performance Computing High-Performance Computing forfor
Processing Earth Observation Processing Earth Observation DataData
ByBy
Dr Ashok KaushalDr Ashok KaushalSenior Divisional DirectorSenior Divisional Director
Enterprise Geospatial & Defense SolutionsEnterprise Geospatial & Defense SolutionsRolta India LimitedRolta India Limited
[email protected] Innovative Technologies for Insightful ImpactInnovative Technologies for Insightful Impact
Agenda • Trends
• Needs
• Process Automation
• GeoImaging Accelerators/
GXL
• Job Processing Systems/ JPS
• Conclusions
Trends • 230 EO [versus 107 in last decade] satellites
projected over next decade for use of satellite imagery– Emerging markets expected to account for 75 satellites
- four-fold increase over last decade
– 41 Nations [currently 26] to have own satellites
• Commercial sale of EO data expected to double– Commercial EO data from satellites expect CAGR of
15% over next 10 years, reaching $4 billion by 2019
– Optical data will represent 79% of overall sales
– Number of high resolution satellites offering commercial data are expected to double from currently 24
‘Satellites to be Built & Launched by 2019, World Market Survey’, Euroconsult
Trends• Exponential increase of volumes of satellite EO
data • Increasing value of EO data with applications in
– Agriculture, Environment, Urban Development, Disaster Management, Surveillance and others
• Increasing value of up-to-date info – RapidEye, GeoEye, Digital Globe, IRS/ Cartosat
• Significant growth of awareness in EO data – Google Earth, Microsoft Bing Maps, Bhuvan
• Increasing importance of collaboration and sharing of current data/information for Situational Awareness
Needs • Satellite Programming
• Timely Data Acquisition
• Process Automation
• Data Pre-Processing
• Data Management
• Data Dissemination
• Information Sharing
• Geo Collaboration
For Production
Move from this
To This
Process Automation
Incoming raw
imageExtract raw Image
to native format
Collect GCP Using Master Image
Refine collected GCP
Compute MathModel
OrthorectifyRaw Image
Load Image toOracle Database
OracleDatabase
Process Automation
Geoimaging Accelerators are
automated workflows
created from linking together
of any number of pluggable
image processing functions
Geoimaging Accelerator (GXL)
Data Ingest Process 1 Process 2 Process 3 DeliveryData Ingest Process 1 Process 2 Process 3 DeliveryData Ingest Process 1 Process 2 Process 3Data Ingest Process 1 Process 2 Process 3 Delivery
Objectives:•Need for large volume image data processing
• to reduce image pre-processing bottlenecks•Demand for greater automation & less user
interaction• to save money on operator time
•Workflows that can scale across multiple processors
• to add capacity as and when needed•Plug & Play architecture
• to add new components or functions to expand•Cost Effective Solution to Remain Competitive
• to run 24/7 with zero or little operator intervention
Geoimaging Accelerator (GXL)
• Distributed processing• Two levels
• Basic• Automated CPU
• Accelerated • Multi-core CPU• Optimized GPU
• Ortho / Ortho XL• Satellite & Airphoto
• PanSharp / PanSharp XL• Mosaic/ Mosaic XL
Geoimaging Accelerator (GXL)
IngestIngest
Job Processing System (JPS)
Job Processing System (JPS)
GXLGXL OutputOutput
Orthorectification GXLOrthorectification GXL Ortho Product
Ortho Product
WorldView-1 Level 1b e.g.
WorldView-1 Level 1b e.g.
RPC Model Calculation
RPC Model Calculation OrthorectificationOrthorectification
DEMDEM
Ortho Product
Ortho Product
UltraCam X Imagery e.g.
UltraCam X Imagery e.g. Orthorectification GXLOrthorectification GXL
AP Model Calculation
AP Model Calculation OrthorectificationOrthorectification
DEMDEM
Ortho / Ortho XL
Airphoto / Airphoto XL
Geoimaging Accelerator (GXL)
Mosaic GXLMosaic GXL
DEM Extraction GXLDEM Extraction GXL
OrthophotosOrthophotos Mosaic ProductMosaic Product
Stereo PairStereo Pair Raster DEMRaster DEM
Colour-Balance
Colour-Balance
Cutline Selection
Cutline Selection MosaicMosaic
Epipolar Rotation
Epipolar Rotation
DEM Extraction
DEM Extraction GeocodingGeocoding
PanSharp GXLPanSharp GXLPan and MS ImageryPan and MS Imagery PanSharp ProductPanSharp Product
Pan SharpeningPan Sharpening
Geoimaging Accelerator (GXL)
Accelerated GXL?• A hardware-based, GPU enabled, high-
performance image processing system • Design to process large volumes
– 40 times faster than desktop product – 2-4 TB per day for desk-side system– 10 TB + for rack mounted system
Orthorectify & Mosaic India in a Day!
Geoimaging Accelerator (GXL)
C++ SDKGPU / HW
C++ SDKGPU / HW
Interface LayerInterface Layer
Processing LayerProcessing Layer
Architecture LayerArchitecture LayerFormats:BIL, TIFF,
etc.
Formats:BIL, TIFF,
etc.
Algorithms: Pansharp, Ortho, etc.
Algorithms: Pansharp, Ortho, etc.
PPFsPPFs
Workflows: GXL
Workflows: GXL
Bindings:Python,
Java
Bindings:Python,
Java
Layer:Layer: Component:Component: Integration:Integration:
Job Processing System Data / Imagery Level
Data / Imagery Level
Operations / Systems Level
Operations / Systems Level
HW / Architecture Level
HW / Architecture Level
Arc
hit
ectu
re
Geoimaging Accelerator (GXL)
• Flexible orthorectification:– Support for several sensors (SPOT, QB, Ikonos, WV, …) – Optional radiometric calibration of SPOT images– Optional GCP collection from multiple reference data types
• Flexible mosaicking:– Mosaics from mixed-resolution raw scenes– Optional tie point collection and refinement– Various types of color balancing– Various tiling schemes
• High quality:– Sub-pixel accuracy of GCPs and orthoimages– Nicely color-balanced mosaics
Hig
hlig
hts
Geoimaging Accelerator (GXL)
Product Type
Dataset Resolution [m]
Volume [TB/Day]
Area [km2/day]
SPOT5
- Level 1A 2.5 meter 8U Pan 2.5 2.0013.7
Million e.g. Europe:
10.1M km2
IKONOS
- Geo Ortho Kit16U Pan Ikonos 1.0 2.94
1.62 Million
e.g. Mongolia:1.56M km2
WorldView-1 and
QuickBird Level 1B 16U Pan 0.5 3.26448k
e.g. Sweden:450k km2
QuickBird- OrthoReady- 4 channel PS
16U Multispectral 0.6 3.52174k
e.g. Florida:170k km2
QuickBird
- Level 1B16U Multispectral 2.4 4.57
3.62 Millione.g. India:3.17M km2
Pro
cessin
g M
etr
ics
Geoimaging Accelerator (GXL)
Product Type
Dataset MB/Sec GB/Min TB/Day
SPOT5
- Level 1A 2.5 meter 8U Pan 24.23 1.42 2.00
IKONOS
- Geo Ortho Kit 16U Pan Ikonos 35.67 2.09 2.94
WorldView-1 and
QuickBird Level 1B 16U Pan 39.59 2.32 3.26
QuickBird- OrthoReady- 4 channel PS
16U Multispectral 42.67 2.50 3.52
QuickBird
- Level 1B 16U Multispectral 55.47 3.25 4.57
Pro
cessin
g
Th
rou
gh
pu
tGeoimaging Accelerator (GXL)
Performance /Day GB 1 - 5TB 5 - 10TB
Batch Processing
20 Orthos per day10GB Project Scale
GXL Deskside Accelerated
2000 Orthos per day1TB Project Scale
GXL RackAccelerated
5000 Orthos per dayPlus
100 Image Mosaic per day5TB Project Scale100 Orthos per day
50GB Project Scale
10
200
500
Cost $1,000
GXL Basic
Geoimaging Accelerator (GXL)
• Environmental• Carbon sequestration• Biomass estimation
• Agricultural• Crop yield• Crop forecasting
• Aerospace & Defense• Border monitoring• Disaster management
• Data Supply• Product delivery• Archive re-processing
GeoImaging Accelerator
Ap
plicati
on
s
Job Processing System• Distributed Processing System
– Run multiple jobs concurrently on multiple servers
Job
Job
Job
Job
Job
Job Job
JPS Processing Server
JPSDatabase
ComputerComputer Computer Computer
JPS Processing Server
JPS Processing Server
JPS Processing Server
• Job:
– An entry in the JPS-DB
– A Process started and monitored by a Processing Server
• Processing Server
– Daemon managing jobs
JPS-DB
Processing ServerJob
Job
Job Processing System
22
• Distributed Cloud Computing (Autonomous Nodes)
• Automatic Load Balancing• Simple Web Interface• Threefold Value:
1. Automation = Increased Throughput (Revenue)2. Job Tracking = Improved QA (Operational
Costs)3. Multi-Platform, Multi-Language =
SustainabilityJPS-DB
GXL1
GXL2 Job
Job
Job
Other Nodes
Job
Job
Job
Job Processing System
Job Processing System
• Effective use of voluminous satellite imagery from numerous high-resolution satellites desires automated pre-processing using HPC
• Distributed processing using multi-core CPU and GPU with CUDA and Open MP provides an ideal platform for faster turn-around-time during pre-processing of geoimaging
Conclusions
Thank you !Thank you !