Upload
kathryn-moore
View
220
Download
1
Tags:
Embed Size (px)
Citation preview
High Performance High Performance Cluster Computing Cluster Computing with PHOTOMOD HPC with PHOTOMOD HPC EditionEdition
Mikhail Drakin Head of Software Development Department
Racurs
September 2011, Tossa de Mar, Spain
11th International Scientific and Technical Conference From Imagery to Map: Digital Photogrammetric Technologies
In BriefIn Brief
Distributed processing of remote sensing data – the key to performance and efficiency
Parallelizing trends in modern hardware
Distributed processing in “conventional” PHOTOMOD:pros and cons
PHOTOMOD HPC Edition for computing clusters
PHOTOMOD Conveyor – the solution for on-line satellite monitoring
RSD – Ideal for Parallel ProcessingRSD – Ideal for Parallel Processing
Most operations on remote sensing data satisfy the conditionsfor effective parallelization – can be easily split into numerous tasks, which:
Are similar in natureand/or
Take source data bits independent from other tasksand/or
Produce data bits independent from output of other tasksand/or
Are performed automatically
RSD – Ideal for Parallel ProcessingRSD – Ideal for Parallel Processing
Common operations, implemented in automatic distributed mode in PHOTOMOD:
Tie points measurement DTM calculation (pickets and “dense” DEM) Orthorectification Mosaics
including sequence of distributed task sets:
• Cutlines creation
• Tie points searching
• Final mosaic building
+ a bunch of service operations (image format conversion, pan-sharpening, DEM reprojection, etc).
Parallel processing helps saving precious time,reducing days of work to hours
Parallel Hardware TrendsParallel Hardware Trends
The same trend at all scales: performance gainedby increasing number of identical computing units.This allows building applications scalable over a wide range
Increasing number of cores in off-the-shelf CPUs
Larger local networks
Supercomputers as clusters of tens to thousands similar nodes
““Conventional” PHOTOMODConventional” PHOTOMOD
Distributed processing first implemented in version 4.4.
Each new version supports several new operations in distributed mode.
Suited for medium-scale parallelization:
Distribution of tasks within single workstation to effectively utilize allprocessor cores
Distribution of tasks between several workstations to employ LAN resources of a department
Ideology: “one server – several workstations”
““Conventional” PHOTOMODConventional” PHOTOMOD
Drawbacks of distributed processing in standard PHOTOMOD:
Targeted at interactive work (flexible, but multiple GUI options complicate automation)
Not optimized for centralizedadministering
Implies storing configurationdata, logs, etc. locally on the workstation
Standard PHOTOMOD does not rely on consistent LAN structure and fixed hardware configuration, thus sacrificing some performance for flexibility.
Hardware SideHardware Side
Massive photogrammetric processing involves:
Sophisticatedcomputingalgorithms
Intensivedata
exchange
Hugestored data
volume
CPU
x K units?
Net
x M Gbps? x N TB?
Storage
Hardware configuration must be carefully planned and balanced to avoid bottlenecks
PHOTOMOD HPC Edition: Key FeaturesPHOTOMOD HPC Edition: Key Features
PHOTOMOD HPC Edition – next level of distributed processing in PHOTOMOD
Ideology: “one workstation – multiple servers”
Oriented at coherent high-end hardware and software complex
Centralized administering, “lessmessage boxes”
No modifications on computing nodes – all installation, configuration, logs storage, etc organized on a dedicated server
Customization for particular customer needs and computing environment is implied.
ApplicationsApplications
Fully automatic or semi-automatic solutions based on PHOTOMOD HPC Edition can pay off in processing of any kind of digital imagery,including:
Conventional aerial survey and rapidly emerging UAV imaging projects, with constantly increasing productivity and positioning quality, which allows fully automatic processing, especially with properly marked ground control points.
Rapid creation of quality orthomaps from HR and VHR satellite images for online monitoring (emergency management in situation centers, reconnaissance, etc).
CounterpartsCounterparts
Examples of counterpart products:
VisionMap A3 Ground Processing System
PCI GeoImaging Accelerator (GXL)
Spot Image Pixel Factory
Photogrammetric systems with distributed processing (LPS, Inpho, etc)
“Counterparts” here is not exactly equal to “Competitors”: each high-end product has specific niche.
PHOTOMOD HPC Edition: DistinctionsPHOTOMOD HPC Edition: Distinctions
Distributed processing subsystem is native for the platform andconstantly improved for specific applied needs
The system can be easily adopted to any hardware meetingbasic requirements, not tied to some specific set of devices. The simplest cluster, which can bring significant productivity benefits with PHOTOMOD HPC, is probably 4 computing nodes (Windows HPC Server) with carefully designed storage system (e.g. FreeBSD-based), starting from $30-40 thousand
A computing cluster with HPC Edition can be tightly integrated with “conventional” interactive PHOTOMOD workstations, taking all advantages and eliminating drawbacks mentioned earlier
The system is initially positioned as a platform for building customized solutions
Testing in KazakhstanTesting in Kazakhstan
Image courtesy of SmartCom
December 2010:PHOTOMOD HPC was successfully tested on KazNTU computing cluster
• 10.9 TFLOPS
• 128 computing nodes
• NetApp storage system
• Windows HPC Server OS
The experiment:Orthomosaic creation• Output GSD 0.16 m • Area 1289 sq. km• DEM cell size 1.6 m • Breaklines 858• Total output files 156 GB
Processing time: 3 hours
PHOTOMOD ConveyorPHOTOMOD Conveyor
Example of HPC Edition customization for specific application
Collaborative project by Racurs and CTT Group (Russia) for heavy-duty mosaic production from VHR satellite data: hardware + software complex solution
+
PHOTOMOD ConveyorPHOTOMOD Conveyor
Fully automatic workflow, including:
• Creation of the processing project from source satellite products
• Adjustment of the project (by RPC)
• Orthorectification by existing DEM
• Exclusion of images with undue cloud coverage
• Building cutlines
• Searching for tiepoints to eliminate residual discrepancies along seams
• Radiometric balancing
• Final stitching into seamless mosaic, saved into standard sheets set
1 000 000 sq. km per day of 1:25 000 mosaic from GeoEye, DG or Alos
FutureFuture
Distributed GPU computing in cluster environment
Extensive set of automatic functions for data processing and enhancement
Library of fully automatic workflows for UAV processing, DTM creation and filtering, etc.
Advanced control over scheduling for convenient sharing of a single cluster between several workgroups
Integration with GIS systems, databases and geoportals
Any modifications and customizations requested by customers
Thank you for attentionThank you for attention!!