Managing and Extracting Gridded Data ─ can database extensions compete with direct use of flat files for extractions? Barrodale Computing Services Ltd. (BCS)
1. can database extensions compete with direct use offlat files
for extractions?Barrodale Computing Services Ltd. (BCS) 2. Dealing
with Data The traditional approach Store metadata in a DBMS Store
complex data in filesDisadvantages Possible that metadata or data
will become separated fromeach other Data mining opportunities
essentially lost Integrity checking opportunities lost2 3. Dealing
with Data The ORDBMS approach ORDBMSs provide support for large
complex objects withoperations that can be performed on them Hence
we can store both the data and its metadata in thesame
databaseAdvantages No separation of data from metadata Data mining
opportunities are possible Integrity checking opportunities 3 4.
BCS Speeds Database Use:DBXten Efficiently and seamlessly store
multidimensional data Conveniently and quickly extract subsets of
the data alongone or more dimensionsGrid DataBlade Store and
manipulate large multidimensional griddeddatasets Extract,
subsample, and reproject data along parallel oroblique axes 4 5. A
Performance Comparison Used 4D OCCAM Model data Geographic extent:
25N to 35S, 20E to 140E 81 NetCDF files Used the Grid Extension for
PostgreSQL, DBXten forPostgreSQL, and traditional NetCDF file
access Several types of extraction were performed Times required to
load and extract were observed andrecorded 5 6. Overall timing
resultsNature ofExtractionExtraction Size DBXtenGrid Extension
NetCDF filesTime profile81x1x1x1 1.03 sec 2.24 sec 1.30 secDepth
profile 1x66x1x1 0.07 sec 0.17 sec 0.72 secHorizontal1x1x50x500.06
sec 0.09 sec 0.03 secplanar slice1x1x100x1000.09 sec 0.13 sec 0.03
sec1x1x200x2000.17 sec 0.23 sec 0.04 secVertical1x33x25x10.07 sec
0.16 sec 0.37 secplanar slice1x33x50x10.09 sec 0.20 sec 0.38
sec1x66x100x1 0.14 sec 0.38 sec 0.80 sec1x66x200x1 0.20 sec 0.59
sec 0.68 sec4D Volume 10x5x20x40 0.19 sec 0.46 sec 0.69
sec10x10x40x800.34 sec 0.99 sec 1.24 sec10x20x80x160 1.12 sec 3.75
sec 2.49 secAverage Time 0.30 sec 0.78 sec 0.73 sec6 7. For more
information Website: http://www.barrodale.com Contact:
[email protected] or (250) 412-7428 More:
http://www.barrodale.com/GriddedDatainDepth.pdf7
