Agenda• Speakers Biography

• Databases & GPUs

• Problem Definition

• SQL Data Types

• SQL Stored Procedures

• System Overview

• Solution Architecture

• Image Processing

• File Formats

• Compression/Decompression

• GPU Image Support

• Image Processing Techniques

• Conclusions

• QA

Speakers Bio

• CEO 3DMashUpStart-Up focused on GPU Accelerated DBMS

4+ years GPU and HPC ProgrammingDeveloped large databases• Navteq Street Maps• NASA EOSDIS Climate Research DBMS• LINZ ( Land Info New Zealand)

Formerly• VP Engineering Oracle• VP Engineering BEA Systems• VP Engineering Informix

• What are we Solving For?

Database & GPU’sInternal Operations

– Query Planning/ Optimizer

• Fastest Route

– Relational Operations

• Select

• Project

• Join

• …

– Ordering

• Sorting

• Group By

GPU Accelerated Applications

– Image Processing

• Medical

• Remote Sensing

• Climate Research

– In-Database Analytics

• Predictive Analytics

• Risk Management

– 3D Content Management

• GIS

• CAD

• Additive Printing

• Artificial Reality

SQL Data Types

• Rows

– Set of Columns

• Tables

– Set of Rows

• Columns Types

– Standard types

• Numbers, Text, Boolean, URL, XML, Date Time ….

– BLOB

• Binary File Stream ( Read/Write, Seek operations )

GPU Types (22)

Vectors

Uchar N, Integer16 N, Integer 32 N, Double N, Float N, …

Images

TimeSeries, Image2D, JPEG

Metadata

N D Range, Image Sampler, LocalStorage

SQL Type ExamplesCreate Table Images

(

Id serial not null,

Name text not null,

Path uri not null,

Image OpenCL.jpeg not null,

Thumbnail OpenCL.jpeg,

WorkImage OpenCL.Image2D,

Histogram TimeSeries,

…

);

Database Image Storage Options• SMART-BLOB, BLOBS

– Opaque Binary File Stream

– Unlogged/Logged

• Tables

– Byte Areas

– Logged

– Unlogged Tables

• External Files

– Store File Path

– SQL Server Filestream

Image Storage Comparison

ACID Size IntegratedBack-Up

Queryable Encryption GPUQuery

Replication

SMART BLOB

Yes SoD Yes Yes Manual No ??

BLOB No SoD No Yes Manual No No

LoggedTable

Yes ~1G Yes Yes Optional Yes Yes

UnloggedTable

No ~1G Yes Yes Optional Yes No

External File

No SoD No No Manual No No

FileStream Yes SoD ?? Yes No No No

SQL UDFs (User Defined Functions)• Functions called by SQL Queries or Triggers

• Many SQL UDF Language Bindings

– TSQL, PLSQL, SQL, C/C++, Perl, Ruby, R, Java, ..

• Implements natural Map-Reduce paradigm

• Can implement Internal DB operations

– Views, Sorts, Joins, …

• GPU Language Bindings

– OpenCL, CUDA

SQL UDF Examples-- An OpenCL UDF

Create Function Histogram256( Input image image2d,

output histogram256 Timeseries )

as $body$

Kernel void Histogram256( read_only image2d_t * image,

write_only Timeseries * histogram )

{

…

}

$body$ language pgopencl;

-- A CUDA UDF

Create Function CudaHistogram256 ( input image integer[],

output histogram integer[] )

As ‘PTX file name’ language pgcuda;

SQL GPU Query-- Insert New Data

Insert into images ( image, name, path) values ( … );

-- Query Creates Histogram and a Thumbnail

Update Images

set

histogram = histogram256( imaged::image2d,

histogram )::Timeseries,

thumbnail = scale( image::image2d, 64, 64) :: jpeg

where id = … ;

-- Query Images for display

Select * from images where id = … ;

SQL GPU Execution Model

Image Processing Goals

Secure Multi-User Shared Managed Scalable Integrated Fast

Image Processing Operations• Geometric Operations

– Scale, Rotate, Transform

• Point Operations

– Contrast, Brightness, ..

• Filters

– Linear, Non-Linear

• Histograms

• Edge & Contours

– Edge Detection, Curve Detection

• Region Processing

• Color/Spectra Processing

• FFT/DCT

System Architecture Overview

File Formats

• Complex• Multi-Layered• Metadata• Opaque

OpenCL GPU Image Support• Pixels

– 1D, 2D, 3D Geometries

• Channels

– Set of sample values (1,2,3,4 per Pixel)

– Data Types

• Byte, Int16, Int32, Float, Half Float, Short_555, …

• Channel Order

– RGBA, ARGB, Intensity, Luminance,…

• 80+ Combinations

– Queryable from the Device Driver

Impedance Mismatch • Impedance Mismatch

– GPU Raw Format

– Files Compressed (mostly)

– JPEG 24 bit (3x8) RGB vs 32 bit (4 x 8) Vector

• Conversion Required

– Extract Relevant Sections

– Decompress/Compress

• DCT, LZW, Huffman, ….

– Color Model Transformation

• RGB, CMYK, YUV, …

– Align to GPU Memory Formats

Image Processing MemoryOptimization Techniques

• Vectorization

– Convert 24 bit RGB Pixel to UChar 4 Vectors

• Image Striding

– Alignment of Pixels Rows to GPU Memory Banks

• Z- Ordering

– Spatial location of Pixels

– Alignment of Pixels to GPU Memory Banks

Vectorization

Host Converts 24 Bit JPEG RGB to 32 Bit Unsigned Char_4 Vector

Wastes 25% RAM for ~3X Speed-Up

Filtering

3x3, 5x5, … Pixels processed per Output Pixel

Un-Coalesced(misaligned)

GPU Memory Access Patterns

Coalesced

Bank Conflict(same Bank)

Image Striding

PaddingWords

Stride Length is an Exact Multiple of 16 Words

Pixel Access Patterns

Z-Order Access• Good Access Pattern

• No Bank Conflicts• Coalesced Access

Linear Access• Poor Access Pattern

• Bank Conflicts• Misalignment

Morton Z Ordering

Access Order(0,0)(1,0)(1,1)(1,1)(2,0)(3,0)(2,1)(3,1)….

Optimizing Memory Access on GPU Using Morton Order Indexing A.E. Nocentio, P.J. Rhodes, ACM SE ‘10

Integer Dilation

Z Order Coordinates• Dilation & Shifting

Dilation• Shifting• Look-Up• Magic Arithmetic

Integer Dilation and Contraction for Quadtrees and Octrees,L Stocco, G Schrack, IEEE 1995

Conclusions• GPU Accelerated Databases

– Bridges Impedance Mismatch

• Persistence of Image File Format

• GPU Memory Access patterns

• Minimal Data Movement

– Supports Image Processing Capabilities

• CUDA and OpenCL Kernel functions

• Support Multiple Image Formats

• Content Based Queries

– GPUs Offload Image Processing Computations

– Support Complex Queries

• Metadata Queries

• Full Text Search

• Geo-Located Queries

• High-Level Workflow Processing

• Combinations of the Above

– Supports Web/Mobile Applications

• Secure Image Content retrieval

Future Directions

• More Image Types

– TIFF (GEOTIFF )

– DICOM

• GPU Side Compress/Decompress

– JPEG

• DCT, Quantization, Huffman

– TIFF

• LZW, RLE, JPEG

Computer Vision Functions ( OpenCV )

Face Detection

Face Recognition

Feature Extraction

Query by Content