Scalable Data Clustering with GPUs

Andrew D. Pangborn

Thesis DefenseRochester Institute of Technology

Computer Engineering DepartmentFriday, May 14th 2010

Intro

• Overview of the application domain• Trends in computing architecture• GPU Architecture, CUDA• Parallel Implementation

Data Clustering

• A form of unsupervised learning that groups similar objects into relatively homogeneous sets called clusters

• How do we define similarity between objects?– Depends on the application domain, implementation

• Not to be confused with data classification, which assigns objects to predefined classes

Data Clustering Algorithms

Clustering Taxonomy from “Data Clustering: A Review”, by Jain et al. [1]

Example: Iris Flower Data

Flow Cytometry

• Technology used by biologists and immunologists to study the physical and chemical characteristics of cells

• Example: Measure T lymphocyte counts to monitor HIV infection [2]

Flow Cytometry

• Cells in a fluid pass through a laser

• Measure physical characteristics with scatter data

• Add fluorescently labeled antibodies to measure other aspects of the cells

Flow Cytometer

Flow Cytometry Data Sets

• Multiple measurements (dimensions) for each event– Upwards of 6 scatter dimensions and 18 colors per

experiment• On the order of 105 – 106 events• ~24 million values that must be clustered• Lots of potential clusters• Clustering can take many hours on a CPU

Parallel Computing

• Fortunately many data clustering algorithms lend themselves naturally to parallel processing

• Typically with clusters of commodity CPUs• Common APIs:– MPI: Message Passing Interface– OpenMP: Open Multi-processing

Multi-core

• Current trends: – Adding more cores– Application specific

extensions• SSE3/AVX, VT-x, AES-NI

– Point-to-Point interconnects, higher memory bandwidths

GPU Architecture Trends

• Throughput Performance

•Pr

ogra

mm

abili

ty

• CPU

•G

PU• Figure based on Intel Larabee Presentation at SuperComputing 2009

• Fixed Function

• Fully Programmable

• Partially Programmable

• Multi-threaded • Multi-core • Many-core• Intel LarabeeNVIDIA CUDA

Tesla GPU Architecture

Tesla Cores

GPGPU

• General Purpose computing on Graphics Processing Units

• Past– Programmable shader languages: Cg, GLSL, HLSL– Use textures to store data

• Present:– Multiple frameworks using traditional general

purpose systems and high-level languages

CUDA: Software Stack

CUDA: Streaming Multiprocessors

CUDA: Thread Model• Kernel

– A device function invoked by the host computer

– Launches a grid with multiple blocks, and multiple threads per block

• Blocks– Independent tasks comprised of

multiple threads– no synchronization between blocks

• SIMT: Single-Instruction Multiple-Thread– Multiple threads executing time

instruction on different data (SIMD), can diverge if neccesary

CUDA: Memory Model

CUDA: Program FlowApplication Start

Search for CUDA Devices

Load data on host

Allocate device memory

Copy data to device

Launch device kernels to process data

Copy results from device to host memory

CPUMain Memory

Device MemoryGPU Cores

PCI-Express

When is CUDA worthwhile?

• High computational density– Worthwhile to transfer data to separate device

• Both coarse-grained and fine-grained SIMD parallelism– Lots of independent tasks (blocks) that don’t require

frequent synchronization map to different multiprocessors on the GPU

– Within each block, lots of individual SIMD threads• Contiguous memory access patterns• Frequently/Repeatedly used data small enough to fit

in shared memory

C-means

• Minimizes square error between data points and cluster centers using Euclidean distance

• Alternates between computing membership values and updating cluster centers

C-means Parallel Implementation

C-means Parallel Implementation

EM with a Gaussian mixture model

• Data described by a mixture of M Gaussian distributions• Each Gaussian has 3 parameters

E-step

• Compute likelihoods based on current model parameters

• Convert likelihoods into membership values

M-step

• Update model parameters

EM Parallel Implementation

EM Parallel Implementation

Performance Tuning

• Global Memory Coalescing– 1.0/1.1 vs 1.2/1.3 devices

Performance Tuning

• Partition Camping

Performance Tuning

• CUBLAS

Multi-GPU Strategy

• 3 Tier Parallel hierarchy

Multi-GPU Strategy

Multi-GPU Implementation

• Very little impact on GPU kernel implementations, just their inputs / grid dimensions

• Discuss host-code changes

Data Distribution

• Asynchronous MPI sends from host instead of each node reading input file from data store

Results - Kernels

• Speedup figures • Speedup figures

Results - Kernels

• Speedup figures • Speedup figures

Results – Overhead

• Time-breakdown for I/O, GPU memcpy, etc

Multi-GPU Results

• Amdahl’s Law vs. Gustafson’s Law– i.e. Strong vs. Weak Scaling– i.e. Fixed Problem Size vs. Fixed-Time – i.e. True Speedup vs. Scaled Speedup

Fixed Problem Size Analysis

Time-Constrained Analysis

Conclusions

Future Work

Questions?

References

1. A. K. Jain, M. N. Murty, and P. J. Flynn, “Data clustering: a review,” ACM Comput. Surv., vol. 31, no. 3, pp. 264–323, 1999.

2. H. Shapiro, J. Wiley, and W. InterScience, Practical flow cytometry. Wiley-Liss New York, 2003.