Unai Lopez Novoa

19 June 2015

Phd Dissertation

Advisors: Jose Miguel-Alonso & Alexander Mendiburu

Contributions to the Efficient Use of

General Purpose Coprocessors:Kernel Density Estimation as Case Study

Outline

• Introduction

• Contributions1) A Survey of Performance Modeling and Simulation Techniques

2) S-KDE: An Efficient Algorithm for Kernel Density Estimation

• And its implementation for Multi and Many-cores

3) Implementation of S-KDE in General Purpose Coprocessors

4) A Methodology for Environmental Model Evaluation based on S-KDE

• Conclusions

2

Introduction

3

High Performance Computing• Branch of computer science related to the use of parallel

architectures to solve complex computational problems

• Today’s fastest supercomputer: Tianhe-2

4Introduction

(China’s National University of Defense and Technology, 33.86 PFLOP/s)

HPC Environments• Traditional HPC systems were homogeneous, built

around single or multi-core CPUs• But supercomputers are becoming heterogeneous

5Introduction

(Coprocessor number evolution in the Top500 list over time)

Compute platforms

Introduction 6

Multi-Core CPUs

• Branch prediction, OoOE• “Versatile”

Up to250 GFLOP/s

GraphicsProcessing

Units

• Hundreds of cores• Handle thousands of threads Up to

1.8 TFLOP/s

Many-CoreProcessors

• Tens of x86 cores• HyperThreading

Up to1 TFLOP/s

Device Features Peak D.P. Performance

Motivation• Examples of successful porting of applications to

accelerators (compared againts multi-core implementations):

• SAXPY: 11.8x• Polynomial Equation Solver: 79x• Image Treatment (MRI): 263x• …

• … but this is not applicable for every HPC code

Introduction 7

Ryoo, Shane, et al. "Optimization principles and application performance evaluation of a multithreaded GPU using CUDA." Proceedings of the 13th ACM SIGPLAN Symposium on Principles and practice of parallel programming. ACM, 2008. (>700 cites on Google Scholar)

Difficulties using accelerators• Suitable codes for accelerators should:

• Expose high levels of parallelism• Have a good spatial/temporal data locality• …

• Porting a code requires extensive program rewriting

• Development tools for accelerators are not as polished as those for CPUs

Effectively exploiting the performance of a coprocessor remains as a challenging task

Introduction 8

Structure of this thesis

Introduction 9

A survey of performance modeling and simulation techniques

Design of a novel algorithm for Kernel Density Estimation: S-KDE

S-KDE for Multi & Many-CoresS-KDE for Accelerators

A methodology for environmental model evaluation based on S-KDE

Motivation:Discuss the issues to efficiently use

general purpose coprocessors

Case Study:Kernel Density Estimation applied to environmental model evaluation

A Survey of Performance Modeling and Simulation Techniques

10

Developing for accelerators• Approaches/aids:

A Survey of Performance Modeling and Simulation Techniques 11

Trial and error

Profilers / Debuggers / …

Performance Models

A survey of models and simulators

• Accelerator & GPGPU trend began ~2005• First performance models appeared ~2007

• Abundant literature• No outstanding models or tools

A Survey of Performance Modeling and Simulation Techniques 12

Taxonomy

A Survey of Performance Modeling and Simulation Techniques 13

Execution timeestimation

Bottleneckhighlighting

Power cons. estimation

Simulators

Model analysis

• We analysed 29 relevant accelerator models

• For each of them we summarized and identified:• Modeling method (Analytical, Machine Learning,…)• Target platforms and test devices• Input preprocessing requirements• Limitations• Highlights over other models

A Survey of Performance Modeling and Simulation Techniques 14

The MWP-CWP model• Presented by Hong & Kim in 2009 (>360 cites in Google Scholar)

• Estimates the execution time of a GPU application• Based on how Warps are scheduled in NVIDIA GPUs

A Survey of Performance Modeling and Simulation Techniques 15

Test platform Input requirements Limitations HighlightsMethod

Analytical NVIDIA GPUs (8800GT,…)

Run µbenchmarks & Parse PTX

Branches notmodeled

Extendable to non-NVIDIA GPUs

The Roofline model• Presented by Williams et al. in 2009 (>450 cites in Google Scholar)

• Outstanding model for bottleneck highlighting• Visual model:

A Survey of Performance Modeling and Simulation Techniques 16

Test platform Input requirements Limitations HighlightsMethod

Analytical Multi-core CPUs& Accelerators

Run µbenchmarks & Analyse application

Depends onarchitecture

Visual output to guide optimizations

Performance tools

• Some models require running performance tools(µbenchmarks, profilers,…)

• We have reviewed them as well

A Survey of Performance Modeling and Simulation Techniques 17

Conclusions

1) There is no accurate model valid for a wide set of architectures

2) Most models are tied to CUDA

3) There is a growing interest in analyzing power

4) It was impossible to make a comparison of the models (lack of details, codes, …)

A Survey of Performance Modeling and Simulation Techniques 18

S-KDE: An Efficient Algorithm for Kernel Density Estimation

(and its implementation for Multi and Many-cores)

19

Case study

• Collaborative Work: EOLO

UPV/EHU Climate and Meteorology Group

• Scenario:Environmental Model Evaluation

• Problem:Excessive execution times of KDE

S-KDE: An Efficient Algorithm for Kernel Density Estimation 20

Kernel Density Estimation• Statistical technique used to estimate the Probability

Density Function (PDF) of a random variable with unknown characteristics

• where:• xi are the samples from the random variables

• K is the Kernel function• H is the bandwidth value

S-KDE: An Efficient Algorithm for Kernel Density Estimation 21

Kernel function• Symmetric function that integrates to one• We classify them according to area of influence

S-KDE: An Efficient Algorithm for Kernel Density Estimation 22

Bounded Unbounded

Bandwidth• Parameter to control the smoothness of the estimation• It must be carefully selected

• Common approaches for its selection• Heuristic as in Silverman, 1986• Iterative technique, e.g., bootstraping

S-KDE: An Efficient Algorithm for Kernel Density Estimation 23

Computing KDE

S-KDE: An Efficient Algorithm for Kernel Density Estimation 24

Naive approach: EP-KDE

for each eval_point e in E

for each sample s in S

d = distance(e,s)

e += density (d)

Our proposal: S-KDE

for each sample s in S

B = findInfluenceArea(s)

for each eval_point e in B

d = distance(e,s)

e += density (d)

Complexity: O(|E|·|S|) Complexity: O(|B|·|S|)

Delimiting the influence area

S-KDE: An Efficient Algorithm for Kernel Density Estimation 25

• Depends on the Kernel• Our case: Epanechnikov kernel • Technique based on a method in Fukunaga, 1990

Chop & Crop• In spaces of dimensionality 3 and higher, the number of

evaluation points outside the influence area increases • We developed a technique to further reduce evaluations:

S-KDE: An Efficient Algorithm for Kernel Density Estimation 26

Step 1: Chop the box into slices Step 2: Crop the slice

Example numbers

500k Samples 3D dataset

194M Evaluation point space

EP-KDE: 9.74 * 1013 distance-density evaluations

102461 Evaluation points per Bounding box (average)

S-KDE: 5.12 * 1010 evaluations

With C&C: 53511 Evaluation point per Bounding box (average)

S-KDE + C&C: 2.67 * 1010 evaluations

S-KDE: An Efficient Algorithm for Kernel Density Estimation 27

S-KDE in OpenMP

S-KDE: An Efficient Algorithm for Kernel Density Estimation 28

Initialization

Distribute samplesto threads

Fit bounding box

Chop into slices

Crop and computedensity

Accumulate densityto evaluation space

#pragma omp for

#pragma simd

#pragma atomic

S-KDE in OpenMP• Targeting Multi and Many core processors

• Tested platforms:• Intel i7 Intel Core i7 3820 CPU (4 Cores @ 3.6 GHz)• Intel Xeon Phi 3120A (57 Cores @ 1.1 GHz, Native mode)

• Public KDE implementations used as yardsticks:• Ks-kde (R Package)• GPUML • Several Python libraries

S-KDE: An Efficient Algorithm for Kernel Density Estimation 29

Execution time comparison

S-KDE: An Efficient Algorithm for Kernel Density Estimation 30

Conclusions

1) S-KDE + Chop & Crop reduces KDE complexity

2) Native, parallel implementation for Multi and Many-core processors

• OpenMP

3) We beat state-of-the-art alternatives

S-KDE: An Efficient Algorithm for Kernel Density Estimation 31

Implementation of S-KDE in General Purpose Coprocessors

32

S-KDE in OpenCL

Implementation of S-KDE in General Purpose Coprocessors 33

Initialization

Fit box & Chop

Crop

Offset calculation

Density computation

Density transfer

Density accumulation

(1)

(2)

(3)

(4)

(5)

(6)

(7)

• Host code• Accelerator code

Execution time comparison

Implementation of S-KDE in General Purpose Coprocessors 34

Conclusions

1) OpenCL version of S-KDE provides good overall performance

2) The consolidation stage is the main bottleneck

3) The code is close to the limits of the accelerators

4) Further performance gains using pipelined execution

Implementation of S-KDE in General Purpose Coprocessors 35

A Methodology for Environmental Model Evaluation based on S-KDE

36

Climate models

• Mathematical representations of a climate system, based on physical, chemical and biological principles

• They predict a trend in a long term time• Recently used to asses the impact of greenhouse gases

A Methodology for Environmental Model Evaluation based on S-KDE 37

Climate model evaluation

• Models must be validated against actual observations

• There is not a universally accepted validation strategy

• Popular approaches:• Averaged values per estimated variable• Evaluating the per-variable Probability Density Functions (PDFs)

A Methodology for Environmental Model Evaluation based on S-KDE 38

PDF-based model evaluation• Current approaches:

1) Compute the PDF per estimated variable

2) Calculate similarity score per-variable against observations

3) Combine the scores to get global performance of the model

• Lack of a universally accepted way to combine the scores

• Our proposal:• An extension of the score by [1] to multiple dimensions• A methodology to evaluate multiple variables in a single step

A Methodology for Environmental Model Evaluation based on S-KDE 39

[1]: Perkins, S. E., et al. "Evaluation of the AR4 climate models' simulated daily maximum temperature, minimum temperature, and precipitation over Australia using probability density functions." Journal of climate 20.17 (2007): 4356-4376.

Methodology

A Methodology for Environmental Model Evaluation based on S-KDE 40

1) Estimate optimal bandwidth

Iterative use of KDE

h = 0.6 h = 0.65

Estimations

MIROCS3.2-MR Model

Observations

3) Compute score S = 0.74

2) Compute PDF with opt. bandwidth

Single useof KDE

PDF (Estimations)h = 0.6

PDF (Observations)h = 0.65

PDF (Observations)h = 0.65

Evaluation• Models: 7 from CMIP3 experiment (with different configurations)

• Dataset: 20C3M (1961 to 1998 on a daily basis)

• Variables:• Global average of surface temperature• Difference in temperature between N and S hemispheres• Difference in temperature between Equator and the poles

• Scores for the models:

A Methodology for Environmental Model Evaluation based on S-KDE 41

NCEP MIROCS3.2MR-2

MIROCS3.2MR-3

HADGEM1

MIROCS3.2MR-1

MIROCS3.2-HR

GFDL-CM2.1

GFDL-CM2.0

BCM2.0 ECHAM5

MRI-RUN03

MRI-RUN04

MRI-RUN01

MRI-RUN02

MRI-RUN05

0,82 0,74 0,73 0,71 0,7 0,67 0,62 0,6 0,51 0,48 0,3 0,29 0,29 0,29 0,28

A Methodology for Environmental Model Evaluation based on S-KDE 42

MIROC3.2-MR-RUN02Score = 0,74

MRI-RUN01Score = 0,29

EvaluationSurface: ObservationsContour: Model

C0: Global average surface temperatureC1: Difference in temperature between Hemispheres

Conclusions

1)We have presented a methodology based on the extension to multiple dimensions of the index by Perkins et al.

2)It allows evaluating multiple variables of an environmental model in a single step

3)It is feasible in time thanks to the use of a fast implementation of KDE: S-KDE

A Methodology for Environmental Model Evaluation based on S-KDE 43

Conclusions

44

Summary of contributions

•We have conducted an extensive survey on performance models using a proposed taxonomy

•We have designed S-KDE, a technique that reduces the complexity of Kernel Density Estimation computations

•We have implemented S-KDE for Multi and Many-cores using OpenMP

• Outperforming the state-of-the-art parallel codes for KDE

Conclusions 45

Summary of contributions

• We have presented an OpenCL implementation of S-KDE for general purpose coprocessors. • It reaches the limits of the devices and acceptable performance,

but requires further work

• We have designed of a methodology for environmental model evaluation based on KDE, that allows to evaluate multiple variables from a model accurately in a simple way• S-KDE is a key, enabling element

Conclusions 46

Future work• We intend to develop a methodology for the performance

evaluation accelerator-based applications, based on the survey presented as first contribution

• We need to improve S-KDE in both multi-cores and coprocessors

• In particular, the consolidation stage

• We intend to design a technique to analyse new climate data from the CMIP Project, with dimensionality up to ten

Conclusions 47

Publications

Conclusions 48

Unai Lopez-Novoa, Alexander Mendiburu, and Jose Miguel-Alonso. A survey of performance modeling and simulation techniques for accelerator-based computing. IEEE Transactions on Parallel and Distributed Systems, 26(1):272–281, Jan 2015

Unai Lopez-Novoa, Jon Saenz, Alexander Mendiburu, and Jose Miguel-Alonso. An efficient implementation of kernel density estimation for multi-core & many-core architectures. International Journal of High Performance Computing Applications, Accepted, 2015, DOI: 10.1177/1094342015576813

Publications

Conclusions 49

Unai Lopez-Novoa, Alexander Mendiburu, and Jose Miguel-Alonso. Kernel density estimation in accelerators: Implementation and performance evaluation. Parallel Computing. To be submitted.

Unai Lopez-Novoa, Jon Saenz, Alexander Mendiburu, Jose Miguel-Alonso, Inigo Errasti, Ganix Esnaola, Agustin Ezcurra, and Gabriel Ibarra-Berastegi. Multi-objective environmental model evaluation by means of multidimensional kernel density estimators: Efficient and multi-core implementations. Environmental Modelling & Software, 63:123 – 136, 2015

Unai Lopez Novoa

19 June 2015

Phd Dissertation

Advisors: Jose Miguel-Alonso & Alexander Mendiburu

Contributions to the Efficient Use of

General Purpose Coprocessors:Kernel Density Estimation as Case Study