• Home

  • Documents

  • * Ghent University, Belgium - ° K.U. Leuven, Belgiumkehoste/ELIS/posters/hoste07... ·...
1
Analyzing Commercial Processor Performance Numbers for Predicting Performance of Applications of Interest Kenneth Hoste*, Lieven Eeckhout* and Hendrik Blockeel° * Ghent University, Belgium - ° K.U. Leuven, Belgium benchmark suite normalized μarch-indep. chars application of interest normalized μarch-indep. chars genetic algorithm performance numbers benchmark space w weights for characteristics Performance Estimation for Applications of Interest Analyzing Commercial Processor Performance Numbers Case study : Intel desktop processors Current Practice in Benchmarking Performance numbers are projected into a three-dimensional space using principal components analysis (PCA). Each point represents one of the 1181 machines, on which the SPEC CPU2000 benchmark suite was run (data obtained from www.spec.org). Choosing colors according to processor characteristics yields valuable insights. Principal Components Analysis (PCA) Principal components analysis is a statistical technique which computes new dimensions (PCs) with the following properties: Var[PC 1 ] ≥ Var[PC 2 ] ≥ ... Var[PC n ] Cov[PC i ,PC j ] = 0 (i ≠ j) Each principal component (PC) is a weighted linear sum of input dimensions. Retaining a limited number of PCs allows reduce the dimensionality of data while controlling the amount of information lost. We retain 3 PCs, which explain 92.5% of total variance. Cluster analysis Using cluster analysis (k-means) in the three-dimensional space described above, various groups of similar performing machines are identified. For each cluster of similar machines, a performance model is constructed. Simply grouping machines by ISA or average performance is insufficient, because of the large variety among machine models (for example, Intel Pentium 4 and Intel Xeon). 1st Principal Component 2nd Principal Component 3rd Principal Component 1st Principal Component 2nd Principal Component ! ! ! ! ! ! ! ! ! ! ! !!! ! ! ! ! ! ! !! ! ! ! ! ! !! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! !! !! ! ! ! ! ! ! ! ! ! !! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! !!! ! ! ! ! !! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! !! ! ! ! ! ! !! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !!! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! !! ! ! ! !!! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !3 !2 !1 0 1 2 3 !1 0 1 2 3 4 5 ! ! ! ! ! ! ! 0!500 (162) 501!1000 (181) 1001!1500 (207) 1501!2000 (392) 2001!2500 (138) 2501!3000 (88) 3001!3500 (13) colored by SPECfp rate 1st Principal Component 2nd Principal Component ! ! ! ! ! ! ! ! ! ! ! !!! ! ! ! ! ! ! !! ! ! ! ! ! !! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! !! !! ! ! ! ! ! ! ! ! ! !! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! !!! ! ! ! ! !! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! !! ! ! ! ! ! !! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !!! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! !! ! ! ! !!! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !3 !2 !1 0 1 2 3 !1 0 1 2 3 4 5 ! ! ! ! ! ! 0!500 (168) 501!1000 (216) 1001!1500 (328) 1501!2000 (350) 2001!3000 (97) 3001!3500 (22) colored by SPECint rate colored by ISA Performance model For each cluster of machines, a performance model is constructed using microarchitecture- independent program characteristics and a genetic algorithm. For a given application of interest, a performance estimation is made by making a weighted harmonic average of the performance numbers of similar benchmarks. Experimental results For each of the SPEC CPU2000 benchmarks, performance numbers (speedups compared to a reference machine) were estimated for 617 machines divided in 26 clusters (by means of cross-validation). This results in an estimated machine ranking. We evaluate the performance deficiency of the estimated best and top 3 machines for each benchmark. ammp applu apsi art bzip2 crafty eon equake facerec fma3d galgel gap gcc gzip lucas mcf mesa mgrid parser perlbmk sixtrack swim twolf vortex vpr wupwise 1st Principal Component (X!axis) 0.00 0.05 0.10 0.15 0.20 ammp applu apsi art bzip2 crafty eon equake facerec fma3d galgel gap gcc gzip lucas mcf mesa mgrid parser perlbmk sixtrack swim twolf vortex vpr wupwise 2nd Principal Component (Y!axis) !0.2 !0.1 0.0 0.1 0.2 0.3 ammp applu apsi art bzip2 crafty eon equake facerec fma3d galgel gap gcc gzip lucas mcf mesa mgrid parser perlbmk sixtrack swim twolf vortex vpr wupwise 3rd Principal Component (Z!axis) !0.2 !0.1 0.0 0.1 0.2 0.3 0.4 SPECint2000 benchmarks SPECfp2000 benchmarks Benchmarking systems is commonly done to assess system performance, for example using the well-known SPEC CPU benchmarks (www.spec.org). Performance numbers for each of the benchmarks are reported by various organizations, including system and processor manufacturers, both for research and marketing purposes. Two problems come forward when trying to interpret these performance numbers: • Because of the huge amount of data (speedup numbers for 26 benchmarks for 1000+ machines for SPEC CPU2000), getting valuable insights is hard. • It is unclear which benchmarks are representative for a given application of interest. Using average performance numbers across a set of benchmarks (for example SPEC rates) to determine which system is best suited for a given task can be misleading, and may lead to suboptimal purchase decisions. This work presents a solution for both problems; we analyze published performance numbers of commercial machines to obtain valuable insights and propose a methodology to estimate performance for a given application of interest. benchmark suite performance numbers application of interest !2 !1 0 1 2 3 !2 !1 0 1 2 3 4 5 !3 !2 !1 0 1 2 3 ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! !! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! Pentium III Pentium 4 (Willamette, 32!bit) Pentium 4 (Northwood, 32!bit) Pentium 4 (Prescott, 32!bit) Pentium 4 (Prescott, 64!bit) Pentium 4 (Prescott 2M, 64!bit) Pentium 4 (Cedar Mill, 64!bit) Pentium 4 Extreme Edition (Gallatin, 32!bit) Pentium 4 Extreme Edition (Prescott 2M, 64!bit) Pentium D (Smithfield) Pentium D (Presler) Pentium Extreme Edition (Smithfield) Pentium Extreme Edition (Presler) Core Duo (Yonah) Core 2 Duo (Merom) Core 2 Duo (Conroe) Core 2 Extreme (Conroe XE) Core 2 Extreme (Kentsfield XE) 3rd Principal Component 1st Principal Component 2nd Principal Component Some observations: ✦ increasing clock speed is more beneficial for SPECint than for SPECfp ✦ the new Core 2 architecture is more targeted to integer workloads than previous systems ✦ Core 2-based systems are well suited for cache-intensive workloads PC1 PC2 X Y average one model per ISA-model per-cluster model per-machine model cumulative average performance deficiency 0% 10% 20% 30% 0 5 10 15 20 26 number of benchmarks estimated best average one model per ISA-model per-cluster model per-machine model 0 5 10 15 20 26 number of benchmarks cumulative average performance deficiency 0% 10% 20% 30% estimated top 3 !2 !1 0 1 2 3 !2 !1 0 1 2 3 4 5 3 2 1 0 !1 !2 !3 ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! !!! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !!! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! AMD 64!bit DEC Alpha Intel 64!bit Intel 32!bit Intel Itanium MIPS PA!RISC IBM PowerPC Sun SPARC

* Ghent University, Belgium - ° K.U. Leuven, Belgiumkehoste/ELIS/posters/hoste07... · 2017-12-05 · ammp applu apsi art bzip2 crafty eon equake facerec fma3d galgel gap gcc gzip

  • Upload
    others

  • View
    3

  • Download
    0

Embed Size (px)

Citation preview

Page 1: * Ghent University, Belgium - ° K.U. Leuven, Belgiumkehoste/ELIS/posters/hoste07... · 2017-12-05 · ammp applu apsi art bzip2 crafty eon equake facerec fma3d galgel gap gcc gzip

Analyzing Commercial Processor Performance Numbers for Predicting Performance of Applications of Interest

Kenneth Hoste*, Lieven Eeckhout* and Hendrik Blockeel°* Ghent University, Belgium - ° K.U. Leuven, Belgium

benchmark

suite

normalized

µarch-indep.

chars

application

of interest

normalized

µarch-indep. chars

genetic

algorithm

performance

numbers

benchmark spacew

weights for characteristics

Performance Estimation for Applications of Interest

Analyzing Commercial Processor Performance Numbers

Case study: Intel desktop processors

Current Practice in Benchmarking

Performance numbers are projected into a three-dimensional space using principal components analysis (PCA). Each point represents one of the 1181 machines, on which the SPEC CPU2000 benchmark suite was run (data obtained from www.spec.org). Choosing colors according to processor characteristics yields valuable insights.

Principal Components Analysis (PCA)Principal components analysis is a statistical technique which computes new dimensions (PCs) with the following properties:

Var[PC1] ≥ Var[PC2] ≥ ... Var[PCn]Cov[PCi,PCj] = 0 (i ≠ j)

Each principal component (PC) is a weighted linear sum of input dimensions. Retaining a limited number of PCs allows reduce the dimensionality of data while controlling the amount of information lost.We retain 3 PCs, which explain 92.5% of total variance.

Cluster analysis Using cluster analysis (k-means) in the three-dimensional space described above, various groups of similar performing machines are identified.

For each cluster of similar machines, a performance model is constructed. Simply grouping machines by ISA or average performance is insufficient, because of the large variety among machine models (for example, Intel Pentium 4 and Intel Xeon).

1st Principal Component

2nd

Prin

cipa

l Com

pone

nt

3rd Principal Component

1st Principal Component

2nd

Prin

cipa

l Com

pone

nt

!!

!

!

!

!

!

!

!!!

! !!!

!

!!!

!!!!!!!

!!!! ! !!!

!

!!

!

! !

!!!!!

!!

!

!

!!!!!

!! !

!! !!

!!! !!

!

!!!

!!

!

!!!!

!!

!!

!

!!

!!

!!

! !

!!

!

!!!

!!!

!!!

!!

!!

!

!!!

!

!!

!!!

!!!

!!

!

!!!!!!!!

!!!!

!!!!

!

!!!

!

!

!!!

!

!!

!

!!!

!

!!

!

!!!!!!!!!

!!

!

!

!

!!!

!!

!!

!!

!!

!

!

!

!

!!

!!!

!!!

!!!!

!!

!

!

!

!!

!

!

!

!

!

!!!!

!

!

!

!

!

!! !!

!!

!!

!

!

! ! !

! !!!

!!

!

!!!!!

!!

!!

!

!

!

!!

!!

!!!

!!!

!!!

!!!!

!

!!

!!

!!!

!!!!!

!!

!

!

!!!!!

!

!!

!!!

!!

!

!!

!

!

!

!

!

!

!!!

!

!!

!

!

!

!

!

!

!!

!!

!

!

!

!!

!

!

!! !!

!!

!!!

!

!

! !!

!

!! !!

!

!!!!

!

!!

!!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

! !!

!!

!!!

!

!!

! ! !

!

!

!!

!

!

!!

!

! !!!!

!

! !

!

!!

!!!!!!

!

!

!

!

!

!!

!

!

!

!!

!

!!

!

!

!

!!

!

!

!

!

!

! !!

!

!!

!

!

!

!

!

!!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!!

!!

!!

!

!

!

!

!

!

!

!

!

!

!

!!!

!!

!

!

!!

!

!!

!

!

!

!

!

!!!

!

!

!

!!

!

!!!

!

!

!

!

!

!

!

!!

!

!

!!

!!

!

!!

! !

!

!

!

!

!!

!!

!

!!

!

!

!

!

!

!

!

!

!!!!

!!!

!

!

!!!

!

!!

!!!!!

!

!!!

!

!

!

!

!!!

!!

!!

!

!!

!

!!

!!

!

!!!

!!

!

!!!!

!!!

!

!

!!

!!!

!

!

!!

!!

!!!!

!!!!

!!

!

! !

!!

!

!

!!

!

!

!

!! ! !

!

!!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!!

!

!

!!!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!!!!!

!

! !

!

!

!!

!

!

!!

!!

!

!

!

!!

!!

!

!

!

!

!!!!!!!!

! ! ! !!

! !!

!

!

!

!

!

!

!!

!

!!!

!

!!

!!

!!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!!

!!

!!! !

!!

!

!!

!

! !

!

!!

!!

!

!

! !

!

!

!!

!!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!!! !

!!

! !!

!

!

!

! !

!!

!!

!!

!

!

!

!!

!!

!

!

!

!!

!! !

!

!!

!!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!!

!!

!

!

!

!

!

!

!

!!

!

!

!

!!!!

!!

!!!!

!

!

!

!!

! !

!

!

!

!

!

!

!!

!!!

!

!

!

!

!!!!

!!

!

!

!!

!

!

!

!

!

!

!

!

!

!!

! !!

!!

!

!

!!

!

!!

!

!!

!!

!

!

!

!

!

!!!

!

!

!

!

!

!!

!!

!

!

!!

!

!

!!!!

!

!

!

!

!!

!

!

!

!

!!

!!

!

!

!

!

! !

!

!

!!!

!

!

!

!

!

!

!

!

!

!!!

!!

!!

!

!!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!!

! !!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!!

!

!!

!

!

!

!!

!

!

!

!!

!

!

!

!!

!

!

!!

!

!

!

!

!

!

!!

!!

! !

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!!

!!

!

!!!!

!! ! !

!

!

! !

!!

!

!

!!

!

!

!!

!

!

!

!!

!

!

!

!!

!

!!

!!

!3 !2 !1 0 1 2 3

!1

01

23

45

1st Principal Component

2rd

Princip

al C

om

ponent

!

!

!

!

!

!

0!500 (168)

501!1000 (216)

1001!1500 (328)

1501!2000 (350)

2001!3000 (97)

3001!3500 (22)

!!

!

!

!

!

!

!

!!!

! !!!

!

!!!

!!!!!!!

!!!! ! !!!

!

!!

!

! !

!!!!!

!!

!

!

!!!!!

!! !

!! !!

!!! !!

!

!!!

!!

!

!!!!

!!

!!

!

!!

!!

!!

! !

!!

!

!!!

!!!

!!!

!!

!!

!

!!!

!

!!

!!!

!!!

!!

!

!!!!!!!!

!!!!

!!!!

!

!!!

!

!

!!!

!

!!

!

!!!

!

!!

!

!!!!!!!!!

!!

!

!

!

!!!

!!

!!

!!

!!

!

!

!

!

!!

!!!

!!!

!!!!

!!

!

!

!

!!

!

!

!

!

!

!!!!

!

!

!

!

!

!! !!

!!

!!

!

!

! ! !

! !!!

!!

!

!!!!!

!!

!!

!

!

!

!!

!!

!!!

!!!

!!!

!!!!

!

!!

!!

!!!

!!!!!

!!

!

!

!!!!!

!

!!

!!!

!!

!

!!

!

!

!

!

!

!

!!!

!

!!

!

!

!

!

!

!

!!

!!

!

!

!

!!

!

!

!! !!

!!

!!!

!

!

! !!

!

!! !!

!

!!!!

!

!!

!!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

! !!

!!

!!!

!

!!

! ! !

!

!

!!

!

!

!!

!

! !!!!

!

! !

!

!!

!!!!!!

!

!

!

!

!

!!

!

!

!

!!

!

!!

!

!

!

!!

!

!

!

!

!

! !!

!

!!

!

!

!

!

!

!!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!!

!!

!!

!

!

!

!

!

!

!

!

!

!

!

!!!

!!

!

!

!!

!

!!

!

!

!

!

!

!!!

!

!

!

!!

!

!!!

!

!

!

!

!

!

!

!!

!

!

!!

!!

!

!!

! !

!

!

!

!

!!

!!

!

!!

!

!

!

!

!

!

!

!

!!!!

!!!

!

!

!!!

!

!!

!!!!!

!

!!!

!

!

!

!

!!!

!!

!!

!

!!

!

!!

!!

!

!!!

!!

!

!!!!

!!!

!

!

!!

!!!

!

!

!!

!!

!!!!

!!!!

!!

!

! !

!!

!

!

!!

!

!

!

!! ! !

!

!!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!!

!

!

!!!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!!!!!

!

! !

!

!

!!

!

!

!!

!!

!

!

!

!!

!!

!

!

!

!

!!!!!!!!

! ! ! !!

! !!

!

!

!

!

!

!

!!

!

!!!

!

!!

!!

!!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!!

!!

!!! !

!!

!

!!

!

! !

!

!!

!!

!

!

! !

!

!

!!

!!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!!! !

!!

! !!

!

!

!

! !

!!

!!

!!

!

!

!

!!

!!

!

!

!

!!

!! !

!

!!

!!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!!

!!

!

!

!

!

!

!

!

!!

!

!

!

!!!!

!!

!!!!

!

!

!

!!

! !

!

!

!

!

!

!

!!

!!!

!

!

!

!

!!!!

!!

!

!

!!

!

!

!

!

!

!

!

!

!

!!

! !!

!!

!

!

!!

!

!!

!

!!

!!

!

!

!

!

!

!!!

!

!

!

!

!

!!

!!

!

!

!!

!

!

!!!!

!

!

!

!

!!

!

!

!

!

!!

!!

!

!

!

!

! !

!

!

!!!

!

!

!

!

!

!

!

!

!

!!!

!!

!!

!

!!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!!

! !!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!!

!

!!

!

!

!

!!

!

!

!

!!

!

!

!

!!

!

!

!!

!

!

!

!

!

!

!!

!!

! !

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!!

!!

!

!!!!

!! ! !

!

!

! !

!!

!

!

!!

!

!

!!

!

!

!

!!

!

!

!

!!

!

!!

!!

!3 !2 !1 0 1 2 3

!1

01

23

45

2nd Principal Component

2rd

Princip

al C

om

ponent

!

!

!

!

!

!

!

0!500 (162)

501!1000 (181)

1001!1500 (207)

1501!2000 (392)

2001!2500 (138)

2501!3000 (88)

3001!3500 (13)

colored by SPECfp rate

1st Principal Component

2nd

Prin

cipa

l Com

pone

nt

!!

!

!

!

!

!

!

!!!

! !!!

!

!!!

!!!!!!!

!!!! ! !!!

!

!!

!

! !

!!!!!

!!

!

!

!!!!!

!! !

!! !!

!!! !!

!

!!!

!!

!

!!!!

!!

!!

!

!!

!!

!!

! !

!!

!

!!!

!!!

!!!

!!

!!

!

!!!

!

!!

!!!

!!!

!!

!

!!!!!!!!

!!!!

!!!!

!

!!!

!

!

!!!

!

!!

!

!!!

!

!!

!

!!!!!!!!!

!!

!

!

!

!!!

!!

!!

!!

!!

!

!

!

!

!!

!!!

!!!

!!!!

!!

!

!

!

!!

!

!

!

!

!

!!!!

!

!

!

!

!

!! !!

!!

!!

!

!

! ! !

! !!!

!!

!

!!!!!

!!

!!

!

!

!

!!

!!

!!!

!!!

!!!

!!!!

!

!!

!!

!!!

!!!!!

!!

!

!

!!!!!

!

!!

!!!

!!

!

!!

!

!

!

!

!

!

!!!

!

!!

!

!

!

!

!

!

!!

!!

!

!

!

!!

!

!

!! !!

!!

!!!

!

!

! !!

!

!! !!

!

!!!!

!

!!

!!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

! !!

!!

!!!

!

!!

! ! !

!

!

!!

!

!

!!

!

! !!!!

!

! !

!

!!

!!!!!!

!

!

!

!

!

!!

!

!

!

!!

!

!!

!

!

!

!!

!

!

!

!

!

! !!

!

!!

!

!

!

!

!

!!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!!

!!

!!

!

!

!

!

!

!

!

!

!

!

!

!!!

!!

!

!

!!

!

!!

!

!

!

!

!

!!!

!

!

!

!!

!

!!!

!

!

!

!

!

!

!

!!

!

!

!!

!!

!

!!

! !

!

!

!

!

!!

!!

!

!!

!

!

!

!

!

!

!

!

!!!!

!!!

!

!

!!!

!

!!

!!!!!

!

!!!

!

!

!

!

!!!

!!

!!

!

!!

!

!!

!!

!

!!!

!!

!

!!!!

!!!

!

!

!!

!!!

!

!

!!

!!

!!!!

!!!!

!!

!

! !

!!

!

!

!!

!

!

!

!! ! !

!

!!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!!

!

!

!!!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!!!!!

!

! !

!

!

!!

!

!

!!

!!

!

!

!

!!

!!

!

!

!

!

!!!!!!!!

! ! ! !!

! !!

!

!

!

!

!

!

!!

!

!!!

!

!!

!!

!!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!!

!!

!!! !

!!

!

!!

!

! !

!

!!

!!

!

!

! !

!

!

!!

!!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!!! !

!!

! !!

!

!

!

! !

!!

!!

!!

!

!

!

!!

!!

!

!

!

!!

!! !

!

!!

!!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!!

!!

!

!

!

!

!

!

!

!!

!

!

!

!!!!

!!

!!!!

!

!

!

!!

! !

!

!

!

!

!

!

!!

!!!

!

!

!

!

!!!!

!!

!

!

!!

!

!

!

!

!

!

!

!

!

!!

! !!

!!

!

!

!!

!

!!

!

!!

!!

!

!

!

!

!

!!!

!

!

!

!

!

!!

!!

!

!

!!

!

!

!!!!

!

!

!

!

!!

!

!

!

!

!!

!!

!

!

!

!

! !

!

!

!!!

!

!

!

!

!

!

!

!

!

!!!

!!

!!

!

!!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!!

! !!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!!

!

!!

!

!

!

!!

!

!

!

!!

!

!

!

!!

!

!

!!

!

!

!

!

!

!

!!

!!

! !

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!!

!!

!

!!!!

!! ! !

!

!

! !

!!

!

!

!!

!

!

!!

!

!

!

!!

!

!

!

!!

!

!!

!!

!3 !2 !1 0 1 2 3

!1

01

23

45

1st Principal Component

2rd

Princip

al C

om

ponent

!

!

!

!

!

!

0!500 (168)

501!1000 (216)

1001!1500 (328)

1501!2000 (350)

2001!3000 (97)

3001!3500 (22)

!!

!

!

!

!

!

!

!!!

! !!!

!

!!!

!!!!!!!

!!!! ! !!!

!

!!

!

! !

!!!!!

!!

!

!

!!!!!

!! !

!! !!

!!! !!

!

!!!

!!

!

!!!!

!!

!!

!

!!

!!

!!

! !

!!

!

!!!

!!!

!!!

!!

!!

!

!!!

!

!!

!!!

!!!

!!

!

!!!!!!!!

!!!!

!!!!

!

!!!

!

!

!!!

!

!!

!

!!!

!

!!

!

!!!!!!!!!

!!

!

!

!

!!!

!!

!!

!!

!!

!

!

!

!

!!

!!!

!!!

!!!!

!!

!

!

!

!!

!

!

!

!

!

!!!!

!

!

!

!

!

!! !!

!!

!!

!

!

! ! !

! !!!

!!

!

!!!!!

!!

!!

!

!

!

!!

!!

!!!

!!!

!!!

!!!!

!

!!

!!

!!!

!!!!!

!!

!

!

!!!!!

!

!!

!!!

!!

!

!!

!

!

!

!

!

!

!!!

!

!!

!

!

!

!

!

!

!!

!!

!

!

!

!!

!

!

!! !!

!!

!!!

!

!

! !!

!

!! !!

!

!!!!

!

!!

!!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

! !!

!!

!!!

!

!!

! ! !

!

!

!!

!

!

!!

!

! !!!!

!

! !

!

!!

!!!!!!

!

!

!

!

!

!!

!

!

!

!!

!

!!

!

!

!

!!

!

!

!

!

!

! !!

!

!!

!

!

!

!

!

!!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!!

!!

!!

!

!

!

!

!

!

!

!

!

!

!

!!!

!!

!

!

!!

!

!!

!

!

!

!

!

!!!

!

!

!

!!

!

!!!

!

!

!

!

!

!

!

!!

!

!

!!

!!

!

!!

! !

!

!

!

!

!!

!!

!

!!

!

!

!

!

!

!

!

!

!!!!

!!!

!

!

!!!

!

!!

!!!!!

!

!!!

!

!

!

!

!!!

!!

!!

!

!!

!

!!

!!

!

!!!

!!

!

!!!!

!!!

!

!

!!

!!!

!

!

!!

!!

!!!!

!!!!

!!

!

! !

!!

!

!

!!

!

!

!

!! ! !

!

!!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!!

!

!

!!!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!!!!!

!

! !

!

!

!!

!

!

!!

!!

!

!

!

!!

!!

!

!

!

!

!!!!!!!!

! ! ! !!

! !!

!

!

!

!

!

!

!!

!

!!!

!

!!

!!

!!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!!

!!

!!! !

!!

!

!!

!

! !

!

!!

!!

!

!

! !

!

!

!!

!!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!!! !

!!

! !!

!

!

!

! !

!!

!!

!!

!

!

!

!!

!!

!

!

!

!!

!! !

!

!!

!!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!!

!!

!

!

!

!

!

!

!

!!

!

!

!

!!!!

!!

!!!!

!

!

!

!!

! !

!

!

!

!

!

!

!!

!!!

!

!

!

!

!!!!

!!

!

!

!!

!

!

!

!

!

!

!

!

!

!!

! !!

!!

!

!

!!

!

!!

!

!!

!!

!

!

!

!

!

!!!

!

!

!

!

!

!!

!!

!

!

!!

!

!

!!!!

!

!

!

!

!!

!

!

!

!

!!

!!

!

!

!

!

! !

!

!

!!!

!

!

!

!

!

!

!

!

!

!!!

!!

!!

!

!!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!!

! !!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!!

!

!!

!

!

!

!!

!

!

!

!!

!

!

!

!!

!

!

!!

!

!

!

!

!

!

!!

!!

! !

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!!

!!

!

!!!!

!! ! !

!

!

! !

!!

!

!

!!

!

!

!!

!

!

!

!!

!

!

!

!!

!

!!

!!

!3 !2 !1 0 1 2 3

!1

01

23

45

2nd Principal Component

2rd

Princip

al C

om

ponent

!

!

!

!

!

!

!

0!500 (162)

501!1000 (181)

1001!1500 (207)

1501!2000 (392)

2001!2500 (138)

2501!3000 (88)

3001!3500 (13)

colored by SPECint ratecolored by ISA

Performance modelFor each cluster of machines, a performance model is constructed using microarchitecture-independent program characteristics and a genetic algorithm. For a given application of interest, a performance estimation is made by making a weighted harmonic average of the performance numbers of similar benchmarks.

Experimental resultsFor each of the SPEC CPU2000 benchmarks, performance numbers (speedups compared to a reference machine) were estimated for 617 machines divided in 26 clusters (by means of cross-validation). This results in an estimated machine ranking. We evaluate the performance deficiency of the estimated best and top 3 machines for each benchmark.

ammp

applu

apsi

art

bzip2

crafty

eon

equake

facerec

fma3d

galgel

gap

gcc

gzip

lucas

mcf

mesa

mgrid

parser

perlbmk

sixtrack

swim

twolf

vortex

vpr

wupwise

1st Principal Component (X!axis)

0.00

0.05

0.10

0.15

0.20

ammp

applu

apsi

art

bzip2

crafty

eon

equake

facerec

fma3d

galgel

gap

gcc

gzip

lucas

mcf

mesa

mgrid

parser

perlbmk

sixtrack

swim

twolf

vortex

vpr

wupwise

2nd Principal Component (Y!axis)

!0.2

!0.1

0.0

0.1

0.2

0.3

ammp

applu

apsi

art

bzip2

crafty

eon

equake

facerec

fma3d

galgel

gap

gcc

gzip

lucas

mcf

mesa

mgrid

parser

perlbmk

sixtrack

swim

twolf

vortex

vpr

wupwise

3rd Principal Component (Z!axis)

!0.2

!0.1

0.0

0.1

0.2

0.3

0.4

SPECint2000 benchmarks SPECfp2000 benchmarks

Benchmarking systems is commonly done to assess system performance, for example using the well-known SPEC CPU benchmarks (www.spec.org). Performance numbers for each of the benchmarks are reported by various organizations, including system and processor manufacturers, both for research and marketing purposes. Two problems come forward when trying to interpret these performance numbers:• Because of the huge amount of data (speedup numbers for 26 benchmarks for 1000+ machines for SPEC CPU2000), getting valuable insights is hard.• It is unclear which benchmarks are representative for a given application of interest. Using average performance numbers across a set of benchmarks (for example SPEC rates) to determine which system is best suited for a given task can be misleading, and may lead to suboptimal purchase decisions.This work presents a solution for both problems; we analyze published performance numbers of commercial machines to obtain valuable insights and propose a methodology to estimate performance for a given application of interest.

benchmark suite

performance numbers application of

interest

!2 !1 0 1 2 3

!2

!1

0 1

2 3

4 5

!3!2

!1 0

1 2

3

1st Principal Component

3rd

Prin

cip

al C

om

po

ne

nt

2n

d P

rin

cip

al C

om

po

ne

nt

!

!

!

!

!!

!

!

!

!

!

!!!

!

!!

!

!

!

!

!!!!

!

!

!

!!

!

!!

!!!!

!!!!!

!

!!

!

!

!

!!

!

!

!

!

!

!

!!!

!!

!

!

!

!

!!

!

!

!!

!!

!

!

!!

!

!

!

!

!!!!

!!

!!

!

!

!!

!!

!

!

!!

!!

!

!

!

!

!

!

!!

!!

!

!

!

!!

!

!!

!

!

!

!!

!

!!!!!

!!

!! !

!

! !!!

!!!!! !

!!!!!!

!!

!!

!

!

!!

!

!!

!

!

!

!

!

!

!

!

!

Pentium III

Pentium 4 (Willamette, 32!bit)

Pentium 4 (Northwood, 32!bit)

Pentium 4 (Prescott, 32!bit)

Pentium 4 (Prescott, 64!bit)

Pentium 4 (Prescott 2M, 64!bit)

Pentium 4 (Cedar Mill, 64!bit)

Pentium 4 Extreme Edition (Gallatin, 32!bit)

Pentium 4 Extreme Edition (Prescott 2M, 64!bit)

Pentium D (Smithfield)

Pentium D (Presler)

Pentium Extreme Edition (Smithfield)

Pentium Extreme Edition (Presler)

Core Duo (Yonah)

Core 2 Duo (Merom)

Core 2 Duo (Conroe)

Core 2 Extreme (Conroe XE)

Core 2 Extreme (Kentsfield XE)

3rd Principal Component

1st Principal Component

2nd

Prin

cipa

l Com

pone

nt

Some observations:

✦ increasing clock speed is more beneficial for SPECint than for SPECfp✦ the new Core 2 architecture is more targeted to integer workloads than previous systems✦ Core 2-based systems are well suited for cache-intensive workloads

PC1PC2

X

Y

!"#!$

!"#$%&'()'$%!*+#,&-.

*"#"/,012%',2%&,3%'4%&)(&#,!*%'5%)1*1%!*6

78

978

:78

;78

7 < 97 9< :7 :=

,2%&,3%(!%'#(5%/4%&'>?@!#(5%/4%&!*/".0%&'#(5%/4%&!#,*+1!%'#(5%/

!"#!%

!"#$%&'()'$%!*+#,&-.

*"#"/,012%',2%&,3%'4%&)(&#,!*%'5%)1*1%!*6

78

978

:78

;78

7 < 97 9< :7 :=

,2%&,3%(!%'#(5%/4%&'>?@!#(5%/4%&!*/".0%&'#(5%/4%&!#,*+1!%'#(5%/

!"#!&

!"#$%&'()'$%!*+#,&-.

*"#"/,012%',2%&,3%'4%&)(&#,!*%'5%)1*1%!*6

78

978

:78

;78

7 < 97 9< :7 :=

,2%&,3%(!%'#(5%/4%&'>?@!#(5%/4%&!*/".0%&'#(5%/4%&!#,*+1!%'#(5%/

!"#!$'

!"#$%&'()'$%!*+#,&-.

*"#"/,012%',2%&,3%'4%&)(&#,!*%'5%)1*1%!*6

78

978

:78

;78

7 < 97 9< :7 :=

,2%&,3%(!%'#(5%/4%&'>?@!#(5%/4%&!*/".0%&'#(5%/4%&!#,*+1!%'#(5%/

Figure 8: Cumulative average performance deficiency as a function of the number of benchmarks considered when predicting the

best performing top-1, top-3, top-5 and top-10 machines.

Figure 9: Performance deficiency for all benchmarks when predicting the top-3 machines.

averageone modelper ISA-modelper-cluster modelper-machine model

cum

ulat

ive

aver

age

perfo

rman

ce d

efic

ienc

y

0%

1

0%

20

%

30

%

0 5 10 15 20 26

number of benchmarks

estimated best

!"#!$

!"#$%&'()'$%!*+#,&-.

*"#"/,012%',2%&,3%'4%&)(&#,!*%'5%)1*1%!*6

78

978

:78

;78

7 < 97 9< :7 :=

,2%&,3%(!%'#(5%/4%&'>?@!#(5%/4%&!*/".0%&'#(5%/4%&!#,*+1!%'#(5%/

!"#!%

!"#$%&'()'$%!*+#,&-.

*"#"/,012%',2%&,3%'4%&)(&#,!*%'5%)1*1%!*6

78

978

:78

;78

7 < 97 9< :7 :=

,2%&,3%(!%'#(5%/4%&'>?@!#(5%/4%&!*/".0%&'#(5%/4%&!#,*+1!%'#(5%/

!"#!&

!"#$%&'()'$%!*+#,&-.

*"#"/,012%',2%&,3%'4%&)(&#,!*%'5%)1*1%!*6

78

978

:78

;78

7 < 97 9< :7 :=

,2%&,3%(!%'#(5%/4%&'>?@!#(5%/4%&!*/".0%&'#(5%/4%&!#,*+1!%'#(5%/

!"#!$'

!"#$%&'()'$%!*+#,&-.

*"#"/,012%',2%&,3%'4%&)(&#,!*%'5%)1*1%!*6

78

978

:78

;78

7 < 97 9< :7 :=

,2%&,3%(!%'#(5%/4%&'>?@!#(5%/4%&!*/".0%&'#(5%/4%&!#,*+1!%'#(5%/

Figure 8: Cumulative average performance deficiency as a function of the number of benchmarks considered when predicting the

best performing top-1, top-3, top-5 and top-10 machines.

Figure 9: Performance deficiency for all benchmarks when predicting the top-3 machines.

averageone modelper ISA-modelper-cluster modelper-machine model

0 5 10 15 20 26

number of benchmarks

cum

ulat

ive

aver

age

perfo

rman

ce d

efic

ienc

y

0%

1

0%

20

%

30

% estimated top 3

!2 !1 0 1 2 3

!2

!1

0 1

2 3

4 5

32

10

!1!2

!3

1st Principal Component

3rd

Prin

cip

al C

om

po

ne

nt

2n

d P

rin

cip

al C

om

po

ne

nt

!!!

!

!

!!

!!

!

!

!

!!

!

!

!

!

!!!

!!

!

!

!!

!

!

!

!

! !

!

!

!

!!

!

!

! !

!

!

!!

!!

!!

!

!!

!

!

!!!

!!

!

!!!

!

!!!

!

!

!

!

!

!

!

!

!

!!!

!

!

!!

!!!

!

!

!!

!

!!

! !!

!

!!

!

!

!!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!!

!

!

! !

!

!

!

!!

!

!

!

!

!

!

!! !

!

!

!

!!!

!!

!

!!

!!!

!

!

!

!!!

!!!

!!

!!

! !

!

!

!!!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!!

! !

!

!

!

!

!

! !

!

!

!

!

!

!

!

!

!

!

!!

!!

!

!

!

!

! !

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!!

!!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!!

!

!

!

! !

!

!

!

!!

!

!!

!

!!

!

!

!

! !

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!!

!

!

!!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

! !

!

!

! !!

!

!!

!

!

!

!

!!

!

!

!

!

!

!

!!

!!

!

!!!!!!

!!

!!

!

!

!!

!

!

!

!

!

!

!

!

!

!!!!!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!!!

!

!

!

!

!

!

!

! !! !

!

!!!

!

!

!

!

!!

!

!

!

!!

!

!

!

!

!

!

!

!

!!

!

!

!

!!

!

!

!!

!

!!

!

!

!

!

!

!

!!

! !

!

!

!

!

!!!

!

!

!

!

!

!

!

!

!!!

!

!

!

!

!

! !

!

!

!

!

!

!

!

!

!

!!!

!

!!

!

!

!

!

!!!

!

!!

!

!! !!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!!!!

!

!

!

! !

!

!

!

!

!

!

!

!

!

!

!!

!

!

!!

!

!

!

!

!

!

!

!

!

!!

!!!!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!!!

!

!

!

!!

!!

!

!

!

!

!

!

!

!!

!

!

!!

!

!

!!

!

!

!!

!

!

!

!

!!

!

!

!

!!

!

!

!!!!

!!

!!

!

!

!!

!

!

!

!

!

!

! !!!

!

!

!

!

!

!

!

!

!

!

!

!! !

!

!

!

!

!

!

!

!

!

!!

!

!!

!

!

! !!

!!

!!

!

!!!!!! !!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!!

!!

!

!!

!

!

!

!

!

!!!!!

!!

!!

!

!

!

!

!

!!

!

!

!

!!

!

!

!!!

!

!

!

!

!

!

!!!

!

!!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!!

!!

!

!

!

!!!

!

!

!

!

!

!!!

!!

!

!

!!

!

!

!

!

!

!!

!

!

!!!!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!!

!

!!!

!

!

!

!

!

!

!

!

!!

!

!

!!

!!

!

!

!!

!!

!

!

!!

!!

!

!

!

!!!

!

!!

!

!

!

!

!

!!!

!

!

!!

!!

!

!

!!

!

!

!!

!

!

!!

!

!

!

!

!

!

!!!!!

!

!

!

!!

!

!

!

!!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!!

!

!

!!!!

!

!

!

!

!

!!

!!

!

!

!!!!

!!!

!

!

!!!

!

!

!

!

!

!

!

!

!

!!!!

!

!!

!

!

!

!!

!

!!

!

!

!

!

!

!

!

!

!

AMD 64!bit

DEC Alpha

Intel 64!bit

Intel 32!bit

Intel Itanium

MIPS

PA!RISC

IBM PowerPC

Sun SPARC

http://www.spec.org
http://www.spec.org
http://www.spec.org
http://www.spec.org

T Q Decoder Human-Centric Compression · 2020. 9. 10. · The processed and bzip2-compressed text transcript is the compressed representation of the original image References 1. Agustsson,

T Q Decoder Human-Centric Compression · 2020. 9. 10. · The processed and bzip2-compressed text transcript is the compressed representation of the original image References 1. Agustsson,

Documents
2005 equake

2005 equake

Documents
Optimizing for MP:s - Uppsala University · LBM LQ ++ + + + 0 0,2 0,4 0,6 0,8 1 1,2 bzip2 Libquantum LBM Geom mean Individually In mix In mix, patched 25% AMD Opteron Performance

Optimizing for MP:s - Uppsala University · LBM LQ ++ + + + 0 0,2 0,4 0,6 0,8 1 1,2 bzip2 Libquantum LBM Geom mean Individually In mix In mix, patched 25% AMD Opteron Performance

Documents
Intel® Software Development Products for MeeGo* · from the SPEC CPU2000 floating point suite (177.mesa, 179.art, 183.equake, 188.ammp) C/C++ Compiler Benchmark Intel® C++ Compiler

Intel® Software Development Products for MeeGo* · from the SPEC CPU2000 floating point suite (177.mesa, 179.art, 183.equake, 188.ammp) C/C++ Compiler Benchmark Intel® C++ Compiler

Documents
10.1 Burrows-Wheeler transform · algorithm (now used in bzip2). The algorithm transforms a given text into the so called Burrows-Wheeler transform (BWT), a permutation of the text

10.1 Burrows-Wheeler transform · algorithm (now used in bzip2). The algorithm transforms a given text into the so called Burrows-Wheeler transform (BWT), a permutation of the text

Documents
ch13 equake - University of Colorado Boulderruby.colorado.edu/~smyth/G1010/13EarthquakesEric.pdfGreat Wave off the Coast of Kanagawa, by Hokusai, a famous late eighteenth- and early

ch13 equake - University of Colorado Boulderruby.colorado.edu/~smyth/G1010/13EarthquakesEric.pdfGreat Wave off the Coast of Kanagawa, by Hokusai, a famous late eighteenth- and early

Documents
win81 7zip bzip2 - brazil.minnesota.edubrazil.minnesota.edu/examples/ex/win81_7zip_bzip2.pdf · The unRAR code is under a mixed license unRAR restrictions. ... \Users\Tim\Documents\bzip2\

win81 7zip bzip2 - brazil.minnesota.edubrazil.minnesota.edu/examples/ex/win81_7zip_bzip2.pdf · The unRAR code is under a mixed license unRAR restrictions. ... \Users\Tim\Documents\bzip2\

Documents
Tdswm Equake Guidance Document

Tdswm Equake Guidance Document

Documents
Validation plan - CORDIS · architecture and the following libraries are prerequisite: curl 7.15 bzip2-libs 1.03 pcre 6.6.2 mysql-server >= 5.0 (only libmysqlclient must be installed)

Validation plan - CORDIS · architecture and the following libraries are prerequisite: curl 7.15 bzip2-libs 1.03 pcre 6.6.2 mysql-server >= 5.0 (only libmysqlclient must be installed)

Documents
Intro to Linux Examples - University of Birmingham · tar -cvpf ~/newfile.tar ~/testdir . Page 12 of 12 gzip or bzip2 These are often used to compress files gzip -9 ~/newfile.tar

Intro to Linux Examples - University of Birmingham · tar -cvpf ~/newfile.tar ~/testdir . Page 12 of 12 gzip or bzip2 These are often used to compress files gzip -9 ~/newfile.tar

Documents
Processor IP product line - RISC-V€¦ · BI-671 SPEC2006 performance 15 BI-671 data is preliminary and collected on FPGA prototype bzip2, gcc, mcf – missing data for Zynq-7000

Processor IP product line - RISC-V€¦ · BI-671 SPEC2006 performance 15 BI-671 data is preliminary and collected on FPGA prototype bzip2, gcc, mcf – missing data for Zynq-7000

Documents
SlowFuzz: Automated Domain-Independent Detection of ...€¦ · -Exploitability & Denial of Service (DoS) ... -PCRE regular expression library -bzip2 ‣ Evolutionary fuzzing as a

SlowFuzz: Automated Domain-Independent Detection of ...€¦ · -Exploitability & Denial of Service (DoS) ... -PCRE regular expression library -bzip2 ‣ Evolutionary fuzzing as a

Documents
Exploiting Application-Level Correctness for Low-Cost ... · the SPECInt CPU2000 workloads are 164.gzip and 256.bzip2, two lossless data compression algorithms, and 175.vpr, a place-and-route

Exploiting Application-Level Correctness for Low-Cost ... · the SPECInt CPU2000 workloads are 164.gzip and 256.bzip2, two lossless data compression algorithms, and 175.vpr, a place-and-route

Documents
full circledl.fullcirclemagazine.org/issueSE01_it.pdf · 2011. 1. 29. · daemon zoo unzip bzip2 arj nomarch lzop cabextract apt-listchanges libnet-ldab-perl libauthen-sasl-perl clamav-docs

full circledl.fullcirclemagazine.org/issueSE01_it.pdf · 2011. 1. 29. · daemon zoo unzip bzip2 arj nomarch lzop cabextract apt-listchanges libnet-ldab-perl libauthen-sasl-perl clamav-docs

Documents
A program and library for data compressionCompress or decompress to standard output.-d --decompress Force decompression. bzip2, bunzip2 and bzcat are really the same program, and the

A program and library for data compressionCompress or decompress to standard output.-d --decompress Force decompression. bzip2, bunzip2 and bzcat are really the same program, and the

Documents
Lossless Data Compression on GPUs · Lossless Data Compression on GPUs - GPU Technology Conference 2012 Ritesh Patel, Jason Mak data compression algorithms, bzip2, burrows wheeler

Lossless Data Compression on GPUs · Lossless Data Compression on GPUs - GPU Technology Conference 2012 Ritesh Patel, Jason Mak data compression algorithms, bzip2, burrows wheeler

Documents
Equake Final

Equake Final

Documents
The Burrows Wheeler transform and the FM indexhaimk/adv-alg-2014/Burrows-Wheeler.pdf · Burrows –Wheeler (bzip2) Currently best algorithm for text compression High level: • Apply

The Burrows Wheeler transform and the FM indexhaimk/adv-alg-2014/Burrows-Wheeler.pdf · Burrows –Wheeler (bzip2) Currently best algorithm for text compression High level: • Apply

Documents
Rationale for a 3D Heterogeneous Multi-core Processor€¦ · Li tt le Core uti li za ti on migration interval astar bzip2 gcc go h264ref hmmer lbm libquantum mcf milc namd omnetpp

Rationale for a 3D Heterogeneous Multi-core Processor€¦ · Li tt le Core uti li za ti on migration interval astar bzip2 gcc go h264ref hmmer lbm libquantum mcf milc namd omnetpp

Documents
Novell SUSE Linux Package Description and Support …support.novell.com/products/server/supported_packages/...busybox The Swiss Army Knife of Embedded Linux L3 bzip2 A program for

Novell SUSE Linux Package Description and Support …support.novell.com/products/server/supported_packages/...busybox The Swiss Army Knife of Embedded Linux L3 bzip2 A program for

Documents
index - Electrical and Computer Engineering - College of …omutlu/pub/mutlu_isca05_talk.pdf · 2008-12-16 · facerec fma3d galgel lucas mesa mgrid sixtrack swim wupwise AVG Increase

index - Electrical and Computer Engineering - College of …omutlu/pub/mutlu_isca05_talk.pdf · 2008-12-16 · facerec fma3d galgel lucas mesa mgrid sixtrack swim wupwise AVG Increase

Documents
The Compressed Permuterm Index - ISTI-CNRhpc.isti.cnr.it/~rossano/files/papers/TALG2010.pdf · 2014. 9. 10. · bzip2 and ppmdi. This way the compressed permuterm index offers a plethora

The Compressed Permuterm Index - ISTI-CNRhpc.isti.cnr.it/~rossano/files/papers/TALG2010.pdf · 2014. 9. 10. · bzip2 and ppmdi. This way the compressed permuterm index offers a plethora

Documents
pl hopl WA Equake 0508 - agents.unigard.com · Earthquake Section Revised - January, 2008 Washington Page 1 Limited Earthquake Index Page Section No. A. - C. Course of Construction

pl hopl WA Equake 0508 - agents.unigard.com · Earthquake Section Revised - January, 2008 Washington Page 1 Limited Earthquake Index Page Section No. A. - C. Course of Construction

Documents
Open-Source License and Copyright Information · bsdutils 2.29.2 GNU General Public License v2.0 or later BusyBox 1.29.3 GNU General Public License v2.0 only bzip2 1.0.6 Bzip2 License

Open-Source License and Copyright Information · bsdutils 2.29.2 GNU General Public License v2.0 or later BusyBox 1.29.3 GNU General Public License v2.0 only bzip2 1.0.6 Bzip2 License

Documents
Uncovering Hidden Loop Level Parallelism in …cccp.eecs.umich.edu/slides/mehrara-hpca08.pdf0.9 1 052.alvinn 056.ear 171.swim 172.mgrid 177.mesa 179.art 183.equake 188.ammp 008.espresso

Uncovering Hidden Loop Level Parallelism in …cccp.eecs.umich.edu/slides/mehrara-hpca08.pdf0.9 1 052.alvinn 056.ear 171.swim 172.mgrid 177.mesa 179.art 183.equake 188.ammp 008.espresso

Documents
Clam AntiVirus 0.99.4 User Manual€¦ · – 7Zip – ARJ (including SFX) – Tar – CPIO – Gzip – Bzip2 – DMG – IMG – ISO 9660. 1 Introduction 5 – PKG – HFS+ partition

Clam AntiVirus 0.99.4 User Manual€¦ · – 7Zip – ARJ (including SFX) – Tar – CPIO – Gzip – Bzip2 – DMG – IMG – ISO 9660. 1 Introduction 5 – PKG – HFS+ partition

Documents
IBM Research and U. Tennessee, Knoxville...facerec-2 ammp-2 lucas-2 fma3d-2 sixtrack-2 apsi-2 SPECJBB LINPACK p4max label Server Power (W) Label power: 308 W P4Max: 273 W Label power

IBM Research and U. Tennessee, Knoxville...facerec-2 ammp-2 lucas-2 fma3d-2 sixtrack-2 apsi-2 SPECJBB LINPACK p4max label Server Power (W) Label power: 308 W P4Max: 273 W Label power

Documents
Profile-Guided Post-Link Stride Prefetching · facerec. 2894 960. fma3d. 3046 460. galgel. 1248 535. lucas. 11261075. mesa. 4098 317. mgrid. 1934 623. sixtrack. 1349 655. swim. 1697

Profile-Guided Post-Link Stride Prefetching · facerec. 2894 960. fma3d. 3046 460. galgel. 1248 535. lucas. 11261075. mesa. 4098 317. mgrid. 1934 623. sixtrack. 1349 655. swim. 1697

Documents
Clam AntiVirus 0.98 User Manual · – Zip (including SFX) – RAR (including SFX) – 7Zip – ARJ (including SFX) – Tar – CPIO – Gzip – Bzip2 – MS OLE2 – MS Cabinet

Clam AntiVirus 0.98 User Manual · – Zip (including SFX) – RAR (including SFX) – 7Zip – ARJ (including SFX) – Tar – CPIO – Gzip – Bzip2 – MS OLE2 – MS Cabinet

Documents
Comparing Benchmarks Using Key Microarchitecture ...kehoste/ELIS/presentations/hoste06compari… · comparing bzip2 (SPEC CPU2000) with blast (BioInfoMark) hardware performance counters

Comparing Benchmarks Using Key Microarchitecture ...kehoste/ELIS/presentations/hoste06compari… · comparing bzip2 (SPEC CPU2000) with blast (BioInfoMark) hardware performance counters

Documents