Embed Size (px)
Citation preview
Performance Prediction of Multigrid-Solver Configurations
Performance Prediction of Multigrid-Solver Configurations
Norbert Siegmund
Harald Köstler
Alexander Grebhahn
Sven Apel
Performance Prediction of Multigrid-Solver Configurations
General Challenge: Finding the Optimal Configuration for a Given Hardware Platform?
2
cgs
preS
preS
preS postS
postS
postS
restriction à
prol
onga
tion à
Stencil code
Performance Prediction of Multigrid-Solver Configurations
General Challenge: Finding the Optimal Configuration for a Given Hardware Platform?
2
cgs
preS
preS
preS postS
postS
postS
restriction à
prol
onga
tion à
Stencil code
Performance Prediction of Multigrid-Solver Configurations
General Challenge: Finding the Optimal Configuration for a Given Hardware Platform?
2
cgs
preS
preS
preS postS
postS
postS
restriction à
prol
onga
tion à
Stencil code
Performance Prediction of Multigrid-Solver Configurations
What is the Influence of Configuration Options on Performance?
3
Performance Prediction of Multigrid-Solver Configurations
What is the Influence of Configuration Options on Performance?
3
Performance Prediction of Multigrid-Solver Configurations
What is the Influence of Configuration Options on Performance?
3
Binary options
Performance Prediction of Multigrid-Solver Configurations
What is the Influence of Configuration Options on Performance?
3
Binary options Numeric options
Performance Prediction of Multigrid-Solver Configurations
What is the Influence of Configuration Options on Performance?
3
Binary options Numeric options
Performance Prediction of Multigrid-Solver Configurations
What is the Influence of Configuration Options on Performance?
3
Binary options Numeric options
Performance Prediction of Multigrid-Solver Configurations
What is the Influence of Configuration Options on Performance?
3
Binary options Numeric options
Performance Prediction of Multigrid-Solver Configurations
What is the Influence of Configuration Options on Performance?
3
Binary options Numeric options
Perf
orm
ance
Performance Prediction of Multigrid-Solver Configurations
Vision: Performance-Influence Models
4
Performance Prediction of Multigrid-Solver Configurations
Vision: Performance-Influence Models
Determine the influence of configuration options and their interactions
4
Performance Prediction of Multigrid-Solver Configurations
Vision: Performance-Influence Models
Determine the influence of configuration options and their interactions
4
Performance Prediction of Multigrid-Solver Configurations
Vision: Performance-Influence Models
Determine the influence of configuration options and their interactions
4
Performance Prediction of Multigrid-Solver Configurations
Vision: Performance-Influence Models
Determine the influence of configuration options and their interactions
– Understanding
4
and use it for:
Performance Prediction of Multigrid-Solver Configurations
Vision: Performance-Influence Models
Determine the influence of configuration options and their interactions
– Understanding – Debugging
4
and use it for:
Performance Prediction of Multigrid-Solver Configurations
Vision: Performance-Influence Models
Determine the influence of configuration options and their interactions
– Understanding – Debugging – Prediction and optimization
4
14.3 + 113.3 - 4.5 - 73.9 + 171.7 = 219.9
and use it for:
Performance Prediction of Multigrid-Solver Configurations
Learning Procedure
5
Performance Prediction of Multigrid-Solver Configurations
Learning Procedure
5
Sampling
Performance Prediction of Multigrid-Solver Configurations
Learning Procedure
5
Sampling Learning
Performance Prediction of Multigrid-Solver Configurations
Learning Procedure
5
Sampling Learning Performance-Influence Model
Performance Prediction of Multigrid-Solver Configurations
Learning Procedure
5
Sampling Learning Performance-Influence Model
Performance Prediction of Multigrid-Solver Configurations
Sampling Binary and Numeric Options
Structured sampling approaches for the different kinds of options
6
Performance Prediction of Multigrid-Solver Configurations
Sampling Binary and Numeric Options
Binary Options
Structured sampling approaches for the different kinds of options
6
Performance Prediction of Multigrid-Solver Configurations
Sampling Binary and Numeric Options
Binary Options Numeric Options
Structured sampling approaches for the different kinds of options
6
Performance Prediction of Multigrid-Solver Configurations
Sampling Binary and Numeric Options
Binary Options Numeric Options
Structured sampling approaches for the different kinds of options
6
Exponential number!
Performance Prediction of Multigrid-Solver Configurations
Sampling Binary and Numeric Options
Binary Options Numeric Options
Structured sampling approaches for the different kinds of options
6
Exponential number!
Performance Prediction of Multigrid-Solver Configurations
Heuristics for Binary-Option Sampling
• Random? – Unlikely to select a valid configuration – Locally clustered solutions using SAT
7
Performance Prediction of Multigrid-Solver Configurations
Heuristics for Binary-Option Sampling
• Random? – Unlikely to select a valid configuration – Locally clustered solutions using SAT
• Heuristics
– Option-Wise (OW): { }, { }, { }, { }, {}
7
Siegmund et al., ICSE‘12
Performance Prediction of Multigrid-Solver Configurations
Heuristics for Binary-Option Sampling
• Random? – Unlikely to select a valid configuration – Locally clustered solutions using SAT
• Heuristics
– Option-Wise (OW): { }, { }, { }, { }, {} – Negative Option-Wise (nOW): { }, { }, { }, { }, { }
7
Siegmund et al., ICSE‘12
Performance Prediction of Multigrid-Solver Configurations
Heuristics for Binary-Option Sampling
• Random? – Unlikely to select a valid configuration – Locally clustered solutions using SAT
• Heuristics
– Option-Wise (OW): { }, { }, { }, { }, {} – Negative Option-Wise (nOW): { }, { }, { }, { }, { } – Pair-Wise (PW) : { }, { }, { }, { }, { },
{ } 7
Siegmund et al., ICSE‘12
Performance Prediction of Multigrid-Solver Configurations
Numeric-Option Sampling (Experimental Designs)
• Fractional factorial designs • Optimal designs
8
Performance Prediction of Multigrid-Solver Configurations
Numeric-Option Sampling (Experimental Designs)
• Fractional factorial designs • Optimal designs
• Pre-study:
8
cgs
preS
preS
preS postS
postS
postS
restriction à
prol
onga
tion à
Multi-grid solver as subject systems
Performance Prediction of Multigrid-Solver Configurations
Numeric-Option Sampling (Experimental Designs)
• Fractional factorial designs • Optimal designs
• Pre-study:
Plackett-Burman Design as best design
8
cgs
preS
preS
preS postS
postS
postS
restriction à
prol
onga
tion à
Multi-grid solver as subject systems
Performance Prediction of Multigrid-Solver Configurations
Plackett-Burman Design (PBD)
• Minimizes the variance of the estimates of the independent variables (numeric options)
• …while using a limited number of measurements
9
Performance Prediction of Multigrid-Solver Configurations
Plackett-Burman Design (PBD)
• Minimizes the variance of the estimates of the independent variables (numeric options)
• …while using a limited number of measurements • Design specifies seeds depending on the number of experiments to
be conducted (i.e., configurations to be measured)
9
Numeric options
Conf
igur
atio
ns
Performance Prediction of Multigrid-Solver Configurations
Plackett-Burman Design (PBD)
• Minimizes the variance of the estimates of the independent variables (numeric options)
• …while using a limited number of measurements • Design specifies seeds depending on the number of experiments to
be conducted (i.e., configurations to be measured)
9
Numeric options
Conf
igur
atio
ns
Min Center Max
Value range of a numeric option
Performance Prediction of Multigrid-Solver Configurations
Plackett-Burman Design (PBD)
• Minimizes the variance of the estimates of the independent variables (numeric options)
• …while using a limited number of measurements • Design specifies seeds depending on the number of experiments to
be conducted (i.e., configurations to be measured)
9
Numeric options
Conf
igur
atio
ns
Min Center Max
Value range of a numeric option
Performance Prediction of Multigrid-Solver Configurations
Plackett-Burman Design (PBD)
• Minimizes the variance of the estimates of the independent variables (numeric options)
• …while using a limited number of measurements • Design specifies seeds depending on the number of experiments to
be conducted (i.e., configurations to be measured)
9
Numeric options
Conf
igur
atio
ns
Min Center Max
Value range of a numeric option
Performance Prediction of Multigrid-Solver Configurations
Learning Procedure
10
Sampling Learning Performance-Influence Model
Performance Prediction of Multigrid-Solver Configurations
Learning Procedure
10
Sampling Learning Performance-Influence Model
Performance Prediction of Multigrid-Solver Configurations
20
50
100
50
20
100
100
0
16
32
32
32
0
16
0
1
1
0
1
0
1
0
1
0
1
1
0
1
Regression Learning
11
1
0
1
1
1
0
1
Individual Options
Conf
igur
atio
ns
Performance Prediction of Multigrid-Solver Configurations
20
50
100
50
20
100
100
0
16
32
32
32
0
16
0
1
1
0
1
0
1
0
1
0
1
1
0
1
Regression Learning
11
1
0
1
1
1
0
1
Perf.
833
411
290
799
753
514
416
=
Individual Options
Conf
igur
atio
ns
Performance Prediction of Multigrid-Solver Configurations
20
50
100
50
20
100
100
0
16
32
32
32
0
16
0
1
1
0
1
0
1
0
1
0
1
1
0
1
Regression Learning
11
1
0
1
1
1
0
1
Perf.
833
411
290
799
753
514
416
=
β1 β2 β3 β4 β5
Individual Options
Conf
igur
atio
ns
* * * * *
Performance Prediction of Multigrid-Solver Configurations
20
50
100
50
20
100
100
0
16
32
32
32
0
16
0
1
1
0
1
0
1
0
1
0
1
1
0
1
Regression Learning
11
1
0
1
1
1
0
1
Perf.
833
411
290
799
753
514
416
=
β1 β2 β3 β4 β5
Individual Options
102.4 84.3 54.1 5.4 1.3
Error:
41%
Conf
igur
atio
ns
* * * * *
Performance Prediction of Multigrid-Solver Configurations
0
800
0 1600
640
0 1600
20
50
100
50
20
100
100
0
16
32
32
32
0
16
0
0
0
1
1
0
1
0
0
0
0
1
0
1
0
1
1
0
1
0
1
0
1
0
1
1
0
1
Regression Learning
11
1
0
1
1
1
0
1
Perf.
833
411
290
799
753
514
416
=
β1 β2 β3 β4 β5 β6 β7 β8
Individual Options Interactions
102.4 84.3 54.1 5.4 1.3
Error:
41%
Conf
igur
atio
ns
* * * * * * * *
Performance Prediction of Multigrid-Solver Configurations
0
800
0 1600
640
0 1600
20
50
100
50
20
100
100
0
16
32
32
32
0
16
0
0
0
1
1
0
1
0
0
0
0
1
0
1
0
1
1
0
1
0
1
0
1
0
1
1
0
1
Regression Learning
11
1
0
1
1
1
0
1
Perf.
833
411
290
799
753
514
416
=
β1 β2 β3 β4 β5 β6 β7 β8
Individual Options Interactions
102.4 84.3 54.1 5.4 1.3
132.3 81.3 56.6 3.5 1.9 -14.1 -5.4 2.4
Error:
41%
25%
Conf
igur
atio
ns
* * * * * * * *
Performance Prediction of Multigrid-Solver Configurations
0
800
0 1600
640
0 1600
20
50
100
50
20
100
100
0
16
32
32
32
0
16
400 2500
10000
2500
400 10000
10000
0
0
0
1
1
0
1
0
0
0
0
1
0
1
0
1
1
0
1
0
1
0
1
0
1
1
0
1
Regression Learning
11
1
0
1
1
1
0
1
N/A
1.2
0.9
1.5
1.5
N/A
1.2
2 Log( ) Perf.
833
411
290
799
753
514
416
=
β1 β2 β3 β4 β5 β6 β7 β8 β9 β10
Individual Options Interactions Functions
102.4 84.3 54.1 5.4 1.3
132.3 81.3 56.6 3.5 1.9 -14.1 -5.4 2.4
Error:
41%
25%
Conf
igur
atio
ns
* * * * * * * * * *
Performance Prediction of Multigrid-Solver Configurations
0
800
0 1600
640
0 1600
20
50
100
50
20
100
100
0
16
32
32
32
0
16
400 2500
10000
2500
400 10000
10000
0
0
0
1
1
0
1
0
0
0
0
1
0
1
0
1
1
0
1
0
1
0
1
0
1
1
0
1
Regression Learning
11
1
0
1
1
1
0
1
N/A
1.2
0.9
1.5
1.5
N/A
1.2
2 Log( ) Perf.
833
411
290
799
753
514
416
=
β1 β2 β3 β4 β5 β6 β7 β8 β9 β10
Individual Options Interactions Functions
102.4 84.3 54.1 5.4 1.3
132.3 81.3 56.6 3.5 1.9 -14.1 -5.4 2.4
130.3 83.5 54.2 0.01 0 -14.1 -5.4 1.4 2.1 8.8
Error:
41%
25%
7.4%
Conf
igur
atio
ns
* * * * * * * * * *
Performance Prediction of Multigrid-Solver Configurations
0
800
0 1600
640
0 1600
20
50
100
50
20
100
100
0
16
32
32
32
0
16
400 2500
10000
2500
400 10000
10000
0
0
0
1
1
0
1
0
0
0
0
1
0
1
0
1
1
0
1
0
1
0
1
0
1
1
0
1
Regression Learning
11
1
0
1
1
1
0
1
N/A
1.2
0.9
1.5
1.5
N/A
1.2
2 Log( ) Perf.
833
411
290
799
753
514
416
=
β1 β2 β3 β4 β5 β6 β7 β8 β9 β10
Individual Options Interactions Functions
102.4 84.3 54.1 5.4 1.3
132.3 81.3 56.6 3.5 1.9 -14.1 -5.4 2.4
130.3 83.5 54.2 0.01 0 -14.1 -5.4 1.4 2.1 8.8
Error:
41%
25%
7.4%
Conf
igur
atio
ns
* * * * * * * * * *
Exponential number!
Performance Prediction of Multigrid-Solver Configurations
0
800
0 1600
640
0 1600
20
50
100
50
20
100
100
0
16
32
32
32
0
16
400 2500
10000
2500
400 10000
10000
0
0
0
1
1
0
1
0
0
0
0
1
0
1
0
1
1
0
1
0
1
0
1
0
1
1
0
1
Regression Learning
11
1
0
1
1
1
0
1
N/A
1.2
0.9
1.5
1.5
N/A
1.2
2 Log( ) Perf.
833
411
290
799
753
514
416
=
β1 β2 β3 β4 β5 β6 β7 β8 β9 β10
Individual Options Interactions Functions
102.4 84.3 54.1 5.4 1.3
132.3 81.3 56.6 3.5 1.9 -14.1 -5.4 2.4
130.3 83.5 54.2 0.01 0 -14.1 -5.4 1.4 2.1 8.8
Error:
41%
25%
7.4%
Conf
igur
atio
ns
* * * * * * * * * *
Exponential number! Unlimited candidates!
Performance Prediction of Multigrid-Solver Configurations 12
Multiple Regression with Feature Subset Selection
Compression Encryption CacheSize PageSize
Performance Prediction of Multigrid-Solver Configurations 12
Multiple Regression with Feature Subset Selection
Compression Encryption CacheSize PageSize
Candidates: Models: Errors: Winner:
1
Performance Prediction of Multigrid-Solver Configurations 12
Multiple Regression with Feature Subset Selection
Compression Encryption CacheSize PageSize
Candidates: Models: Errors: Winner:
1
Performance Prediction of Multigrid-Solver Configurations 12
Multiple Regression with Feature Subset Selection
Compression Encryption CacheSize PageSize
Candidates: Models: Errors: Winner:
1 β0+ * β1
β0+ * β1
β0+ * β1
β0+ * β1
Performance Prediction of Multigrid-Solver Configurations 12
Multiple Regression with Feature Subset Selection
Compression Encryption CacheSize PageSize
Candidates: Models: Errors: Winner:
1 β0+ * β1 50%
125%
72%
35%
β0+ * β1
β0+ * β1
β0+ * β1
Performance Prediction of Multigrid-Solver Configurations 12
Multiple Regression with Feature Subset Selection
Compression Encryption CacheSize PageSize
Candidates: Models: Errors: Winner:
1 β0+ * β1 50%
125%
72%
35% 2 2
log( )
β0+ * β1
β0+ * β1
β0+ * β1
β0+ * β1
β0+ log( ) * β1
Performance Prediction of Multigrid-Solver Configurations 12
Multiple Regression with Feature Subset Selection
Compression Encryption CacheSize PageSize
Candidates: Models: Errors: Winner:
1 β0+ * β1 50%
125%
72%
35% 2 2
log( )
29%
34%
β0+ * β1
β0+ * β1
β0+ * β1
β0+ * β1
β0+ log( ) * β1
Performance Prediction of Multigrid-Solver Configurations 12
Multiple Regression with Feature Subset Selection
Compression Encryption CacheSize PageSize
Candidates: Models: Errors: Winner:
1 β0+ * β1 50%
125%
72%
35% 2 2
log( )
29%
34%
β0+ * β1
β0+ * β1
β0+ * β1
β0+ * β1
β0+ log( ) * β1
Performance Prediction of Multigrid-Solver Configurations 12
Multiple Regression with Feature Subset Selection
Compression Encryption CacheSize PageSize
Candidates: Models: Errors: Winner:
1 β0+ * β1 50%
125%
72%
35% 2 2
log( )
29%
34%
β0 + * β1 2
β0+ * β1
β0+ * β1
β0+ * β1
β0+ * β1
β0+ log( ) * β1
Performance Prediction of Multigrid-Solver Configurations 12
Multiple Regression with Feature Subset Selection
Compression Encryption CacheSize PageSize
Candidates: Models: Errors: Winner:
1 β0+ * β1 50%
125%
72%
35% 2 2
log( )
29%
34%
β0 + * β1 2
2
…
log( )
β0+ * β1
β0+ * β1
β0+ * β1
β0+ * β1
β0+ log( ) * β1
Performance Prediction of Multigrid-Solver Configurations 12
Multiple Regression with Feature Subset Selection
Compression Encryption CacheSize PageSize
Candidates: Models: Errors: Winner:
1 β0+ * β1 50%
125%
72%
35% 2 2
log( )
29%
34%
β0 + * β1 2
2
…
* β2
…
log( )
β0 + * β1 + 2
* β2 β0 + * β1 + 2
* β2 β0 + * β1 + 2
* β2 β0 + * β1 + 2
* β2 β0 + * β1 +log( ) 2
β0+ * β1
β0+ * β1
β0+ * β1
β0+ * β1
β0+ log( ) * β1
Performance Prediction of Multigrid-Solver Configurations 12
Multiple Regression with Feature Subset Selection
Compression Encryption CacheSize PageSize
Candidates: Models: Errors: Winner:
1 β0+ * β1 50%
125%
72%
35% 2 2
log( )
29%
34%
β0 + * β1 2
2
…
* β2 5% …
12%
18%
27%
… …
log( )
β0 + * β1 + 2
* β2 β0 + * β1 + 2
* β2 β0 + * β1 + 2
* β2 β0 + * β1 + 2
* β2 β0 + * β1 +log( ) 2
β0+ * β1
β0+ * β1
β0+ * β1
β0+ * β1
β0+ log( ) * β1
Performance Prediction of Multigrid-Solver Configurations 12
Multiple Regression with Feature Subset Selection
Compression Encryption CacheSize PageSize
Candidates: Models: Errors: Winner:
1 β0+ * β1 50%
125%
72%
35% 2 2
log( )
29%
34%
β0 + * β1 2
2
…
* β2 5% …
12%
18%
27%
… …
2
log( )
β0 + * β1 + 2
* β2 β0 + * β1 + 2
* β2 β0 + * β1 + 2
* β2 β0 + * β1 + 2
* β2 β0 + * β1 +log( ) 2
β0+ * β1
β0+ * β1
β0+ * β1
β0+ * β1
β0+ log( ) * β1
β0 + * β1 + * β2
Performance Prediction of Multigrid-Solver Configurations
Learning Procedure
13
Sampling Learning Performance-Influence Model
Performance Prediction of Multigrid-Solver Configurations
Learning Procedure
13
Sampling Learning Performance-Influence Model
Performance Prediction of Multigrid-Solver Configurations
1. Experiment: Leveraging Domain Knowledge
• Test system: HSMGP
– Different smoothers, coarse-grid solvers, # pre-smoothing, # post-smoothing, # cores
14
Performance Prediction of Multigrid-Solver Configurations
1. Experiment: Leveraging Domain Knowledge
• Test system: HSMGP
– Different smoothers, coarse-grid solvers, # pre-smoothing, # post-smoothing, # cores
14
Performance Prediction of Multigrid-Solver Configurations
1. Experiment: Leveraging Domain Knowledge
15
• Assumptions: 1. The runtime of the coarse-grid solver is not affected
by the selection of the smoother and the number of pre-smoothing and post-smoothing steps.
2. Pre-smoothing sweeps and post-smoothing sweeps needs the same time.
Performance Prediction of Multigrid-Solver Configurations
1. Experiment: Leveraging Domain Knowledge
15
• Assumptions: 1. The runtime of the coarse-grid solver is not affected
by the selection of the smoother and the number of pre-smoothing and post-smoothing steps.
2. Pre-smoothing sweeps and post-smoothing sweeps needs the same time.
Performance Prediction of Multigrid-Solver Configurations
1. Experiment: Leveraging Domain Knowledge
15
• Assumptions: 1. The runtime of the coarse-grid solver is not affected
by the selection of the smoother and the number of pre-smoothing and post-smoothing steps.
2. Pre-smoothing sweeps and post-smoothing sweeps needs the same time.
Performance Prediction of Multigrid-Solver Configurations
1. Experiment: Leveraging Domain Knowledge
15
• Assumptions: 1. The runtime of the coarse-grid solver is not affected
by the selection of the smoother and the number of pre-smoothing and post-smoothing steps.
2. Pre-smoothing sweeps and post-smoothing sweeps needs the same time.
Performance Prediction of Multigrid-Solver Configurations
1. Experiment: Leveraging Domain Knowledge
15
• Assumptions: 1. The runtime of the coarse-grid solver is not affected
by the selection of the smoother and the number of pre-smoothing and post-smoothing steps.
2. Pre-smoothing sweeps and post-smoothing sweeps needs the same time.
Performance Prediction of Multigrid-Solver Configurations
2. Experiment: Code Generator
• 20 configuration options from different domains (hardware-specific, multigrid-specific, optimizations)
• Global Model vs. Local Model
16
Performance Prediction of Multigrid-Solver Configurations
2. Experiment: Code Generator
• 20 configuration options from different domains (hardware-specific, multigrid-specific, optimizations)
• Global Model vs. Local Model
16
Performance Prediction of Multigrid-Solver Configurations
2. Experiment: Code Generator
• 20 configuration options from different domains (hardware-specific, multigrid-specific, optimizations)
• Global Model vs. Local Model
16
Performance Prediction of Multigrid-Solver Configurations
Findings and Future Work
17
Performance Prediction of Multigrid-Solver Configurations
Findings and Future Work
17
Performance Prediction of Multigrid-Solver Configurations
Findings and Future Work
17
Performance Prediction of Multigrid-Solver Configurations
Findings and Future Work
17
Performance Prediction of Multigrid-Solver Configurations
Findings and Future Work
17
https://github.com/nsiegmun/SPLConqueror http://fosd.de/SPLConqueror/
Performance Prediction of Multigrid-Solver Configurations
Findings and Future Work
17
https://github.com/nsiegmun/SPLConqueror http://fosd.de/SPLConqueror/
Thank You!
