1

Simulation Scenarios

2

Computer Based Experiments

• Systematically planning and conducting scientific studies that change experimental variables together in order to determine their effect of a given response.

• Controlled changes to input variables in order to gain maximum amounts of information on cause and effect relationships.

3

BASIC STEPS

1. Design the scenario, 2. Collect data, 3. Simulation and analysis, and4. The conclusions reached and recommendations made as a result of the scenario.

4

TERMINOLOGY

• Replication – repetition of a basic experiment without changing any factor settings, allows the experimenter to estimate the experimental error (noise) in the system used to determine whether observed differences in the data are “real” or “just noise”, allows the experimenter to obtain more statistical power (ability to identify small effects)

5

TERMINOLOGY

• Factors – experimental factors or independent variables (continuous or discrete) an investigator manipulates to capture any changes in the output of the process. Other factors of concern are those that are uncontrollable and those which are controllable but held constant during the experimental runs.

6

TERMINOLOGY

• Responses – dependent variable measured to describe the output of the process.

• Treatment Combinations (run) – experimental trial where all factors are set at a specified level.

7

TERMINOLOGY

• Fixed Effects Model - If the treatment levels are specifically chosen by the experimenter, then conclusions reached will only apply to those levels.

• Random Effects Model – If the treatment levels are randomly chosen from a population of many possible treatment levels, then conclusions reached can be extended to all treatment levels in the population.

8

PLANNING SCENARIO

• Everyone involved in the computer based experiment should have a clear idea in advance of exactly what is to be studied, the objectives of the simulation experiment, the questions one hopes to answer and the results anticipated

9

PLANNING SCENARIO

• Select a response/dependent variable (variables) that will provide information about the problem under study and the proposed measurement method for this response variable, including an understanding of the measurement system variability

10

PLANNING SCENARIO

• Select the independent variables/factors (quantitative or qualitative) to be investigated in the experiment, the number of levels for each factor, and the levels of each factor chosen either specifically (fixed effects model) or randomly (random effects model).

11

FACTORIAL (2k) DESIGNS

• Experiments involving several factors ( k = # of factors) where it is necessary to study the joint effect of these factors on a specific response.

• Each of the factors are set at two levels (a “low” level and a “high” level) which may be qualitative (machine A/machine B, fan on/fan off) or quantitative (temperature 800/temperature 900, line speed 4000 per hour/line speed 5000 per hour).

12

FACTORIAL (2k) DESIGNS

• Factors are assumed to be fixed (fixed effects model)

• Designs are completely randomized (experimental trials are run in a random order, etc.)

• The usual normality assumptions are satisfied.

13

FACTORIAL (2k) DESIGNS

• Particularly useful in the early stages of experimental work when you are likely to have many factors being investigated and you want to minimize the number of treatment combinations (sample size) but, at the same time, study all k factors in a complete factorial arrangement (the experiment collects data at all possible combinations of factor levels).

14

FACTORIAL (2k) DESIGNS

• As k gets large, the sample size will increase exponentially. If experiment is replicated, the # runs again increases.

k # of runs2 43 84 165 326 647 1288 2569 512

10 1024

15

FACTORIAL (2k) DESIGNS (k = 2)• Two factors set at two levels (normally

referred to as low and high) would result in the following design where each level of factor A is paired with each level of factor B.

RUN Factor A Factor B RESPONSE RUN Factor A Factor B RESPONSE1 low low y1 1 -1 -1 y1

2 high low y2 2 +1 -1 y2

3 low high y3 3 -1 +1 y3

4 high high y4 4 +1 +1 y4

Generalized Settings Orthogonal Settings

16

FACTORIAL (2k) DESIGNS (k = 2)

• Estimating main effects associated with changing the level of each factor from low to high. This is the estimated effect on the response variable associated with changing factor A or B from their low to high values.

2)(

2)( 3142 yyyyEffectAFactor

2)(

2)( 2143 yyyyEffectBFactor

17

FACTORIAL (2k) DESIGNS (k = 2): GRAPHICAL OUTPUT

• Neither factor A nor Factor B have an effect on the response variable.

18

FACTORIAL (2k) DESIGNS (k = 2): GRAPHICAL OUTPUT

• Factor A has an effect on the response variable, but Factor B does not.

19

FACTORIAL (2k) DESIGNS (k = 2): GRAPHICAL OUTPUT

• Factor A and Factor B have an effect on the response variable.

20

FACTORIAL (2k) DESIGNS (k = 2): GRAPHICAL OUTPUT

• Factor B has an effect on the response variable, but only if factor A is set at the “High” level. This is called interaction and it basically means that the effect one factor has on a response is dependent on the level you set other factors at. Interactions can be major problems in a DOE if you fail to account for the interaction when designing your experiment.

DISCUSSION

21