Transcript Experimental Design, Response Surface Analysis, and

Experimental Design,
Response Surface
Analysis, and
Optimization
1
Outline











Motivation and Terminology
Difficulties in Solving the Basic Problem
Examples of Factors and Responses
Types/Examples of Experimental Design
Full Factorial Designs
Randomness of Effects
Example: Full Factorial Design
Situations with Many Factors
Response Surfaces and Metamodels
Regression Analysis
Response Surface Methodology
2
Motivation and Terminology






Useful when there are many alternatives to
consider (e.g., numerous capacity levels of
various types, numerous parameters for a
proposed inventory system)
Two basic types of variables: factors and
responses
Factors: input parameters:
-controllable or uncontrollable
-quantitative or qualitative
Responses: outputs from the simulation model:
-uncertain in nature
Basic problem: find the best levels (or values
of the parameters) in terms of the responses
Experimental Design can tell you which
alternatives to simulate so that you obtain the
desired information with the least amount of
simulation
3
Difficulties in Solving
the Basic Problem

Multiple Responses

Uncertain Responses
4
Examples of Factors
1. Mean interarrival time (uncontrollable,
2.
3.
4.
5.
6.
7.
quantitative)
Mean service time (controllable or
uncontrollable, quantitative)
Number of servers (controllable, quantitative)
Queuing discipline (controllable, qualitative)
Reorder point (controllable, quantitative)
Mean interdemand time (uncontrollable,
quantitative)
Distribution of interdemand time
(uncontrollable, qualitative)
5
Examples of Responses
1.
2.
3.
4.
Mean daily production rate
Mean time in the system for patients
Mean inventory level
Number of customers who wait for more than 5
minutes
6
Types/Examples of
Experimental Designs

Completely Randomized Designs

Randomized Complete Block Designs

Nested Factorial Designs

Split Plot Type Designs

Latin Square Type Designs

Full Factorial Designs

Fractional Factorial Designs
7
2k Factorial Designs

Suppose that we have k (k > 2) factors.
A 2k factorial design would require that
two levels be chosen for each factor,
and that n simulation runs (replications)
be made at each of the 2k possible
factor-level
combinations
(design
points). For 3 factors, this yields a
Design Matrix:
8
2k Factorial Designs
Design Matrix for 3 Factors
Factor 1 Factor 2 Factor 3
Design Point
Level
Level
Level
Response
1
O1
2
+
O2
3
+
O3
4
+
+
O4
5
+
O5
6
+
+
O6
7
+
+
O7
8
+
+
+
O8
“+” refers to one level of a factor and “-” refers to
the other level. Normally, for quantitative factors,
the smallest and largest levels for each factor are
chosen.
9
2k Factorial Designs:
Estimating Main Effects

The main effect of factor 1 is the change in the
response variable as a result of the change in
the level of the factor, averaged over all levels
of all of the other factors
e1 


O
2
 O1   O4  O3   O6  O5   O8  O7 
4
If the effect of some factor depends on the
level of another factor, these factors are said to
interact.
The degree of interaction (two-factor
interaction effect) between two factors i and j is
defined as half the difference between the
average effect of factor i when factor j is at its
“+” level ( and all factors other than i and j are
held constant) and the average effect of i when
j is at its “-” level; for example,
1  (O  O3 )  (O8  O7 ) (O2  O1 )  (O6  O5 ) 
e12   4


2
2
2
10
Example: Full Factorial (2k)
Design

Consider a simulation model of reorder point,
reorder quantity inventory system. The two
decision variables, or factors, to consider are
the reorder point (P) and the reorder quantity
(Q) for the inventory system. The maximum
and minimum allowable values for each are
given below:
Factor
P
Q

Minimum Maximum
Value
Value
20
40
15
50
Suppose that the response variable output by
the model is the long-run average monthly cost
(composed of three components: holding cost,
shortage costs, and ordering costs) in
thousands of dollars.
11
Example: Full Factorial (2k)
Design

A 22 factorial design matrix with simulation results
(for 10 replications at each design point) might be
given by:
Design Point Factor Level for
P
Q
1
2
+
3
+
4
+
+
Response
135.6
128.2
121.7
131.5
where a factor level of “-” indicates the minimum
possible value for that factor, and a factor level of
“+” indicates the maximum possible level; for
example, design point 2 has P=40 and Q=15.
12
Example: Full Factorial (2k)
Design

The response given is the average cost over
the 10 replications. Now, the main effects are
given by:
eQ 
(O3  O1 )  (O4  O2 ) (1217
.  1356
. )  (1315
.  128.2)

 53
.
2
2
(O2  O1 )  (O4  O3 ) (128.2  1356
. )  (1315
.  1217
.)
eP 

 12
.
2
2

The interaction effect (ePQ) is given by:
ePQ

135.6  128.2  1217
.  1315
.

 8.6
2
Therefore, the average effect of increasing P
from 20 to 40 is to increase monthly cost by
1.2, and the average effect of increasing Q
from 15 to 50 is to decrease monthly cost by
5.3. Hence, it would be advisable to set P as
low as possible and set Q as high as possible.
13
Example: Full Factorial (2k)
Design

Also since the interaction effect is positive,
it would seem advisable to set P and Q at
opposite levels. (Of course, all of the
above could be inferred from a cursory
analysis of the responses for the various
design points.) Note also that the literal
interpretation of main effects assumes no
interaction effects (pages 669 and 670 of
Law and Kelton, 1991).
14
Randomness of Effects

Note that the main and interaction effects
computed in the previous examples are
just random variables. To determine if the
effects are “significant” or real, and not
due to random fluctuations, one could
compute values for the main and
interaction effects 10 times (once for each
replication) and form confidence intervals
for each of the main effects, and the
interactions effect. If the confidence
interval contains 0, then the effect is not
statistically significant. (Note that
statistical significance does not
necessarily imply practical significance).
15
Situations with Many Factors



When there are many factors to consider,
full factorial, or even fractional factorial
designs may not be feasible from a
computational standpoint.
Other types of design (e.g., PlackettBurman designs or “supersaturated
designs”) may be appropriate (Mauro,
1986).
Another approach is to reduce the number
of factors to consider via “factor screening”
techniques, involving, for example, group
screening in which a whole group is
treated as a factor.
16
Response Surfaces and
Metamodels



A response surface is a graph of a
response variable as a function of the
various factors.
A metamodel (literally, model of the
simulation model), is an algebraic
representation of the simulation model,
with the factors as independent variables
and a response as the dependent
variable. It represents an approximation
of the response surface.
The typical metamodel used in a
simulation application is a regression
model.
17
Response Surfaces and
Metamodels


A metamodel through the use of response
surface methodology can be used to find
optimal values for a set of factors. It can
also be used to answer “what if” questions.
(Experimentation with a metamodel is
typically much less expensive than using a
simulation model directly).
An experimental design process assumes
a particular metamodel, e.g.,
E[ C( P, Q )]  B0  B1 P  B2 Q  B12 PQ
18
Basic Concepts of
Regression Analysis


Regression is used to determine the
“best” functional relation among
variables.
Suppose that the functional relationship
is represented as:
E(Y) = f (X1, ..., Xp / B1, ..., BE)
where E(Y) is the expected value of the
response variable Y; the
X1, ..., Xp
are factors; and the
B1, ..., BE are
function parameters; e.g.,
E(Y) = B1 + B2 X1 + B3 X2 + B4 X1 X2
19
Basic Concepts of
Regression Analysis

The observed value for Y, for a given
set of X ’s, is assumed to be a random
variable, given by:
Y = f (X1, ..., Xp/B1, ..., BE) +

Where  is a random variable with
2
mean equal to 0 and variance
.The
E
values for B1,...,BE are obtained by
minimizing the sum of squares of the
deviations.
20
Response Surface
Methodology
Source: (Fu, 1994)

Response surface methodology (RSM)
involves a combination of
metamodeling (i.e., regression) and
sequential procedures (iterative
optimization).
21
Response Surface
Methodology

RSM involves two phases:

Fit a linear regression model to some initial
data points in the search space (through
replications of the simulation model).
Estimate a steepest descent direction from
the linear regressions model, and a step
size to find a new (and better) solution in
the search space. Repeat this process
until the linear regression model becomes
inadequate (indicated by when the slope of
the linear response surface is
approximately 0; i.e., when the interaction

effects become larger than than the main
effects).
Fit a nonlinear quadratic regression
equation to this new area of the search
space. Then find the optimum of this
equation.
22
Terminology in Experimental
Design
Source:(Ostle, 1963)
 Replication - the repetition of the basic
experiment
 Treatment - a specific combination of several
factor levels
 Experimental Unit - the unit to which a single
treatment is applied to one replication of the
basic experiment
 Experimental Error - the failure of two
identically treated experimental units to yield
identical results
 Confounding - the “mixing up” of two or more
factors so that it’s impossible to separate the
effects
23
Terminology in Experimental
Design



Randomization - randomly assigning
treatments to experimental units (assures
independent distribution of errors)
Main Effect (of a factor) - a measure of the
change in a response variable to changes in
the level of the factor averaged over all levels
of all the other factors
Interaction is an additional effect (on the
response) due to the combined influence of
two or more factors
24