Transcript Simulation with Arena - Virginia Commonwealth University

Steady-State Statistical Analysis
By Dr. Jason Merrick
What We’ll Do ...
• Statistical analysis of steady-state simulations
–
–
–
–
Warm up and run length
Truncated replications
Batching in a single run
Automatic run-time confidence intervals via batch means
Simulation with Arena - Steady-state Output Analysis
C7/2
Statistical Analysis of SteadyState Simulations
• Recall:
Difference between terminating, steadystate simulations
– Which is appropriate depends on model, study
• Now, assume steady-state is desired
– Be sure this is so, since running and analysis is a lot harder
than for terminating simulations
• Naturally, simulation run lengths can be long
– Opportunity for different internal computation order
– Can change numerical results
– Underscores need for statistical analysis of output
Simulation with Arena - Steady-state Output Analysis
C7/3
Warm Up and Run Length
• Most models start empty and idle
–
–
–
–
Empty: No entities present at time 0
Idle: All resources idle at time 0
In a terminating simulation this is OK if realistic
In a steady-state simulation, though, this can bias the
output for a while after startup
• Bias can go either way
• Usually downward (results are biased low) in queueing-type models
•
that eventually get congested
Depending on model, parameters, and run length, the bias can be
very severe
Simulation with Arena - Steady-state Output Analysis
C7/4
Warm Up and Run Length
(cont’d.)
• Remedies for initialization bias
– Better starting state, more typical of steady state
• Throw some entities around the model
• Can be inconvenient to do this in the model
• How do you know how many to throw and where?
you’re trying to estimate in the first place.)
(This is what
– Make the run so long that bias is overwhelmed
• Might work if initial bias is weak or dissipates quickly
– Let model warm up, still starting empty and idle
• Simulate module: Warm-Up Period (time units!)
• “Clears” all statistics at that point for summary report, any cross-
replication data saved with Statistics module’s Outputs area (but not
Time-Persistent or Tallies)
Simulation with Arena - Steady-state Output Analysis
C7/5
Warm Up and Run Length
(cont’d.)
• Warm-up and run length times?
– Most practical idea: preliminary runs, plots
– Simply “eyeball” them
– Statistics module, Time-Persistent and Tallies areas, then
Plot with Output Analyzer
• Be careful about variability — make multiple
replications, superimpose plots
• Also, be careful to note “explosions”
• Model 5.1:
– Run for 1 day (=1440 minutes), 4 replications
– Save within-run Shipped parts flow time values
Simulation with Arena - Steady-state Output Analysis
C7/6
Warm Up and Run Length
•
•
•
•
Simulation with Arena - Steady-state Output Analysis
(cont’d.)
No explosions
All seem to be
settling into
steady state
Run length
seems
adequate to
reach steady
state
Hard to judge
warm-up ...
C7/7
Warm Up and Run Length
•
•
•
Simulation with Arena - Steady-state Output Analysis
(cont’d.)
“Crop” plots to
time 0 - 5,000
– Plot dialog,
“Display Time
from … to …”
Conservative
warm-up:
maybe 2,000
If measures
disagreed, use
max warm-up
C7/8
Truncated Replications
• If you can identify appropriate warm-up and runlength times, just make replications as for
terminating simulations
– Only difference: Specify Warm-Up Period in Simulate
module
– Proceed with confidence intervals, comparisons, all
statistical analysis as in terminating case
Simulation with Arena - Steady-state Output Analysis
C7/9
Collecting Truncated Replications
• Model 5.1:
–
–
–
–
–
Warm-Up Period = 6 hours (=360 minutes)
Run length 1800 minutes (1440 proper + 360 warm-up)
10 replications
Collect flowtimes for shipped, salvaged and scrapped parts
Statistics module, Outputs area entries to save summary
statistics (averages) across replications
Simulation with Arena - Steady-state Output Analysis
C7/10
Truncated Replications
•
•
•
•
(cont’d.)
Output Analyzer,
Classical Confidence
Intervals
Separate invocations
due to different units
Interpretation for steadystate expectations here
Want smaller?
– More reps, same length
– Longer reps, same number
of them
Simulation with Arena - Steady-state Output Analysis
C7/11
Batching in a Single Run
• If model warms up very slowly, truncated
replications can be costly
– Have to “pay” warm-up on each replication
• Alternative:
Just one R E A L L Y long run
– Only have to “pay” warm-up once
– Problem: Have only one “replication” and you need more
than that to form a variance estimate (the basic quantity
needed for statistical analysis)
• Big no-no:
•
Use the individual points within the run as “data” for
variance estimate
Usually correlated (not indep.), variance estimate biased
Simulation with Arena - Steady-state Output Analysis
C7/12
Batching in a Single Run
(cont’d.)
• Break each output record from the run into a few
large batches
– Tally (discrete-time) outputs: Observation-based
– Time-Persistent (continuous-time): Time-based
• Take averages over batches as “basic” statistics
for estimation: Batch means
– Tally outputs: Simple arithmetic averages
– Time-Persistent: Continuous-time averages
• Treat batch means as IID
– Key: batch size for low correlation (details in text)
– Still might want to truncate (once, time-based)
Simulation with Arena - Steady-state Output Analysis
C7/13
Batching in a Single Run
(cont’d.)
• Picture for WIP (time-persistent):
– For observation-based Tallies, just count points
• To batch and analyze (details in text):
– Statistics module, Time-Persistent, Tally areas to save
within-run records (could be big files)
– Output Analyzer, Analyze/Batch/Truncate or
• Warning if batches are too small for IID
– Get means .flt file; Classical C.I. as before
Simulation with Arena - Steady-state Output Analysis
C7/14
Collecting Batch Means
• Model 5.1:
– Warm-Up Period 6 hours (=360 minutes)
– Run length 14760 minutes
• 10 * 1440 proper + 360 warm-up
• Saves 9 warm-ups of 360 minutes from terminating version
– Collect flowtimes for shipped, salvaged and scrapped parts
– Statistics module, Outputs area entries to save tallies
through simulation run
– Output Analyzer, Correlogram for 500 lags
– Output Analyzer, Analyze/Batch/Truncate Obsn’s
• Truncate Time 360, Batch Observations try 150, 200, 250, …
• Save batch means to .flt file
Simulation with Arena - Steady-state Output Analysis
C7/15
Automatic Run-Time Confidence
Intervals via Batch Means
• Arena will automatically attempt to form 95%
confidence intervals on steady-state output
measures via batch means
–
–
–
–
“Half Width” column in summary output
Ignore if you’re doing a terminating simulation
Uses internal rules for batch sizes (details in text)
Won’t report anything if your run is not long enough
• “(Insuf)” if you don’t have the minimum amount of data Arena
•
requires even to form a c.i.
“(Correl)” if you don’t have enough data to form nearly-uncorrelated
batch means, required to be safe
Simulation with Arena - Steady-state Output Analysis
C7/16
Recommendations, Other Methods
for Steady-State Analysis
• What to do?
– Frankly, try to avoid steady-state simulations
• Look at goal of the study
– If you really do want steady-state
• First try warm-up and truncated replications
• Automatic run-time batch-means c.i.’s
• Batch-means c.i.’s “by hand” if necessary
• Other methods, goals
– Large literature on steady-state analysis
Simulation with Arena - Steady-state Output Analysis
C7/17