Transcript Model Verification and Validation

Simulation Modeling and
Analysis
Verification and
Validation
1
Outline
• Model Building, Verification and Validation
• Verification
• Calibration
2
Objectives of Verification and
Validation
• To produce a representative model of the
system under study
• To increase the model credibility
• To gradually refine the model during the
development process
3
Verification and Validation
• Verification
– Building the model right
• Validation
– Building the right model
• Verification and Validation must be
conducted simultaneously throughout the
model development process
4
Model Building, Verification and
Validation
Steps in Model Building
1.- Observe the real system
2.- Construct conceptual model and perform
conceptual validation
3.- Translate conceptual model into a computer
model and perform verification
4.- Calibrate, verify and validate at every step
5
Verification
• Verification checks that the computer model
accurately represent the conceptual model.
6
Verification Strategies
• Peer review
• Flow diagrams of all
possible actions
• Detailed output
examination
• Final check of input
parameters
• Make model selfdocumenting
• Use the Interactive
Run Controller
• Use the Graphical
User Interface
• Examine current
contents, total count
and traces
• Compare against
baselines
7
Validation and Calibration
• Validation compares the model to the real
system.
• Calibration adjusts the model to make it
more representative of the real system.
• Validation and Calibration must be
performed all the time and until the very last
minute.
8
Steps in Validation
1.- Build a model with high face validity.
2.- Validate model assumptions.
3.- Compare the model’s input-output
transformations against those in the real
system.
9
Face Validity
• Face validity is concerned with the
reasonableness of the model to
knowledgeable peers.
• Sensitivity analysis can help checking for
face validity.
10
Validating Model Assumptions
• Types of Assumptions
– Structural
– Data
• Structural assumptions must be checked
against the real system.
• Data assumptions must be checked by
statistical testing.
11
Validating Transformations
• The range of outputs of the model for a
given range of inputs must resemble the one
observed in the real system.
• Use historical data.
• Validate on the main response variables.
• What to do if the model represents a nonexisting system?
12