Transcript Model Risk - WordPress.com

Model Review & Model
Risk
[CHAPTER 8]
1
Sources
 Federal Reserve ?comptroller of the Currency
2011 Supervisory Guidance on Model Risk
Management
 Massimo Morini, Understanding and
Managing Model Risk (2011)
 Emanuel Derman, “Markets and Models”
(2001)
 Riccardo Rebonato, “Theory and Practice of
Model Risk Management” (2003)
2
How important is model risk?
 Models are just convenient mathematical
shorthand
 Felix Salmon: “Li’s Gaussian Copula model
will go down in history as instrumental in
causing the unfathomable losses that
brought the world financial system to its
knees.”
 Modeler quoted by Morini: “models were not
a problem . The problem was in the data and
the parameters! The problem was in the
3
Model risk evaluation and
control
 Scope of model review and control
 Roles and responsibilities
 Model verification
 Model validation
 Continuous review
 Periodic review
4
Model risk evaluation and
control
 Scope of model review and control
 Broad definition
 Methodology
 Purpose
 Vendor products
 Roles and responsibilities
 Model verification
 Model validation
 Continuous review
 Periodic review
5
Model risk evaluation and
control

Scope of model review and control
 Roles and responsibilities
 Business unit
 Independent reviewer as “second set of eyes”
 Effective challenge
 External resources
 Documentation standards
 Senior management

Model verification

Model validation

Continuous review

Periodic review
6
Model risk evaluation and
control

Scope of model review and control

Roles and responsibilities
 Model verification
 Independent implementation
 Degenerate cases and extreme inputs
 Graphs to build intuition
 Benchmarking against previous model
 Deal representation
 Approximations

Model validation

Continuous review

Periodic review
7
Model risk evaluation and
control
 Scope of model review and control
 Roles and responsibilities
 Model verification
 Model validation
 Continuous review
 Daily P&L reconciliation
 Back-testing
 Analysis of overrides
 Periodic review
8
Model risk evaluation and
control

Scope of model review and control

Roles and responsibilities

Model verification

Model validation

Continuous review
 Periodic review
 Changes in population of transactions
 Changes in market environment
 Cahnges in academic literature or market
practices
 Changes in technology
9
Model risk evaluation and
control

Scope of model review and control

Roles and responsibilities

Model verification
 Model validation
 Interpolation approach
 Cost of hedging approach
 Prevailing market model approach
 Matching model validation to model purpose
 Liquid Instruments
 Illiquid instruments
 Trading models

Continuous review

Periodic review
10
Position that has
illiquidity thrust
upon it
Liquid position in a
market that
experiences a
temporary bout of
illiquidity
Example: moderate
size position in a
liquid stock during a
market crash
Risk management:
measure risk by VaR
and stress test
scenarios
[Chapter 7]
Position that achieves Position in a liquid
illiquidity
asset that is so large
as to be illiquid
Example: LTCM’s
large positions in
liquid interest rate
swaps
Risk management:
extra risk beyond that
measured by standard
stress test should be
measured by
potential market
moves during
reasonable liquidation
period [Section 6.1.4]
Position that is born
illiquid
Example: Tranches of
subprime CDOs
Risk management:
liquid proxy + model
review to establish
required limits and
reserves [Sections
6.1.2 and 8.4]
Asset that never had
any liquidity
11
Model Risk

Liquid positions
 Models may be used for interpolation between liquid price quotes
 Models will be used for calculation of the impact of market changes on positions
in the VaR and stress test calculations
 These model uses are robust and easy to test, since they can constantly be
checked against actual liquid market quotes
 Only need to be concerned with current Greeks (you can always ask that positions
be reduced in future in reaction to changing Greeks)

Illiquid positions
 Model becomes critical and hard to validate
 Must employ as much liquid market data in the model as possible to avoid
unnecessary staleness of MTM
 Must clearly identify illiquid inputs and estimate liquidation risk through
conservative assumptions (relative to net exposure to illiquid inputs)
 Model Greeks are only useful in representing sensitivities to liquid market data;
1212they cannot be used in identifying the potential cost of being wrong about
the illiquid inputs (since your risk on the illiquid inputs is not to small hedgable
changes)
 Even for Greeks related to liquid market data, must be concerned about future
evolution, since you may not be able to change positions easily
4
The Li Model
• Is it fair to blame the Li model for the collapse of the
subprime mortgage CDO market?
– The major firms whose losses led to the largest part of the
crisis were all using more sophisticated models than the Li
model
– The Li model was used in very much the same way for
CDOs as the BSM model is successfully used for vanilla
options, with market-implied correlation skews playing a
parallel role to market-implied volatility surfaces
– The big difference is liquidity: the CDO market does not
possess somewhat liquid instruments with characteristics
that approximate the CDOs needing to be hedged
– The Li model could be used to reasonably accurately
compute the dependence of tranches on undiversifiable
risk and to produce stress test results
13
Options position risk

Because of the wide variety of options contract specifications, there
needs to be some underlying principle for integrating risk reporting




Puts and calls
Variety of dates that options expire
Variety of strikes at which options can be exercised
The Black-Scholes model is the key to managing and reporting options
position risk (see Chapter 11, particularly Section 11.1 and 11.7)




It is used to interpolate market prices for all options from reported prices for a
handful of options
It is used to identify a few key variables that impact the prices of all options
It is used to measure the change in options prices to changes in these key
variables
There are known flaws in the model but risk managers and traders have
developed procedures for dealing with them
14
Flaws in the Black-Scholes
formula
 Trivial flaws
 BS assumes that assets prices are log-normally
distributed when in fact they have fat tails
 Any distribution can be accommodated by a slightly
more complex model and closely approximated by BS
using different volatilities at different strike levels
 BS assumes a constant risk-free interest rate when in fact
rates vary by maturity date
 Variability of rates can be accommodated by basing
BS on the price of a forward contract on the asset
 BS assumes that hedging will take place continuously
 Experience and simulations show that reasonably
frequent hedging is very nearly as effective as
continuous hedging
15
Flaws in the Black-Scholes
formula
 Significant flaws concerning hedging
 BS assumes that hedging can take place without transaction
costs
 Large options trading desks only need to hedge net exposures
which holds transaction costs down to a very small part of
overall trading costs
 Other users of options can count on large options trading
desks to eliminate pricing discrepancies
 BS assumes that asset prices follow a smooth path with no
sudden jumps
 Risk reports need to be developed to show exposure to price
jumps
 By engaging in both buying and selling options, this exposure
can be kept under control
16
Flaws in the Black-Scholes
formula
 Significant flaws concerning volatility
 BS assumes that volatility is known in advance
 Risk reports need to be developed to show exposure to
volatility uncertainty
 By engaging in both buying and selling options, this exposure
can be kept under control
 BS assumes that volatility is constant when in fact volatility varies
by time period and price level
 Risk reports on volatility exposure need to show details of
exposure bucketed by tenor and strike
17
Option position reporting using
the Black-Scholes Greeks



The “Greeks” are the sensitivities of an option portfolio based on changes to the
inputs to the BS formula
Sensitivities that need to be consolidated with non-option positions
 Delta is the sensitivity to a change in asset price
 Delta is used to consolidate option exposures into position reports for the
asset
 Very important in making sure that undiversifiable exposure to stock indices
and government bond price levels is captured
 Rho is the sensitivity to changes in interest rates
 Rho is used to consolidate option exposures into interest rate position reports
 Two interest rate positions may be needed (e.g, Dollars and Euros on an FX
option)
Sensitivities that are unique to options
 Vega is the sensitivity to changes in volatility
 Exposure to a parallel shift in volatility is the most important risk exposure that
is unique to options
 Theta is sensitivity to changes in time to expiry
 Theta reports show exposure to time decay – getting closer to expiration of
the option with no further price moves
 Gamma is the sensitivity of delta to a change in asset prices
 Gamma reports are a first approximation to exposure to jumps in asset prices
18
Option position reporting that
goes beyond the Greeks
 Price-volatility matrix reporting
 Shows the exposure of an options book that is currently delta-
hedge (hedged against small changes in asset prices) to different
combinations of large changes in asset price and volatility
 Very important for getting a complete picture of exposure to
price jumps (look at Table 11.6 to see a portfolio with 0 gamma
and 0 vega that still has exposure to price jumps)
 Vega bucket reporting
 The principal exposure to volatility changes is picked up by the
vega measure
 Basis risks between options of different maturities requires
bucketing vega by time to expiry
 Basis risks between options of different strikes requires bucketing
vega by strike
19
Price-volatility matrix for
a call
20
Price-volatility matrix for a
call-spread (“risk reversal”)
21
Price-volatility matrix for
a calendar spread
22
How do we know that these fixes to
BSM can be used to control risk?
 Historical experience shows there have been no
major blow-ups due to market-making in reasonably
liquid vanilla options
 Simulations can be performed to show that while
risk has not been eliminated it has been reduced to
controllable levels (see section 11.3 for details)
 Adequate risk control requires frequent (but not
continuous) trading in a liquid underlying asset and
infrequent trading in a less liquid (but somewhat
liquid) set of options with strike and tenor
characteristics that reasonably approximate the
options being hedged
23
Dynamic hedging strategies
24
Dynamic hedging simulation
25
The Li Model
• Is it fair to blame the Li model for the collapse of the
subprime mortgage CDO market?
– The major firms whose losses led to the largest part of the
crisis were all using more sophisticated models than the Li
model
– The Li model was used in very much the same way for
CDOs as the BSM model is successfully used for vanilla
options, with market-implied correlation skews playing a
parallel role to market-implied volatility surfaces
– The big difference is liquidity: the CDO market does not
possess somewhat liquid instruments with characteristics
that approximate the CDOs needing to be hedged
– The Li model could be used to reasonably accurately
compute the dependence of tranches on undiversifiable
risk and to produce stress test results
26
Calculation of the Liquidation Margin for
Product with Sporadic Liquidity
 Need to build a model that relates pricing on this product




to products with full liquidity
Must change MTM continuously based on changes to
these related products, using the model
Estimate the degree of added liquidation risk caused by
the uncertainty of the modeled relationship
The key rule is that the illiquid product must show an
added degree of liquidation risk above that of the more
liquid proxy product
The model is relatively easy to check, since it can be
recalibrated every time a new sporadic quote becomes
available
27
Calculation of the Liquidation Margin for
Position of Illiquid Size
 Illiquidity of position size must be judged relative
to normal trading volume
 Can estimate liquidation risk by VaR-like price
moves over a time period in which orderly
liquidation can take place
 Alternatively, can model the market impact of
large volume
 In either case, this is an adjustment for added
cost of liquidation beyond the normal liquidation
costs of a smaller position
 Need to be concerned with the impact of
changing Greeks throughout the liquidation
period
28
The nature of actuarial risk
 Actuarial instrument risk can arise from
 Instruments with one-way market – instruments for which
almost all customer interest is on one side of the market
and ability to lay off risk is limited by all dealers having
similar positions (you can’t take comfort from price
quotes that can’t be acted on)
 Instruments that require extensive information disclosure
and negotiation to realize anything other than fire-sale
prices
 Instruments that can only be liquidated under restrictive
conditions
29
The nature of actuarial risk
 Actuarial risk requires a different approach from
management, traders and marketers than
market risk does
 Longer time horizon – willingness to wait until sale or




expiry to fully recognize gains
Willingness to live with larger degree and duration of
uncertainty of results
Capital structure that allows for patience
Different approach to assuring adequate capital
Actuarial risk is not for suitable for everyone!!!
30
Countering pro-cyclicality of
regulatory capital requirements

Pro-cyclicality can be eliminated by the ability to sell liquid assets and by
using a fixed target for non-liquid assets
 There are two ways to deal with the pro-cyclical problem of higher
demands for capital during economic downturns, when it is hardest
to raise capital:
 Positions can be liquidated
 Adequate capital can be raised prior to the economic downturn
 This implies that for liquid positions, capital can be tied to changes
from current mark-to-market. Even though this will result in higher
capital requirements during economic downturns, the ability to
liquidate positions makes this manageable.
 For illiquid positions, capital should not be tied to mark-to-market. In
an economic downturn, capital will rise severely because losses on
MTM cost capital on top of rising capital requirements tied to
increased volatility. The option of position liquidation is not available
to relieve this bind. [Sections 8.4.4 and 5.5.8.1]
31
Regulatory capital for illiquid
positions
 The alternative for illiquid positions is capital requirements tied to a
fixed stress-level (e.g., default levels three times historical
averages). This causes adequate capital to be raised when assets are
first booked. When an economic downturn occurs, the capital drain
of MTM losses is balanced by reduction of capital requirements,
since current MTM is closer to the fixed stress-level.
 Accounting standards should impact disclosure to shareholders and
lenders, not regulatory capital standards
 This is a natural outcome of the recommendation to tie required
capital to fixed stress-level losses.
 De-coupling regulatory capital requirements from earnings
reporting would ease regulatory and investor concerns that
building up reserves during good times is a way of manipulating
the reporting of earnings volatility
32
Using liquid proxies to estimate
illiquid instrument risk
 Use liquid proxies to represent these trades in computations designed
for the firm’s liquid positions, such as MTM, VaR and stress tests
 Avoids stale MTMs
 Measures risk concentrations and encourages diversification
 Every model is a potential liquid proxy representation; every liquid proxy
representation is a model subject to error
 Model difference between actual product and liquid proxy to create
conservative valuation assumptions
 Modeling of differences must go all the way to final payout and
must reflect the possibility that the model used for pricing and
trading the product may be wrong
 Modeling should be by Monte Carlo simulation to reflect a full
range of possible outcomes
 The liquid proxy is just a form of representation; it is not intended to
dictate hedging action to the trading desk
 Traders who believe that they have better hedging strategies should
be given sufficient limit room to act on these (plausible) beliefs and
should reap the resulting gains (and losses)
33
Estimating illiquid
instrument risk
 Derman: “Because of their illiquidity, many of these positions [in
long-term or exotic over-the-counter derivative securities that have
been designed to satisfy the risk preferences of their customers] will
be held for years. Despite their long-term nature, their daily values
affect the short-term profit and loss of the banks that trade them.”
 Rebonato: “What differentiates trading in opaque instruments from
other trading activities is the possibility that the bank might
accumulate a large volume of aggressively-marked opaque
instruments. When, eventually, the true market prices are
discovered, the book-value readjustment is sudden, and can be very
large. Stop-loss limits are ineffective to deal with this situation, since
the gates can only be shut once the horse has well and truly bolted.”
 Derman: “Derivative models work best when they use as their
constituents underlying securities that are one level simpler and one
level more liquid than the derivative itself.” (my emphasis)
34
Derman on estimating
illiquid instrument risk

“It’s never clear what profit and loss will result from hedging a derivative
security to its expiration. Markets will move in unexpected ways, sometimes
intensifying transactions costs and often dismantling what seemed a
reasonable hedging strategy. These effects are rarely captured by the
conventional models used in front-office valuation systems.”

“Therefore, for illiquid positions, it is important to estimate the adjustments
to conventional marked values that can occur as a result of long-term
hedging. One should build Monte Carlo models that simulate both underlyer
behavior and a trader’s hedging strategy to create distributions of the
resultant profit or loss of the whole portfolio. These distributions can be
used to determine a realistic adjustment to the trading desk’s conventional
marks that can be withheld until the trade is unwound and their realized
profit or loss determined…Monte Carlo analysis provides a good sense of
the variation in portfolio value that will be exhibited over the life of the trade
due to transactions costs, hedging error and model risk. Ultimately, such
analyses should be part of the desk’s own front-office valuation system.”
35
Rebonato on estimating
illiquid instrument risk
•
•
•
•
“Model risk is the risk of occurrence of a significant difference between the mark-tomodel value of a complex and/ or illiquid instrument, and the price at which the same
instrument is revealed to have traded in the market.”
“From the perspective of the risk manager the first and foremost task in model risk
management is the identification of the model (‘right’ or ‘wrong’ as it may be)
currently used by the market to arrive at traded prices.”
 “…market intelligence and contacts with the trader community at other institutions
are invaluable”
 Requires a variety of models to reverse-engineer observed prices
 Requires information about as many observed prices as possible
“The next most important task of the risk manager is to surmise how today’s accepted
pricing methodology might change in the future.” (including changes to model,
changes to calibration, and changes to numerical implementation).
 “Being aware of the latest market developments and of academic papers can be
very useful in guessing which direction the market might evolve tomorrow.”
“To a large extent, the model risk management task can be described as an
interpolation and extrapolation exercise that simply cannot be carried out in an
informational vacuum [without anchor points of solid knowledge about the levels and
nature of actual market transactions]”
36
Integrating Derman’s approach &
Rebonato’s approach
Derman emphasizes simulation of long-term hedging while Rebonato emphasizes
anticipating changes in pricing methodology due to model changes and calibration
changes. Are these contrasting approaches or two aspects of the same general
methodology?
 Simulation of long-term hedging is a way to calculate the financial impact of changes
in model calibration. Can it also handle changes in model specification?
Pro: Since you are simulating all the way through to final payout, model changes are
irrelevant
Con: “…the sudden occurrence of large-notional trades for which it is difficult to
establish a clear rationale on the basis of customer-driven demand” or a shift in
market structure (e.g., exit of a large dealer) could potentially force liquidation of a
portfolio prior to final payout, at which point the prevailing model will be very relevant.
 Can you calculate the impact of potential changes in pricing methodology without
simulation of long-term hedging?
 Maybe you could just reprice a portfolio with a range of potential models and
calibrations.
But since you are vulnerable to the impact of model and calibration changes not just
at the present time but over time, including changes in price levels, it’s hard to see
how this can be done without some form of long-term simulation.
37
Estimating Illiquid
Instrument Risk
 Model difference between actual product and liquid proxy to
create conservative valuation assumptions
 Modeling of differences must go all the way to final
payout and must reflect the possibility that the model
used for pricing and trading the product may be wrong
 Modeling should be by Monte Carlo simulation to reflect
a full range of possible outcomes
 Derman recommends a full simulation that includes both
underlying behavior and trader hedging strategy. This
represents an ideal that may sometimes be difficult to
achieve in practice.
 Simulation by assuming infrequent rehedging can make
use of more public information and can be easier to
implement, but at a cost of greater conservatism, since the
full range of trader hedging strategies will not be captured.
38
Liquid proxy example: variance
swaps [Section 12.1.1]
 Some non-liquid instruments can be so closely estimated by




liquid proxies that the residual risk is quite small
These instruments are sometimes called “quasi-vanillas”
The Breeden-Litzenberger theorem says that any option whose
payout is a smooth function of a terminal price can be perfectly
replicated by an (infinite) package of forwards and plain vanilla
calls and puts
In practice, a finite package can be used to replicate closely,
with degree of residual risk easily estimated
Variance swaps (swaps with payout based on the square of
realized volatility) are popular instruments, since they can be
used to express a vie on volatility that is independent of price
level
 Variance swaps can be shown to be precisely replicated by
delta hedging in forwards plus an option with payout based
on the logarithm of terminal price
39
Liquid proxy example: Illiquid
swap [Section 10.2.2]
 Suppose there is a liquid two-way market in a certain currency’s





swaps out to 7 years, but only a one-way market to pay 10 year
fixed
A full simulation of a hedging strategy could be quite complex.
An alternative is to use a 7-year swap as the liquid proxy for the
10-year swap and conservatively estimate the illiquid piece as the
cost of a one-time trade, 3 years from now, to reverse a 4-year
swap and put on a 7-year swap (this is called a “stack-and-roll”)
Every day for which public liquid quotes of 4 and 7 year swap
rates are available contributes an independent data point for
estimating this cost
If the liquid market only extends to 5 years, you would need two
trades: a 2 year swap into a 5 year swap in 3 years, followed by a 3
year swap into a 5 year swap after 2 more years
This approach generalizes to illiquid maturity options, though a
range of strikes needs to be used to offset the impact of price
changes prior to the one-time trade date and the full vol-surface
(skew as well as time) needs to be considered
18
Liquid proxy example: digital options
[Section 12.1.4]
•
•
•
•
•
•
•
•
•
Delta hedging of a digital option can be very dangerous, given the reality of
discrete hedging. When the market is near the strike close to expiry, the delta
hedge called for could result in very large losses
Full simulation of an optimal delta hedging strategy would be very difficult,
depending on hard-to-determine information about the liquidity of large
trades
An alternative is to use a call spread (selling a call at a strike lower than the
digital’s strike and buying a call at a strike above the digital’s strike) as a
proxy for selling a digital option
Estimate the illiquid piece as the difference between the potential payout on
the digital and payout of the call spread
Hypothetical size of the call spread depends on how wide the strikes are set –
the narrower the difference in strikes, the larger is the required size
The representation by the liquid proxy allows correct representation in risk
reports showing sensitivity to volatility level and volatility skew.
The closer the strikes for the call spread, the smaller the potential loss on the
illiquid piece.
Selection of the strikes for the call spread needs to be wide enough to make
for liquid trade size and manageable deltas and gammas as the trade evolves
Risk of the illiquid piece can be easily estimated by Monte Carlo simulation. If
you have a large book of digital options, negative impacts for some digitals
on a given Monte Carlo path may be offset by positive impacts for some other
digitals.
41
Liquid proxy example: barrier
options [Section 12.3.3]
• Suppose there is a one-way market for selling a down-and-out call
•
•
•
•
•
option (a call option that has zero payoff if the asset price goes below
a certain barrier any time during the life of the option)
A liquid proxy would be selling a European call option with the same
strike and expiry date as the barrier option along with the purchase of
European put options struck below the barrier
If the barrier is never struck, the puts expire worthless
Estimate the illiquid piece by the cost of reversing the European call
and put positions at the time the barrier is hit, relative to a set of
scenarios
• Each scenario would specify the time the barrier is hit, the level of
implied volatility at the time the barrier is hit, the volatility skew at
the time the barrier is hit, and the slope of the forward price curve
(the drift) at the time the barrier is hit
The representation by the liquid proxy allows correct representation
in risk reports showing sensitivity to volatility level and volatility skew.
This is similar to Peter Carr’s approach (and gives the same results
when volatility skew and drift are zero) but may utilize more expiry
dates for the puts and explicitly simulates hedge slippage costs for
varying volatility skew and drift
42
Liquid proxy example: barrier options
(continued)
• Every day for which public liquid quotes for the volatility surface of the
European options is available contributes an independent data point for
estimating the cost of the illiquid piece
• An approach in line with Derman’s recommendation is to pick the best
model you can for evolution of prices, incorporating stochastic volatility
and the possibility of jumps, and use Greeks to identify the liquid proxy.
 But it then requires much more work (Monte Carlo simulation of
recalibration of the Greeks) to calculate the potential cost of difference
between the barrier trade and the liquid proxy
 You can’t just utilize current Greeks of non-liquid inputs, such as jump
frequency, since these might not be stable
 Very important: must be consistent in liquid proxy choice and simulation
of cost of liquidity. If you are going to change the liquid proxy based on
model calibration, then you must use the more detailed simulation
method!
• This approach can easily be extended to other exotic options such as
double barriers, partial barriers, lookback options, and compound options
43
Liquid proxy example: illiquid CDO
tranche [Section 13.4]








Create a set of scenarios corresponding to different states of the economy,
i.e., correlation is enforced through a principal component.
Use a model that fully incorporates exact modeling of cash flows in a given
scenario.
Do a complete simulation to price each CDO tranche in each of these
scenarios, assuming a fully diversified portfolio (i.e., no residual correlation
beyond the principal component).
Probability weights can be assigned to scenarios based on observed prices of
the underlying basket and observed prices of liquid CDO tranches (if
available).
The model can be used to determine a good hedging portfolio, utilizing the
underlying basket and liquid CDO tranches . This hedging portfolio can be
used as a liquid proxy in risk reports.
Model risk can be measured by a Monte Carlo simulation of final payouts of
the illiquid CDO tranche plus the hedging portfolio, including the impact of
lack of full diversification.
Model risk must be judged by final payouts to maturity without any
assumption of ability to sell a CDO.
A full simulation would include periodic recalculation of the hedging
portfolio. A more conservative, but more easily computable, simulation
would assume the initial hedge is held throughout.
44
Derman & Wilmott on What
Makes a Good Model

Physics, because of its astonishing success at predicting the future behavior of material
objects from their present state, has inspired most financial modeling…Financial theory has
tried hard to emulate physics and discover its own elegant, universal laws. But finance and
economics are concerned with the human world of monetary value.

There are no fundamental laws in finance. And even if there were, there is no way to run
repeatable experiments to verify them. Financial theories written in mathematical
notation—aka models—imply a false sense of precision. Good modelers know that.

What makes a good model, then? We think the best ones use only a few variables and are
explicit about their assumptions. In this regard, we believe that the Black-Scholes model for
options valuation—now often maligned—is a model for models. It is clear and robust. Clear
because it is based on true engineering: It gives you a method for manufacturing an option
out of stocks and bonds, and it tells you what, under ideal circumstances, the option should
be worth…The world of markets doesn't exactly fulfill the ideal conditions Black-Scholes
requires. But the model allows an intelligent trader to see what real-world dirt has been
swept under the rug—and to adjust his or her risk estimates accordingly.

I will never sacrifice reality for elegance without explaining why I have done so. Nor will I
give the people who use my model false comfort about its accuracy. Instead, I will make
explicit its assumptions and oversights.
45
Derman on What Makes a Good
Model Review




One should…ask the following set of open ended questions:
 Does the model embody an accurate description of…the derivative’s payoffs?
 Does the model provide a realistic (or at least plausible) description of the factors that
affect a derivative’s value?...All models are simplifications…Is the assigned model an
appropriate simplification?
 Has the model been appropriately calibrated to the observed behavior, parameters and
prices of the simpler, liquid constituents that comprise the derivative?
 Is the software reliable?
Model documentation: Since software and models often have long lives…documentation
provide[s] a long-term corporate memory of the principles and implementation of the
model.
Comprehensive price verification: Nothing is better than a completely independent check
of price and hedge ratios. A strategist knowledgeable about the market, but
organizationally separate from the trading desk, should start with the confirmed trade
details and build an independent model…
Periodic comprehensive model review: …as markets mature and market participants gain
experience of the supply, demand and shocks that their underlyers and derivatives can
experience, prices often change character and start to reflect these realities more
accurately….It is therefore advisable to periodically revisit entire derivative markets and
their models…
46