Reliance on models to price, trade, and manage risks carries risk. Models are susceptible to errors.
In liquid and more or less efficient securities markets, the market price is, on average, the best indicator of the asset’s value. In the absence of liquid markets and price discovery mechanisms, there is no alternative than theoretical valuation models to mark-to-model the position, to assess the risk exposure along the various risk factors, and to derive the appropriate hedging strategy.
The pace of model development has accelerated to support the rapid growth of financial innovations such as caps, floors, and others. These have been made possible because of developments in theory, which allow a better capture of financial risks. At the same time, these models would have never been implemented in practice, had the growth in computing power not accelerated so dramatically. The main causes of model risk are model error and implementing a model wrongly.
Model error
A model is incorrect if there are mistakes in the analytical solution. A model is also incorrect if it is based on wrong assumptions about the underlying asset price process. Finance is littered with examples of trading strategies based on shaky assumptions – some model risks are really just a formalisation of this.
The most frequent error in model building is to assume that the distribution of the underlying asset is stationary, when it changes. The ideal solution would be to acknowledge that volatility is stochastic and to develop an option-pricing model, but option-valuation models become difficult when any sort of stochastic volatility is included.
Implementing a model wrongly
With models that require extensive programming, there is always a chance that a bug may affect output. Some implementations rely on techniques that exhibit inherent errors and limited validity. Many programmes that seem error-free have been tested only under normal conditions.
For models evaluating complex derivatives, data are collected from different sources. The implicit assumption is that for each period, the data for all relevant assets and rates pertain to exactly the same instant, and thus reflect simultaneous prices. Using nonsimultaneous price inputs may be but can lead to wrong pricing.
When implementing a pricing model, tools are used to estimate parameters. But how frequently should input parameters be refreshed? Should the adjustment be made on a periodic basis, or should it be triggered by an event? Should parameters be adjusted according to qualitative judgements, or should these be based on statistics?
Statistical estimators are subject to errors involving inputs. A major problem in the estimation procedure is the treatment of outliers. Are the outliers really outliers, in the sense that they do not reflect the true distribution? Or are they important observations that should not be dismissed? The results of the estimation procedure will be vastly different depending on how observations are treated. Each bank may use a different procedure to estimate parameters. Some may use daily closing prices, while others may use transaction data.
The quality of a model depends on the accuracy of inputs and parameter values. This is particularly true in the case of new markets, where best-practice procedures and controls are evolving. Volatilities and correlations are the hardest parameters to judge.
Most institutions use internal data as well as external databases. The responsibility for accuracy is often not clearly assigned.
Adding observations improves the power of tests and tends to reduce errors; but the longer the sampling perio, the more weight is given to obsolete information.
The gap between the bid and ask prices may be large enough to complicate the process of finding a single value. Choices made about the price data at the time of data selection can have a major impact on the model.
How can we mitigate model risk?
One way is to invest in research to improve models and to develop statistical tools. It is critical for an institution to keep up with developments.
An even more vital way of reducing model risk is to establish a process for independent vetting of how models are constructed.
The role of vetting is to assure management that any model for the valuation of a given security proposed is reasonable. It provides assurance that the model offers a reasonable representation of how the market itself values the instrument, and that the model has been correctly implemented.
1. Documentation
This should be independent of any implementation, such as a spreadsheet or code, and should include:
(a) the term sheet or;
(b) a statement of the model
2. Soundness of model
The vetter needs to verify that the mathematical model is a reasonable representation of the instrument.
3. Independent access to financial rates
The vetter should check that the bank’s middle office has independent access to an independent market-risk-management rates database.
4. Benchmark modelling
The vetter should develop a benchmark model based on the assumptions and on deal specifications. The vetter may use an implementation that is different from that proposed.
5. Health check and stress test
6. Build a formal treatment of risk into the overall risk-management procedures
Large trading profits tend to lead to large bonuses for senior managers, and this creates an incentive for these managers to believe the traders who are reporting the profits. Often, traders use their expertise in formal pricing models to confound internal critics, or they may claim to have some sort of profound insight. Senior managers should approach any model that seems to record or deliver above-market returns with a healthy skepticism, to insist that models are made transparent, and to make sure that all models are independently vetted.
This article is an edited version of an entry in the “Encyclopedia of Quantitative Risk Analysis and Assessment”, Copyright © 2008 John Wiley & Sons Ltd. Used by permission.
