Journal of Risk Model Validation

Steve Satchell

University of Cambridge

One of the most interesting documents I have read recently on stress testing is an address1 by Mr A. G. Haldane, Executive Director for Financial Stability, Bank of England, to a stress testing conference in February 2009. Here he deals with some of the, probably, many reasons why stress testing banking risk models failed to predict the catastrophic losses that recently occurred. Among other reasons, he looks at disaster myopia, a blindness in an industry to the storm that comes after a periodof calm. I can affirm its existence in our industry.

This unfortunate condition is often found, for example, among some of my former students after they have received their first bonus. I recall one of them, who I shall not name and shame, blithely informing me that risk management had advanced so far that crashes were a thing of the past. Returning to Mr Haldane, whose ability to juggle abstract academic ideas with central banking realities is very impressive, I note that he also gives far reaching advice on how to set-up stress testing in the future.

This material should be of great interest to our readers. It is not fully appreciated that stress testing is a critical part of model validation. Often the output that comes out of such an exercise is seen more as information about the portfolio involved rather than the model involved. Investors use stress testing to see how the profitability of a strategy reacts to extreme changes in inputs. However, this is just one transformation of the possible outputs a model can produce. Properly designed stress testing should be able to give us information about both the model and the current portfolio.

In this issue we have a number of papers which look at issues of validation and stress testing. Li et al present “a practical framework for empirically evaluating the performance of loss given default models as part of a greater credit risk management infrastructure. It discusses the use of quantitative metrics such as the confusion matrix, the expected loss shortfall, and the loss capture ratio as part of a periodic validation/approval process”.

Mager and Schmieder present a paper on stress testing German credit portfolios using an internal-ratings based model and an alternative correlation based model. They present interesting results based on historical data.

Next, Meyer looks at the estimation of intra-sector asset correlations. The author discusses how to validate the parameters using both cross sectional and time series information.

Finally, Petrov and Pomazanov look at Basel II capital requirement maturity adjustment formulas. They propose methods of calculation based on rating-agency data and also analytical formulas.

As always, the Journal is indebted to our contributors for producing such a good range of relevant, intellectually challenging papers.