Journal of Risk Model Validation

Steve Satchell

University of Cambridge

This editorial is being written against a backdrop of seismic shocks to the financial sector and it is hard to make any forecasts about outcomes in the short or the long run. Indeed, events have been so peculiar that validating risk models seems rather second order. I think it might be conceded by even fanatical quants that certain forms of extreme events do not really admit to quantification, unless we have a proviso that market structures stay the same through time and we have 20,000 years of data.

It also seems clear that the clock to measure observations from is not time, but volatility. In risk realization terms, September 2008 may prove to be equivalent to 100 months of “normal” market activity. With these preliminaries out of the way, I shall discuss the contents of our current issue. The broad theme is the role of macroeconomics in modeling credit and default risk.

Our first paper is by Antioch and it is concerned with recalibrating credit risk models. This is an interesting topic and in a more general context can be thought of in terms of how frequently one should re-estimate/rebuild risk models. Factor rotation is a serious problem in this context so that certain risk factors will rise and fall in importance through time. In some cases, they may disappear altogether while entirely new ones will emerge. Recalibration thus has a number of dimensions; it is not just about data frequency and re-estimation.

The second paper by Breeden and Thomas is a follow-up to the Breeden et al (2008) paper from the previous issue and is concernedwith the relationship between default and economic cycles for retail portfolios across countries. I think this is a valuable contribution; as I have mentioned in previous editorials, trying to get macroeconomic information to bear upon default is a challenging statistical exercise. Typically the macro data lacks sufficient variability to lead to any useful relationships. However, looking at it in a multiple country or sector perspective can be a good way to proceed.

The third paper is by Christodoulakis and the Editor-in-Chief of the journal. Modesty forbids me from talking about it too much except to say that it addresses some theoretical issues to do with receiver operating curves (ROC). These are normally treated in the literature as empirical entities and this paper discusses their population analogue. We see that ROC analysis has a strong relationship to an analysis of the quantiles of the sick and healthy loan population. We illustrate our methodologies with macroeconomic shocks.

The final paper, by Cipollini andMissaglia, is titled “Forecasting industry sector default rates through dynamic factor models”. The authors use macroeconomic variables to explain industry sector default rates in Italy. Their chosen methodology is a dynamic factor model. They assess different procedures for predicting portfolio credit risk. Such an analysis is a very useful tool in validating risk models.