Journal of Risk Model Validation

I recently attended the Inquire Europe conference, where the special topic was environmental, social and governance (ESG) criteria. Much of the research that was presented was at an early stage, but there were attempts to include ESG criteria in parts of the risk-modeling process. At some point, this will evolve into risk model validation, and I await the resulting papers with interest. The offerings in this issue of The Journal of Risk Model Validation cover many aspects of risk model validation without any particular theme emerging, although we have two papers on aspects of the recently published Fundamental Review of the Trading Book (FRTB), which is part of Basel III.

Our first paper is “A comprehensive evaluation of value-at-risk models and a comparison of their performance in emerging markets” by Saeed Shaker-Akhtekhane, Mohsen Seighali and Solmaz Poorabbas. This looks at the performance of different value-at-risk (VaR) calculation methods. The emphasis here is on emerging markets. The authors apply several widely used methods for calculating VaR, including both parametric and nonparametric methods. They consider different confidence levels for the VaR as well as different sample sizes. The countries the authors consider are Iran, Turkey and Russia. The best-performing models, according to their analysis, involved time-varying asymmetric volatility with fat tails in the distribution. This is a pleasing result because, a priori, these are the features we might expect to see in emerging market returns. The popularity of cross-asset investment and the integration of emerging market assets into a much wider range of portfolios makes this paper very topical.

Carsten S. Wehn’s “Back to backtesting: integrated backtesting for value-at-risk and expected shortfall in practice” is the second paper in this issue. It looks at the validation of market risk forecasts by means of backtesting. According to the author, the upcoming FRTB emphasizes the importance of expected shortfall as a risk measure, which, in turn, should influence the discussion on appropriate approaches to its proper validation. There is an emphasis in this paper on providing a structure for validation and some practical applications. In addition to a classification in this context, the paper offers a practical application for backtesting via an example portfolio with three comparative models, and it shows how to integrate the different backtesting methods in an integrated framework.

“Model risk in the Fundamental Review of the Trading Book: the case of the Default Risk Charge” by Sascha Wilkens and Mirela Predescu, the issue’s third paper, also deals with the recent FRTB. As a result of this review, default risk needs to be measured and capitalized through a dedicated Default Risk Charge (DRC). The authors have previously worked on this topic using a Gaussian factor copula model. They extend their previous analysis by assessing the resulting model risk, investigating alternative copulas (Gaussian, Student t and Clayton) and their influence on DRC figures with the help of a set of example portfolios. The copula choice can affect the DRC considerably, especially for less diversified, directional portfolios. The influence on typical larger-scale, diversified portfolios is much less pronounced. The uncertainty arising from the calibration of any copula using only a few data points – as implied by the regulation – is of at least equal importance to the selection of the dependence model itself. It is interesting to note that much has been written criticizing the Gaussian copula, and these results may shed some light on the validity of that criticism.