Podcast: Montoro on FRTB thresholds and non-modellable risks

Senior risk manager also argues Kolmogorov-Smirnov test is better than Chi-squared

Photo: Marc Sargent

Adolfo Montoro, a director in the market risk management and risk methodology team at Deutsche Bank, visited our offices in London to discuss his new paper, The revised P&L attribution test and the suitability of new proposed thresholds, co-written by two of his colleagues, Marco Spinaci and Marc Georgi.

The Basel Committee on Banking Supervision published the latest consultation on its revised market risk capital requirements – the Fundamental Review of the Trading Book – in March this year, more than two years after the regulator published the finalised rules.

The FRTB requires each trading desk at a bank to individually pass a so-called profit and loss (P&L) attribution test in order to use the internal models approach, which is much less punitive than the fall-back standardised approach.

Following industry criticism that the test is extremely difficult to pass, even for well-hedged portfolios, the regulator introduced an 'amber zone' to allow for a smooth transition between pass and failure of the test, marked by the green and red zones, respectively.

In his paper, Montoro shows the thresholds for the amber zone are too narrow, by reverse-engineering the process used by the committee to calibrate the thresholds.

“When the latest consultation paper came out, we were interested to understand how those thresholds were calibrated in the first place…it took us a while to reverse-engineer those thresholds…but we clearly noticed the amber zone was very narrow and the red zone was quite big,” says Montoro.

The issue stemmed from what seems to be an incorrect assumption within the test, Montoro argues.

The P&L attribution test requires two different measures of P&L to be compared – the hypothetical P&L (HPL) generated by front-office pricing models, and the risk-theoretical P&L (RTPL), generated by the bank’s risk models. In the original proposals, the measures were compared in two different ways – one looks at the gap between the two using a mean ratio, and the other looks at the variance of that gap using a variance ratio. Both ratios should fall within certain established thresholds – if not, a breach is counted. Four breaches within any 12-month period will force a desk on to the standardised approach.

Our view is we are in favour of the Kolmogorov-Smirnov test, as the test has a slightly better ability to discriminate between different models used by different banks

Adolfo Montoro, Deutsche Bank

Following criticism that the tests are too sensitive, in its latest consultation, the regulator accepted industry requests to replace the mean ratio with the better statistical test of the Spearman correlation method. The regulator provided an option between the Kolmogorov-Smirnov and Chi-squared tests as a potential replacement for the variance ratio test.

The issue with the calibration of the two sets of tests are that they assume the underlying cumulative distributions of the HPL and the RTPL are independent – but they aren’t in practice.

“On reverse-engineering, which we justify in the paper, when you look at the usage of this Kolmogorov-Smirnov test and the Chi-squared test, they are parametric tests, but we shouldn’t forget the fact that they are used in the context which is not a natural context in which this test is supposed to be [used] – [that is,] when you test independent distributions,” argues Montoro.

The analysis by Montoro and his co-authors finds that the regulatory thresholds may have been calibrated using the p-values generated by the tests, which, given the invalid assumption, would not be the right way to go.

“In the paper…we move away from calibrating those thresholds using a p-value approach, but we’d rather look at an approach where we assess what a better model is…and then from there, saying let’s calibrate [the thresholds],” says Montoro.

Montoro and his co-authors instead calibrate the new thresholds by assuming what a good model is and what a good portfolio would look like as a benchmark, then build the thresholds from there.

“If I assume a good model should be characterised by this level of correlation, this level of bias and this level of under- or over-estimation [of risk], then from there, reusing the reverse-engineering process of how the thresholds should be calibrated in terms of intervals of confidence, the mathematical ground is very clear. And clearly, it has been shown there is a miscalibration, particularly for the amber and red zones,” Montoro added.

In the paper, the quants also argue why the Kolmogorov-Smirnov test is a better choice over the Chi-squared test.

“Our view is we are in favour of the Kolmogorov-Smirnov test, as the test has a slightly better ability to discriminate between different models used by different banks,” he said.

Index

0:00 Introduction

0:54 The P&L attribution test – outstanding issues

7:14 The amber zone

8:10 Paper in Risk

10:23 Kolmogorov-Smirnov test versus Chi-squared test

14:30 Are current thresholds set too narrow?

27:03 Calibration of thresholds

32:15 Assumption of independent distributions  

36:45 Outstanding issues in the NMRF rules

41:16 Data pooling: regional fragmentation concerns

To hear the full interview, listen in the player above, or download. Future podcasts in our Quantcast series will be uploaded to Risk.net. You can also visit the main page here to access all tracks, or go to the iTunes store or Google Podcasts to listen and subscribe.