Setting boundaries for neural networks

Quants unveil new technique for controlling extrapolation by neural networks

According to a popular internet meme, there are two types of people in this world: those who can extrapolate from incomplete data.

Neural networks will probably struggle with that one. As quants race to deploy neural networks in finance, they are running into a common problem. Neural networks require huge datasets to train, but they do not always extrapolate well when faced with new situations.  

“Neural networks can fit the data very well within the region of training, but can produce completely unpredictable and uncontrolled results outside of that,” says Michael Konikov, head of quantitative development at Numerix.

Konikov, along with Alexandre Antonov, chief analyst at Danske Bank, and Vladimir Piterbarg, head of quantitative analytics and development at NatWest Markets, propose a two-step solution to this extrapolation problem.

The authors tackle the common issue of approximating a derivatives pricing function using neural networks. They start by approximating the pricing function with a cubic spline, a set of third-degree polynomial curves joined together. This allowed them to closely replicate the asymptotes of the pricing function, and capture the limits of the function for large values of its parameters. Asymptotes are normally known or relatively easy to calculate for many common functions in finance, making it possible for the pricing function to be approximated by a spline-like function. By subtracting the cubic spline from the pricing function, they created a function that tends to zero for large values of parameters, de facto eliminating explosive or uncontrollable values.

This effectively reduces the problem to the approximation of a neural network to a pricing function with null asymptotes. Next, an additional mapping function is inserted into the neural network to ensure that, outside a selected domain, asymptotes are kept to zero.   

“We look at the expected behaviour of the model for large values of input parameters, outside the region where we would normally use neural networks, and incorporate that into the overall approximation,” explains Piterbarg.

The paper describes how the technique works on the SABR model, which has the advantage of being neither too simple nor too complex, and for that reason is commonly used to test neural network applications.

“We use the SABR model [to show] the power of our method. The good thing about it is one can calculate its true values for any set of input parameters exactly,” Piterbarg says.

We look at the expected behaviour of the model for large values of input parameters, outside the region where we would normally use neural networks, and incorporate that into the overall approximation

Vladimir Piterbarg, NatWest Markets

The technique, however, is fully general and can be applied to any function that can be approximated with a neural network and has known asymptotes in some dimensions.

Piterbarg sees XVA calculation as one possible application.

“It is particularly suitable for complex pricing functions that involve expensive Monte Carlo simulations, like those of Bermudan options or other exotics, with a reasonable number of parameters, like rates, vols etc,” says Antonov.

He is planning to “implement it to a low/mid-dimensional problem of a spread option between two SABR rates”.

Numerix also intends to put the research to work. “While this technique has not been implemented in our production library yet, we plan to do this at some point in the future,” reveals Konikov.

Piterbarg himself has been critical of hasty applications of neural networks in finance, comparing them to a hammer looking for a nail. Using classical functional analysis to control extrapolation by neural networks will mitigate some of the risks inherent in deploying this new technology. “What we’re trying to do is focus neural network research in the direction where it actually works together with traditional research,” he says.