Interpreting neural network decisions

Terry Benzschawel

Methods for interpreting neural network decisions do exist. Although neural networks have often been criticised as being “black boxes”, there are methods for reliably determining and understanding neural network decisions. Some of these are

•  

  • the list-of-variables method,

•  

  • univariate relationships,

•  

  • the variable exclusion method,

•  

  • Garson’s method, and

•  

  • analysis of derivatives.

Each is considered in this chapter and these are used to interpret the results of the Fraud Detection and Mutual Fund networks presented earlier.

11.1 THE LIST-OF-VARIABLES METHOD

The simplest, most straightforward but least informative method for imputing network decision-making is to consider the list of input variables. For example, Table 11.1 presents a set of variables used as inputs for a network to predict mutual fund performance.¹¹ The mutual fund network is presented in Chapter 10. Clearly, variables such as the change in one-month return, the change in three-month return and the sales load are related to mutual fund performance.

A list of variables like that in Table 11.1 is useful in that we can see the full spectrum of information available to the model. Of course, the usefulness of this method is limited