Goodness-of-fit for discrete-choice models of borrower default

Arden Hall

Abstract

Discrete choice models of the probability of default (PD) have several applications in finance. In some applications, such as credit scoring, their value is in ranking applicants or customers by PD. Other applications, such as estimating losses as part of determining capital requirements under Basel, require accuracy for the estimated PD. There is a well-developed set of tests to assess models’ ability to rank-order, and these are sometimes relied upon to assess the accuracy of models’ probability estimates. This paper demonstrates that the rank-order tests are unreliable for assessing models to be used to predict probabilities. This is true even when estimated probabilities will only be used to assign observations to segments. There are other tests, such as the Hosmer–Lemeshow test, which assess magnitude as well as order. While there are some practical difficulties in applying these alternative tests to the data sets typically used for default model estimation, they can provide a better assessment of how well a model fits the data.