Performance validation of representative sample-balancing methods in loan credit-scoring scenarios

Ling-Jia Chen and Runchi Zhang

Abstract

Data sets used to construct a credit-scoring model are always imbalanced in the real world, leading to the recognition ability of the model becoming biased toward the majority and low-risk samples and away from the minority and high-risk samples. In the past few decades, many sample-balancing methods have been designed to balance the two classes of samples before modeling, but they lack sufficient performance verification, especially on large data sets. This paper quantitatively validates 12 of the most representative balancing methods. The results show that, in terms of performance, a method combining the synthetic minority oversampling technique (SMOTE) and the Edited Nearest Neighbor method is optimal, followed by the SMOTE-Tomek method, whose performance is significantly different from the other methods tested. All 12 balanced methods can maintain stability and thus meet the relevant requirements of the regulatory authorities. The performance of each credit-scoring model is also influenced by the balancing ratio and the number of variables in the data set. In general, the user needs to determine the proper balancing ratio according to the comprehensive characteristics of the scoring model and the balancing method, and a data set containing a larger number of related variables will improve the performance and robustness for most balancing methods.