Journal of Credit Risk
ISSN:
1744-6619 (print)
1755-9723 (online)
Editor-in-chief: Linda Allen and Jens Hilscher
![Risk.net](https://www.risk.net/sites/default/files/styles/print_logo/public/2018-09/print-logo.png?itok=1TpHrpuP)
Validation of credit rating systems using multi-rater information
Kurt Hornik, Rainer Jankowitsch, Manuel Lingo, Stefan Pichler, Gerhard Winkler
Abstract
ABSTRACT
We suggest a new framework for the use of multi-rater information in the validation of credit rating systems, applicable in any validation process where rating information from different sources on the same set of objects is available. As our validation framework does not rely on historical default information, it appears to be particularly useful in situations where such information is inaccessible. We focus on the degree of similarity or – more generally – proximity of rating outcomes stemming from different sources and show that it is important to analyze three major aspects of proximity: association, agreement and rating bias. In contrast to the existing literature, we suggest as a measure of association, which is based on the Kemeny–Snell metric and, opposed to other measures, is consistent with a set of basic axioms and should therefore be used in the context of multi-rater information. Furthermore, we suggest using a weighted version of Cohen’s to measure the agreement between two rating systems and we introduce a new measure for rating bias. We provide an illustrative empirical example using rating information stemming from the Austrian Credit Register on partially overlapping sets of customers of 27 banks. Using a multi-dimensional scaling technique in connection with a minimal spanning tree, we show that it is possible to consistently detect “outliers”, ie, banks with a low degree of similarity to other banks. The results indicate that banks that are less diversified across the size of their loans are more likely to be outliers than others.
Copyright Infopro Digital Limited. All rights reserved.
As outlined in our terms and conditions, https://www.infopro-digital.com/terms-and-conditions/subscriptions/ (point 2.4), printing is limited to a single copy.
If you would like to purchase additional rights please email info@risk.net
Copyright Infopro Digital Limited. All rights reserved.
You may share this content using our article tools. As outlined in our terms and conditions, https://www.infopro-digital.com/terms-and-conditions/subscriptions/ (clause 2.4), an Authorised User may only make one copy of the materials for their own personal use. You must also comply with the restrictions in clause 2.5.
If you would like to purchase additional rights please email info@risk.net