Journal of Risk Model Validation
ISSN:
1753-9579 (print)
1753-9587 (online)
Editor-in-chief: Steve Satchell
Quantifying model selection risk in macroeconomic sensitivity models
Need to know
- Forecast accuracy is compared from lists of candidate models for a range of loan products.
- Risks are also compared for ensembles of models.
- Monte Carlo error propagation was found to accurately quantify model risk.
- Error propagation showed that a single model’s forecasts can be systematically biased relative to the full distribution.
Abstract
Model selection is a critical step in model development and brings with it a significant level of model risk. This analysis considers a real-world example and compares the forecasts and uncertainties of three possibilities: the model selected as best, the best ensemble and the model not selected. A key part of this analysis is propagating the prediction intervals for the individual forecasts using a Monte Carlo simulation all the way through the lifetime probability of default calculations to obtain distributions for the lifetime forecasts. The results highlight two sources of risk: a systematic difference between the expectation value of the forecast and the median of the simulated distribution, and the 95% confidence interval from the distribution. The difference between the forecast and the simulated median should be included as part of the loss reserve. The confidence interval from the distribution should contribute to the model risk component of the economic capital. This paper also illustrates how uncertainties can be computed for Current Expected Credit Losses or International Financial Reporting Standard 9 Stage 2 lifetime loss estimates.
Copyright Infopro Digital Limited. All rights reserved.
As outlined in our terms and conditions, https://www.infopro-digital.com/terms-and-conditions/subscriptions/ (point 2.4), printing is limited to a single copy.
If you would like to purchase additional rights please email info@risk.net
Copyright Infopro Digital Limited. All rights reserved.
You may share this content using our article tools. As outlined in our terms and conditions, https://www.infopro-digital.com/terms-and-conditions/subscriptions/ (clause 2.4), an Authorised User may only make one copy of the materials for their own personal use. You must also comply with the restrictions in clause 2.5.
If you would like to purchase additional rights please email info@risk.net