Hints on quantification approaches

Tiziano Bellini, head of risk integration competence line, international markets at Prometeia, examines the key components of successful model risk management (MRM), focusing on the importance of integration, processes, governance and IT solutions to strengthening resilience

Tiziano Bellini, Prometeia

Models are an integral part of modern decision-making in financial institutions. They are used to price transactions, value portfolios and optimise returns. Models are also key to regulatory frameworks – for example, they are used to determine requirements for capital and liquidity. Models, however, require constant vigilance – risk measurements and financial analytics always need to be monitored for effectiveness and relevance. Simplifications and assumptions models must necessarily employ sometimes come at the cost of accuracy and structural integrity under stress. This exposes banks to model risk.

A thorough process is needed to ensure effective model management, and this can be achieved by spreading the know-how across different competence lines. Change management programmes, in conjunction with integrated IT solutions, are the key success factors to tackling such a challenge. Leveraging Prometeia’s experience, the following building blocks summarise the critical areas for an advanced MRM process:

• Identification – Risks related to a specific model are mapped throughout their entire lifecycle. A flexible IT platform extracts information from various sources (such as validation reports or internal audit findings) to classify models based on a customised set of rules. Holistic taxonomy plays a key role in ensuring consistency across the whole model management process.
• Control – A comprehensive process is designed to monitor risks and trigger appropriate actions. Residual risks, related to unavoidable potential adverse events, are appropriately scrutinised to inform effective decisions.
• Governance – Rules are defined and embedded into a comprehensive IT solution to align model risks to a financial institution’s specific risk appetite. Qualitative and quantitative assessments are crucial for setting and monitoring appetite, tolerance and risk capacity.
• Quantification – Well-shaped methods and consistent IT tools are the key ingredients to represent models’ effectiveness in capturing the key features of the phenomenon under analysis. Traditional statistics integrated with Bayesian methods and machine learning and artificial intelligence (AI) are crucial to quantify model risk. Silos and complex networks of model analyses are performed to summarise their weaknesses on a standalone basis, and when they are part of integrated systems.

Model risk is interpreted as the potential loss an institution may incur as a consequence of decisions based on the output of models. Errors may occur in the development, implementation or use of the models. Uncertainty depending on model errors is at the very heart of such potential losses. In the past few years, financial institutions have been developing model risk quantification solutions to set and monitor the appetite for model risk, design new margin-of-conservatism paradigms, measure the impact of extreme scenarios on model performance, and so on. Prometeia is pioneering this field by proposing a comprehensive frame based on the following pillars:

• Mis-specification and calibration analysis – You need to ensure the chosen model is capable of representing the phenomenon under analysis. The first question is whether the model is suitable for the purpose. The second is whether the model is effectively calibrated. You must specify appropriate metrics to address these questions. A comprehensive definition of modelling error is required to embed issues on model choice and parameter fitting. The aim is to disentangle these two aspects to set model risk appetite limits (for example, in the development phase), monitor them (along model usage) and facilitate the entire MRM process.
• Sensitivity and what-if enquiries – A model developed on all available data and using appropriate techniques has the potential to represent the phenomenon under analysis as it develops. Nevertheless, changes in internal conditions (for example, portfolio composition) or external factors (macroeconomic situations) may reduce a model’s potential. For this reason, you may examine the circumstances leading the model to fail its mission. What‑if analysis, based on a stressed setup alongside reverse-stress mechanics are particularly effective in defining model risk tolerance limits and monitoring them over time. Bayesian statistics, machine learning and AI procedures may be very useful in exploring and generating scenarios to highlight models’ potential weaknesses.
• Complex modelling interactions – Integrated modelling frameworks characterise banking processes and management decisions. Uncertainty in each component may impact on dependencies. Functional analyses, in combination with a solid simulation architecture, are key success factors in quantifying interaction uncertainties (for example, stress-testing exercises or balance sheet planning).

In essence, thorough model management requires integrated efforts. Processes, governance and IT solutions, in conjunction with enhanced quantification methods, are crucial to strengthen a financial institution’s resilience by fostering an effective decision-making process.

Sponsored content