Not stressed enough

Recent turmoil in the US subprime mortgage market has seen mass downgrades, falling prices and a drying up of liquidity in the secondary market. Did banks' stress tests prepare them adequately and were scenario simulations up to the task? Clive Davidson investigates

The recent market troubles arising from the US subprime mortgage market have caught many investors and dealers on the hop. Delinquencies have far exceeded historical precedent, the rating agencies have downgraded hundreds of securities backed by subprime loans, and prices in the secondary market have plummeted amid a drying up of liquidity. Spooked investors have refused to extend credit to virtually any structured credit investor, while hedge funds, facing margin calls on loss-making credit exposures, have been forced to liquidate positions in other, more liquid markets, causing volatility across asset classes.

The turmoil has posed many questions - not least of all, tough ones about the risk management capabilities of the institutions involved in the securitisation market. Underwriting standards had been weakening for some time in the US, predicated on the assumption that US house prices would continue to rise. Did originators, arrangers and investors in residential mortgage-backed securities (RMBSs) and collateralised debt obligation (CDO) tranches contemplate a potential fall in house prices or a sharp rise in defaults? Liquidity had been plentiful in benign market conditions, but was a complete drying up of liquidity in the CDO markets, as well as increased volatility in other asset classes, considered? Conventional risk measures such as value-at-risk estimate expected losses under the normal range of market conditions. But firms also need to consider the unexpected and how they would fare under extreme conditions. This is the realm of stress testing.

While the current crisis is still playing itself out, it is impossible to say just how well institutions were able to stress test their exposures. Nevertheless, supervisors have been warning for some time that stress testing within financial institutions continues to fall short of best practice, especially with regard to stressing aggregated risks across multiple business units.

Regulators require that financial institutions stress test their overall liquidity in the light of possible adverse developments in macroeconomic factors. At the portfolio level, stress testing is expected, although requirements differ across risk factors. Stress testing portfolios for market risk is well established, but it has taken Basel II to introduce the practice for credit risk and operational risk.

"Stress testing and scenario analysis are essential tools for firms' planning and risk management processes," said the UK's Financial Services Authority (FSA) in its Financial Risk Outlook 2007. "By using stress testing and scenario analysis, senior management can assess and adjust their view of the risks that face their firm, plan mitigating action and identify risk concentrations."

However, the FSA admits it is still early days in terms of devising methodologies to aggregate risks in stress tests, meaning there is increased risk that institutions may be unaware of hidden correlations in their portfolios. The supervisor is in an ongoing dialogue with banks and other regulators about how stress testing should evolve, and last year published a thematic review of stress-testing practices at 10 large banks and building societies, which highlighted good practice and what it saw as areas of concern. The FSA concluded that most of the institutions were developing and improving their processes, rather than being at any stage of maturity with stress testing. Market risk stress testing was best developed, while "stressing some risk factors in combination such as market and credit risk factors was embryonic".

Similarly, the majority of banks in Germany currently use separate stress tests for market risk and credit risk, and find it difficult to handle combined stress tests, explains an official at the Bundesanstalt fur Finanzdienstleistungsaufsicht (BaFin), the country's regulator. US regulators, such as the Federal Reserve Bank of New York, talk of the frontier where stress-testing innovation is taking place, but like the FSA and BaFin, admit that the industry is still some way off solving the problems and agreeing best practice.

So what are the problems? For a start, there are several theoretical questions still to be answered. There are also technology and resources challenges, as well as management issues - for instance, what does the organisation do with the output of its stress tests, especially if it threatens a profitable line of business? And finally, there is the simple failure of imagination: can an organisation, or even the industry as a whole, overcome its inherent optimism and conjure up the range of extreme but plausible scenarios that reality might actually throw at it?

Theoretical issues arise because risk management is still part science and part art. "For mature markets, you can rely more on the science than the art, but for newer markets the art needs to play a role," says Andy Aziz, managing director of risk solutions at Toronto-based risk management systems specialist Algorithmics. "Most of the models used for stress testing, even if they are robust and capture all the risk factors, and even if you are able to predict the worst-case scenarios, are still often based on the assumption of perfect capital markets. One of the key assumptions is that there is perfect liquidity in the market - there can be a shock, but there are still as many buyers as sellers. In mature markets, that's usually a valid assumption, but if you look at some of the major market dislocations - the 1987 crash, Long-Term Capital Management in the late 1990s and the recent crisis - then, if not the cause, at least an accelerator to the impact has been a breakdown in liquidity."

Another problem is that models must be applied consistently across books in a stress-testing regime to accurately capture risk. This can be more difficult in a decentralised risk management structure, where valuation is only conducted in the front office. "The large banks are more likely to take a decentralised approach just because of the size and the number of systems they have and the number of trades they do," says Dan Travers, product manager for analytics at SunGard's Adaptiv risk management systems division in London. "These banks tend to put the onus on the front-office systems to produce VAR sensitivities at a summary level, which is a lot easier in terms of data volumes. But they now have a challenge - how do they stress test these sensitivities?"

The only solution to this problem is for the risk management division to create stress tests and distribute them among the front-office systems for all the various asset classes. "Then it is very important to have a way of constructing stress tests so they are consistent - so that bond spreads are moving with credit prices in a consistent way, and so on," says Travers. Furthermore, the front-office systems must have the ability to run the stress tests. "The question arises: can all the front-office systems return present values (PVs) for a particular scenario they are given? Providing they can, the risk management department still has to aggregate the PVs of all various books across all asset classes."

Ensuring consistency can be a particular problem with new and more exotic asset classes, where there is no standard for valuing the security or where the institution sees a competitive advantage in a proprietary modelling approach developed in-house. "Inconsistency and built-for-purpose are often two sides of the same coin," says Mat Newman, vice-president, product management for SunGard's Adaptiv division. The challenge is to integrate the new proprietary models with more standardised systems, avoiding operational risks such as incomplete or inconsistent data.

Adding to the difficulty is the fact that front-office models are usually built for accuracy, while risk management systems must look for economy in the time they take to run in order to operate viable stress-testing regimes. The front office might be able to live with a 10-second process for valuing an instrument, but the risk management division may want to revalue the instrument 250 times for an historical scenario simulation or 5,000 times for Monte Carlo simulation. This takes time, and the valuation has to be conducted for each deal in the portfolio.

"It gets even worse when you get into credit risk because you have to model the price over the whole life of the deal. So, if it's a five-year swap, you may have to revalue at 20-50 data points, multiplying the amount of times you need to revalue," explains Newman. Banks can address this by using faster models and relying on the breadth of the Monte Carlo simulations to restore accuracy, or they can throw hardware at it by using high-performance computer grids.

"Then what becomes an issue is not just having the models in place but having the data to drive them," says Newman. "If you are a large bank and you have 40,000 or so counterparties, trying to correlate their probabilities of default with all the relevant market rates is a huge and onerous task."

The current market turmoil has highlighted a particular problem in relation to securitised instruments and stress testing. A key purpose of securitisation is to enable institutions to parcel up risk and get it off their books. This distributes the risk across a wide number of participants, which has systemic risk mitigation advantages. But at the same time, it disconnects the buyers of the assets from the underlying risks and, more importantly, disconnects them from the data and tools that could be applied to model their exposures.

"There is a question of whether banks should be running Monte Carlo simulations for portfolios or should you be doing something more conventional that banks have been doing for more than 30 years - asset and liability management (ALM)," says Arun Pingaley, head of the product management group at governance, risk and compliance systems specialist Reveleus, based in Bangalore. "ALM systems allow you to measure two kinds of risk - interest rate risk and, much more important in this case, liquidity risk."

While mortgages remained on their books, banks measured and managed liquidity risk through their ALM systems. "If you have a 30-year mortgage on your books, and if there is an option for an annual reset, then you are going to treat it like a one-year exposure and will hedge it with liquid assets in case there is a demand on this. Then you would cover that with stress scenarios," explains Pingaley. But once the risk is parcelled up and passed on, those banks that have taken the assets into their trading books don't necessarily apply their ALM tools, he adds.

Modern risk management systems, such as those from Reveleus, Algorithmics and SunGard, offer ALM integrated with risk management with common data sources, so there is no technological obstacle to doing this. But one reason ALM analysis has not been conducted on RMBS and CDO investments is that the necessary data has not come along with the assets.

"If I am the originator of the mortgages, I will have all the data relating to the mortgages," says Pingaley. "If I have sold them to an investment bank, then they only have the information that I have provided at a portfolio level." The bank might then repackage the mortgages into a structured investment vehicle and sell them on, so the investor who ends up holding the instrument is several stages removed from the underlying mortgages. "What we need to ideally happen is for all the data to be carried with the mortgages so they can be modelled appropriately. Typically, this is not done," Pingaley adds.

On top of the modelling challenges, some analysts claim there has also been a failure of imagination to foresee likely risks - if not in terms of new risk factors, then in terms of the severity that existing risks might pose. Nick Hill, a credit analyst at Standard & Poor's (S&P), says banks attempted to examine their subprime exposure under adverse conditions but did not go far enough. "We've seen specific stresses applied to subprime books at banks as a part of broader stress tests," says Hill. "The issue with the current environment is that the stress we have had has been larger than most stress tests anticipated. It is not that the processes weren't being applied - it is really a scale issue."

Too mild

In its thematic review of stress testing at top UK institutions, the FSA commented: "We were struck by how mild the firm-wide stress events were at some of the firms we visited. On the evidence of our review, few firms were seeking out scenarios such as those that might require a dividend cut, generate an annual loss, or result in shortfalls against capital requirements while still remaining plausible. The current high level of profitability and capitalisation in many firms may provide part of the explanation. But additional reasons could be that firms might underestimate the likelihood of severe events or, where mitigating action is envisaged, they might overestimate their ability to take action that is both effective and timely. As a result, senior management may not be presented with scenarios that force consideration of more challenging issues."

In August, S&P conducted its own stress test - this time on how bank revenues might be affected if the current crisis developed into a scenario similar to the troubles of 1998, when Russia defaulted on its debt and Connecticut-based hedge fund Long-Term Capital Management collapsed. "The results show aggregate investment banking and trading revenues for the largest firms down 47%. This is more severe than in 1998, reflecting in particular potential mark-downs on leveraged finance exposures and structured credit," S&P said, adding that if banks do take a hit, they could find themselves subject to a ratings downgrade if it was found the firm's risk management systems were deficient.

"What we are saying is that where there is a financial effect from the current environment, our ratings might be sensitive to the risk management aspect of the bank, even if the financial impact is not that big by itself," says Hill. However, it is too early to say whether the current environment has thrown up risk management problems for the banks covered in its report, he adds.

The FSA and the Committee of European Banking Supervisors are among the regulators that have published discussion papers on stress testing over the past year or two, and are actively engaged in trying to improve stress-testing practice. Reports suggest the current crisis has prompted considerable discussion about stress testing behind closed doors. However, it is a highly sensitive issue, and none of the 10 banks Risk approached in the course of writing this article would agree to discuss stress-testing practice.

Observers say stress testing has been fairly low on the priority list at many banks, especially when markets were relatively benign and there were attractive profits to be had from new securitisation instruments. "It's been a bit of a 'who is going to blink first' situation," says Newman. "There is an acceptance that stress testing needs to happen, but there's not a great consensus about what those stress tests should be and what should happen to their results."