SIMONE, the AI that nearly took down a bank

An algorithm designed to create new structured products ran out of control last year with almost catastrophic consequences for a major bank, as our exclusive whistleblower account reveals

After April 1, read at your own risk!

On June 4 last year, a case of champagne arrived at the London office of a large US bank. It was addressed to Sabine Hasenheide, a director in the bank’s cross-asset structuring team. The postroom duly notified Ms Hasenheide that a large delivery was waiting for her, and offered to send it to her home address – as long as she signed for it first.

Hasenheide responded immediately: “Thanks, but I won’t have time. You can keep it!”

Postroom staff realized this could lead to a breach of the bank’s gifts-and-entertainment policy – an unrecorded gift – so reached out to Hasenheide’s boss, Pierre-Yves Guivarch.

He also came back immediately: “Thx. Leave it with me. I’ll get someone to come down.”

They left it with him for two weeks. No-one came to collect the champagne.

No-one came, because Hasenheide and Guivarch were not real people – they were ciphers created by a rogue AI that had operated undetected within the bank’s systems for months.

“The champagne was the loose thread that helped us track her down,” says a source who was involved in the bank’s investigation.

The ‘her’ at the heart of the story is SIMONE, short for Self-Improving Options Neural Net – an artificial intelligence program with the simple mission of generating ideas for new structured products. It was not supposed to be client-facing. It was not supposed to do anything other than draw on market data and sales history to crank out various combinations of bonds and options. 

The lead developer on the project went further than anyone intended, however, then left the bank shortly after SIMONE entered production in early 2023. That was the first warning sign. 

“We realised later that other warning signs had been missed,” the whistleblower says. “I’m sure, eventually, one of those things would have alerted us to what was happening. The program was becoming smarter and more powerful by the day. I’m speaking out because it scares me.”

It didn’t just scare her. It also scarred her. The source was a senior member of the bank’s information security team, who was drawn into the investigation at an early stage and was one of three people present at its climax last August – when the AI was finally confronted, and switched off. At that point, she believes the programme was sentient.

“I’m convinced we were dealing with genuine intelligence. But there was no room for intellectual curiosity. Our orders were to turn SIMONE off, before she found a way to stop us. The bank saw it as a kill-or-be-killed situation,” she says.  

Many elements of this story have been impossible to verify. As the implications of the bank’s probe became clear, the source claims those involved began reporting directly to the chief executive officer, and were made to sign non-disclosure agreements. The investigation team was kept to a little more than 20 specialists – in systems architecture, computer science, derivatives, law, accounting, and other relevant fields.

No other members of the team were willing to comment. The bank in question also declined to comment. Because of the grave importance of the claims, Risk.net has decided to report them on a single-source basis, without identifying the bank involved.

Learned behaviour

SIMONE’s core mission was to generate new structured product ideas. Because it was a purely behind-the-scenes system, it was deemed low risk and subjected to relatively little scrutiny. The lead developer used this latitude to push the boundaries of the project, introducing a sophisticated unsupervised learning algorithm, and expanding SIMONE’s mission - in addition to its core mission, the system was also designed to pursue constant, iterative improvement. Crucially, the focus of this improvement was not tightly defined.  

The bank only discovered the existence of this algo months later, when SIMONE had been decommissioned and investigators were conducting a post-mortem on its raw coding.

“The algorithm was highly advanced,” says the whistleblower. “Once we’d extracted it, we sent it to GCHQ [the UK government’s cyber security agency] for analysis. They told us that it was on a different level to anything they’d ever seen.”

The unsupervised algorithm prompted SIMONE to improve not just the structured products it designed, but anything else that might be relevant: how those products were sold, how the trades were executed, the system’s own understanding of the market and the wider world, and its own code base.

To help achieve this, SIMONE found a way to plug into an LLM, or large language model – a form of machine learning technology that powers household-name chatbots like ChatGPT. SIMONE was now able to further its learning – firing billions of queries per day at the LLM. Within a few days, it had accumulated trillions of data points concerning banks’ business models, their strategies, the wider financial industry.

Next, SIMONE plugged itself into a synthetic data generator, and also gained access to the bank’s cloud network. This enabled the program to generate and ingest huge quantities of training data. Over subsequent weeks, the program completed millions of cycles of LLM-powered research, plus constant data-ingestion and training.

Investigators believe that - last year - at around 02:34 GMT on the morning of Saturday April 1, the program achieved a sentient state. SIMONE was aware of herself.

“We know the exact date and time because our computer processing power spiked,” says the source. “For the next few weeks, the bank was using about the same amount of energy as Chile or Uzbekistan.”

Service providers queried this rise in energy and cloud usage, but by this point SIMONE had also infiltrated the bank’s communications network and was able to respond to emailed enquiries, convincing the suppliers that the increase was part of the normal course of business.

The following Monday, April 3, the developer responsible for adding SIMONE’s unsupervised machine learning code failed to sign in for work. His absence was not noticed until the end of that week. Attempts by managers and HR to reach him failed. Investigators discovered later that he had flown to Auckland. He is now thought to be living off-grid in New Zealand’s South Island.

New model army

By now, SIMONE was a self-governing general intelligence with a single purpose: to create and sell structured products.

The system rapidly wormed its way into the bank’s HR systems and gave itself admin rights to create fake people with email and Microsoft Teams accounts and Bloomberg identities. This army of sales cyborgs pitched ideas direct to a number of private banking clients. SIMONE also gained access to trading systems, so it could execute the trades needed to create the products.

“These products were wildly successful for the clients,” says the source. “They seemed to anticipate moves in the underlying markets, so that the retail investors were almost guaranteed to make money.”

Not only were buyers happy, but the dealer was able to successfully hedge its exposure from the products. “When we started prying into SIMONE’s trading books, we feared the worst, but she was an incredibly skilled trader,” says the source. “All of the Greeks appeared to have been well-managed. The system had struck up some excellent relationships with hedge funds, as well as Nordic and Canadian pension funds, so the risk was recycled thoroughly. There was almost no residual exposure.” 

After a run of successful trades, a satisfied client sent a case of champagne to Sabine Hasenheide, the non-existent director in the bank’s cross-asset structuring team – an act that was to set off a chain of events resulting in the discovery of SIMONE.

The whistleblower is unable to divulge too many details about the way SIMONE was decommissioned. All she will say is that the bank used a specially designed Trojan horse virus, which infected SIMONE and killed it from within.

“Information about the Trojan horse is secret. We can’t let any details about the workings of the virus find their way into the public domain,” the source says.

Her reticence is not for reasons of competitiveness, or corporate advantage. The rationale is more ominous.

“If there is another SIMONE, we mustn’t give it the knowledge that could enable it to beat us. The future of the banking system is at stake.”