Last summer, it was reported that OpenAI, the company behind ChatGPT, had information about the chatbot’s design stolen.

According to the New York Times, the perpetrator got hold of details about how the company builds its AI tools by snooping on a forum used by OpenAI employees.

But while it’s not believed that the systems used to build and run the AI were hacked, the incident showed how AI systems – and the data centers used to house and train them – are prime targets for cyber attackers.

The reason is simple: as AI becomes further integrated into our lives, the more valuable the algorithms behind the AI and the datasets they’re trained on are. And not only are the vast swathes of information being processed by AI vendors and their platforms extremely valuable to cyber criminals and malicious threat actors, there’s also the risk that attackers will attempt to get hold of the secrets and the source code that power AI.

Attacking AI

Because, as with any sort of new technology, software, or product, there are those who want to get hold of it by any means necessary. That can include hacking into the data center where it’s housed, be it to make off with the intellectual property required to build their own version, steal information, or carry out a range of other malicious acts.

“What you have is this growing value of data that’s housed in these data centers, so they’re becoming a bigger and bigger target – and more and more of the world is running on infrastructure that’s in these environments,” says Paul Shaver, global operational technology security lead at Google Cloud Security.

“They’ve become critical infrastructure pretty quickly in the way we’re leaning into leveraging more and more cloud and AI technology.”

As with other forms of critical infrastructure, data centers housing AI tools quickly become a very tempting target for cyber attackers. And while more traditional forms of critical infrastructure like energy, food processing, and water supply systems may be run on air-gapped systems, the very nature of AI means that the data centers powering it are directly facing the Internet. But while necessary, it means this form of critical infrastructure is potentially more vulnerable to unauthorized intrusions

“It’s growing the attack surface,” says Shaver. “You’re putting more out on the Internet with more connectivity into these environments, which presents another attack vector that needs to be monitored, controlled, and updated.”

Naturally, cyber crooks are aware of this expanded attack surface around data centers hosting AI and are trying to take advantage of it. And with many AI systems hosted on cloud-based infrastructure, hackers will attempt to exploit the same techniques they use to target cloud-based applications: such as using phishing or impersonation attacks to gain hold of legitimate credentials, or perhaps even delivering malware to provide them with a secret backdoor into the system. And they’re increasingly using AI tools to help conduct attacks.

“The attackers are leveraging AI to identify ways to exploit these systems, to write phishing emails, and to write malicious code,” says Shaver.

But there’s also another threat vector to think about when protecting the data centers storing the models: the supply chain.

“Because by their very nature, LLMs and AI are very heavily dependent on integrations and APIs, which means there are a lot of third-party connections and frameworks,” says Michael Adjei, director of systems engineering at Illumio.

“The problem with that is that most of them are new. If you have anything that’s new, that hasn’t been stress-tested, it’s a sweet spot for cyber gangs to target. Especially if you’ve done a lot of R&D to get to the point where that IP is commercial,” he explains.

Move fast and break things (and forget to secure them)

It’s a tale as old as technology; when something is new and exciting, companies are keen to roll their new product out as quickly as possible – especially if they are competing with rivals for market share.

But the speed at which products are rushed to market can result in problems, not least around the security of the systems. An organization that wants to ensure it’s fastest to market is, unfortunately, unlikely to test every aspect of its product, let alone examine the tools and companies used in its supply chain. And that’s creating opportunities for opportunistic attackers.

“If I’m a nation-state and I see your company is doing well, I’ll put resources towards trying to steal that information, rather than building it from scratch,” Adjei says. “If they know it works, they can steal it and make it their own.”

The rapid pace at which AI and LLMs entered the public eye and became widely-used tools by businesses resulted in a shortage of GPUs, as AI vendors scooped them up to enable the high-powered processing required to run and train their models.

There are still only a handful of major players - led by Nvidia and AMD - which make and supply GPUs suitable for AI data centers. A savvy, well-resourced attacker could take advantage of this by corrupting the supply chain in a way that provides them with direct access to the data centers.

“If I were to prioritise the biggest concerns, I would say that most of the big enterprises out there are using certain LLM providers and they’re all using certain hardware – the fanciest, newest, shiniest GPUs. If you can learn those AI vendors and hardware, then maybe an attacker could plant something into the supply chain,” suggests Itamar Golan, CEO and co-founder of Prompt Security.

“You could potentially poison the hardware used for training LLMs. If you can get into the supply chain of the GPUs, you can make it vulnerable to certain inputs and keywords, and now you can take control from the hardware level. It’s something to consider.”

Poisoning the data supply

Data poisoning is when a malicious actor intentionally inserts false data into an AI’s training data with the goal of manipulating or changing how the model operates. More often than not, in a way designed to disrupt how the AI works, or trick it into releasing sensitive or harmful data.

“They can do it in plenty of ways. They can hack your system and add fake samples. They can use what you add, like customer feedback, but intentionally skew it to non-average behaviour. But in the end, your model is being trained on biased or maliciously impacted data,” explains Golan.

“The consequences can range from simply not generating good results because the training set is impacted, to being able to generate harmful content. It could be vulnerable to a secret token that you can use to take control of the model and other sophisticated malicious capabilities.”

Protecting the data centers being used to train AI models is therefore essential, especially as AI becomes more widely used in business and society.

“AI has become ubiquitous and we can’t really opt out of it – so there’s a greater responsibility on the vendors to ensure it’s protected properly,” says Adjei.

“Whether it’s nation-states wanting to steal your model, or a threat actor wanting to use prompt injections to manipulate your model, that risk is real.”

So, what actions should AI data center operators be taking to ensure that their infrastructure remains protected against cyber attacks, data poisoning, and other threats? In many ways, operators should be following the same cybersecurity strategies as those securing any other systems or infrastructure.

“We think about AI as this new thing – and it very much is – but technology as a whole has the same lifecycle of good security hygiene, good detections, testing those capabilities against real-world attack scenarios based on the threat landscape,” says Shaver.

“We’re not reinventing the wheel, the wheel is just moving faster. And so, it comes back to paying attention to the threat landscape, developing your security hygiene and defences against what the attackers are doing. The attacks change, the exploits change, but the lifecycle of securing AI is really no different,” he concludes.