Hackers could poison AI models by tampering with core infrastructure | #hacking | #cybersecurity | #infosec | #comptia | #pentest | #ransomware

Dive Brief:

Threat actors can sabotage AI models by tampering with their underlying infrastructure, such as the firmware and software powering the computers that run the models, the security firm Eclypsium said in a report published on Thursday.
“A subtle runtime change could bias the model toward weakening certificate validation, selecting an unsafe dependency, or generating a specific insecure pattern,” according to the report.
Eclypsium’s findings highlight an underappreciated area of risk that requires safeguards on the part of frontier AI labs and oversight by businesses using them.

Dive Insight:
When an AI model launches, the Eclypsium report said, “its runtime state is controlled by a hierarchy of software, firmware, memory controllers, management processors, and I/O devices,” each of which represents an opportunity to bias a model and its results if hackers can tamper with it.
The easiest target is the host machine running the model, which relies on an operating system, a kernel, graphics drivers and other hardware and software components. Those computers are vulnerable to the same malware as any other computer, including rootkits that establish persistence below the OS level by reaching into a device’s firmware.
Third-party components, such as firmware update services, AI orchestration agents and management controller hardware, also create pathways for tampering with models. “Compromise of the management plane can enable firmware modification, changes to isolation controls, direct control of attached devices, or other privileged paths into the workload,” Eclypsium said in its report.
Chip makers use data processing units (DPU) to handle part of the workload associated with running an AI model, but because DPUs perform such important processing tasks, they can give hackers a way of altering a model’s operating state. “The same positioning that makes this access sensitive is also what makes a properly secured DPU valuable for independent monitoring and enforcement,” said Eclypsium, which partnered with Nvidia on its research.
The report describes a demonstration of a model-tampering technique that altered the operating state after traditional security checks took place, effectively fooling the model into thinking its integrity had been successfully verified.
“The model file does not need to change,” researchers said. “The container, model hash, and signed artifacts on disk can remain valid while selected runtime weights are changed in memory.”
In addition to Nvidia, systems from AMD, Intel and Arm are also vulnerable to these techniques, according to the report.
Eclypsium said chipmakers should follow Nvidia’s lead in relying on DPUs to enforce security policies independently of the AI system itself, but with adequate protections for the DPUs as well. The security firm also recommended that businesses maintain accurate inventories of all the components in their AI systems, incorporate those components into vulnerability-handling processes and carefully scrutinize third-party software running on critical hardware like DPUs.
“Every AI infrastructure stack contains privileged components, each with its own firmware, software, update process, and supply chain,” Eclypsium researchers wrote. “Trusted AI therefore requires the ability to see and verify those components rather than assuming the model boundary begins at the container.”

——————————————————-
Click Here For The Original Source.

Related