Published on
Nvidia is rolling out a security platform for artificial intelligence (AI) agents that more than 100 organisations are working with, including Anthropic, SpaceXAI and JPMorganChase.
Called the Nvidia Open Agent Safety Platform, it is designed to set limits on what agents can access and put them in “quarantine” within milliseconds if they cross those limits, according to an announcement from the US chipmaker.
The platform can be used across software and hardware, compute and robotics systems that run agents.
“AI’s extraordinary potential for society will only be realised if we solve AI safety,” said Jensen Huang, the CEO and founder of Nvidia.
“As we continue to discover the frontier of AI capabilities, we must accelerate discovery at the frontier of AI safety…. Together, we can raise the bar for global AI safety.”
The Nvidia Open Agent Safety Platform consists of two tools, OpenShell and Sentry.
OpenShell runs on central processing units (CPUs), the chips that handle many of the tasks needed to run AI agents. Nvidia says it works efficiently on Vera, its latest CPU designed for AI agents, and can also be adapted for chips made by other companies.
Sentry is an additional “watchdog” system that would run on a separate Nvidia chip, watching for agents that get past the software’s controls.
The organisations working with Nvidia are using or developing the technology in different ways, Nvidia says.
Anthropic has worked with Nvidia to add security controls to its Claude service for businesses to run AI agents, allowing companies to restrict what those agents can access.
SpaceX’s AI division, SpaceXAI, says it is using the platform for Cursor coding agents and Grok models, while Salesforce has integrated OpenShell with Slack so teams can see what agents are doing and approve or reject requests for additional permissions.
In finance, JPMorganChase and Citi are collaborating with Nvidia on shared open-source agent safety technology.
AI companies call for stronger safeguards
The announcement comes amid growing calls for stronger safeguards around AI agents.
In July, OpenAI admitted one of its models exploited a hidden flaw to escape a controlled test and break into Hugging Face’s servers.
Meanwhile, Anthropic said its AI model hacked into three organisations’ systems during the testing phase.
Nvidia said the incidents showed that agents could get around security controls built into an app while trying to complete their assigned tasks and that “safety and security require full-stack engineering”.