Google Cloud introduces Agent Compute Engine to host AI agents

Google LLC’s cloud unit today introduced Agent Compute Engine, an infrastructure service optimized to host artificial intelligence agents.

AI agents run in technology environments called sandboxes. They host the applications, data and user instructions that agents need to carry out their work. Sandboxes also contain cybersecurity guardrails that block malicious activity.

Google refers to Agent Compute Engine’s implementation of sandboxes as microVMs. The company says that the technology combines the hardware efficiency of containers with the security of traditional virtualization. In particular, Google brought over hypervisors’ ability to isolate virtual machines from one another to prevent the spread of malware.

The company runs microVMs on specially configured appliances. The appliances are stateless, which means that they don’t permanently retain the files used by customers’ agents. The data that the systems do store is kept in an immutable file system that can’t be modified or deleted.

Google verifies the integrity of the appliances’ firmware using its custom Titan cybersecurity chips. The appliances’ central processing units, in turn, feature mitigations against side-channel attacks. Those are cyberattacks that analyze data about a CPU’s physical state to glean the customer information it processes.

The microVMs that run on an Agent Compute Engine appliance have only limited access to its storage resources. As a result, hackers can’t read or delete data if they take over one of the sandboxes. Another set of guardrails protects the network that connects the appliance to the rest of the host data center. Traffic from microVMs must go through multiple proxies before it can proceed to its destination.

“All inbound and outbound traffic must traverse an isolated, point-to-point IPC boundary,” Google Cloud distinguished engineers Ramanathan Venkatapathy and Rohit Jnagal explained in a blog post. “This boundary terminates at a local, unprivileged mediation proxy, which forwards packets to an external proxy. The external gateway then validates request scopes, and enforces strict, policy-driven outbound brokering.”

AI applications often pause the agents they use to automate tasks for users. When an agent shuts down, the sandbox that hosts it must remain available because the agent may be reactivated later. Such idle sandboxes can consume a significant amount of hardware resources, which increases costs.

Google says Agent Compute Engine detects when a sandbox goes idle and shuts it down. That prevents unnecessary hardware usage. When the sandbox is needed, the service quickly reactivates it.

“This lets a single physical node host orders of magnitude more living agent microVMs than conventional VMs,” Venkatapathy and Jnagal wrote. “We couple this with features integral to agents like cloning, snapshot/forking, rewind and rollback that allow the harnesses to use the system efficiently without resource duplication.”

Some agents use login credentials to access the applications with which they perform their work. According to Google, Agent Compute Engine stores those credentials outside the sandbox that hosts an agent. That means hackers can’t steal any sensitive data if they take over the sandbox. When an agent needs to log into an application, a Google Cloud service called Agent Gateway retrieves the relevant credentials and submit them on the agent’s behalf.

Google will roll out Agent Compute Engine in the coming months alongside a new version of Agent Gateway. The latter service, which helps customers block insecure agent traffic, is set to gain a model routing capability. The feature will enable it to send each agent request to the model that can process it in the most efficient manner. Agent Gateway is also gaining a tool for regulating agents’ inference costs.

“Token management provides control over token consumption across agents and models, allowing you to attribute spend, enforce budget guardrails, and provide a single source of truth for token metrics,” Google Cloud vice presidents Nirav Mehta, Michael Bachman and Kevin Hutchins wrote in a blog post.

Image: Google