Caterpillar and CoreWeave shorten the learning loop for physical AI

Demand for new data centers, power plants and highways is fueling a construction boom, even as the industry wrestles with declining productivity and a shortage of skilled machine operators. Physical AI, which lets machines perceive their surroundings and act on them, is emerging as one answer.

Caterpillar Inc. has years of experience with autonomous equipment in mining. Extending that technology to construction requires systems that can adapt to more variable conditions, according to Brandon Hootman (pictured, right), vice president of physical AI platforms and construction autonomy at Caterpillar.

“Once that mine site gets instantiated, it does change, but it doesn’t change frequently,” he said. “You go to construction as an industry. Polar opposite of mining. If you think about all of the dynamics of what has to happen, taking a structured system and matching it to an unstructured environment, it’s really, really challenging to do.”

Hootman and Richard Ahlfeld (left), senior vice president of Physical AI at CoreWeave Inc., spoke with theCUBE Research’s Dave Vellante and John Furrier at the Fully Connected event, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed why physical AI reshapes the infrastructure behind autonomy and how the two companies are shortening the learning loop for construction machines. (* Disclosure below.)

Physical AI changes what data centers must deliver

AI clouds were first built to train foundation models, then tuned for agentic inference at scale. Training an autonomous excavator means ingesting telemetry and vision data, simulating a digging scenario a million times and layering reinforcement learning on top. CoreWeave recently launched a Physical AI Field Engineering service that embeds its engineers with customers’ domain experts, according to Ahlfeld.

“Physical AI now is an entirely different beast,” he said. “That requires, first of all, a lot of storage. It requires a different infrastructure.”

Caterpillar’s digital ecosystem already holds about 18 petabytes of federated data from machines, dealers and customers, Hootman noted. Training autonomous equipment also requires perception data synchronized with machine control and performance data.

“That 18 petabytes that we’ve collected so far is just a drop in the bucket to the amount of data that it actually takes to go train physical AI to operate in a world like a construction site,” he said. “But you take [Light Detection and Ranging] data, camera data, that multi-second control data and performance data coming back in, you’re talking about terabytes of data within a given day for just one machine.”

CoreWeave named Caterpillar among its enterprise customers in its second-quarter results announcement. Caterpillar began working with the company this year, drawn by both graphics processing unit capacity and applied expertise, according to Hootman. Working with Nvidia, the partners use AI models to annotate and label incoming field data.

“Now you’re taking things that were months to maybe weeks,” Hootman said. “And now you’re getting it down to hours and your feedback loop between that happening and being able to use it in your simulation environment, or your training environment, happens within the given workday.”

Here’s the complete video interview, part of SiliconANGLE’s and theCUBE’s coverage of the Fully Connected event:

(* Disclosure: TheCUBE is a paid media partner for the Fully Connected event. Neither CoreWeave, the sponsor of theCUBE’s event coverage, nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)

Photo: SiliconANGLE