IBM inks $240M infrastructure deal with AI-optimized cloud operator Together AI

IBM Corp. will provide infrastructure to artificial intelligence startup Together AI Inc. as part of a $240 million deal announced today.

The partnership comes a few weeks after the latter company raised $800 million in funding. The consortium that provided the capital included Nvidia Corp., whose chips IBM will use to run Together AI’s workloads. 

San Francisco-based Together AI operates a cloud platform optimized for machine learning. The platform runs on infrastructure that the company rents from partners such as Amazon Web Services Inc. Developers can use it to train custom AI models, fine-tune existing ones and perform inference. IBM disclosed today that the platform processes about 400 trillion tokens per month for customers.

The infrastructure deal announced today will see the tech giant deploy a “large cluster” of HGX B300 systems for Together AI. The hardware will become available through IBM Cloud in the first half of 2027.

The HGX B300 is a motherboard equipped with eight Blackwell Ultra chips. The Blackwell Ultra was Nvidia’s most advanced graphics card until the introduction of the Rubin in January. It can provide 15 petaflops of performance when processing data in the NVFP4 format.

The Blackwell Ultra comprises two chiplets linked together by a custom interconnect. They host 160 computing modules called streaming multiprocessors, which each comprise more than 130 cores. They also contain memory optimized to store intermediate results, temporary data that AI models generate while processing prompts.

The HGX B300 uses eight ConnectX-8 SuperNIC chips to move data between its Blackwell Ultra accelerators. A BlueField-3 data processing unit, or DPU, coordinates traffic to and from external systems.

Hardware makers use the HGX B300 to build AI servers that also include contain components. According to Nvidia, the machines include at least two terabytes of memory and a pair of central processing units with 56 cores apiece. Those CPUs connect to two PCIe switches that enable users to attach several terabytes NVME flash memory.

Together AI will use IBM’s HGX B300 cluster to host workloads for customers of its public cloud. In particular, it will run inference workloads that use open-source AI models.

The AI startup offers five different inference services as part of its platform. One is optimized for media generation models that run in containers. Another service, Provisioned Throughput, is charged by the token to help customers optimize their costs. Together AI’s other inference offerings provide serverless infrastructure, dedicated machines and a batch processing environment that trades off some performance for a price reduction. 

“This cluster lets us bring production-grade inference to more companies, faster, and it’s a big step in our push to make open-source AI the obvious choice for enterprises,” said Together AI Chief Executive Vipul Ved Prakash.

Image: IBM