Confirming rumors, Anthropic reveals plan to develop custom chip

Anthropic PBC today disclosed that it plans to develop a custom artificial intelligence chip.

A spokesperson told Business Insider that the company will co-design the processor with its future large language models. Usually, co-design initiatives focus on tailoring a chip to a specific workload. Such customization can significantly increase hardware efficiency.

Off-the-shelf AI chips’ specifications are often mismatched with the models they run. For example, an accelerator might feature slightly more memory than an LLM requires or slightly less. Co-designing processors with the software they run removes such inefficiencies.

According to Business Insider, Anthropic is assembling an in-house chip development team to lead its semiconductor push. The company is reportedly seeing processor designers and verification experts. Verification is the process of ensuring that a chip design will work as expected.

A job posting indicates that Anthropic plans to automate some verification tasks using AI. According to the listing, the company plans to develop simulations in which Claude can learn how to test newly developed chip designs. The effort will place particular emphasis on an evaluation method called formal valuation. It checks a chip design for flaws by simulating every combination of operating conditions in which it will run.

Semiconductor engineers also use a variety of other verification methods. During the initial phase of the testing process, they run a virtual version of their chip in a simulation. They later implement it in an FPGA, or field-programmable gate array. An FPGA is a chip that can mimic other processors at the hardware level, which enables it to provide more detailed telemetry than a simulation.

OpenAI Group is also using AI to accelerate its chip design efforts. In June, it debuted a custom inference accelerator called Jalapeño that was developed through a collaboration with Broadcom Inc. The chip took only nine months to design because the companies automated a number of manual tasks with AI.

It’s likely that Anthropic’s chip will be optimized for inference workloads much like Jalapeño. Inference, or the task of running LLMs in production once training is complete, often represents AI providers’ biggest infrastructure expense. Custom silicon can be much more cost-efficient than off-the-shelf graphics cards.

Rumors of Anthropic’s chip design effort first emerged in April. In early June, The Information reported that the company may partner with Samsung Electronics Co. to manufacture its processor.

Samsung has a much smaller share of the contract chipmaking market than Taiwan Semiconductor Manufacturing Co. However, it recently introduced a technology called zHBM that could significantly increase the efficiency of AI chips. Partnering with Samsung may enable Anthropic to incorporate zHBM into its upcoming accelerator.

A graphics card’s logic cores and HBM memory sit next to one another on a shared base layer, or substrate. Samsung says that its zHBM technology makes it possible to place memory directly atop logic cores. That arrangement reduces the distance data must travel between memory and logic circuits, which in turn lowers power use.

Image: Anthropic