Inference chip startup Etched launches with $800M in funding

Etched Inc., a developer of artificial intelligence inference chips, launched today with $800 million in funding.

The startup raised the capital over multiple rounds. The most recent investment, which closed in December, valued Etched at $5 billion. It included the participation of VentureTech Alliance, a startup fund associated with Taiwan Semiconductor Manufacturing Co. It was joined by more than a dozen other backers, including Geoffrey Hinton, Fei-Fei Li and Andrej Karpathy.

Etched plans to produce its inference chips using TSMC’s N4P process. It’s an enhanced version of the chip giant’s five-nanometer node that provides 11% better performance than the original. According to Etch, its first prototype chips rolled off TMSC’s N4P production line earlier this year.

Nvidia Corp.’s flagship Rubin graphics processing unit is optimized for both AI training and inference. Focusing solely on inference, the approach that Etched has taken, makes it possible to reduce power usage by removing training-optimized circuits. Alternatively, engineers can add more inference circuitry to boost processing speeds.

Etched has also equipped its chip with several other performance optimizations.

The more calculations a GPU performs per second, the more power it draws, which in turn increases its operating temperature. Past a certain threshold, the extra heat can cause malfunctions. Graphics cards mitigate the issue by lowering their clock rate when they start approaching their peak speeds. Thermal throttling, as the practice is known, slows down inference. 

Etched has developed a technology called LVI that reduces the need for thermal throttling. According to the company, its chip can run a trillion-parameter AI model at “80%+ peak FLOPs” without lowering its clock rate. The result is a significant inference speedup. Etched says that its chip’s FLOP density, a measure of performance, is several times higher than the existing AI processors on the market.

The company plans to ship its silicon as part of a rack-scale inference appliance. The system features multiple chips installed on custom circuit boards. Etched has also developed custom cold plates, components that play an important role in liquid cooling systems. A cold plate is a flat piece of metal that channels heat generated by chips into a rack’s coolant.

Etched’s appliance includes a mix of SRAM and HBM memory. AI chips use SRAM, the fastest RAM variety on the market, to store their workloads’ most important data. Other information is sent to HBM memory, which trades off some speed for significantly increased capacity.

The AI chips in a rack often require the ability to access data in one another’s memory. Etched’s appliance uses a custom interconnect to facilitate such data movement. According to the company, the machine features a system-wide shared memory pool that can process requests with less latency than earlier technologies.

Etched is currently in the process of ramping up chip production and plans to ship its first racks this summer. The company disclosed today that it has received more than $1 billion worth of customer orders.

Image: Etched