-advertisement-

Etched, a startup specializing in transformer-focused chips, unveiled Sohu on June 25. This ASIC claims to outperform Nvidia's H100 in AI large language model (LLM) inference. According to reports, an 8xSohu server matches the performance of 160 H100 GPUs, potentially reducing both initial and operational costs for data centers.

Image credit: Etched

Current AI accelerators, whether CPUs or GPUs, support various AI architectures (CNNs, LSTMs, state space models, etc.), necessitating significant computing power for programmability. Etched notes that Nvidia's H100 GPUs use only 3.3% of their transistors for matrix multiplication, a key task for most LLMs, with the remaining 96.7% allocated to other necessary functions for general-purpose AI chips.

Transformer AI architectures, like that of ChatGPT, have recently surged in popularity. Etched anticipated this trend a couple of years ago and began the Sohu project to create a chip specifically for transformer models, optimizing transistor use for AI compute tasks. This specialization is akin to how GPUs handle graphics more efficiently than CPUs.

Instead of making a chip that can accommodate every single AI architecture, Etched built one that only works with transformer models. When it started the project in 2022, ChatGPT didn’t even exist. But then it exploded in popularity in 2023, and the company’s gamble now looks like it is about to pay off — big time, especially as transformer models, including ChatGPT, Sora, Gemini, Stable Diffusion, and DALL-E, dominate the AI landscape.

Nvidia, a leading AI GPU provider, saw record revenues and shipped 3.76M data center GPUs in 2023. However, Sohu could challenge Nvidia's dominance, especially for transformer-exclusive companies. Efficiency is crucial in AI, and those running models on the fastest, most cost-effective hardware will lead.

AI data centers' power consumption has raised concerns, with figures like Mark Zuckerberg and the U.S. government discussing potential power crises. Last year's GPU sales consumed more power than 1.3M homes. Etched's Sohu could help manage AI power demands more sustainably, aiding the electricity grid in meeting growing computing needs.

Editor：Lulu

▼▼▼

S. Korea to launch $13 bn financial support for chip industry in July

TSMC plans third 2nm fab at Nanzih Technology Industrial Park

Rumors are spreading that ByteDance is collaborating with Broadcom to develop AI chips

Samsung confirms further delay to its Texas fab

Vishay tops out its second factory in Itzehoe

NSIG to spend $1.8 billion on silicon wafer production