Google open-sources speedy DiffusionGemma text diffusion model

Google LLC today released DiffusionGemma, a large language model based on an emerging machine learning approach known as text diffusion.

The company says the algorithm can generate text four times faster than traditional LLMs. Furthermore, DiffusionGemma does so using less RAM. The model’s memory efficiency enables it to run on high-end consumer graphics cards that usually struggle to support LLMs.

DiffusionGemma’s text diffusion architecture is derived from a method that AI models use to generate images. The image generation workflow begins with a blurry photo that contains a type of error called Gaussian noise. An AI model removes a small portion of the noise, analyzes the enhanced photo and uses its findings to restore another batch of pixels. It then repeats the process until arriving at a usable image. 

When DiffusionGemma receives a prompt, it generates a placeholder response that comprises random words. It then replaces a subset of the random text with words that will form part of its answer to the user’s prompt. DiffusionGemma reviews the edits, generates a few more words and repeats the process until its prompt response is ready.

AI models usually generate prompt responses one token at a time. DiffusionGemma’s text diffusion architecture, by contrast, enables it to produce 256 tokens at once. That parallelization is what makes the model faster than standard LLMs.

Google says that DiffusionGemma can generate more than 1,000 tokens per second when running on a single H100, a server-grade GPU that Nvidia Corp. launched in 2022. The model can generate over 700 tokens per second on the chipmaker’s desktop-grade GeForce RTX 5090 chip.

One reason DiffusionGemma can run on consumer GPUs is that it’s based on a mixture-of-experts architecture. The model includes 26 billion parameters but activates only 3.8 billion of them to answer the prompt, which lowers memory usage. DiffusionGemma further lowers RAM consumption by keeping information in a lightweight data format called NVFP4.

DiffusionGemma is based on an LLM called Gemma 4 26B A4B that Google released in April. To facilitate text diffusion, the search giant replaced the latter model’s attention mechanism, the software module it uses to interpret prompts. The original mechanism inferred the meaning of each word in a prompt by analyzing the preceding text. The new attention module also reviews the text that follows a given word.

“While the AI research community has explored diffusion-based text generation for years, applying it to large models has remained a challenge,” Google research scientists Brendan O’Donoghue and Sebastian Flennerhag wrote in a blog post today. “DiffusionGemma changes this by shifting how models use hardware.”

DiffusionGemma is available on Hugging Face under an open-source license.

Image: Google