Google DeepMind released DiffusionGemma today, an open 26B Mixture of Experts model that generates text up to 4x faster than autoregressive Gemini and Gemma 4 models on dedicated GPUs. The model clocks 1,000+ tokens per second on a single Nvidia H100 and 700+ tokens per second on a GeForce RTX 5090, numbers achieved by abandoning the token-by-token decoding that every mainstream LLM relies on. Instead, DiffusionGemma drafts 256 tokens in parallel on each forward pass and iteratively refines them, the same broad approach diffusion models use for images.
The model activates 3.8B of its 26B parameters during inference and, when quantized, fits inside the 18GB VRAM ceiling of high-end consumer cards. That makes a single RTX 5090 or 4090 a viable target. DeepMind worked with Nvidia on NVFP4 4-bit kernels for Hopper and Blackwell, and certified deployment on DGX Spark, DGX Station, and RTX PRO workstations alongside cloud delivery through Gemini Enterprise Agent Platform Model Garden and Nvidia NIM.
DeepMind is positioning DiffusionGemma as an experimental release rather than a Gemma 4 replacement. The blog post, co-authored by research scientists Brendan O'Donoghue and Sebastian Flennerhag, frames it as a tool for in-line editing, code infilling, and other low-latency interactive workloads where the standard left-to-right typewriter model leaves a local GPU mostly idle waiting for the next keystroke.
“DiffusionGemma is designed for researchers and developers exploring speed-critical, interactive local workflows such as in-line editing, rapid iteration, and generating non-linear text structures.”— Brendan O'Donoghue, Research Scientist, Google DeepMind
Key facts
- 01DiffusionGemma is a 26B Mixture of Experts model with 3.8B active parameters, released under Apache 2.0.
- 02It generates 1,000+ tokens per second on an Nvidia H100 and 700+ tokens per second on a GeForce RTX 5090.
- 03The model drafts 256 tokens in parallel per forward pass instead of one token at a time.
- 04Quantized, it fits inside the 18GB VRAM ceiling of high-end consumer GPUs.
- 05Google explicitly says output quality trails standard Gemma 4 and recommends Gemma 4 for production.
The speed comes from a structural shift in how the model uses the hardware. Autoregressive decoding is memory-bandwidth bound on a single accelerator at low batch sizes, which is the common case for local inference. DiffusionGemma converts that into a compute-bound workload by giving the GPU a full 256-token block to chew on at once. DeepMind compares the shift to moving from a typewriter to a printing press.
Bi-directional attention is the other architectural payoff. Because every token in the 256-token block can attend to every other token during generation, the model handles tasks where future context matters as much as past context. DeepMind highlights a fine-tuned version from Unsloth that solves Sudoku, a problem autoregressive models stumble on because each square's correct value depends on squares the model has not yet generated. Code infilling, amino acid sequences, and mathematical graphs land in the same category.
The trade-off is quality. DeepMind is direct about it: DiffusionGemma's output quality is lower than standard Gemma 4, and the company recommends Gemma 4 for any application where quality outranks latency. The throughput advantage also narrows or disappears in high-QPS cloud serving, where autoregressive models batch user requests together and saturate compute on their own. Parallel decoding is a local and low-concurrency play.
“For applications that demand maximum quality, we recommend deploying standard Gemma 4.”— Sebastian Flennerhag, Research Scientist, Google DeepMind
Tooling support is broad at launch. Weights are on Hugging Face under Apache 2.0. Serving works on MLX, vLLM with Red Hat integration, and Hugging Face Transformers. Fine-tuning is supported through Hackable Diffusion, a JAX toolbox DeepMind is releasing alongside the model, plus Unsloth and Nvidia NeMo. llama.cpp support is on the way.
DiffusionGemma builds on research from Gemini Diffusion, which Google has signaled for over a year as its bet on parallel text generation. Text diffusion has been a research target across the field for years, but most attempts have stalled at small parameter counts or narrow domains. Shipping a 26B MoE under a permissive license with first-party Nvidia kernel work is the most concrete step any major lab has taken toward making the approach usable outside a paper.
The honesty about quality matters. DeepMind could have buried the Gemma 4 comparison; instead it sits in the launch post. That points to where this model is actually aimed: developers building latency-sensitive features who can tolerate weaker single-shot outputs in exchange for sub-second generation on a workstation. Speculative decoding and other autoregressive tricks have eaten into the latency gap from the other direction, and it remains unclear whether parallel diffusion will hold its lead once those techniques mature further on the same hardware.
For the broader market, DiffusionGemma is most interesting as a pressure test on the assumption that autoregressive transformers are the only path to useful text generation at scale. If the diffusion approach holds up in fine-tuned production workloads — coding assistants, real-time editors, structured-output tools where the 4x speedup is felt directly by the user — it gives Google a genuine differentiator on local and edge deployments, exactly the segments where OpenAI and Anthropic have no consumer-hardware story. The next signal worth watching is whether third-party developers ship anything on top of it that Gemma 4 cannot match on latency.
Working on something we should cover, or seeing a story we missed? Send leads, documents, or feedback to hello@aichatdaily.com. For sensitive tips, see our secure tips page for Signal and PGP options.
Spotted an error? Email hello@aichatdaily.com with the URL and the issue, or read our full corrections policy.
