Most people who’ve used an AI chatbot have met the same underlying trick: the model predicts one word at a time, left to right, until it finishes. Image generators work differently. They start with static and clean it up step by step until a picture emerges. A group of researchers and startups are now asking whether that second approach, called diffusion, can write code and prose too.
It’s a fair question and a useful way to understand how generative AI works at all.
The two recipes
Autoregressive generation
This is the standard approach for chat models. The model looks at everything written so far, picks the most likely next token (a word or word piece), adds it, and repeats. It’s like writing a sentence while only ever being allowed to look backward.
It works remarkably well, but it has built-in limits. Each token depends on the previous one, so you can’t easily produce many tokens at once. Speed is tied to length.
Diffusion generation
Diffusion starts with a rough, noisy version of the whole output and refines it in several passes. For images, noise means random pixels. For text, researchers use different tricks, such as masking out tokens and then gradually filling them in, or working in a continuous representation of words.
The key difference is that the model works on the whole output at once. In each pass it can adjust many parts in parallel.
Why anyone would bother
There are three arguments in favor of trying diffusion for language.
- Speed. Refining many tokens at the same time can be faster than producing them one by one, especially for long outputs. For code completion, where latency is felt immediately, that matters.
- Editing and revision. A diffusion model can revisit earlier parts of the output. A left-to-right model commits to each choice and can’t easily go back, which is why it sometimes paints itself into a corner.
- Global coherence. Code has structure: brackets must match, variables must be defined, functions must agree with how they’re called. Looking at the whole thing at once might help keep that structure consistent.
The catches
None of this is solved. Text is made of discrete tokens, and diffusion was originally built for smooth, continuous data like pixels. Adapting it takes cleverness, and early systems have often lagged behind the best standard models in quality.
Training recipes are also less mature. The autoregressive approach has years of accumulated tooling, tuning knowledge, and hardware optimization. A newcomer has to catch up on all of it while proving a clear benefit.
Another open question is controlling the number of refinement steps. More steps usually mean better results but slower output, which eats into the speed advantage. The trade-off is a tuning problem teams are still working on.
Why code is a natural testing ground
Code is a good proving ground for a few reasons. You can check it objectively: it compiles or it doesn’t, tests pass or fail. That gives researchers clear feedback, unlike essays, where quality is subjective.
Developers also care a lot about response time. If a suggestion appears instantly while you type, you use it. If it lags, you ignore it. A model that fills in a function body quickly could feel better in an editor even if it’s only equal on quality.
And code involves a lot of editing in the middle of existing text, such as inserting a line between two others. Diffusion-style models can condition on text both before and after a gap, which fits that task nicely.
What to watch for
- Independent benchmark results on coding tasks, not just company demos.
- Whether speed gains hold at longer outputs and real workloads.
- Hybrid approaches that combine both methods. Many researchers expect the answer won’t be either-or.
- Cost per request, since hardware efficiency decides whether a technique makes business sense.
A reasonable take
It would be premature to say diffusion will replace the models behind today’s chatbots. The incumbent method is strong and keeps improving. But the idea has real merit, and healthy competition between approaches is how the field finds better answers. If diffusion models can deliver comparable quality at noticeably lower latency, they’ll earn a place, particularly in tools where speed is felt, like coding assistants.
For now, treat bold claims with patience. Look for open evaluations, try the products yourself, and remember that a clever architecture only matters if people can feel the difference when they use it.
Featured image: Dülmen, Münsterstraße, Viktorkirmes, Riesenrad — 2023 — 9053 (kreativ 2).jpg by Dietmar Rabich, CC BY-SA 4.0, via Wikimedia Commons.
