> To resolve this, we propose a parallel multimodal diffusion framework, MMaDA-Parallel, that enables continuous, bidirectional interaction between text and images throughout the entire denoising trajectory.

> (ParaRL), a novel strategy that applies semantic rewards along the trajectory to enforce cross-modal consistency.

(emphasis mine)

This sounds really cool. The fact that one generation "attends" to the other is really interesting. I'm curious if this would hold for other modalities. I'm thinking coding specific applications, where things can change once something is generated. My hunch is that coding would benefit a lot from this approach, because the "manual" way of writing code often resembles diffusion more than autoregressive (that is, we often edit something here, then because we did that we have to import something, then change something there, then that leads to further changes, etc).

For now coding seems to benefit a lot from <thinking> -> <coding> -> <env_feedback> -> <reflexion> -> <thinking> -> <coding>, but this seems at a glance to be shoehorned in for autoregressive generation... GPT5 in particular seems to be better at this, with multiple "tool calls" interleaved in its thinking sessions. I wonder if this would get better with the paralel denoising thing proposed here, where both thinking and coding are done in paralel, and one can "attend" to the other. Add some feedback (linters, compilers, LSPs, tests, etc.) and this can go places. If it works.

▲

soulofmischief 11 hours ago | parent [-]

Diffusion text models aren't new, I've made them at home. Also, plenty of frontier models are good at tool calling, GPT-5 has just been trained to do it more so that it appears to do better at coding exercises with codex/IDEs.

If you haven't tried an agentic IDE such as Cursor yet, or at least an extension such as Copilot, I would recommend checking them out and trying out Anthropic's models as well.

▲

NitpickLawyer 9 hours ago | parent [-]

Do you have any examples / papers where they do the parallel thing proposed here? I've tried googles diffusion coding model, but AFAICT they don't do parallel thinking & coding. It seems to just take a prompt and output code.

What's cool with this thinking & generation in parallel is that one can attend to the other. So you're not limited by prompt influences code, but can do prompt influences both thinking and code, and code can influence thinking and thinking can influence code.

▲

lossolo 7 hours ago | parent [-]

They use bidirectional attention between modalities, not within the same modality. This doesn’t change much in the context you're referring to (coding). How do you think "thinking" works in current SOTA models like GPT-5-Thinking/Pro? When generating code, the model's "thinking" already attends to the code, and both influence each other during generation. "Reasoning" models modify the code as they generate it, they delete it, revise it, and adjust their internal reasoning based on the new tokens they produce during the "thinking" process. There are dozens of denoising models created for text, they are not good at it and parallel sampling between modalities will not change that.

▲

ricardobeat 4 hours ago | parent [-]

They cannot "edit" the code though, like you can with diffusion. They must emit all tokens again, or a patch/diff which is not directly connected to the previous stream of tokens.

	▲	lossolo 2 hours ago \| parent [-]
		LLMs can "edit" code, but as you say, they do it differently from diffusion models. They operate directly on long text sequences and use much more context, which is one reason they currently work better for coding. Diffusion models for code aren't a new idea, people have tried different designs, but so far they tend to underperform autoregressive LLMs, probably because denoising over discrete tokens is harder to make work than straightforward next token prediction.