DiffusionGemma Technical Report

kamranjon 3 hours ago | parent | next [-]

Just wanted to share this, I found it was a really nice resource to understand how diffusion Gemma worked: https://newsletter.maartengrootendorst.com/p/a-visual-guide-...

The really interesting thing to me was that they didn’t need to train this model from scratch they just used their existing MOE checkpoint:

“To convert a decoder-only model (Gemma 4 26B A4B) into a denoiser, we can make use of something it is not directly using when generating tokens, namely the logits of all tokens!”

What makes me hopeful about this release is that possibly this same conversion can be applied to other open models and we might see a bunch of diffusion versions of existing local models. It’s exciting stuff!

	▲	jerpint an hour ago \| parent [-]
		Oh wow that’s really cool!

▲

mmastrac 26 minutes ago | parent | prev | next [-]

I re-implemented this one for macOS over the last couple of months: https://github.com/mmastrac/diffgemma

I like the model a lot and it's fairly good at reasoning. You can also really bend it to your needs. It's designed for machines with more compute than memory bandwidth but IMO does really well on metal.

I've got it up to ~15tok/s on M3-class machines, but I wager there's a bunch of perf on M5 that I just don't have hardware access to unlock.

I tried to implement MTP using the other Gemma MTP heads but I failed to move that perf needle. There's some interesting research to be done about pre-seeding the diffusion canvas from draft models. DiffusionGemma with the right drafter can hit 20-30 tok/s on my machine, but I've been unable to combine the two together to make it faster than what it's been running at so far.

▲

jermaustin1 3 hours ago | parent | prev | next [-]

I'm very interested in Diffusion text models. The concept of taking noise and adding words starting randomly all over the response, and filling in the noise from there on breaks my brain.

I'm sure I have a fundamental misunderstanding of the technology, though.

▲

embedding-shape 3 hours ago | parent | next [-]

DiffusionGemma goes one step further even, and does this denoising over multiple "canvases" which lets it do reasoning and separate out a "final reply" canvas, looks something like this: https://gist.github.com/embedding-shapes/f4cb46bad704b6d0168...

Diffusion text models for me is the more interesting type of LLMs for local usage, as it really makes good use of single GPUs for single responses, rather than auto-regressive ones, and is a lot faster! Probably the fastest model I've been able to run so far, ending up doing ~670 tok/s (depending on the type of text) on a Pro 6000

▲

moffkalast 2 hours ago | parent | prev [-]

How does that break your brain? It's how basically every human writes and iterates on text..?

▲

jermaustin1 2 hours ago | parent [-]

Because my brain thinks through text in a forward motion. Pausing at the end of each word and searching for the next.

My entire brain runs on sentences and words since I have no inner eye or whatever. So my thinking and writing both work kind of forward only.

I wouldn’t have thought that was too unique. But maybe it is?

	▲	pebbly_bread an hour ago \| parent [-]
		Normally people have feelings about things before they are able to put them into words, I would imagine if you were asked a question like "what city would you most like to visit" then unless you've already thought about it a lot, then you would have to do substantial non-verbal thinking before you can come up with an answer, and once you have the answer you may respond "my favorite city is X" and you decided what X would be before you started the sentence.

▲

anentropic 2 hours ago | parent | prev | next [-]

Appealing results... do we think there is scope to close the accuracy gap against AR models? or even leverage the "Bidirectional Reasoning and Self-Correction" into an overall advantage?

▲

keel-control 3 hours ago | parent | prev [-]

there's still JEPA to be integrated before AGI.

Would DiffusionGemma be suitable candidate for DFlash 2?

▲

jatora 14 minutes ago | parent | next [-]

Has JEPA shown a shred of viability yet?

▲

discobot2 2 hours ago | parent | prev [-]

its more of a competitive approach to improve compute utilisation at lower batch sizes

	▲	Tostino an hour ago \| parent [-]
		(to finish your thought) Which is important for consumer hardware to be better suited to running these models. Cloud providers are already able to batch as many requests as they want together to improve resource utilization, so they will not see a big benefit from diffusion models.