It is a well-known trick, given that the timestep is between 0 to 1, you can slicing them at any resolution (1000, or 10000, give or take), and then keep a look-up table for modulation scale / bias etc for each. It is quite different from quantization and it is indeed lossless.

It is also only applicable to diffusion models as only these operates at per-timestep.

▲

xienze 3 hours ago | parent [-]

So... why didn't the model ship this way to begin with? They just wanted to waste VRAM for fun?

	▲	liuliu 3 hours ago \| parent \| next [-]
		They ship a complete checkpoint for easily management (inference & training) in their own infrastructure. Moving to a LUT would make training on these layers impossible. BTW, these are not useful for lightweight fine-tuning, but might still be useful if you do serious post-training work. Of course, these are also not an issue for things like FLUX.2 which adopts DiT-Air arch, that doesn't have this wasted space issue.
	▲	doctorpangloss 2 hours ago \| parent \| prev [-]
		It may or may not be true. The people who made this modification and the other commenters didn't do anything rigorous to verify what they did. They just eyeball it. They could very well make some other error - this has happened frequently - that developing on prod, not knowing what they are doing, has and hopefully will again solve.