I started creating internal models using it, then the Gated Deltanet 2 came out( https://arxiv.org/abs/2605.22791), and it seems like an evolution of it in expressiveness. And in our tests it is really better than.

▲

iandanforth 3 hours ago | parent [-]

Is it just me or does this read like a re-implementation of LSTMs?

	▲	muricula an hour ago \| parent [-]
		I'm no expert but it seems like a descendent of LSTMs. There's a series of papers which show how to reformulate attention as RNNs which arrives at linear attention. Then they add a decay term to get mamba2. Then they add modified the decay term as like a scale to apply both to the existing state and the new update to get delta net. Then they added a gate matrix on the output to get gated delta net. Then Kimi Linear Attention seems to be gated delta net with a more expressive gate. The Gated DeltaNet paper recaptilulates this evolution decently well. But yeah, it feels like they're starting with the same lego blocks and assembling them in similar shapes to accomplish similar but slightly distinct modules.