Haven't read the paper yet, but it is interesting how seemingly simple many breakthroughs in ML are. Even transformers are like that. Maybe it's hindsight bias.

I suppose we just don't have a deeper underlying theory to lean on and help us 'design' anything.

▲

christophilus 14 hours ago | parent | next [-]

A lot of discoveries are like that. In fact, simplicity is often the hallmark of correctness, and complexity is often a sign that our understanding is incomplete and we’re still stumbling towards the right model. Not always, but often. It’s been a good rule of thumb in my programming career.

	▲	heeton 14 hours ago \| parent \| next [-]
		100%. I have a guiding approach when solving problems: keep reframing and exploring until the solution becomes obvious. I often find, if I've got a complicated solution, it’s because I haven’t fully examined the problem.
	▲	Teever 9 hours ago \| parent \| prev \| next [-]
		A designer knows he has achieved perfection not when there is nothing left to add, but when there is nothing left to take away. -- Antoine de Saint-Exupery
	▲	GandalfHN 9 hours ago \| parent \| prev \| next [-]
		[flagged]
	▲	GandalfHN 7 hours ago \| parent \| prev [-]
		[flagged]

▲

GandalfHN 8 hours ago | parent | prev [-]

[flagged]