Remix.run Logo
▲ dist-epoch 5 hours ago

I think it also addresses the issue that there is a finite amount of computation you can do in one pass.

So you use one pass to compute one relevant word, let's say you are working on a sound analysis task, and you output "FFT".

Now in the second pass, FFT is right there at the input, so you can push that idea forward more until you are out of layers again and then you use the output token to prime the next pass.