Remix.run Logo
woopsn 2 days ago

It's a little arbitrary. Look at the graph on page 6, there's no steep gap in the spectrum there. 16 just about the balance point

moi2388 2 days ago | parent [-]

But there is a steep gap in the spectrum at 16 on page 7

yorwba 16 hours ago | parent [-]

That's the spectrum of LoRAs, which are LoW RAnk by design.

moi2388 7 hours ago | parent [-]

Yes. But from their paper: “In our analysis, we present compelling empirical evidence for the existence of universal subspaces within LoRA adapters across different modalities and tasks.”

I also don’t understand what they write under figure 2, since resnet50 has 50 layers, not 31.