Remix.run Logo
fodkodrasz 2 hours ago

> I wonder, is there a way to make the RAM usage tunable?

In LM Studio I can tune it by selecting different quantation of the model, by selecting how many layers of the neural net to be loaded to GPU (rest stays in main mem, evaluated by the CPU), and by adjusting context window.