Remix clone Hacker News

new | show | ask | jobs Github

▲

xyzsparetimexyz 2 days ago

Well yeah, it'd require the models to be loaded on the same system and the cache to be shared between them somehow

▲

inigyou 2 days ago | parent [-]

Cache sharing is not possible. The numbers in the cache are completely specific to the model.

▲

xyzsparetimexyz 2 days ago | parent [-]

Currently. I'm sure that you could make a system where the cache values are a superset C of e.g. models A and B where C is probably bigger than max(A,B) but smaller than A+B

▲

wonnage a day ago | parent | next [-]

Reddit post so take with a grain of salt but this does seem possible. But unclear whether this is actually a viable architecture https://www.reddit.com/r/LocalLLaMA/comments/1t8s83r/nvidia_...

▲

inigyou 2 days ago | parent | prev [-]

It's equal to A+B. There is literally no sharing possible.

	▲	xyzsparetimexyz a day ago \| parent [-]
		Even when training both models together in a novel way?