You can get something relatively capable and usable with your configuration and Strata: https://github.com/Niko1221/Strata

▲

Balinares 6 hours ago | parent [-]

Strata is intriguing and it definitely looks like there's something useful happening there with weight streaming at usable speeds, but I've not managed to have it serve a full answer without inference crashing before long. Pity.

	▲	MaxikCZ 3 hours ago \| parent [-]
		I dont think its strata problem, its been running on my system for days without any crash.