Remix.run Logo
QuadmasterXLII 3 hours ago

headline hundred billion parameter, none of the official models are over 10 billion parameters. Curious.

Tuna-Fish 3 hours ago | parent [-]

The project is an inference framework which should support 100B parameter model at 5-7tok/s on CPU. No one has quantized a 100B parameter model to 1 trit, but this existing is an incentive for someone to do so.