| ▲ | rs38 7 hours ago | |
is there a realistic way to distill 2 consumer hardware friendly models with max ~200B and ~20B? Qwen did it, but would it be possible for 3rd parties (unsloth etc)? | ||
| ▲ | embedding-shape 7 hours ago | parent [-] | |
Yeah, why not. Toughest part is running the hardware so you can create the traces for downstream training, but once over that hump, nothing would stop you from doing that no. | ||