Remix.run Logo
onion2k 2 hours ago

The G in GPT stands for Generalized. You don't need that for specialist models, so the size can be much smaller. Even coding models are quite general as they don't focus on a language or a domain. I imagine a model specifically for something like React could be very effective with a couple of billion parameters, especially if it was a distill of a more general model.

christkv 31 minutes ago | parent | next [-]

Thats what i want and orchestrator model that operates with a small context and then very specialized small models for react etc

MzxgckZtNqX5i 2 hours ago | parent | prev [-]

I'll be that guy: the "G" in GPT stands for "Generative".