Probably need to be taught by someone of Latino origin. Learning to roll them "r"s could help.

I always thought this should be easier by telling it to write each letter one a new line, then count, any token separator ought to suffice, so much so you'd think they'd have trained in this strategy given tokens make individual letters opaque

Because the open model might have been distilled from outputs of the closed model doesn’t mean they are architecturally equivalent. There is almost certainly innovations in architecture present in the open models that the closed labs didn’t think of. It also is almost certainly true that they aren’t completely built out of a distilled corpus, that reinforcement is equivalent, etc. Therefore closed labs will also benefit from being able to inspect in totality the architecture, activations, weights, and be able to train against it at scale in an ensemble of other models and their internal work.

The only way there is no benefit would be is if the open models are literal copies of the closed model, which unless there was direct theft, is highly improbable.

When I say distill I also mean mine it architecturally for insights but I doubt seriously the model training is entirely distillation of Claude, it’s almost certainly a mixture of both original corpus and reinforcement as well as distillation. I think it’s a little condescending to imply that these new open models are cheap ripoffs with nothing original to them. These teams and labs are top tier as well, working under unreasonable constraints imposed by the USG. That’s a powerful combination for creativity.

If weights are the source for models then ELF binaries are the source for software.

I want to caution against this line of thinking. NSA's fast16 program silently altered data during nuclear simulations, and that is also possible within open weight models. There is no reason to think there is anything like that currently, it also cannot be dismissed. Open source would include the training data so you could create a similarly capable model.

The weights are the output artifact, the training corpus and system are the source.

What to do with $20 on Fable? One prompt or two ?

I'm a $20 user and was given $100 extra usage credit today, "for Fable" (I'll be sticking to Sonnet and sometimes Opus TYVM).

True. However, I believe most non-programmers don't need access to the fanciest model, but just want to use a good LLM without constant nagging about usage limits. Then 19 vs. 20 is true?

You're presenting further evidence of lack of effective control over these systems as... a mitigating factor...?

Well considering it was just a few months ago that OpenAI had the brilliant idea of connecting these systems to autonomously run an actual protein synthesis wetlab, very possibly all of us.