> and by "source" they meant the binary they shipped

Common misconception. Weights are not binary. Weights are hardcoded values that you load into an (open-source or closed-source) engine and you run that engine. The source code for LLMs is both in the architecture (i.e. what to do with those hardcoded values) and the inference engines.

As opposed to binaries, you can modify weights. You can adjust them, tune them for downstream tasks and so on. And more importantly, in theory you the downloader and "company x" the releaser of the model use the same methods and technologies to modify the weights. (in contrast to a binary release where you can only modify only the machine language while the creator can modify the source-code and recompile).

LLamas aren't open source because the license under which they're released isn't open source. There are plenty of models that are open source tho: mistrals (apache 2.0), qwens (apache2.0), deepseeks (mit), glms (mit) and so on.

what you describe reminds me pretty much of a binary blob that is loaded into a machine or software.

additionally modifying data in binary form was a longtime practice last time I looked, but I might not remember correctly.