Ok, let's say "cash is king". What is queen?

Arithmetic is king = royalty + male, while queen = royalty + female

But then it makes all these words just arithmetic values without meaning. Even if the words "royalty" and "male" can be sum or difference of some other words and so on - all are just numbers, no meaning at all.

The representation might not need to explicitly encode "meaning", if it does so implicitly by preserving essential properties of how things relate to each other.

For instance, a CAD object has no notion of what an airplane wing or car wheel are, but it can represent those in a way that how a wing relates to a fuselage is captured in numerical simulations. This is because it doesn't mangle the geometry the user wanted to represent ("what it means", in a geometric sense), although it does make it differ in certain ways that are "meaningless" (e.g. spurious small features, errors under tolerances), much like this representation might do with words.

Back to words, how do you define meaning anyways? I believe I was taught what words "mean" by having objects pointed to as a word was repeated: "cat", says the the parent as they point to a cat, "bird", as they point to a bird. Isn't this also equality/correspondence by relative frequency?

The meaning is not in the numbers themselves, but in how they each relate to one another.

Also those are not mere numbers here, but vectors. Dimensionality and orthogonality is key to define complex relationships.

I really think you should actually read the article. None of what you are saying has to do with the content of it, and it will explain how you can do arithmetic with these words.

It doesn’t work because that’s just wrong. The semantic meaning of “king” is much more than simply “royalty” and “male”. And it will be different for different people based on their experiences and familiarity with English and world history as well.

Then there’s the phonetic angle in addition to the semantic one. Why isn’t cash emperor? Because “cash is king” is alliterative.

Then there’s the orthographic angle: it’s a lot easier to write “king” than “emperor”.

This is my problem with people claiming that LLMs "understand". What we usually call "meaning" in intricately related to an encyclopedic knowledge of the world around us. How does this kind of knowledge not get into the same kind of loop you've just described? It is ultimately founded on our direct experience of the world, on sense data. It is ultimately embodied knowledge.

If Johnny Cash is king, then Queen is queen.

cash - king + queen = cashing, cash - male + female = cashing (in qwen3-embedding:0.6b). Make of that what you will

I asked deepseek and after many options the recommended one was:

Cash flow, "because you need the ongoing stream, not just a pile of cash, to reign successfully".

If I had to guess, cash - king + queen = credit (or money or something?). You are just asking the same thing as cash - man + woman, or "What is the feminine version of cash?" because queen - king ~= woman - man.

I say credit, because it is not as physical and direct as cash, so perhaps it is perceptually more feminine?

But I will have to check the next time I work with word2vec.

Can you embed a phrase like that in the same way?

Cash is the Man in Black. Elvis is King.

    "cash is king". What is queen?

This is the kind of question that made us doubt that AI would happen one day, cause meaning is quite remote from its expression.

There was always these contextual meanings that differ widely.

Like "carrots are orange" is a fact that's generally okay, but is not true at all, carrots come in a very wide range of colors.

But LLMs completely crushed through these problems. And vector embeddings are a bit part of why it worked.

So yeah, somewhere in those vectors is something that says that when cash is king, "king" has no relationship to monarchy.

queen is going to be a funny vector, mostly royalty, slightly chess, a bit gay, a bit rock and roll, a bit bee. Finally:

Queen + One = King