LLM's generating text referencing nonexistent up API's or machine instructions seems to me to be exactly like LLMs generating text referencing nonexistent legal cases or generating fictional text referencing nonexistent people.

They've been fed enough real text in each of these various styles that they can parrot the shallow surface style without reference to the internals.