Complete LLM internals noob here: Wouldn't this make GPTs awful at languages like German with separable word prefixes?

E.g. Er macht das Fenster. vs Er macht das Fenster auf.

(He makes the window. vs He opens the window.)

Or exceptionally good at german because they have to keep better track of what is meant and anticipate more?

No I don't think it makes any noticeable difference :)