I'm not sure that's a great takeaway. Lots of problems are undecidable and have reasonably effective solutions in practice (e.g. finding bugs -> testing, static analysis, etc). Mind you, I don't expect we'll find anything like that for transformers, but there's a surprisingly large gap between what's possible in general and what's possible in the cases we care about.