It's not possible to do reliably. None of the models in current use today use any esoteric math. It's extremely easy to implement the math behind both the inference and learning of all modern models.

Not everything is AI and dot products of massive vectors, there are still applications that do other maths on GPUs

My thinking was rather that most of our current programming languages put memory layout fully into the programmer’s responsibility - I can think off hand of a language where the compiler makes performance decisions like whether your structure are SoA, AoS or SoAoS, what alignment, padding, strides and float types to use.

Automatic decisions about when to use cooperative loads through shared local mem versus gathers from global mem and hardware caches are also something that such a hypothetical compiler would have to make.