CAG sounds like fake solution for LLM's. Math problems are not custom data, they are limited in amount, and do not refresh like product manuals.

Hence math can always be part either generic llm or math fine tuned llm, without weird layer made for human ( entire wolfram) and dependencies.

Wolfram alpha was always an extra translation layer between machine and human. LLM's are a universal translation layer that can also solve problems, verify etc.

You wouldn't use an LLM to solve a big Linear Programming problem, because it would cost way more than using the Simplex Method, and you'd be worried that it might be wrong.