Isn't the "gardening" aspect part of it though? It's where you naturally review and mentally correlate topics, infer connections in your brain and spark new paths?

This is true. It would be beneficial to do such a task

However, if one doesn't want (or just doesn't have time) to do the task but still want a tidy cross-referenced set of notes, one could outsource to a LLM