I understand that part, but I don't see why do this instead of basic Makefile or CMake setup. It seems like more work than a regular linker at that point. For what purpose?

Because not everyone is using Makefiles or CMake.

A true header-only library should be build-system agnostic and this is one way to do that.

We can argue about build systems for C++ all day long and never come to an agreement. With this approach this piece of code can be used anywhere.