Freeing memory to the OS or doing things like munmap involves a TLB shootdown which by definition is a performance bottleneck. Probably what you ended up experiencing.

Thanks for that. That was an interesting read!

I ended up in this situation like I end up in most, by just learning as I go.

With this I learned that the libc allocator has a max amount that it'll add to the heap, and how mmap works, and that doing frequent munmap is slow.

Now I learned why it is slow!