| ▲ | Orphis 6 hours ago | ||||||||||||||||||||||
ripgrep is there to be FAST. Trading any speed to improve memory efficiency over a longer period of time for a process with a short life-time doesn't make sense in this case. It's also a development tool. If your development machine is having RAM issues because it's doing a grep, you have bigger problems to solve. As for other workloads that might use less RAM with mallocng compared to other performant ones, I'm curious to know about the magnitudes we're talking about. From 10MB to 20MB or from 100MB to 2GB? How was the speed of the program? Was there any multithreading involved? | |||||||||||||||||||||||
| ▲ | tripflag 5 hours ago | parent [-] | ||||||||||||||||||||||
For context, the Python workload was a general-purpose fileserver with file indexing and image thumbnailing, largely IO-bound. Switching from mallocng to mimalloc resulted in a slight speedup (150% of baseline), but over twice the memory usage, going from 250 to 670 MiB, primarily for thumbnailing. Some pathological cases (libvips calling imagemagick to decode heif) was a 10x multiplier. There was multithreading, and yes, mallocng visibly became a bottleneck beyond 5 threads. However already at 3 threads there was diminishing returns for both allocators, so this was not an issue in this case. The speed gain was hardly noticeable in practice since the program was already plenty fast, but the additional memory usage was a very real inconvenience. | |||||||||||||||||||||||
| |||||||||||||||||||||||