The way most programs achieve being fast is by re-using allocations. You don't need a fast allocator if you don't allocate. Nothing of what ripgrep does inherently requires frequent allocations.

▲

entrope 3 hours ago | parent [-]

The ISO C definition of opendir() requires an allocation in practice because it returns a DIR* and it's bad practice for the library to arbitrarily limit how many of those an application has at a time.

Maybe a C library could preallocate several DIRs and only use the heap when those are exhausted, but this ripgrep use case (lots of threads running in parallel on a large tree) would still be likely to trigger that.

▲

mort96 an hour ago | parent [-]

There is no ISO C definition of opendir(), that's a POSIX thing.

But yeah, those DIR objects which opendir() returns a pointer to are probably some kind of heap allocated. But we're talking about a system call involving the filesystem here. The time taken by the allocator is gonna be dwarfed by the time spent in the syscall even with the slowest allocator.

Ripgrep reads through every byte of most files and matches it against a regex. That is the tight inner loop where you want to avoid allocations.

Not that you even need to call the C functions. I don't think ripgrep would gain anything from it, but the syscall to read directory contents just needs a file descriptor.

	▲	entrope 5 minutes ago \| parent [-]
		I think ripgrep does avoid allocations in that innermost loop. That's why this report says all the crashes are associated with opendir() and its memory allocation: the opendir() call is outside of ripgrep's innermost loop but still runs often enough to trigger the race condition.