| ▲ | bmenrigh 8 hours ago |
| I recently started using Microsoft's mimalloc (via an LD_PRELOAD) to better use huge (1 GB) pages in a memory intensive program. The performance gains are significant (around 20%). It feels rather strange using an open source MS library for performance on my Linux system. There needs to be more competition in the malloc space. Between various huge page sizes and transparent huge pages, there are a lot of gains to be had over what you get from a default GNU libc. |
|
| ▲ | skavi 7 hours ago | parent | next [-] |
| We evaluated a few allocators for some of our Linux apps and found (modern) tcmalloc to consistently win in time and space. Our applications are primarily written in Rust and the allocators were linked in statically (except for glibc). Unfortunately I didn't capture much context on the allocation patterns. I think in general the apps allocate and deallocate at a higher rate than most Rust apps (or more than I'd like at least). Our results from July 2025: rows are <allocator>: <RSS>, <time spent for allocator operations> app1:
glibc: 215,580 KB, 133 ms
mimalloc 2.1.7: 144,092 KB, 91 ms
mimalloc 2.2.4: 173,240 KB, 280 ms
tcmalloc: 138,496 KB, 96 ms
jemalloc: 147,408 KB, 92 ms
app2, bench1
glibc: 1,165,000 KB, 1.4 s
mimalloc 2.1.7: 1,072,000 KB, 5.1 s
mimalloc 2.2.4:
tcmalloc: 1,023,000 KB, 530 ms
app2, bench2
glibc: 1,190,224 KB, 1.5 s
mimalloc 2.1.7: 1,128,328 KB, 5.3 s
mimalloc 2.2.4: 1,657,600 KB, 3.7 s
tcmalloc: 1,045,968 KB, 640 ms
jemalloc: 1,210,000 KB, 1.1 s
app3
glibc: 284,616 KB, 440 ms
mimalloc 2.1.7: 246,216 KB, 250 ms
mimalloc 2.2.4: 325,184 KB, 290 ms
tcmalloc: 178,688 KB, 200 ms
jemalloc: 264,688 KB, 230 ms
tcmalloc was from github.com/google/tcmalloc/tree/24b3f29.i don't recall which jemalloc was tested. |
| |
| ▲ | hedora 6 hours ago | parent | next [-] | | I’m surprised (unless they replaced the core tcmalloc algorithm but kept the name). tcmalloc (thread caching malloc) assumes memory allocations have good thread locality. This is often a double win (less false sharing of cache lines, and most allocations hit thread-local data structures in the allocator). Multithreaded async systems destroy that locality, so it constantly has to run through the exception case: A allocated a buffer, went async, the request wakes up on thread B, which frees the buffer, and has to synchronize with A to give it back. Are you using async rust, or sync rust? | | |
| ▲ | skavi 6 hours ago | parent [-] | | modern tcmalloc uses per CPU caches via rseq [0]. We use async rust with multithreaded tokio executors (sometimes multiple in the same application). so relatively high thread counts. [0]: https://github.com/google/tcmalloc/blob/master/docs/design.m... | | |
| ▲ | usrnm 5 hours ago | parent [-] | | How do you control which CPU your task resumes on? If you don't then it's still the same problem described above, no? | | |
| ▲ | skavi an hour ago | parent [-] | | on the OS scheduler side, i'd imagine there's some stickiness that keeps tasks from jumping wildly between cores. like i'd expect migration to be modelled as a non zero cost. complete speculation though. tokio scheduler side, the executor is thread per core and work stealing of in progress tasks shouldn't be happening too much. for all thread pool threads or threads unaffiliated with the executor, see earlier speculation on OS scheduler behavior. | | |
| ▲ | packetlost 40 minutes ago | parent [-] | | Correct. The Linux scheduler has been NUMA aware + sticky for awhile (which is more or less what this reduces to in common scenarios). |
|
|
|
| |
| ▲ | ComputerGuru 6 hours ago | parent | prev | next [-] | | That’s a considerable regression for mimalloc between 2.1 and 2.2 – did you track it down or report it upstream? Edit: I see mimalloc v3 is out – I missed that! That probably moots this discussion altogether. | | | |
| ▲ | codexon 5 hours ago | parent | prev [-] | | This is similar to what I experienced when I tested mimalloc many years ago. If it was faster, it wasn't faster by much, and had pretty bad worst cases. |
|
|
| ▲ | pjmlp 7 hours ago | parent | prev | next [-] |
| If you go into Dr Dobbs, The C/C++ User's Journal and BYTE digital archives, there will be ads of companies whose product was basically special cased memory allocator. Even toolchains like Turbo Pascal for MS-DOS, had an API to customise the memory allocator. The one size fits all was never a solution. |
|
| ▲ | m463 6 hours ago | parent | prev | next [-] |
| I remember in the early days of web services, using the apache portable runtime, specifically memory pools. If you got a web request, you could allocate a memory pool for it, then you would do all your memory allocations from that pool. And when your web request ended - either cleanly or with a hundred different kinds of errors, you could just free the entire pool. it was nice and made an impression on me. I think the lowly malloc probably has lots of interesting ways of growing and changing. |
| |
| ▲ | Sesse__ 5 hours ago | parent | next [-] | | This is called “an arena” more generally, and it is in wide use across many forms of servers, compilers, and others. | |
| ▲ | jra_samba 6 hours ago | parent | prev [-] | | Look into talloc, used inside Samba (and other FLOSS projects like sssd). Exactly this. |
|
|
| ▲ | adgjlsfhk1 8 hours ago | parent | prev | next [-] |
| One of the best parts about GC languages is they tend to have much more efficient allocation/freeing because the cost is much more lumped together so it shows up better in a profile. |
| |
| ▲ | pjmlp 7 hours ago | parent | next [-] | | Agreed, however there is also a reason why the best ones also pack multiple GC algorithms, like in Java and .NET, because one approach doesn't fit all workloads. | | |
| ▲ | nevdka 7 hours ago | parent [-] | | Then there’s perl, which doesn’t free at all. | | |
| ▲ | hedora 6 hours ago | parent | next [-] | | Perl frees memory. It uses refcounting, so you need to break heap cycles or it will leak. (99% of the time, I find this less problematic than Java’s approach, fwiw). | |
| ▲ | cermicelli 7 hours ago | parent | prev | next [-] | | Freedom is overrated... :P | |
| ▲ | NooneAtAll3 7 hours ago | parent | prev [-] | | doesn't java also? I heard that was a common complaint for minecraft | | |
| ▲ | xxs 6 hours ago | parent [-] | | What do you mean - if Java returns memory to the OS? Which one - Java heap of the malloc/free by the JVM? | | |
| ▲ | cogman10 6 hours ago | parent [-] | | Java is pretty greedy with the memory it claims. Especially historically it was pretty hard to get the JVM to release memory back to the OS. To an outsider, that looks like the JVM heap just steadily growing, which is easy to mistake for a memory leak. | | |
| ▲ | k_roy 6 hours ago | parent | next [-] | | > Especially historically it was pretty hard to get the JVM to release memory back to the OS. This feels like a huge understatement. I still have some PTSD around when I did Java professionally between like 2005 and 2014. The early part of that was particularly horrible. | |
| ▲ | xxs 6 hours ago | parent | prev | next [-] | | Java has a quite strict max heap setting, it's very uncommon to let it allocate up to 25% of the system memory (the default). It won't grow past that point, though. Baring bugs/native leaks - Java has a very predictable memory allocation. | |
| ▲ | adgjlsfhk1 6 hours ago | parent | prev [-] | | This only really ends up being a problem on windows. On systems with proper virtual memory setups, the cost of unused memory is very low (since the the OS can just page it out) | | |
| ▲ | cogman10 3 hours ago | parent | next [-] | | Unfortunately, the JVM and collectors like the JVM's plays really bad with virtual memory. (Actually, G1 might play better. Everything else does not). The issue is that through the standard course of a JVM application running, every allocated page will ultimately be touched. The JVM fills up new gen, runs a minor collection, moves old objects to old gen, and continues until old gen gets filled. When old gen is filled, a major collection is triggered and all the live objects get moved around in memory. This natural action of the JVM means you'll see a sawtooth of used memory in a properly running JVM where the peak of the sawtooth occasionally hits the memory maximum, which in turn causes the used memory to plummet. | |
| ▲ | snackbroken 3 hours ago | parent | prev [-] | | For video games it is pretty bad, because reading back a page from disk containing "freed" (from the application perspective, but not returned to the OS) junk you don't care about is significantly slower than the OS just handing you a fresh one. A 10-20ms delay is a noticeable stutter and even on an SSD that's only a handful of round-trips. | | |
| ▲ | cogman10 3 hours ago | parent [-] | | Games today should be using ZGC. There's a lot of bad tuning guides for minecraft that should be completely ignored and thrown in the trash. The only GC setting you need for it is `-XX:+UseZGC` For example, a number of the minecraft golden guides I've seen will suggest things like setting pause targets but also survivor space sizes. The thing is, the pause target is disabled when you start playing with survivor space sizes. |
|
|
|
|
|
|
| |
| ▲ | bluGill 6 hours ago | parent | prev | next [-] | | When it works. Many programs in GC language end up fighting the GC by allocating a large buffer and managing it by hand anyway because when performance counts you can't have allocation time in there at all. (you see this in C all the time as well) | | |
| ▲ | cogman10 6 hours ago | parent [-] | | That's generally a bad idea. Not always, but generally. It was a better idea when Java had the old mark and sweep collector. However, with the generational collectors (which are all Java collectors now. except for epsilon) it's more problematic. Reusing buffers and objects in those buffers will pretty much guarantees that buffer ends up in oldgen. That means to clear it out, the VM has to do more expensive collections. The actual allocation time for most of Java's collectors is almost 0, it's a capacity check and a pointer bump in most circumstances. Giving the JVM more memory will generally solve issues with memory pressure and GC times. That's (generally) a better solution to performance problems vs doing the large buffer. Now, that said, there certainly have been times where allocation pressure is a major problem and removing the allocation is the solution. In particular, I've found boxing to often be a major cause of performance problems. | | |
| ▲ | drob518 4 hours ago | parent | next [-] | | If your workload is very regular, you can still do better with an arena allocator. Within the arena, it uses the same pointer-bump allocation as Java normally uses, but then you can free the whole area back to the start by resetting the pointer to its initial value. If you use the arena for servicing a single request, for instance, you then reset as soon as you're done with the request, setting you up with a totally free area for the next request. That's more efficient than a GC. But it also requires your algorithm to fall into that pattern where you KNOW that you can and should throw everything from the request away. If you can't guarantee that, then modern collectors are pretty magical and tunable. | |
| ▲ | CyberDildonics 5 hours ago | parent | prev [-] | | If people didn't need to do it, they wouldn't generally do it. Not always, but generally. | | |
| ▲ | cogman10 5 hours ago | parent [-] | | People do stuff they shouldn't all the time. For example, some code I had to clean up pretty early on in my career was a dev, for unknown reasons, reinventing the `ArrayList` and then using that invention as a set (doing deduplication by iterating over the elements and checking for duplicates). It was done in the name of performance, but it was never a slow part of the code. I replaced the whole thing with a `HashSet` and saved ~300 loc as a result. This individual did that sort of stuff all over the code base. | | |
| ▲ | CyberDildonics 4 hours ago | parent [-] | | Reinventing data structures poorly is very common. Heap allocation in java is something trivial happens constantly. People typically do funky stuff with memory allocation because they have to, because the GC is causing pauses. People avoid system allocators in C++ too, they just don't have to do it because of uncontrollable pauses. | | |
| ▲ | cogman10 4 hours ago | parent [-] | | > People typically do funky stuff with memory allocation because they have to This same dev did things like putting what he deemed as being large objects (icons) into weak references to save memory. When the references were collected, invariably they had to be reloaded. That was not the source of memory pressure issues in the app. I've developed a mistrust for a lot of devs "doing it because we have to" when it comes to performance tweaks. It's not a never thing that a buffer is the right thing to do, but it's not been something I had to reach for to solve GC pressure issues. Often times, far more simple solutions like pulling an allocation out of the middle of a loop, or switching from boxed types to primatives, was all that was needed to relieve memory pressure. The closest I've come to it is replacing code which would do an expensive and allocation heavy calculation with a field that caches the result of that calculation on the first call. | | |
|
|
|
|
| |
| ▲ | CyberDildonics 5 hours ago | parent | prev [-] | | Any extra throughput is far overshadowed by trying to control pauses and too much heap allocations happening because too much gets put on the heap. For anything interactive the options are usually fighting the gc or avoiding gc. |
|
|
| ▲ | pocksuppet 7 hours ago | parent | prev | next [-] |
| In many cases you can also do better than using malloc e.g. if you know you need a huge page, map a huge page directly with mmap Yes, if you want to use huge pages with arbitrary alloc/free, then use a third-party malloc. If your alloc/free patterns are not arbitrary, you can do even better. We treat malloc as a magic black box but it's actually not very good. |
|
| ▲ | Dylan16807 6 hours ago | parent | prev | next [-] |
| I think some operating system improvements could get people motivated to use huge pages a lot better. In particular make them less fragile on linux and make them not need admin rights on windows. The biggest factor causing problems there is that neither OS can swap a 2MB page. So someone needs to care enough to fix that. |
|
| ▲ | codexon 7 hours ago | parent | prev | next [-] |
| I've been using jemalloc for over 10 years and don't really see a need for it to be updated. It always holds up in benchmarks against any new flavor of the month malloc that comes out. Last time I checked mimalloc which was admittedly a while ago, probably 5 years, it was noticebly worse and I saw a lot of people on their github issues agreeing with me so I just never looked at it again. |
| |
| ▲ | adgjlsfhk1 7 hours ago | parent | next [-] | | Mimalloc v3 has just come out (about a month ago) and is a significant improvement over both v2 and v1 (what you likely last tested) | |
| ▲ | hrmtst93837 7 hours ago | parent | prev [-] | | Benchmarks age fast. Treating a ten-year-old allocator as done just because it still wins old tests is tempting fate, since distros, glibc, kernel VM behavior, and high-core alloc patterns keep moving and the failures usually show up as weird regressions in production, not as a clean loss on someone's benchmark chart. | | |
| ▲ | codexon 7 hours ago | parent [-] | | It still beat mimalloc when I checked 4-5 years ago. | | |
| ▲ | imp0cat 7 hours ago | parent [-] | | You really need to benchmark your workloads, ideally with the "big 3" (jemalloc, tcmalloc, mimalloc). They all have their strengths and weaknesses. Jemalloc can usually keep the smallest memory footprint, followed by tcmalloc. Mimalloc can really speed things up sometimes. As usually, YMMV. | | |
| ▲ | codexon 6 hours ago | parent | next [-] | | I've benchmarked them every few years, they never seem to differ by more than a few percent, and jemalloc seems to fragment and leak the least for processes running for months. Mimalloc made the claim that they were the fastest/best when they released and that didn't hold up to real world testing, so I am not inclined to trust it now. | | |
| ▲ | ComputerGuru 6 hours ago | parent [-] | | > Mimalloc made the claim that they were the fastest/best when they released and that didn't hold up to real world testing That’s… ahistorical, at least so far as I remember. It wasn’t marketed as either of those; it was marketed as small/simple/consistent with an opt-in high-severity mode, and then its performance bore out as a result of the first set of target features/design goals. It was mainly pushed as easy to adopt, easy to use, easy to statically link, etc. | | |
| |
| ▲ | HackerThemAll 5 hours ago | parent | prev [-] | | Look up the numbers in other comments above. When it comes to performance, the Google's tcmalloc is unconquered. |
|
|
|
|
|
| ▲ | anthk 7 hours ago | parent | prev | next [-] |
| I used mimalloc to run zenlisp under OpenBSD as it would clash with the paranoid malloc of base. |
|
| ▲ | IshKebab 7 hours ago | parent | prev | next [-] |
| I feel like the real thing that needs to change is we need a more expressive allocation interface than just malloc/realloc. I'm sure that memory allocators could do a significantly better job if they had more information about what the program was intending to do. |
| |
|
| ▲ | jeffbee 7 hours ago | parent | prev | next [-] |
| Just out of curiosity are you getting 1GB huge pages on Xeon or some other platform? I always thought this class of page is the hardest to exploit, considering that the machine only has, if I recall correctly, one TLB slot for those. |
| |
| ▲ | bmenrigh 7 hours ago | parent [-] | | Modern x86_64 has supported multiple page sizes for a long time. I'm on commodity Zen 5 hardware (9900X) with 128 GiB of RAM. Linux will still use a base page size of 4kb but also supports both 2 MiB and 1 GiB huge pages. You can pass something like `default_hugepagesz=2M hugepagesz=1G hugepages=16` to your kernel on boot to use 2 MiB pages but reserve 16 1 GiB pages for later use. The nice thing about mimalloc is that there are a ton of configurable knobs available via env vars. I'm able to hand those 16 1 GiB pages to the program at launch via `MIMALLOC_RESERVE_HUGE_OS_PAGES=16`. EDIT: after re-reading your comment a few times, I apologize if you already knew this (which it sounds like you did). | | |
| ▲ | jeffbee 6 hours ago | parent [-] | | Right but on Intel the 1G page size has historically been the odd one. For example Skylake-X has 1536 L2 shared TLB entries for either 4K or 2M pages, but it only has 16 entries that can be used for 1G pages. It wasn't unified until Cascade Lake. But Skylake-like Xeon is still incredibly common in the cloud so it's hard to target the later ones. | | |
| ▲ | Dylan16807 5 hours ago | parent [-] | | So for any process that's using less than 16GB, it's a significant performance boost. And most processes using more RAM, but not splitting accesses across more than 16 zones in rapid succession, will also see a performance boost. My old Intel CPU only has 4 slots for 1GB pages, and that was enough to get me about a 20% performance boost on Factorio. (I think a couple percent might have been allocator change but the boost from forcing huge pages was very significant) | | |
| ▲ | jeffbee 4 hours ago | parent [-] | | That strikes me as a common hugepages win. People never believe you, though, when you say you can make their thing 20% faster for free. |
|
|
|
|
|
| ▲ | sylware 8 hours ago | parent | prev [-] |
| If there is so much performance difference among generic allocators, it means you need semantic optimized allocators (unless performance is actually not that much important in the end). |
| |
| ▲ | Cloudef 7 hours ago | parent | next [-] | | You are not wrong and this is indeed what zig is trying to push by making all std functions that allocate take a allocator parameter. | |
| ▲ | codexon 5 hours ago | parent | prev [-] | | Agreed mostly. Going from standard library to something like jemalloc or tcmalloc will give you around 5-10% wins which can be significant, but the difference between those generic allocators seem small. I just made a slab allocator recently for a custom data type and got speedups of 100% over malloc. |
|
