Your agent(s) need to work on something, ie. running TypeScript, your app, your tests, Docker, Redis and/or database you need to run harness and user side apps ie. VScode, browser etc. it all adds up quickly.
Single user conversation spawns multiple parallel backend conversations, you need extra room for it as well, not just single context.
This plus usual apps like Mail, Spotify, iTerm2, SourceTree etc. also fill in memory.
Also 4 bit quantization is already quite aggressive compromise (measurable but sometimes acceptable loss, compared to ie. 8 bits which are often practically lossless) – for weights it's ok'ish, sometimes (especially if model was trained as 4 bit quants aware), but activations need to stay at higher bits taking more memory, otherwise quality degrades a lot.
For a dedicated headless setup, I’d probably use something like NVIDIA DGX Spark rather than a Mac (to be more precise NVIDIA GB10 Grace Blackwell from other suppliers than directly NVidia, they are much cheaper and have same insides). Linux is much better for running headless server, you also get standard NVIDIA/CUDA ecosystem instead of being tied to Metal/macOS.
For people who are interested in buying IMHO I'd wait a bit – next generation of Spark and/or Macs that are going to come out next year will be much better / will cross the line of being actually useful, not just a toy with goldfish LLM.
