| ▲ | embedding-shape 7 hours ago | |
> I rewrote some Bash and Python scripts into C# that were previously processing gigabytes of JSON Hah, incredibly funny, I remember doing the complete opposite about 15 years ago, some beginner developer had setup a whole interconnected system with multiple processes and what not in order to process a bunch of JSON and it took forever. Got replaced with a bash script + Python! > Just how much data do you need when these sort of clustered approaches really start to make sense? I dunno exactly what thresholds others use, but I usually say if it'd take longer than a day to process (efficiently), then you probably want to figure out a better way than just running a program on a single machine to do it. | ||