Remix.run Logo
saberience 7 hours ago

Well, at my old company we had some datasets in the 6-8 PB range, so tell me how we would run analytics on that dataset on an Intel NUC.

Just because you don't have experience of these situations, it doesn't mean they don't exist. There's a reason Hadoop and Spark became synonymous with "big data."

dapperdrake 5 hours ago | parent | next [-]

These situations are rare not difficult.

The solutions are well known even to many non-programmers who actually have that problem:

There are also sensor arrays that write 100,000 data points per millisecond. But again, that is a hardware problem not a software problem.

6 hours ago | parent | prev [-]
[deleted]