> all the signals from the offline files in mere seconds

That sounds unlikely unless those machines had access to crazy disk I/O. A local RAID 5/10 with 8 drives would still take 30-50 minutes just to read that much data. Even with a mid-range SAN you still would have spent 15-25 minutes just reading data. This assumes 7.2K SAS/SATA since SSD/NVMe were not ubiquitous in 2015, but even with 2015-era SSDs you're still looking at half that much time spent reading.

▲

farazbabar 3 hours ago | parent | next [-]

Lots of EBS volumes mounted via 25GBPs or higher network, positioned carefully onto a single rack where possible and carefully tuned network/ip/os for both clients and servers. It is not feasible in most deployments as this would not have scaled to real production loads (RAID configurations requiring redundancy alone would slow you down, not to mention costs of using that many EBS volumes on extra high network/IO/provisioned IOPS nodes). This was only meant to prove what was possible in AWS at the time.

	▲	tomnipotent 2 hours ago \| parent [-]
		Not in 2015. 25GBps didn't even arrive until 2016, and dedicated EBS bandwidth didn't arrive until 2017.

▲

jeffbee 3 hours ago | parent | prev [-]

Your statement about SSD availability and performance does not fit with my personal recollections of history. Smoking fast SSDs were completely vanilla items you could order from anyone in 2015. For example, the Intel DC P35/6/700 series. Each of these was as good as 25 HDDs for linear reads and more like 5000 HDDs for random reads. The U.2 specification for putting SSDs in boxes was formalized in 2011. Outside of the mass market, cloud builders had private SSD designs long before 2015.

	▲	tomnipotent an hour ago \| parent [-]
		Less than 5% of commercial storage capacity was SSD of any flavor in 2015 (NVMe probably less than 1%), and the price-per-GB was anywhere from 5-15x. NVMe hardware RAID didn't happen until 2017, and AWS SSD via EBS was capped at 320 MB/s (~500 MB/s with striped EBS volumes). Before 2015 it was capped at 160 MB/s. In practice even NVMe SSDs at the time were maybe 10x faster for sequential R/W and maybe only 200-500x for random I/O. Most databases in 2015 were limited by serial dependency chains like walking a B-tree to figure out what nodes to read next and couldn't max out what an NVMe was capable of. Not to mention we were stuck with libaio in 2015 that made O_DIRECT ops synchronous. So the OPs claim of reading multiple terabytes of data in seconds on AWS in 2015 literally wasn't technically possible. SSDs simply were not as ubiquitous in 2015 as they are today, and I didn't know anyone using them outside of mission critical systems were the cost made sense.