i used to work, 15 years ago, on a (permissive, not covert) monitoring service for a UK national public service, the NHS spine core. We used switches to mirror ports and capture traffic in promisciouse mode on a few dozen servers split across a few datacentres that all the traffic went througg. We had certs installed to decode https. We could get enough hardware to do this step easily, but fast enough storage was an issue, we had 1 petabyte of usable storage across all sitesn that could hold a few days of content. We aimed to get this data filtered and forwarded into our central Splunk (seperate storage) and also into our bespoke dashboards within 60s. We often lagged...