This can’t be right. 2 PB of flash is like $200k. It’s within reach of many individuals. Then again I guess you don’t need that much storage so maybe it is.

▲

tjwebbnorfolk 3 hours ago | parent | next [-]

Also my first thought: "Is that... a lot?"

You can put 6PB (244TB * 24) into a single box these days.

▲

devttyeu 6 hours ago | parent | prev | next [-]

More like $1M at current prices at this scale / level of performance.

If you go with HDD arrays probably $50k

	▲	arjie 6 hours ago \| parent [-]
		Boy pricing is pretty nuts these days. I have half a petabyte in Seagate enterprise drives myself and I didn’t pay anything close to that to acquire it. Such a pity about the flash storage. 2 years ago we built 200 TiB or something of flash using Samsung PM1633 or something and it was a fraction of the cost per gigabyte that $1m would imply.

▲

metadat 6 hours ago | parent | prev [-]

Your numbers are a little off but the point remains- 2PB is nothing, not newsworthy imo. What’s special about this?

	▲	vidarh 5 hours ago \| parent [-]
		What's special about it is not the flash but training an LLM based on the content, much of which is still in copyright and which the library has restrictions on how they are allowed to use (irrespective of the legal position of training on it) and which required an agreement with the copyright holders.