| ▲ | huntaub 14 hours ago | |||||||
This is pretty different than s3fs. s3fs is a FUSE file system that is backed by S3. This means that all of the non-atomic operations that you might want to do on S3 (including edits to the middle of files, renames, etc) are run on the machine running S3fs. As a result, if your machine crashes, it's not clear what's going to show up in your S3 bucket or if would corrupt things. As a result, S3fs is also slow because it means that the next stop after your machine is S3, which isn't suitable for many file-based applications. What AWS has built here is different, using EFS as the middle layer means that there's a safe, durable place for your file system operations to go while they're being assembled in object operations. It also means that the performance should be much better than s3fs (it's talking to ssds where data is 1ms away instead of hdds where data is 30ms away). | ||||||||
| ▲ | mgaunard 4 hours ago | parent | next [-] | |||||||
It also means that you need to pay for EFS, which is outrageously expensive, to use S3, whose whole purpose is to be cheap. | ||||||||
| ||||||||
| ▲ | ChocolateGod 13 hours ago | parent | prev [-] | |||||||
You can also use something like JuiceFS to make using S3 as a shared filesystem more sane, but you're moving all the metadata to a shared database. | ||||||||
| ||||||||