▲ | texthompson 4 days ago | ||||||||||||||||||||||
Why would you PUT an object, then download it again to a central server in the first place? If a service is accepting an upload of the bytes, it is already doing a pass over all the bytes anyway. It doesn't seem like a ton of overhead to calculate SHA256 in the 4092-byte chunks as the upload progresses. I suspect that sort of calculation would happen anyways. | |||||||||||||||||||||||
▲ | willglynn 4 days ago | parent | next [-] | ||||||||||||||||||||||
You're right, and in fact S3 does this with the `ETag:` header… in the simple case. S3 also supports more complicated cases where the entire object may not be visible to any single component while it is being written, and in those cases, `ETag:` works differently. > * Objects created by the PUT Object, POST Object, or Copy operation, or through the AWS Management Console, and are encrypted by SSE-S3 or plaintext, have ETags that are an MD5 digest of their object data. > * Objects created by the PUT Object, POST Object, or Copy operation, or through the AWS Management Console, and are encrypted by SSE-C or SSE-KMS, have ETags that are not an MD5 digest of their object data. > * If an object is created by either the Multipart Upload or Part Copy operation, the ETag is not an MD5 digest, regardless of the method of encryption. If an object is larger than 16 MB, the AWS Management Console will upload or copy that object as a Multipart Upload, and therefore the ETag will not be an MD5 digest. https://docs.aws.amazon.com/AmazonS3/latest/API/API_Object.h... | |||||||||||||||||||||||
▲ | danielheath 4 days ago | parent | prev [-] | ||||||||||||||||||||||
S3 supports multipart uploads which don’t necessarily send all the parts to the same server. | |||||||||||||||||||||||
|