▲ | josnyder 4 days ago | |
While it can't be done server-side, this can be done straightforwardly in a signer service, and the signer doesn't need to interact with the payloads being uploaded. In other words, a tiny signer can act as a control plane for massive quantities of uploaded data. The client sends the request headers (including the x-amz-content-sha256 header) to the signer, and the signer responds with a valid S3 PUT request (minus body). The client takes the signer's response, appends its chosen request payload, and uploads it to S3. With such a system, you can implement a signer in a lambda function, and the lambda function enforces the content-addressed invariant. Unfortunately it doesn't work natively with multipart: while SigV4+S3 enables you to enforce the SHA256 of each individual part, you can't enforce the SHA256 of the entire object. If you really want, you can invent your own tree hashing format atop SHA256, and enforce content-addressability on that. I have a blog post [1] that goes into more depth on signers in general. [1] https://josnyder.com/blog/2024/patterns_in_s3_data_access.ht... | ||
▲ | JoshTriplett 3 days ago | parent [-] | |
That's incredibly interesting, thank you! That's a really creative approach, and it looks like it might work for me. |