▲ | viccis 5 days ago | |||||||
>In EC2, you can now change security groups and IAM roles without shutting the instance down to do it. Hasn't it been this way for many years? >Spot instances used to be much more of a bidding war / marketplace. Yeah because there's no bidding any more at all, which is great because you don't get those super high spikes as availability drops and only the ones who bid super high to ensure they wouldn't be priced out are able to get them. >You don’t have to randomize the first part of your object keys to ensure they get spread around and avoid hotspots. This one was a nightmare and it took ages to convince some of my more pig headed coworkers in the past that they didn't need to do it any more. The funniest part is that they were storing their data as millions and millions of 10-100kb files, so the S3 backend scaling wasn't the thing bottlenecking performance anyway! >Originally Lambda had a 5 minute timeout and didn’t support container images. Now you can run them for up to 15 minutes, use Docker images, use shared storage with EFS, give them up to 10GB of RAM (for which CPU scales accordingly and invisibly), and give /tmp up to 10GB of storage instead of just half a gig. This was/is killer. It used to be such a pain to have to manage pyarrow's package size if I wanted a Python Lambda function that used it. One thing I'll add that took me an embarrassingly long time to realize is that your Python global scope is actually persisted, not just the /tmp directory. | ||||||||
▲ | Inufu 2 days ago | parent | next [-] | |||||||
> You don’t have to randomize the first part of your object keys to ensure they get spread around and avoid hotspots. Sorry, this is absolutely still the case if you want to scale throughput beyond the few thousand IOPS a single shard can serve. S3 will automatically reshard your key space, but if your keys are sequential (eg leading timestamp) all your writes will still hit the same shard. Source: direct conversations with AWS teams. | ||||||||
▲ | indigodaddy 4 days ago | parent | prev [-] | |||||||
Re: SG, yeah I wasnt doing any cloud stuff when that was the case. Never had to restart anything for an SG change and this must be at least 5-6 years.. | ||||||||
|