That's only how it works when misapplying mean() to characterise application performance. In your scenario, every request is hitting either an overloaded instance, or when scaled down, a 20-30 second wait for spin up, better hope they're neither interactive nor some jobs configured with a short timeout. The mechanisms are so completely different they're hard to even make meaningful comparisons between

    > In your scenario, every request is hitting either an overloaded instance,

The container instances auto-scale by default to a configured n (0 - n) with the option to define specific scale points. If I'm sensitive to high bursts, this can be a matter of scaling at 50% capacity and a warm instance is available while the application still has headway.

Cases where this falls apart: DDoS in which case, I argue that 1) it is better in any case that it falls apart rather than absorbing that type of traffic spike, 2) such traffic is better filtered upstream in any case in an API Gateway + WAF.

I have worked with both types of solutions for different use cases (was invited presenter at Azure Serverless Conf[0]), but my take nowadays is that serverless containers are almost always a better approach since it provides much more flexibility.

[0] https://learn.microsoft.com/en-us/shows/azure-serverless-con...