Yep you do need to be careful with uncaught errors.

When you setup your project you choose the default number of retries and back-off settings. Generally people don't go as high as 50 and setup alerts when runs fail. Then you can use the bulk replaying feature when things do wrong, or if services you rely on have long outages.

I think on balance it is the correct behaviour.