> theoretically you could fail a server within 200-300ms, instead of 15seconds in your post.

You need to be careful here, though, because the server might just be a little sluggish. If it's doing something like garbage collection, your responses might take a couple hundred milliseconds temporarily. A blip of latency could take your server out of rotation. That increases load on your other servers and could cause a cascading failure.

If you don't need sub-second reactions to failures, don't worry too much about it.