But you can rent on-prem servers in some datacenter near you where all that is done for you.

First off, servers on someone else's premises are by definition not on-prem; and second, it still leaves you with a lot of the maintenance, management, and documentation overhead that comes with operating infrastructure equipment.

Do not forget that it is also cheaper. Main difference would be scalability which you do not inherently need. Not for ordinary bau.