I am developing a self-hosted solution for this [1]. It’s true that it’s somewhat of a pain but JIT runners allow a lot of flexibility that we don’t find elsewhere.

[1] https://runs-on.com

I did something very similar.

Create/start/stop/terminate EC2.

https://github.com/redoapp/fast-actions-runner-ec2

In my experience even with runs-on it is a bit painful with runner cancellations.