Remix.run Logo
abidlabs an hour ago

It's not too hard to switch over from GitHub actions to other runners. For example, I wrote up the steps needed in order to use Hugging Face Jobs instead, which also enables GPU runners and other flavors. I use this for several of the repos I manage (such as Trackio):

https://huggingface.co/blog/github-ci-hf-jobs

yashap 43 minutes ago | parent | next [-]

We use other runners (Blacksmith), and it hasn't helped, still hard down all day long. Brutal outage.

As a side note, I started getting these symptoms (actions staying queued forever or not running at all) at 5pm PDT yesterday, intermittently. And definitely full outage by 8:30pm PDT. So for sure full outage for 7 hours and counting, even if you use other runners, and I strongly believe partial outage for ~15.5+ hrs before that, even if it hasn't been acknowledged by GitHub yet.

timwis an hour ago | parent | prev | next [-]

Is that not still down though because it still relies on GH actions to orchestrate?

tom1337 an hour ago | parent [-]

It most likely is. We are using Ubicloud (which seems to work the same way hf jobs does) and it's also down. GitHub is not sending out Webhooks and is also not able to queue external jobs. If we had to emergency deploy anything now I'd have no idea how to really do that...

nodesocket 21 minutes ago | parent | prev [-]

Also use Blacksmith for arm64 runners but this outage apparently affected webhooks so their runners are not picking up jobs either.