Remix.run Logo
larkost 6 days ago

GitHub has still been managing the orchestration and monitoring of runs that you run on your own (or other cloud) hardware. They have just decided that they are no longer going to do this for free.

So the question becomes: is $0.002/minute a good price for this. I have never run GitHub Actions, so I am going to assume that experience on other, similar, systems applies.

So if your job takes an hour to build and run though all tests (a bit on the long side, but I have some tests that run for days), then you are going to pay GitHub $.12 for that run. You are probably going to pay significantly more for the compute for running that (especially if you are running on multiple testers simultaneously). So this does not seem to be too bad.

This is probably going to push a lot of people to invest more in parallelizing their workloads, and/or putting them on faster machines in order to reduce the number of minutes they are billed for.

I should note that if you are doing something similar in AWS using SMS (Systems Management Service), that I found that if you are running small jobs on lots of system that the AWS charges can add up very quickly. I had to abandon a monitoring system idea I had for our fleet (~800 systems) because the per-hit cost of just a monitoring ping was $1.84 (I needed a small mount of data from an on-worker process). Running that every 10 minutes was going to be more than $250/day. Writing/running my own monitoring system was much cheaper.

featherless 6 days ago | parent | next [-]

As a solo Founder who recently invested in self-hosted build infrastructure because my company runs ~70,000 minutes/month, this change is going to add an extra $140/month for hardware I own. And that's just today; this number will only go up over time.

I am not open to GitHub extracting usage-based rent for me using my own hardware.

This is the first time in my 15+ years of using GitHub that I'm seriously evaluating alternative products to move my company to.

larkost 6 days ago | parent | next [-]

But it is not for hardware you own. It is for the use of GutHubs coordinators, which they have been donating the use of to you for free. They have now decided that that service is something they are going to charge for. Your objection to GitHub "extracting usage-based rent from me" seems to ignore that you have been getting usage of their hardware for free up to now.

So, like I said, the question for you is whether that $140/month of service is worth that money to you, or can you find a better priced alternative, or build something that costs less yourself.

My guess is that once you think about this some more you will decide it is worth it, and probably spend some time trying to drive down your minutes/month a bit. But at $140 a month, how much time is that worth investing?

featherless 6 days ago | parent | next [-]

No. It is not worth a time-scaled cost each month for them to start a job on my machines and store a few megabytes of log files.

I'd happily pay a fixed monthly fee for this service, as I already do for GitHub.

The problem here is that this is like a grocery store charging me money for every bag I bring to bag my own groceries.

> But at $140 a month, how much time is that worth investing?

It's not $140/month. It's $140/month today, when my company is still relatively small and it's just me. This cost will scale as my company scales, in a way that is completely bonkers.

breppp 6 days ago | parent | next [-]

> The problem here is that this is like a grocery store charging me money for every bag I bring to bag my own groceries.

Maybe they can market it as the Github Actions corkage fee

__turbobrew__ 6 days ago | parent | prev | next [-]

> It is not worth a time-scaled cost each month for them to start a job on my machines and store a few megabytes of log files

If it is so easy why don’t you write your own orchestrator to run jobs on the hardware you own?

otterley 6 days ago | parent | prev [-]

> The problem here is that this is like a grocery store charging me money for every bag I bring to bag my own groceries.

This is an odd take because you're completely discounting the value of the orchestration. In your grocery store analogy, who's the orchestrator? It isn't you.

featherless 6 days ago | parent [-]

Do you feel that orchestration runs on a per-minute basis?

otterley 6 days ago | parent [-]

As long as they're reserving resources for your job during the period of execution, it does.

featherless 6 days ago | parent [-]

Charging people to maintain a row in a database by the minute is top-tier, I agree.

6 days ago | parent | next [-]
[deleted]
otterley 6 days ago | parent | prev [-]

If you really think that's all it is, I would encourage you to write your own.

featherless 6 days ago | parent [-]

It would be silly to write a new one today. Plenty of open source + indy options to invest into instead.

For scheduled work, cron + a log sink is fine, and for pull request CI there's plenty of alternatives that don't charge by the minute to use your own hardware. The irony here, unfortunately, is that the latter requires I move entirely off of GitHub now.

PunchyHamster 6 days ago | parent | prev [-]

so they are selling cent of their CPU time for a minute's worth

> My guess is that once you think about this some more you will decide it is worth it, and probably spend some time trying to drive down your minutes/month a bit. But at $140 a month, how much time is that worth investing?

It's $140 right now. And if they want to squeeze you for cents worth of CPU time (because for artifact storage you're already paying separately), they *will* squeeze harder.

And more importantly *RIGHT NOW* it costs more per minute than running decent sized runner!

nebezb 6 days ago | parent | prev | next [-]

I get the frustration. And I’m no GitHub apologist either. But you’re not being charged for hardware you own. You’re being charged for the services surrounding it (the action runner/executor binary you didn’t build, the orchestrator configurable in their DSL you write, the artefact and log retention you’re getting, the plug-n-play with your repo, etc). Whether or not you think that is a fair price is beside the point.

That value to you is apparently less than $140/mo. Find the number you’re comfortable with and then move away from GH Actions if it’s less than $140.

More than 10 years of running my own CI infra with Jenkins on top. In 2023 I gave up Jenkins and paid for BuildKite. It’s still my hardware. BuildKite just provides the “services” I described earlier. Yet I paid them a lot of money to provide their services for me on my own hardware. GH actions, even while free, was never an option for me. I don’t like how it feels.

This is probably bad for GitHub but framing it as “charging me for my hardware” misses the point entirely.

hugs 6 days ago | parent | prev [-]

feels like a new generation is learning what life is like when microsoft has a lot of power. (tl;dr: they try to use it.)

AJRF 6 days ago | parent | next [-]

I was born in 1993. I kind of heard lots of rumbling about Microsoft being evil as I grew up, but I wasn't fully understanding of the anti trust thing.

It used to suprise me that people saw cool tech from Microsoft (like VSCode) and complain about it.

I now see the first innings of a very silly game Microsoft are going to start playing over the next few years. Sure, they are going to make lots of money, but a whole generation of developers are learning to avoid them.

Thanks for trying to warn us old heads!

janc_ 6 days ago | parent | prev | next [-]

ABuse it.

PunchyHamster 6 days ago | parent | prev [-]

Feels like listening to Halo generation being surprised MS fucks them over, because they thought they were Good Guys, coz they Made Thing They like

gen220 6 days ago | parent | prev | next [-]

Yeah, I'm no GitHub apologist, but I'll be one in this context. This is actually a not-unreasonable thing to charge for. And a price point that's not-unreasonable.

It makes sense to do usage-based pricing with a generously-sized free tier, which seems to be what they're doing? Offering the entire service for free at any scale would imply that you're "paying" for/subsidizing this orchestration elsewhere in your transactions with GitHub. This is more-transparent pricing.

Although, this puts downward pressure on orgs' willingness to pay such a large price for GH enterprise licenses, as this service was hitherto "implicitly" baked into that fee. I don't think the license fees are going to go down any time soon, though :P

gallexme 6 days ago | parent | prev | next [-]

I run about 1 action a day taking 18h running on 2 runners One being self hosted 24gb ram 8 core ARM vps and one being a 64gb 13900k x86 dedicated server

Now the GitHub pricing change definitely? costs more than both servers combined a month ... (They cost about 60$ together )

3 step GitHub action builds around 1200 nix packages and derivations , but produces only around 50 lines of logs total if successful and maybe 200 lines of log once when a failure occurs And I'm supposed to pay 4$ a day for that ? Wonder what kind of actual costs are involved on their side of waiting for a runner to complete and storing 50 lines of log

alexellisuk 5 days ago | parent | next [-]

It sounds like you'd be better off self-hosting Jenkins. The other issue with GHA is they cap all runs at 6 hours.

Despite what people say about "maintaining" Jenkins (whatever that means to them personally) - you can set it up in an IaaC way including the jobs. You can migrate/create jobs en masse via its API (I did this about 10 years ago for a large US company converting from what was then called TFS)

franktankbank 5 days ago | parent | next [-]

What problem does Jenkins solve? When we got jenkins working how we wanted it was a giant groovy script that was handling checkout manually.

gallexme 5 days ago | parent | prev [-]

I'll likely check out buildbot or just switch to gitlab

janc_ 6 days ago | parent | prev [-]

Somewhere around 0.00004$ probably.

Nice profit margin…

deathanatos 6 days ago | parent | prev | next [-]

You know, one might ask what the base fee of $4k/mo (in my org's case) is covering, if not the control plane?

Unless you're on the free org plan, they're hardly doing it "for free" today…

numbsafari 6 days ago | parent [-]

Exactly this. It’s not like they don’t have plenty of other fees and charges. What’s next, charging mil rates for webhook deliveries?

dragonwriter 6 days ago | parent | prev | next [-]

> They have just decided that they are no longer going to do this for free.

Right, instead, they now charge the full cost of orchestration plus runner for just the orchestration part, making the basic runner free.

(Considering that compute for "self-hosted" runners is often also rented from some party that isn't Microsoft, this is arguably leveraging the market power in CI orchestration that is itself derived from their market power in code hosting to create/extend market power in compute for runners, which sounds like a potential violation of both the Sherman Act and the Clayton Act.)

ajford 6 days ago | parent | prev | next [-]

Sure, but that shouldn't be a time-dependent charge. If my build takes an hour to build on GH's hardware, sure thing, charge me for that time. But if my build takes an hour to build on _my_ hardware, then why am I paying GH for that hour?

I get being charged per-run, to recoup the infra cost, but what about my total runtime on my machine impacts what GH needs to spend to trigger my build?

csomar 5 days ago | parent | prev | next [-]

> $0.002/minute a good price for this.

It is not only not good. It is outrageous. The amount of compute required for orchestration is small (async operations) and also they already charge your for artifacts storage. You need to understand that the orchestration just receives details (inbound) from the runner. It needs very little resources.

whynotmaybe 6 days ago | parent | prev | next [-]

> is $0.002/minute a good price for this

It was free, so anything other than free isn't really a good price. It's hard to estimate the cost on github's side when the hardware is mine and therefore accept this easily.

(Github is already polling my agent to know it's status so whether is "idle" or "running action" shouldn't really change a lot on their side.)

...And we already pay montly subscription for team members and copilot.

I have a self-hosted runner because I must have many tools installed for my builds and find it kinda counter productive to always reinstall those tools for each build as this takes a long time. (Yeah, I know "reproducible builds" aso, but I only have 24h in most of my days)

Even for a few hundreds minutes a month, we're still under a few $ so not worth spending two days to improve anything... yet.

saagarjha 6 days ago | parent [-]

Is it polling the runner, or is the runner sending it progress?

ExoticPearTree 6 days ago | parent [-]

The runner sends progress info, polls for jobs and so on. The runners don't have to be accessible from GitHub, they just needs general internet access (like through a NAT device).

skilning 6 days ago | parent | prev | next [-]

> is $0.002/minute a good price for this

Absolutely not, since it's the same price as their cheapest hosted option. If all they're doing is orchestration, why the hell are they charging per-minute instead of per-action or some other measure that recognizes the difference in their cost between self-hosted and github-hosted?

swiftcoder 5 days ago | parent | prev | next [-]

> is $0.002/minute a good price for this

I think a useful framing of this question is: would you run a c7gn.large instance just to do this orchestration?

j45 6 days ago | parent | prev | next [-]

Additionally, they could just self-host their code since code is data is a moat.

solatic 5 days ago | parent | prev [-]

> GitHub has still been managing the orchestration and monitoring of runs that you run on your own (or other cloud) hardware. They have just decided that they are no longer going to do this for free.

This argument is disingenuous. Companies pay GitHub per seat for access to PR functionality etc. What's next, charging per repository? Because of a decision to no longer provide the repositories "for free"? It's not for free, you're paying already, it's included in the per-seat pricing. If you charge per seat then sometimes there are users who hardly use it and sometimes there are users who use it a lot. The per-seat pricing model is supposed to make the service profitable overall regardless of the usage levels of individual users.