Remix.run Logo
siliconc0w 4 days ago

Pretty bullish, anyone who has tried to setup and manage their own compute knows the pain they're solving.

Plus I predict more companies will exit the cloud once they realize how thick the margins have become or want better guarantees over sovereignty.

jillesvangurp 3 days ago | parent | next [-]

I actually think that having more cloud providers might deflate a lot of the pricing. If you think about it, companies like Amazon buy server hardware and then rent it out by the vcpu (with throttling if they can get away with it) per month. Add memory and IO and you are looking at bills that pay for the server in mere months/weeks several tenants carving up all the hardware and each paying tens/hundreds per month.

There are of course benefits to using cloud based VMs and I use them as well. But you are paying a very steep premium for what is a pitiful amount of compute and memory. There's a lot of wiggle room for price decreases and the only thing preventing that is a lack of competition. There's a reason Amazon is so rich: nobody seems to challenge them on AWS pricing. There's value in having them do all the faffing about with hardware of course. That's why companies use them. I'm in GCP; but same principle. I don't want to have to worry about replacing hard disks in the middle of the night, deal with network routers that are misbehaving, cooling issues, etc. That's why I pay them the big bucks. But I'm well aware that it's not that great of a deal.

I used Hetzner a decade ago and paid something like 50 euros per month for a quad core xeon with a raid 1 disks, 32 GB, etc. Bare metal of course. But also, 50 euro. We had five of those. Forget about getting anything close to that with modern cloud providers for anything resembling a reasonable price. Your first monthly bill might actually add up to enough to buy your own hardware. Very tempting. They have beefed up their specs since then. You now get more for less. And they also do VMs now.

throwup238 3 days ago | parent | next [-]

I knew it was bad but I didn’t realize just how bad the pricing spread can be until I started dealing with the GPU instances (8x A100 or H100 pods). Last I checked the on-demand pricing was $40/hr and the 1-year reserved instances were $25/hr. That’s over $200k/yr for the reserved instances so within two years I’d spend enough to buy my own 8x H100 pod (based on LambdaLabs pricing) plus enough to pay an engineer to babysit five pods at a time. It’s insane.

With on-demand pricing the pod would pay for itself (and the cost to manage it) within a year.

dilyevsky 4 hours ago | parent | next [-]

It's actually not that bad for GPUs considering their useful life is much shorter than regular compute. DC-grade CPU servers cost 12-24mo of typical public cloud prices but you can run em for 6-8 years.

wordpad 3 days ago | parent | prev [-]

That's just hardware. If you need to build and maintain your own devops tooling it can balloon in complexity and cost real quick.

It would still likely be much cheaper to do everything in house, but you would be assuming a lot of risks and locking yourself in losing flexibility.

There is a reason people go with AWS over many competing cheaper cloud providers.

echelon 3 days ago | parent [-]

> There is a reason people go with AWS over many competing cheaper cloud providers.

Opportunity cost.

The evolutionary fitness landscape hasn't yet provided an escape hatch for paying this premium, but in time it will.

time0ut 3 days ago | parent | prev [-]

In my experience, companies seem to want to pay the cloud provider tax in order to avoid capacity planning. Sometimes it makes sense because it is hard to predict when something is going to take off. I have also worked at companies with very predictable growth paying insane amounts. I didn’t understand the logic, but they still were profitable and paid well so whatever.

AndrewKemendo 3 days ago | parent | prev | next [-]

Having done web applications for oh…30 years now the key pain is network routing

That’s the only thing I can tell is useful about “the cloud”

I can build racks and servers easily, but the challenge is availability and getting past everyone’s firewalls

So the real win is any service that allows for instant DNS table updates and availability of DNS whitelisting.

This is why Google, Msft etc win in email because they have trusted endpoints

Alternative routes with self signed DKIM etc is more or less blocked by default forcing you onto a provider

We need more cloud flare tunnel and local hosting via commercial ISP routes and less new centralized data centers

chrisweekly 3 days ago | parent | next [-]

+1 Insightful.

I've also been doing web-related work for a living for over 25y and yours is the most spot-on take I've seen in this discussion.

AndrewKemendo 3 days ago | parent [-]

Thank you :)

davinoishi 3 days ago | parent | prev [-]

[dead]

bambax 3 days ago | parent | prev | next [-]

At the (very) low end it's pretty easy to build your own "cloud" with a NAS, containers, and reverse proxies and tunnels to the outside world. And this will get you suprisingly far.

But at the high end, I think the market is litterally infinite. Every large company should want this, and want it now. Cloud providers are extremely expensive and, outside of the 1-tier where prices are really outrageous, they perform poorly and often offer little support.

This really feels like the future.

naikrovek 3 days ago | parent | next [-]

like us old assholes were saying when the cloud really started to take off: "this is nuts, it's just someone else's computer! and they're making a profit off of this service, meaning it's more expensive than what we were doing!"

Now a lot of the things that were done pre-cloud were done in bad ways, and I'm not saying that we were right about those things. Having APIs for provisioning and monitoring are far better than submitting a request to some queue and having your VM provisioned manually 1 week later by someone who gets a key detail wrong. APIs and granular permissions are how this should be done, and "the cloud" taught everyone that very early. But a lot of companies are really stuck in the cloud mindset now, and won't let go of it.

I think companies like Oxide and product lines like theirs are going to start becoming common. Microsoft, of course, completely fumbled the ball with Azure Stack, and I've never even heard of anyone deploying AWS Outpost, both for the same reason: the costs for these are absolutely insane for what they provide.

What most folks really want is their own infrastructure running their own stuff using APIs that are either written in-house or provided by some vendor. Oxide is betting that they can sell you a working scalable system for less money than it would take to hire a team to write the APIs that would allow a company to do the same with off-the-shelf hardware. I think that they're probably right about that.

jeffbee 3 days ago | parent [-]

> they're making a profit off of this service, meaning it's more expensive than what we were doing!

I hope you can see that this is a logical fallacy known as "zero-sum thinking". It is not only possible for a business to profit while lowering prices, it is universal throughout the economy. Tomato farmers make a profit selling tomatoes at a price much lower than the cost to grow tomatoes at home. Bakeries radically undercut the cost of home baking. It is obviously cheaper to buy motor fuel at the gas station than it would be to buy crude and refine it yourself.

The main reason people think their on-prem is cheaper than cloud is that they are bad at accounting.

transpute 3 days ago | parent | next [-]

https://arstechnica.com/information-technology/2024/10/basec...

> [2023] 37Signals expected to save $7 million over five years by buying more than $600,000 worth of Dell server gear and hosting its own apps.. [2024] update: it's more like $10 million (and, he told the BBC, more like $800,000 in gear). By squeezing more hardware into existing racks and power allowances.. transferring its 10 petabytes of S3 storage into a dual-DC Pure Storage flash array, 37Signals expects to save money, run faster, and have more storage available.

solatic 2 days ago | parent | prev | next [-]

Your overall point isn't wrong, but...

> Bakeries radically undercut the cost of home baking

This is not always true anymore. I bought a (nice, expensive for its product category) bread machine on Amazon for something like $300. I can bake a loaf of spelt flour bread with nuts and raisins for ~$2 in ingredients at unoptimized grocery store prices. The more-or-less same loaf from a bakery would cost me probably ~$8. Now, that loaf from the bakery would be nicer - artisan etc. - but the bread machine gives a nice crust, I slice it up to throw in the freezer anyway, and the bakery loaf isn't worth the premium to me. I'll see a return on my investment in about a year.

Industrialization should reduce prices while improving quality. In many cases this is true. But it's worth it, sometimes, to have the actual numbers in mind, in case the industrial producers start to forget that they are competing with home processes and raise their prices without consideration. Sometimes, owning your means of production will be cheaper than paying someone else to pay R&D, rent, labor, and marketing.

naikrovek 2 days ago | parent | prev | next [-]

You’re right but this isn’t tomato farming. This is writing software to automate certain actions and expose that automation as an api.

This is an attainable goal for just about any skilled software development team, and the analogy of a tomato farmer doesn’t really fit.

AWS has to provide a generalized solution to serve a wide range of customers and needs, while a company writing its own stuff needs only provide a solution for its own needs.

Damn near everything I do can be expressed as EC2, S3, and Lambda. And IAM.

Those are not challenging APIs to write in order to expose basic functionality and authentication, and such an API existing in my employer 10 years ago would have proven cheaper than AWS, especially for bandwidth; it’s all the manual steps and manual checks that needed to be done within my employer which drove us to AWS quickly.

We could definitely do EC2, S3, and Lambda cheaper than AWS charges us today.

steveklabnik 3 days ago | parent | prev [-]

So, to dig into (and maybe stretch) one of your analogies a little bit:

> Tomato farmers make a profit selling tomatoes at a price much lower than the cost to grow tomatoes at home.

This is because at home, you have an elastic need for tomatoes. One week you need a few, the next week you need none, the next week you need a lot. In that case, yes, growing your own tomatoes would be very silly. (This part of the analogy works with on-demand instances.)

However, if you aren't a home, but you're a busy restaurant, you're not buying tomatoes from the grocery store. You have a regular, fairly fixed capacity, and so you go with a produce vendor who's able to serve that need at a decent price. Part of the reason you're able to get a cheaper price is through volume and due to the regular-ness of the business. (this part of the analogy works with reserved instances.)

Okay, so let's move from tomatoes to the actual building of a restaurant itself. Similar to a reserved instance, renting a building is a decent way to get started with less capital, and you have fairly consistent capacity requirements. But at some point, you realize what McDonalds did: owning the building and land beneath it ends up being a great deal at certain scale. Because that ends up being cheaper still, in the long run. This is closer to on-prem. (Okay at this point this analogy is getting pretty silly but it was fun to try and work through it...)

So, the trick is, for a lot of organizations, they could realize the benefits of owning their own hardware, but to get back to your original analogy, running your own hardware comes with its own set of costs that may make it not worth it. You have to have staff to operate everything, you have to manage all of the various supplier relationships, keep track of software licensing fees, etc etc etc. Even with all of this, as my sibling comment shows, often this can be cheaper than using the cloud.

But one way of looking at Oxide is, we are making it so that it's simpler to own your own hardware, thanks to all of the integration work we do. You don't have to manage a ton of vendor relationships, you have "one throat to choke," as they say. You don't have to keep track of software licensing fees, there are none. You don't need to build out your own software to put the whole thing together, we give it to you. Etc.

So yes, just like a household is best served by going to the supermarket, individuals aren't ever going to buy Oxide. But there's a lot more out there than just households. And larger organizations have fundamentally different needs than they do.

> The main reason people think their on-prem is cheaper than cloud is that they are bad at accounting.

I mean, there's also base accounting stuff that differs significantly between the two, like opex vs capex spend. Not that I'm an expert in that, mind you.

jeffbee 3 days ago | parent [-]

My comment wasn't really about Oxide, it's about the fallacy. A person can't succeed in life thinking they can do something cheaper on the sole basis that the other guy makes a profit. The much more likely explanation is that the other guy makes their profit by being much, much better than you.

To stretch the metaphor to a grotesque extent, I think Oxide stands in the middle between the home-grown and the industrial tomato. Your average corporate IT installation has the same economics as home-grown. Even if they have 1000 potted plants, they are still potted plants, and they are still $100 tomatoes. EC2 is a 5000-acre California tomato grower where the fields have been leveled using lasers and the fruits are harvested by robots driving themselves using on-board GPUs. Their tomatoes cost 5¢. An Oxide computer is like having a 1-acre kitchen garden where the tomatoes are in rows. These are more like $1 each. The economics are undoubtedly better.

steveklabnik 3 days ago | parent [-]

> A person can't succeed in life thinking they can do something cheaper on the sole basis that the other guy makes a profit.

That's fair! Just like, pointing out that on-prem can make financial sense.

> To stretch the metaphor to a grotesque extent,

I like it, haha.

boricj 3 days ago | parent | prev [-]

> At the (very) low end it's pretty easy to build your own "cloud" with a NAS, containers, and reverse proxies and tunnels to the outside world. And this will get you suprisingly far.

Anyone can throw together a bunch of parts and software to run Internet-facing services from a closet. That doesn't mean that you're safe from issues that Oxide aims to solve, especially at that small scale.

My homelab (which hosts my blog and a couple of other things) runs off a Topton N17 micro-ATX motherboard ordered on AliExpress, featuring an AMD Ryzen 7 7840HS. Yes, that's a mobile CPU shoehorned onto a desktop platform with a funky mounting bracket to take AM4/AM5 coolers.

Anyways, I wanted to run SmartOS on it, but this system is so janky that the Illumos kernel couldn't find any PCIe devices at all. After spending an afternoon reconfiguring PCIe bridges by hand with the kernel debugger in an attempt to troubleshoot PCIe initialization, I gave up and installed Proxmox.

Admittedly, as far as janky hardware this takes the cake, but the point stands. To paraphrase Bryan, buggy firmware is the sysadmin's worst enemy.

master_crab 3 days ago | parent | prev | next [-]

I don’t disagree that there are some fat margins in the cloud, but how is vendor lock-in any different here? Companies could end up paying fat margins to oxide too while still managing physical gear and plant.

TZubiri 3 days ago | parent [-]

Well, their servers are mostly hypervisors, so the interface is mostly any virtual machine.

You can "just" migrate by exporting or importing the vms.

yencabulator 3 days ago | parent [-]

You're literally ignoring that the Oxide management stack is very much custom and effectively vendor-locking the purchase to be maintained by them. They are not general purpose PC servers.

You can "just" migrate away from Oxide but that would mean throwing away the hardware you now own. That's the grandparent's point; if you're migrating out of a cloud to avoid the margins demanded by the cloud vendor, now you're at the mercy of whatever Oxide thinks your support contract is worth.

Sure, the convenience may be worth it, but watch how many companies are now struggling to get off of VMWare after Broadcom moves.

TZubiri 3 days ago | parent [-]

This reminds me of people complaining about github being closed source and moving to gitlab, or people obsessing over terraform to avoid cloud locking.

Sure you will have vendor locking at the periferies, but the core is what's important, the guest vms. The hypervisor is whatever. If you have 100 vms running on ec2, you have done a great job of designing portable software, don't obsess over the last 1%.

turnsout 3 days ago | parent | prev | next [-]

I've run my own servers for over 20 years now, but I guess I don't understand the pain point. Can you elaborate? They write:

  > Our system delivers all the hardware and software you need to run cloud…
To "run cloud?" Does this mean treating your own servers like "serverless?" Does it mean running Kubernetes? Is this primarily for people who want to self-host LLMs?

I'm literally so old that I write programs that run on a server and never think about infrastructure.

steveklabnik 3 days ago | parent | next [-]

> To "run cloud?"

I agree that's a bit awkwardly phrased, let me send in a patch for that.

> Does this mean treating your own servers like "serverless?" Does it mean running Kubernetes? Is this primarily for people who want to self-host LLMs?

Not exactly any of that. We let you treat an entire rack as a single pool of resources, spinning up virtual machines that our control plane manages for you. Think "VPS provider but you own it." There's an API, but if you want to see what our console looks like, you can poke around with a demo here: https://console-preview.oxide.computer/

turnsout 3 days ago | parent [-]

That's helpful, thanks!

throw0101c 3 days ago | parent | prev | next [-]

> To "run cloud?" Does this mean treating your own servers like "serverless?" Does it mean running Kubernetes? Is this primarily for people who want to self-host LLMs?

Machine/system/service deployment via API.

See §Essential Characteristics

> On-demand self-service; Broad network access; Resource pooling; Rapid elasticity; Measured service

* https://en.wikipedia.org/wiki/Cloud_computing

None are generally applicable to standalone pizza boxes as managed individual (as opposed to being herded by (e.g.) OpenStack).

skybrian 3 days ago | parent | prev [-]

No GPU’s yet, so it’s not good for LLM’s. But they have a lot of funding now, so perhaps that will change?

geodel 3 days ago | parent | prev [-]

Agree. But those fat margins can get starting to be shared with CIOs, CTOs and other managers with purchasing power. IMO the constant hectoring at workplace about migration to cloud or cloud native crap is not coming from some deep technical principles. It is more of Do it before you get fired for non-compliance