The only caveat being this assumes all your data can fit on a single machine, and all your processing can fit on one machine. You can get a a u-24tb1.112xlarge with 448 vcores, 24TB RAM for 255/hour and attach 64TB of EBS -- that's a lot of runway.

▲

tuhgdetzhh 5 hours ago | parent | next [-]

Or rent a bare-metal machine from hetzner with 2-3x performance per core and 90% less costs[1].

[1] Various HN posts regarding Hetzner vs AWS in terms of costs and perf.

▲

tetha 5 hours ago | parent | next [-]

In my experience, a decently managed database scales very hard.

3x EX44 running Patroni + PostgreSQL would give you 64GB of working memory, at least 512 GB NVMe of dataset (configurable with more for a one-time fee) at HA + 1 maintenance node. Practically speaking, that would have carried the first 5 - 10 years of production at the company I work at with ease, for 120 Euros hardware cost/month + a decent sysadmin.

I also know quite a few companies who toss 3-4x 20k - 30k at DELL every few years to get a database cluster on-prem so that database performance ceases to be a problem (unless the application has bad queries).

▲

ethanwillis 4 hours ago | parent | prev | next [-]

This might be true in terms of direct monetary costs.

I want to like Hetzner but the bureaucratic paper process of interacting with them and continuing to interact with them is just... awful.

Not that the other clouds don't also have their own insane bureaucracies so I guess it's a wash.

I'm just saying, I want a provider that leaves me alone and lets me just throw money at them to do so.

Otherwise, I think I'd rather simply deploy my own oversized server in a colo even with the insanely overpriced hardware prices currently.

edit: And shortly after writing this comment I see: "Microsoft won't let me pay a $24 bill, blocking thousands in Azure spending" https://news.ycombinator.com/item?id=46124930

▲

earthnail 4 hours ago | parent | next [-]

Can you elaborate on what the bureaucracy is you experienced? I'm a Hetzner customer since last month and so far I thoroughly enjoy it. Have not encountered any bureaucracy yet.

	▲	ethanwillis 2 hours ago \| parent \| next [-]
		I think I was still being a bit too harsh even after throwing into my comment that other providers aren't perfect either. But basically after the initial paperwork I had some issues with my account getting flagged even though I wasn't using it 99.999% of the time. It's not a huge deal for me because I wasn't trying them out for anything serious. I just questioned how often that might happen if I was actually using it seriously and what kind of headaches it could cause me while re-verifying everything with them. From people I know if everything is going good then their service is great. Server performance is good, pricing is good, etc.
	▲	wesammikhail 16 minutes ago \| parent \| prev [-]
		I was asked for a passport photo when I tried to open an account. They literally asked for a passport photo immediately after the signup form. Like WHAT? I couldn't believe my eyes. The most insane shit I've ever seen.

▲

Nextgrid 4 hours ago | parent | prev [-]

> I want a provider that leaves me alone and lets me just throw money at them to do so.

That’s been my experience with Hetzner.

A lot of people get butthurt that a business dares to verify who they’re dealing with as to filter out the worst of the worst (budget providers always attract those), but as long as you don’t mind the reasonable requirement to verify your ID/passport they’re hands-off beyond that.

	▲	ethanwillis 2 hours ago \| parent [-]
		That's fair and I don't have any major issues with that. I guess my concern on the bureaucracy is if you are unlucky enough to get flagged as a false positive it can be an annoying experience. And I can't really blame them too hard for having to operate that way in an environment of bad actors. You're definitely right that the budget providers do attract the types of people trying to do bad things/exploit them in some way.

▲

5 hours ago | parent | prev [-]

[deleted]

▲

adityaathalye 5 hours ago | parent | prev | next [-]

Heh, the documentation calls out the limits. Maximum (theoretical) DB size is 281TB: https://sqlite.org/limits.html

> This particular upper bound is untested since the developers do not have access to hardware capable of reaching this limit.

> However, tests do verify that SQLite behaves correctly and sanely when a database reaches the maximum file size of the underlying filesystem (which is usually much less than the maximum theoretical database size) and when a database is unable to grow due to disk space exhaustion.

▲

DenisM 5 hours ago | parent | prev | next [-]

Scale-up solves a lot of problems for stable workloads. But elasticity is poor, so you either live with overprovisinoed capacity (multiples, not percentages) or fail under spiky load which often time is the most valuable moment (viral traffic, Black Friday, etc).

No one has solved this problem. Scale out is typically more elastic, at least for reads.

▲

kragen 5 hours ago | parent | next [-]

That's a good point, but when one laptop can do 102545 transactions per second, overprovisioned capacity is kind of a more reasonable thing to use than back when you needed an Amdahl mainframe to hit 100 transactions per second.

▲

DenisM 3 hours ago | parent [-]

As compute becomes cheaper your argument becomes more and more true.

But it only works if workloads remain fixed. If workloads grow at similar rates you’re back to the same problem.

	▲	kragen 3 hours ago \| parent [-]
		Well, it doesn't work for the newly added workloads. But for the most part we instead have the same workloads performed less efficiently.

▲

CuriouslyC 3 hours ago | parent | prev | next [-]

I love hetzner for internal resources because they're not spikey. For external stuff I like to do co-processing, you can load balance to cloudflare/aws/gcp services like containers/Run/App Runner/etc.

▲

masterj 2 hours ago | parent | prev [-]

I suspect that for a large number of orgs accepting over-provisioning would be significantly cheaper than the headcount required for a more sophisticated approach while allowing faster movement due to lower overall complexity

▲

5 hours ago | parent | prev | next [-]

[deleted]

▲

embedding-shape 5 hours ago | parent | prev | next [-]

> The only caveat being this assumes all your data can fit on a single machine

Does my data fit in RAM? https://yourdatafitsinram.net/

Not sure using EC2/AWS/Amazon is a good example here, if you're squeezing for large single-node performance you most certainly go for dedicated servers, or at least avoid vCPUs like a plague.

	▲	jandrese 5 hours ago \| parent \| next [-]
		That site is a bit questionable. I entered "64TB" as the answer and it was very happy to show me a bunch of servers that maxed out at 6 or 8TB. Even the one server that listed 64TB of RAM might be questionable since it's not leaving room for the OS or your applications. That said 64 TB is a gargantuan amount of data, so I'm not too worked up over it not fitting in RAM. Lord help you if you have a power outage and have to reload the data from disk.
	▲	ErroneousBosh 4 hours ago \| parent \| prev \| next [-]
		Getting on for ten year's worth of forum posts on https://rangerovers.pub/ comes to about 32MB of SQL dump. So yeah, easily.
	▲	paulddraper 3 hours ago \| parent \| prev [-]
		How does 25 TiB fit in RAM when the max machine has 24 TB?

▲

kiitos 3 hours ago | parent | prev [-]

and that your application doesn't need to be resilient to host or network faults