| ▲ | belter 16 hours ago |
| Many of these scaling issues would be solved, if they simply used AWS and Aurora Postgres. https://pages.cs.wisc.edu/~yxy/cs764-f20/papers/aurora-sigmo... https://youtu.be/pUqVCK7Ggh0 |
|
| ▲ | sjdidhshab 13 hours ago | parent | next [-] |
| Aurora is Postgres compatible but it’s not equivalent. It was my understanding that the underlying implementation was distinctly different from regular Postgres. So while maybe not as risky as dropping in MySQL, dropping in aurora is still akin to switching to a brand new db and certainly not simple. Or are you saying they should have started on Aurora from the start? |
| |
| ▲ | belter 11 hours ago | parent [-] | | Yes they should have started with Aurora Postgres. Note that for Aurora Postgres you can just import a Postgres backup and its the same SQL Code. Its Postgres+plus.... |
|
|
| ▲ | sgarland 13 hours ago | parent | prev | next [-] |
| Oh yeah, separating compute and storage by a large physical distance was a great idea that certainly had no downsides! It's so awesome that they added "Optimized Reads" as an option, which is literally just running the DB on a server with a local NVMe drive - you know, how people used to do things. The only feature that Aurora (MySQL) has that is remotely impressive is its ability to restart the DB process without losing the buffer pool. Aurora (Postgres) has no interesting differentiations. I've benchmarked both, with prod-like workloads, against some 12 year old Dell R620s I have, which have NVMe drives exposed via Ceph over Infiniband. The ancient servers handily beat Aurora and RDS on everything except when the latter had an instance type with a local NVMe drive, at which point it's just superior clock speed and memory throughput. I despise Aurora with a burning passion. AWS successfully hoodwinked companies everywhere with bullshit, and are absolutely raking in cash because of it. |
| |
| ▲ | evanelias 8 hours ago | parent | next [-] | | > The only feature that Aurora (MySQL) has that is remotely impressive Aurora is one of the only options if you need low-lag physical replication in a MySQL-compatible environment. That makes it operationally feasible to execute large/heavy writes or DDL which would normally cause too much replication lag on traditional (async binlog-based) MySQL replicas. Granted, there's some important fine print: long transactions will still block InnoDB purge of old row versions, and in Aurora that's cluster-wide. But in any case, personally I'd consider nearly-lag-free replication to be an important differentiator. This can be leveraged in interesting ways, for example CashApp's `spirit` OSC tool (https://github.com/block/spirit) can do online schema changes blazing-fast because it doesn't need to throttle its write rate to avoid replication lag. Scale-to-zero is also nice for dev/test environments. That said, I do agree with your overall point that Aurora was majorly over-marketed. And Amazon's capture of so much revenue in the MySQL space has been extremely detrimental for the MySQL ecosystem, especially considering Aurora's modifications are proprietary/closed-source. | | |
| ▲ | sgarland 2 hours ago | parent [-] | | Re: lag, that’s a fair point, but IME devs don’t care if it’s 10 msec or 1000 msec, they don’t trust it either way. “I have to read after write.” I know MySQL doesn’t have RETURNING, maddeningly, but I’ve tried explaining how they could use LAST_INSERT_ID() to no avail. |
| |
| ▲ | belter 11 hours ago | parent | prev [-] | | > The only feature that Aurora (MySQL) has that is remotely impressive is its ability to restart the DB process I dont really care about Aurora MySQL...only Aurora Postgres, but you forgot about Parallel Query and Clones. For clones you dont pay for the extra
storage for the new database, only the delta if you add new data... https://aws.amazon.com/blogs/aws/new-parallel-query-for-amaz... https://aws.amazon.com/blogs/aws/amazon-aurora-fast-database... "...AWS successfully hoodwinked companies everywhere with bullshit, and are absolutely raking in cash because of it." Really?... "How Twilio modernized its billing platform on Amazon Aurora MySQL" - https://aws.amazon.com/blogs/database/how-twilio-modernized-... "No observable Aurora downtime taken in over 5 months of experimentation, and almost 2 months of running shadow production.. Steady state metrics on over 40 accumulated days of live production data across all Aurora clusters: - Over 46 billion transaction records indexed and available, compared to less than one billion stored in the former online Redis system - 4.8 TB of data across all tables - Over 11,000 active database connections to all clusters - Less than 10 milliseconds median end-to-end transaction run latency - Less than 60 milliseconds 99th percentile end-to-end transaction run latency..." "Increasing Scalability and Reducing Costs Using Amazon Aurora Serverless with BMW" - https://aws.amazon.com/solutions/case-studies/bmw-group-auro... "FINRA CAT selects AWS for Consolidated Audit Trail" - https://aws.amazon.com/blogs/publicsector/finra-cat-selects-... https://aws.amazon.com/rds/aurora/customers/ | | |
| ▲ | sgarland 9 hours ago | parent [-] | | > For clones you dont pay for the extra storage for the new database, only the delta if you add new data... Considering how much they’re charging you just to query storage, that’s still a net negative. If anything, you’re going to pay MORE since you’re probably querying more. > No observable Aurora downtime taken in over 5 months of experimentation I manage somewhere north of 500 Aurora instances spread across dozens of clusters. We have one drop out at least weekly, if not more often. > Over 46 billion transaction records indexed and available, compared to less than one billion stored in the former online Redis system This isn’t unique to Aurora. > 4.8 TB of data across all tables Neither is this; also, it’s honestly not that big. I doubt we’re going to convince each other of anything here. | | |
| ▲ | belter 9 hours ago | parent [-] | | > We have one drop out at least weekly, if not more often. You mean an instance? A cluster wont go down because of that. I dont work for AWS :-) and dont want to convince you of anything. But there is a reason why they developed Aurora, and DynamoDB and it was not because some software developer had hours to waste... | | |
| ▲ | sgarland 2 hours ago | parent [-] | | Yes, an instance. You know you can set up hot standby for vanilla MySQL and Postgres as well and achieve the same thing, right? They developed those to make a shit ton of money, and on that front, they succeeded. |
|
|
|
|
|
| ▲ | belter 15 hours ago | parent | prev | next [-] |
| Surely someone can articulate the flaw...Unless, of course, there isn’t one worth mentioning.... |
| |
| ▲ | cbg0 15 hours ago | parent [-] | | They're deployed on Azure and have a deep partnership with Microsoft, so they can't "simply" use a different cloud. Also, recommending a black box managed solution isn't an option for some large companies that have their own hardware & datacenters and which may want to use open source solutions they can easily deploy, fork and support themselves to keep costs under control. | | |
| ▲ | belter 14 hours ago | parent [-] | | They are one of the most well capitalized company/startup/foundation/non-profit in the planet and just spent 6,5 billion to hire a designer. They should be using the best technical and cheapest solution, and they owe it to their investors. At their scale they will never be able to use anything else than a cloud solution. They could solve these issues at the number of users they report, for a monthly bill below 25 million dollars. "6,311 database instances running the PostgreSQL-compatible and MySQL-compatible editions of Amazon Aurora processed more than 376 billion transactions, stored 2,978 terabytes of data, and transferred 913 terabytes of data" - https://aws.amazon.com/blogs/aws/how-aws-powered-prime-day-2... | | |
| ▲ | CharlesW 10 hours ago | parent | next [-] | | > At their scale they will never be able to use anything else than a cloud solution. That's definitely not true, and there are many companies doing higher volumes at a fraction of the cost-per-query. Although scale doesn't force companies into public-cloud database systems, considerations like capital, time-to-market, and business strategy often do. In this case, OpenAI is trading a significantly higher per-query cost for benefits like improved agility, turnkey compliance, etc. | |
| ▲ | iampims 14 hours ago | parent | prev [-] | | but that'd be real money, not the Monopoly money they used to buy Ive/Windsurf... |
|
|
|
|
| ▲ | v5o 15 hours ago | parent | prev [-] |
| [dead] |
