Sure, but if you did a good job, the gradual deployment can go relatively quickly and smoothly, which is how $FAANG roll out new features and products to very large audiences. The actual rollout is usually a bit of an implementation detail of what first needed to be architected to handle that larger scale.

▲

coliveira a day ago | parent | next [-]

The issue with FAANG is that they already have the infrastructure to make these large scale deployments. So any new system - by necessity - needs to conform to that large scale architecture.

▲

brendoelfrendo 20 hours ago | parent [-]

The other nice thing about FAANG is that almost nothing they do is actually necessary. If Facebook rolls out a new feature and breaks something for a few hours, it doesn't actually matter. It's harder to move fast and break things if you're, say, a bank, and every minute of downtime is a minute where your customers can't access their money. Enough minutes go by and you may have a very, very expensive crisis on your hands.

	▲	lanstin 11 hours ago \| parent \| next [-]
		Replying to myself in sibling: except maybe people paying for ads, which is more of a faith based action; it's well known a lot of ad traffic is fraudulent, but not which traffic. So if you pay for ads, who can tell what happened.
	▲	lanstin 11 hours ago \| parent \| prev [-]
		Yeah, people get real upset about even 1 messed up money transaction.

▲

vlovich123 a day ago | parent | prev | next [-]

You get certain big pieces correct maybe but you’d be surprised how many mistakes get made. For example, I had designed the billing system for a large distributed product that the engineer ended up implementing not as described in the spec which fell down fairly quickly with even a modicum of growth.

▲

eru a day ago | parent | prev | next [-]

Well, Google got good at large scale rollouts, because they are doing large scale rollouts all the time. _And_ most of the time, the system they are rolling out is a small iteration from the last system they rolled out: the new GMail servers look almost exactly like the last GMail servers, but they have on extra feature flag you can turn on (and which is disabled by default) or have one bug fixed.

That's a very different challenge from rolling out a brand new system once.

▲

mmooss 20 hours ago | parent | prev [-]

FAANG tests first on test beds, and on subsets of their user base.

	▲	agos 19 hours ago \| parent [-]
		also, see what happened last week when Cloudflare pushed out a bad configuration without trying it on a subset