I don't have any first hand experience with customer facing startups, SaaS or otherwise. How common is rigorous testing in the first place?

As you scale it improves. More often at a small scale you ask users and they’ll give you invaluable information. As you scale you abstract folks into buckets. At about 1million MAU I’ve found A/B testing and p-value starts to make sense.