Remix.run Logo
0xbadcafebee 6 days ago

I see they're still using the old 2000s-style "dev", "test", "prod" environments. How many developers here are aware that this is a bad pattern? Do you use it even though you know it's a bad pattern, and if so, why?

aloknnikhil 5 days ago | parent | next [-]

Since when did having isolated environments for dev / test and prod become a bad pattern?

0xbadcafebee 4 days ago | parent [-]

Since around the time "cattle not pets" philosophy was coined. dev and test are pets, ephemeral environments are cattle.

overfeed 5 days ago | parent | prev | next [-]

> How many developers here are aware that this is a bad pattern?

I've been around the block and I think dev/staging/prod is a great pattern! Anyone telling otherwise may be cargo-culting hyperscalers who:

1. Can't replicate their prod environment without spending tens/hundreds of billions in capex

2. Have spent an aggregate of billions investing in developing tools, processes, core-infrastructure teams, and specialized SREs to monitor and keep prod healthy.

2.1 Have tools supporting gated release to a fraction of their users with automated anomaly detection. FANGs can easily test changes against 0.01% of their userbase and get meaningful data back.

If you're not constrained by (1) and don't have the benefits of (2), dev/test/prod is an excellent strategy you should use instead of testing in production for no reason.

0xbadcafebee 4 days ago | parent [-]

I have seen tiny teams fail to replicate their single-host prod environment in dev and test.

I have seen tiny teams with 3 release branches cause outage after outage of their tiny SaaS product as they couldn't keep track of what changes they merged where.

I have seen days of downtime when nobody could re-create their old-ass environments that nobody ever documented or updated, their code dependent on systems that didn't even have an active repository anymore.

I have seen C-beams glitter in the dark by the tanhauser gate.

All these moments, lost in time, like ssh keystrokes in a container.

diamondfist25 5 days ago | parent | prev [-]

is it a bad pattern? i'd love to know more cuz im starting on this route for my hetzner k3s setup

0xbadcafebee 4 days ago | parent [-]

I hesitate to call it an anti-pattern but it kind of is. It leads you down some bad paths.

- In the best possible scenario: 1) you're spending time to duplicate configuration between dev, test, and prod, any time the environments change, 2) you hope your testing in one environment is actually borne out in prod, 3) you are shutting down dev & test when they're not in use (to save money), 4) you rebuild it all regularly to ensure it's still actually repeatable to repair or rebuild it, 5) you do maintenance on all 3 regularly to stay up to date on security patches and dependencies, 6) you still do trunk-based development that just deploys to each environment successively with automated functional tests, which makes one of the environments (either dev or test) unnecessary, and the other inaccurate

- In the more likely scenario: 1) your "duplication" of configuration is inaccurate or literally not the same, leading to differences which cause bugs, 2) your results in prod don't mimic those in test, 3) you aren't shutting down dev/test when not in use, leading to not only unnecessary charges, but potential security issues if they get compromised (because who secures dev/test?), 4) you never rebuild it, so when there's an issue and you need to repair or replace it, you have no idea how to do it again, or the automation doesn't work, so you spend more time fixing it while your stuff is down, 5) you never do maintenance, leading to failed disks, out of date software, and general maintenance headaches, 6) you are probably not doing trunk based development, but cherry-picking changes from a "dev", "test", and "prod" branch, ending up with diverging codebases which don't match each other, and is a nightmare to debug and maintain

That's the very short version of it, anyway. Thankfully there is a modern method which solves all those problems.

Only use ephemeral environments, and production. The ephemeral environment is built from scratch whenever you want to test, and is shut down automatically after idleness. This ensures trunk-based development, allows multiple independent test deployments with specific feature-branch changes to exist simultaneously, re-tests all your automation constantly, uses the latest updates to avoid dependency stagnation, is versioned, saves money and resources, and eliminates security concerns.

Faster development, fewer bugs, less money, less maintenance toil, less downtime, more security. But it's not popular on HN so nobody does it.