I’ll bite. We’re expanding into Europe. I literally copied and pasted our entire infrastructure into a new folder named “Europe”

Now there’s a new requirement that only applies to Europe and nowhere else and it’s super easy and straight forward to change the infrastructure.

I don’t see how it was a poor choice to literally copy and paste configs that result in hundreds of thousands of lines of yaml and I have 25 yoe.

▲

chipsrafferty 9 days ago | parent | next [-]

I think the most common way to approach that problem would be to have a "default config", and overrides. Could you go into more detail about why you didn't do this instead?

Downsides with your approach is:

1. Now whenever you want to change something both in Europe and (assuming) USA you have to do it in 2 places. If the change is the same for both, in my system, you could just update the default/shared config. If the change is different for both it's equally easy, but faster, since the overrides are smaller files.

2. It's not clear what the difference is between Europe and USA if there is 1 line different amongst thousands. If there are more differences in the future, it becomes increasingly difficult to tell the difference easily.

3. If in the future you also need to add Africa, you just compounded the problems of 1. and 2.

▲

MrDarcy 9 days ago | parent [-]

I don’t do this because with a complete copy I get progressive rollouts across regions without the complexity of if statements and feature flags. That is to say, making the change twice is a feature not a bug when the changes are staggered in time.

From an operational perspective it’s much more important to ensure the code is clear and readable during an incident.

Overrides are like inheritance. They are themselves complex and add unnecessary cognitive load.

Composition is better for the common pieces that never change across regions. Think of an import statement of a common package into both the Europe and North America folders.

I easily see the one line diff among hundreds of thousands using… diff.

Regarding Africa, we’ve established 1 is a feature and 2 is a non issue, so I’d copy it again.

This approach scales both as the team scales and as the infrastructure scales. Teammates can read and comprehend much more easily than hierarchies of overrides, and changes are naturally scoped to pieces of the whole.

	▲	seadan83 9 days ago \| parent \| next [-]
		The "rules" for config are different. Code, test code, and config are different, their complexity scales in different ways of course. By way of analogy for why the two configs are different, for example Two beaches are not the same because they both have very similar sand. You really have two different configs.. You also have one set of configs. You didn't set up an application that also fetches some config that is already provided. It would be like having a test flag in both config and database, sane flag - two places. Where config duplication goes bad is when repeatedly the same change is made across all N, local variations have to be reconciled each time and it is N sets of testing you need to do. Something like that in code is potentially more complex, more obviously a duplication of a module, just more likely to be a problem overall.
	▲	chipsrafferty 6 days ago \| parent \| prev [-]
		> with a complete copy Ok, so make a build script that creates a complete copy for each separate config from the default config and the overrides. Then use the complete copies. > I easily see the one line diff among hundreds of thousands using… diff. Yes, and in 5 years when it's not just 1 line? Your approach does NOT scale. You just haven't scaled yet and haven't realized this.

▲

tasuki 9 days ago | parent | prev [-]

> I don’t see how it was a poor choice to literally copy and paste configs that result in hundreds of thousands of lines of yaml

Perhaps one day you will. I'm a dev who worked with infra people who had your philosophy: many copy pasted config files.

Sometimes I needed to add an env var to a service. Expressing "default to false and only set it to true in these three environments" took changing about 30 files. I always made mistakes (usually of omission), and the infra people only ever caught them at deployment time. It was hell.