Remix.run Logo
PunchyHamster a day ago

> I would argue that a good CI/CD system should not support secrets as a first-class object at all. Instead steps may have privileges assigned. At most there should be an adapter, secure enclave style, that may hold a secret and give CI/CD steps the ability to do something with that secret, to be used for APIs that don’t support OIDC or some other mechanism to avoid secrets entirely.

CI/CD does not exist in the vacuum. If you had CI/CD entirely integrated with the rest of the infrastructure it might be possible to do say an app deploy without passing creds to user code (say have the platform APIs that it can call to do the deployment instead of typical "install the client, get the creds, run k8s/ssh/whatever else needed for deploy").

But that's a high level of integration that's very environment specific, and without all that many positives (so what you don't need creds, you still have permission to do a lot of mess if it gets hijacked), and a lot, lot more code to write vs "run a container and pass it some env vars" that had become a standard

amluto a day ago | parent | next [-]

You seem to be talking mostly about the CD part. Some thoughts:

On the one hand, CD workflows are less exposed than CI workflows. You only deploy code that has made it through your review and CI processes. In a non-continuous deployment model, you only deploy code when you decide to. You are not running your CD workflow on a third-party pull request.

On the other hand, the actual CD permission is a big deal. If you leak a credential that can deploy to your k8s cluster, you are very, very pwned. Possibly in a manner that is extremely complex to recover from.

I also admit that I find it rather surprising that so many workflows have a push model of deployment like this. My intuition for how to design a CD-style system would be:

1. A release is tagged in source control.

2. Something consumes that release tag and produces a production artifact. This might be some sort of runner that checks out the tagged release, builds it, and produces a ghcr image. Bonus points if that process is cleanly reproducible and more bonus points if there's also an attestation that the release artifact matches the specified tag and all the build environment inputs. (I think that GitHub Actions can do this, other than the bonus points, without any secrets.)

3. Something tells production to update to the new artifact. Ideally this would trigger some kind of staged deployment. Maybe it's continuous, maybe it needs manual triggering. I think that, in many production systems, this could be a message from the earlier stages that tells an agent with production privileges to download and update. It really shouldn't be that hard to make a little agent in k8s or whatever that listens to an API call from a system like GitHub Actions, authenticates it using OIDC, and follows its deployment instructions.

P.S. An attested-reproducible CD build system might be an interesting startup idea.

PunchyHamster a day ago | parent [-]

Well, in my mind the build system should build an artifact (a container, or a .deb package), and then the separate system should deploy it (with smaller amount of permitted people), and have option to roll it back. So in principle I agree on that .

...but I saw that anti-pattern of "just add a step that does the deploy after CI in same" often enough that I think it might be the most common way to do it.

Kinrany a day ago | parent | prev [-]

CI shouldn't do deployments, deployment pipelines should run separately when a new release passes CI

Of course the general purpose task runner that both run on does need to support secrets

arccy a day ago | parent | next [-]

you conveniently ignored the "CD" part of CI/CD

Kinrany 21 hours ago | parent [-]

As mentioned by sibling comments, these two parts are very different. GHA and alternatives are primarily for CI; many people choose to use it solely for CI. For these reasons I did ignore CD.

Kinrany a day ago | parent | prev | next [-]

Hmm, I have long assumed that a perfectly executed CI/CD setup would be based on a generic task runner... But maybe not?

Only the CI part needs to build; it needs little else and it's the only part of a coherent setup that needs to build.

regularfry a day ago | parent | prev [-]

We're iterating towards GHA for CI, AWS CodeBuild for the CD. At least on AWS projects. Mainly because managing IAM permissions to permit the github runner to do everything the deployment wants is an astonishingly large waste of time. But you need a secret to trigger one from the other.

jamescrowley a day ago | parent [-]

You actually don’t need (long-lived / hard-coded) secrets in this scenario if you use OIDC:

https://docs.github.com/en/actions/how-tos/secure-your-work/...

regularfry a day ago | parent | next [-]

Technically yes. It depends on whether you consider the account ID to be a secret or not (AWS say "sensitive but not secret" which doesn't help much). But also it can make sense to treat all environment variables as secrets by default just so you don't accidentally end up putting something somewhere that turns out to have been Wrong.

Kinrany 21 hours ago | parent [-]

GP is saying that GHA would need zero information about AWS if CodeBuild used a Github token and listened for GHA runs.

regularfry 19 hours ago | parent [-]

That may be true, but it's not what the link describes.

Kinrany 19 hours ago | parent [-]

Fair!

everfrustrated a day ago | parent | prev [-]

And even better can scope assuming an AWS IAM role to a specific branch name & workflow filename so only code/workflows that have been through review have access to CD secrets/prod infra.

IE no prod access by editing the workflow definition and pushing it to a branch.