> I've not understood the propensity for using yaml for CI pipelines and workflows in general. A decent programming language would be a big improvement.

Because it's clear to write and read. You don't want your CI/CD logic to end up being spaghetti because a super ninja engineer decided they can do crazy stuff just because they can. Same reason why it's a bad idea to create your infrastructure directly in a programming language (unless creating infrastructure is a core part of your software).

> Why not just build the workflows themselves as docker images? I guess running other docker images in the workflow would then become a problem.

That's how Drone CI handled it. GitLab kind of does the same, where you always start as a docker image, and thus if you have a custom one with an entrypoint, it does whatever you need it to.

IME on a Pulumi for IaC team, writing infra in a real language (TypeScript) is MILES better than- you can do conditions, have typed outputs, etc and not have it be a bastardized imperative YAML mess.

YAML is fine for data, but inevitably stuff like workflows end up tacking on imperative features to a declarative language.