Pretty much all pipeline services share the same architecture pattern:
* A pipeline run is comprised of one or more build jobs,
* Pipeline runs are triggered by external events
* Build jobs have contexts and can output artifacts,
* Build jobs are grouped into stages,
* Stages are organized as a directed graph,
* Transitions between stages in the directed graph is ruled by a set of rules, some supported by default (i.e., if a job fails then the stage fails) complemented by custom rules (manual or automatic approvals, API tests, baking periods, etc).
This is the textbook scenario ideal for DSLs. You already are bound to an architecture pattern, this there is no point of reinventing the wheel each time. Just specify your stages and which jobs run as part of each stage, manage artifacts and promotion logic, and you're done.
You do not need to take my word for it. Take a look at GitLab CICD for a pipeline with build, test, and delivery stage. See what a mess you will put together if you support the same feature set with whatever scripting language you choose. There is no discussion or debate.
I can’t understand how you can say DSL and YAML in the same sentence and say it’s fine. YAML is a serialization format. A bad DSL would be a welcome improvement over GHA pipelines in YAML. You’re fundamentally confusing concepts here, you want to restrict flexibility (I agree with that btw) by using a simplistic language, but what it actually does is increase complexity of the code comprising the pipeline with zero hard restrictions.
The problem starts when that graph cannot be determined in advance and needs to be computed in runtime. It's a bit better when it's possible to compute that graph as a first step, and it's a lot worse when one needs to do a couple of stages before being able to compute the next elements of the graph. The graph computation is terrible enough in e.g. Groovy, but having to do it in YAML is absolutely horrendous.
> Take a look at GitLab CICD for a pipeline with build, test, and delivery stage
Yeah, if your workflow fits in a kindergarten example of "build, test, and delivery", then yeah, it's YAML all the way baby. Not everyone is so fortunate.
It's funny how you say this is the textbook scenario ideal for DSLs and I see it as the textbook scenario ideal for a real programming language. Organizing stages as a DAG with "transition ruled by a set of rules" is bonkers, I know how to write code with conditional logic and subroutine calls, give that to me.
Wrapping it in a DSL encoded as YAML has zero benefit other than it being easier for a team with weak design skills to implement and harder for users to migrate off of.
We don't disagree here. There are tools which support you in doing this, and I mentioned a few of them in my post (Make, Just, doit, mage). There are many more. I also think that re-inventing these tools is a waste of time, but it is still better than shoehorning this into YAML. You seem to think YAML is some kind of DSL for pipelines. It really is not.
No, it really isn't. I'll clarify why.
Pretty much all pipeline services share the same architecture pattern:
* A pipeline run is comprised of one or more build jobs,
* Pipeline runs are triggered by external events
* Build jobs have contexts and can output artifacts,
* Build jobs are grouped into stages,
* Stages are organized as a directed graph,
* Transitions between stages in the directed graph is ruled by a set of rules, some supported by default (i.e., if a job fails then the stage fails) complemented by custom rules (manual or automatic approvals, API tests, baking periods, etc).
This is the textbook scenario ideal for DSLs. You already are bound to an architecture pattern, this there is no point of reinventing the wheel each time. Just specify your stages and which jobs run as part of each stage, manage artifacts and promotion logic, and you're done.
You do not need to take my word for it. Take a look at GitLab CICD for a pipeline with build, test, and delivery stage. See what a mess you will put together if you support the same feature set with whatever scripting language you choose. There is no discussion or debate.