Yes, cron is a bit undervalued in that for one off (well locked) tasks it's perfectly fine to create a crontab entry. And simplicity is king. I feel people throw frameworks at problems where a simple shell/go script in a cron would be just enough.
As for the pipeline definition. One goal is to have a notion of pipelines that is both comprehensive and declarative.
As for a database, what would you store there? Container image to run? Past execution data (e.g. output path, time, errors)?
The software world has many pipeline-y things, such as CI definitions and these definitions usually live in configuration files.
What is difficult at time is the tracking of done tasks. Is the output a file or a new row in some database or many files or many rows or anything else?
As for the pipeline definition. One goal is to have a notion of pipelines that is both comprehensive and declarative.
As for a database, what would you store there? Container image to run? Past execution data (e.g. output path, time, errors)?
The software world has many pipeline-y things, such as CI definitions and these definitions usually live in configuration files.
What is difficult at time is the tracking of done tasks. Is the output a file or a new row in some database or many files or many rows or anything else?