Split builds into releases
Proposal
Currently the whole pipeline first builds service images, then os-images, followed by the dune core images, dependency images and the discretization module images. Only if all of them are build successfully, the images are uploaded to the registry. The whole process of building all the images takes 4-5 hours (see #12 (closed)) and when run overnight fail most of the time.
I suggest to split these big build chains into smaller chunks that can be build independently:
- service images
- os images
- master debian images + releases 2.4/2.5/2.6/2.7/2.8 images
- dependency images
- distribution modules
If the service images are rebuild, the other images do not need to be rebuild as well. But, if the os images are rebuild, all the other chains should be updated. I think, it is not necessary to update the debian or ubuntu images every night. There should not be a big change in the images. Maybe a frequent update is fine, to get latest security updated included.
The master branches should be build nightly, but the release branches should be build only on request since they should be quiet stable.
The dependency images and also the distribution images should only be rebuild if from a master branch or if the corresponding upstream images (i.e. the core image) was updated.
These dependency chains could be implemented with branches and pipeline triggers in the deploy step. Let's say the os images are rebuild. If this succeeds the images are pushed to the registry, triggers a pipeline and in the next step all the core images are rebuild. IF the core image of one branch are pushed to the registry this triggers another pipeline to start the dependency or distribution branches to be build.
One could even trigger a pipeline to rebuild the release branches only if someone pushes to the corresponding release branch in a dune module. The (and the update of the os image) are the only reasons to rebuild a release version of dune.