Avoid cache-from.
According to https://github.com/moby/moby/issues/34715, using --cache-from
disables use of the local cache. Instead, if multiple images are used to
cache from, and multiple of them have a match, in some circumstances one of
them is chosen and the rest is ignored.
Try to avoid this problem by simply fetching all tags of an image, and hope that they serve as a local cache.
Another problem are multi-stage builds, since the intermediate stages are not
tagged, and thus cannot be retreived to re-fill the cache. But this happens
with or without --cache-from
.