Feature #4040
closedUse "COPY --from" in Dockerfiles to avoid cloning from scratch again and again
0%
Description
Some of our Dockerfiles clone a lot of git repositories (e.g. those for the ttcn3 tests), and due to the sequential nature of docker cache invalidation, a change in osmo-ttcn3-hacks.git not only means we will re-clone that git repository from scratch, but we also will clone all of the 'deps' repositories from scratch.
If we had a docker container hosting pre-created clones of those repositories, we could use the "COPY --from" syntax in our Dockerfile, copying those initial clones of the git repositories over. We'd then only have to pull whatever changes (if any) over the network.
See https://docs.docker.com/develop/develop-images/multistage-build/ for a description of this feature towards the bottom.
This would- significantly reduce the network traffic on the buildhost during re-generation of images
- reduce the load to the respective git servers
- speed up the build process
The container hosting the "pre cloned git repos" could then be updated only once per week or even once per month. It should not be rebuilt too often, as otherwise it will have to be pulled all the time, avoiding the benefits stated above and just move traffic from git to docker.
Updated by laforge almost 5 years ago
The section on the docker documentation is Use an external image as a “stage”
Updated by osmith about 3 years ago
Looks like we can close this with the changes from #5017, which solves the problem of cloning deps over and over.
Now it gets only updated when:The container hosting the "pre cloned git repos" could then be updated only once per week or even once per month. It should not be rebuilt too often, as otherwise it will have to be pulled all the time, avoiding the benefits stated above and just move traffic from git to docker.
- deps/Makefile in osmo-ttcn3-hacks.git changes
- debian-stretch-titan/* changes in docker-playground.git
laforge: can we mark it as rejected (duplicate)?