Project

General

Profile

Feature #4040

Use "COPY --from" in Dockerfiles to avoid cloning from scratch again and again

Added by laforge 5 months ago. Updated 5 months ago.

Status:
New
Priority:
Low
Assignee:
sysmocom
Target version:
-
Start date:
06/04/2019
Due date:
% Done:

0%

Spec Reference:

Description

Some of our Dockerfiles clone a lot of git repositories (e.g. those for the ttcn3 tests), and due to the sequential nature of docker cache invalidation, a change in osmo-ttcn3-hacks.git not only means we will re-clone that git repository from scratch, but we also will clone all of the 'deps' repositories from scratch.

If we had a docker container hosting pre-created clones of those repositories, we could use the "COPY --from" syntax in our Dockerfile, copying those initial clones of the git repositories over. We'd then only have to pull whatever changes (if any) over the network.

See https://docs.docker.com/develop/develop-images/multistage-build/ for a description of this feature towards the bottom.

This would
  • significantly reduce the network traffic on the buildhost during re-generation of images
  • reduce the load to the respective git servers
  • speed up the build process

The container hosting the "pre cloned git repos" could then be updated only once per week or even once per month. It should not be rebuilt too often, as otherwise it will have to be pulled all the time, avoiding the benefits stated above and just move traffic from git to docker.

History

#1 Updated by laforge 5 months ago

The section on the docker documentation is Use an external image as a “stage”

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 48.8 MB)