Project

General

Profile

Actions

Feature #4040

closed

Use "COPY --from" in Dockerfiles to avoid cloning from scratch again and again

Added by laforge almost 5 years ago. Updated about 3 years ago.

Status:
Rejected
Priority:
Low
Assignee:
Target version:
-
Start date:
06/04/2019
Due date:
% Done:

0%

Spec Reference:

Description

Some of our Dockerfiles clone a lot of git repositories (e.g. those for the ttcn3 tests), and due to the sequential nature of docker cache invalidation, a change in osmo-ttcn3-hacks.git not only means we will re-clone that git repository from scratch, but we also will clone all of the 'deps' repositories from scratch.

If we had a docker container hosting pre-created clones of those repositories, we could use the "COPY --from" syntax in our Dockerfile, copying those initial clones of the git repositories over. We'd then only have to pull whatever changes (if any) over the network.

See https://docs.docker.com/develop/develop-images/multistage-build/ for a description of this feature towards the bottom.

This would
  • significantly reduce the network traffic on the buildhost during re-generation of images
  • reduce the load to the respective git servers
  • speed up the build process

The container hosting the "pre cloned git repos" could then be updated only once per week or even once per month. It should not be rebuilt too often, as otherwise it will have to be pulled all the time, avoiding the benefits stated above and just move traffic from git to docker.

Actions #1

Updated by laforge almost 5 years ago

The section on the docker documentation is Use an external image as a “stage”

Actions #2

Updated by laforge about 3 years ago

  • Assignee changed from 4368 to osmith
Actions #3

Updated by osmith about 3 years ago

Looks like we can close this with the changes from #5017, which solves the problem of cloning deps over and over.

The container hosting the "pre cloned git repos" could then be updated only once per week or even once per month. It should not be rebuilt too often, as otherwise it will have to be pulled all the time, avoiding the benefits stated above and just move traffic from git to docker.

Now it gets only updated when:
  • deps/Makefile in osmo-ttcn3-hacks.git changes
  • debian-stretch-titan/* changes in docker-playground.git

laforge: can we mark it as rejected (duplicate)?

Actions #4

Updated by laforge about 3 years ago

  • Status changed from New to Rejected
Actions

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 48.8 MB)