Project

General

Profile

Feature #3870

jenkins job generating tarballs of all tagged releases

Added by laforge 5 months ago. Updated about 1 month ago.

Status:
In Progress
Priority:
Normal
Assignee:
Target version:
-
Start date:
03/28/2019
Due date:
% Done:

80%

Spec Reference:

Description

It would be great to have a script (which could subsequently be executed by a jenkins job) that would generate tarball releases for all our tags.

So basically something like:

foreach repository:
   foreach tag:
      autoreconf -fi && ./configure && make dist

If some older tags don't build [anymore] we could either explicitly blacklist them, or we simply build only tags no older than N years.

The resulting tarball releases should be collected somewhere in a directory which could then be subsequently rsync'ed to a public server/directory.


Checklist

  • merge osmo-ci.git patches
  • merge docker-playground.git patches
  • enable this job's e-mail notifications (currently disabled, because patches are not merged)
  • rsync the result to another server

Related issues

Related to OsmoMGW - Bug #4084: make dist: No rule to make target 'osmocom/mgcp_client/mgcp_common.h'Resolved07/01/2019

History

#1 Updated by osmith about 2 months ago

(Missing permissions :/)

State: In Progress

#2 Updated by laforge about 2 months ago

On Fri, Jun 28, 2019 at 09:50:43AM +0000, osmith [REDMINE] wrote:

(Missing permissions :/)

sorry once more, should be better now.

#3 Updated by osmith about 2 months ago

Unfortunately I still don't have permissions to change the status of this issue etc.

EDIT: it works now

#4 Updated by osmith about 2 months ago

  • Related to Bug #4084: make dist: No rule to make target 'osmocom/mgcp_client/mgcp_common.h' added

#5 Updated by osmith about 2 months ago

  • Status changed from New to In Progress
  • % Done changed from 0 to 50

#6 Updated by osmith about 2 months ago

  • Checklist item merge osmo-ci.git patches added
  • Checklist item merge docker-playground.git patches added
  • Checklist item enable this job's e-mail notifications (currently disabled, because patches are not merged) added
  • Checklist item rsync the result to another server added
  • % Done changed from 50 to 70
osmocom-release-tarballs.sh is implemented:
  • This iterates over all Osmocom repos and tags, and builds release tarballs.
  • I've added a blacklist for versions where the release tarball can't be generated.
  • For osmo-mgw I've added a workaround, because the release tarball can't be built for the last 5 versions or so, including the latest one (#4084)
  • The script creates a _release_tarballs dir in the jenkins workspace, and puts all generated tarballs there. It skips the ones that are already generated.
  • Tags are queried from the git server without cloning the repos. The git repos only get cloned when we need to generate a new tarball. When the script is done, the git repos are deleted again to save space (all checked out git repos are about 500 MB in size).

I have also added the jenkins job. The script needs to run in docker, otherwise ./configure will complain that various libraries are missing. I've added "debian-stretch-build-dist" to docker-playground.git, which installs the various -dev packages on top of "debian-stretch-build". The jenkins job builds that docker image with a new function in osmo-ci.git's script/common.sh, and then runs osmocom-release-tarballs.sh inside the container.

The output of the script looks like this:

...
osmo-hlr
  osmo-hlr-0.0.1.tar.bz2 (ignored)
  osmo-hlr-0.1.0.tar.bz2 (exists)
  osmo-hlr-0.2.0.tar.bz2 (exists)
  osmo-hlr-0.2.1.tar.bz2 (exists)
  osmo-hlr-1.0.0.tar.bz2 (creating)
...

Patches:
https://gerrit.osmocom.org/q/topic:tagged-release-tarballs

Jenkins job:
https://jenkins.osmocom.org/jenkins/job/Osmocom-release-tarballs/

Generated tarballs:
https://jenkins.osmocom.org/jenkins/job/Osmocom-release-tarballs/ws/_release_tarballs/

#7 Updated by osmith about 2 months ago

  • Checklist item merge osmo-ci.git patches set to Done
  • Checklist item merge docker-playground.git patches set to Done
  • Checklist item enable this job's e-mail notifications (currently disabled, because patches are not merged) set to Done

#8 Updated by osmith about 2 months ago

Almost done, only missing the rsync now.

laforge, I assume that we want to send the files to somewhere at: https://ftp.osmocom.org/

Could you set up a new account for the releases, like the existing docs account?

Then we can upload the releases similar to how the docs are uploaded.

#9 Updated by osmith about 2 months ago

  • % Done changed from 70 to 80

#10 Updated by laforge about 2 months ago

On Wed, Jul 03, 2019 at 11:54:33AM +0000, wrote:

Could you set up a new account for the releases, like the existing docs
account?

I do not have the required credentials with me on the old, non-work laptop that I
carry with me on the motorbike tour during holidays. I can look into
this once I'm back, please ping me then (July 16).

#11 Updated by osmith about 1 month ago

I do not have the required credentials with me on the old, non-work laptop that I
carry with me on the motorbike tour during holidays. I can look into
this once I'm back, please ping me then (July 16).

laforge: ping :)

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 48.8 MB)