https://osmocom.org/https://osmocom.org/favicon.ico?16647414092019-06-28T09:50:43ZOpen Source Mobile CommunicationsMiscellaneous Projects - Feature #3870: jenkins job generating tarballs of all tagged releaseshttps://osmocom.org/issues/3870?journal_id=150292019-06-28T09:50:43Zosmith
<ul></ul><p>(Missing permissions :/)</p>
<p>State: In Progress</p> Miscellaneous Projects - Feature #3870: jenkins job generating tarballs of all tagged releaseshttps://osmocom.org/issues/3870?journal_id=150342019-06-28T21:10:11Zlaforge
<ul></ul><p>On Fri, Jun 28, 2019 at 09:50:43AM +0000, osmith [REDMINE] wrote:</p>
<blockquote>
<p>(Missing permissions :/)</p>
</blockquote>
<p>sorry once more, should be better now.</p> Miscellaneous Projects - Feature #3870: jenkins job generating tarballs of all tagged releaseshttps://osmocom.org/issues/3870?journal_id=150372019-07-01T07:43:11Zosmith
<ul></ul><p>Unfortunately I still don't have permissions to change the status of this issue etc.</p>
<p>EDIT: it works now</p> Miscellaneous Projects - Feature #3870: jenkins job generating tarballs of all tagged releaseshttps://osmocom.org/issues/3870?journal_id=150402019-07-01T11:18:32Zosmith
<ul><li><strong>Related to</strong> <i><a class="issue tracker-1 status-3 priority-2 priority-default closed" href="/issues/4084">Bug #4084</a>: make dist: No rule to make target 'osmocom/mgcp_client/mgcp_common.h'</i> added</li></ul> Miscellaneous Projects - Feature #3870: jenkins job generating tarballs of all tagged releaseshttps://osmocom.org/issues/3870?journal_id=150432019-07-02T07:01:45Zosmith
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>In Progress</i></li><li><strong>% Done</strong> changed from <i>0</i> to <i>50</i></li></ul> Miscellaneous Projects - Feature #3870: jenkins job generating tarballs of all tagged releaseshttps://osmocom.org/issues/3870?journal_id=150482019-07-02T11:57:18Zosmith
<ul><li><b>Checklist item</b> <input type='checkbox' class='checklist-checkbox' disabled> <i>merge osmo-ci.git patches</i> added</li><li><b>Checklist item</b> <input type='checkbox' class='checklist-checkbox' disabled> <i>merge docker-playground.git patches</i> added</li><li><b>Checklist item</b> <input type='checkbox' class='checklist-checkbox' disabled> <i>enable this job's e-mail notifications (currently disabled, because patches are not merged)</i> added</li><li><b>Checklist item</b> <input type='checkbox' class='checklist-checkbox' disabled> <i>rsync the result to another server</i> added</li><li><strong>% Done</strong> changed from <i>50</i> to <i>70</i></li></ul>osmocom-release-tarballs.sh is implemented:
<ul>
<li>This iterates over all Osmocom repos and tags, and builds release tarballs.</li>
<li>I've added a blacklist for versions where the release tarball can't be generated.</li>
<li>For osmo-mgw I've added a workaround, because the release tarball can't be built for the last 5 versions or so, including the latest one (<a class="issue tracker-1 status-3 priority-2 priority-default closed" title="Bug: make dist: No rule to make target 'osmocom/mgcp_client/mgcp_common.h' (Resolved)" href="https://osmocom.org/issues/4084">#4084</a>)</li>
<li>The script creates a _release_tarballs dir in the jenkins workspace, and puts all generated tarballs there. It skips the ones that are already generated.</li>
<li>Tags are queried from the git server without cloning the repos. The git repos only get cloned when we need to generate a new tarball. When the script is done, the git repos are deleted again to save space (all checked out git repos are about 500 MB in size).</li>
</ul>
<p>I have also added the jenkins job. The script needs to run in docker, otherwise ./configure will complain that various libraries are missing. I've added "debian-stretch-build-dist" to docker-playground.git, which installs the various -dev packages on top of "debian-stretch-build". The jenkins job builds that docker image with a new function in osmo-ci.git's script/common.sh, and then runs osmocom-release-tarballs.sh inside the container.</p>
<p>The output of the script looks like this:<br /><pre>
...
osmo-hlr
osmo-hlr-0.0.1.tar.bz2 (ignored)
osmo-hlr-0.1.0.tar.bz2 (exists)
osmo-hlr-0.2.0.tar.bz2 (exists)
osmo-hlr-0.2.1.tar.bz2 (exists)
osmo-hlr-1.0.0.tar.bz2 (creating)
...
</pre></p>
<p>Patches:<br /><a class="external" href="https://gerrit.osmocom.org/q/topic:tagged-release-tarballs">https://gerrit.osmocom.org/q/topic:tagged-release-tarballs</a></p>
<p>Jenkins job:<br /><a class="external" href="https://jenkins.osmocom.org/jenkins/job/Osmocom-release-tarballs/">https://jenkins.osmocom.org/jenkins/job/Osmocom-release-tarballs/</a></p>
<p>Generated tarballs:<br /><a class="external" href="https://jenkins.osmocom.org/jenkins/job/Osmocom-release-tarballs/ws/_release_tarballs/">https://jenkins.osmocom.org/jenkins/job/Osmocom-release-tarballs/ws/_release_tarballs/</a></p> Miscellaneous Projects - Feature #3870: jenkins job generating tarballs of all tagged releaseshttps://osmocom.org/issues/3870?journal_id=150542019-07-03T11:46:30Zosmith
<ul><li><b>Checklist item</b> <input type='checkbox' class='checklist-checkbox' checked disabled> <i>merge osmo-ci.git patches</i> set to Done</li><li><b>Checklist item</b> <input type='checkbox' class='checklist-checkbox' checked disabled> <i>merge docker-playground.git patches</i> set to Done</li><li><b>Checklist item</b> <input type='checkbox' class='checklist-checkbox' checked disabled> <i>enable this job's e-mail notifications (currently disabled, because patches are not merged)</i> set to Done</li></ul> Miscellaneous Projects - Feature #3870: jenkins job generating tarballs of all tagged releaseshttps://osmocom.org/issues/3870?journal_id=150552019-07-03T11:54:32Zosmith
<ul></ul><p>Almost done, only missing the rsync now.</p>
<p><a class="user active" href="https://osmocom.org/users/7">laforge</a>, I assume that we want to send the files to somewhere at: <a class="external" href="https://ftp.osmocom.org/">https://ftp.osmocom.org/</a></p>
<p>Could you set up a new account for the releases, like the existing docs account?</p>
<p>Then we can upload the releases <a href="https://git.osmocom.org/osmo-gsm-manuals/tree/build/Makefile.common.inc?id=001c5787fe0e7385a1a16211a45b2cacf0e415b8#n33" class="external">similar to how the docs are uploaded</a>.</p> Miscellaneous Projects - Feature #3870: jenkins job generating tarballs of all tagged releaseshttps://osmocom.org/issues/3870?journal_id=150562019-07-03T11:54:48Zosmith
<ul><li><strong>% Done</strong> changed from <i>70</i> to <i>80</i></li></ul> Miscellaneous Projects - Feature #3870: jenkins job generating tarballs of all tagged releaseshttps://osmocom.org/issues/3870?journal_id=150872019-07-06T12:42:20Zlaforge
<ul></ul><p>On Wed, Jul 03, 2019 at 11:54:33AM +0000, <a class="email" href="mailto:redmine@lists.osmocom.org">redmine@lists.osmocom.org</a> wrote:</p>
<blockquote>
<p>Could you set up a new account for the releases, like the existing docs<br />account?</p>
</blockquote>
<p>I do not have the required credentials with me on the old, non-work laptop that I<br />carry with me on the motorbike tour during holidays. I can look into<br />this once I'm back, please ping me then (July 16).</p> Miscellaneous Projects - Feature #3870: jenkins job generating tarballs of all tagged releaseshttps://osmocom.org/issues/3870?journal_id=151542019-07-16T07:49:39Zosmith
<ul></ul><blockquote>
<p>I do not have the required credentials with me on the old, non-work laptop that I<br />carry with me on the motorbike tour during holidays. I can look into<br />this once I'm back, please ping me then (July 16).</p>
</blockquote>
<p><a class="user active" href="https://osmocom.org/users/7">laforge</a>: ping :)</p> Miscellaneous Projects - Feature #3870: jenkins job generating tarballs of all tagged releaseshttps://osmocom.org/issues/3870?journal_id=165662019-11-29T10:29:09Zosmith
<ul></ul><p><a class="user active" href="https://osmocom.org/users/7">laforge</a>: ping</p> Miscellaneous Projects - Feature #3870: jenkins job generating tarballs of all tagged releaseshttps://osmocom.org/issues/3870?journal_id=169892020-01-05T12:04:19Zlaforge
<ul></ul><p>Sorry for dragging this too long.</p>
<p>I tried to make it work, but the fact that it's built inside docuer doesn't make things easier.</p>
<p>In <a class="external" href="https://jenkins.osmocom.org/jenkins/job/Osmocom_API/configure">https://jenkins.osmocom.org/jenkins/job/Osmocom_API/configure</a> we use the jenkins credential store with SSH agent forwarding to make the upload work</p>
<p>I would prefer a similar solution. There's now a <a class="external" href="http://ftp.osmocom.org/releases/">http://ftp.osmocom.org/releases/</a> directory that you can rsync to (port 48) <code>releases@rita.osmocom.org:web-files/</code></p>
<p>I will add a related 'releases' ssh key to the jenkins credential store.</p> Miscellaneous Projects - Feature #3870: jenkins job generating tarballs of all tagged releaseshttps://osmocom.org/issues/3870?journal_id=172402020-01-17T14:56:47Zosmith
<ul><li><b>Checklist item</b> <input type='checkbox' class='checklist-checkbox' checked disabled> <i>rsync the result to another server</i> set to Done</li><li><strong>% Done</strong> changed from <i>80</i> to <i>90</i></li></ul><blockquote>
<p>the fact that it's built inside docuer doesn't make things easier.</p>
</blockquote>
<p>Let's run rsync outside of docker then :)</p>
<p><a class="external" href="https://gerrit.osmocom.org/c/osmo-ci/+/16917">https://gerrit.osmocom.org/c/osmo-ci/+/16917</a></p> Miscellaneous Projects - Feature #3870: jenkins job generating tarballs of all tagged releaseshttps://osmocom.org/issues/3870?journal_id=172412020-01-17T15:09:31Zosmith
<ul><li><strong>Status</strong> changed from <i>In Progress</i> to <i>Resolved</i></li><li><strong>% Done</strong> changed from <i>90</i> to <i>100</i></li></ul>