https://osmocom.org/https://osmocom.org/favicon.ico?16647414092021-04-29T20:42:02ZOpen Source Mobile CommunicationsCore testing infrastructure - Bug #5139: rpi4build3 out of disk spacehttps://osmocom.org/issues/5139?journal_id=220342021-04-29T20:42:02Zlaforge
<ul></ul><p>already 28GB recovered by 'docker system prune' by now, and still runnning.</p> Core testing infrastructure - Bug #5139: rpi4build3 out of disk spacehttps://osmocom.org/issues/5139?journal_id=220362021-04-30T05:47:43Zlaforge
<ul></ul><blockquote>
<p>Total reclaimed space: 38.19GB</p>
</blockquote> Core testing infrastructure - Bug #5139: rpi4build3 out of disk spacehttps://osmocom.org/issues/5139?journal_id=220372021-04-30T09:56:43Zosmith
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>In Progress</i></li><li><strong>% Done</strong> changed from <i>0</i> to <i>90</i></li></ul><p>There is an ansible rule that sets up a cron job to run docker-cleanup.sh: <a class="external" href="https://gerrit.osmocom.org/c/osmo-ci/+/7716/">https://gerrit.osmocom.org/c/osmo-ci/+/7716/</a></p>
<p>This has been set up on the rpi nodes successfully:</p>
<pre>
# cat /var/spool/cron/crontabs/osmocom-build
# DO NOT EDIT THIS FILE - edit the master and reinstall.
# (/tmp/crontabPVPUxS installed on Mon Mar 8 09:17:27 2021)
# (Cron version -- $Id: crontab.c,v 2.13 1994/01/17 03:20:37 vixie Exp $)
#Ansible: cleanup-docker-images
0 */3 * * * test -x /home/osmocom-build/osmo-ci/scripts/docker-cleanup.sh && /home/osmocom-build/osmo-ci/scripts/docker-cleanup.sh >/dev/null
</pre>
<p>However, the cron job did not get executed because the file was owned by the wrong uid 1001 instead of 1000:<br /><pre>
# ls -l /var/spool/cron/crontabs/
total 4
-rw------- 1 1001 crontab 366 Mar 8 09:17 osmocom-build
</pre></p>
On the raspbian nodes this has happened because:
<ul>
<li>there was a "pi" user with uid=1000</li>
<li>existing ansible rules created "osmocom-build" without specifying the uid, so it became uid=1001</li>
<li>running the docker role created the crontab with uid=1001</li>
<li>as docker-playground.git assumes that its containers run as uid=1000, I had modified the ansible rules to remove the "pi" user and set "osmocom-build"'s uid to 1000
<ul>
<li><a class="external" href="https://gerrit.osmocom.org/c/osmo-ci/+/23275/1">https://gerrit.osmocom.org/c/osmo-ci/+/23275/1</a></li>
<li><a class="external" href="https://gerrit.osmocom.org/c/osmo-ci/+/23274/1">https://gerrit.osmocom.org/c/osmo-ci/+/23274/1</a></li>
</ul>
</li>
<li>after running the updated rule, ansible changed the uid of the already created user to 1000 but did not change ownership of the crontab file</li>
</ul>
<p>So it's a problem with these three provisioned nodes specifically and won't happen when we provision new nodes. I've changed ownership of the file manually on these three:</p>
<pre>
# chown osmocom-build /var/spool/cron/crontabs/osmocom-build
# systemctl restart cron
</pre>
<p>I'll check back later to verify that the cronjob actually ran and then I'll close this issue.</p> Core testing infrastructure - Bug #5139: rpi4build3 out of disk spacehttps://osmocom.org/issues/5139?journal_id=220432021-05-03T07:17:06Zosmith
<ul><li><strong>Status</strong> changed from <i>In Progress</i> to <i>Resolved</i></li><li><strong>% Done</strong> changed from <i>90</i> to <i>100</i></li></ul><p>Working as expected.</p>