Project

General

Profile

Feature #4563

install test for centos packages (like the debian install test)

Added by osmith 6 months ago. Updated 6 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Target version:
-
Start date:
05/25/2020
Due date:
% Done:

100%

Spec Reference:

Description

Create jenkins jobs, that install all centos packages from the official repositories for nightly and latest.


Related issues

Related to Cellular Network Infrastructure - Bug #3176: osmocom debian packages are not install / upgrade testedResolved04/16/2018

Related to Cellular Network Infrastructure - Bug #3369: no automatic testing of Debian/Ubuntu packagesResolved06/29/2018

Related to Cellular Network Infrastructure - Feature #4550: Integrate RPM spec files into Osmocom repositories and build them on OBSResolved05/13/2020

Related to Cellular Network Infrastructure - Bug #4567: rpm is failing on jenkins slave admin2-deb9build in dockerStalled05/29/2020

History

#1 Updated by osmith 6 months ago

  • Related to Bug #3176: osmocom debian packages are not install / upgrade tested added

#2 Updated by osmith 6 months ago

  • Related to Bug #3369: no automatic testing of Debian/Ubuntu packages added

#3 Updated by osmith 6 months ago

  • Related to Feature #4550: Integrate RPM spec files into Osmocom repositories and build them on OBS added

#4 Updated by osmith 6 months ago

Concept:
  • move scripts that install all packages and try to run the services from docker-playground.git to osmo-ci.git. In docker-playground.git, we only have scripts for one image (nothing shared), so it fits better in osmo-ci.git
  • create a new image centos-repo-install-test in docker-playground.git
  • in osmo-ci.git:
    • trigger building the images from docker-playground.git (there's already code for that)
    • mount the shared script dir inside the containers
    • run the scripts in the containers, with a distro parameter to do distro specific installation etc.
  • adjust jenkins job builder yml to make use of the new structure and run it for centos, too

#5 Updated by osmith 6 months ago

  • Status changed from New to In Progress

#6 Updated by osmith 6 months ago

  • Tracker changed from Bug to Feature

#7 Updated by osmith 6 months ago

  • % Done changed from 0 to 70

Patches submitted:
https://gerrit.osmocom.org/q/topic:centos-install-test

In jenkins:

I just noticed that the workspace is not cleaned up properly yet:

java.nio.file.FileSystemException: /home/osmocom-build/jenkins/workspace/Osmocom-repo-install-debian/feed/nightly/label/osmocom-master-debian9/_repo_install_test_data/blacklist.txt: Operation not permitted

Furthermore, "Osmocom-repo-install-centos/feed=next" found a real error in the repository that needs to be fixed. This should not be problematic however, as it only occurs when installing osmo-trx-ipc and its debug symbols package (which AFAIK nobody is doing/trying at the moment):

Error: Transaction check error:
file /usr/lib/debug/usr/bin/ipc-driver-test-1.2.0.60.bb89-2.1.x86_64.debug conflicts between attempted installs of osmo-trx-ipc-debuginfo-1.2.0.60.bb89-2.1.x86_64 and osmo-trx-ipc-1.2.0.60.bb89-2.1.x86_64

#8 Updated by osmith 6 months ago

osmith wrote:

I just noticed that the workspace is not cleaned up properly yet:

java.nio.file.FileSystemException: /home/osmocom-build/jenkins/workspace/Osmocom-repo-install-debian/feed/nightly/label/osmocom-master-debian9/_repo_install_test_data/blacklist.txt: Operation not permitted

Resolved with: https://gerrit.osmocom.org/c/osmo-ci/+/18518

#9 Updated by osmith 6 months ago

  • % Done changed from 70 to 90

osmith wrote:

Furthermore, "Osmocom-repo-install-centos/feed=next" found a real error in the repository that needs to be fixed. This should not be problematic however, as it only occurs when installing osmo-trx-ipc and its debug symbols package (which AFAIK nobody is doing/trying at the moment):

Error: Transaction check error:
file /usr/lib/debug/usr/bin/ipc-driver-test-1.2.0.60.bb89-2.1.x86_64.debug conflicts between attempted installs of osmo-trx-ipc-debuginfo-1.2.0.60.bb89-2.1.x86_64 and osmo-trx-ipc-1.2.0.60.bb89-2.1.x86_64

also fixed (in osmo-trx.git osmith/ipc branch, which is what the script for next is currently building).

#10 Updated by osmith 6 months ago

rpm is failing on jenkins slave admin2-deb9build for some reason (is the machine broken?). But I can't look further into that now, I've configured the job to only run on build2-deb9build-ansible for now, this works fine.

error: rpmdb: BDB0060 PANIC: fatal region error detected; run recovery
error: db5 error(-30973) from db->close: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery

https://jenkins.osmocom.org/jenkins/job/Osmocom-repo-install-centos/feed=next,label=osmocom-master-debian9/11/console

#11 Updated by laforge 6 months ago

On Wed, May 27, 2020 at 01:02:57PM +0000, osmith [REDMINE] wrote:

rpm is failing on jenkins slave admin2-deb9build for some reason (is the machine broken?).

the machine is fine, it runs tons of other services. There is plenty of disk space, and no errors
in the kernel log or other suspicious behavior.

I suggest:
  • rebuilding the docker container
  • restarting the deb9build lxc (can use "reboot" like a physical machine).

#12 Updated by osmith 6 months ago

  • Status changed from In Progress to Resolved
  • % Done changed from 90 to 100

laforge wrote:

On Wed, May 27, 2020 at 01:02:57PM +0000, osmith [REDMINE] wrote:
I suggest:
  • rebuilding the docker container

I've tried this yesterday, same result.

  • restarting the deb9build lxc (can use "reboot" like a physical machine).

Debugging this now doesn't seem a good use of time, given that I also need to get the centos8 ttcn-3 implementation done. With the workaround of hardcoding a node, it is working now, and I'm certain that it is an issue with that specific jenkins node (admin2-deb9build).

I'll set this issue to resolved, as the centos packages install and systemd services test script is now rolled out.

#13 Updated by laforge 6 months ago

On Thu, May 28, 2020 at 10:13:13AM +0000, osmith [REDMINE] wrote:

Debugging this now doesn't seem a good use of time, [...]
I'll set this issue to resolved, [...]

Then please at least create a separate ticket about the bug, as otherwise we
lose all knowledge of it.

#14 Updated by osmith 6 months ago

laforge wrote:

On Thu, May 28, 2020 at 10:13:13AM +0000, osmith [REDMINE] wrote:

Debugging this now doesn't seem a good use of time, [...]
I'll set this issue to resolved, [...]

Then please at least create a separate ticket about the bug, as otherwise we
lose all knowledge of it.

https://osmocom.org/issues/4567

#15 Updated by osmith 6 months ago

  • Related to Bug #4567: rpm is failing on jenkins slave admin2-deb9build in docker added

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 48.8 MB)