Project

General

Profile

Actions
Copy link

Bug #4567

closed

rpm is failing on jenkins slave admin2-deb9build in docker

Added by osmith over 3 years ago. Updated almost 3 years ago.

Status:
Resolved
Priority:
Low
Assignee:
osmith
Target version:
-
Start date:
05/29/2020
Due date:
% Done:

100%

Spec Reference:

Description

When running Osmocom-repo-install-centos on admin2-deb9build, rpm throws strange errors (see below).

It works fine on build2-deb9build-ansible and locally.

Rebuilding the docker container did not help. We could try to restart the LXC container.

As a workaround, the job is confiugred to only run on build2-deb9build-ansible.

https://jenkins.osmocom.org/jenkins/job/Osmocom-repo-install-centos/feed=next,label=osmocom-master-debian9/9/console

error: rpmdb: BDB0060 PANIC: fatal region error detected; run recovery
error: db5 error(-30973) from db->close: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
error: rpmdb: BDB0060 PANIC: fatal region error detected; run recovery
error: db5 error(-30973) from db->close: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
error: rpmdb: BDB0060 PANIC: fatal region error detected; run recovery
error: db5 error(-30973) from db->close: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
error: rpmdb: BDB0060 PANIC: fatal region error detected; run recovery
error: db5 error(-30973) from db->close: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
error: rpmdb: BDB0060 PANIC: fatal region error detected; run recovery
error: db5 error(-30973) from db->close: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
error: rpmdb: BDB0060 PANIC: fatal region error detected; run recovery
error: db5 error(-30973) from db->close: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
error: rpmdb: BDB0060 PANIC: fatal region error detected; run recovery
error: db5 error(-30973) from db->close: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
error: rpmdb: BDB0060 PANIC: fatal region error detected; run recovery
error: db5 error(-30973) from db->close: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
error: rpmdb: BDB0060 PANIC: fatal region error detected; run recovery
error: db5 error(-30973) from db->close: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
error: rpmdb: BDB0060 PANIC: fatal region error detected; run recovery
error: db5 error(-30973) from db->close: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
error: rpmdb: BDB0060 PANIC: fatal region error detected; run recovery
error: db5 error(-30973) from db->close: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
error: rpmdb: BDB0060 PANIC: fatal region error detected; run recovery
error: db5 error(-30973) from db->close: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
error: rpmdb: BDB0060 PANIC: fatal region error detected; run recovery
error: db5 error(-30973) from db->close: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
error: rpmdb: BDB0060 PANIC: fatal region error detected; run recovery
error: db5 error(-30973) from db->close: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
error: rpmdb: BDB0060 PANIC: fatal region error detected; run recovery
error: db5 error(-30973) from db->close: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
error: rpmdb: BDB0060 PANIC: fatal region error detected; run recovery
error: db5 error(-30973) from db->close: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
error: rpmdb: BDB0060 PANIC: fatal region error detected; run recovery
error: db5 error(-30973) from db->close: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
error: rpmdb: BDB0060 PANIC: fatal region error detected; run recovery
error: db5 error(-30973) from db->close: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
error: rpmdb: BDB0060 PANIC: fatal region error detected; run recovery
error: db5 error(-30973) from db->close: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
error: rpmdb: BDB1581 File handles still open at environment close
error: rpmdb: BDB1582 Open file handle: /var/lib/rpm/__db.001
error: rpmdb: BDB1582 Open file handle: /var/lib/rpm/__db.001
error: rpmdb: BDB1582 Open file handle: /var/lib/rpm/__db.002
error: rpmdb: BDB1582 Open file handle: /var/lib/rpm/__db.003
error: rpmdb: BDB1582 Open file handle: /var/lib/rpm/Packages
error: rpmdb: BDB1582 Open file handle: /var/lib/rpm/Name
error: rpmdb: BDB1582 Open file handle: /var/lib/rpm/Basenames
error: rpmdb: BDB1582 Open file handle: /var/lib/rpm/Group
error: rpmdb: BDB1582 Open file handle: /var/lib/rpm/Requirename
error: rpmdb: BDB1582 Open file handle: /var/lib/rpm/Providename
error: rpmdb: BDB1582 Open file handle: /var/lib/rpm/Conflictname
error: rpmdb: BDB1582 Open file handle: /var/lib/rpm/Obsoletename
error: rpmdb: BDB1582 Open file handle: /var/lib/rpm/Triggername
error: rpmdb: BDB1582 Open file handle: /var/lib/rpm/Dirnames
error: rpmdb: BDB1582 Open file handle: /var/lib/rpm/Installtid
error: rpmdb: BDB1582 Open file handle: /var/lib/rpm/Sigmd5
error: rpmdb: BDB1582 Open file handle: /var/lib/rpm/Sha1header
error: rpmdb: BDB1582 Open file handle: /var/lib/rpm/Filetriggername
error: rpmdb: BDB1582 Open file handle: /var/lib/rpm/Transfiletriggername
error: rpmdb: BDB1582 Open file handle: /var/lib/rpm/Recommendname
error: rpmdb: BDB1582 Open file handle: /var/lib/rpm/Suggestname
error: rpmdb: BDB1582 Open file handle: /var/lib/rpm/Supplementname
error: rpmdb: BDB1582 Open file handle: /var/lib/rpm/Enhancename
error: rpmdb: BDB0060 PANIC: fatal region error detected; run recovery
error: db5 error(-30973) from dbenv->close: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
Error: Transaction failed

Related issues

Related to Cellular Network Infrastructure - Feature #4563: install test for centos packages (like the debian install test)Resolvedosmith05/25/2020

Actions
Actions
Copy link
 #1

Updated by osmith over 3 years ago

  • Related to Feature #4563: install test for centos packages (like the debian install test) added
Actions
Copy link
 #2

Updated by osmith over 3 years ago

  • Status changed from New to In Progress
  • Assignee set to osmith

I'm also running into this problem with the ttcn3 tests for centos. On admin2-deb9build, osmo-bts-master-centos8 cannot be built, because installing rpm packages causes the same error.
I've just tried to force building centos8-build with docker build --no-cache but that did not help it.

Actions
Copy link
 #3

Updated by osmith over 3 years ago

Also tried without success, right before the failing "dnf install":
  • "rpm --rebuilddb"
  • "rm -rf /var/cache/dnf"
Actions
Copy link
 #4

Updated by osmith over 3 years ago

In TTCN3-centos-sip-test, osmo-sip-connector and the testsuite both core dumped on admin2-deb9build, and did not core dump on build2-deb9build-ansible.

https://jenkins.osmocom.org/jenkins/view/TTCN3-centos/job/TTCN3-centos-sip-test/1/artifact/logs/

Actions
Copy link
 #5

Updated by osmith over 3 years ago

  • Status changed from In Progress to Stalled
  • Assignee deleted (osmith)
Actions
Copy link
 #6

Updated by laforge over 3 years ago

  • Assignee set to osmith
  • Priority changed from Normal to Low
Actions
Copy link
 #7

Updated by laforge almost 3 years ago

is this still an issue?

Actions
Copy link
 #8

Updated by osmith almost 3 years ago

  • % Done changed from 0 to 90

Not an issue anymore: the rpm errors are gone when running on admin2-deb9build: https://jenkins.osmocom.org/jenkins/job/Osmocom-repo-install-centos8/feed=nightly,label=admin2-deb9build/80/console

This patch will make it run on all build slaves again: https://gerrit.osmocom.org/c/osmo-ci/+/22788

EDIT: fixup: https://gerrit.osmocom.org/c/osmo-ci/+/22806

Actions
Copy link
 #9

Updated by osmith almost 3 years ago

  • Status changed from Stalled to Resolved
  • % Done changed from 90 to 100
Actions
Copy link

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 48.8 MB)