Project

General

Profile

Jenkins build verification jobs

We are building the various Osmocom projects when patches get posted to Gerrit for review and when patches have been merged to the master branch.
The jenkins-job-builder configuration files are in osmo-ci.git/jobs.

Workflow

  • Pushing commits to the master branch or to Gerrit triggers a one of the jobs generated by master-builds.yml/gerrit-verifications.yml (e.g. master-osmo-bsc, gerrit-osmo-bsc)
  • Jenkins triggers a new job on a build slave and waits until it completes (to see the actual console output, click on the job ID in the "Build History" on the left, then "Console Output", then follow the link in the "Triggering..." line, click the same job ID in the "Build History", click "Console Output")
  • This job most likely runs contrib/jenkins.sh of the git repository in question, inside docker. There are some variations of this, depending on the repository. Look for the repository name in the appropriate .yml config file for details.
  • The contrib/jenkins.sh typically builds the source code and runs the unit tests.

Writing jenkins.sh scripts

The following environment variables are available:

Environment variable Description
ARTIFACT_STORE Logs and other files that should be downloadable after the job ran, can be placed here.
ASCIIDOC_WARNINGS_CHECK Always "1".
HOME Points to the top dir of the git repository when running in docker. Avoid using in jenkins.sh and rely on $PWD instead, so we don't modify/clutter people's home directories if they run jenkins.sh without docker.
JOB_NAME Jenkins job name.
MAKE Always "make".
OSMOPY_DEBUG_TCP_SOCKETS Always "1".
PARALLEL_MAKE Jobs flag that should be passed to make, e.g. "-j5".
PWD Top dir of the git repository (save in a variable if you need to cd and go back).
PATH Starts with the path to scripts from osmo-ci.git. This means, that all scripts form there, like osmo-deps.sh, can be used from jenkins.sh.
PUBLISH This variable is only available in master-builds.yml. Usually "1" to enable publishing manuals, can be "0" if building with multiple axes and only one axis should publish the manuals.
WITH_MANUALS Usually "1" to enable building manuals, can be "0" if building with multiple axes and only one axis should build the manuals.

Running jenkins.sh in docker locally

In order to run VTY tests in parallel, almost all jobs don't run directly on a build slave, but in docker instead (#3726). When making changes to a job or to a jenkins.sh file, you should run the docker container locally to make sure that everything is working as it should. Clone osmo-ci.git and start building the relevant docker images, it is going to take a while:

$ git clone https://git.osmocom.org/osmo-ci
$ osmo-ci/scripts/osmo-ci-docker-rebuild.sh

Read the appropriate jenkins job configuration for the project you are changing in gerrit-verifications.yml and master-builds.yml, then create and run a shell script that emulates what jenkins does. Make sure that the scripts dir from osmo-ci.git gets properly mounted to /build_bin (CI_SCRIPTS in the example below).

#!/bin/sh -ex

PROJECT=osmo-hlr
SRC=~/code/osmo-dev/src
PARALLEL_MAKE=-j5
CI_SCRIPTS=$SRC/osmo-ci/scripts
JOB_NAME="local-test" 
DOCKER_IMG="$USER/debian-stretch-jenkins" 

cd $SRC/$PROJECT

# Based on "docker_run" and "cmd" variables from gerrit-verifications.yml
ARTIFACT_STORE="/tmp/jenkins_build_artifact_store" 
mkdir -p "$ARTIFACT_STORE" 
docker run --rm=true \
  --cap-add SYS_PTRACE \
  -e ARTIFACT_STORE=/artifact_store \
  -e ASCIIDOC_WARNINGS_CHECK="1" \
  -e HOME=/build \
  -e JOB_NAME="$JOB_NAME" \
  -e MAKE=make \
  -e OSMOPY_DEBUG_TCP_SOCKETS="1" \
  -e PARALLEL_MAKE="$PARALLEL_MAKE" \
  -e PATH="$PATH:/build_bin" \
  -e WITH_MANUALS="1" \
  -w /build -i -u build \
  -v "$PWD:/build" -v "$CI_SCRIPTS:/build_bin" \
  -v "$ARTIFACT_STORE:/artifact_store" \
"$DOCKER_IMG" /build/contrib/jenkins.sh

How the jenkins slaves are rebuilding the docker image

The Dockerfile of relevant images are stored in docker-playground.git. Rebuilding them works as follows:

  • update-osmo-ci-on-slaves (update-osmo-ci-on-slaves.yml) gets triggered on changes in osmo-ci.git's and docker-playground.git's master branch
  • runs contrib/jenkins.sh of osmo-ci.git
  • runs scripts/osmo-ci-docker-rebuild.sh of osmo-ci.git

When a new jenkins slave is set up, ansible/roles/osmocom-jenkins-slave/tasks/osmo-ci.yml in osmo-ci.git runs scripts/osmo-ci-docker-rebuild.sh as well.

Add picture from clipboard (Maximum size: 48.8 MB)