Titan TTCN3 Testsuites » History » Revision 23

« Previous | Revision 23/49 (diff) | Next »
osmith, 10/01/2018 10:45 AM

Osmocom Titan TTCN3 Testsuites

In 2017, Osmocom started to create extensive tests suites for the Cellular Network Infrastructure elements. Those suites are written in the TTCN-3 programming language and use the compiler and executor of the Eclipse_TITAN project. There are special cases, but in general, the idea is to test one component (e.g. osmo-mgw) independently.

Source Code / Patches

The source code of our test suites lives in the osmo-ttcn3-hacks repository

We're following the Gerrit process for patch review.

Test Suites

Implementation under Test Testsuite Jenkins
bsc-nat -
OsmoPCU -

Running a testsuite

You have multiple options on how to execute a test suite.

Running test suite in the dockerized environment

This is the much easier way to run the test suite. No manual configuration of Osmocom programs or the test suite are required. Both the Osmocom program (Implementation Under Test) as well as the test suite are packaged as docker containers, which are then executed next to each other using a docker network with the "right" addresses between IUT and testsuite.

Also, using this setup you will be guaranteed to run in the exact same environment as the automatically-executed tests on, i.e. your results should be exactly identical, without any differences introduced by your runtime environment, whether specific library versions or intentional or inadvertent configuration differences.

Installing docker

Make sure you have *docker-ce* installed (not to be confused with the possibly outdated version that may be shipped in your distribution's repositories). Check that the version string contains -ce as follows:

$ docker -v
Docker version 18.06.1-ce, build e68fc7a

Next, add yourself to the docker group, log off and on again, and verify that you can use docker with your user:

$ docker info

Cloning docker-playground

Osmocom related containers are stored in the docker-playground git repository.

$ git clone git://
$ cd docker-playground

In theory you could run make now to build all containers. However, not all of them are maintained, so this will most likely fail. Instead take a look at the Jenkins script of a specific project to figure out its dependencies and then build these containers.

Running a testsuite

Let's assume you'd want to run the testsuite for OsmoBSC. You would need to
  1. look at its to understand which containers are required.
  2. build those respective containers - at time of writing, this would be:
    $ make osmo-stp-master osmo-bsc-master osmo-bts-master ttcn3-bsc-test
  3. run the script of the ttcn3-bsc-test suite:
    $ cd ttcn3-bsc-test && ./
The script will make sure to
  • create the needed docker network[s]
  • run all the required containers (IUT, helpers, testsuite)
  • shut them down after test suite execution
  • collect the log files after execution, /tmp/logs will contain them

Running only a sub-set of the test cases

See running the testsuite outside of Docker for reference (below).

Running it native on your machine

This is the most complicated bit to set-up, as you will have to run the respective Osmocom Program (Implementation Under Test) in the right configuration with all the IP addresses, port numbers, config file, etc. exactly like it's expected by the test suite.

You will need a deeper understanding about how the test suite works, and what its requirement are. Oftentimes looking at how it's done in the Docker scripts is helpful.


$ apt install eclipse-titan
$ git clone git://
$ cd osmo-ttc3-hacks/deps
$ make

Compile a testsuite

Let's compile the mgw testsuite for example:

$ cd mgw
$ ./
$ ./regen_makefile

The next command will transform the TTCN3 test data into C++ code. Do not use -j here, that won't work (and it is pretty fast anyway).

$ make compile

Finally compile the testsuite:

$ make -j5

Run a testsuite

Start the component that is about to get tested with the config that the testsuite expects. Again, with mgw as example. If you don't have a suitable config file, use the one from Jenkins and replace all IPs with

$ osmo-mgw -c osmo-mgw.cfg

Then run the testsuite:

$ cd osmo-ttcn3-hacks/mgw
$ ../ ./MGCP_test MGCP_Test.cfg

Afterwards you can merge and format the logs as follows:

$ ttcn3_logmerge MGCP_test*.log > ./merged.log
$ ttcn3_logformat ./merged.log > ./result.log

Running only a sub-set of the test cases

Running whole testsuites may take a long time. If you only want to run one specific test, or a subset of tests, open up the _*Test.cfg file in the folder of the testsuite and edit the [EXECUTE] section. For example, in MGCP_Test.cfg from the mgw folder it looks like the following:


The only line that is not commented out is MGCP_Test.control. This *.control line will cause all tests to be executed. So comment this line out, and then activate one of the lines of which you would like to execute the tests (see the example below). Then execute the testsuite as usually, and it will skip all commented out tests.


Reading the logs

  • xfail means, that a test is known to fail, FAIL are unexpected test failures
  • search for setverdict( to find the results of single tests. -> fail should be the location where a test is failing.

Further information

Updated by osmith over 3 years ago · 23 revisions

Add picture from clipboard (Maximum size: 48.8 MB)