• Table of contents
• Osmocom Titan TTCN3 Testsuites
  • Source Code / Patches
  • Test Suites
  • Running a testsuite
    • Running test suite in the dockerized environment
      • Building the docker containers
      • Example: running the OsmoBSC testsuite
      • Running only a sub-set of the test cases
    • Running it native on your machine
      • Running only a sub-set of the test cases
  • Further information

Osmocom Titan TTCN3 Testsuites¶

In 2017, Osmocom started to create extensive tests suites for the Cellular Network Infrastructure elements. Those suites are written in the TTCN-3 programming language and use the compiler + executor of the Eclipse_TITAN project.

Source Code / Patches¶

The source code of our test suites lives in the osmo-ttcn3-hacks repository

• cgit web interface: http://git.osmocom.org/osmo-ttcn3-hacks/
• clone: git clone git://git.osmocom.org/osmo-ttcn3-hacks
• patch review: https://gerrit.osmocom.org/#/q/project:osmo-ttcn3-hacks

We're following the Gerrit process for patch review.

Test Suites¶

Implementation under Test	Testsuite	Jenkins
bsc-nat	http://git.osmocom.org/osmo-ttcn3-hacks/tree/bsc-nat	-
OsmoBSC	http://git.osmocom.org/osmo-ttcn3-hacks/tree/bsc	https://jenkins.osmocom.org/jenkins/view/TTCN3/job/ttcn3-bsc-test/
OsmoBTS	http://git.osmocom.org/osmo-ttcn3-hacks/tree/bts	https://jenkins.osmocom.org/jenkins/view/TTCN3/job/ttcn3-bts-test/
OsmoGGSN	http://git.osmocom.org/osmo-ttcn3-hacks/tree/ggsn_tests	https://jenkins.osmocom.org/jenkins/view/TTCN3/job/ttcn3-ggsn-test/
OsmoHLR	http://git.osmocom.org/osmo-ttcn3-hacks/tree/hlr	https://jenkins.osmocom.org/jenkins/view/TTCN3/job/ttcn3-hlr-test/
OsmoMGW	http://git.osmocom.org/osmo-ttcn3-hacks/tree/mgw	https://jenkins.osmocom.org/jenkins/view/TTCN3/job/ttcn3-mgw-test/
OsmoMSC	http://git.osmocom.org/osmo-ttcn3-hacks/tree/msc	https://jenkins.osmocom.org/jenkins/view/TTCN3/job/ttcn3-msc-test/
OsmoPCU	http://git.osmocom.org/osmo-ttcn3-hacks/tree/pcu	-
OsmoSGSN	http://git.osmocom.org/osmo-ttcn3-hacks/tree/sgsn	https://jenkins.osmocom.org/jenkins/view/TTCN3/job/ttcn3-sgsn-test/
osmo-sip-connector	http://git.osmocom.org/osmo-ttcn3-hacks/tree/sip	https://jenkins.osmocom.org/jenkins/view/TTCN3/job/ttcn3-sip-test/

Running a testsuite¶

You have multiple options on how to execute a test suite.

Running test suite in the dockerized environment¶

This is the much easier way to run the test suite. No manual configuration of Osmocom programs or the test suite are required. Both the Osmocom program (Implementation Under Test) as well as the test suite are packaged as docker containers, which are then executed next to each other using a docker network with the "right" addresses between IUT and testsuite.

Also, using this setup you will be guaranteed to run in the exact same environment as the automatically-executed tests on jenkins.osmocom.org, i.e. your results should be exactly identical, without any differences introduced by your runtime environment, whether specific library versions or intentional or inadvertent configuration differences.

Building the docker containers¶

First install docker on your system and add yourself to the docker group. Then clone and build the containers from the docker-playground git repository:

$ git clone git://git.osmocom.org/docker-playground
$ cd docker-playground
$ make

Tagging images¶

If the build fails with a message like the following:

repository docker.io/laforge/debian-jessie-build not found: does not exist or no pull access

Then you need to tag your own image with the laforge/debian-jessie-build name. In docker 1.13.1 this works with the following command (replace user with the username that ran the make command above):

$ docker tag user/debian-jessie-build laforge/debian-jessie-build

Example: running the OsmoBSC testsuite¶

Let's assume you'd want to run the testsuite for OsmoBSC. You would need to

1. look at http://git.osmocom.org/docker-playground/tree/ttcn3-bsc-test/jenkins.sh to understand which containers are required.
   • at the time of writing, this is osmo-stp-master, osmo-bsc-master, osmo-bts-master and ttcn3-bsc-test
2. build those respective containers by issuing make in the above-mentioned directories, e.g.
   

   $ (cd osmo-stp-master && make)
   $ (cd osmo-bsc-master && make)
   $ (cd osmo-bts-master && make)
   $ (cd ttcn3-bsc-test && make)
   

3. run the jenkins.sh script of the ttcn3-bsc-test suite:
   

   $ cd ttcn3-bts-test && ./jenkins.sh
   

The jenkins.sh script will make sure to

• create the needed docker network[s]
• run all the required containers (IUT, helpers, testsuite)
• shut them down after test suite execution
• collect the log files after execution, /tmp/logs will contain them

Running only a sub-set of the test cases¶

FIXME

Running it native on your machine¶

This is the most complicated bit to set-up, as you will have to run the respective Osmocom Program (Implementation Under Test) in the right configuration with all the IP addresses, port numbers, config file, etc. exactly like it's expected by the test suite.

You will need a deeper understanding about how the test suite works, and what its requirement are.

Running only a sub-set of the test cases¶

FIXME

Further information¶

• April 2018 talk by Harald on State of he Osmocom TTCN-3 Test Suites