Project

General

Profile

Actions

Bug #6411

open

ttcn3: execute testsuites with a more realistic link performance

Added by fixeria 23 days ago. Updated 9 days ago.

Status:
Stalled
Priority:
Normal
Assignee:
-
Target version:
-
Start date:
03/20/2024
Due date:
% Done:

0%

Spec Reference:
Tags:

Description

This idea by laforge originates from #6375: we should try running at least some of our BTS / BSC tests over an Abis link with simulated latency and non-zero packet loss. I did some experiments using tc-netem, running the whole ttcn3-bts-test with a simulated delay: #6375#note-14. Let's continue discussing this here.

Actions #1

Updated by fixeria 23 days ago

This patch:

https://gerrit.osmocom.org/c/docker-playground/+/36372 ttcn3-*-test: allow running tests with artificial link delay

makes simulating the link delay as easy as passing an additional environment variable:

$ export DOCKER_ARGS="--cap-add=NET_ADMIN -e 'OSMO_TTCN3_NETEM_DELAY=30'" 
$ cd ttcn3-bsc-test
$ ./jenkins.sh

Somehow, the --cap-add=NET_ADMIN (needed for the tc-netem) breaks tcpdump/dumpcap:

dumpcap: The file to which the capture would be saved ("/data/BTS_Tests.TC_est_dchan.pcap") could not be opened: Permission denied.

This needs to be investigated and fixed/worked around before we merge the patch.

Actions #2

Updated by fixeria 23 days ago

I gave a few testsuites a spin with OSMO_TTCN3_NETEM_DELAY=30.

ttcn3-bsc-test

Most of the testcases are passing, except the following ones:

  • pass->FAIL BSC_Tests.TC_paging_500req
  • pass->FAIL BSC_Tests.TC_paging_450req_no_paging_load_ind
  • pass->FAIL BSC_Tests.TC_ho_out_fail_no_msc_response
  • pass->FAIL BSC_Tests.TC_ho_out_fail_no_result_after_ho_cmd
  • pass->FAIL BSC_Tests.TC_ho_in_fail_no_detect
  • pass->FAIL BSC_Tests.TC_ho_neighbor_config_5
  • pass->FAIL BSC_Tests.TC_ho_neighbor_config_6
  • pass->FAIL BSC_Tests.TC_ratectr_all_available_allocated_dyn
  • pass->FAIL BSC_Tests.TC_lost_sdcch_during_assignment

It took ~1.5h to execute the testsuite, so the time overhead is not significant.

ttcn3-bts-test

Most of the testcases are passing, except the following ones:

  • pass->FAIL BTS_Tests.TC_chan_act_stress
  • pass->FAIL BTS_Tests.TC_rsl_ms_pwr_ctrl
  • pass->FAIL BTS_Tests.TC_paging_tmsi_200percent
  • pass->FAIL BTS_Tests.TC_pcu_time_ind
  • pass->FAIL BTS_Tests.TC_pcu_rts_req
  • pass->FAIL BTS_Tests.TC_data_rtp_tchh48
  • pass->FAIL BTS_Tests.TC_data_rtp_tchh24
  • pass->FAIL BTS_Tests.TC_acch_overpower_always_on_sacch
  • pass->FAIL BTS_Tests_SMSCB.TC_cbc_sdcch4_load_overload
  • pass->FAIL BTS_Tests_SMSCB.TC_etws_p1ro
  • pass->FAIL BTS_Tests_LAPDm.TC_iframe_timer_recovery
  • pass->FAIL BTS_Tests_LAPDm.TC_rec_invalid_frame
  • pass->FAIL BTS_Tests_LAPDm.TC_segm_concat_sacch
  • pass->FAIL BTS_Tests_LAPDm.TC_t200_n200
  • pass->FAIL BTS_Tests_LAPDm.TC_normal_reestablishment

The generic profile took ~1h to execute, so the time overhead is not significant.

Actions #3

Updated by fixeria 9 days ago

  • Status changed from New to Stalled
Actions

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 48.8 MB)