Project

General

Profile

Bug #3851

External tests error

Added by msuraev 7 months ago. Updated 7 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
-
Target version:
-
Start date:
03/20/2019
Due date:
% Done:

100%

Resolution:
Spec Reference:

Description

Recently following error appeared in gerrit:

ERROR: testBSCreload (__main__.TestVTYNAT)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./vty_test_runner.py", line 803, in testBSCreload
    b0 = nat_bsc_sock_test(0, "lol", verbose=True, proc=self.proc)
  File "./vty_test_runner.py", line 1112, in nat_bsc_sock_test
    ipa_handle_small(bsc, verbose)
  File "./vty_test_runner.py", line 1052, in ipa_handle_small
    raise Exception("expected to receive 4 bytes, but got %d (%r)" % (len(s)/2, s))
Exception: expected to receive 4 bytes, but got 0 ('')

======================================================================
ERROR: testUssdSideChannelProvider (__main__.TestVTYNAT)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./vty_test_runner.py", line 935, in testUssdSideChannelProvider
    data = ussdSocket.recv(4)
error: [Errno 104] Connection reset by peer

This can be rather reliably reproduced manually by running:
./configure --disable-iu --disable-mgcp-transcoding --enable-vty-tests --enable-external-tests
make check

Initially suspected b3e34435b3bde227805c8dcac39c28e57144e500 in libosmo-netif is unrelated: error reappers even with earlier libosmo-netif version. It could be that this is somehow related to I435aaa33605bd48635715a2c81aa2d231c1abf51 in libosmocore or maybe test expectations are wrong.

disable-other-tests.patch disable-other-tests.patch 1.2 KB osmith, 03/25/2019 01:24 PM
out_nok out_nok 3.87 KB vty_test_runner.py not passing the test (like it fails in jenkins right now) osmith, 03/25/2019 01:28 PM
out_ok out_ok 3.37 KB vty_test_runner.py passing the test (libosmocore patch reverted) osmith, 03/25/2019 01:28 PM
3851-test-nok.pcapng 3851-test-nok.pcapng 56.6 KB osmith, 03/25/2019 01:30 PM
3851-test-ok.pcapng 3851-test-ok.pcapng 64.9 KB osmith, 03/25/2019 01:30 PM

History

#1 Updated by osmith 7 months ago

Reverting https://gerrit.osmocom.org/#/c/libosmocore/+/10216/ makes it go away.

I have tested 3x with the patch reverted, and it passes the tests every time. And 3x with master and it fails every time. (Just to make sure, because I had assumed earlier that it was a race condition.)

#2 Updated by osmith 7 months ago

  • Status changed from New to In Progress
  • Assignee set to osmith

I'm checking if this is just a wrong test expectation in openbsc, or if we need to revert the patch to stay compatible with openbsc's releases.

#3 Updated by osmith 7 months ago

I've been looking into this for over two hours, but I can't tell why exactly the tests are failing. I have submitted a revert of the patch:

https://gerrit.osmocom.org/#/c/libosmocore/+/13396

Some notes:
  • most irrelevant tests can be disabled with the appended patch file
  • then run the failing script directly with: $ python2 ./vty_test_runner.py -w/home/user/code/osmo-dev/src/openbsc/openbsc -v
  • pcap files of both OK and NOK runs, as well as the stdout/stderr output, are attached
  • I could not find anything odd in the IPA messages in wireshark (they look the same, except for timestamps, sequence numbers etc.)

Maybe somebody else can take a look at this?

#4 Updated by laforge 7 months ago

  • Assignee set to pespin

#5 Updated by pespin 7 months ago

  • Status changed from Stalled to Feedback
  • % Done changed from 0 to 90

Fixed by
libosmocore:
https://gerrit.osmocom.org/#/c/libosmocore/+/13431/ ipa: Document ipa_ccm_idtag_parse_off and fix ipa_ccm_idtag_parse
https://gerrit.osmocom.org/#/c/libosmocore/+/13432/ ipa: Fix ipa_ccm_id_resp_parse on big endian systems

openbsc:
https://gerrit.osmocom.org/#/c/openbsc/+/13433 Replace broken ipa_ccm_idtag APIs with new ipa_ccm_id ones

#6 Updated by laforge 7 months ago

all 3 patches merged, FYI. is this still open?

#7 Updated by pespin 7 months ago

  • Status changed from Feedback to Resolved
  • % Done changed from 90 to 100

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 48.8 MB)