Project

General

Profile

Actions

Bug #4005

open

TTCN3 crashes when receiving an RTP MGCP_DUMMY_LOAD from osmo-mgw during CRCX

Added by pespin almost 5 years ago. Updated almost 3 years ago.

Status:
Stalled
Priority:
Normal
Assignee:
Category:
libosmo-mgcp
Target version:
-
Start date:
05/16/2019
Due date:
% Done:

10%

Spec Reference:

Description

Running test TC_two_crcx_and_rtp on my local machine (without docker) as of osmo-ttcn3-hacks master (a2d59c6e6efa1fa979aa5e1cd61a9eb21a775cb6):

Executing all items of [EXECUTE] section.
MC2> MTC@pespin-thinkpad: Starting external command `../ttcn3-tcpdump-start.sh MGCP_Test.TC_two_crcx_and_rtp'.
sh: ../ttcn3-tcpdump-start.sh: No such file or directory
MTC@pespin-thinkpad: Warning: External command `../ttcn3-tcpdump-start.sh MGCP_Test.TC_two_crcx_and_rtp' returned unsuccessful exit status (127).
MTC@pespin-thinkpad: Test case TC_two_crcx_and_rtp started.
MTC@pespin-thinkpad: Warning: dec_MgcpMessage(): Data remained at the end of the stream after successful decoding: '00'O
 ./mgw/MGCP_Test: Segmentation fault occurred
/usr/ttcn3/lib/libttcn3-parallel-dynamic.so(_Z14signal_handleri+0xa8)[0x7f41841c8238]
/usr/lib/libc.so.6(+0x378b0)[0x7f41833878b0]
/usr/lib/libc.so.6(+0x160f7e)[0x7f41834b0f7e]
/usr/ttcn3/lib/libttcn3-parallel-dynamic.so(_ZN11TTCN_Buffer5put_sEmPKh+0x3b)[0x7f418414da6b]
mgw/RTP_EncDec.so(_ZN10RTP__Types11f__RTP__decERK11OCTETSTRING+0xf3)[0x7f418448b605]
mgw/RTP_CodecPort.so(_ZN14RTP__CodecPort23IPL4__to__RTP__RecvFromERKN14IPL4asp__Types13ASP__RecvFromERNS_13RTP__RecvFromE+0x192)[0x7f4184f865ff]
mgw/RTP_CodecPort.so(_ZN14RTP__CodecPort14RTP__CODEC__PT16incoming_messageERKN14IPL4asp__Types13ASP__RecvFromEi+0x123)[0x7f4184f85abb]
mgw/RTP_CodecPort.so(_ZN14RTP__CodecPort14RTP__CODEC__PT16incoming_messageERKN14IPL4asp__Types13ASP__RecvFromE+0x28)[0x7f4184f85fec]
mgw/IPL4asp_PT.so(_ZN17IPL4asp__PortType20IPL4asp__PT_PROVIDER24Handle_Fd_Event_ReadableEi+0x1159)[0x7f41844c0dfb]
/usr/ttcn3/lib/libttcn3-parallel-dynamic.so(_ZN19Fd_And_Timeout_User13call_handlersEi+0x5eb)[0x7f418418c31b]
/usr/ttcn3/lib/libttcn3-parallel-dynamic.so(_ZN13TTCN_Snapshot8take_newEb+0x392)[0x7f418418c8f2]
mgw/RTP_Emulation.so(_ZN14RTP__Emulation7f__mainEv+0x2bc2)[0x7f4184f4060a]
mgw/RTP_Emulation.so(_ZN14RTP__Emulation18start_ptc_functionEPKcR8Text_Buf+0x14c)[0x7f4184f415f9]
/usr/ttcn3/lib/libttcn3-parallel-dynamic.so(_ZN11Module_List14start_functionEPKcS1_R8Text_Buf+0x2c)[0x7f4184172fec]
/usr/ttcn3/lib/libttcn3-parallel-dynamic.so(_ZN12TTCN_Runtime14start_functionEPKcS1_R8Text_Buf+0x26)[0x7f4184185086]
/usr/ttcn3/lib/libttcn3-parallel-dynamic.so(_ZN18TTCN_Communication13process_startEv+0x44)[0x7f4184149344]
/usr/ttcn3/lib/libttcn3-parallel-dynamic.so(_ZN18TTCN_Communication23process_all_messages_tcEv+0x31f)[0x7f4184149b9f]
/usr/ttcn3/lib/libttcn3-parallel-dynamic.so(_ZN12TTCN_Runtime8ptc_mainEv+0xf1)[0x7f4184188cd1]
/usr/ttcn3/lib/libttcn3-parallel-dynamic.so(main+0x3c8)[0x7f4183e8be18]
/usr/lib/libc.so.6(__libc_start_main+0xf3)[0x7f4183373ce3]
./mgw/MGCP_Test(+0x204e)[0x5603395fa04e]
MTC@pespin-thinkpad: Warning: dec_MgcpMessage(): Data remained at the end of the stream after successful decoding: '00'O
MC@pespin-thinkpad: Unexpected end of PTC connection (3) from localhost [127.0.0.1].
MTC@pespin-thinkpad: Warning: The last outgoing messages on port RTPEM[0] may be lost.

So apparently the issue is in my local osmo-mgw.cfg I was not using the option no rtp keep-alive, and as a result osmo-mgw sends MGCP_DUMMY_LOAD (0x23) upon receiving a CRCX (see #2650).
See send_dummy called from handle_create_con in mgcp_protocol.c from osmo-mgw.git.

This kind of packet is used to make a hole in the NAT. We should check:
  • Whether that payload (0x23) is valid in RTP
  • In case it's invalid, change it to something valid
  • In case it's valid, improve TTCN3 RTP code to being able to handle that kind of packet without crashing.

I aleady dropped its use in Osmux and using correct Osmux Dummy format for that.


Related issues

Related to OsmoMGW - Bug #2650: how can osmo-mgw send a dummy RTP packet before CRCX ACK?New11/17/2017

Actions
Actions #1

Updated by pespin almost 5 years ago

  • Related to Bug #2650: how can osmo-mgw send a dummy RTP packet before CRCX ACK? added
Actions #2

Updated by laforge about 3 years ago

  • Assignee changed from 4368 to dexter

I don't think the 0x23 was ever a valid payload and it was simply a hack from the old osmo-bsc_nat days. As a result, we should move away from it. The question is to what? It must be on the same UDP port, of course, but it should not be a real RTP header, as that would mean it affects SSRC-change-detection-logic, timestamp/counters etc. in the receiver, which we must avoid.

pespin, do you have any specific ideas?

Actions #3

Updated by dexter almost 3 years ago

  • Status changed from New to In Progress
Actions #4

Updated by dexter almost 3 years ago

I am able to reproduce the crash. I have the feeling that the very short dummy packet produces some kind of overflow because the TTCN3 code might expect more bytes. I am not sure yet, I also couldn't make out any exact point in the TTCN3 code where the crash happens, its somewhere in RTP_Emulation.ttcn3 in the while loop of f_main.

Actions #5

Updated by dexter almost 3 years ago

  • % Done changed from 0 to 10

In order to be able to debug this problem I have refactored/fixed the dummy packet definition and handling in osmo-mgw a bit:
See also: https://gerrit.osmocom.org/c/osmo-mgw/+/24315

Actions #6

Updated by dexter almost 3 years ago

I have merged patch #24315 now.

Actions #7

Updated by dexter almost 3 years ago

  • Status changed from In Progress to Stalled
Actions

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 48.8 MB)