Project

General

Profile

Actions

Bug #3182

closed

OSMO-BSC: Intermittent Segmentation fault (core dumped)

Added by ron.menez@entropysolution.com almost 6 years ago. Updated over 5 years ago.

Status:
Resolved
Priority:
Urgent
Assignee:
Category:
-
Target version:
-
Start date:
04/18/2018
Due date:
% Done:

80%

Spec Reference:

Description

Segmentation Fault (core dumped) is experienced intermittently in running OSMO-BSC.

Hardwares used are the following:

  • Ettus B210
  • UPBOARD

UPBOARD information:

  1. lscpu
    Architecture: x86_64
    CPU op-mode(s): 32-bit, 64-bit
    Byte Order: Little Endian
    CPU: 4
    On-line CPU list: 0-3
    Thread(s) per core: 1
    Core(s) per socket: 4
    Socket(s): 1
    NUMA node(s): 1
    Vendor ID: GenuineIntel
    CPU family: 6
    Model: 92
    Model name: Intel(R) Pentium(R) CPU N4200 @ 1.10GHz
    Stepping: 9
    CPU MHz: 800.000
    CPU max MHz: 1101.0000
    CPU min MHz: 800.0000
    BogoMIPS: 2188.79
    Virtualization: VT-x
    L1d cache: 24K
    L1i cache: 32K
    L2 cache: 1024K
    NUMA node0 CPU: 0-3
    Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 ds_cpl vmx est tm2 ssse3 sdbg cx16 xtpr pdcm sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave rdrand lahf_lm 3dnowprefetch intel_pt tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust smep erms mpx rdseed smap clflushopt sha_ni xsaveopt xsavec xgetbv1 dtherm ida arat pln pts
  1. lshw (kindly see attached file)

OS used:

  1. lsb_release -a
    No LSB modules are available.
    Distributor ID:
    Ubuntu
    Description:
    Ubuntu 16.04.4 LTS
    Release: 16.04
    Codename: xenial

The following osmocom elements and libraries were installed last April 14 through git:

  • libosmocore
  • libosmo-abis
  • libosmo-crypt-a53
  • libosmo-dsp
  • libosmo-netif
  • libosmo-sccp
  • libsmpp34
  • libgtpnl
  • osmo-mgw
  • openbsc
  • osmo-bsc
  • osmo-bts
  • osmo-hlr
  • osmo-msc
  • osmo-trx

Also run a backtrace after segfault occurs. Logs, pcap network trace and core dump file are also provided. Kindly see attached.

(gdb) bt
#0 0x000000000041aa51 in rsl_rx_conn_fail (msg=msg@entry=0x8a4090) at abis_rsl.c:1380
#1 0x000000000042172b in abis_rsl_rx_dchan (msg=0x8a4090) at abis_rsl.c:1646
#2 abis_rsl_rcvmsg (msg=0x8a4090) at abis_rsl.c:2853
#3 0x00007ffff710f7e8 in handle_ts1_read (bfd=0x895760) at input/ipaccess.c:282
#4 ipaccess_fd_cb (bfd=0x895760, what=1) at input/ipaccess.c:397
#5 0x00007ffff7329352 in osmo_fd_disp_fds (_eset=0x7fffffffe3b0, _wset=0x7fffffffe330, _rset=0x7fffffffe2b0)
at select.c:216
#6 osmo_select_main (polling=polling@entry=0) at select.c:256
#7 0x00000000004075ef in main (argc=<optimized out>, argv=<optimized out>) at osmo_bsc_main.c:532

(gdb) bt full
#0 0x000000000041aa51 in rsl_rx_conn_fail (msg=msg@entry=0x8a4090) at abis_rsl.c:1380
dh = 0x8a411e
lchan = 0x7ffff7fb3290
tp = {lv = {{len = 0, val = 0x0} <repeats 26 times>, {len = 1, val = 0x8a4124 "\001"}, {len = 0,
val = 0x0} <repeats 229 times>}}
cause = 1 '\001'
#1 0x000000000042172b in abis_rsl_rx_dchan (msg=0x8a4090) at abis_rsl.c:1646
rslh = 0x8a411e
rc = 0
sign_link = 0x8941e0
#2 abis_rsl_rcvmsg (msg=0x8a4090) at abis_rsl.c:2853
sign_link = 0x8941e0
rslh = 0x8a411e
rc = 0
#3 0x00007ffff710f7e8 in handle_ts1_read (bfd=0x895760) at input/ipaccess.c:282
line = 0x894ba0
ts_nr = <optimized out>
link = <optimized out>
e1i_ts = <optimized out>
hh = 0x8a411b
msg = 0x8a4090
ret = <optimized out>
rc = <optimized out>
#4 ipaccess_fd_cb (bfd=0x895760, what=1) at input/ipaccess.c:397
rc = 0
#5 0x00007ffff7329352 in osmo_fd_disp_fds (eset=0x7fffffffe3b0, _wset=0x7fffffffe330, _rset=0x7fffffffe2b0)
at select.c:216
flags = 1
ufd = 0x895760
tmp = 0x895d38
work = 1
#6 osmo_select_main (polling=polling@entry=0) at select.c:256
readset = {
_fds_bits = {0 <repeats 16 times>}}
writeset = {__fds_bits = {0 <repeats 16 times>}}
exceptset = {__fds_bits = {0 <repeats 16 times>}}
rc = <optimized out>
no_time = {tv_sec = 0, tv_usec = 0}
#7 0x00000000004075ef in main (argc=<optimized out>, argv=<optimized out>) at osmo_bsc_main.c:532
msc = 0x6bdab8
data = <optimized out>
rc = <optimized out>


Files

segfault_osmo-bsc.pcap segfault_osmo-bsc.pcap 50.7 KB ron.menez@entropysolution.com, 04/18/2018 04:40 AM
segfault_osmo-bsc.log segfault_osmo-bsc.log 128 KB ron.menez@entropysolution.com, 04/18/2018 04:40 AM
osmo-bts_standalone_demo.cfg osmo-bts_standalone_demo.cfg 2.26 KB ron.menez@entropysolution.com, 04/18/2018 04:40 AM
osmo-bsc_standalone_demo.cfg osmo-bsc_standalone_demo.cfg 4.57 KB ron.menez@entropysolution.com, 04/18/2018 04:40 AM
osmo-mgw_standalone_demo.cfg osmo-mgw_standalone_demo.cfg 1.16 KB ron.menez@entropysolution.com, 04/18/2018 04:40 AM
osmo-stp_standalone_demo.cfg osmo-stp_standalone_demo.cfg 424 Bytes ron.menez@entropysolution.com, 04/18/2018 04:40 AM
osmo-trx_standalone_demo.cfg osmo-trx_standalone_demo.cfg 903 Bytes ron.menez@entropysolution.com, 04/18/2018 04:40 AM
osmo-hlr_standalone_demo.cfg osmo-hlr_standalone_demo.cfg 305 Bytes ron.menez@entropysolution.com, 04/18/2018 04:40 AM
osmo-msc_standalone_demo.cfg osmo-msc_standalone_demo.cfg 1.78 KB ron.menez@entropysolution.com, 04/18/2018 04:40 AM
core core 3.3 MB ron.menez@entropysolution.com, 04/18/2018 04:40 AM
gdb_segfault_04202018.log gdb_segfault_04202018.log 88.4 KB ron.menez@entropysolution.com, 04/20/2018 02:59 AM

Checklist

  • do a thorough audit of all lchan->conn dereferences
  • fix the actual bug reported
  • implement TTCN3 tests to trigger the problem
Actions

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 48.8 MB)