Bug #3231: osmo-hlr crashes on "LU RECEIVED", address sanitizer reports stack-buffer-underflow on gsup_encode() - OsmoHLR - Open Source Mobile Communications

Actions

Copy link

Bug #3231

closed

osmo-hlr crashes on "LU RECEIVED", address sanitizer reports stack-buffer-underflow on gsup_encode()

Added by neels almost 6 years ago. Updated almost 6 years ago.

Status:

Resolved

Priority:

Urgent

Assignee:

Target version:

Start date:

05/03/2018

Due date:

% Done:

100%

Spec Reference:

Description

20180503175555423 DLINP DEBUG ipa.c:340 127.0.0.1:32814 message received
20180503175555423 DMAIN DEBUG luop.c:160 LU OP state change: NULL -> LU RECEIVED
=================================================================
==20030==ERROR: AddressSanitizer: stack-buffer-underflow on address 0x7fffffffd9c0 at pc 0x7ffff6e9b6c2 bp 0x7fffffffd900 sp 0x7fffffffd0b0
READ of size 2 at 0x7fffffffd9c0 thread T0
    #0 0x7ffff6e9b6c1  (/usr/lib/x86_64-linux-gnu/libasan.so.4+0x766c1)
    #1 0x7ffff6314419 in tlv_put ../../../../src/libosmocore/include/osmocom/gsm/tlv.h:107
    #2 0x7ffff6314419 in msgb_tlv_put ../../../../src/libosmocore/include/osmocom/gsm/tlv.h:299
    #3 0x7ffff6314419 in encode_pdp_info ../../../../src/libosmocore/src/gsm/gsup.c:419
    #4 0x7ffff6314419 in osmo_gsup_encode ../../../../src/libosmocore/src/gsm/gsup.c:535
    #5 0x555555580016 in _luop_tx_gsup ../../../src/osmo-hlr/src/luop.c:54
    #6 0x5555555809d8 in lu_op_tx_insert_subscr_data ../../../src/osmo-hlr/src/luop.c:264
    #7 0x55555558b356 in rx_upd_loc_req ../../../src/osmo-hlr/src/hlr.c:306
    #8 0x55555558b356 in read_cb ../../../src/osmo-hlr/src/hlr.c:365
    #9 0x555555586671 in osmo_gsup_server_read_cb ../../../src/osmo-hlr/src/gsup_server.c:105
    #10 0x7ffff5b35911 in ipa_server_conn_read ../../../src/libosmo-abis/src/input/ipa.c:356
    #11 0x7ffff5b35911 in ipa_server_conn_cb ../../../src/libosmo-abis/src/input/ipa.c:387
    #12 0x7ffff5e5541f in osmo_fd_disp_fds ../../../src/libosmocore/src/select.c:216
    #13 0x7ffff5e5541f in osmo_select_main ../../../src/libosmocore/src/select.c:256
    #14 0x5555555791b6 in main ../../../src/osmo-hlr/src/hlr.c:600
    #15 0x7ffff4707a86 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21a86)
    #16 0x555555579679 in _start (/usr/local/bin/osmo-hlr+0x25679)

Address 0x7fffffffd9c0 is located in stack of thread T0 at offset 16 in frame
    #0 0x7ffff63131ff in osmo_gsup_encode ../../../../src/libosmocore/src/gsm/gsup.c:481

  This frame has 1 object(s):
    [32, 64) 'bcd_buf' <== Memory access at offset 16 underflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism or swapcontext
      (longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-buffer-underflow (/usr/lib/x86_64-linux-gnu/libasan.so.4+0x766c1) 
Shadow bytes around the buggy address:
  0x10007fff7ae0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10007fff7af0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10007fff7b00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10007fff7b10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10007fff7b20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x10007fff7b30: 00 00 00 00 00 00 f1 f1[f1]f1 00 00 00 00 f3 f3
  0x10007fff7b40: f3 f3 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10007fff7b50: 00 00 00 00 f1 f1 f1 f1 00 00 00 00 00 03 f2 f2
  0x10007fff7b60: f2 f2 f2 f2 00 00 00 00 00 00 00 00 00 00 00 00
  0x10007fff7b70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10007fff7b80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==20030==ABORTING

Files

pcap-recorder_any(filters=_host 10.42.42.2_).pcap

1.7 KB

pespin, 05/04/2018 12:48 PM

Actions

Copy link

Updated by fixeria almost 6 years ago

Seems I've also faced this segfault during the external USSD interface development.
Exactly during Location Update procedure.

I thought it was somehow related to my local changes in GSUP implementation, so
didn't save any details... But now it turns out that it isn't related to my changes.

Actions

Copy link

Updated by pespin almost 6 years ago

Triggered in osmo-gsm-tester aoip_ussd:trx-sysmocell5000 / assert_extension.py:

[0;m20180504142001416 [1;33mDMAIN[0;m <0000> hlr.c:563 hlr starting
[0;m[1;31m20180504142001416 [1;33mDDB[0;m[1;31m <0001> db.c:221 using database: /home/jenkins/workspace/osmo-gsm-tester_run-prod/trial-1101/run.2018-05-04_13-43-53/aoip_ussd:trx-sysmocell5000/assert_extension.py/osmo-hlr_10.42.42.2/hlr.db
[0;m[1;31m20180504142001416 [1;32mDDB[0;m[1;31m <0001> db.c:222 Compiled against SQLite3 lib version 3.16.2
[0;m[1;31m20180504142001416 [1;32mDDB[0;m[1;31m <0001> db.c:223 Running with SQLite3 lib version 3.16.2
[0;m[1;31m20180504142001416 [1;34mDDB[0;m[1;31m <0001> db.c:231 SQLite3 compiled with 'COMPILER=gcc-6.3.0 20170516'
[0;m[1;31m20180504142001416 [1;34mDDB[0;m[1;31m <0001> db.c:231 SQLite3 compiled with 'ENABLE_COLUMN_METADATA'
[0;m[1;31m20180504142001416 [1;34mDDB[0;m[1;31m <0001> db.c:231 SQLite3 compiled with 'ENABLE_DBSTAT_VTAB'
[0;m[1;31m20180504142001416 [1;34mDDB[0;m[1;31m <0001> db.c:231 SQLite3 compiled with 'ENABLE_FTS3'
[0;m[1;31m20180504142001416 [1;34mDDB[0;m[1;31m <0001> db.c:231 SQLite3 compiled with 'ENABLE_FTS3_PARENTHESIS'
[0;m[1;31m20180504142001416 [1;34mDDB[0;m[1;31m <0001> db.c:231 SQLite3 compiled with 'ENABLE_FTS4'
[0;m[1;31m20180504142001416 [1;34mDDB[0;m[1;31m <0001> db.c:231 SQLite3 compiled with 'ENABLE_FTS5'
[0;m[1;31m20180504142001417 [1;34mDDB[0;m[1;31m <0001> db.c:231 SQLite3 compiled with 'ENABLE_JSON1'
[0;m[1;31m20180504142001417 [1;34mDDB[0;m[1;31m <0001> db.c:231 SQLite3 compiled with 'ENABLE_LOAD_EXTENSION'
[0;m[1;31m20180504142001417 [1;34mDDB[0;m[1;31m <0001> db.c:231 SQLite3 compiled with 'ENABLE_RTREE'
[0;m[1;31m20180504142001417 [1;34mDDB[0;m[1;31m <0001> db.c:231 SQLite3 compiled with 'ENABLE_UNLOCK_NOTIFY'
[0;m[1;31m20180504142001417 [1;34mDDB[0;m[1;31m <0001> db.c:231 SQLite3 compiled with 'ENABLE_UPDATE_DELETE_LIMIT'
[0;m[1;31m20180504142001417 [1;34mDDB[0;m[1;31m <0001> db.c:231 SQLite3 compiled with 'HAVE_ISNAN'
[0;m[1;31m20180504142001417 [1;34mDDB[0;m[1;31m <0001> db.c:231 SQLite3 compiled with 'LIKE_DOESNT_MATCH_BLOBS'
[0;m[1;31m20180504142001417 [1;34mDDB[0;m[1;31m <0001> db.c:231 SQLite3 compiled with 'MAX_SCHEMA_RETRY=25'
[0;m[1;31m20180504142001417 [1;34mDDB[0;m[1;31m <0001> db.c:231 SQLite3 compiled with 'OMIT_LOOKASIDE'
[0;m[1;31m20180504142001417 [1;34mDDB[0;m[1;31m <0001> db.c:231 SQLite3 compiled with 'SECURE_DELETE'
[0;m[1;31m20180504142001417 [1;34mDDB[0;m[1;31m <0001> db.c:231 SQLite3 compiled with 'SOUNDEX'
[0;m[1;31m20180504142001417 [1;34mDDB[0;m[1;31m <0001> db.c:231 SQLite3 compiled with 'SYSTEM_MALLOC'
[0;m[1;31m20180504142001417 [1;34mDDB[0;m[1;31m <0001> db.c:231 SQLite3 compiled with 'TEMP_STORE=1'
[0;m[1;31m20180504142001417 [1;34mDDB[0;m[1;31m <0001> db.c:231 SQLite3 compiled with 'THREADSAFE=1'
[0;m[1;31m20180504142001417 [1;34mDDB[0;m[1;31m <0001> db.c:248 Not setting SQL log callback: SQLite3 compiled without support for it
[0;m20180504142001442 [1;33mDLCTRL[0;m <000b> control_if.c:863 CTRL at 10.42.42.2 4259
[0;m20180504142001763 [1;33mDLINP[0;m <0006> input/ipa.c:265 accept()ed new link from 10.42.42.1 to port 4222
[0;m20180504142001763 [1;32mDLGSUP[0;m <000e> gsup_server.c:274 New GSUP client 10.42.42.1:40750 (IND=0)
[0;m20180504142002026 [1;34mDLINP[0;m <0006> input/ipa.c:385 connected read/write
[0;m20180504142002027 [1;34mDLINP[0;m <0006> input/ipa.c:340 10.42.42.1:40750 message received
[0;m20180504142002027 [1;32mDLGSUP[0;m <000e> gsup_server.c:181 CCM Callback
[0;m20180504142002027 [1;32mDLGSUP[0;m <000e> gsup_server.c:137 0: MSC-00-00-00-00-00-00
[0;m20180504142002027 [1;32mDLGSUP[0;m <000e> gsup_server.c:140 0: 4d 53 43 2d 30 30 2d 30 30 2d 30 30 2d 30 30 2d 30 30 2d 30 30 00 00 
[0;m20180504142002027 [1;32mDLGSUP[0;m <000e> gsup_server.c:137 1: MSC-00-00-00-00-00-00
[0;m20180504142002027 [1;32mDLGSUP[0;m <000e> gsup_server.c:140 1: 4d 53 43 2d 30 30 2d 30 30 2d 30 30 2d 30 30 2d 30 30 2d 30 30 00 00 
[0;m20180504142002027 [1;32mDLGSUP[0;m <000e> gsup_server.c:137 2: 00:00:00:00:00:00
[0;m20180504142002027 [1;32mDLGSUP[0;m <000e> gsup_server.c:140 2: 30 30 3a 30 30 3a 30 30 3a 30 30 3a 30 30 3a 30 30 00 00 
[0;m20180504142002027 [1;32mDLGSUP[0;m <000e> gsup_server.c:137 3: 00:00:00:00:00:00
[0;m20180504142002027 [1;32mDLGSUP[0;m <000e> gsup_server.c:140 3: 30 30 3a 30 30 3a 30 30 3a 30 30 3a 30 30 3a 30 30 00 00 
[0;m20180504142002027 [1;32mDLGSUP[0;m <000e> gsup_server.c:137 4: 00:00:00:00:00:00
[0;m20180504142002027 [1;32mDLGSUP[0;m <000e> gsup_server.c:140 4: 30 30 3a 30 30 3a 30 30 3a 30 30 3a 30 30 3a 30 30 00 00 
[0;m20180504142002027 [1;32mDLGSUP[0;m <000e> gsup_server.c:137 5: 00:00:00:00:00:00
[0;m20180504142002027 [1;32mDLGSUP[0;m <000e> gsup_server.c:140 5: 30 30 3a 30 30 3a 30 30 3a 30 30 3a 30 30 3a 30 30 00 00 
[0;m20180504142002027 [1;32mDLGSUP[0;m <000e> gsup_server.c:137 7: 00:00:00:00:00:00
[0;m20180504142002027 [1;32mDLGSUP[0;m <000e> gsup_server.c:140 7: 30 30 3a 30 30 3a 30 30 3a 30 30 3a 30 30 3a 30 30 00 00 
[0;m20180504142002027 [1;32mDLGSUP[0;m <000e> gsup_server.c:137 8: 0/0/0
[0;m20180504142002027 [1;32mDLGSUP[0;m <000e> gsup_server.c:140 8: 30 2f 30 2f 30 00 00 
[0;m20180504142002027 [1;34mDLINP[0;m <0006> input/ipa.c:385 connected read/write
[0;m20180504142002028 [1;34mDLINP[0;m <0006> input/ipa.c:340 10.42.42.1:40750 message received
[0;m20180504142002028 [1;34mDLINP[0;m <0006> input/ipa.c:385 connected read/write
[0;m20180504142002028 [1;34mDLINP[0;m <0006> input/ipa.c:340 10.42.42.1:40750 message received
[0;m20180504142018965 [1;34mDLINP[0;m <0006> input/ipa.c:385 connected read/write
[0;m20180504142018965 [1;34mDLINP[0;m <0006> input/ipa.c:340 10.42.42.1:40750 message received
[0;m20180504142018965 [1;34mDMAIN[0;m <0000> luop.c:160 LU OP state change: NULL -> [0;mLU RECEIVED
[0;m=================================================================
==734==ERROR: AddressSanitizer: stack-buffer-underflow on address 0x7ffdb0216a50 at pc 0x7f00dd2c6f7f bp 0x7ffdb0216990 sp 0x7ffdb0216140
READ of size 2 at 0x7ffdb0216a50 thread T0
    #0 0x7f00dd2c6f7e  (/usr/lib/x86_64-linux-gnu/libasan.so.3+0x5cf7e)
    #1 0x7f00dc78455a in tlv_put ../../include/osmocom/gsm/tlv.h:107
    #2 0x7f00dc78455a in msgb_tlv_put ../../include/osmocom/gsm/tlv.h:299
    #3 0x7f00dc78455a in encode_pdp_info /home/osmocom-build/jenkins/workspace/osmo-gsm-tester_build-osmo-hlr/libosmocore/src/gsm/gsup.c:419
    #4 0x7f00dc78455a in osmo_gsup_encode /home/osmocom-build/jenkins/workspace/osmo-gsm-tester_build-osmo-hlr/libosmocore/src/gsm/gsup.c:535
    #5 0x562da53c7626 in _luop_tx_gsup /home/osmocom-build/jenkins/workspace/osmo-gsm-tester_build-osmo-hlr/osmo-hlr/src/luop.c:54
    #6 0x562da53c7e8d in lu_op_tx_insert_subscr_data /home/osmocom-build/jenkins/workspace/osmo-gsm-tester_build-osmo-hlr/osmo-hlr/src/luop.c:264
    #7 0x562da53d1ce7 in rx_upd_loc_req /home/osmocom-build/jenkins/workspace/osmo-gsm-tester_build-osmo-hlr/osmo-hlr/src/hlr.c:306
    #8 0x562da53d1ce7 in read_cb /home/osmocom-build/jenkins/workspace/osmo-gsm-tester_build-osmo-hlr/osmo-hlr/src/hlr.c:365
    #9 0x562da53cd653 in osmo_gsup_server_read_cb /home/osmocom-build/jenkins/workspace/osmo-gsm-tester_build-osmo-hlr/osmo-hlr/src/gsup_server.c:105
    #10 0x7f00dbfc86a5 in ipa_server_conn_read input/ipa.c:356
    #11 0x7f00dbfc86a5 in ipa_server_conn_cb input/ipa.c:387
    #12 0x7f00dc2e05d8 in osmo_fd_disp_fds /home/osmocom-build/jenkins/workspace/osmo-gsm-tester_build-osmo-hlr/libosmocore/src/select.c:216
    #13 0x7f00dc2e05d8 in osmo_select_main /home/osmocom-build/jenkins/workspace/osmo-gsm-tester_build-osmo-hlr/libosmocore/src/select.c:256
    #14 0x562da53c0636 in main /home/osmocom-build/jenkins/workspace/osmo-gsm-tester_build-osmo-hlr/osmo-hlr/src/hlr.c:600
    #15 0x7f00dabc52e0 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x202e0)
    #16 0x562da53c0aa9 in _start (/home/jenkins/workspace/osmo-gsm-tester_run-prod/trial-1101/inst/osmo-hlr/bin/osmo-hlr+0x25aa9)

Address 0x7ffdb0216a50 is located in stack of thread T0 at offset 16 in frame
    #0 0x7f00dc78335f in osmo_gsup_encode /home/osmocom-build/jenkins/workspace/osmo-gsm-tester_build-osmo-hlr/libosmocore/src/gsm/gsup.c:481

  This frame has 1 object(s):
    [32, 64) 'bcd_buf' <== Memory access at offset 16 underflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism or swapcontext
      (longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-buffer-underflow (/usr/lib/x86_64-linux-gnu/libasan.so.3+0x5cf7e) 
Shadow bytes around the buggy address:
  0x10003603acf0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10003603ad00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10003603ad10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10003603ad20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10003603ad30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x10003603ad40: 00 00 00 00 00 00 00 00 f1 f1[f1]f1 00 00 00 00
  0x10003603ad50: f3 f3 f3 f3 00 00 00 00 00 00 00 00 00 00 00 00
  0x10003603ad60: 00 00 00 00 00 00 f1 f1 f1 f1 00 00 00 00 00 03
  0x10003603ad70: f4 f4 f2 f2 f2 f2 00 00 00 00 00 00 00 00 00 00
  0x10003603ad80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10003603ad90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Heap right redzone:      fb
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack partial redzone:   f4
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==734==ABORTING

Actions

Copy link