Project

General

Profile

Bug #4119

osmo-remsim-client-st2 leaking memory

Added by roh 5 months ago. Updated 14 days ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
remsim-client
Target version:
-
Start date:
07/18/2019
Due date:
% Done:

100%


Description

i just realized the client is leaking too:

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root     14138  0.1  1.7 191976 68708 ?        Ssl  Jul10  22:04 /usr/bin/osmo-remsim-client-st2 -s 10.9.99.10 -V 1d50 -P 4004 -C 1 -I 0 -H 2-1.1 -c 0 -n 0

after restart:

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root     28923  0.2  0.1  62816  4776 ?        Ssl  16:04   0:00 /usr/bin/osmo-remsim-client-st2 -s 10.9.99.10 -V 1d50 -P 4004 -C 1 -I 0 -H 2-1.1 -c 0 -n 0

talloc is also not interresting:

Jul 18 16:03:12 remsimtest osmo-remsim-client-st2[14138]: full talloc report on 'global' (total   2885 bytes in  22 blocks)
Jul 18 16:03:12 remsimtest osmo-remsim-client-st2[14138]:     struct bankd_client            contains   1996 bytes in  15 blocks (ref 0) 0x55d6e0b9cbe0
Jul 18 16:03:12 remsimtest osmo-remsim-client-st2[14138]:         struct osmo_fsm_inst           contains    240 bytes in   3 blocks (ref 0) 0x55d6e0b9d260
Jul 18 16:03:12 remsimtest osmo-remsim-client-st2[14138]:             BANKD_CONN(bankd)[0x55d6e0b9d260] contains     34 bytes in   1 blocks (ref 0) 0x55d6e0b9d400
Jul 18 16:03:12 remsimtest osmo-remsim-client-st2[14138]:             bankd                          contains      6 bytes in   1 blocks (ref 0) 0x55d6e0b9d390
Jul 18 16:03:12 remsimtest osmo-remsim-client-st2[14138]:         struct osmo_fsm_inst           contains    756 bytes in  10 blocks (ref 0) 0x55d6e0b92c20
Jul 18 16:03:12 remsimtest osmo-remsim-client-st2[14138]:             struct osmo_fd                 contains     48 bytes in   1 blocks (ref 0) 0x55d6e0b926b0
Jul 18 16:03:12 remsimtest osmo-remsim-client-st2[14138]:             struct ipa_client_conn         contains    464 bytes in   6 blocks (ref 0) 0x55d6e0b92590
Jul 18 16:03:12 remsimtest osmo-remsim-client-st2[14138]:                 struct osmo_fsm_inst           contains    277 bytes in   4 blocks (ref 0) 0x55d6e0b9d010
Jul 18 16:03:12 remsimtest osmo-remsim-client-st2[14138]:                     struct ipa_fsm_priv            contains     32 bytes in   1 blocks (ref 0) 0x55d6e0b9d1d0
Jul 18 16:03:12 remsimtest osmo-remsim-client-st2[14138]:                     IPA-KEEPALIVE(server)[0x55d6e0b9d010] contains     38 bytes in   1 blocks (ref 0) 0x55d6e0b9d140
Jul 18 16:03:12 remsimtest osmo-remsim-client-st2[14138]:                     server                         contains      7 bytes in   1 blocks (ref 0) 0x55d6e0b92dc0
Jul 18 16:03:12 remsimtest osmo-remsim-client-st2[14138]:                 10.9.99.10                     contains     11 bytes in   1 blocks (ref 0) 0x55d6e0b92750
Jul 18 16:03:12 remsimtest osmo-remsim-client-st2[14138]:             RSPRO_CLIENT(server)[0x55d6e0b92c20] contains     37 bytes in   1 blocks (ref 0) 0x55d6e0b92500
Jul 18 16:03:12 remsimtest osmo-remsim-client-st2[14138]:             server                         contains      7 bytes in   1 blocks (ref 0) 0x55d6e0b92d50
Jul 18 16:03:12 remsimtest osmo-remsim-client-st2[14138]:         ClientSlot_t                   contains     40 bytes in   1 blocks (ref 0) 0x55d6e0b80d70
Jul 18 16:03:12 remsimtest osmo-remsim-client-st2[14138]:     logging                        contains    889 bytes in   5 blocks (ref 0) 0x55d6e0b80200
Jul 18 16:03:12 remsimtest osmo-remsim-client-st2[14138]:         struct log_target              contains    208 bytes in   2 blocks (ref 0) 0x55d6e0b805f0
Jul 18 16:03:12 remsimtest osmo-remsim-client-st2[14138]:             struct log_category            contains     40 bytes in   1 blocks (ref 0) 0x55d6e0b80700
Jul 18 16:03:12 remsimtest osmo-remsim-client-st2[14138]:         struct log_info                contains    680 bytes in   2 blocks (ref 0) 0x55d6e0b80270
Jul 18 16:03:12 remsimtest osmo-remsim-client-st2[14138]:             struct log_info_cat            contains    640 bytes in   1 blocks (ref 0) 0x55d6e0b80300
Jul 18 16:03:12 remsimtest osmo-remsim-client-st2[14138]:     asn1                           contains      0 bytes in   1 blocks (ref 0) 0x55d6e0b80190

after restart:

Jul 18 16:04:02 remsimtest systemd[1]: Stopping Osmocom Remote SIM Client 0...
Jul 18 16:04:02 remsimtest systemd[1]: Stopped Osmocom Remote SIM Client 0.
Jul 18 16:04:02 remsimtest systemd[1]: Started Osmocom Remote SIM Client 0.
Jul 18 16:04:02 remsimtest osmo-remsim-client-st2[28923]: <0000> fsm.c:423 RSPRO_CLIENT(server)[0x558090c62b20]{INIT}: Allocated
Jul 18 16:04:02 remsimtest osmo-remsim-client-st2[28923]: <0000> fsm.c:423 BANKD_CONN(bankd)[0x558090c61800]{INIT}: Allocated
Jul 18 16:04:05 remsimtest osmo-remsim-client-st2[28923]: <0003> input/ipa.c:128 10.9.99.10:9998 connection done
Jul 18 16:04:05 remsimtest osmo-remsim-client-st2[28923]: <0000> rspro_client_fsm.c:105 RSPRO_CLIENT(server)[0x558090c62b20]{INIT}: RSPRO link to 10.9.99.10:9998 UP
Jul 18 16:04:05 remsimtest osmo-remsim-client-st2[28923]: <0000> rspro_client_fsm.c:107 RSPRO_CLIENT(server)[0x558090c62b20]{INIT}: Received Event SRVC_E_TCP_UP
Jul 18 16:04:05 remsimtest osmo-remsim-client-st2[28923]: <0000> rspro_client_fsm.c:210 RSPRO_CLIENT(server)[0x558090c62b20]{INIT}: state_chg to ESTABLISHED
Jul 18 16:04:05 remsimtest osmo-remsim-client-st2[28923]: <0000> rspro_client_fsm.c:78 RSPRO_CLIENT(server)[0x558090c62b20]{ESTABLISHED}: Tx RSPRO connectClientReq
Jul 18 16:04:06 remsimtest osmo-remsim-client-st2[28923]: <0000> rspro_client_fsm.c:141 RSPRO_CLIENT(server)[0x558090c62b20]{ESTABLISHED}: Received RSPRO [L1]> 00 40 ee 07 [L2]> 30 3d 80 01 02 81 02 09 26 a2 34 a3 32 30 2d 0a 01 01 16 0a 66 69 78 6d 65 2d 6e 61 6d 65 80 0d 72 65 6d 73 69 6d 2d 73 65 72 76 65 72 81 0d 30 2e 31 2e 30 2e 32 31 2d 63 38 62 63 0a 01 00
Jul 18 16:04:06 remsimtest osmo-remsim-client-st2[28923]: <0000> simtrace2-remsim_client.c:647 RSPRO_CLIENT(server)[0x558090c62b20]{ESTABLISHED}: Received Event SRVC_E_CLIENT_CONN_RES
Jul 18 16:04:06 remsimtest osmo-remsim-client-st2[28923]: <0000> rspro_client_fsm.c:254 RSPRO_CLIENT(server)[0x558090c62b20]{ESTABLISHED}: state_chg to CONNECTED
Jul 18 16:04:19 remsimtest osmo-remsim-client-st2[28923]: full talloc report on 'global' (total   2885 bytes in  22 blocks)
Jul 18 16:04:19 remsimtest osmo-remsim-client-st2[28923]:     struct bankd_client            contains   1996 bytes in  15 blocks (ref 0) 0x558090c61110
Jul 18 16:04:19 remsimtest osmo-remsim-client-st2[28923]:         struct osmo_fsm_inst           contains    240 bytes in   3 blocks (ref 0) 0x558090c61800
Jul 18 16:04:19 remsimtest osmo-remsim-client-st2[28923]:             BANKD_CONN(bankd)[0x558090c61800] contains     34 bytes in   1 blocks (ref 0) 0x558090c619a0
Jul 18 16:04:19 remsimtest osmo-remsim-client-st2[28923]:             bankd                          contains      6 bytes in   1 blocks (ref 0) 0x558090c61930
Jul 18 16:04:19 remsimtest osmo-remsim-client-st2[28923]:         struct osmo_fsm_inst           contains    756 bytes in  10 blocks (ref 0) 0x558090c62b20
Jul 18 16:04:19 remsimtest osmo-remsim-client-st2[28923]:             struct osmo_fd                 contains     48 bytes in   1 blocks (ref 0) 0x558090c625a0
Jul 18 16:04:19 remsimtest osmo-remsim-client-st2[28923]:             struct ipa_client_conn         contains    464 bytes in   6 blocks (ref 0) 0x558090c62480
Jul 18 16:04:19 remsimtest osmo-remsim-client-st2[28923]:                 struct osmo_fsm_inst           contains    277 bytes in   4 blocks (ref 0) 0x558090c61540
Jul 18 16:04:19 remsimtest osmo-remsim-client-st2[28923]:                     struct ipa_fsm_priv            contains     32 bytes in   1 blocks (ref 0) 0x558090c61770
Jul 18 16:04:19 remsimtest osmo-remsim-client-st2[28923]:                     IPA-KEEPALIVE(server)[0x558090c61540] contains     38 bytes in   1 blocks (ref 0) 0x558090c616e0
Jul 18 16:04:19 remsimtest osmo-remsim-client-st2[28923]:                     server                         contains      7 bytes in   1 blocks (ref 0) 0x558090c61670
Jul 18 16:04:19 remsimtest osmo-remsim-client-st2[28923]:                 10.9.99.10                     contains     11 bytes in   1 blocks (ref 0) 0x558090c62640
Jul 18 16:04:19 remsimtest osmo-remsim-client-st2[28923]:             RSPRO_CLIENT(server)[0x558090c62b20] contains     37 bytes in   1 blocks (ref 0) 0x558090c62c50
Jul 18 16:04:19 remsimtest osmo-remsim-client-st2[28923]:             server                         contains      7 bytes in   1 blocks (ref 0) 0x558090c50600
Jul 18 16:04:19 remsimtest osmo-remsim-client-st2[28923]:         ClientSlot_t                   contains     40 bytes in   1 blocks (ref 0) 0x558090c50f30
Jul 18 16:04:19 remsimtest osmo-remsim-client-st2[28923]:     logging                        contains    889 bytes in   5 blocks (ref 0) 0x558090c3e200
Jul 18 16:04:19 remsimtest osmo-remsim-client-st2[28923]:         struct log_target              contains    208 bytes in   2 blocks (ref 0) 0x558090c3e5f0
Jul 18 16:04:19 remsimtest osmo-remsim-client-st2[28923]:             struct log_category            contains     40 bytes in   1 blocks (ref 0) 0x558090c3e700
Jul 18 16:04:19 remsimtest osmo-remsim-client-st2[28923]:         struct log_info                contains    680 bytes in   2 blocks (ref 0) 0x558090c3e270
Jul 18 16:04:19 remsimtest osmo-remsim-client-st2[28923]:             struct log_info_cat            contains    640 bytes in   1 blocks (ref 0) 0x558090c3e300
Jul 18 16:04:19 remsimtest osmo-remsim-client-st2[28923]:     asn1                           contains      0 bytes in   1 blocks (ref 0) 0x558090c3e190

starting one of those in valgrind too (in screen)


Related issues

Related to osmo-remsim - Bug #4097: osmo-remsim-bankd leaking memoryResolved07/10/2019

History

#1 Updated by roh 5 months ago

just a short run of a few seconds:

==29064== 8,610 bytes in 6 blocks are definitely lost in loss record 50 of 51
==29064==    at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
==29064==    by 0x54D9661: talloc_named_const (in /usr/lib/x86_64-linux-gnu/libtalloc.so.2.1.8)
==29064==    by 0x590BB61: msgb_alloc_c (in /usr/lib/x86_64-linux-gnu/libosmocore.so.12.1.0)
==29064==    by 0x4E6C1C8: ipa_msg_alloc (in /usr/lib/x86_64-linux-gnu/libosmogsm.so.12.0.0)
==29064==    by 0x4E6CB3A: ipa_msg_recv_buffered (in /usr/lib/x86_64-linux-gnu/libosmogsm.so.12.0.0)
==29064==    by 0x50A99F5: ??? (in /usr/lib/x86_64-linux-gnu/libosmoabis.so.6.0.2)
==29064==    by 0x590B6F6: osmo_select_main (in /usr/lib/x86_64-linux-gnu/libosmocore.so.12.1.0)
==29064==    by 0x10B279: run_mainloop (simtrace2-remsim_client.c:504)
==29064==    by 0x10B279: main (simtrace2-remsim_client.c:961)
==29064==
==29064== 9,216 bytes in 9 blocks are possibly lost in loss record 51 of 51
==29064==    at 0x4C2BADF: malloc (vg_replace_malloc.c:298)
==29064==    by 0x4C2DE5F: realloc (vg_replace_malloc.c:785)
==29064==    by 0x52C9073: ??? (in /lib/x86_64-linux-gnu/libusb-1.0.so.0.1.0)
==29064==    by 0x52CA173: ??? (in /lib/x86_64-linux-gnu/libusb-1.0.so.0.1.0)
==29064==    by 0x52C8912: ??? (in /lib/x86_64-linux-gnu/libusb-1.0.so.0.1.0)
==29064==    by 0x52C014C: libusb_init (in /lib/x86_64-linux-gnu/libusb-1.0.so.0.1.0)
==29064==    by 0x10AF63: main (simtrace2-remsim_client.c:823)
==29064==
==29064== LEAK SUMMARY:
==29064==    definitely lost: 8,898 bytes in 8 blocks
==29064==    indirectly lost: 369 bytes in 7 blocks
==29064==      possibly lost: 16,302 bytes in 55 blocks
==29064==    still reachable: 12,986 bytes in 19 blocks
==29064==         suppressed: 0 bytes in 0 blocks

#2 Updated by laforge 5 months ago

  • Status changed from New to In Progress
  • % Done changed from 0 to 60

https://gerrit.osmocom.org/c/osmo-remsim/+/14829 should fix this, roh feel free to test

#3 Updated by laforge 5 months ago

  • Related to Bug #4097: osmo-remsim-bankd leaking memory added

#4 Updated by roh 5 months ago

tested with the binaries i got via mail yesterday i think we still have a small msgb leak

Jul 24 16:11:39 remsimtest osmo-remsim-client-st2[2859]: #033[0;mfull talloc report on 'global' (total 6665765 bytes in 4999 blocks)
Jul 24 16:11:39 remsimtest osmo-remsim-client-st2[2859]:     struct bankd_client            contains   2012 bytes in  15 blocks (ref 0) 0x5638a7e9d9c0
Jul 24 16:11:39 remsimtest osmo-remsim-client-st2[2859]:         struct osmo_fsm_inst           contains    240 bytes in   3 blocks (ref 0) 0x5638a7e9fb00
Jul 24 16:11:39 remsimtest osmo-remsim-client-st2[2859]:             BANKD_CONN(bankd)[0x5638a7e9fb00] contains     34 bytes in   1 blocks (ref 0) 0x5638a7e9fca0
Jul 24 16:11:39 remsimtest osmo-remsim-client-st2[2859]:             bankd                          contains      6 bytes in   1 blocks (ref 0) 0x5638a7e9fc30
Jul 24 16:11:39 remsimtest osmo-remsim-client-st2[2859]:         struct osmo_fsm_inst           contains    772 bytes in  10 blocks (ref 0) 0x5638a7e9ddf0
Jul 24 16:11:39 remsimtest osmo-remsim-client-st2[2859]:             struct osmo_fd                 contains     48 bytes in   1 blocks (ref 0) 0x5638a7ea20b0
Jul 24 16:11:39 remsimtest osmo-remsim-client-st2[2859]:             struct ipa_client_conn         contains    480 bytes in   6 blocks (ref 0) 0x5638a7ea1f90
Jul 24 16:11:39 remsimtest osmo-remsim-client-st2[2859]:                 struct osmo_fsm_inst           contains    293 bytes in   4 blocks (ref 0) 0x5638a7e9f870
Jul 24 16:11:39 remsimtest osmo-remsim-client-st2[2859]:                     struct ipa_fsm_priv            contains     48 bytes in   1 blocks (ref 0) 0x5638a7e9fa10
Jul 24 16:11:39 remsimtest osmo-remsim-client-st2[2859]:                     IPA-KEEPALIVE(server)[0x5638a7e9f870] contains     38 bytes in   1 blocks (ref 0) 0x5638a7e9e020
Jul 24 16:11:39 remsimtest osmo-remsim-client-st2[2859]:                     server                         contains      7 bytes in   1 blocks (ref 0) 0x5638a7e9f9a0
Jul 24 16:11:39 remsimtest osmo-remsim-client-st2[2859]:                 10.9.99.10                     contains     11 bytes in   1 blocks (ref 0) 0x5638a7ea2150
Jul 24 16:11:39 remsimtest osmo-remsim-client-st2[2859]:             RSPRO_CLIENT(server)[0x5638a7e9ddf0] contains     37 bytes in   1 blocks (ref 0) 0x5638a7e9df90
Jul 24 16:11:39 remsimtest osmo-remsim-client-st2[2859]:             server                         contains      7 bytes in   1 blocks (ref 0) 0x5638a7e9df20
Jul 24 16:11:39 remsimtest osmo-remsim-client-st2[2859]:         ClientSlot_t                   contains     40 bytes in   1 blocks (ref 0) 0x5638a7e9f3c0
Jul 24 16:11:39 remsimtest osmo-remsim-client-st2[2859]:     logging                        contains    889 bytes in   5 blocks (ref 0) 0x5638a7e88270
Jul 24 16:11:39 remsimtest osmo-remsim-client-st2[2859]:         struct log_target              contains    208 bytes in   2 blocks (ref 0) 0x5638a7e88660
Jul 24 16:11:39 remsimtest osmo-remsim-client-st2[2859]:             struct log_category            contains     40 bytes in   1 blocks (ref 0) 0x5638a7e88770
Jul 24 16:11:39 remsimtest osmo-remsim-client-st2[2859]:         struct log_info                contains    680 bytes in   2 blocks (ref 0) 0x5638a7e882e0
Jul 24 16:11:39 remsimtest osmo-remsim-client-st2[2859]:             struct log_info_cat            contains    640 bytes in   1 blocks (ref 0) 0x5638a7e88370
Jul 24 16:11:39 remsimtest osmo-remsim-client-st2[2859]:     msgb                           contains 6662864 bytes in 4977 blocks (ref 0) 0x5638a7e88200
Jul 24 16:11:39 remsimtest osmo-remsim-client-st2[2859]:         IPA Multiplex                  contains   1339 bytes in   1 blocks (ref 0) 0x5638a8577b80
Jul 24 16:11:39 remsimtest osmo-remsim-client-st2[2859]:         IPA Multiplex                  contains   1339 bytes in   1 blocks (ref 0) 0x5638a85775d0
Jul 24 16:11:39 remsimtest osmo-remsim-client-st2[2859]:         IPA Multiplex                  contains   1339 bytes in   1 blocks (ref 0) 0x5638a8577020
Jul 24 16:11:39 remsimtest osmo-remsim-client-st2[2859]:         IPA Multiplex                  contains   1339 bytes in   1 blocks (ref 0) 0x5638a8576a70
Jul 24 16:11:39 remsimtest osmo-remsim-client-st2[2859]:         IPA Multiplex                  contains   1339 bytes in   1 blocks (ref 0) 0x5638a85764c0
Jul 24 16:11:39 remsimtest osmo-remsim-client-st2[2859]:         IPA Multiplex                  contains   1339 bytes in   1 blocks (ref 0) 0x5638a8575f10
Jul 24 16:11:39 remsimtest osmo-remsim-client-st2[2859]:         IPA Multiplex                  contains   1339 bytes in   1 blocks (ref 0) 0x5638a8575960
Jul 24 16:11:39 remsimtest osmo-remsim-client-st2[2859]:         IPA Multiplex                  contains   1339 bytes in   1 blocks (ref 0) 0x5638a85753b0
Jul 24 16:11:39 remsimtest osmo-remsim-client-st2[2859]:         IPA Multiplex                  contains   1339 bytes in   1 blocks (ref 0) 0x5638a8574e00
...
Jul 24 16:11:42 remsimtest osmo-remsim-client-st2[2859]:         IPA Multiplex                  contains   1339 bytes in   1 blocks (ref 0) 0x5638a7e9e150
Jul 24 16:11:42 remsimtest osmo-remsim-client-st2[2859]:         IPA Multiplex                  contains   1339 bytes in   1 blocks (ref 0) 0x5638a7e9fdd0
Jul 24 16:11:42 remsimtest osmo-remsim-client-st2[2859]:     asn1                           contains      0 bytes in   1 blocks (ref 0) 0x5638a7e88190

#5 Updated by laforge 5 months ago

#6 Updated by laforge 4 months ago

  • Status changed from In Progress to Feedback
  • Assignee changed from laforge to roh
  • % Done changed from 60 to 70

#7 Updated by roh 3 months ago

i think this can be closed.

#8 Updated by laforge 14 days ago

  • Status changed from Feedback to Closed
  • % Done changed from 70 to 100

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 48.8 MB)