Bug #1994
closedCrash in osmo-hnbgw testing 3G setup
0%
Description
I was testing a 3G setup successfully as described in [1], with a ip.access nano3G [2], all branches were fetched+compiled today. At some point in time (I think it happened while registering a second UE, but not sure) I saw osmo-hnbgw had crashed.
I attach a log file + a backtrace in case somebody can easily hunt the issue. I also attach the core file generated from gdb. At first glance it looks like a linked list corruption or the cn object being used after freed.
[1] http://osmocom.org/projects/cellular-infrastructure/wiki/Getting_Started_with_3G
[2] http://osmocom.org/projects/cellular-infrastructure/wiki/Configuring_the_ipaccess_nano3G
Files
Updated by neels about 7 years ago
- Status changed from New to In Progress
- Assignee set to pespin
- Priority changed from Normal to High
On first sight, it looks like an llist entry was talloc_free()d without llist_del()ing first.
Be aware: talloc allocates trees of items, so if the linked list item was allocated from a "too detailed" context and that context was freed, then the list item would be deallocated along with it, without an explicit llist_del() on that item.
In osmo-iuh/src/context_map.c in context_map_alloc_by_hnb(), the the hnbgw_context_map item is allocated, using 'hnb' as talloc context:
map = talloc_zero(hnb, struct hnbgw_context_map);
Through various indirections (msgb->dst, osmo_stream_srv data), this hnb is a struct hnb_context and comes from hnbgw.c, i.e. the 'ctx' in:
osmo_stream_srv_create(tall_hnb_ctx, link, new_fd, hnb_read_cb, hnb_close_cb, ctx);
I notice that the hnb_close_cb() is empty. It should probably remove the hnb_context from the gw->hnb_list, and it should also go through its map_list to explicitly llist_del the hnbgw_context_map items created in context_map_alloc_by_hnb() -- both llist_del(map->hnb_list) and llist_del(map->cn_list).
But actually it looks like nothing is ever freeing the hnb (if the hnb_close_cb doesn't, who else will), so while this is wrong/missing, it doesn't look like the real cause for an invalid hnb_context_map item. A cause could be that hnb_close_cb() leaves the hnb configured even though its link pointers have become invalid. It might be used again later but should be gone, causing some corruption leading to a hnb_context_map error?
And still, this error should only appear if you leave the hnbgw running while reconnecting an hnodeb. In our chat you said that you are restarting the hnbgw process before the crash appears.
You said that you put the phone in flight mode, then restart all core network programs including the osmo-hnbgw, wait for the femto to re-attach to hnbgw and then disable flight mode, causing the crash. That would to me sound like the identical scenario to just starting up the 3G system in the first place (as far as osmo-hnbgw is concerned). Because the hnbgw program is restarted, there is no state from the previous connection in the hnbgw. Does the crash not happen the first time you launch the hnb? Are you positive that you're really restarting the osmo-hnbgw? =)
Given your input, I'm not sure that I have really found what is causing this, even though I have found something that needs fixing.
Pau, would you like to continue on this? Otherwise set me as assignee of this issue.
Updated by pespin about 7 years ago
The following patch fixes the issue for me: https://gerrit.osmocom.org/#/c/2232/
