Cellular Network Infrastructure » Radio Access Network » OsmoBTS

Turns out, libosmo-abis' RSL link strongly depends on the IPA ID GET message to be sent, or it will readily dereference an uninitialized link.
To illustrate, this libosmo-abis patch fixes the failure:

--- a/src/input/ipaccess.c
+++ b/src/input/ipaccess.c
@@ -753,7 +753,8 @@ static int ipaccess_bts_read_cb(struct ipa_client_conn *link, struct msgb *msg)
        if (ret == 1 && hh->proto == IPAC_PROTO_IPACCESS) {
                uint8_t msg_type = *(msg->l2h);

-               if (msg_type == IPAC_MSGT_ID_GET) {
+               if (msg_type == IPAC_MSGT_ID_GET
+                   || msg_type == IPAC_MSGT_ID_ACK) {
                        sign_link = link->line->ops->sign_link_up(msg,
                                        link->line,
                                        link->ofd->priv_nr);

This works because I know we're sending an IPA ID ACK now, and triggering on the ID ACK to initialize the link solves the problem in this instance.
Of course libosmo-abis should never crash, whether an initial message was received or not, so we still need a fix for the crash situation regardless.

The crash happens here:

        else if (link->port == IPA_TCP_PORT_RSL)
                e1i_ts = &link->line->ts[link->ofd->priv_nr-1];  <-- got the RSL link

        OSMO_ASSERT(e1i_ts != NULL);

        /* look up for some existing signaling link. */
        sign_link = e1inp_lookup_sign_link(e1i_ts, hh->proto, 0);  <-- entering
---
struct e1inp_sign_link *e1inp_lookup_sign_link(struct e1inp_ts *e1i,
                                                uint8_t tei, uint8_t sapi)
{
        struct e1inp_sign_link *link;

        llist_for_each_entry(link, &e1i->sign.sign_links, list) {
                if (link->sapi == sapi && link->tei == tei)       <--- crash
                        return link;
        }

        return NULL;
}

I assume the sign_links list is not initialized?? Checking now.

about the detailed failure cause:

in osmo-bts, sign_link_up() calls e1inp_ts_config_sign(), which sets INIT_LLIST_HEAD(&ts->sign.sign_links);
Before that, the ts->sign.sign_links llist is indeed still uninitialized.
(It would be nice to call INIT_LLIST_HEAD() sooner, i.e. in e1inp_line_create(), but some callers, e.g. bs11_config, call only e1inp_ts_config_sign() and never call e1inp_line_create().)

There should definitely be a safeguard to not use the e1i_ts if it hasn't been initialized: https://gerrit.osmocom.org/#/c/libosmo-abis/+/10600

There are other nightmares in there:

• directly using unvalidated link->ofd->priv_nr as array index.
• sometimes link->ofd->priv_nr is used, sometimes link->ofd->priv_nr-1 is used, sometimes trx_nr = link->ofd->priv_nr - E1INP_SIGN_RSL (==2), and there is no explanation of this magic.

It is very tempting to eradicate these horrors from the code, but I am not going to touch those now.

A remaining open question: which IPA clients should work with which init behaviors.
TLDR: make ttcn3 IPA server configurable.

We know of an SCCPlite MSC that, when IPA connects to it, directly sends an IPA ID ACK, never an IPA ID GET.
Thus it seems too fragile to depend on exactly an IPA ID GET to initialize RSL; but actually all our own current IPA servers do that.
The only reason this whole issue shows up is that I adjusted the ttcn3 IPA server to match the behavior of that SCCPlite MSC.

To work around the current stalemate, I could make the ttcn3 IPA server's behavior configurable, so that it sends IPA ID ACK when it acts as SCCPlite-MSC, and sends IPA ID GET when it acts as OML/RSL server. That seems the simplest solution.

What I don't like though is that we then have two different IPA standards, one on the OML/RSL side, and another on the MSC A-interface side.
The warmest fuzziest would be to make both implementations (OML/RSL client in libosmo-abis and the SCCPlite ipa_asp_fsm in libosmo-sccp) work with both IPA ID GET and IPA ID ACK behaviors, and then we can leave the current ttcn3 IPA behavior unchanged to always send only IPA ID GET.

But, we anyway still want to test exactly this SCCPlite MSC's behavior in ttcn3!

In conclusion, either way, step 1 is to make the ttcn3 IPA server configurable, to pick between sending IPA ID GET or IPA ID ACK.
After that follows:

• make libosmo-abis client work when it doesn't get IPA ID GET (optional).
• make ipa_asp_fsm client work when it doesn't get IPA ID ACK (optional, pending other MSCs' behavior).
• after that, also test in ttcn3 that ipa_asp_fsm works with the IPA ID GET behavior

The above paragraph is a bit off topic from this issue, which is frankly just about crashing in the absence of IPA ID GET.
Above mentioned https://gerrit.osmocom.org/#/c/libosmo-abis/+/10600 is the fix for this issue.

merged