Feature #6135
openImplement routing of SMS-over-GSUP messages between connected MSC/VLRs and SMSCs
90%
Description
As of #3587, OsmoMSC can suppress its built-in SMSC and instead pass SMS-PP to and from external SMSCs via MAP-mimicking GSUP, following classic GSM architecture. However, current OsmoHLR is unable to route these SMS-over-GSUP messages: if I implement my own GSUP-speaking SMSC and connect it to OsmoHLR, the latter won't know how to forward MO-forwardSM.req from OsmoMSC to the SMSC based on the SC-address in SM-RP-DA, or how to forward MT-forwardSM.req in the opposite direction based on the IMSI.
Themyscira Wireless is currently in need of a custom SMSC, I am starting to implement one, and I am looking to do it in the most GSM-architecture-classic manner. The mechanism of SMS-over-GSUP looks like the right way to do what I seek, or at least from OsmoMSC side the protocol does exactly what it should, pass SM-RP to an external SMSC with minimal transformation. My SMSC would connect to OsmoHLR via GSUP much like EUSEs do for USSD - copy the same model, so far so good. But the routing piece in OsmoHLR is missing.
I propose the following design:
- Similarly to how EUSEs connect with IPA names of the form EUSE-foobar, require SMSCs to connect with IPA names of the form SMSC-12345, where 12345 is the SC-address of this SMSC, i.e., the number to be programmed as the Service-Centre-Address in the EF_SMSP record on SIM cards.
- When MO-forwardSM.req comes from MSC/VLR, route it based on SM-RP-DA: require the address type to be SMSC, and look for a connected GSUP peer with name SMSC-12345 or whatever digit string is in the SC address field coming from the MS.
- To route MO-forwardSM.{resp,err} correctly, the SMSC will be responsible for setting destination_name in the response to a copy of source_name from the request, ideally using osmo_gsup_req_respond().
- When MT-forwardSM.req comes from an SMSC, route it to the serving VLR based on the IMSI just like MT USSD coming from an EUSE.
- For MT-forwardSM.{resp,err} routing, OsmoMSC will need to be patched to fill response destination_name with a copy of request source_name, which it currently fails to do.
The only part for which I don't have any solution currently is READY-FOR-SM.req: it's a request coming from MSC/VLR, but there is no SMSC address to route to. In the official architecture the HLR is expected to keep a list of SMSC that attempted and failed MT SMS delivery, and notify all of them upon a single "ready" notification from MSC/VLR - but implementing all that complexity would be a bit too much for me at the present moment. I am thinking of having OsmoHLR simply reply with "success" to READY-FOR-SM.req as a FIXME placeholder in the initial implementation, just so that the MS gets an RP-ACK rather than RP-ERROR (the MS did nothing wrong!), and leave it as a problem to be solved later.
Any better ideas?
Related issues
Updated by falconia 8 months ago
- Status changed from New to In Progress
- Assignee set to falconia
- % Done changed from 0 to 20
The initial implementation is here:
https://cgit.osmocom.org/osmo-hlr/?h=falconia/os6135
https://cgit.osmocom.org/osmo-msc/?h=falconia/os6135
More work needs to be done, particularly in testing - so far only the MO path has been tested and proven working. Further updates will follow.
Updated by falconia 8 months ago
- % Done changed from 20 to 40
Both MO and MT SMS paths have now been proven working. See the links above for the branch in osmo-hlr and osmo-msc (patches unchanged), plus these test fixture components:
https://www.freecalypso.org/hg/osmo-playpen/
https://www.freecalypso.org/hg/sms-coding-utils/
The only part which remains to be implemented is routing of READY-FOR-SM messages, which is a non-immediately-obvious problem because neither OsmoMSC nor OsmoHLR remembers which SMSC have tried and failed to deliver SMS to a given IMSI. When fixeria and I discussed this issue, we both concluded that the simplest solution is to replicate READY-FOR-SM.req from an MSC to all connected SMSCs - obviously not particularly efficient, but realistically speaking, how many different SMSCs will be connected to OsmoHLR in a typical Osmocom network...
Updated by falconia 8 months ago
- % Done changed from 40 to 50
READY-FOR-SM handling is now implemented, same falconia/os6135 branch in osmo-hlr as before. The work that is currently implemented on the branch is expected to be good for production use at Themyscira Wireless; the next step is to rebase the patches to current master and submit to Gerrit for review.
Updated by falconia 7 months ago
- % Done changed from 50 to 60
OsmoHLR patches have been rebased to current master and submitted to Gerrit:
https://gerrit.osmocom.org/c/osmo-hlr/+/34448
https://gerrit.osmocom.org/c/osmo-hlr/+/34449
https://gerrit.osmocom.org/c/osmo-hlr/+/34450
https://gerrit.osmocom.org/c/osmo-hlr/+/34451
Updated by falconia 7 months ago
- % Done changed from 70 to 80
OsmoMSC patches have been rebased to current master and submitted to Gerrit:
https://gerrit.osmocom.org/c/osmo-msc/+/34516
https://gerrit.osmocom.org/c/osmo-msc/+/34517
https://gerrit.osmocom.org/c/osmo-msc/+/34518
Updated by falconia 7 months ago
- Status changed from In Progress to Stalled
- % Done changed from 80 to 90
Code implementation is complete: all necessary patches to both OsmoHLR and OsmoMSC have been merged. However, documentation remains to be written: an entirely new facility has just been implemented (ability to connect external SMSC implementations to an otherwise standard and self-contained Osmocom CNI setup), hence it needs to be documented. I reason that OsmoHLR manual would probably be the proper place for the new chapter, as the required SMSC routing configuration needs to be done in OsmoHLR vty.
Documentation is a type of work I tend to be good at, at least in my own projects under FreeCalypso umbrella, but I don't know what kind of difficulties I may run into with editing Osmocom manuals. Common sense tells me that prior to submitting any OsmoHLR manual patches to Gerrit, I will need to make sure that my changes to adoc source compile into PDF locally, and that the result looks as desired - but I have yet to try building any Osmocom component with --enable-manuals, and I have a reasonable fear of running into dependency hell once again.
Because of these considerations, I am marking this feature ticket as Stalled for now.
Updated by falconia 4 months ago
There is still this outstanding sub-issue: OsmoHLR applies active routing (explicit routing code written for SMS over GSUP) only to request messages in each of the 3 possible exchanges (MO-forwardSM, MT-forwardSM and ReadyForSM), while responses to all 3 are expected to be routed passively, based on the destination_name IE. The entity responding to the request (SMSC responding to MO-forwardSM.req and ReadyForSM.req, and OsmoMSC responding to MT-forwardSM.req) must set destination_name in the response to source_name from the request - fair enough. But the outstanding problem is that for this passive routing to work in OsmoHLR, the original sender of the response message must set not only destination_name, but also source_name, i.e., explicitly name itself. Why is it needed? Currently this quirk is needed because the passive routing path through OsmoHLR does not decode and re-encode, it just forwards the packet blob, hence source_name must be set from the beginning in order for the final recipient to get it correctly.
The problem right now is that OsmoMSC can properly name itself in the source_name IE only when a custom ipa-name has been set in its vty config. If it runs with its default ipa-name (which should be fine whenever there is only one MSC), it is currently unable to set the source_name IE in GSUP responses, and thus cannot correctly receive and respond to MT-forwardSM.
There is a proposed fix here:
https://gerrit.osmocom.org/c/osmo-msc/+/35384
but it is still waiting for laforge and neels to weigh in.
If my proposed fix is bad, let us by all means address the problem in some other way - but simply ignoring the issue does not look good.
Updated by fixeria 4 months ago
falconia wrote in #note-9:
There is a proposed fix here:
The patch has been merged, and since then we're seeing regressions in ttcn3-msc-test:
https://jenkins.osmocom.org/jenkins/view/TTCN3/job/ttcn3-msc-test/2260/ +36 failures
Those failures are caused by presence of the Source Name IE
in GSUP PDUs coming from osmo-msc, which is not expected in our TTCN-3 templates.
So it's not a fault of Mychaela's patch, but rather a problem of our testsuite: the templates are not flexible enough to match optional IEs.
Ideally, we should rework the templates, so that we could handle new / optional IEs gracefully. But as discussed with Harald, this would require significant effort.
For now, let's add the missing IE to our templates and ensure that both -lastest and -master versions of osmo-msc are sending it:
https://gerrit.osmocom.org/c/docker-playground/+/35520 ttcn3-msc-test: set 'hlr' / 'ipa-name' explicitly [NEW]
https://gerrit.osmocom.org/c/osmo-ttcn3-hacks/+/35517 MSC_Tests: indicate the failure reason in setverdict() [NEW]
https://gerrit.osmocom.org/c/osmo-ttcn3-hacks/+/35518 library/GSUP_Types: set Message Class IE in send templates for SMS [NEW]
https://gerrit.osmocom.org/c/osmo-ttcn3-hacks/+/35519 library/GSUP_Types: add Source Name IE to receive templates for SMS [NEW]
With these patches applied, the failing testcases (at least some of them) are passing again for the current master.