Feature #4092


Osmux: Support dynamic allocation of osmux sockets

Added by pespin over 4 years ago. Updated over 1 year ago.

Target version:
Start date:
Due date:
% Done:


Spec Reference:


Initial idea came from #2551 (no need to read all that):

Right now, we have 1 unique Osmux socket per MGW instance.

Osmux streams (one direction) are identified by tuple: <srcIPaddr, srcPort, dstIPaddr, dstPort, dstCID>
Since we currently only have 1 socket, dstIPaddr and dstPort are constant, so we identify received messages by: <srcIPaddr, srcPort, dstCID>.

if the other peer is behind a NAT, srcIPaddr and srcPort are not really reliable easily, so we can only rely on dstCID to identify streams, and thus we share the CID range against all BSCs or connections.
We actually do something similar for RTP, let's think about it:
tuple identified by: <srcIPaddr, srcPort, dstIPaddr, dstPort, dstCID>, but in this case dstPort changes per-conn, so we actually identify streams per dstPort in this case.

So in Osmux streams, we identify based on dstCID, and in RTP, in dstPort. The problem is that dstCID is actually in range 0..255, which means we can get out of CIDs (not like with ports, were range is a lot higher).

In order to improve this situation (only 255 concurrent call legs in MGW using Osmux), here's my proposal:

VTY specifies an osmux IP (IPmsc) and a port range (PORTmsc1..100), or a port to start counting from, and dynamically allocates/binds and closes ports based on demand. These ports are allocated in MGW based on received MGCP <remoteIP,remotePort> tuple. This way we end up having 256 CIDs per <remoteIP,remotePort> tuple, aka per BSC, and which is the same, per <localPort>. Each BSC can actually open new osmux sockets themselves once the first one is full, meaning each BSC can handle also unlimited CIDs, and MSC will be fine because upon receival of this new <remoteIP,remotePort> tuple, a new osmux socket will be created.

All that also plays well with the fact that we want to group Osmux CIDs under same <remotIP,remotePort>, since the trunk frames need to be sent to that same address.

So summary, in MGW:
  • Add a table to match <remoteIP,remotePort> <-> <localPort>, with extra information like: publicRemoteIP, publicRemotePort, Osmux handler.
  • Upon CRCX/MDCX update table: If an entry <remoteIP,remotePort> doesn't exist, create osmux socket + handler on next empty localPort, and fill the table.
  • Upon receival of Osmux pkt, take <dstPort>, and use it in the table to lookup based on localPort, then we know the private and public <remoteIPaddr,port>, and we know to which Osmux handler we need to send the message.
  • When we need to send on an osmux handler, we know localIp, localPort and publicRemoteIP and publicRemotePort, so we can send osmux frames correctly.

Drawbacks: We need at least 1 Osmux port per BSC.
Wins: We can handle unlimited Osmux call legs per BSC and per MSC (as long as ports are free, each port provides 256 streams). It allows us to support multiple MGWs when using Osmux, since now it's not possible afaict.

Related issues

Related to OsmoMGW - Feature #2551: generalization of OSMUX support in osmo-mgwResolvedpespin10/06/2017

Actions #1

Updated by pespin over 4 years ago

  • Related to Feature #2551: generalization of OSMUX support in osmo-mgw added
Actions #2

Updated by pespin over 4 years ago

Forgot to say: There should be a VTY flag to change the behavior explained above in the case there's no NAT in the setup, and remoteIP and remotePort can be used. Then no lookup is needed and localPort can be re-used for each <remoteIP,remotePort> tuple.

Actions #3

Updated by pespin over 1 year ago

Now that we are adding Osmux on the BTS<->BSC link, there are also some more topics to take into account:

Let's call each set of CIDs 0-255 an "Osmux CID namespace", which is a CID to be used a destination over an Osmux link identified by <IPaddr,port>. The namespace and its values is hence controlled by the local host, assigning one namespace (0-255) to a <localIP+localPort>.
However, since in general we want to group payloads of the same namespace towards the same field, we actually better want to maintain a namespace based on <localIP+localPort,remIP+remPort>.

Example: Since there are several BTS connected to the BSC, that means that in general we want to group all osmux CIDs of the same BTS under the same Osmux CID namespace, so that they can be grouped when sending Osmux frames.

Similarly, a osmux node may handle one or more remote namespaces, each assigned to a given <remoteIP,remotePort>.

Further ranting regarding >256 Osmux users:

[Option A]

  • On the BSC, when we do the first MGCP CRCX for the BTS side conn (X-Osmux: *), we still don't know the IPaddr and Osmux port which will be used by the BTS, which means osmo-mgw is unable to select a proper namespace from which to allocate a local CID and give it back in MGCP OK:
    • Right now we only support one namespace so it just simply picks the first available one.
    • When we add support for several namespaces, we probably want to answer MGCP OK with "(X-Osmux: *)" in the case were no remote IPaddr+port is received, then osmo-bsc signal to the BTS IPA CRCX with Osmux-CID IE with special value (0xff?) meaning we want to use Osmux but we still don't know the local CID of the BSC/MGW. Then, in IPA CRCX ACK the BTS provides the BSC with its local CID and IPaddr+port for osmux. Then BTS does MGCP MDCX providing the IPaddr+port+CID and get back the BSC local IPaddr+port+CID, which is set back on the BTS with IPAC MDCX.

With the above proposal, however, we'd have problems on the MSC+MGW, because the MSC also asks for the "X-Osmux: *" without knowing the remote IPaddr+port of the BSC+MGW, and has to send the local CID it got from its MGW before knowing it in BSSMAP Assignment Request. Hence the above proposal wouldn't work there.

[Option B]

What we could do is adding a new MGCP Extension by the side of "X-Osmux", called "X-Osmux-Namespace-Id" containing an unsigned integer. That would tell the MGW to select/allocate/use a CID from CID namespace identified by that unsigned integer. That Namespace-Id could be mapped dynamically (created first time a Namespace-Id is seen) with hashmap/tree in osmo-mgw in {key=namespace_id -> value=osmux_listen<local_ip,local_port>}.
The BSC can then set "X-Osmux-Namespace-Id: <bts-nr>" and "X-Osmux-Namespace-Id: 10000+<msc_nr>" for instance. Similarly, the MSC can set "X-Osmux-Namespace-Id: <bsc-nr>".
This way the BSC and the MSC give hints to the MGW on how to group osmux CIDs into namespaces which will potentially have the same IP address and port (same BTS in the case of BSC, same BSC+MGW in the case of MSC).

[Option C]

We could simply increase Osmux-CID uint8_t -> uint16_t, which means we automatically don't have any problem having to many connections on one Osmux link. Some drawbacks:
  • We break compatibility with older Osmux implementations (not sure there's any running right now). To minimize the breakage, we could use uint8_t if CID < 0x80, and use first bit to indicate a uint16_t is used. This way at least implementations using up to 128 concurrent CIDs would remain compatible. This way we could also extend it to uint32_t if needed in the future (not sure if a MGW_msc would ever handle more than 65536 osmux conns anyway...).
  • Finding a new free CID becomes more complex now (right now it's done with a 256bitstring in a 256/8 byte array). This can be done now maintaining an ordered list or tree of allocated CIDs, then iterate until a spot is found.
  • In this case we'd basically maintain a shared unique local CID namespace mapped to a <localIP+localPort> since it would be big enough.

Right now "Option B" or "Option C" seem the acceptable ways to me.

Actions #4

Updated by laforge over 1 year ago

On Tue, Aug 09, 2022 at 05:44:25PM +0000, pespin wrote:

Right now "Option B" or "Option C" seem the acceptable ways to me.

I agree, both look fine to me, too.


Also available in: Atom PDF

Add picture from clipboard (Maximum size: 48.8 MB)