Cellular Network Infrastructure » Core Network (CN) » OsmoGGSN (former OpenGGSN) » Linux Kernel GTP-U

This is what the failing ttcn3 does:

    /* Test PDP context activation for dynamic IPv6 EUA with IPv6 DNS in PCO and router solicitation/advertisement.
       Test we can send ICMPv6 ping over GTPU to DNS server. */
    testcase TC_pdp6_act_deact_gtpu_access() runs on GT_CT {
        f_init();
        var PdpContext ctx := valueof(t_DefinePDP(f_rnd_imsi('26242'H), "1234", c_ApnInet6, valueof(t_EuaIPv6Dyn)));
        ctx.pco_req := valueof(ts_PCO_IPv6_DNS);
        f_pdp_ctx_act(ctx);

        f_send_gtpu(ctx, f_icmpv6_rs_for_pdp(ctx));
        f_wait_rtr_adv(ctx);
        f_send_gtpu(ctx, f_gen_icmpv6_neigh_solicit_for_pdp(ctx));

        var OCT16 dns1_addr := f_PCO_extract_proto(ctx.pco_neg, '0003'O);

//////////////////////////////////////////////
// PESPIN: THIS IS THE PART FAILING!!!!
//////////////////////////////////////////////
        /* Check if we can use valid link-local src addr.  */
        var OCT16 saddr_ll := f_ipv6_link_local(ctx.eua.endUserAddress.endUserAddressIPv6.ipv6_address);
        f_send_gtpu(ctx, f_gen_icmpv6_echo(saddr_ll, dns1_addr));
        if (m_ggsn_impl == GGSN_IMPL_OSMOCOM) {
            f_wait_icmp6_echo_reply(ctx);
        } else {
            f_wait_gtpu_fail(ctx);
        }
//////////////////////////////////////////////

        /* Check if we can use valid global src addr, should work */
        var OCT16 saddr_glob := f_ipv6_global(ctx.eua.endUserAddress.endUserAddressIPv6.ipv6_address);
        f_send_gtpu(ctx, f_gen_icmpv6_echo(saddr_glob, dns1_addr));
        f_wait_icmp6_echo_reply(ctx);

        f_pdp_ctx_del(ctx, '1'B);
        f_shutdown_helper();
    }

So basically this test creates the pdp context against osmo-ggsn, and then tries 2 things:
1- Sending a ping to the reported DNS IP address using the source link-local assigned IPv6 address
1- Sending a ping to the reported DNS IP address using the source global assigned IPv6 address

The DNS IP address is an IP address assigned on the same host but on another interface (eth in the same docker container where osmo-ggsn runs).

When using osmo-ggsn in userspace, the process receives the packet from the UDP/GTP-U socket fd and sees the source (link-local) IPv6 address matches the registration, so it let's it pass/forward by injecting the packet into kernel network stack through the tun fd.

When using osmo-ggsn gtp kernel module, the gtp kernel module handles the UDP/GTP-U code and gtp_check_ms() calls gtp_check_ms_ipv6() which does:

    if ((ipv6_addr_type(&ip6h->saddr) & IPV6_ADDR_LINKLOCAL) ||
        (ipv6_addr_type(&ip6h->daddr) & IPV6_ADDR_LINKLOCAL))
        return false;

That means "don't forward the packet through the kernel network stack, but push it to the userspace program through the fd1u socket).
When osmo-ggsn receives the UDP/GTP-U/IPv6(ll-addr) packet, it decapsulates it and again (as in userspace setup explained above), tries to inject it over the "tun->fd", which in this case was never initialized to a valid fd, but to fd = -1 in tun_new().

So this raises several questions we should agree on:
1- Is there a way to obtain the "tun->fd" equivalent to the generic tundev? (aka open("/dev/net/tun", O_RDWR)). If yes, we could implement that to be able to inject packets like we do in the generic tunnel scenario. If not, we assume this is not possible.
2- Should osmo-ggsn actually let pass/forward out of the tunnel packets received of the form UDP/GTP-U/IPv6(ll-addr) packet ?
2.1- Forwarding them should allow for instance pinging an IP address set on the tunnel interface
2.1- Forwarding them also allows, well, ending up with a link-local-source-address packet forwarded to some other interface, which may be something not wanted? IPv6 experts here welcome.

pespin wrote in #note-1:

This is what the failing ttcn3 does:
[...]

So basically this test creates the pdp context against osmo-ggsn, and then tries 2 things:
1- Sending a ping to the reported DNS IP address using the source link-local assigned IPv6 address
1- Sending a ping to the reported DNS IP address using the source global assigned IPv6 address

The DNS IP address is an IP address assigned on the same host but on another interface (eth in the same docker container where osmo-ggsn runs).

When using osmo-ggsn in userspace, the process receives the packet from the UDP/GTP-U socket fd and sees the source (link-local) IPv6 address matches the registration, so it let's it pass/forward by injecting the packet into kernel network stack through the tun fd.

When using osmo-ggsn gtp kernel module, the gtp kernel module handles the UDP/GTP-U code and gtp_check_ms() calls gtp_check_ms_ipv6() which does:
[...]
That means "don't forward the packet through the kernel network stack, but push it to the userspace program through the fd1u socket).
When osmo-ggsn receives the UDP/GTP-U/IPv6(ll-addr) packet, it decapsulates it and again (as in userspace setup explained above), tries to inject it over the "tun->fd", which in this case was never initialized to a valid fd, but to fd = -1 in tun_new().

Hm.

So this raises several questions we should agree on:
1- Is there a way to obtain the "tun->fd" equivalent to the generic tundev? (aka open("/dev/net/tun", O_RDWR)). If yes, we could implement that to be able to inject packets like we do in the generic tunnel scenario. If not, we assume this is not possible.

It is userspace that creates the socket and it passes it on to the GTP driver, why is this set to -1 in osmo-ggsn?

2- Should osmo-ggsn actually let pass/forward out of the tunnel packets received of the form UDP/GTP-U/IPv6(ll-addr) packet ?
2.1- Forwarding them should allow for instance pinging an IP address set on the tunnel interface
2.1- Forwarding them also allows, well, ending up with a link-local-source-address packet forwarded to some other interface, which may be something not wanted? IPv6 experts here welcome.

pablo wrote in #note-2:

So this raises several questions we should agree on:
1- Is there a way to obtain the "tun->fd" equivalent to the generic tundev? (aka open("/dev/net/tun", O_RDWR)). If yes, we could implement that to be able to inject packets like we do in the generic tunnel scenario. If not, we assume this is not possible.

It is userspace that creates the socket and it passes it on to the GTP driver, why is this set to -1 in osmo-ggsn?

It's not the socket that doesn't exist - it's the tun-device-fd. We receive an encapsulated packet through the UDP socket, but then apparently wants to decapsulate it and make it appear on the net-device. For tun, this works as a tun-device has a file-descriptor through which the userspace program can inject packets into the tun device. I don't know why this feature is needed, unfortunately the issue doesn't describe it. But certainly the gtp net-device is not a tun-device and hence we cannot inject any packets to it from userspace. the gtp net-device only receives packets decapsulated by the kernel.

2- Should osmo-ggsn actually let pass/forward out of the tunnel packets received of the form UDP/GTP-U/IPv6(ll-addr) packet ?
2.1- Forwarding them should allow for instance pinging an IP address set on the tunnel interface

to clarify: When you say forwarding you actually mean decapsulating, and not forwarding in the sense usually used in the linux kernel when packets are forwareded from one-net device to another?

Are there any other use cases where the UE would need to reach an IP address configured locally on the GGSN?

So basically the question is: Is our test case using a realistic test case, or is it testing something esoteric that is implementation dependant and not guaranteed to work anyway.

Did anyone try to ping a link-local GGSN/PGW address on a commercial cellular network?

2.1- Forwarding them also allows, well, ending up with a link-local-source-address packet forwarded to some other interface, which may be something not wanted? IPv6 experts here welcome.

link-local packets are never routed, they are local to a link. The kernel ipv6 routing/forwarding code should make sure of that, if you attempt it.

laforge wrote in #note-4:

pablo wrote in #note-2:

So this raises several questions we should agree on:
1- Is there a way to obtain the "tun->fd" equivalent to the generic tundev? (aka open("/dev/net/tun", O_RDWR)). If yes, we could implement that to be able to inject packets like we do in the generic tunnel scenario. If not, we assume this is not possible.

It is userspace that creates the socket and it passes it on to the GTP driver, why is this set to -1 in osmo-ggsn?

It's not the socket that doesn't exist - it's the tun-device-fd. We receive an encapsulated packet through the UDP socket, but then apparently wants to decapsulate it and make it appear on the net-device. For tun, this works as a tun-device has a file-descriptor through which the userspace program can inject packets into the tun device. I don't know why this feature is needed, unfortunately the issue doesn't describe it. But certainly the gtp net-device is not a tun-device and hence we cannot inject any packets to it from userspace. the gtp net-device only receives packets decapsulated by the kernel.

Right, this is a listener socket, which cannot be used to send the reply.

I guess that, in case this is ever needed, it can be entirely handled by userspace through a UDP sender socket which already sends a GTP packet? The packet in the receiving side is decapsulated from userspace, in this case, userspace would also fully deal with this control traffic by encapsulating the reply in a GTP header.

On Mon, Mar 04, 2024 at 05:22:48PM +0000, pablo wrote:

laforge wrote in #note-4:

It's not the socket that doesn't exist - it's the tun-device-fd. We receive an encapsulated packet through the UDP socket, but then apparently wants to decapsulate it and make it appear on the net-device. For tun, this works as a tun-device has a file-descriptor through which the userspace program can inject packets into the tun device. I don't know why this feature is needed, unfortunately the issue doesn't describe it. But certainly the gtp net-device is not a tun-device and hence we cannot inject any packets to it from userspace. the gtp net-device only receives packets decapsulated by the kernel.

Right, this is a listener socket, which cannot be used to send the reply.

The UDP socket is a normal datagram socket and can very well be used to send UDP replies to the remote GSN[s], as far as I know. This is exactly how GTP ECHO RESP are transmitted in respose to inbound GTP ECHO REQ, right?

As I stated above, this is NOT about the UDP socket for encapsulated packets. It is about the decapsulated inner IP packet!

I guess that, in case this is ever needed, it can be entirely handled by userspace through a UDP sender socket which already sends a GTP packet? The packet in the receiving side is decapsulated from userspace, in this case, userspace would also fully deal with this control traffic by encapsulating the reply in a GTP header.

The point is: What about GTP packets arriving from a remote GSN, which the kernel module does not understand
itself (e.g. extension headers, or whatever else). They are passed down the UDP socket into user space. If
those packets actually need to be passed on (in decapsulated form), this is not possible like in the
tun-device case.

Basically at this point userspace wants a feature "inject this decapsulated IP packet into the netwokr stack
as if it was received by the gtpX net-device".

The question is: How many valid use cases exist for it. We already established that we don't need it for
SLAAC. Pinging a link-local IPv6 address of the gtp-device is all we know so far, which by itself is likely
not a relevant/important feature.

I think our tests should change to not ping a link-local IP address but some other "real" IP address that can be reached through/behind the tunnel. This way we are not testing a niche feature but something a production user wants: Excahnging IP traffic with hosts behind The GGSN/PGW.

The question remains: Can we think of other cases, where a GTP packet is not decapsulated in the kernel but
passed to userspace via the UDP socket, and which userspace would need to pass on as decapsulated packet?

• GTP ECHO is not a problem as we just respond to it and send a UDP response. No need to decapsulate
• SLAAC is handled entirely in userspace and also just generates UDP/GTP responses

Fundamentally, you can think the kernel GTP moduel is an acceleated data path for the most common case. All
other cases nod handled in the kernel are passed to the slow path in userspace. We're just discovering that
that slow path can really only send GTP responses and not implement any actual decapsulation.

Userspace could of course create a RAW or even PACKET socket and send the decapsulated IP datagram that way.
However, to the rest of the network stack this packet would then look like a locally-generated packet (coming
from sockets, traversing through OUTPUT hook) and not like the fast-path packets which look like they
were receiced on the 'gtpX' net-device and then pass through FORWARD hook. This means that the RAW or PACKET socket packets would traverse different packet filter / tc / policy routing / NAT rules, so I don't think that is a proper solution.

So, conceptually we are missing some functionality here. The question remains: Is there a real-world use case that neesd it?

pespin wrote in #note-6:

pablo wrote in #note-2:
I guess that, in case this is ever needed, it can be entirely handled by userspace through a UDP sender socket which already sends a GTP packet? The packet in the receiving side is decapsulated from userspace, in this case, userspace would also fully deal with this control traffic by encapsulating the reply in a GTP header.

You are talking about forwarding GTP here, but I meant decapsulating. I think in any case if the userspace process still wants to decapsulate + inject it could do so by creating a RAW socket?

Yes, handle this entirely from userspace, or even use the existing UDP socket that is handed over to the kernel as Harald mentioned. Userspace would need to build the GTP header and the inner headers entirely, then send it through the UDP socket.

So far my opinion is that it's fine not easily supporting this scenario with the "gtp" tun device, unlike with the general "tun" device. If this really specific test case is needed, then it's up to the user to create a RAW socket or whatever to inject the IP packets. Just wanted laforge and pablo to also analyse the problem to see that we all agree on this.

Correct, "gtp" kernel device goal never really meant to provide a 1:1 mapping with "tun" device semantics, and benefits of allowing to inject the reply packet through the "gtp" kernel driver is limited compared to the complexity of handling this in userspace.

I assume tests need to be amended to sort out this issue. Thanks!

laforge wrote in #note-7:

The UDP socket is a normal datagram socket and can very well be used to send UDP replies to the remote GSN[s], as far as I know. This is exactly how GTP ECHO RESP are transmitted in respose to inbound GTP ECHO REQ, right?

ACK, the fd0 and fd1u sockets are working fine and can be used to eg. answer the Router Solicitation with Router Advertisement during IPv6 SLAAC. That's working fine. GTP ECHO RESP also works fine, I just tested that with ttcn3 on osmo-epdg (IPv4), where gtp_u_kmod writes on fd1u to answer the gtp echo req with a reply.

I think our tests should change to not ping a link-local IP address but some other "real" IP address that can be reached through/behind the tunnel. This way we are not testing a niche feature but something a production user wants: Excahnging IP traffic with hosts behind The GGSN/PGW.

The ttcn3 test actually validates 2 things as mentioned in the initial post:

1- Sending a ping to the reported DNS IP address using the source link-local assigned IPv6 address
1- Sending a ping to the reported DNS IP address using the source global assigned IPv6 address

The problem is not really the "destination IP address". The problem in the test is the "source IP address" being link-local. In any case, the gtp kernel will forward the packet to userspace UDP socket if any of the source or destination inner addresses are link-local (see also initial post, there's the code from the kernel there).

Fundamentally, you can think the kernel GTP moduel is an acceleated data path for the most common case. All
other cases nod handled in the kernel are passed to the slow path in userspace. We're just discovering that
that slow path can really only send GTP responses and not implement any actual decapsulation.

ACK, that's a great summary of the "issue" :)

So, conceptually we are missing some functionality here. The question remains: Is there a real-world use case that neesd it?

Another hypothetical use case would be also to explicitly avoid configuring the gtp kernel for a given (set of) pdp contexts, to do extra mangling in userspace before reinjecting the packet (eg. lawful interception or alike).
But I guess anyway lawful interception can be done by other means like matching UE IP address on the decapsulated traffic.
So I agree everything is really not "standard" use, and so probably we shouldn't worry about them.

I say we simply edit the test to expect that scenario to not work, same as we do with open5gs.
Now an extra question would be: do we want to allow this to work when osmo-ggsn uses the generic tun dev? Or we want to discard packets with src link-local address which are currently being reinjected into the tun dev to be processed by the linux ip stack?
If we want the same behavior, we should add in osmo-ggsn userplane path the same check done in the gtp kernel module.