Feature #6096
openadd support for kernel-GTP IPv6
Added by laforge 10 months ago. Updated about 1 month ago.
80%
Description
pablo has implemented [inner] IPv6 support in the kernel GTP driver and libgtpnl, see #1952
In order to end-to-end test it in our TTCN3 test suite (which already tests ipv6 when used with userspace GTP), we would need to add supprot for it to osmo-ggsn
I think pespin is currently too busy to look at this, hence assigned to osmith, but that's not mandatory. Feel free to pass around as needed.
Related issues
Updated by osmith 10 months ago
- % Done changed from 0 to 30
With https://gerrit.osmocom.org/c/osmo-ggsn/+/33797 and https://gerrit.osmocom.org/c/docker-playground/+/33802 the kernel module can be used for ipv6 and some of the ipv6 and ipv4v6 tests start passing.
However it is not possible currently to configure multiple APNs with the kernel module (in the osmo-ggsn.cfg that does not use the kernel module, we have extra APNs for ipv4, ipv4v6 and ipv6). I'm not sure if this is in scope of this issue or needed at all.
Updated by laforge 10 months ago
On Tue, Jul 18, 2023 at 03:10:20PM +0000, osmith wrote:
However it is not possible currently to configure multiple APNs with the kernel module (in the osmo-ggsn.cfg that does not use the kernel module, we have extra APNs for ipv4, ipv4v6 and ipv6). I'm not sure if this is in scope of this issue or needed at all.
I'm a bit surprised by that constraint. Why would the kernel module care about APNs?
After all, the APN is a completely logical construct, nothing more than an
identifier (spoken only on the GTP-C control plane) to identify the network
device (and hence address pool and where to route packets).
Updated by laforge 10 months ago
On Tue, Jul 18, 2023 at 03:10:20PM +0000, osmith wrote:
However it is not possible currently to configure multiple APNs with the kernel module
I've now re-checked and couldn't find any mention of that constraint in the user manual, nor would
I see why code in gtp-kernel.c would impose such a constraint. gtp_kernel_create() is called
for every net-device, and gtp_kernel_tunnel_add is called for every PDP-context.
Updated by osmith 10 months ago
laforge wrote in #note-7:
On Tue, Jul 18, 2023 at 03:10:20PM +0000, osmith wrote:
However it is not possible currently to configure multiple APNs with the kernel module
I've now re-checked and couldn't find any mention of that constraint in the user manual, nor would
I see why code in gtp-kernel.c would impose such a constraint. gtp_kernel_create() is called
for every net-device, and gtp_kernel_tunnel_add is called for every PDP-context.
When creating the second APN:
tun.c:213 cannot create GTP tunnel device: Device or resource busy
EBUSY gets set here in the kernel: https://git.kernel.org/pub/scm/linux/kernel/git/pablo/gtp.git/tree/drivers/net/gtp.c?id=4b215ddfdb88c8addfa2d775aa1014e7363313fc#n1498
As I understand it, the problem is that we pass the same gsn->fd0 and gsn->fd10 to gtp_kernel_create in lib/tun.c:tun_new().
Besides the problem with multiple APNs, I realized that I have been testing without the libgtpnl changes, and that these will need further changes in osmo-ggsn. I'm working on it and have set the patches to WIP.
full osmo-ggsn logfull osmo-ggsn log
+ osmo-ggsn -c /data/osmo-ggsn.cfg 20230720093833153 DGGSN <0002> ../../../../src/osmo-ggsn/ggsn/ggsn.c:186 APN(internet): Starting 20230720093833154 DGGSN <0002> ../../../../src/osmo-ggsn/ggsn/ggsn.c:204 APN(internet): Opening Kernel GTP device tun4 20230720093833155 DGGSN <0002> ../../../../src/osmo-ggsn/ggsn/ggsn.c:210 APN(internet): Skipping APN start 20230720093833155 DGGSN <0002> ../../../../src/osmo-ggsn/ggsn/ggsn.c:186 APN(inet6): Starting 20230720093833156 DGGSN <0002> ../../../../src/osmo-ggsn/ggsn/ggsn.c:204 APN(inet6): Opening Kernel GTP device tun6 20230720093833157 DGGSN <0002> ../../../../src/osmo-ggsn/ggsn/ggsn.c:210 APN(inet6): Skipping APN start 20230720093833157 DGGSN <0002> ../../../../src/osmo-ggsn/ggsn/ggsn.c:186 APN(inet46): Starting 20230720093833158 DGGSN <0002> ../../../../src/osmo-ggsn/ggsn/ggsn.c:204 APN(inet46): Opening Kernel GTP device tun46 20230720093833159 DGGSN <0002> ../../../../src/osmo-ggsn/ggsn/ggsn.c:210 APN(inet46): Skipping APN start 20230720093833159 DGGSN <0002> ../../../../src/osmo-ggsn/ggsn/ggsn.c:817 GGSN(ggsn0): Starting GGSN 20230720093833160 DLGTP <000d> ../../../../src/osmo-ggsn/gtp/gsn.c:431 GTP: gtp_newgsn() started at 172.18.136.201 20230720093833161 DLGTP <000d> ../../../../src/osmo-ggsn/gtp/gsn.c:386 State information file (/tmp/gsn_restart) not found. Creating new file. 20230720093833162 DGGSN <0002> ../../../../src/osmo-ggsn/ggsn/ggsn.c:853 GGSN(ggsn0): Successfully started 20230720093833163 DGGSN <0002> ../../../../src/osmo-ggsn/ggsn/ggsn.c:186 APN(internet): Starting 20230720093833163 DGGSN <0002> ../../../../src/osmo-ggsn/ggsn/ggsn.c:204 APN(internet): Opening Kernel GTP device tun4 20230720093833164 DGGSN <0002> ../../../../src/osmo-ggsn/lib/gtp-kernel.c:79 Initialized GTP kernel mode (genl ID is 31) 20230720093833166 DTUN <0001> ../../../../src/osmo-ggsn/lib/tun.c:217 GTP kernel configured 20230720093833167 DGGSN <0002> ../../../../src/osmo-ggsn/ggsn/ggsn.c:230 APN(internet): Setting tun IP address 176.16.16.0/20 20230720093833168 DGGSN <0002> ../../../../src/osmo-ggsn/ggsn/ggsn.c:242 APN(internet): Setting tun IPv6 address 2001:780:44:2000::/56 20230720093833168 DGGSN <0002> ../../../../src/osmo-ggsn/ggsn/ggsn.c:288 APN(internet): Creating IPv4 pool 176.16.16.0/20 20230720093833169 DGGSN <0002> ../../../../src/osmo-ggsn/ggsn/ggsn.c:167 APN(internet): Blacklist tun IP 176.16.16.0/20 20230720093833170 DGGSN <0002> ../../../../src/osmo-ggsn/ggsn/ggsn.c:305 APN(internet): Creating IPv6 pool 2001:780:44:2000::/56 20230720093833171 DGGSN <0002> ../../../../src/osmo-ggsn/ggsn/ggsn.c:167 APN(internet): Blacklist tun IP 2001:780:44:2000::/56 20230720093833172 DGGSN <0002> ../../../../src/osmo-ggsn/ggsn/ggsn.c:320 APN(internet): Successfully started 20230720093833173 DGGSN <0002> ../../../../src/osmo-ggsn/ggsn/ggsn.c:186 APN(inet6): Starting 20230720093833173 DGGSN <0002> ../../../../src/osmo-ggsn/ggsn/ggsn.c:204 APN(inet6): Opening Kernel GTP device tun6 20230720093833174 DTUN <0001> ../../../../src/osmo-ggsn/lib/tun.c:213 cannot create GTP tunnel device: Device or resource busy 20230720093833175 DGGSN <0002> ../../../../src/osmo-ggsn/ggsn/ggsn.c:215 APN(inet6): Failed to configure Kernel GTP device 20230720093833176 DGGSN <0002> ../../../../src/osmo-ggsn/ggsn/ggsn.c:186 APN(inet46): Starting 20230720093833177 DGGSN <0002> ../../../../src/osmo-ggsn/ggsn/ggsn.c:204 APN(inet46): Opening Kernel GTP device tun46 20230720093833178 DTUN <0001> ../../../../src/osmo-ggsn/lib/tun.c:213 cannot create GTP tunnel device: Device or resource busy 20230720093833178 DGGSN <0002> ../../../../src/osmo-ggsn/ggsn/ggsn.c:215 APN(inet46): Failed to configure Kernel GTP device
Updated by laforge 10 months ago
On Thu, Jul 20, 2023 at 01:35:30PM +0000, osmith wrote:
When creating the second APN:
> tun.c:213 cannot create GTP tunnel device: Device or resource busy >
that should only happen if the gtp device is actually busy.
As I understand it, the problem is that we pass the same gsn->fd0 and gsn->fd10 to gtp_kernel_create in lib/tun.c:tun_new().
Ah. Ok, that explains. The way how the kernel GTP driver works is that for each UDP socket there can only be one gtp-net-device. After all, the kernel and all of GTP-U know nothing about APNs. All they know is packets arriving, and TEID-to-IP address mappings.
This means, indeed, for every UDP socket, there can only be one APN configured. However, multiple different
UDP sockets can serve multiple APNs. The way how GTP-C vs GTP-U works, one can actually hand out a different GTP-U endpoint/socket/port for each PDP context that is being activated. So osmo-ggsn could create one different GTP-U socket for each APN, and then hand out the respective GTP-U IP address during PDP context activate.
- document the current constraint in the osmo-ggsn user manual (explaining it's not really a kernel GTP
restriction but one of osmo-ggsn) - create an osmo-ggsn issue so we keep track of this constraint and hopefully fix it eventually.
Besides the problem with multiple APNs, I realized that I have been testing without the libgtpnl changes, and that these will need further changes in osmo-ggsn. I'm working on it and have set the patches to WIP.
yes, that part is the actual IPv6 support for kernel-gtp mode in osmo-ggsn. :)
Updated by osmith 10 months ago
laforge wrote in #note-9:
As the above is not within the scope of this ticket, please
- document the current constraint in the osmo-ggsn user manual (explaining it's not really a kernel GTP
restriction but one of osmo-ggsn)
-> https://gerrit.osmocom.org/c/osmo-ggsn/+/33865
- create an osmo-ggsn issue so we keep track of this constraint and hopefully fix it eventually.
-> #6106
Updated by osmith about 1 month ago
- % Done changed from 50 to 80
- osmo-ggsn has been adjusted, patches are in master
- libgtpnl has been adjusted, patches are in master
- kernel has been adjusted (#1952), and patches are being upstreamed
Temporary TTCN3 testsuite job with the kernel patches:
https://jenkins.osmocom.org/jenkins/job/ttcn3-ggsn-test-kernel-pablo-gtp/
