Bug #6455
open
dumpcap failing to access /data inside docker
Added by osmith about 1 month ago.
Updated about 1 month ago.
Description
From https://gerrit.osmocom.org/c/docker-playground/+/36836:
if a ttcn3 container was started in jenkins.sh with
--cap-add=NET_ADMIN, then osmo-ttcn3-hacks.git/ttcn3-tcpdump-start.sh
would select dumpcap instead of tcpdump, to attempt to record a pcap
file to /data/$testcase.pcap.
Due to some unknown way of dropping privileges done by dumpcap though,
it was unable to open the /data/$testcase.pcap due to /data being owned
to another user (tcpdump doesn't have this problem).
The patch that was supposed to fix this causes other problems, so I'm reverting it now:
https://gerrit.osmocom.org/c/docker-playground/+/36858
Let's discuss how to fix it instead here.
pespin: how about we change ttcn3-tcpdump-start.sh to just always use tcpdump instead?
- Description updated (diff)
As discussed:
- We could re-apply Pau's patch, if we make sure that after each testsuite run, we fix the permissions again. The osmo-ggsn and osmo-bts testsuites broke, because we run the testsuites multiple times with multiple configurations there, and the subnets in the configs need to be replaced before each testsuite run, at which point the files must have permissions of the outside-of-docker user. This is probably the simplest solution.
- Would be nice to replace the implicit logic for choosing dumpcap/tcpdump with explicitly setting which one to use.
I also had the idea of making sure that we always have the same uid inside and outside of docker, but this is more effort and probably not worth doing now.
Some considerations/tests I did:
If I map the user ID "docker run --user $(id -u)", then:
1. I cannot run "ip xfrm" inside the docker container because I get "Operation not permitted". I guess I'd need to use setcap on the ip binary in Dockerfile beforehand.
2. ttcn3-tcpdump-start.sh tries to use tcpdump, but it fails because sudo is not installed. Without sudo it also fails because it cannot listen on "-i any".
So moving all ttcn3 testsuite docker containers to use non-root user, despite desirable, may take some work to fix permissions here and there.
osmith I fixed the originating issue so far by skip using dumpcap if the script detects it would fail to create the .pcap file:
https://gerrit.osmocom.org/c/osmo-ttcn3-hacks/+/36867 ttcn3-tcpdump-start.sh: Avoid using dumpcap if it has no access to pcap
I think in the long term we should move towards an architecture where we add "setcap" to required binaries inside docker container when in Dockerfile, and then run with a non-root user during "docker run". Even some steps like compiling ttcn3 could be done without root user.
On Fri, May 17, 2024 at 09:24:58AM +0000, osmith wrote:
pespin: how about we change ttcn3-tcpdump-start.sh to just always use tcpdump instead?
I don't know the specific use case in all its details, but dumpcap is usually superior to
tcpdump. For example, you can simultaneously capture traffic of several interfaces (by multiple "-i ..."
arguments) without going straight away for "any".
Also available in: Atom
PDF
