Cellular Network Infrastructure

I prefer this approach to the solution we arrived in #4044 (enforce generating the documentation with each commit in jenkins' gerrit-verfication job), if it is feasible.

Hoernchen, you have explained in detail how the issue is going yesterday on the phone. Can you post a quick summary here too? Thanks!

We want to extract this from the code, so the only reasonable choice is Clang/LLVM, more specifically LibTooling, because LibClang offers a stable API, but accessing the AST through a C interface is cumbersome. Looking for the known struct names should suffice, there is also the possibility of attaching annotations to functions and variables to augment this approach.
It's also possible to modify all build scripts to support compiling and linking everything into one large bitcode blob, and the optimizing away all unused code in order to produce one large bitcode blob that contains all vty commands (including those provided by the used libs) that can be used, but I don't think that is necessary at this point.

Please update the ticket with a description of the latest status, and set the perceived percentage (and probably "stalled")

The current solution consists of two parts, step one is linking everything into one big bitcode file, step two is optimizing this to get rid of unused code, and then getting the debug information for occurences of vty command registration calls and so on, which are then used to look up and parse the corresponding code using a compilaton database file.

The issue with this approach is that most projects do not feature nice single make targets for the various parts like libs/tests/whatever due to the atrocious libtool and just adding a nice makefile rule that allows outputting a nice compilation database (per file) is not possible either, so one way to circumvent the latter issue is by using a unix domain socket as "filename", since writes up to one page size are atomic this should work with concurrent compiler (clang) invocations, the other is to embed the bitcode into all object files and extract them afterwards, and then hand-craft a bit of find magic to get only the files that we need without tests or multiple main functions and stuff like that to make the linking work.

It looks like this:

CC=clang-8 CFLAGS="-O0 -g -fembed-bitcode" ./configure
#liboso-netif
make clean
bear make -j

find . -iname "*.o" | xargs -I{} objcopy -O binary --only-section=.llvmbc {} {}.bc

#make clean # remove all *.o
cat compile_commands.json | jq '.[] | select( .directory | contains("/src") )' | jq ' .arguments[0] |= "clang-8 -emit-llvm"' | jq -r '.arguments |= join(" ") |  "cd \(.directory) && \(.arguments)"' | xargs -d '\n' -n1 -P4 sh -c
find . -iname "*.o" -print0 |xargs -0 -n1 file | grep LLVM |cut -d: -f1 | grep \.libs | xargs llvm-link-8 -o all.bc

So the actual issue is not so much extracting the information, but getting a nice compilation database per project/a sane subset of bitcode files without having to resort to brittle hand-crafted lists of grep and find magic for all projects that need to be updated all the time, due to the way the arcane build system works.

I think we can probably give up on this approach, as it has not materialized fully so far. We'll continue to rely on starting the program and introspecting it, I suppose.