Project

General

Profile

Feature #3385

Move project specific manuals from osmo-gsm-manuals to each respective git repository

Added by pespin 4 months ago. Updated about 8 hours ago.

Status:
In Progress
Priority:
Normal
Assignee:
Target version:
-
Start date:
07/06/2018
Due date:
% Done:

70%

Spec Reference:

Description

Following subdirs in osmo-gsm-manuals.git should be moved under doc subdir of each respective git repository (osmo-bsc.git, osmo-bts.git, etc.)

OsmoBSC
OsmoBTS
OsmocomBB
OsmoGGSN
OsmoGSMTester
OsmoHLR
OsmoMGCP
OsmoMGW
OsmoMSC
OsmoNAT
OsmoNITB
OsmoPCU
OsmoSGSN
OsmoSTP
OsmoTRX

The idea is to keep the common parts and build related scripts (makefile, etc) in osmo-gsm-manuals.git, and then conditionally enable building them by using a configure flag (eg. --enable-man --with-man-path=/path/to/osmo-gsm-manuals.git/).

We can have most of the Makefile logic in a .mk file in osmo-gsm-manuals.git and include it in every repository, similar to what we do with osmo-release.mk from libosmocore.git:
In configure.ac:

dnl include release helper
RELMAKE='-include osmo-release.mk'
AC_SUBST([RELMAKE])

This way we can have user manual and VTY documentation up to date and sync with code, and force to have documentation changes together with code changes in the same gerrit patch before +2 it.

This will also allow to easily generate man pages from a specific .adoc file (which is also included in user manual).


Related issues

Related to OsmoBSC - Feature #3583: OsmoBSC manual: copy/move bts-examples chapter from OsmoNITBNew2018-09-20

Precedes Cellular Network Infrastructure - Feature #3386: Generate man pages at build time from adoc filesNew2018-07-092018-07-09

History

#1 Updated by pespin 4 months ago

  • Precedes Feature #3386: Generate man pages at build time from adoc files added

#2 Updated by neels about 2 months ago

  • Related to Feature #3583: OsmoBSC manual: copy/move bts-examples chapter from OsmoNITB added

#3 Updated by osmith 14 days ago

  • Status changed from New to In Progress
  • Assignee changed from sysmocom to osmith

Assigning this to myself, because another issue that is assigned to me (#3386) depends on this one.

#4 Updated by osmith 14 days ago

The idea is to keep the common parts and build related scripts (makefile, etc) in osmo-gsm-manuals.git, and then conditionally enable building them by using a configure flag (eg. --enable-man --with-man-path=/path/to/osmo-gsm-manuals.git/).

pespin: I wonder if that works properly with packaging the man pages for Debian later on. We could also extend osmo-gsm-manuals.git with a "make install" target and write a .pc file, that points to the shared code (pkg-config allows custom variables). Then we could create a "osmo-gsm-manuals-dev" package as build time dependency for the other osmo packages. What do you think?

#5 Updated by pespin 14 days ago

Fine for me. Keep in mind that it may be interesting to have some Makefile installed by "make install" in osmo-gsm-manuals.git and reachable through pkg-config custom variable which can then be included as described in initial post. This way we avoid copy&paste same stuff on how to build adoc stuff into each project. Up to you to find best solution.

#6 Updated by osmith 13 days ago

I have researched how this issue relates to the Jenkins jobs.

Right now we have:
  • push to master:
    • osmo-gsm-manuals.git - jenkins job "master-osmo-gsm-manuals" (from master-builds.yml)
      • calls ./contrib/jenkins.sh --publish
      • builds the manuals for all projects
      • runs rsync to publish the "./out" folder
  • push to gerrit:
    • osmo-gsm-manuals.git - jenkins job "gerrit-osmo-gsm-manuals" (from gerrit-verifications.yml)
      • calls ./contrib/jenkins.sh
      • builds the manuals for all projects
After moving the manuals to the project folders, I think we need:
  • push to master:
    • osmo-gsm-manuals.git
      • build and publish the documentation of all projects
    • osmo-*.git
      • build and publish the documentation of that project
  • push to gerrit:
    • osmo-gsm-manuals.git
      • build the documentation of all projects
    • osmo-*.git
      • build the repo with its documentation

#7 Updated by laforge 13 days ago

On Tue, Oct 30, 2018 at 03:39:46PM +0000, osmith [REDMINE] wrote:

After moving the manuals to the project folders, I think we need:
  • push to master:
    • osmo-gsm-manuals.git
      • build and publish the documentation of all projects

that would be rather nice, but I would guess we could live without any
automatism here.

  • osmo-*.git
    • build and publish the documentation of that project

this part is mandatory.

I guess with the lines below you're referring to the gerrit build verification?

  • push to gerrit:
    • osmo-gsm-manuals.git
      • build the documentation of all projects
    • osmo-*.git
      • build the repo with its documentation

I guess in this case it would be sufficient to just verify for osmo-*.git when a commit happens
to that specific repo. I don't think it's strictly required to built-test osmo-*.git after any
change to osmo-gsm-manuals.git. I cannot really think of anything being committed to the 'common'
part that would break the builds right now (short of removing a file that others need), so we can
probably ignore that as a rather esoteric case.

In terms of the generated/published manuals, this new scheme woould also allow us
to build + publish manuals for specific versions, particularly the tagged versions.

So basically once we tag a given version of osmo-msc (e.g. 1.2.3) , it
should be possible to build the manuals for exactly that version of the
osmo-msc user manual. Ideally that would run automatically, but of
course it is also possible to trigger this manually.

One interesting questions is what kind of "common" part to use in that
case. Some pinned commit hash of osmo-gsm-manuals.git that's stored in osmo-msc.git
repository at the 1.2.3 tag? Or simply the masteer of
osmo-gsm-manuals.git?

Regards,
Harald

#8 Updated by osmith 6 days ago

  • % Done changed from 0 to 30

Thanks for the feedback!

Time for a status update. I've pushed my WIP code to osmith/move-manuals-to-project-repos branches in both osmo-msc.git and osmo-gsm-manuals.git. My plan is to make it work with MSC first (as proof of concept), then with all other projects.

  • moved osmo-gsm-manuals.git/OsmoMSC to osmo-msc.git/doc/man
  • osmo-gsm-manuals.git installs the common files to $prefix/share/osmo-gsm-manuals (e.g. /usr/local/share/osmo-gsm-manuals):
    $ autoreconf -fi
    $ ./configure
    $ make
    $ make install
    
  • a pkg-config file gets generated, with a custom "topdir" variable that points to $prefix/share/osmo-gsm-manuals
  • the new topdir variable is queried from pkg-config and used by the project's manual build Makefile.am instead of the old "TOPDIR := .."
  • "../build", "../common" was hardcoded in a lot of places, this was changed to use that variable where possible (asciidoc), or adjusted to use a symlink to $topdir/common (doxygen)
  • osmo-msc.git's pdf is building now (needs more polishing though, images are probably missing etc.)
  • osmo-msc.git builds the pdf only when --enable-man is specified at the configure line
  • the check for installed dependencies was moved from osmo-gsm-manuals.git's global Makefile to "osmo-gsm-manuals-check-depends" installed to $prefix/bin. It gets called inside osmo-msc.git's configure when --enable-man is set.
  • I have moved libosmocore.git/doc/vty/merge_doc.xsl to osmo-gsm-manuals.git/merge_doc.xsl. It was not used anywhere in libosmocore, but it is needed for building the manuals.

Also I've noticed that GIT_VERSION and GIT_DATE are problematic here:
https://git.osmocom.org/osmo-gsm-manuals/tree/build/Makefile.asciidoc.inc

GIT_VERSION := $(shell git describe --abbrev=4 --dirty --always --tags)
GIT_DATE := $(shell $(TOPDIR)/build/unix-time-to-fmt.py `git log -n 1 "--pretty=%at" ../.`)

This won't work if ./configure is running in a source folder extracted from a release tarball.

#9 Updated by osmith 4 days ago

  • % Done changed from 30 to 50
update:
  • all common pages from osmo-gsm-manuals.git can be built as standalone test now (to see if there are any syntax errors etc. in the common pages, and to see if the common build scripts are working)
  • made --enable-man work with out-of-tree builds
  • unix-time-to-fmt.py: use "unknown" if we're not in a git directory
  • both pdfs for msc get generated now, though there's content missing in the vty reference

#10 Updated by osmith 3 days ago

  • % Done changed from 50 to 60
update:
  • fixed missing pages in the vty-reference pdfs
  • make checks from asciidoc Makefile work again
  • update Makefile.*.inc text comments
  • remove --publish from contrib/jenkins.sh, add a "publish" target to the common Makefile that gets included in each project

And I've thought about creating a "osmo-gsm-manuals-dev" Debian package as part of this issue. But there doesn't seem to be any benefit to that until we can build the manuals as man pages (#3386). So instead of blowing up the patchset further with a new "debian"-folder, let's do that as part of the man pages issue.

I'll clean up the code, then submit the patches for review on Monday :)

#11 Updated by osmith about 8 hours ago

  • % Done changed from 60 to 70
Patches submitted:

The manuals are only added to MSC so far. When this is reviewed, I can add them to the other project repositories as well.

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 48.8 MB)