Project

General

Profile

Actions

Bug #2435

open

Osmocom asn1c/libasn1c is based on old fork

Added by laforge over 6 years ago. Updated about 1 year ago.

Status:
New
Priority:
Low
Assignee:
-
Target version:
-
Start date:
08/14/2017
Due date:
% Done:

0%

Spec Reference:

Description

We have a fork of asn1c/libasn1c that's from April 2015.

We should rebase our changes on current upstream master (which has picked up development speed again) and keep rebasing.


Related issues

Related to OsmoHNBGW - Feature #2436: Test cases for asn1c APER encodingNew08/14/2017

Actions
Related to OsmoHNBGW - Bug #2437: synchronize different asn1c APER forksNew08/14/2017

Actions
Related to OsmoHNBGW - Feature #3862: Add support for asn1sccRejected03/26/2019

Actions
Actions #1

Updated by laforge over 6 years ago

  • Related to Feature #2436: Test cases for asn1c APER encoding added
Actions #2

Updated by laforge over 6 years ago

  • Related to Bug #2437: synchronize different asn1c APER forks added
Actions #3

Updated by laforge about 6 years ago

FYI: I just used current asn1c master 4cc779fd9bd7f556699b5863cf111b359da10b66 (last commit November 21, 2017) to successfully compile HNBAP and RANAP from wireshark.gig:

asn1c -fcompound-names -fline-refs -funnamed-unions HNBAP-CommonDataTypes.asn  HNBAP-Constants.asn  HNBAP-Containers.asn  HNBAP-IEs.asn  HNBAP-PDU-Contents.asn  HNBAP-PDU-Descriptions.asn
asn1c -fcompound-names -fline-refs -funnamed-unions RANAP-CommonDataTypes.asn RANAP-Constants.asn RANAP-Containers.asn RANAP-IEs.asn RANAP-PDU-Contents.asn RANAP-PDU-Descriptions.asn
asn1c -fcompound-names -fline-refs -funnamed-unions RUA-CommonDataTypes.asn  RUA-Constants.asn  RUA-Containers.asn  RUA-IEs.asn  RUA-PDU-Contents.asn  RUA-PDU-Descriptions.asn

This means it is capable to parse the full extent of information object classes contained in the relevant 3GPP specs, without relying on any asn1tostruct.py hacks!

It still is missing the APER and type prefixing which we need, so those would have to be forward-ported, and our existing code written against the assumptions of asn1tostruct.py ported over.

Actions #4

Updated by neels about 6 years ago

note that there already is a branch aper-prefix-onto-upstream which I had rebased onto the upstream master at that time. Last time I checked it worked well (with our current asn1tostruct.py affairs). It is probably far behind the current master, but would be a good basis to avoid some of the merge conflicts already resolved there.

Also note that our current asn1c.git repository contains two completely separated histories. In my gitk --all it is visible by showing the aper-prefix-onto-upstream branch, with a new "Initial" commit right above that and a looong history towering on top of that; notice there is no line connecting the two, so it is a completely separate initial commit.

About all of this, I wrote:
http://lists.osmocom.org/pipermail/openbsc/2016-July/009488.html
"merging iu specific branches to master"
Wed Jul 6 16:58:40 UTC 2016

> >   asn1c: * aper-prefix

First of all, it looks like we imported from svn, but Lev Walkin later did
another migration to git, which we fetched as well. So our master is on our old
svn import, while the upstream master has different hashes in its history.  Our
aper-prefix branch is based on the upstream history, not our "stale" master,
which made a rebase a bit easier.

Furthermore, on https://github.com/vlm/asn1c/commits/master there are scores of
new commits we don't have in our asn1c.  our last commit from Lev Walkin is
from 2015-04-28, "?= was confusing some environments", 62913d8b8e1eb96d74315ff

I have thus:
* fetched upstream master from github's vlm/asn1c,
  pushed as new branch vlm/master to our git.osmocom.org/asn1c
* rebased our aper-prefix branch to that master (with minor conflicts),
  pushed as new branch aper-prefix-onto-upstream

We should probably:
* reset --hard our master to vlm/master
* reset --hard our aper-prefix to aper-prefix-onto-upstream (after testing)

The last of which hasn't happened yet but still makes sense to me today, followed by pulling in the updated current master and rebasing our (still required) changes onto that.

(I guess we also might want to get rid of that disjunct evil twin history in our asn1c.git somehow.)

Actions #5

Updated by msuraev almost 5 years ago

Actions #6

Updated by laforge over 4 years ago

  • Priority changed from Normal to Low
Actions #7

Updated by laforge about 4 years ago

  • Assignee deleted (4368)
Actions #8

Updated by pespin over 2 years ago

neels wrote:

Also note that our current asn1c.git repository contains two completely separated histories. In my gitk --all it is visible by showing the aper-prefix-onto-upstream branch, with a new "Initial" commit right above that and a looong history towering on top of that; notice there is no line connecting the two, so it is a completely separate initial commit.
(I guess we also might want to get rid of that disjunct evil twin history in our asn1c.git somehow.)

I also run into this. I'd start with cleaning this up. I found the same commit (with description: "attribution") in both the svn-to-git and the git separated lines:
svn-to-git: f5ab8ba08c82d6283fe35a9678b6bac4e1fe2337 (branch origin/git-svn)
git: 83f2a1cd8b088264b64a62b51733243281f590e3 (no branch name)

So created a branch named "pespin/master" on 83f2a1cd8b088264b64a62b51733243281f590e3, and then cherry-picked all commits we added starting from f5ab8ba08c82d6283fe35a9678b6bac4e1fe2337 up to "master".

So it's now a matter of:

git branch -D master
git push origin :master
git push origin :git-svn
git branch -m pespin/master master
git push -f origin master

neels agree?

Actions #9

Updated by neels over 2 years ago

neels agree?

oof I really don't know. Maybe laforge can confirm that the twin history is
not intentional?

When removing the twin is approved: I am not firm enough with git's internals
to confirm that those are the right commands... I hope you know what you're
doing?? Could be worth to test that e.g. buildbot clones don't break.

Actions #10

Updated by osmith over 2 years ago

neels wrote in #note-9:

When removing the twin is approved: I am not firm enough with git's internals
to confirm that those are the right commands... I hope you know what you're
doing?? Could be worth to test that e.g. buildbot clones don't break.

regarding git commands, I'd suggest to just force push master instead of deleting it first:

$ git checkout master # old master
$ git checkout -b name-of-legacy-branch
$ git push --set-upstream origin name-of-legacy-branch # push the old history here
$ git checkout master
$ git reset --hard pespin/master
$ git push --force

I'd assume that we can just clear workspaces of buildbots if they should complain.

Actions #11

Updated by osmith about 1 year ago

The master-osmo-asn1c job was failing with asn1p_y.y:357:13: error: 'param' undeclared, as we have seen it from time to time already. The 'fix' was to wipe the workspace.

This made me wonder what we are building in this job, and I looked at the branches again:
  • master:
    • based on a 2010 tree (with a separate history from upstream as mentioned above)
    • has aper patches on top
    • has a patch for contrib/jenkins.sh, this is what we have been building in jenkins
  • aper:
    • based on a 2015 tree from upstream
    • has aper patches on top
  • aper-prefix:
    • same as aper branch + an ASN1C_PREFIX patch and small fixes on top
    • this is what we have actually been using, this branch is required to be able to regenerate files for osmo-iuh.git
    • but this is not what we were building with jenkins...
  • aper-prefix-onto-upstream:
    • should be closer to upstream, but did not build for me
  • pespin/master:
    • it seems, here patches were taken from aper-prefix and actually backported to the much older 2010 tree on master

So all of this is very confusing, and it's not useful to build a branch in jenkins that we don't use at all, and not build the one that we do use.

I've done the following to make clear what code we are actually using:
  • Previous master is now in an "old-master" branch
  • "osmo-iuh/master" is now the same as "aper-prefix", but with a working contrib/jenkins.sh on top (force pushed)
  • I had to disable 'make check' in contrib/jenkins.sh as it didn't work in that branch, but still build testing a branch we use is more useful than build testing + running 'make check' in a branch we don't use at all.

=> now "osmo-iuh/master" is the tree that we can use for osmo-iuh, and what jenkins builds.

(gitea probably takes a bit to sync again, the repository can also be browsed on gerrit)

EDIT: Pau pointed out that for osmo-cbc another tree/branch is used: https://github.com/mouse07410/asn1c/tree/vlm_master/; but I still think it's good to have the master branch of our fork point to what we actually use for osmo-iuh, and the old master branch wasn't used in practice for anything from what I can tell.

EDIT: As discussed with Pau, using osmo-iuh/master for the proper osmo-iuh branch now, and master follows upstream.

Actions

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 48.8 MB)