Project

General

Profile

Actions

Feature #2537

closed

package wireshark with osmocom patches

Added by laforge over 6 years ago. Updated 2 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Target version:
-
Start date:
10/05/2017
Due date:
% Done:

100%

Spec Reference:

Description

We should have some jenkins-ci job that
  • packages our wireshark as packages in OBS as part of the feeds
  • possibly even re-bases our non-mainline patches before each build

This is particularly important for the AMR voice playback, as wireshark has refused to include this upstream. We can also include any other not-yet-mainline code that we have at osmocom, like osmux statistics, Jacob's EGPRS patch, Ericsson RSL support, ...

It would probably be best if both the package as well as the file names are modified to not conflict with the distribution-standard wireshark package that mostly everyone has installed. Let's hope this is not too hard to do. Alternatively we could install into something like an /opt/osmocom PREFIX. A bit ugly, but then it would avoid having to patch the wireshark source?

Please coordinate with Daniel on this.

Actions #1

Updated by pespin about 6 years ago

Daniel, can you add some details/links here on the patches which should be included in here? Let's first make a list and then we can continue with it.

If anyone else wants any related patch to be included, please share it here.

Actions #2

Updated by pespin almost 6 years ago

  • Status changed from New to Feedback
  • Assignee changed from pespin to daniel

Assigning to Daniel so he can give me feedback on where can I find the different interesting patches.

Actions #3

Updated by pespin almost 6 years ago

  • Assignee changed from daniel to laforge

AMR decode support: https://git.osmocom.org/wireshark/commit/?h=daniel/osmux&id=0ac2c285e965d7ae104f857c829c3978cdd7b83c

Other patches from daniel/osmux should be already merged in upstream.

There seem to be other branches in our wireshark repo at osmocom.org:

laforge/gsup    Dissector for the Osmocom GSUP Protocol    Harald Welte    3 months
laforge/pending    Add dissector for OsmoTRX protocol    Harald Welte    2 weeks
laforge/qcdiag    qcdiag_log: Implement RRC dissection    Harald Welte    15 months
laforge/trx    Add dissector for OsmoTRX protocol    Harald Welte    5 weeks

What's the status of these branches? Are they merged upstream? Should I copy all those patches into the same branch and use the branch as source to build our own wireshark?

Assigning to laforge as the branches are from him.

Actions #4

Updated by laforge almost 6 years ago

  • Assignee changed from laforge to pespin
  • gsup has just been merged upstream
  • trx is about to get merged upstream
  • qcdiag is nowhere near
  • for "pending" you will have to rebase/review. I think at least the EGPRS patch as well as the "hand RLC payload to LLC" are not upstream.
Actions #5

Updated by pespin almost 6 years ago

  • Status changed from Feedback to In Progress

I started adding commits to osmocom/master branch.

gsup: Already in master, we get it for free.
trx: Available in laforge/trx and in wireshark's gerrit, still not merged.
qcdiag: Several commits in laforge/qcdiag. I didn't look at those yet.
pendign: In laforge/pendign. Several L2TP, GPRS-RLC and EGPRS patches.
AMR: Original commit in daniel/osmux. During rebase I had to fix some small bits and add all the CMake build bits since autoconf was dropped in master. New branch can be found in pespin/amr and is already merged in osmocom/master.

I also started work on building wireshark in OBS: https://build.opensuse.org/package/show/home:pespin:branches:network:osmocom:nightly/wireshark#
I had to modify debian/control to depend on libgnutsl28-dev instead of libgnutls-dev. Debian package also has this patch. See 7580784d4bc98544684607c70c6d490c0f9316cc in our wireshark repo in osmocom/master branch.

I also had to modify the Project Config to tell OBS which package to use as preferred, otherwise it won't build:

Prefer: libjpeg-dev

Actions #6

Updated by pespin over 5 years ago

  • Status changed from In Progress to Stalled

Set to stalled since I have this task paused for a while due to having other more urgent tasks.

Actions #7

Updated by laforge about 4 years ago

  • Status changed from Stalled to Feedback

this sounded like it was already rather complete before it was stalled. Can you estimate the amount of time required to finish it?

Actions #8

Updated by pespin about 4 years ago

I think during last year the building system (cmake->ninja) and dependencies (dropped gtk, neq qt afair) have changed, I'd need to check again.

Actions #9

Updated by pespin over 2 years ago

  • Status changed from Feedback to Stalled
Actions #10

Updated by osmith over 2 years ago

  • Assignee changed from pespin to osmith
Actions #11

Updated by osmith over 2 years ago

  • Status changed from Stalled to In Progress
  • % Done changed from 0 to 10

It would probably be best if both the package as well as the file names are modified to not conflict with the distribution-standard wireshark package that mostly everyone has installed. Let's hope this is not too hard to do. Alternatively we could install into something like an /opt/osmocom PREFIX. A bit ugly, but then it would avoid having to patch the wireshark source?

There's a set(_project_name wireshark line in CMakeLists.txt. I've changed it to osmoshark in a local build to test what it does and found that it changes some of the paths from wireshark to osmoshark, but not all of them (e.g. /usr/share/wireshark). Also the binaries and libwireshark.so are not renamed, so this doesn't resolve conflicts with distributions packaging.

So installing to /opt seems to be the best solution to avoid conflicts. Passing -DCMAKE_INSTALL_PREFIX=/opt/wireshark-osmo/ to cmake does the trick.

I'll add wireshark-osmo, tshark-osmo etc. wrappers that point to the binaries in /opt, so it is still convenient to run the custom wireshark build.

Actions #12

Updated by osmith over 2 years ago

  • Status changed from In Progress to Stalled

I'm working on other issues right now, but for future reference:

I had built the daniel/osmux branch (2016) with AMR support to test if I had a good AMR sample that I could play and test with a rebased version. With that version wireshark was able to decode the file (showed the waveform, which does not show in wireshark without these patches) but audio playback didn't work. (Looks like it didn't work in general in that build, maybe I would have needed more build flags for that.)

However with the pespin/amr branch (2018), opening the same sample causes the wireshark gui to freeze. So there's a regression from the rebase that needs to be fixed.

Also the patches need to be rebased again on a recent version. If I understood it correctly, we could also refactor the patches to use the plugin api so we don't have problems with further rebases (or at least less problems).

Actions #13

Updated by laforge 12 months ago

rebased AMR patch now in https://gitea.osmocom.org/osmocom/wireshark/src/branch/laforge/amr-2023 - so far just build tested, not playback-tested.

Actions #14

Updated by laforge 12 months ago

The decoder/playback for AMR in RTP works fine using the rebased patch/branch (used the file from #5944).

I did some research on the patent situation for AMR-NB and determined that all patents must have expired by now. Hence, I've done another attempt at merging it at https://gitlab.com/wireshark/wireshark/-/merge_requests/10025

Actions #15

Updated by laforge 12 months ago

osmith: In terms of packaging, a basis can be found at https://obs.osmocom.org/project/show/home:laforge:wireshark - I used the packaging from debian unstable as basis.

Later on I found out that wireshark itself also supports package building (ln -s packaging/debian debian && dpkg-buildpackage), which might have been simpler. For some strange reason the wireshark developers have .symbols files listing each and every symbol of their library in the debian packaging and don't appear to have CI to verify it, so build fails right now as the library and symbol list disagree.

In either case, we'd need a jenkins job that will automatically build packages from a given osmocom.org/wireshark git branch whenever we update it. We could then create an osmocom/all-in-one branch that contains rspro, amr, qcdiag and/or other patches we have floating around.

Actions #16

Updated by osmith 12 months ago

  • Status changed from Stalled to In Progress
Actions #17

Updated by osmith 12 months ago

  • % Done changed from 10 to 90
I've built it now as discussed in the weekly meeting:
  • A jenkins job that runs every night
  • Wireshark from upstream gets cloned
  • Several Osmocom specific branches get merged on top (this works without conflict)
  • A debian source package gets built
  • That package gets pushed to osmocom:wireshark

The Osmocom branches are defined in scripts/obs/update_obs_wireshark.sh in osmo-ci.git, as of writing:

BRANCHES=" 
    osmith/deb-packaging
    laforge/amr-2023
    laforge/doc-fixes
    osmocom/qcdiag
    laforge/rspro
" 

  • osmocom/qcdiag is the same as laforge/qcdiag, but without WIP: RSL Ericsson support since that caused tests to fail due to missing/conflicting IE values and msg types in the value strings... after spending some time on trying to fix it, I've left it out for now.
  • osmith/deb-packaging is the packaging from home:laforge:wireshark. I've also looked at packaging/debian from wireshark upstream, but as you mentioned it seems incomplete / not maintained with automatic tests at this point.

I've also revisited the idea of changing installation paths so it does not conflict with the main wireshark package. But since I already spent some effort on it and found it not to be trivial, I've also left this out for now. I could look into it again if it's really important. With the current state, we have an extra repository for wireshark so users will at least not install the Osmocom Wireshark version by accident. I've also added "~osmocom" at the end of the package version so it's easy to recognize.

Patches:
https://gerrit.osmocom.org/q/topic:osmocom-wireshark

Actions #18

Updated by osmith 12 months ago

  • Status changed from In Progress to Resolved
  • % Done changed from 90 to 100

Patches are merged. Added some documentation about the repository here:
https://osmocom.org/projects/cellular-infrastructure/wiki/Wireshark#Binary-packages-with-Osmocom-patches

Actions #19

Updated by osmith 2 months ago

I've also looked at packaging/debian from wireshark upstream, but as you mentioned it seems incomplete / not maintained with automatic tests at this point.

This has changed, there is automated testing for it now:
https://gitlab.com/wireshark/wireshark/-/blob/af408571e5d3f967fc67c77402aacdbfa1a849cd/.gitlab-ci.yml#L295

...and with the latest failures I've changed it to use the upstream packaging (with some adjustments on top):
https://gitea.osmocom.org/osmocom/wireshark/commits/branch/osmith/deb-packaging

Related: https://gerrit.osmocom.org/c/osmo-ci/+/35404

Actions

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 48.8 MB)