Project

General

Profile

Bug #1581

IP payload compression

Added by laforge over 2 years ago. Updated about 2 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
-
Target version:
-
Start date:
02/23/2016
Due date:
% Done:

100%

Spec Reference:

Description

3GPP specifies optional IP payload compression that can be negotiated. We should offer it at least to those phones that support it in order to save scarece air-inteface bandwidth.

main.c main.c 4.13 KB dexter, 08/01/2016 04:45 PM
v42bis.c v42bis.c 25.7 KB dexter, 08/01/2016 04:45 PM
v42bis.h v42bis.h 8.34 KB dexter, 08/01/2016 04:45 PM

Related issues

Related to OsmoSGSN - Feature #1655: KPIs for OsmoSGSNStalled2016-03-11

History

#2 Updated by laforge over 2 years ago

  • Assignee set to spaar

#3 Updated by lynxis over 2 years ago

#4 Updated by laforge over 2 years ago

  • Assignee changed from spaar to msuraev

#5 Updated by laforge over 2 years ago

  • Priority changed from Normal to High

#6 Updated by laforge over 2 years ago

  • Assignee changed from msuraev to dexter
  • Priority changed from High to Normal

SNDCP spec refers to both V.42bis and V.44 as options.

A free software V.42bis implementation is part of iaxmodem (asterisk universe): Upstream source repo is still svn, at

#7 Updated by dexter about 2 years ago

Also looked a bit around, Freeswitch has an implementation too: https://github.com/jart/freeswitch/blob/master/libs/spandsp/src/v42bis.c

#8 Updated by dexter about 2 years ago

I have looked at the verious sources. It seems that all implementation come from the same origin (SPANDSP). I managed to separete the code from their spandsp-0.0.6pre21 package and it compiles without issues. However, I did not try it out yet.

In the .c file it says the following:

/* THIS IS A WORK IN PROGRESS. IT IS NOT FINISHED.
Currently it performs the core compression and decompression functions OK.
However, a number of the bells and whistles in V.42bis are incomplete. */

So its not complete yet. I am currently not aware which features are missing and if that is a problem for us.

#9 Updated by dexter about 2 years ago

  • Status changed from New to In Progress
  • % Done changed from 0 to 10

Status update: I updated gprs_sndcp_xid.c/h to also support generation of the parameter fields for v42bis and v44. So, these parameters can now be generated. Decoding is still missing.

#10 Updated by dexter about 2 years ago

The changes are on dexter/draft: http://git.osmocom.org/openbsc/?h=dexter/draft

#11 Updated by dexter about 2 years ago

  • % Done changed from 10 to 20

Status update: Now we also have the decoding functionality for v42bis and v44 SNDCP-XID fields

#12 Updated by dexter about 2 years ago

We now have a working v42bis implementation. Working means, I was able to compress and decompress a buffer with test-data in a small test program. It was not to easy to find a working version of SPANDSP's v42bis. There seem to be some broken versions around, the test rig that is included in the package also has a lot of problems. Missing test data, broken dependencies...

I have attached the test program and the modified v42bis.c/h files. In case someone is interested or needs it as standalone program.

#13 Updated by dexter about 2 years ago

After some longer time without update, I want to inform about the current status of V.42bis:

  • The SPANDSP V.42bis implementation has been ported to run inside the openbsc code, there is also a unit-test that tests the compression by generating some test data, compressing and decompressing it. If the original data and the decompressed data match up. The test passes.
  • With the infrastructure we have built for to turn on the RFC1144 header compression, it is very simple to turn on V.42bis as well. Since we are not able to decode the incoming packets it will quickly run into connection problems, but the compressed packets the phone sends make up a good testcase to enhance our unit-test.
  • While the V.42bis works fine with our generated test data, it is unable to decompress the packet we got from the experiment mentioned above. After some investigation, it turned out that the SPANDSP implementation behaves different from the ITU standard. The start conditions are different, and the escape character handling is also different. I have changed the ininital state and the Escape character handling, then It decodes the packet partially (only the uncompressed part at the beginning). I am currently unaware of what the problem is, but oviously there are different V.42bis dialects around. I am currently investigating on it. Finding the problem will be very difficult since V.42bis is a rather complicated algorithm and works with different word sizes (8 bit, 9 bit...) which makes debugging painfully. We will see. I am on it. Traces are very appreciated!
  • There is also some good news: 3GPP TS 44.065, states in 6.6.2.3 that when SN-UNITDATA is used (which we do) the compression state shall reset before any compression/decompression operation of an npdu (the compression is done over the entire npdu). This makes the implementation simple.

#14 Updated by dexter about 2 years ago

  • % Done changed from 30 to 60

After some investigation I found out that the V.42bis implementation I got from SPANDSP seems to be incomplete or wrong (wrong startup states, wrong escaping...) The author also mentions that "THIS IS A WORK IN PROGRESS. IT IS NOT FINISHED...". Since fixing turned out to be a very complex task I checked the Implementation I got against the implementation others are using. Finally I found a completed implementation at IAXmodem. I am glad to tell that we are now able to decode the compressed packets we sniffed while we turned on V.42bis experimentally.

#15 Updated by laforge about 2 years ago

On Thu, Aug 25, 2016 at 10:53:42AM +0000, dexter [REDMINE] wrote:

  • There is also some good news: 3GPP TS 44.065, states in 6.6.2.3 that
    when SN-UNITDATA is used (which we do) the compression state shall
    reset before any compression/decompression operation of an npdu (the
    compression is done over the entire npdu). This makes the
    implementation simple.

So this means compression is only done over one particular N-PDU, and
never extends into other PDUs? That's of course quite inefficient, but
understandable in the absence of reliable transmission.

--
- Harald Welte <> http://laforge.gnumonks.org/ ============================================================================
"Privacy in residential applications is a desirable marketing option."
(ETSI EN 300 175-7 Ch. A6)

#16 Updated by laforge about 2 years ago

Hi!
On Fri, Aug 26, 2016 at 04:19:52PM +0000, dexter [REDMINE] wrote:

Finally I found a completed implementation at IAXmodem. I am glad to
tell that we are now able to decode the compressed packets we sniffed
while we turned on V.42bis experimentally.

Great news!

--
- Harald Welte <> http://laforge.gnumonks.org/ ============================================================================
"Privacy in residential applications is a desirable marketing option."
(ETSI EN 300 175-7 Ch. A6)

#17 Updated by dexter about 2 years ago

  • % Done changed from 60 to 90

V.42bis is now integrated into the sgsn. While decoding incoming, compressed packets worked immediately, I had a hard time sending compressed packets. It seemed obvious that the V.42bis implementation was still faulty, so the debugging focussed there. Checking and testing various aspects of the V.42bis implementation revealed that compressor output was 100% correct. Since the packet shrinks during compression, the message buffer has to be realinged before moving on. This was done by setting msg->len to the new length. This turned out to be not enough, because this does not re-aling the tail pointer of the message buffer. After fixing this by using proper API functions the V.42bis immediately worked.

Summary:
  • V.42bis works for both directions!
  • VTY-Functionality to configure for V.42bis is added

#18 Updated by dexter about 2 years ago

  • Rebased changes to current origin/master
  • Optimized compression handling (no more memcpy() or talloc_zero_size() when compression is disabled)

#19 Updated by dexter about 2 years ago

  • Status changed from In Progress to Resolved
  • % Done changed from 90 to 100

#20 Updated by laforge about 2 years ago

  • Status changed from Resolved to Closed

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 48.8 MB)