Project

General

Profile

Actions

Feature #6036

open

Implement ternary SID classification for HR1 uplink

Added by falconia about 1 year ago. Updated 21 days ago.

Status:
Stalled
Priority:
Low
Assignee:
Category:
-
Target version:
-
Start date:
05/17/2023
Due date:
% Done:

10%

Spec Reference:
GSM 06.41

Description

The DTX specs for all 3 classic non-AMR codecs (FR1, HR1 and EFR) prescribe rather symmetric handling for uplink Rx in a BTS: for each of the 3 codecs, each received UL frame is ternary-classified as valid SID, invalid SID or non-SID speech. Furthermore, in the original E1-based architecture this classification operation (for all 3 codecs) always takes place in the BTS, immediately adjacent to the GSM 05.03 channel decoder, and the Abis output (TRAU frames) includes not only the frame payload bits, but also out-of-band control bits indicating BFI, UFI (for HR only) and this SID classification decision. There is a serious problem, however, in that these out-of-band indication bits have been dropped in the migration from E1 TRAU frames to RTP, and there are many different negative fallouts from this "simplification".

There is one further way how the problem of SID classification is more difficult for HR than for FR and EFR. In the case of both FR and EFR, the respective DTX specs (06.31 and 06.81) prescribe exact bit-counting rules whereby a received frame is declared to be a valid SID, an invalid SID or non-SID speech. Given these exact bit-counting rules, the missing C13 & C14 indication bits that have been lost in the transition from TRAU-UL frames to RTP can be trivially reconstructed on the RTP receiving end, as now implemented in osmo_{fr,efr}_sid_classify() functions in libosmocodec, and the output of these functions (or their equivalents in Themyscira libraries) can be used by all processes that need to receive an RTP stream that comes or may be coming from the uplink of a GSM call leg: network edge transcoders that need to feed this UL RTP stream to their Rx DTX handlers, and the RTP-to-DL path in OsmoBTS that needs to be prepared for the possibility of TrFO.

However, the DTX spec for HR (GSM 06.41) does NOT prescribe exact thresholds of how many non-1 bits in the SID field mark the transition from valid to invalid SID, and how many non-1 bits in this SID field mark the transition from invalid SID to non-SID speech classification. Instead this design decision is left to BTS implementors, fitting well into the architecture where this SID classification logic is closely coupled with the GSM 05.03 channel decoder. Furthermore, while the SID fields of both FR and EFR fall fully into class 1 bits (all subject to convolutional coding), the SID field of HR1 is split between class 1 and class 2 bits. A BTS uplink receiver making the SID classification decision based on how many non-1 bits occur in the SID field may very well give different weights to non-1 bit errors in class 1 vs class 2 bits, and the GSM 06.41 spec leaves the door open for BTS implementors to apply such more complex logics than a simple count of "wrong" bits as in FR and EFR SID classifiers.

So what are we going to do about this problem in Osmocom? Given the lack of "solid" SID classification rules that can be "legitimately" applied by RTP stream receivers, I propose that instead of replicating what we did for FR and EFR, we handle HR1 differently:

  • Implement a SID classifier in OsmoBTS at the point of UL output, rather than at the point of DL input. For the sake of simplicity, our initial implementation of this SID classifier will look only at the hard-decision output bits from the 05.03 decoder, but we leave the door open for the possibility of tighter coupling between this classifier and the channel decoder.
  • If the HR1 RTP output format from OsmoBTS is RFC 5993, the ToC octet provides the proper place to indicate the SID classification result. If this RTP output format is instead vty-configured to be TS 101 318 (which should be the non-recommended option given this SID classification problem), valid SID can be indicated by "rejuvenating" the SID frame (resetting all 79 SID field bits to 1), while invalid SID frames will have to be simply dropped from the output - treated like BFIs.
  • At the point of RTP input for DL in OsmoBTS, if the RTP format is RFC 5993 (preferred), take the SID classification status from the ToC octet and use this status for TFO-like logic, i.e., for the logic of #5996 that mimics parts of TS 28.062 section C.3.2.1.1. If the RTP format is TS 101 318 (non-preferred), apply the current osmo_hr_check_sid() function, i.e., treat the frame as SID iff all 79 bits of SID field are ones.

Related issues

Related to OsmoBTS - Bug #5996: DTXd and SID logic is broken for non-AMR codecsResolvedfalconia04/06/2023

Actions
Related to OsmoBTS - Bug #5688: osmo-bts-sysmo and osmo-bts-trx use different GSM-HR RTP packet formatResolveddexter09/20/2022

Actions
Related to Cellular Network Infrastructure - Feature #6448: Support Themyscira RTP extensions in Osmocom BSSResolvedfalconia05/08/2024

Actions
Actions #1

Updated by falconia about 1 year ago

  • Related to Bug #5996: DTXd and SID logic is broken for non-AMR codecs added
Actions #2

Updated by falconia about 1 year ago

  • Related to Bug #5688: osmo-bts-sysmo and osmo-bts-trx use different GSM-HR RTP packet format added
Actions #3

Updated by falconia 28 days ago

When I created this ticket a year ago, I thought along the lines of extending the FT field in the ToC octet of RFC 5993 to indicate all 3 possible SID classification states of GSM 06.41. This idea has now been formalized in a proper spec:

https://www.freecalypso.org/specs/tw-ts-002-v010100.txt

However, before this spec can be properly implemented in OsmoBTS, we still need a SID classifier function for TCH/HS Rx, producing one of 3 possible states based on the output from the channel decoder - and the problem of having no prescribed rules to go by still remains. My idea is to see what others have done:

  • Look at the SID status bits produced by TI Calypso DSP given different crafted TCH DL inputs, and hopefully reverse-eng the SID classification logic implemented in this GSM MS DSP;
  • Do similar behavioral reverse eng on at least one historical E1-based BTS: feed different crafted inputs via Um interface, and see what SID classification the BTS puts out in TRAU-UL frames.

For the Calypso experiment, I can do it myself, but I need a little more time: it will be much easier for me to perform these experiments after I make some planned improvements to Themyscira CN software for test calls. For the E1 BTS experiment, I will need help from other members of Osmocom community, as I don't have any such hardware.

Actions #4

Updated by falconia 28 days ago

  • Related to Feature #6448: Support Themyscira RTP extensions in Osmocom BSS added
Actions #5

Updated by falconia 22 days ago

  • Status changed from New to In Progress
  • Assignee set to falconia

In 2024-06-05 RetroNetCall, laforge offered to send me a Nokia InSite E1-based BTS. If and when I receive that hardware, I will be able to complete this ticket myself, without needing work involvement from other community members. I assume, however, that improved support for HRv1 codec is probably a low-priority item for Osmocom, and it is likewise a low-priority item for Themyscira Wireless. Therefore, I am not expecting any particular urgency, neither in getting the BTS from Harald nor in the work I will need to do upon receiving it, unless someone tells me otherwise.

In the meanwhile, I am going to perform experiments on my end to see how TI Calypso DSP performs HRv1 SID classification on TCH/HS downlink Rx.

Actions #6

Updated by falconia 21 days ago

  • Status changed from In Progress to Stalled
  • % Done changed from 0 to 10

I performed an experiment in which I fed bit-corrupted SID frames to TCH/HS DL and looked at the TCH DL tap capture on a Calypso GSM MS. Result: at least under these very constrained conditions (perfect radio Rx, zero bit errors at the convolutional decoder level, good GSM 05.03 CRC-3, only the payload bit content has been intentionally corrupted), the logic performed by TI Calypso DSP matches the BCI=0 code path in the swSidDetection() function in the REID utility in GSM 06.06 code drop - a part of the code that is officially classified as an example and not normative. However, my test was very incomplete: because I used PHY-based sysmoBTS and not osmo-bts-trx as the TCH DL Tx, I had no ability to test and see what the Calypso DSP will do in the presence of bit errors at the convolutional decoder level, or if CRC-3 fails.

Thinking more about this problem, I am now of the mind that it would be saner to try to reverse-eng this logic in a historical E1 BTS before coming back to Calypso DSP. In the case of FR and EFR I had no problem figuring out the meaning of each bit in TI's a_dd_00 status word just by looking at captures, but unfortunately the HR version turned out to be much less obvious. The two SID classification bits are in the same place, but the meaning of channel decoder error flags is currently unknown (TI's version appears to have 3 of them!), and there is a high likelihood that these channel decoder error flags do interact with the SID detector. OTOH, if we were doing these experiments with an E1 BTS, we know exactly where BFI and UFI flag bits are in the TRAU-UL frame structure.

For these reasons, I am marking this ticket as Stalled until the following two future developments:

  1. I will need to receive the Nokia InSite BTS from Harald;
  2. I will need to get my UmTRX working (or acquire another SDR if this currently-untested UmTRX turns out to be dead), climb the SDR learning curve, and put together a test setup where I can use this SDR in the role of a GSM MS to send crafted UL to the InSite BTS, with intentionally induced convolutional decoding errors, with inverted CRC and whatever other hacks will be needed.
Actions #7

Updated by falconia 21 days ago

  • Priority changed from Normal to Low

Given how much work it will be to recover the elusive truth of HRv1 channel decoder error flagging and SID classification, and how much other work there is in Themyscira queendom, including work of potential value to Osmocom community, I am changing the priority of this ticket to Low.

Actions

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 48.8 MB)