Project

General

Profile

Bug #2978

OsmoBTS rxlev/rxqual SUB computation completely broken [AMR DTX]

Added by laforge over 2 years ago. Updated 23 days ago.

Status:
Stalled
Priority:
High
Assignee:
Category:
-
Target version:
-
Start date:
02/21/2018
Due date:
% Done:

80%

Spec Reference:

Description

In the osmo-bts codebase, I can only see readers of bts_ul_meas.is_sub, but not see any code that sets this:

src/common/measurement.c:               if (m->is_sub) {


Checklist

  • {RxQual,RxLev}-SUB for AMR DTX

Related issues

Related to OsmoBTS - Bug #2987: OsmoBTS RxQual/RxLev averaging broken if bursts are missignResolved02/23/2018

History

#1 Updated by laforge over 2 years ago

  • Status changed from New to In Progress
  • % Done changed from 0 to 70

#2 Updated by laforge over 2 years ago

  • Subject changed from OsmoBTS rxlev/rxqual SUB computation likely broken to OsmoBTS rxlev/rxqual SUB computation completely broken

#3 Updated by laforge over 2 years ago

  • Related to Bug #2987: OsmoBTS RxQual/RxLev averaging broken if bursts are missign added

#4 Updated by laforge over 2 years ago

  • Checklist item {RxQual,RxLev}-SUB for AMR DTX added
  • Status changed from In Progress to Stalled

#5 Updated by laforge over 2 years ago

  • Subject changed from OsmoBTS rxlev/rxqual SUB computation completely broken to OsmoBTS rxlev/rxqual SUB computation completely broken [AMR DTX]

#6 Updated by laforge about 2 years ago

  • Assignee changed from laforge to dexter

#7 Updated by dexter about 2 years ago

  • Status changed from Stalled to In Progress

measurement.c expects the lower layers to set the is_sub flag correctly. We need a small FSM that looks into the packets and switches between silent periods and non silent periods so that we know when to set the is_sub flag and when not. The SID_UPDATE frames should come in a fixed interval, so for those we know when to expect them.

While we tag the measurements in the lower layers we also must record some information about the silent periods. We need to keep a record of the exact number of SUB measurements we expected and this number must then be reported to measurement.c. If we don't do this we would have correctly tagged SUB measurements but we still wouldn't be able say anything about lost is-sub frames.

I need to try a few more things out. I now have a setup where AMR with DTX is turned on. I can see and hear that it is enabled. Next we need some code to look into each voice frame (RTP) in order to detect the SID_FIRST and ONSET frames.

#8 Updated by dexter about 2 years ago

I have tried to detect when SID frames. The plan is (as described above) to make an FSM similar to dtx_dl_amr_fsm.c that observes the incoming RTP stream. For my experiment I have tapped the incoming frames directly in l1sap.c:l1sap_tch_ind(). Than I used osmo_amr_rtp_dec() to decode the frames. This gives me the frame type (ft) and in theory I should be able to check with osmo_amr_is_speech(ft) if I see a voice frame or a SID frame.

In my experiment this unfortunately did not work. I don't think that there is something wrong with the utilities. When I look at the trace I can see that there are no SID frames in it, only voice frames are visible. Also all the marks are missing. I think osmo-trx has some problems here, which we need to fix first.

When comparing to a trace from a sysmo-bts things look different. Here the marks are visible and there are also some comfort noise AMR-SID frames. However, I would expect to see some more SID frames. I miss ONSET and SID_FIRST frames. Presumably there are also some lower level problems here as well.

Attached one findes thw two traces I made

#9 Updated by dexter about 2 years ago

  • Status changed from In Progress to Stalled

#10 Updated by dexter about 2 years ago

I have set this to stalled. We will pick this up again when all remaining measurement and frame number calculation/scheduling problems are resolved.

#11 Updated by dexter 8 months ago

I now have a bit of an understanding what to expect but I am unable to get smart out of the frames we actually receive. A SID frame should be distributed over 4 sub frames, where one sub frame should be one MAC-Block. The spec says thet the RX SCR determines the frame type to determine what the frame actually is. (3GPP TS 26.093, Chapter E.2, Table 8)

in rx_tchf_fn() in the switchcase GSM48_CMODE_SPEECH_AMR, I get Macblocks like this:

2014e959f35fdfe5e9667ffbc088818088010100000101

02 is the header I also can see in Wireshark. 1 = 0001 ==> 000, which should mean "SPEECH_GOOD". I can not see any other frame, I would expect if there are SID frames that they are transmitted in four consecutive mac blocks and that they are marked with 101 = "POST1" (Correct SID update frame).

My Idea is to match on the SID frames on block level. I wonder if this is even possible like i Thought it out. I think no. Because the spec mentions a deframing unit which does the tagging. Any hints would be very much appreciated.

#12 Updated by dexter 8 months ago

  • Status changed from Stalled to In Progress

I came a bit further with this. I now have some code to detect the SID_FIRST frame and I am now working on the SID_UPDATE detection. The problem why I wasn't able to detect the marker patterns was because there is an interleaving in between. The docs aren't all too clear about this.

#13 Updated by dexter 8 months ago

  • % Done changed from 70 to 80

I now got my head around the interleaving. I have programmed some detector functions that can detect the various different types of SID frames by their identification pattern or by their coded inband data repetition sequence (depends on the sid frame type). We can now detect the following sid frames:

AFS_SID_FIRST
AFS_SID_UPDATE
AFS_ONSET
AHS_SID_UPDATE
AHS_SID_FIRST_P1
AHS_SID_FIRST_P2
AHS_ONSET
AHS_SID_FIRST_INH
AHS_SID_UPDATE_INH

AHS SID_FIRST_INH

(coded like an onset frame since it also occurs on sub-block 3 and 4)

xBxBxBxBxBxBxBxBxBxBxBxBxBxBxBxBx1x0x1x1x0x0x0x0x1x1x0x1x1x0x0x0x0x1x1x0x1x1x0x0x0x0x1x1x0x1x1x0x0x0x0x1x1x0x1x1x0x0x0x0x1x1x0x1x1x0x0x0x0x1x1x0x1x1x0x0x0x0x1x1x0x1x1x0x0x0x0x1x1x0x1x1x0x0x0x0x1x1x0x1x1x0x0x0x0x1x1x0x1x1x0x0x0x0x1x1x0x1x1x0x0x0x0x1x1x0x1x1x0x0x0x0x1x1x0x1x1x0x0x0x0x1x1x0x1x1x0x0x0x0x1x1x0x1x1x0x0x0x0x1x1x0x1x1x0x0x0x0x1x1x0x1x1x0x0x0x0x1x1x0x1x1x0x0x0x0x1x1x0x1x1x0x0x0x0x1x1x0x1x1x0x0x0x0x1x1x0x1x1x0x0x0x0x1x1x0x1x1x0x0x0x0x1x1x0x1x1x0

AHS SID_UPDATE_INH

(coded like an onset frame since it also occurs on sub-block 3 and 4)

xBxBxBxBxBxBxBxBxBxBxBxBxBxBxBxBx0x1x0x0x1x1x1x1x0x0x1x0x0x1x1x1x1x0x0x1x0x0x1x1x1x1x0x0x1x0x0x1x1x1x1x0x0x1x0x0x1x1x1x1x0x0x1x0x0x1x1x1x1x0x0x1x0x0x1x1x1x1x0x0x1x0x0x1x1x1x1x0x0x1x0x0x1x1x1x1x0x0x1x0x0x1x1x1x1x0x0x1x0x0x1x1x1x1x0x0x1x0x0x1x1x1x1x0x0x1x0x0x1x1x1x1x0x0x1x0x0x1x1x1x1x0x0x1x0x0x1x1x1x1x0x0x1x0x0x1x1x1x1x0x0x1x0x0x1x1x1x1x0x0x1x0x0x1x1x1x1x0x0x1x0x0x1x1x1x1x0x0x1x0x0x1x1x1x1x0x0x1x0x0x1x1x1x1x0x0x1x0x0x1x1x1x1x0x0x1x0x0x1x1x1x1x0x0x1x0x0x1

For the last two I am not sure if I have done it right since I have no real world sample of those, but to me it looks correct.

The next problem to solve would be to make sure that the incoming frames are correctly tagged and handed upwards. Also we do not have any deciding yet. I do not know whats the coded inband data is for but I think we need to be able to decode the SID_UPDATE frames since those contain comfort noise updates. The SID_UPDATE has a 1/4 convolutional coder in between and is CRC protected.

#14 Updated by laforge 8 months ago

Hi dexter,

On Tue, Feb 04, 2020 at 02:23:34PM +0000, dexter [REDMINE] wrote:

I now got my head around the interleaving. I have programmed some detector functions that can detect the various different types of SID frames by their identification pattern or by their coded inband data repetition sequence (depends on the sid frame type). We can now detect the following sid frames:

this is great progress, I'm really happy to read about it!

The next problem to solve would be to make sure that the incoming frames are correctly tagged and handed
upwards. Also we do not have any deciding yet.

I'm not sure what you are trying to express here: "do not have any deciding yet"?

I do not know whats the coded inband data is for but I think we need
to be able to decode the SID_UPDATE frames since those contain comfort
noise updates. The SID_UPDATE has a 1/4 convolutional coder in between
and is CRC protected.

We need to decode them only as far as needed int order to propagate them
to the RTP side. Not sure if that helps you

#15 Updated by dexter 8 months ago

There was a typo in the last message "deciding" => "decoding"

The current code that is used to detect the DTX frames is now in gerrit, however I think it is not perfect yet.
https://gerrit.osmocom.org/c/libosmocore/+/17095 dtx: add functions to determine DTX frame types

The experimental code that detects the DTX frames in osmo-bts-trx can be found here: pmaier/dtxtest Currently I only print the frame type. Next I will make sure that the is_sub flag is set whenever a DTX frame is received.

I had a look on how osmo-bts-sysmo does the DTX frame handling in tch.c:l1if_tch_rx(). There it seems not do do anything with the information other than setting the rtp_tx_marker to true for ONSET and the INH frames. I wonder if we should find a way here to set the is_sub flag here too.

(also helpful: https://ftp.osmocom.org/docs/latest/rtp-amr.pdf)

#16 Updated by dexter 7 months ago

The next step here is to tag the incoming frames as correctly "is_sub". For osmo-bts-trx this is now easy, but it should be made working for osmo-bts-sysmo as well. There we face two problems. The first is that measurement data and payload data are sent separately through l1sap. This has to be resolved first. The other is that the phy only identifies SID_FIRST and ONSET for us, the SID_UPDATE frames are not identified. I am not sure whats the best way to resolve this. A state flag could be enough but I think its better when we somehow identify the frames, this should be possible.

Also the marker bit for AMR_ONSET must be set as well. Osmo-bts-sysmo does this by setting lchan->rtp_tx_marker to true. I have copied this method to osmo-bts-trx

#17 Updated by dexter 7 months ago

Found out that in osmo-bts-sysmo also the SID_UPDATE is not identified. I think those frames are then IF2 formatted and we need to look into the frames to check if they are SID frames.

However, we still need to decode the SID_UPDATE frames we receive in osmo-bts-trx. I have done some experiments and I can now extract the data from SID_UPDATE frames. I can also apply the convolutional decoder and the CRC check on them for the two frames I experimented with the CRC check is successful, that means the data is decoded correctly. Now I have to integrate that somehow in libosmocore. I also think we also will have to mangle the BITs a bit so that we get a valid AMR TRAU frame (in IF2 format?).

#18 Updated by laforge 7 months ago

On Fri, Feb 28, 2020 at 03:58:43PM +0000, dexter [REDMINE] wrote:

Found out that in osmo-bts-sysmo also the SID_UPDATE is not identified. I think those frames are then IF2 formatted and we need to look into the frames to check if they are SID frames.

In general, one selsects either RTP or IF2 mode, and I would be surprised if even in RTP
mode some would be reported as IF2.

However, we still need to decode the SID_UPDATE frames we receive in osmo-bts-trx. I have done some experiments and I can now extract the data from SID_UPDATE frames. I can also apply the convolutional decoder and the CRC check on them for the two frames I experimented with the CRC check is successful, that means the data is decoded correctly.

great.

Now I have to integrate that somehow in libosmocore. I also think we also will have to mangle the BITs a bit so that we get a valid AMR TRAU frame (in IF2 format?).

Where do we need the TRAU format? That format is only used between the
BTS and BSC over a circuit-switched E1/T1 Abis interface (16k sub-slots
with TRAU frames). So that's only relevant to Nokia/Siemens/Ericsson
BTSs, but not to the IP based BTS mdoels such as osmo-bts-*

#19 Updated by dexter 7 months ago

I see TRAU format is only relevant for E1/T1, I have confused this with the RTP format.

I have done some integration now in osmo-bts-trx. In FR AMR the sid update decodes fine but in HR AMR I get a lot of CRC errors. Only a few frames decode properly. Since there is no direct connection between BTS and MS, this could be due to interference. The CRC errors get less when I shorten the distance between BTS and MS. However I still have the feeling that something might be wrong here.

There is also another problem I see. For the convolutional encoded section of the SID_UPDATE i get an n_bis_total and an n_errors value but for the othe SID frames I do not determine those values. I could fix this. All i need to do is to count out the deviations to the marker pattern. But then the question is what to do with frames other than SID_UPDATE and SID_FIRST. Those won't generate an RTP packet but they would generate a measurement. Presumably I would need to pass an AMR_BAD frame or something up to the higher layers.

#20 Updated by dexter 7 months ago

I have worked out the integration of the SID_FIRST and SID_UPDATE decoding into osmo-bts-trx to the point that the decoder in libosomocore outputs a frame that should be suitable to be sent via RTP. However, re-reading the spec revealed that there needs to be a frame type prepended to the 35 decoded bits. This is very confusing, i first thought that those 4 bit frame types were already the RTP header, but apparently the CMR and TOC data is something independent. After all we never really see a proper IF2 frame in our chain, its basically the RTP format without header that leaves the decoder. I also investigated if there is any reordering specified for the comfort noise bits themselves, fortunately there is no reordering. The original order of the bits is preserved.

It also looks like if there were a solution for the problem that we can not pass SID frames up that would not translate into any RTP frame. There is a frame type 15 which basically means that the frame does not contain any useful data, I think the SID frames that terminate at the BTS should generate a type 15 frame and send it to the higher layer. There then the measurement data is processed, but no RTP packet is emitted.

Also we still need to think about counting out the error bits. At the moment the decoder for the SID frames only counts out the errors roughly and does not report them back. This needs to be extended at least to the marker patterns so that we can deliver a value for n_bits_total and n_errors.

#21 Updated by dexter 6 months ago

The error bits are now counted out. However I was noticing that we can not just accept any pattern of coded inband data. The spec defines 4 distinctive patterns. I have changed the checks now. Fortunately since the coded inband data can now only consist of 4 well defined patterns and not just random bits checking becomes much easier. I also hope that checking for ONSET frames now becomes more reliable.

I was also noticing that there were a lot of bad frames in the RTP stream which only consisted of 0x00 bytes. I saw that the phy based BTSs call lchan_set_marker() allow for skipping of the not sent DTX frames. We did not call this function before, presumably because we did not have SID frame detection but now we have. However, I can see the amount of RTP frames is vastly reduced during the silence periods. I need to do a couple more tests, I also did not test HR yet.

#22 Updated by dexter 6 months ago

While testing I noticed that on FR none of the SID_UPDATE frame could be decoded anymore. The reason turned out to be quite surprising. Actually we have now the same behavior as we are experiencing with HR. I did not get my head around it yet but it makes sense together with the fact that we have two times coded-inband-data inside the SID_UPDATE. The sudden change of the behavour is due to fact that we now also pass nope indications for the tch channels. I think that we were seeing SID_UPDATE + Marker pattern in one frame was just because some blocks were counted wrongly. I now need to fix this so that it works the same as it does on HR. I ran a quick test on HR today and to my surprise also the decoding there has been improved (could also be due to more correct configs for osmo-bts-trx and osmo-trx-uhd).

#23 Updated by dexter 6 months ago

  • % Done changed from 80 to 90

Change Ice45d5986610d9bcef2a7e41f0a395ec779e3928 has fixed some timing/clocking issues so that some of the behavior during DTX looked different, what seemed to be correct really was not correct. I have revisited the DTX changes now and also fixed the remaining problems. One major problem was that during a silent period osmo-btx-trx continued to emit bad RTP packets. Also the first (ONSET) frame of the beginning of the voice spurt was always bad. This now fixed and I feel confident that we can now review and merge it.

#24 Updated by dexter 5 months ago

The following patches are currently in review:

https://gerrit.osmocom.org/c/osmo-bts/+/17221 dtx: add detection of AMR DTX frames for osmo-bts-trx
https://gerrit.osmocom.org/c/osmo-bts/+/17928 measurement: remove unecessary is_amr_sid_update parameter
https://gerrit.osmocom.org/c/osmo-bts/+/17929 measurement: expect at least 1 SUB frame for AMR

While the mechanics look good (we receive plausible DTX frames and we can decode them), I am noticing
that the measurement results (RSSI-VALUES) are actually looking very strange:

The following is for FR-AMR:

Without DTX enable I get the following:

Thu Apr 23 12:49:06 2020 <0004> measurement.c:706 (bts=0,trx=0,ts=1,ss=0) Computed TA256(   7) BER-FULL( 0.00%), RSSI-FULL(- 33dBm), BER-SUB( 0.00%), RSSI-SUB(- 33dBm)
Thu Apr 23 12:49:06 2020 <0004> measurement.c:706 (bts=0,trx=0,ts=2,ss=0) Computed TA256(  27) BER-FULL( 0.00%), RSSI-FULL(- 42dBm), BER-SUB( 0.00%), RSSI-SUB(- 42dBm)

This looks normal to me but when I enable DTX I get much lower RSSI values, this is strange because during the whole test I did not change the position of the mobile phone, the external condition are the same, I assume tat I get values that are at least in the ballpark of the values I get when DTX is off.

Thu Apr 23 12:52:57 2020 <0004> measurement.c:706 (bts=0,trx=0,ts=1,ss=0) Computed TA256(   0) BER-FULL(72.00%), RSSI-FULL(- 87dBm), BER-SUB( 0.00%), RSSI-SUB(- 60dBm)
Thu Apr 23 12:52:57 2020 <0004> measurement.c:706 (bts=0,trx=0,ts=2,ss=0) Computed TA256(  -4) BER-FULL(72.00%), RSSI-FULL(- 83dBm), BER-SUB( 0.00%), RSSI-SUB(- 48dBm)

I found the cause of the problem, however I do not get why it is programmed that way but first lets look at the following log:

====FN_CALL_WITH_RSSI=> tn=1, rssi=-98
====FN_CALL_WITH_RSSI=> tn=2, rssi=-98
====FN_CALL_WITH_RSSI=> tn=1, rssi=-96
====FN_CALL_WITH_RSSI=> tn=2, rssi=-98
====FN_CALL_WITH_RSSI=> tn=1, rssi=-98
====FN_CALL_WITH_RSSI=> tn=2, rssi=-98
====FN_CALL_WITH_RSSI=> tn=1, rssi=-97
Thu Apr 23 17:49:33 2020 <0007> scheduler_trx.c:1283 1684972/1270/16/34/28 (bts=0,trx=0,ts=1) TCH/F: Received bad data (68/104)
================> ts=1 -97, 0
Thu Apr 23 17:49:33 2020 <0004> measurement.c:345 1684964/1270/08/26/20 (bts=0,trx=0,ts=1,ss=0) adding measurement (is_sub=0), num_ul_meas=15, fn_mod=60
====FN_CALL_WITH_RSSI=> tn=2, rssi=-98
Thu Apr 23 17:49:33 2020 <0007> scheduler_trx.c:1283 1684972/1270/16/34/28 (bts=0,trx=0,ts=2) TCH/F: Received bad data (68/104)
================> ts=2 -98, 0
Thu Apr 23 17:49:33 2020 <0004> measurement.c:345 1684964/1270/08/26/20 (bts=0,trx=0,ts=2,ss=0) adding measurement (is_sub=0), num_ul_meas=12, fn_mod=60
====FN_CALL_WITH_RSSI=> tn=1, rssi=-97
====FN_CALL_WITH_RSSI=> tn=2, rssi=-49
====FN_CALL_WITH_RSSI=> tn=1, rssi=-97
====FN_CALL_WITH_RSSI=> tn=2, rssi=-50
====FN_CALL_WITH_RSSI=> tn=1, rssi=-99
====FN_CALL_WITH_RSSI=> tn=2, rssi=-49
====FN_CALL_WITH_RSSI=> tn=1, rssi=-97
Thu Apr 23 17:49:33 2020 <0007> scheduler_trx.c:1283 1684976/1270/20/38/32 (bts=0,trx=0,ts=1) TCH/F: Received bad data (72/104)
================> ts=1 -97, 0
Thu Apr 23 17:49:33 2020 <0004> measurement.c:345 1684969/1270/13/31/25 (bts=0,trx=0,ts=1,ss=0) adding measurement (is_sub=0), num_ul_meas=16, fn_mod=65
====FN_CALL_WITH_RSSI=> tn=2, rssi=-49
====RSSI-FOR-SID=> tn=2, rssi=-49
Thu Apr 23 17:49:33 2020 <0007> scheduler_trx.c:1235 1684976/1270/20/38/32 (bts=0,trx=0,ts=2) TCH/F: Received AMR SID frame: AFS_SID_UPDATE (marker)
Thu Apr 23 17:49:33 2020 <0007> scheduler_trx.c:1288 1684976/1270/20/38/32 (bts=0,trx=0,ts=2) TCH/F: Received bad data (72/104) with invalid codec mode 3
================> ts=2 -49, 1
Thu Apr 23 17:49:33 2020 <0004> measurement.c:345 1684969/1270/13/31/25 (bts=0,trx=0,ts=2,ss=0) adding measurement (is_sub=1), num_ul_meas=13, fn_mod=65
====FN_CALL_WITH_RSSI=> tn=1, rssi=-98
====FN_CALL_WITH_RSSI=> tn=2, rssi=-97
====FN_CALL_WITH_RSSI=> tn=1, rssi=-98
====FN_CALL_WITH_RSSI=> tn=2, rssi=-97
====FN_CALL_WITH_RSSI=> tn=1, rssi=-98
====FN_CALL_WITH_RSSI=> tn=2, rssi=-98
====FN_CALL_WITH_RSSI=> tn=1, rssi=-97
Thu Apr 23 17:49:33 2020 <0007> scheduler_trx.c:1283 1684980/1270/24/42/36 (bts=0,trx=0,ts=1) TCH/F: Received bad data (76/104)
================> ts=1 -97, 0
Thu Apr 23 17:49:33 2020 <0004> measurement.c:345 1684973/1270/17/35/29 (bts=0,trx=0,ts=1,ss=0) adding measurement (is_sub=0), num_ul_meas=17, fn_mod=69
====FN_CALL_WITH_RSSI=> tn=2, rssi=-98
====RSSI-FOR-SID=> tn=2, rssi=-98
Thu Apr 23 17:49:33 2020 <0007> scheduler_trx.c:1235 1684980/1270/24/42/36 (bts=0,trx=0,ts=2) TCH/F: Received AMR SID frame: AFS_SID_UPDATE_CN (audio)
<pre>

"FN_CALL_WITH_RSSI" is just a marker that tells me that rx_tchf_fn() was called and what the RSSI of the burst was that came in. "RSSI-FOR-SID" is the point that maks that the DTX block is complete and which RSSI the block has. We see that we receive the AFS_SID_UPDATE (marker) with an RSSI of -49, this is plausible but already problematic sind the the code only uses the RSSI value of the last burst in the block. All other values are ignored. I do not know if this is a bug or if we just assume that the value wouln't change much anyway, so taking the last one is enough. So far everything is still fine but when the AFS_SID_UPDATE_CN (audio) is received we get an RSSI value of -98, which can not be.

In the sourcecode I can see that the burst is first built up at the high side of the buffer (see line 1166) but when the burst is complete the code runs on the beginning of the buffer. Once the code is through with its work on the buffer it copies the high side of the buffer to the beginning. I do not really get whats the purpose of this shifting, I also don't see that the high side of the buffer is used somewhere until it gets copied to the low side. This partially explains the why the RSSI of the CN data is messed up. However I do not fully understand this yet. The problem is caused by the fact that when the burst data is processed it uses the RSSI from the burst that concludes the block, but the data is from the block before? And why is looking everything like if there were marker and CN data in the same block when observing the RSSIs while this seems not to be the case when looking at the data (marker in one block, CN data in the following block)

I will investigate this further, but I think we need to find a way to somehow lock the RSSI values that arrive on burst level onto the actual data so that we are sure that the RSSI levels actually belong the the block that we pass up to the higher layers.

#25 Updated by laforge 5 months ago

On Thu, Apr 23, 2020 at 04:22:07PM +0000, dexter [REDMINE] wrote:

problematic sind the the code only uses the RSSI value of the last burst in the block. All other values are ignored. I do not know if this is a bug or if we just assume that the value wouln't change much anyway, so taking the last one is enough.

This is a bug. It should be the average of those bursts, IMHO

In the sourcecode I can see that the burst is first built up at the high side of the buffer (see line 1166) but when the burst is complete the code runs on the beginning of the buffer. Once the code is through with its work on the buffer it copies the high side of the buffer to the beginning. I do not really get whats the purpose of this shifting,

It relates to the spreading / interleaving of one voice frame across 8
bursts. Every four bursts one voice frame is complete, but you still
need 8 bursts to decode it.

I also don't see that the high side of the buffer is used somewhere until it gets copied to the low side.

it is used when the next voice frame is to be decoded...

I will investigate this further, but I think we need to find a way to somehow lock the RSSI values that arrive on burst level onto the actual data so that we are sure that the RSSI levels actually belong the the block that we pass up to the higher layers.

I you want to do it correct and the same way as the data bits, you would
have to store the per-burst RSSI values also in an array with eight
entries, and apply the same shifting logic. Then every time you report
a voice frame upwards, you take the average of those eight values.

#26 Updated by fixeria 5 months ago

Hi Harald, Philipp,

This is a bug. It should be the average of those bursts, IMHO

ACK.

I you want to do it correct and the same way as the data bits, you would
have to store the per-burst RSSI values also in an array with eight
entries, and apply the same shifting logic. Then every time you report
a voice frame upwards, you take the average of those eight values.

This is exactly how it's implemented in trxcon, using a stack allocated ring-buffer.
I am planning to port my changes to osmo-bts-trx as soon as I find some time for that.
See https://git.osmocom.org/osmocom-bb/commit/?id=2060b5b7cc3b63b64e651d3cda5ed50b44593a05.

#27 Updated by dexter 5 months ago

I see, gsm0503_tch_ahs_decode_dtx() is accessing the far end of the buffer as well, now its clear to me.

eight entries, and apply the same shifting logic.

I had the same idea last night, thanks for confirming. I will implement it that way.

#28 Updated by dexter 5 months ago

I have now fixed the remaining problems with the sub frames. We now get meaningful RSSI values for the SUB frames and we also average the RSSI values that we pass up to higher layers. I have also tested the patch with all possible voice codecs.

The following patches are now in review:
https://gerrit.osmocom.org/c/osmo-bts/+/18034 measurement: make measurements more debugable
https://gerrit.osmocom.org/c/osmo-bts/+/18035 scheduler_trx: fix RSSI calculation for SUB frames
https://gerrit.osmocom.org/c/osmo-bts/+/17929 measurement: expect at least 1 SUB frame for AMR

#29 Updated by dexter 3 months ago

Patch https://gerrit.osmocom.org/c/osmo-bts/+/18035 is still in review, unfortunately it suffers from merge conflicts. A quick inspection of the situation reveals that there were drastic changes in scheduler_trx.c recently.

#30 Updated by dexter 3 months ago

  • % Done changed from 90 to 50

#31 Updated by dexter 3 months ago

  • Assignee changed from dexter to fixeria

#32 Updated by fixeria 3 months ago

  • Status changed from In Progress to Feedback
  • Assignee changed from fixeria to dexter
  • % Done changed from 50 to 80

dexter wrote:

Patch https://gerrit.osmocom.org/c/osmo-bts/+/18035 is still in review, unfortunately it suffers from merge conflicts. A quick inspection of the situation reveals that there were drastic changes in scheduler_trx.c recently.

I implemented measurement averaging using a simple ring buffer, so now not only RSSI is handled, but also ToA256 and C/I. Change by dexter has been rebased on top of it (thus on top of the current master) and now waiting for review in Gerrit.

I would still want to look at the related parts of the spces., because right now this ("4 low bursts of 8" or "2 middle bursts of 6") looks like a magic to me, and I guess would look so for other people reading the code. dexter any hints? Thanks!

#33 Updated by laforge 3 months ago

On Tue, Jun 23, 2020 at 08:51:07AM +0000, fixeria [REDMINE] wrote:

I would still want to look at the related parts of the spces., because right now this ("4 low bursts of 8" or "2 middle bursts of 6") looks like a magic to me, and I guess would look so for other people reading the code. dexter any hints? Thanks!

I don't think it matters much. Measurements are aggregated / averaged over time at various
different levels. So as long as all of the bursts we are using are from the same logical
channel, we can't really do anything wrong.

#34 Updated by fixeria 3 months ago

Measurements are aggregated / averaged over time at various different levels.

Ok, that's clear. So far it worked even with measurements of the last burst of a block, and (almost) nobody complained ;)

So as long as all of the bursts we are using are from the same logical channel, we can't really do anything wrong.

Unless we deal with DTX, where some bursts are intentionally skipped. If we pick wrong entries in the measurement history, we get wrong measurements (basically noise measurements). That's why I am curious. After all, I believe that dexter took those "4 first bursts of 8" or "2 middle bursts of 6" averaging modes from some document, and they are not just random guesses.

#35 Updated by fixeria 3 months ago

After all, I believe that Philipp took those "4 first bursts of 8" or "2 middle bursts of 6" averaging modes from some document, and they are not just random guesses.

dexter ping?

#36 Updated by dexter 3 months ago

I found the burst combination through experimenting. While in non-dmx mode the selection of the bursts does not have much effect, however I think averaging over 8 bursts instead of 4 is not all too good because of the diagonal interleaving the the measurement results are valid for the current and for the adjacent burst at the same time. Taking four should be ok, especially when the results are getting averaged by the higher layers it should not be all too important if the computation was done over 4 or over 8 bursts (4 or 2 for HR) - but that is not the problem here.

When using DTX there may be blocks where only one half of bursts is used to transmit the DTX frame. So I decided to pick the bursts I use for averaging in a way so that when a DTX frame that announces the begin of a silence interval is received the 4 (2) bursts with actual information are picked instead of the ones that contain silence. It is difficult to detect this situation in advance.

You can make an experiment: Enable AMR DTX, make a call and trigger between silence periods and voice spurts by blowing into the mic of the phone. When you print the RSSI for each burst individually you will see what I mean.

#37 Updated by fixeria 3 months ago

Thank you, Philipp!

I found the burst combination through experimenting.

Nice! I'll then add comments to the code stating that those magic measurement modes were found experimentally.

When using DTX there may be blocks where only one half of bursts is used to transmit the DTX frame.

Let's just make sure that we understand "only one half" correctly. You cannot just send half of a burst, it must always be two halves (57 bit each, 114 payload bits in total) together, plus a training sequence for demodulator. In context of the block-diagonal interleaving, one half means either odd bits of a burst, or even bits of a burst. So a L2 block is equally distributed over N consecutive bursts. That's why I believe that we should be calculating AVG for the whole distribution period. But as was mentioned, averaging in the higher layers makes this aspect less critical.

When using DTX there may be blocks where only one half of bursts is used to transmit the DTX frame.

So in case of e.g. TCH/F, where each L2 frame is interleaved over 8 consecutive bursts,

  • unused halves (odd numbered bits) of the first 4 bursts are set to zero or simply reflect the old content of the buffers;
  • unused halves (even numbered bits) of the last 4 bursts are filled according to 3GPP TS 45.002, section A.2.2.

So here is an extract from section A.2.2 named "Half burst filling":

For downlink DTX, when a given traffic frame is scheduled for transmission and one of its adjacent traffic frames is not scheduled for transmission, half of the "encrypted bits" belonging to the normal bursts associated with the scheduled traffic frame need to be filled. These bits are referred to as "half burst filling bits". These half bursts filling bits contain either:
  • A.2.2.1 Partial SID information from any associated SID frame; or
  • A.2.2.2 The mixed bits of the dummy bursts (encrypted or not encrypted).

So the specs. say half of the "encrypted bits", not half of the normal burst itself. Regarding the interleaving/mapping of these partial SID frames, we should refer to sections 3.9 and 3.10 of 3GPP 45.003.

So I decided to pick the bursts I use for averaging in a way so that when a DTX frame that announces the begin of a silence interval is received the 4 (2) bursts with actual information are picked instead of the ones that contain silence. It is difficult to detect this situation in advance.

Ah, I think I understand your idea now. So we're talking about SID_FIRST (TCH/AFS) and SID_FIRST{P1,P2}_ (TCH/AHS).

SID_FIRST (TCH/AFS) is interleaved in the same way as a regular full-rate speech or FACCH/F frame - over 8 consecutive bursts:

3.9.2.3 Interleaving

The interleaving is done as specified for the TCH/FS in subclause 3.1.3.

3.9.2.4 Mapping on a Burst

The mapping is done as specified for the TCH/FS in subclause 3.1.4. The last 4 bursts shall not be transmitted unless the SID_FIRST frame is immediately followed by a speech frame.

SID_FIRST{P1,P2}_ is interleaved in the same way as a regular half-rate speech frame - over 4 consecutive bursts.

3.10.3.3 Interleaving

The interleaving is done as specified for the TCH/HS in subclause 3.2.3.

3.10.3.4 Mapping on a Burst

The mapping is done as specified for the TCH/HS in subclause 3.2.4.

It looks like we should handle them in the same way as regular speech frames, not like "first 2 bursts of 6" or "first 4 bursts of 8". Correct me if I am wrong. Other SID frames are interleaved in a different way, I didn't look further.

Another interesting question is how do we handle the measurements triggered by NOPE.ind after reception of SID_FIRST (on TCH/AFS) or SID_FIRST{P1,P2}_ (on TCH/AHS). The reported RSSI values will be equal to the noise levels, so we should make sure that we ignore them during the intervals of silence. Not sure if and where it's implemented.

Best regards,
Vadim.

#38 Updated by dexter 23 days ago

  • Status changed from Feedback to Stalled
  • Assignee changed from dexter to fixeria

It is not possible to handle the SID frames like voice frames as they do not fill up the whole time period of a voice frame. They are more like half-frames. There is some printed documentation available in the office.

As far as I know we get RSSI levels on a per burst level. We also can examine the bit errors in the SID frames, so it is possible to pass up RSSI and bit error levels there.

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 48.8 MB)