Project

General

Profile

Bug #3938

Get Attribute Response parsing is broken: NM_ATT_SW_DESCR is ignored

Added by fixeria 4 months ago. Updated 3 months ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
-
Target version:
-
Start date:
04/18/2019
Due date:
% Done:

100%

Spec Reference:
3GPP TS 52.021

Description

For some reason, OsmoBSC doesn't parse the attribute 'NM_ATT_SW_DESCR' from 'Get Attributes Response':

DNM DEBUG <0004> abis_nm.c:1716 Get Attr (bts=0,trx=255)
DNM DEBUG <0004> abis_nm.c:1716 Get Attr (bts=0,trx=0)

...

DNM DEBUG <0004> abis_nm.c:621 OC=BTS(01) INST=(00,ff,ff): Get Attributes Response for BTS0
DNM DEBUG <0004> abis_nm.c:489 BTS0 Get Attributes Response Info: 70 bytes total with 0 unreported attributes
DNM NOTICE <0004> abis_nm.c:538 BTS0 feature 'EGPRS' reported via OML does not match statically set feature: 0 != 1. Please fix.
DNM NOTICE <0004> abis_nm.c:538 BTS0 feature 'OML Alerts' reported via OML does not match statically set feature: 1 != 0. Please fix.
DNM NOTICE <0004> abis_nm.c:538 BTS0 feature 'CBCH' reported via OML does not match statically set feature: 1 != 0. Please fix.
DNM NOTICE <0004> abis_nm.c:538 BTS0 feature 'Fullrate speech V1' reported via OML does not match statically set feature: 1 != 0. Please fix.
DNM NOTICE <0004> abis_nm.c:538 BTS0 feature 'Halfrate speech V1' reported via OML does not match statically set feature: 1 != 0. Please fix.
DNM NOTICE <0004> abis_nm.c:538 BTS0 feature 'Fullrate speech EFR' reported via OML does not match statically set feature: 1 != 0. Please fix.
DNM NOTICE <0004> abis_nm.c:538 BTS0 feature 'Fullrate speech AMR' reported via OML does not match statically set feature: 1 != 0. Please fix.
DNM NOTICE <0004> abis_nm.c:538 BTS0 feature 'Halfrate speech AMR' reported via OML does not match statically set feature: 1 != 0. Please fix.
DNM DEBUG <0004> abis_nm.c:621 OC=BTS(01) INST=(00,ff,ff): Get Attributes Response for BTS0
DNM DEBUG <0004> abis_nm.c:489 BTS0 Get Attributes Response Info: 32 bytes total with 1 unreported attributes
DNM ERROR <0004> abis_nm.c:494 BTS0 Attribute Manufacturer Dependent State is unreported

while the old OpenBSC does:

DNM DEBUG <0005> abis_nm.c:1601 Get Attr (bts=0)
DNM DEBUG <0005> abis_nm.c:1601 Get Attr (bts=0)

...

DNM DEBUG <0005> abis_nm.c:550 OC=BTS(01) INST=(00,ff,ff) Get Attributes Response for BTS0
DNM DEBUG <0005> abis_nm.c:463 BTS0 Get Attributes Response Info: 70 bytes total with 0 unreported attributes
DNM NOTICE <0005> abis_nm.c:507 BTS0 feature 'EGPRS' reported via OML does not match statically set feature: 0 != 1. Please fix.
DNM NOTICE <0005> abis_nm.c:507 BTS0 feature 'OML Alerts' reported via OML does not match statically set feature: 1 != 0. Please fix.
DNM NOTICE <0005> abis_nm.c:507 BTS0 feature 'CBCH' reported via OML does not match statically set feature: 1 != 0. Please fix.
DNM NOTICE <0005> abis_nm.c:573 BTS0: ARI reported sw[0/2]: osmobts is 0.8.1.255-7209
DNM NOTICE <0005> abis_nm.c:446 BTS0 reported variant: omso-bts-trx
DNM DEBUG <0005> abis_nm.c:550 OC=BTS(01) INST=(00,ff,ff) Get Attributes Response for BTS0
DNM DEBUG <0005> abis_nm.c:463 BTS0 Get Attributes Response Info: 32 bytes total with 1 unreported attributes
DNM ERROR <0005> abis_nm.c:468 BTS0 Attribute Manufacturer Dependent State is unreported
DNM NOTICE <0005> abis_nm.c:573 BTS0: ARI reported sw[0/1]: TRX_PHY_VERSION is Unknown

Observed with the recent osmo-bts-trx 0.8.1.255-7209.

Please find the OML captures for both cases attached. I could't find any difference in both pairs of 'Get Attributes' / 'Get Attributes Response' messages. The following is my hand-parsed representation of these messages:

OML BTS(ff,ff,ff) Get Attributes
RAW: 1e41

OML BTS(00,ff,ff) Get Attributes Response
RAW (len=70): 001e000242fc421200076f736d6f62747313000e302e382e312e3235352d37323039421200104254535f545950455f56415249414e5413000c6f6d736f2d6274732d74727800

0000  00 1e 00 02 42 fc 42 12  00 07 6f 73 6d 6f 62 74   ....B.B. ..osmobt
0010  73 13 00 0e 30 2e 38 2e  31 2e 32 35 35 2d 37 32   s...0.8. 1.255-72
0020  30 39 42 12 00 10 42 54  53 5f 54 59 50 45 5f 56   09B...BT S_TYPE_V
0030  41 52 49 41 4e 54 13 00  0c 6f 6d 73 6f 2d 62 74   ARIANT.. .omso-bt
0040  73 2d 74 72 78 00                                  s-trx.

00 - the amount of unreported attributes

1e - NM_ATT_MANUF_ID
    0002 - 2 octets
    42fc - BTS supported features

42 - NM_ATT_SW_DESCR
    12 - NM_ATT_FILE_ID
        0007 - 7 octets
        6f736d6f627473 - "osmobts" 
    13 - NM_ATT_FILE_VERSION
        000e - 14 octets
        302e382e312e3235352d37323039 - "0.8.1.255-7209" 

42 - NM_ATT_SW_DESCR
    12 - NM_ATT_FILE_ID
        0010 - 16 octets
        4254535f545950455f56415249414e54 - "BTS_TYPE_VARIANT" 
    13 - NM_ATT_FILE_VERSION
        000c - 13 octets
        6f6d736f2d6274732d74727800 - "osmo-bts-trx" + \0

OML Baseband Transceiver(00,00,ff) Get Attributes
RAW: 1c41

OML BTS(00,ff,ff) Get Attributes Response
RAW (len=32): 011c4212000f5452585f5048595f56455253494f4e130007556e6b6e6f776e00

0000  01 1c 42 12 00 0f 54 52  58 5f 50 48 59 5f 56 45   ..B...TR X_PHY_VE
0010  52 53 49 4f 4e 13 00 07  55 6e 6b 6e 6f 77 6e 00   RSION... Unknown.

01 - the amount of unreported attributes
1c - NM_ATT_MANUF_STATE

42 - NM_ATT_SW_DESCR
    12 - NM_ATT_FILE_ID
        000f - 15 octets
        5452585f5048595f56455253494f4e - "TRX_PHY_VERSION" 
    13 - NM_ATT_FILE_VERSION
        0007 - 7 octets (WTF?!? \0 is not accounted)
        556e6b6e6f776e00 - "Unknown" + \0

I am not familiar with OML, though I found some things odd:

  • In the first 'Get Attributes' message we ask for 0x1e - NM_ATT_MANUF_ID and 0x41 - NM_ATT_SW_CONFIG. However, instead of 0x41 - NM_ATT_SW_CONFIG we receive 0x42 - NM_ATT_SW_DESCR in the response. Same problem with the second 'Get Attributes Response' message.
  • In the first 'Get Attributes Response' message, 'NM_ATT_SW_DESCR' tag is present twice. Is it allowed to indicate the same tag twice?
  • In the first 'Get Attributes Response' message, one value of 'NM_ATT_FILE_VERSION' is \0-terminated, another is not.
  • In the second 'Get Attributes Response' message, the value of 'NM_ATT_FILE_VERSION' is \0-terminated, but \0 is not accounted in the TLV's length.
  • The secound 'Get Attributes' message is for Baseband Transceiver (00,00,ff), however the response is for BTS (00,ff,ff).
abis_oml_open_bsc.pcapng.gz abis_oml_open_bsc.pcapng.gz 4.02 KB fixeria, 04/18/2019 08:26 AM
abis_oml_osmo_bts.pcapng.gz abis_oml_osmo_bts.pcapng.gz 4.09 KB fixeria, 04/18/2019 08:26 AM

Related issues

Related to OsmoBSC - Bug #1614: better identification of BTS model / capabilities to BSCStalled02/23/2016

History

#1 Updated by fixeria 4 months ago

  • Related to Bug #1614: better identification of BTS model / capabilities to BSC added

#2 Updated by fixeria 4 months ago

The secound 'Get Attributes' message is for Baseband Transceiver (00,00,ff), however the response is for BTS (00,ff,ff).

I just had a quick look at the OML captures from OS#3814, in particular - 'msmax0_startup_abis.pcapu'. Unlike OsmoBTS, the Attribute Response from nanoBTS does match the request. Indeed it's a bug of OsmoBTS. Please see:

https://gerrit.osmocom.org/#/c/osmo-bts/+/13701/

After this change, the behaviour of OsmoBTS is similar to nanoBTS. One problem less ;)

#3 Updated by fixeria 4 months ago

In the first 'Get Attributes Response' message, one value of 'NM_ATT_FILE_VERSION' is \0-terminated, another is not.

In the second 'Get Attributes Response' message, the value of 'NM_ATT_FILE_VERSION' is \0-terminated, but \0 is not accounted in the TLV's length.

Looking at 'msmax0_startup_abis.pcap' from #3814, I found that nanoBTS does include several 'NM_ATT_SW_DESCR' blobls into a separate attribute called 0x41 'SW Configuration' (not 0x44 'Get ARI' as we do in OsmoBTS). Here is my hand-crafted representation of these blobs:

0000  42 12 00 08 31 36 38 64  34 37 34 00 13 00 0b 76   B...168d 474....v
0010  32 30 30 62 31 36 34 64  30 00 42 12 00 08 31 36   200b164d 0.B...16
0020  38 64 34 37 32 00 13 00  0b 76 32 30 30 62 31 36   8d472... .v200b16
0030  34 64 30 00 42 12 00 08  31 36 38 64 34 37 32 00   4d0.B... 168d472.
0040  13 00 0b 76 32 30 30 62  31 35 31 64 30 00 42 12   ...v200b 151d0.B.
0050  00 08 31 36 38 64 34 37  31 00 13 00 0b 76 32 30   ..168d47 1....v20
0060  30 62 31 36 34 64 30 00  42 12 00 08 31 36 38 64   0b164d0. B...168d
0070  34 37 31 00 13 00 0b 76  32 30 30 62 31 35 31 64   471....v 200b151d
0080  30 00                                              0.

42 - NM_ATT_SW_DESCR
    12 - NM_ATT_FILE_ID
        0008 - 8 octets
        3136386434373400 - "168d474" + \0
    13 - NM_ATT_FILE_VERSION
        000b - 11 octets
        7632303062313634643000 - "v200b164d0" + \0
42 - NM_ATT_SW_DESCR
    12 - NM_ATT_FILE_ID
        0008 - 8 octets
        3136386434373200 - "168d472" + \0
    13 - NM_ATT_FILE_VERSION
        000b - 11 octets
        7632303062313634643000 - "v200b164d0" + \0
42 - NM_ATT_SW_DESCR
    12 - NM_ATT_FILE_ID
        0008 - 8 octets
        3136386434373200 - "168d472" + \0
    13 - NM_ATT_FILE_VERSION
        000b - 11 octets
        7632303062313531643000 - "v200b151d0" + \0
42 - NM_ATT_SW_DESCR
    12 - NM_ATT_FILE_ID
        0008 - 8 octets
        3136386434373100 - "168d471" + \0
    13 - NM_ATT_FILE_VERSION
        000b - 11 octets
        7632303062313634643000 - "v200b164d0" + \0
42 - NM_ATT_SW_DESCR
    12 - NM_ATT_FILE_ID
        0008 - 8 octets
        3136386434373100 - "168d471" + \0
    13 - NM_ATT_FILE_VERSION
        000b - 11 octets
        7632303062313531643000 - "v200b151d0" + \0

I have no idea what is the meaning of these magic NM_ATT_FILE_ID / NM_ATT_FILE_VERSION values, but it's clear that all values are \0-terminated, and \0 is accounted in the TLV's length. Please see:

https://gerrit.osmocom.org/#/c/libosmocore/+/13702

#4 Updated by fixeria 4 months ago

In the second 'Get Attributes Response' message, the value of 'NM_ATT_FILE_VERSION' is \0-terminated, but \0 is not accounted in the TLV's length.
In the first 'Get Attributes Response' message, one value of 'NM_ATT_FILE_VERSION' is \0-terminated, another is not.

As it turns out, this is actually a result of reading uninitialized memory:

DOML DEBUG oml.c:539 OC=BTS(01) INST=(ff,ff,ff) Rx GET ATTR
DOML INFO oml.c:265 BTS Tx Get Attribute Response
==25467== Syscall param socketcall.sendto(msg) points to uninitialised byte(s)
==25467==    at 0x623E0BD: send (send.c:27)
==25467==    by 0x5685846: __handle_ts1_write (ipaccess.c:358)
==25467==    by 0x5683F8B: ipa_client_write (ipa.c:79)
==25467==    by 0x5683F8B: ipa_client_fd_cb (ipa.c:140)
==25467==    by 0x5F1DC23: osmo_fd_disp_fds (select.c:223)
==25467==    by 0x5F1DC23: osmo_select_main (select.c:263)
==25467==    by 0x42980B: bts_main (main.c:354)
==25467==    by 0x6160F44: (below main) (libc-start.c:287)
==25467==  Address 0x7d83895 is 23,669 bytes inside a block of size 102,528 alloc'd
==25467==    at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==25467==    by 0x589A6B4: talloc_pool (in /usr/lib/x86_64-linux-gnu/libtalloc.so.2.1.5)
==25467==    by 0x5F1E28B: msgb_talloc_ctx_init (msgb.c:316)
==25467==    by 0x4293D0: bts_main (main.c:234)
==25467==    by 0x6160F44: (below main) (libc-start.c:287)
==25467==  Uninitialised value was created by a stack allocation
==25467==    at 0x415FE5: oml_tx_attr_resp (oml.c:259)
==25467==    by 0x415FE5: oml_rx_get_attr (oml.c:561)
==25467==

where "at 0x415FE5: oml_tx_attr_resp (oml.c:259)" corresponds to:

/* send 3GPP TS 52.021 ยง8.11.2 Get Attribute Response */
static int oml_tx_attr_resp(const struct gsm_abis_mo *mo,
                            const uint8_t *attr, uint16_t attr_len)
{
        struct msgb *nmsg;
        uint8_t resp[MAX_VERSION_LENGTH * attr_len * 2]; /* heuristic for Attribute Response Info space requirements */
        // ...

and adding:

memset(resp, 0xff, sizeof(resp))

turns those fake \0-terminators into 0xff. Why ASAN didn't catch this?

#5 Updated by fixeria 4 months ago

As it turns out, this is actually a result of reading uninitialized memory [...]

Please see: https://gerrit.osmocom.org/#/c/osmo-bts/+/13707/

#6 Updated by fixeria 4 months ago

  • Status changed from New to Feedback
  • Priority changed from Normal to High
  • % Done changed from 0 to 60

In the first 'Get Attributes Response' message, 'NM_ATT_SW_DESCR' tag is present twice. Is it allowed to indicate the same tag twice?

Since in parse_attr_resp_info() we do:

abis_nm_tlv_parse(tp, bts, data, data_len);

of course, the TLV parser does not handle the repeating tag properly. In the best case, it would parse only the last one, while the preceding ones would be overridden. But actually, basic debugging in gdb shows that NM_ATT_MANUF_ID is the only one parsed by abis_nm_tlv_parse():

gdb-peda$ b abis_nm.c:608
Breakpoint 1 at 0x40b05b: file abis_nm.c, line 608.

gdb-peda$ r -c osmo-bsc.cfg
...

[-------------------------------------code-------------------------------------]
   0x40b04f <parse_attr_resp_info+222>: mov    rax,QWORD PTR [rbp-0x40]
   0x40b053 <parse_attr_resp_info+226>: mov    rdi,rax
   0x40b056 <parse_attr_resp_info+229>: call   0x40920e <abis_nm_tlv_parse>
=> 0x40b05b <parse_attr_resp_info+234>: mov    rcx,QWORD PTR [rbp-0x40]
   0x40b05f <parse_attr_resp_info+238>: mov    rdx,QWORD PTR [rbp-0x38]
   0x40b063 <parse_attr_resp_info+242>: mov    rsi,QWORD PTR [rbp-0x30]
   0x40b067 <parse_attr_resp_info+246>: mov    rax,QWORD PTR [rbp-0x28]
   0x40b06b <parse_attr_resp_info+250>: mov    rdi,rax
[------------------------------------stack-------------------------------------]
Breakpoint 1, parse_attr_resp_info (bts=0x989d20, trx=0x0, foh=0x9a0062, tp=0x7fffffffb780) at abis_nm.c:608
608             return parse_attr_resp_info_attr(bts, trx, foh, tp);

gdb-peda$ p *tp
$2 = {
  lv = {{
      len = 0x0, 
      val = 0x0
    } <repeats 30 times>, {
      len = 0x2, 
      val = 0x9a006e "B\374B\022" 
    }, {
      len = 0x0, 
      val = 0x0
    } <repeats 225 times>}
}

gdb-peda$ p tp->lv[NM_ATT_MANUF_ID]
$4 = {
  len = 0x2, 
  val = 0x9a006e "B\374B\022" 
}

Most likely, because neither NM_ATT_SW_CONFIG nor NM_ATT_SW_DESCR is defined in bts->model->nm_att_tlvdef (see ./src/osmo-bsc/bts_*.c), so it's ignored.

In the first 'Get Attributes' message we ask for 0x1e - NM_ATT_MANUF_ID and 0x41 - NM_ATT_SW_CONFIG. However, instead of 0x41 - NM_ATT_SW_CONFIG we receive 0x42 - NM_ATT_SW_DESCR in the response. Same problem with the second 'Get Attributes Response' message.

I think this is the spec. violation: if the BSC asks for NM_ATT_SW_CONFIG, we should get NM_ATT_SW_CONFIG from the BTS as expected. A possible solution is to wrap both NM_ATT_SW_DESCR TLVs into a single NM_ATT_SW_CONFIG as nanoBTS does.

Why does the current implementation work with the old OpenBSC? Because it doesn't rely on abis_nm_tlv_parse(), and the parsing is done by sequential iteration over the raw buffer. Thus wrapping the NM_ATT_SW_DESCR blobs with NM_ATT_SW_CONFIG would break the compatibility.

I am not sure how can we keep the backwards compatibility, since the current approach is a rough spec. violation. Though, I can write a patch for OpenBSC too. laforge, how should I proceed?

#7 Updated by fixeria 4 months ago

  • Project changed from OsmoBSC to OsmoBTS
  • Category deleted (A-bis OML)
  • % Done changed from 60 to 90
  • Spec Reference set to 3GPP TS 52.021

Indeed, this is a bug of OsmoBTS. The problem was that a list of 'SW Descriptions' was not properly encapsulated into 'NM_ATT_SW_CONFIG' attribute. Please see: https://gerrit.osmocom.org/#/c/osmo-bts/+/13709/.

The question about backwards compatibility with old OpenBSC remains open.

#8 Updated by fixeria 3 months ago

  • Status changed from Feedback to Resolved
  • % Done changed from 90 to 100

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 48.8 MB)