Project

General

Profile

Actions

Feature #5599

open

trxcon: integrate with osmo-trx-ms

Added by laforge 5 months ago. Updated 6 days ago.

Status:
Feedback
Priority:
High
Assignee:
Category:
trxcon
Target version:
Start date:
07/01/2022
Due date:
% Done:

80%

Resolution:
Spec Reference:
Tags:

Description

As was discussed with Harald and Eric, the TDMA scheduler from trxcon should be integrated into osmo-trx-ms for the sake of minimizing delays. This can be achieved by separating trxcon's major components into shared libraries, let's say: libosmobb-l1sched.so, libosmobb-l1ctl.so, and libosmobb-trxif.so (naming suggestions are welcome!).

Scheduler interface

Clock

  • trx_data_rx_cb()
  • sched_clck_handle()

Uplink path

  • trx_data_rx_cb()
  • sched_trx_handle_rx_burst()
  • lchan_handler(), e.g. rx_data_fn()
  • (... burst buffers ... )
  • sched_send_dt_ind()
  • l1ctl_tx_dt_ind()
  • l1ctl_link_send()

Downlink path

  • l1ctl_rx_dt_req()
  • sched_prim_init() & sched_prim_push()
  • (... prim queue ...)
  • sched_frame_clck_cb()
  • lchan_handler()
  • trx_if_tx_burst()

Files

osmotrx_ms_arch.png View osmotrx_ms_arch.png 35.4 KB laforge, 07/01/2022 06:21 AM

Checklist

  • Separate the scheduler into a library (lisched.a)
  • Introduce an abstract PHY interface
  • Make the clock mode configurable: timer driven vs PHY driven
  • PHYIF: Implement Ready-to-Receive API
  • PHYIF: Implement Ready-to-Send API

Related issues

Related to OsmocomBB - Feature #3761: Separate TDMA scheduler into a library (libtdmasched)Resolvedfixeria01/20/2019

Actions
Actions #1

Updated by laforge 5 months ago

  • Related to Feature #3761: Separate TDMA scheduler into a library (libtdmasched) added
Actions #2

Updated by fixeria 5 months ago

  • Subject changed from trxcon: Separate TDMA scheduler into a library to trxcon: integrate with osmo-trx-ms
Actions #3

Updated by Hoernchen 5 months ago

Additonally, the current scheduler is kind of timer driven, instead of being driven by rx frames or some sort of fn/tn indication, which also causes jitter, and the old osmo-bts/trx split approach basically duplicates channel layout knowledge, the trxcon scheduler knows which fn and tn to handle, so there is no reason to duplicate the decision logic/channel layout for the lower parts.

Actions #4

Updated by fixeria about 1 month ago

  • Checklist item Separate the scheduler into a library (lisched.a) added
  • Checklist item Introduce an abstract PHY interface added
  • Checklist item Make the clock mode configurable: timer driven vs PHY driven added
  • % Done changed from 0 to 60

Most notable changes:

https://gerrit.osmocom.org/c/osmocom-bb/+/28555 trxcon: separate the scheduler into libl1sched.la
https://gerrit.osmocom.org/c/osmocom-bb/+/29868 trxcon: implement an abstract PHYIF API
https://gerrit.osmocom.org/c/osmocom-bb/+/28809 trxcon: rework trxcon_fsm, move into a separate file

Actions #5

Updated by laforge about 1 month ago

  • Priority changed from Normal to High

Adding @Hoernschen as a watcher here.

As per our review call today, this is currently the highest priority task within the ARDC/DARC project about the SDR-MS.

Please collaborate towards replacing the copy+paste old trxcon code from the osmo-trx-ms codebase with a git submodule of this new "linkable" trxcon code.

Actions #6

Updated by fixeria about 1 month ago

Hoernchen wrote in #note-3:

Additonally, the current scheduler is kind of timer driven, instead of being driven by rx frames or some sort of fn/tn indication, which also causes jitter, and the old osmo-bts/trx split approach basically duplicates channel layout knowledge, the trxcon scheduler knows which fn and tn to handle, so there is no reason to duplicate the decision logic/channel layout for the lower parts.

Here is a very first step towards this goal:

https://gerrit.osmocom.org/c/osmocom-bb/+/29954 trxcon: drop sched->clock_cb(), expose l1sched_trigger()

Actions #7

Updated by fixeria 24 days ago

  • % Done changed from 60 to 80

https://gerrit.osmocom.org/c/osmocom-bb/+/30095 trxcon: cosmetic: rename trxcon.c to trxcon_main.c [NEW]
https://gerrit.osmocom.org/c/osmocom-bb/+/30096 trxcon: cosmetic: s/trx_log_init()/trxcon_logging_init()/ [NEW]
https://gerrit.osmocom.org/c/osmocom-bb/+/30097 trxcon: move l1sched_logging_init() from l1sched.h to logging.h [NEW]
https://gerrit.osmocom.org/c/osmocom-bb/+/30098 trxcon: move trxcon_inst_{alloc,free}() to a separate file [NEW]
https://gerrit.osmocom.org/c/osmocom-bb/+/30099 trxcon: use L1CTL_HEADROOM in l1ctl_alloc_msg() [NEW]
https://gerrit.osmocom.org/c/osmocom-bb/+/30100 trxcon: abstract L1CTL codec from the l1ctl_server API [NEW]
https://gerrit.osmocom.org/c/osmocom-bb/+/30101 trxcon: reorganize the core logic into libtrxcon.la [NEW]
https://gerrit.osmocom.org/c/osmocom-bb/+/30102 trxcon: use 'osmo_trxcon_' prefix for public libtrxcon API [NEW]
https://gerrit.osmocom.org/c/osmocom-bb/+/30103 trxcon: move shim l1sched/phyif API to libtrxcon.la [NEW]
https://gerrit.osmocom.org/c/osmocom-bb/+/30104 trxcon: drop unneeded imports in trxcon_main.c [NEW]
https://gerrit.osmocom.org/c/osmocom-bb/+/30105 trxcon: turn libtrxcon into shared library libosmo-trxcon [NEW]

Actions #8

Updated by Hoernchen 20 days ago

Still missing:
  • a way to "be" the trxcon main loop
  • the interface to access the scheduler for the one trx instance (itself) that matters to the ms-transceiver to check if the ts is active
  • bidirectional calls into the transceiver to submit a tx burst, and to push a rx burst up after handling it
  • same approach for ctrl message cmd/response

See https://gitea.osmocom.org/cellular-infrastructure/osmo-trx/commit/2ecd9f698f90b9d00612384008bec08c14ed2b1d?style=split&whitespace=show-all for the current hack that shows the changes that are required (based on the OLD libosmocore, ctrl missing, still handled using sockets)

Actions #9

Updated by fixeria 18 days ago

I am currently waiting for code review on some patches (see #note-7). Once merged, you can proceed with the integration as follows:

General

  • Allocate a struct trxcon_inst using trxcon_inst_alloc(), then set both trxcon->phyif and trxcon->l2if.
    • Optionally set trxcon->gsmtap if you want osmo-trx-ms originated GSMTAP.

PHYIF abstraction

Related commits:

https://cgit.osmocom.org/osmocom-bb/commit/src/host/trxcon?id=c6324f89520fc8a003e091cbe17508c1aa67c65b
https://cgit.osmocom.org/osmocom-bb/commit/src/host/trxcon?id=5fbc7e50a9301cfc23ccb8bf85eccd0789f83a18

  • Implement the following PHYIF shim functions (see how trxcon_main.c does this):
int trxcon_phyif_handle_burst_req(void *phyif, const struct trxcon_phyif_burst_req *br);
int trxcon_phyif_handle_cmd(void *phyif, const struct trxcon_phyif_cmd *cmd);
void trxcon_phyif_close(void *phyif);
  • From your PHYIF implementation call the following functions (pass struct trxcon_inst * as priv):
int trxcon_phyif_handle_burst_ind(void *priv, const struct trxcon_phyif_burst_ind *bi);
int trxcon_phyif_handle_rsp(void *priv, const struct trxcon_phyif_rsp *rsp);

L1CTL interface

I guess you want your own L1CTL server implementation, which does not require osmo_select_loop()?
Currently libtrxcon intentionally does not include the L1CTL server and provides an abstract API.

https://cgit.osmocom.org/osmocom-bb/commit/src/host/trxcon?id=394793b1c1596d0c788d105ec874a67b8024fcd4

  • Implement the following L1CTL shim functions (see how trxcon_main.c does this):
int trxcon_l1ctl_send(struct trxcon_inst *trxcon, struct msgb *msg);
void trxcon_l1ctl_close(struct trxcon_inst *trxcon);
  • Call the following function from your L1CTL server implementation:
int trxcon_l1ctl_receive(struct trxcon_inst *trxcon, struct msgb *msg);
Actions #10

Updated by fixeria 18 days ago

  • Checklist item PHYIF: Implement Ready-to-Receive API added
  • Checklist item PHYIF: Implement Ready-to-Send API added

Hoernchen wrote in #note-8:

Still missing:
  • a way to "be" the trxcon main loop

With the abstract L1CTL codec interface, you can write your own L1CTL server which does not require osmo_select_loop(). Other parts of the libtrxcon should not require osmo_select_loop(), except the trxcon_fsm timers: they'll be set by fsm.c but apparently will never fire. We also still have "timer driven Uplink" in the l1sched, but I am already working on this.

  • the interface to access the scheduler for the one trx instance (itself) that matters to the ms-transceiver to check if the ts is active

By analogy with Ready-to-Send, let's call it Ready-to-Receive. I am working on it.

  • bidirectional calls into the transceiver to submit a tx burst, and to push a rx burst up after handling it

For Downlink BURST.ind simply call trxcon_phyif_handle_burst_ind().
For Uplink BURST.req the libtrxcon calls trxcon_phyif_handle_burst_req(), so you need to implement it.
What's missing is the Ready-to-Send API, which will trigger the scheduler (so no CLCK timer is needed anymore). I am on it.

  • same approach for ctrl message cmd/response

As I said in #note-9, you need to implement the control command handler trxcon_phyif_handle_cmd().

Actions #11

Updated by fixeria 17 days ago

  • Checklist item Make the clock mode configurable: timer driven vs PHY driven set to Done
  • Checklist item PHYIF: Implement Ready-to-Send API set to Done

Here is the proposed implementation for the Ready-to-Send:

https://gerrit.osmocom.org/c/osmocom-bb/+/30243 trxcon: separate l1sched_clck_handle() from BURST.ind handling [NEW]
https://gerrit.osmocom.org/c/osmocom-bb/+/30238 trxcon: make l1sched_trigger() accept the [advanced] TDMA Fn [NEW]
https://gerrit.osmocom.org/c/osmocom-bb/+/30239 trxcon: rework l1sched_trigger(), split l1sched_trigger_all() [NEW]
https://gerrit.osmocom.org/c/osmocom-bb/+/30240 trxcon: adjust coding style in l1sched_trigger() [NEW]
https://gerrit.osmocom.org/c/osmocom-bb/+/30241 trxcon: implement Ready-to-Send PHYIF API [NEW]
https://gerrit.osmocom.org/c/osmocom-bb/+/30242 trxcon: rx_sch_fn(): do not use sched->fn_counter_proc [NEW]

Hoernchen: The PHY needs to call the trxcon_phyif_handle_rts_ind() for each TDMA Fn/Tn. This function in its turn would trigger the scheduler to emit an Uplink burst (or NOPE.req with burst_len=0) and send it back to the PHY via the trxcon_phyif_handle_burst_req(). This way you can bypass the internal timer-driven Uplink scheduler and pull Uplink bursts synchronously whenever you need them.

Actions #12

Updated by fixeria 15 days ago

  • Checklist item PHYIF: Implement Ready-to-Receive API set to Done

https://gerrit.osmocom.org/c/osmocom-bb/+/30262 trxcon: implement Ready-to-Receive PHYIF API [NEW]

Actions #13

Updated by fixeria 15 days ago

  • Status changed from In Progress to Feedback
  • Assignee changed from fixeria to Hoernchen

I am done implementing what was requested, assigning to Eric for feedback/review.
Please do code/API review, your input is very welcome.

Actions #14

Updated by Hoernchen 12 days ago

  • Assignee changed from Hoernchen to fixeria

There is a problem: the structs don't really map to the "old" strings, and are less generic:


struct trxcon_phyif_cmdp_setfreq_h0 {
    uint16_t band_arfcn;
};

implies arfcns, but the goal here is random frequencies not related to the usual arfcns at all, so the transceiver currently can't tune. Both rx and tx need to be able to be independently tuned by supplying frequencies, not arfcns.

This is also true for

struct trxcon_phyif_cmdp_measure {
    uint16_t band_arfcn;
};

/* param of TRXCON_PHYIF_CMDT_MEASURE (response) */
struct trxcon_phyif_rspp_measure {
    uint16_t band_arfcn;
    int dbm;
};
Actions #15

Updated by fixeria 12 days ago

  • Assignee changed from fixeria to Hoernchen

Hoernchen wrote in #note-14:

There is a problem: the structs don't really map to the "old" strings, and are less generic:
[...]

implies arfcns, but the goal here is random frequencies not related to the usual arfcns at all, so the transceiver currently can't tune. Both rx and tx need to be able to be independently tuned by supplying frequencies, not arfcns.

For the record, this was discussed yesterday on the IRC:

04:43 < fixeria> I understand your point regarding arbitrary frequencies, but I would like to propose slightly different approach
04:45 < Hoernchen_> well i can just handle it manually in my config we just have to come up with a mapping of "my secret frequency" to "your made up arfcn"+
04:45 < Hoernchen_> it affects the whole stack of everyhting anyway, nothing works using frequencies, everything expects arfcns
04:46 < fixeria> indeed, you have to use ARFCNs in the L1CTL PDUs anyway
04:47 < fixeria> we already have 'freq-offset' parameter in osmo-trx, are you aware of it?
04:47 < Hoernchen_> no, this needs some additional config anyway
04:48 < fixeria> we could have different offsets for UL and DL if needed
04:48 < Hoernchen_> there is no point expressing it as offset
04:48 < Hoernchen_> as long as both sides agree what the afcn "should" be it's fine
04:49 < fixeria> why not?  you simply shift the whole band to whatever freq. band you want using the same offset on both MS and BTS sides and that's it
04:50 < Hoernchen_> because then you need to calculcate offsets to configure it properly which is... not great.
04:50 < fixeria> what do you suggest then?
04:51 < Hoernchen_> actual frequencies that override all messages that might try to modify it
04:51 < Hoernchen_> on both sides
04:52 < Hoernchen_> since i'm in charge of the whole trxcon "app" anyway I can just add config options as i see fit anyway
04:52 < Hoernchen_> and osmotrx can also just have two new settings, we do already have weird delay stuff anyway
04:53 < Hoernchen_> it's not a big deal
04:53 < fixeria> well, yeah, do whatever you want with the ARFCNs you get over the PHYIF :)
04:54 < fixeria> so, is there anything I should do in https://osmocom.org/issues/5599?  asking because you assigned it back to me
04:55 < Hoernchen_> no not really i just wanted to point out that we have implicit arfcns that do not map to the actual frequencies

We used to do ARFCN/freq. mapping in trxcon, but now it's a task of the PHY implementation. What Hoernchen describes is a specific requirement of the ARDC/DARC: running both MS and BTS on non-standard frequencies. I see no problem with doing the mapping in osmo-trx-ms, you can make it configurable via the VTY. Assigning back to you.

Actions #16

Updated by fixeria 11 days ago

Eric has uploaded the state-of-art osmo-trx-ms:

https://cgit.osmocom.org/osmo-trx/log/?h=mstx_newtrxcon

In PM he told me he's having a problem with the RTR and Uplink burst handling. According to him, the scheduler complains about bursts received for disabled timeslots. I could not reproduce this with hacked fake_trx.py so far. The only potentially causing this thing I noticed so far is how the actual RTR.ind is sent:

@@ -128,29 +162,32 @@ SoftVector *upper_trx::pullRadioVector(GSM::Time &wTime, int &RSSI, int &timingO
        const auto is_sch = gsm_sch_check_ts(burst_time.TN(), burst_time.FN());
        const auto is_fcch = gsm_fcch_check_ts(burst_time.TN(), burst_time.FN());

+       // trxcon::trxcon_phyif_rtr_ind i = { static_cast<uint32_t>(burst_time.FN()),
+       //                                 static_cast<uint8_t>(burst_time.TN()) };
+       // trxcon::trxcon_phyif_rtr_rsp r;
+       // trxcon_phyif_handle_rtr_ind(trxcon::g_trxcon, &i, &r);
+       // if (!(r.flags & TRXCON_PHYIF_RTR_F_ACTIVE))
+       //      return false;

Note that the RTR.rsp (trxcon::trxcon_phyif_rtr_rsp) is not initialized here. I am not 100% sure, but I doubt C++ does implicit struct initialization. You may need to zero-initialize it. From the API usability point of view, it might be more handy if trxcon_phyif_handle_rtr_ind() would do initialization for the caller?

Actions #17

Updated by fixeria 11 days ago

BTW, I just stumbled upon Figure 5.1 in 3GPP TS 44.004, defining the L1 PHY FSM. The trxcon_fsm I implemented recently is pretty much what 3GPP defines, with a few insignificant differences (e.g. separate states for DCCH and PDCH, separate state for power scanning). Looking at the reference FSM state diagram I submitted a few patches with the improvements for the trxcon_fsm:

https://gerrit.osmocom.org/c/osmocom-bb/+/30309 trxcon: add a spec. reference for the trxcon_fsm
https://gerrit.osmocom.org/c/osmocom-bb/+/30315 trxcon: trxcon_fsm: disallow invalid state transitions [NEW]
https://gerrit.osmocom.org/c/osmocom-bb/+/30316 trxcon: trxcon_fsm: shorten DCCH related event names [NEW]
https://gerrit.osmocom.org/c/osmocom-bb/+/30317 trxcon: trxcon_fsm: separate handling of TRXCON_EV_DCCH_EST_REQ [NEW]
https://gerrit.osmocom.org/c/osmocom-bb/+/30318 trxcon: trxcon_fsm: permit loop DCH transitions [NEW]
https://gerrit.osmocom.org/c/osmocom-bb/+/30319 trxcon: trxcon_fsm: permit direct {TCH,DCCH}/PDCH transitions [NEW]

Actions #18

Updated by fixeria 10 days ago

fixeria wrote in #note-16:

Eric has uploaded the state-of-art osmo-trx-ms:

https://cgit.osmocom.org/osmo-trx/log/?h=mstx_newtrxcon

In PM he told me he's having a problem with the RTR and Uplink burst handling. According to him, the scheduler complains about bursts received for disabled timeslots. I could not reproduce this with hacked fake_trx.py so far.

I think I found a more serious problem in 9b6db11ee7fcc17a7960ea1a41ebdd69957c3bae (current HEAD):

+       if (pullRadioVector(burstTime, RSSI, TOA)) {
+               // trxcon::trx_data_rx_handler(trxcon::trxcon_instance, (uint8_t *)&response);
+               trxcon::trxcon_phyif_burst_ind bi;
+               bi.fn = htonl(burstTime.FN());
+               bi.tn = burstTime.TN();
+               bi.rssi = RSSI;
+               bi.toa256 = htons(TOA);
+               bi.burst = (sbit_t *)demodded_softbits;
+               bi.burst_len = sizeof(demodded_softbits);
+               // trxcon_phyif_handle_clock_ind(trxcon::g_trxcon, bi.fn);
+               trxcon_phyif_handle_burst_ind(trxcon::g_trxcon, &bi);

You are converting between the host and network byte order when using the PHYIF API. This is not required and is actually wrong, because libtrxcon is using the native byte order internally. This is only required when communicating with other processes over L1CTL or TRXD. Apparently increasing the trxcon's logging levels would have shown weird Fn/Tn numbers? Anyway, you need to remove all hton[sl]/ntoh[sl] calls.

P.S. It would have been a lot easier to comment on you patches if they were in Gerrit.

Actions #19

Updated by fixeria 8 days ago

All patches have been merged, my part of this ticket is done.

Actions #20

Updated by fixeria 7 days ago

There was an issue reported by Hoernchen about an endless make loop when trying to build trxcon as a submodule. Here is a single-line patch fixing the problem:

https://cgit.osmocom.org/osmo-trx/commit/?h=fixeria/ms&id=02f65668559f88c0c715e43728b794b028a9f844

Unfortunately I am still unable to build osmo-trx-ms, specifically the UHD variant. The linker fails with:

/usr/bin/ld: ms/osmo_trx_ms_uhd-ms_rx_lower.o: in function `detect_burst(std::complex<float> const*, std::complex<float>*, int, char*) [clone .resolver]':
/home/fixeria/projects/osmocom/osmo-trx/Transceiver52M/ms/ms_rx_lower.cpp:344: undefined reference to `detect_burst(std::complex<float> const*, std::complex<float>*, int, char*) [clone .avx]'
/usr/bin/ld: /home/fixeria/projects/osmocom/osmo-trx/Transceiver52M/ms/ms_rx_lower.cpp:344: undefined reference to `detect_burst(std::complex<float> const*, std::complex<float>*, int, char*) [clone .sse4_2]'
/usr/bin/ld: /home/fixeria/projects/osmocom/osmo-trx/Transceiver52M/ms/ms_rx_lower.cpp:344: undefined reference to `detect_burst(std::complex<float> const*, std::complex<float>*, int, char*) [clone .sse3]'
/usr/bin/ld: /home/fixeria/projects/osmocom/osmo-trx/Transceiver52M/ms/ms_rx_lower.cpp:344: undefined reference to `detect_burst(std::complex<float> const*, std::complex<float>*, int, char*) [clone .sse2]'
/usr/bin/ld: /home/fixeria/projects/osmocom/osmo-trx/Transceiver52M/ms/ms_rx_lower.cpp:344: undefined reference to `detect_burst(std::complex<float> const*, std::complex<float>*, int, char*) [clone .sse]'
/usr/bin/ld: /home/fixeria/projects/osmocom/osmo-trx/Transceiver52M/ms/ms_rx_lower.cpp:344: undefined reference to `detect_burst(std::complex<float> const*, std::complex<float>*, int, char*) [clone .default]'
collect2: error: ld returned 1 exit status
make[3]: *** [Makefile:920: osmo-trx-ms-uhd] Error 1

I am giving up. No luck even after fixing stuff here and there. See other patches in:

https://cgit.osmocom.org/osmo-trx/log/?h=fixeria/ms

Actions #21

Updated by fixeria 7 days ago

fixeria wrote in #note-20:

Unfortunately I am still unable to build osmo-trx-ms, specifically the UHD variant. The linker fails with: [...]

Actually, building with CC=clang CXX=clang++ works. Looks like GCC is not happy about something?

Actions #22

Updated by laforge 7 days ago

On Wed, Nov 30, 2022 at 10:28:09AM +0000, fixeria wrote:

Actually, building with CC=clang CXX=clang++ works. Looks like GCC is not happy about something?

let me clarify that the standard C/C++ compiler in Osmocom is gcc/g++.
If it also builds with other compilers, fine, that's most welcome in
addition
to gcc All our infrastructure for build validation, etc. is
runnig gcc, which should be generally known.

Actions #23

Updated by Hoernchen 6 days ago

Those are leftover debug macros related to xray tracing that will be removed as soon as it's pushed to gerrit.

Actions

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 48.8 MB)