Bug #2344

OsmoTRX is not using SCHED_RR

Added by laforge over 4 years ago. Updated about 2 years ago.

Target version:
Start date:
Due date:
% Done:


Spec Reference:


It's quite odd that OsmoTRX, being real-time critical is not using the SCHED_RR scheduling policy (see, similar to what OsmoBTS is using.

osmo-trx-sched_rr.diff osmo-trx-sched_rr.diff 595 Bytes laforge, 06/29/2017 01:23 PM

Related issues

Related to OsmoBTS - Bug #2325: sporadic shutdown of osmo-bts-trx in osmo-gsm-tester runs (no clock from trx)Resolved06/13/2017

Associated revisions

Revision 81486e05 (diff)
Added by laforge about 4 years ago

Add '-t' command line option to enable SCHED_RR

SCHED_RR allows us to operate osmo-trx reliable even under exceptionally
high system load, as the realtime scheduler priority will have higher
priority than the other "regular" tasks on the system.

Change-Id: Ia2452b9763960b2be37fbeee9d832554da68a53f
Closes: OS#2344

Revision 67aa91b2 (diff)
Added by pespin about 2 years ago

Drop old setPriority related code

This code is not needed anymore since we are setting SCHED_RR scheduler
with a real time priority in main thread during startup, so all threads
will inherit same rt priority, which should be enough to keep the
process working reliably even on high system loads (from non rt

osmo-trx was tested to be reliable during test with stress-ng as
explained in related ticket below.

Related: OS#2344
Change-Id: I3a88946dd71e9aeeaac9d19d396e2236c302b608


#1 Updated by laforge over 4 years ago

  • Status changed from New to In Progress

#2 Updated by laforge over 4 years ago

  • Related to Bug #2325: sporadic shutdown of osmo-bts-trx in osmo-gsm-tester runs (no clock from trx) added

#3 Updated by laforge over 4 years ago

  • % Done changed from 0 to 20

Using a quick exerimental patch to enable SCHED_RR, I can get osmo-trx and osmo-bts-trx to run on a system with load > 100, created by the following stress-ng command line:
stress-ng --vm 10 --hdd 10 --cpu 10 --open 10 --sem 10 --sock 10 --float 10 --io 10 --timer 10

With my SCHED_RR patch, osmo-trx and osmo-bts-trx are running perfectly fine, despite the high system load. Prior to the SCHED_RR patch, osmo-trx basically immediately throws errors like

NOTICE 140480399431424 15:19:43.5 Transceiver.cpp:381:pushRadioVector: dumping STALE burst in TRX->USRP interface

and osmo-bts-trx shows high clock deviation like

<0006> scheduler_trx.c:1704 We were 16 FN faster than TRX, compensating

so definitely, using SCHED_RR significantly improves performance under high system load.

#4 Updated by laforge over 4 years ago

attaching patch for unconditional use of SCHED_RR.

#5 Updated by laforge over 4 years ago

  • % Done changed from 40 to 80

proper patch now in

However, I think we should actually make it a default.

#6 Updated by ttsou over 4 years ago

OsmoTRX already uses SCHED_RR by default, however, Harald's positive result confirms a counterintuitive result. Instead of setting individual RT priorities on the primary I/O threads (as below), better results are obtained by calling sched_setscheduler() from main(). The main() thread simply loops on sleep() during operation, which is why priorities were not setup this way before.

The likely reason for the improved behavior is that the libusb event handler, which runs below osmo-trx was not being prioritized before. The scheduling setting in main() prioritizes UHD threads and likely prevents osmo-trx from preempting libusb and UHD which both run in userspace.

I've seen similar improvements in higher bandwidth LTE test cases, but never reached the same conclusion in osmo-trx.

void RxLowerLoopAdapter(Transceiver *transceiver)

  while (1) {
  return NULL;

#7 Updated by ttsou over 4 years ago

Can you remove or disable the calls to RadioDevice::setPriority() and still maintain the same performance?

The current settings set priorities through the UHD scheduling wrapper, which then enabled SCHED_RR and priorities by calling pthread_setschedparam(). I'm not sure these calls have a significant effect when RT scheduling is already configured by the parent thread.

I also prefer not to have separate scheduling implementations that could potentially conflict.

#8 Updated by laforge almost 4 years ago

  • Status changed from In Progress to Stalled

#9 Updated by laforge over 3 years ago

  • Assignee changed from laforge to pespin

#10 Updated by msuraev almost 3 years ago

On somewhat related note: I've noticed that when I run osmo-trx manually on my system as a user, it fails to set SCHED_RR but running it with systemctl --user it works fine. I've got system.conf and user.conf altered to set proper DefaultLimitRTPRIO.

#11 Updated by pespin about 2 years ago

  • Status changed from Stalled to Feedback

Related patch submitted in Drop old setPriority related code

Once merged this ticket can be closed.

#12 Updated by pespin about 2 years ago

  • Status changed from Feedback to Resolved
  • % Done changed from 80 to 100

Merged, closing.

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 48.8 MB)