Project

General

Profile

Support #4021

long-term end-to-end osmo-remsim test setup

Added by laforge 8 months ago. Updated 6 days ago.

Status:
In Progress
Priority:
High
Assignee:
Category:
tests
Target version:
-
Start date:
05/23/2019
Due date:
% Done:

0%


Description

let's build a setup consisting of:
  • 5-slot CCID card reader
  • sysmoQMOD with four Gobi2000 modems
    • four random SIM cards
  • APU2 with Debian9
  • one instance of remsim-server from osmocom:nightly feed running on the host
  • Debian 9 VM for remsim-bankd
    • USB port towards card reader routed to this VM
    • one instance of osmo-bankd from osmocom:nightly feed running
  • Debian 9 VM for remsim-client
    • USB port towards QMOD routed to this VM
    • four instances of osmo-remsmim from osmocom:nightly feed running

The host machine should run tc netem to simulate a non-zero latency, such as 200ms between client/server/bankd

All software should automatically start up after boot.


Checklist

  • CP2102 USB UARTs for QMOD

History

#1 Updated by roh 8 months ago

  • assembled hardware
  • configured host system for libvirt, remsim-server, sshd
  • moved to osmocom-network segment (vlan25)
  • added cardreader

#2 Updated by roh 8 months ago

done:
  • 2x kvm debian9 inside the debian9 on apu2 for remsimbank and remsimclient(s)
  • added nightly feeds to all debian
  • usb from acs reader routed into remsimbank vm
  • autostart vms
  • installed osmo-* daemons in respective vm
  • network segment for vms: 10.9.99.0/24 - .1 server .10 client .30 simbank
todo:
  • tc netem
  • systemd servicefiles for all daemons
  • qmod/modems
  • simcards

#3 Updated by roh 8 months ago

done:
  • systemd servicefiles for -bankd and -server
  • qmod/modems connected - may need to move to a different usb controller (issues pushing all parts of the device to vm)

#4 Updated by roh 8 months ago

done:
  • tested usb forwarding via complete seperate usb port - no go on apu due to hw limits.
  • swapped the vm for -client and host for -server due to usb/pci forwarding limitations (qmod needs to be on host)
  • systemd servicefile template (copy 4 times and configure for specific modems)

#5 Updated by roh 7 months ago

done:
  • systemd servicefile for osmo-remsim-client using proper templating (no copying servicefile around anymore), just add more configs.
todo:
  • netem
  • manual testing/verification

#6 Updated by roh 7 months ago

done:
  • updated all install/vm
  • the clients now have proper ids (-n 0-3)
  • basic netem setup (inmarsat-style 300ms / 150ms jitter / 50% corrlation / 512kBit)
    on network output of remsimserver-vm - do we need a more complicated setup here? it feels proper laggy to me.
todo:
  • real tests/load/traffic on all components (do we have a list of things to probe already?)
  • which testrunner to use?

#7 Updated by roh 7 months ago

#8 Updated by laforge 6 months ago

  • Priority changed from Normal to High

#9 Updated by roh 4 months ago

i think we need a bigger ssd for this machine. it ran out of disk twice now.

it has 2 images for the vm (4g each) and 4g swap, which results in about 500mb free when i delete all old logs.
i tried adding it to the monitoring but we have another blocking issue there. (#4664)

kevin got access a few days ago, so he can help get some traffic on those simcards.

how will this work exactly? jenkins will log in via ssh and key and trigger some script?

#10 Updated by roh 4 months ago

done:
  • swapped the simcards to sysmoUSIM.

#11 Updated by roh 3 months ago

done: * upgraded ssd to 240G, copied system, fixed uuid, resized / to 100Gb, grown swap to 16Gb, added remaining diskspace to /space

#12 Updated by laforge about 2 months ago

I cannot find any mention of the IP adresss of thee system. roh please add.

#13 Updated by roh about 2 months ago

its on 10.9.25.90/24

i noted it in https://osmocom.org/projects/osmo-remsim/wiki/Osmo-remsim#Test-Suite (only on the graphic)

#14 Updated by laforge about 1 month ago

  • Status changed from New to In Progress
  • Assignee changed from roh to laforge
  • % Done changed from 0 to 50

I'm currently upgrading the system and the VMs to buster. We shouldn't start to deploy new systems based on oldstable, it just means we need to migrate at some point sooner than neccessary.

#15 Updated by roh about 1 month ago

sorry, yes.
i think this was due to buster being released in july and the install happening beforehand.

#16 Updated by laforge about 1 month ago

  • Checklist item CP2102 USB UARTs for QMOD added
  • Assignee changed from laforge to roh
  • % Done changed from 50 to 0
we should extend this testbed as follows:
  • CP2102 USB UART for ST12 of QMOD
  • CP2102 USB UART for ST34 of QMOD

#17 Updated by roh 6 days ago

  • added a usb-hub (apu only has 2 ports and both were used)
  • added 2x cp2102
  • soldered pinheader
  • added 2.5mm to pinheader adapters.

it took me quite a while to get the osmo-remsim-client working again (the usb path changed obviously) and to figure out that the cp2102 i used were broken before or have problems with the high baudrate (looks like binary)

is there any (standard distro shipped) tool to display usb-paths besides lsusb.py?

packaging is now a cardboard box to contain the cp2102 and usbhub.

for some reason i cannot edit the checklist item (grayed out).

also somebody disabled the start of osmo-remsim-client@[0-3] for reasons i do not know.

#18 Updated by laforge 6 days ago

On Wed, Jan 15, 2020 at 07:54:18PM +0000, roh [REDMINE] wrote:

is there any (standard distro shipped) tool to display usb-paths besides lsusb.py?

I don't know, but in this specific use case simtrace2-list (part of simtrace2 host tools)
should work just fine.

also somebody disabled the start of osmo-remsim-client@[0-3] for reasons i do not know.

That was me while doing some manual testing

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 48.8 MB)