Project

General

Profile

Feature #3246

osmo-gsm-tester: Support config resource modifiers in scenario files

Added by pespin 4 months ago. Updated about 6 hours ago.

Status:
Feedback
Priority:
Normal
Assignee:
Target version:
-
Start date:
05/08/2018
Due date:
% Done:

90%

Estimated time:
Spec Reference:

Description

From an e-mail of mine:

I have been looking into the osmo-gsm-tester stuff for a while today, and althrough having this kind of stuff in config files instead of using them programatically may be handy in some cases, I think it's not worth the amount of hours required to create that and I still think it's not entirely useful for more complex test cases.

What's clear from my side: there needs to be a 2 step process in any case:
1- selection/filtering of requested resources.
2- Config specific modifications/overlay for each requested resource.

The first one is the one already done by suite+scenario dictionary mangling.

Each state must handle their own data structure (dictionary), since the algorithm runs on the data structures merging/overlaying them, and the twoprocesses are exclusive one from each other on how they process the input
dictionaries.

For instance, if you add extra info (overlay modifications), during comined() (filtering) time it would fail later during match() because no resource would be found with exactly does values. So the selectors and the modifiers need to be stated clearly separated.

Then, as a result, we'd need some way to map selectors to modifiers in the sense: we want to apply this modifiers to this exact object resource with this selectors.

From "user" point of view, I guess one could expect a solution like having an extra section in the suite+scenario combination:
        resources:
          bts:
          - type: sysmo
          - ciphers:
            - a5_0
          modem:
          - ciphers:
            - a5_0

        modifiers:
          bts:
          - cipher:
            - a5_0

          modem: {}

So the current "resources" is the "selector/filter" part, and the modifiers is the modifier part which is overlayed on top of the config used for the resource object. However, it doesn't seem easy to pass this modifier for each object in the way we generate and reserve all resources and generate objects from there. Furthermore, this method forces to us move all objects to be resources. For instance, if we want to modify cipher parameter for a given msc, we need to define it in the suite and provide a resources in resources.conf so it can be allocated, etc. Another issue, is that this forces us to static configs from tests, instead of being able to play with features at runtime at some point.

Another solution would be to apply modifiers to "classes of objects" 
(bts_osmo_trx, bsc, msc, bsc_bts, etc.), like we do in defaults.conf. But this doesn't work for all complex cases, because we may want to have for instance 2 BTS or 2 BSC, each with a different set of configurations.

So all that being said, my plan is to keep using all the dictionary stuff as we do now for selection/filtering of required resource objects, and then add bits to the API so that the test can programatically configure several aspects of each object. I'm not willing to spend more time for now adding this kind of features which make the system more complex instead of going straight to add more tests, I think we already had to invest too much time in this kind of topics. If at some point anybody else wants to add more features like having "modifiers" through dictionaries, or we see ourselves that it's really more useful, we can look into adding them.

I think I'll move the registration of syspath python include dir inside suite.py to have the "lib" subdir of the suite being run added by default, and unregistering the path after the test is finished.
Then tests in the same suite can share code in any file under the lib subdir.

I think it'd probably make sense to support both approaches in the scenario files:
- Per object overlay config section (as per resource requested in the suite.conf file).
- defaults overlay section (which let's overlay on top of the "defaults" dictionary).

This way we can configure with per-object granularity in resources, and per object class in non-resource objects (like msc, esme, etc.)

History

#1 Updated by pespin about 6 hours ago

  • Status changed from New to Feedback
  • Assignee changed from osmo-gsm-tester to pespin
  • % Done changed from 0 to 90

Patches introducing support for modifiers and using them in:
https://gerrit.osmocom.org/#/c/osmo-gsm-tester/+/10562 Introduce scenario modifiers
https://gerrit.osmocom.org/#/c/osmo-gsm-tester/+/10563 Replace suites dyn_ts_ipa* and voice using scenario modifiers

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 48.8 MB)