osmo-gsm-tester: allow running binaries for a whole suite, or only for a single test
There are two different use cases for reserving resources / running programs, for example:
a) A given test script launches an osmo-nitb which should be stopped/torn down before the next test script starts.
b) In another suite that contains a number of tests, an earlier test script may launch osmo-nitb and bts processes, which should stay running until the entire suite is through. This is not necessarily the first test script and it should not necessarily have to run until the end of the suite, it should be up to the test scripts to start/end processes / allocate/free resources at will.
Add test API to distinguish between the two cases: should a resource be freed / program be terminated at the end of the test?
This should be generically similar for processes run or "passive" resources allocated. Details follow.
Make sure that at the end of a suite run, all those programs/resources not torn down by the tests are definitely cleaned up and resources are freed.
Also free resoures / terminate processes that were not requested to last beyond the current test script at the end of each test script automatically.
Think of a way how a later test script may obtain a resource allocated / process launched in an earlier test script, without causing a new resource to be allocated (and probably failing because the earlier test already launched it).
The way to keep a resource should be tied to a name given, used by a later test script to look it up. For example:
01_prepare.py: nitb = suite.nitb() nitb.start() suite.keep(nitb, 'nitb1') 02_use.py: nitb = suite.earlier('nitb1') 03_restart.py: nitb = suite.earlier('nitb1') nitb.stop() nitb.start()
This should work the same for nitb, bts, modems, ...
Right now we assume that each test script is independent on its own. Hence we should automatically terminate processes and un-use resources for each test script that is done.
Later on we can add a feature to let certain resources survive across several test scripts. But I am not entirely sure that we will really use this.
Related: the idea so far is to reserve resources for an entire suite; so a given suite is intended to be a unit in terms of what it uses, which makes sense when several test scripts use the same core network processes. With each test being independent on its own this idea doesn't really deliver, and it might also make sense to change this.
We shall keep this in mind as we further develop tests. When more complex test setups arise, we will hopefully see quite easily how this should work. At the moment it's neither: we don't have automatic cleanup after each test script, but we also can't re-use a process in a subsequent test script.