SIM card related Projects » pySim

dexter what about this, it has been 3 months? at least a one-line input/opinion from you would be much appreciated.

I had a look at it, first of all the current master of pysim and sysmo-usim-tool test both fine.

From time to time we see problems when sysmo-usim-tool does changes to the file system which then cause the tests of pysim to fail and vice versa. At least in my opinion this wasn't to much of a problem yet, but I agree, the way we test is a bit clumsy.

When I get you right you want to use the APIs provided by pysim to check on the contents of the files and implement those checks as python unit-tests? Maybe we could also make pySim-read.py a bit more scripting friendly. Maybe have commandline arguments to specify which file/parameter to read out. Then we would also have some kind of separation and at the same time we woule improve pySim-read.py so that it can be used from scripts.

One more change related to this ticket: https://gerrit.osmocom.org/c/pysim/+/18073.

I'm sorry, but this is starting to seriously impede the development of pySim.

From my point of view, what is needed is to properly reset the system to a sane state after executing tests. "system" here includes the contents of the SIM cards.

• for user-writable files on all card types
• for ADM1-writable files on all card types

If some files cannot be written even with ADM1, this shouldn't be a problem - it is safe to assume that any [future] patch we test will then also not be able to write those files.

The resetting of the file system contents should take place before the test executes. My idea is to use the erase() method in cards.py. I think at the moment this method is only programmed for one card. What I plan to do is to reset the contents of the files that pysim writes to 0xff, so that they are basically uninitalized. We can do this with a separate tool pySim-erase.py, this way we do not have to touch pySim-prog.py. The erase() method would use functions that do a read-after-write verification of the file contents, so that we can be sure that the card really has been erased. I already have a draft implementation of pySim-erase.py and some methods for the cards class that can do an erase of binary files and records in one line, so filling the erase() method for each card should be easy. For sysmo-isim-sjs2 I already filled the erase() method.

The following patches are in review now:
https://gerrit.osmocom.org/c/pysim/+/18205 cards: remove len calculation
https://gerrit.osmocom.org/c/pysim/+/18206 cards: reset uninitalized EF.AD
https://gerrit.osmocom.org/c/pysim/+/18207 pysim-prog: move ADM sanitation to utils.py
https://gerrit.osmocom.org/c/pysim/+/18208 utils: do not crash when all bytes of EF.IMSI are 0xFF
https://gerrit.osmocom.org/c/pysim/+/18209 commands: add method to determine size of a non record oriented file
https://gerrit.osmocom.org/c/pysim/+/18211 commands: add features to verify data written to files

dexter wrote:

The resetting of the file system contents should take place before the test executes. My idea is to use the erase() method in cards.py. I think at the moment this method is only programmed for one card. What I plan to do is to reset the contents of the files that pysim writes to 0xff, so that they are basically uninitalized.

The problem is that this will probably break once somebody submits code for a new file to gerrit, which is not yet understood by pysim master. This way the card contents changes juts because we tried to verify any random submitted but not yet approved patch.

Hence I think it would be better to really do a full dump / backup of all files on each of the cards once, and then restore that backup before each test run. The "Backup" part is basically what "osmo-sim-test -o" from libosmocore/utils is generating. Maybe all that's needed is the inverse part, i.e. restoring such a file/directory based backup?

We can do this with a separate tool pySim-erase.py, this way we do not have to touch pySim-prog.py.

I agree this test pre-setup should be kept out of pySim-prog. Whether it's C code or python, I don't really have a preference.

The erase() method would use functions that do a read-after-write verification of the file contents, so that we can be sure that the card really has been erased.

If you want to stick with erase rather than restore, you should make sure to generate a list of all files/paths on each of the cards once, and then erase all their contents [assuming one can do this with ADM1 pin on all relevant cards]

The build verification is broken again. It looks like one of the recent pySim patches has changed something on the test card(s):

https://gerrit.osmocom.org/c/pysim/+/23631 Use zero padding for EF['ACC'] field
https://gerrit.osmocom.org/c/pysim/+/23632 Use zero padding for EF['ACC'] field

I tried to upload another patch that does not touch the actual parsing logic, and it fails too:

https://gerrit.osmocom.org/c/pysim/+/23633 Py2 -> Py3: do not inherit classes from object

Hello fixeria

There is indeed something changed on the card:
Administrative data: 00000002
MS operation mode: normal operation

is now:

Administrative data: 80000002
MS operation mode: type approval operations

This is a bit odd, at least the patches I added recently do not touch this field. I will take care of this after lunch.

I have now fixed the problem by resetting EF.AD to its original state.