https://osmocom.org/https://osmocom.org/favicon.ico?16647414092018-07-04T09:12:51ZOpen Source Mobile CommunicationspySim - Bug #3376: create test scripts to verify pysim-prog ans sysmo-usim-tool with jenkinshttps://osmocom.org/issues/3376?journal_id=102102018-07-04T09:12:51Zdexter
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>In Progress</i></li><li><strong>% Done</strong> changed from <i>0</i> to <i>50</i></li></ul><p>We now have a bash script to verify pysim-prog.py. The script isself is generic. The user has to supply a config file for each card (also scripts but with variables only). The script can then be used to generage a .ok file with pysim-read.py. The cards are automatically probed (see also: <a class="external" href="https://gerrit.osmocom.org/#/c/pysim/+/9884">https://gerrit.osmocom.org/#/c/pysim/+/9884</a>), so one does not pay too much attention on which card goes into which reader. Also -re enumeration of the readers by pcscd will not hurt.</p> pySim - Bug #3376: create test scripts to verify pysim-prog ans sysmo-usim-tool with jenkinshttps://osmocom.org/issues/3376?journal_id=102112018-07-04T12:36:35Zlaforge
<ul></ul><p>dexter wrote:</p>
<blockquote>
<p>We now have a bash script to verify pysim-prog.py. The script isself is generic.</p>
</blockquote>
<p>great!</p>
<blockquote>
<p>The cards are automatically probed (see also: <a class="external" href="https://gerrit.osmocom.org/#/c/pysim/+/9884">https://gerrit.osmocom.org/#/c/pysim/+/9884</a>), so one does not pay too much attention on which card goes into which reader.</p>
</blockquote>
<p>But I presume we do make sure that all expected cards are actually found? We just don't care in whcih slot they are? I'm just asking as the absence of an expected card from the system should cause the test to fail.</p>
<blockquote>
<p>Also -re enumeration of the readers by pcscd will not hurt.</p>
</blockquote>
<p>great.</p> pySim - Bug #3376: create test scripts to verify pysim-prog ans sysmo-usim-tool with jenkinshttps://osmocom.org/issues/3376?journal_id=102122018-07-04T12:41:51Zlaforge
<ul><li><strong>Related to</strong> <i><a class="issue tracker-1 status-3 priority-2 priority-default closed" href="/issues/3377">Bug #3377</a>: physical setup for automatic pySim tests against different cards</i> added</li></ul> pySim - Bug #3376: create test scripts to verify pysim-prog ans sysmo-usim-tool with jenkinshttps://osmocom.org/issues/3376?journal_id=102412018-07-04T14:56:36Zdexter
<ul><li><strong>% Done</strong> changed from <i>50</i> to <i>80</i></li></ul><blockquote>
<p>But I presume we do make sure that all expected cards are actually found? We just don't care in whcih slot they are? I'm just asking as the absence of an expected card from the system should >cause the test to fail.</p>
</blockquote>
<p>Thats a good point. I have now added that. If it does not see each expected card once it will exit with 1. Also if a card becomes unresponsive the script will exit with 1.</p>
<p>For sysmo-usim-tool I now have a script that tries out each of the commandline options. I have set -e on here as well, so if sysmo-usim-tool detects an error it will fail. We also might consider to dump the output into an .ok file and diff afterwards. Then we also can verify the actual data contents as well.</p>
<p>All in all I think we have a good coverage now. I will coordninate the jenkins integration with lynxis when I see him next time.</p>
<p>(I also have extended sysmo-usim-tool a bit so that it senses on the ATR of the card. By this we make sure that we always pick the first available sysmo-usim-sjs1 even when various readers with various other cards are connected)</p> pySim - Bug #3376: create test scripts to verify pysim-prog ans sysmo-usim-tool with jenkinshttps://osmocom.org/issues/3376?journal_id=102772018-07-10T14:30:54Zdexter
<ul></ul><p>I have discussed the next steps with lynxis today. As a first step I have pushed the pysim-test.sh script I have created into review so it can checked out together with the repository and also be used by other developers if needed. Lynxis will take care of the jenkins stuff (contrib directory).</p>
<p>See also: <a class="external" href="https://gerrit.osmocom.org/#/c/pysim/+/9947">https://gerrit.osmocom.org/#/c/pysim/+/9947</a> tests: add test program to verify pysim-prog.py (and pysim-read.py)</p>
<p>I now negotiate with Roh to get a debian host where we can run the build slave on.</p>
<p>(Also I changed the script a bit so that it can be run from any location.)</p> pySim - Bug #3376: create test scripts to verify pysim-prog ans sysmo-usim-tool with jenkinshttps://osmocom.org/issues/3376?journal_id=103092018-07-16T07:49:31Zdexter
<ul></ul><p>Roh has set up the test host and connected the readers. Next step is try it on that test-host. I expect that there will be minor fixes required until it works fine with jenkins. We should do the tests with a private branch first and merge the gerrit changes when we are done with everything.</p> pySim - Bug #3376: create test scripts to verify pysim-prog ans sysmo-usim-tool with jenkinshttps://osmocom.org/issues/3376?journal_id=103362018-07-16T18:36:56Zlynxis
<ul></ul><p>I've setted up the jenkins jobs and ansible scripts.</p>
<p><a class="external" href="https://gerrit.osmocom.org/#/c/osmo-ci/+/10018/">https://gerrit.osmocom.org/#/c/osmo-ci/+/10018/</a><br /><a class="external" href="https://jenkins.osmocom.org/jenkins/job/pysim-hardware-test/">https://jenkins.osmocom.org/jenkins/job/pysim-hardware-test/</a></p>
<p><a class="user active" href="https://osmocom.org/users/15">dexter</a> can you take a look, why it is failing?</p> pySim - Bug #3376: create test scripts to verify pysim-prog ans sysmo-usim-tool with jenkinshttps://osmocom.org/issues/3376?journal_id=103732018-07-19T09:25:10Zdexter
<ul><li><strong>% Done</strong> changed from <i>80</i> to <i>90</i></li></ul><p>Status update: The patches that contain the test script are now in master. So far the tests work fine on pmaier/simtest. We now can switch to master for the tests as well.</p> pySim - Bug #3376: create test scripts to verify pysim-prog ans sysmo-usim-tool with jenkinshttps://osmocom.org/issues/3376?journal_id=103832018-07-19T22:32:01Zlynxis
<ul></ul><p>all changes are now in gerrit and awaiting review</p>
<p><a class="external" href="https://gerrit.osmocom.org/#/c/pysim/+/10069/">https://gerrit.osmocom.org/#/c/pysim/+/10069/</a><br /><a class="external" href="https://gerrit.osmocom.org/#/c/osmo-ci/+/10018/">https://gerrit.osmocom.org/#/c/osmo-ci/+/10018/</a></p> pySim - Bug #3376: create test scripts to verify pysim-prog ans sysmo-usim-tool with jenkinshttps://osmocom.org/issues/3376?journal_id=105322018-07-30T10:20:11Zdexter
<ul><li><strong>% Done</strong> changed from <i>90</i> to <i>80</i></li></ul><p>I have tried it out today. Pushing a change to gerrit and let it verify by jenkins forts fine. This means we are complete for pysim.</p>
<p>Now we need to see what we can do about sysmo-usim-tool. For this we should discuss tomorrow what we need there. We might have problems there since sysmo-usim-tool is on git.sysmocom.de.</p> pySim - Bug #3376: create test scripts to verify pysim-prog ans sysmo-usim-tool with jenkinshttps://osmocom.org/issues/3376?journal_id=105422018-07-30T13:50:07Zlaforge
<ul></ul><p>On Mon, Jul 30, 2018 at 10:20:12AM +0000, dexter [REDMINE] wrote:</p>
<blockquote>
<p>We might have problems there since sysmo-usim-tool is on git.sysmocom.de.</p>
</blockquote>
<p>Not sure why that would cause problems. git.sysmocom.de is a public git server,<br />just like git.osmocom.org. The build job is simply cloning the publicly readable<br />repository, it doesn't care from which server.</p> pySim - Bug #3376: create test scripts to verify pysim-prog ans sysmo-usim-tool with jenkinshttps://osmocom.org/issues/3376?journal_id=105972018-08-06T12:19:14Zdexter
<ul></ul><p>The repository for sysmo-usim-tool is not in gerrit. I do not know if this is a problem or not. Probably we can just trigger the testrun from jenkins directly after we have pushed something. Maybe lynxis can say more about that?</p> pySim - Bug #3376: create test scripts to verify pysim-prog ans sysmo-usim-tool with jenkinshttps://osmocom.org/issues/3376?journal_id=106042018-08-06T12:59:38Zlaforge
<ul></ul><p>On Mon, Aug 06, 2018 at 12:19:15PM +0000, dexter [REDMINE] wrote:</p>
<blockquote>
<p>The repository for sysmo-usim-tool is not in gerrit. I do not know if this is a problem or not.</p>
</blockquote>
<p>gerrit is only required if you want pre-merge tests before accepting a given patch. Those are the<br />"gerrit" jenkins jobs listed in <a class="external" href="https://jenkins.osmocom.org/jenkins/view/gerrit/">https://jenkins.osmocom.org/jenkins/view/gerrit/</a></p>
<blockquote>
<p>Probably we can just trigger the testrun from jenkins directly after we have pushed something. Maybe lynxis can say more about that?</p>
</blockquote>
<p>this is how all our "master" jenkins jobs in <a class="external" href="https://jenkins.osmocom.org/jenkins/view/master/">https://jenkins.osmocom.org/jenkins/view/master/</a> work.</p>
<p>"Build Trigger" is set to "Poll SCM" every 5 minutes. If any commits are detected at the time<br />of the poll, a new build is started.</p>
<p>There is nothing special required for this, other than a public readable git repo, like it is the<br />case for sysmo-usim-tool.</p> pySim - Bug #3376: create test scripts to verify pysim-prog ans sysmo-usim-tool with jenkinshttps://osmocom.org/issues/3376?journal_id=106582018-08-08T10:37:48Zdexter
<ul><li><strong>Assignee</strong> changed from <i>dexter</i> to <i>lynxis</i></li><li><strong>% Done</strong> changed from <i>80</i> to <i>90</i></li></ul><p>I have now fixed and extended the already existing unit-tests for sysmo-usim-tool. The repository is now updated and all tests are working. It does not matter in which state the test-card is, as long as the ADM1 key in test-data is correct it should work. The tests are executed by running tests/run-tests.</p>
<p>The ADM1 key that is currently in test-data matches the one of the card we have in our test setup so we can now integrate those tests with jenkins.</p> pySim - Bug #3376: create test scripts to verify pysim-prog ans sysmo-usim-tool with jenkinshttps://osmocom.org/issues/3376?journal_id=107512018-08-13T16:12:39Zlynxis
<ul><li><strong>Status</strong> changed from <i>In Progress</i> to <i>Stalled</i></li></ul><p>I'll work on this tomorrow together with dexter.</p> pySim - Bug #3376: create test scripts to verify pysim-prog ans sysmo-usim-tool with jenkinshttps://osmocom.org/issues/3376?journal_id=108782018-08-20T16:42:02Zdexter
<ul><li><strong>Status</strong> changed from <i>Stalled</i> to <i>In Progress</i></li></ul><p>I had a look at the sysmo-usim-tool test. It works fine, but the pysim tests seem to have problems. It looks just like a problem with gerrit. lets have a look tomorrow.</p> pySim - Bug #3376: create test scripts to verify pysim-prog ans sysmo-usim-tool with jenkinshttps://osmocom.org/issues/3376?journal_id=109872018-08-27T08:43:05Zdexter
<ul></ul><p>I have updated the the simtester now so that it also tests gr1. Everything looks good.</p>
<p>But I found it very confusing to find the right location to put the new .data and .ok files, here are the directories I have cycled trough:</p>
<p>simcard testdata is in:<br />/home/jenkins/workspace/gerrit-pysim/a1/default/a2/default/a3/default/a4/default/label/simtester/pysim-testdata</p>
<p>seems to be abandonned:<br />/home/jenkins/pysim-testdata<br />/home/jenkins/workspace/pysim-hardware-test/pysim-testdata<br />/home/jenkins/workspace/gerrit-pysim-hardware-test/pysim-testdata</p>
<p>not tried yet:<br />./workspace/gerrit-pysim/a1/default/a2/default/a3/default/a4/default/label/simtester/pysim-testdata/pysim-testdata/</p>
<p>I wonder what all those are for and why there are so many of them. Wouldn't be just one enough? If manual triggering requires a commit to gerrit this is no problem. The the only reason why I would want to trigger the test manually is when I make changes to the setup and then I can just do a commit to gerrit. This is also better because then we have more coverage when trying the whole process out.</p> pySim - Bug #3376: create test scripts to verify pysim-prog ans sysmo-usim-tool with jenkinshttps://osmocom.org/issues/3376?journal_id=170922020-01-08T22:50:14Zlaforge
<ul><li><strong>Status</strong> changed from <i>In Progress</i> to <i>Resolved</i></li><li><strong>Assignee</strong> changed from <i>lynxis</i> to <i>dexter</i></li><li><strong>% Done</strong> changed from <i>90</i> to <i>100</i></li></ul>