Project

General

Profile

Actions

Bug #3068

open

osmo-gsm-tester: resource objects leaked or destroyed too late

Added by pespin about 6 years ago. Updated 12 months ago.

Status:
New
Priority:
Low
Assignee:
Target version:
-
Start date:
03/15/2018
Due date:
% Done:

0%

Spec Reference:

Description

It was noticed during development of nanoBTS support that some objects are being kept alive after suite.objects_cleanup() is called.

It was noticed because nanobts keeps a PowerSupplySispm which in turn used to keep (changed now) a usbdevice object from libusb which in turn kept the usb device "reclaimed" during the lifespan of the object. When the 2nd test was run, even on a differnet suite, the new PowerSupplySispm failed due to libusb returned "Resource Busy", because the previous nanoBTS+PowerSupplySispm were still alive.

Adding the following code to suite.objects_cleanup() helped showing that devices are being kept alive at that point, because rfcount is > 1 (it should be 1 since we should only have 1 reference to it in "obj" var after cleanup() is called. Actually this is not entirely true since other resource objects could be referencing them, so we should first call cleanup() on all of them, then keep them in another array, then check the refcount of each of them to make sure they are going to be cleared.

My bet is that they are still kept referenced by the test .py file which used them, since the test is run in the file directly and not in a function with a scope. I think we should either find a way to "unload" that file, and if not easy to do, then call a function inside that file instead of running the file directly. The test could still leak objects by having global objects in the file, but we could check it with the refcount once the test/suite is finished.

diff --git a/src/osmo_gsm_tester/log.py b/src/osmo_gsm_tester/log.py
index 7c4ae44..ca56781 100644
--- a/src/osmo_gsm_tester/log.py
+++ b/src/osmo_gsm_tester/log.py
@@ -397,6 +397,9 @@ class Origin:
         if find_parent:
             self._set_parent(Origin.find_on_stack(except_obj=self))

+    def __del__(self):
+        self.log('PESPIN: OBJ DESTROYED!')
+
     def _set_parent(self, parent):
         # make sure to avoid loops
         p = parent
--- a/src/osmo_gsm_tester/suite.py
+++ b/src/osmo_gsm_tester/suite.py
@@ -20,6 +20,7 @@
 import os
 import sys
 import time
+import gc
 import pprint
 from . import config, log, template, util, resource, schema, event_loop, test
 from . import osmo_nitb, osmo_hlr, osmo_mgcpgw, osmo_mgw, osmo_msc, osmo_bsc, osmo_stp, osmo_ggsn, osmo_sgsn, modem, esme
@@ -100,6 +101,9 @@ class SuiteRun(log.Origin):
                 obj.cleanup()
             except Exception:
                 log.log_exn()
+            refc = sys.getrefcount(obj)
+            if refc > 1:
+                self.err('refc for %r: %d > 1, probably going to be leaked. referrers: %r' % (repr(obj), refc, gc.get_referrers(obj)))

     def mark_start(self):
         self.start_timestamp = time.time()
Actions #1

Updated by pespin about 6 years ago

  • Description updated (diff)
Actions #2

Updated by pespin 12 months ago

  • Subject changed from resource objects leaked or destroyed too late to osmo-gsm-tester: resource objects leaked or destroyed too late
  • Priority changed from Normal to Low
Actions

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 48.8 MB)