Performance - meta issue for hunting down memory leaks
We recently had some tests fail due to running out of memory (@DaveD , @seamuslee ). Concurrently in performance testing locally I discovered that my tests of 2000 donations went in at 1/4 of the speed of my tests of 200 & of course @jamie has blogged about slow downs.
This set me off on the path of digging into memory leaks and I seem to have found my first leak. It's in the unit tests rather than in the main code (which is where I started looking). I'll talk through my steps
-
I started from https://alanstorm.com/php-meminfo-and-memory-leaks/ and installed the linked extension & despite some initial confusion was able to get it going in the context of MAMP.
-
I then applied this patch and ran the address test class (update - I realise I need to find somewhere after the test class as they could still be removed during deconstruct for true leaks - but this would show excessive memory use - esp with dataproviders)
--- a/tests/phpunit/api/v3/AddressTest.php
+++ b/tests/phpunit/api/v3/AddressTest.php
@@ -53,6 +53,7 @@ class api_v3_AddressTest extends CiviUnitTestCase {
$this->contactDelete($this->_contactID);
$this->quickCleanup(['civicrm_address', 'civicrm_relationship']);
parent::tearDown();
+ meminfo_dump(fopen('/tmp/my_dump_file.json', 'w'));
}
- I analysed the file with the summary command
eileenmcnaughton@Eileens-MacBook-Pro analyzer % bin/analyzer summary /tmp/my_dump_file.json
-
This screen tells me there are instances of the above that are not cleaned up. I picked ``CRM_Core_DAO_LocationType ``` as it seemed the most leaky one that seemed somewhat straight forward
-
I used the method from the instructions to find an instance id
bin/analyzer query -v -f "class=CRM_Core_DAO_LocationType" -f "is_root=0" /tmp/my_dump_file.json
- I could determine which class was loading it bin/analyzer ref-path -v 0x1177b4380 /tmp/my_dump_file.json and it was just the AddressTest
- And again (from the instructions) to find what it was interacting with
- In the interests of finding the circularity preventing the memory from being freed I went through each of the child objects & looked to see if they had dependencies - most had none but when I tried on
_query: 0x1177b4408
I found it had 11 children
I'm still working on figuring out what it's children are - but I do have a quick & easy fix for the specific instance & test since we only need to put the id, not the whole class, as an object on the AddressTest
- After applying https://github.com/civicrm/civicrm-core/pull/18632 I got a small improvement