Aggressive cache clearing significantly increases test time
The test suites are significantly slower in Civi 5.4.beta1/5.5.alpha1 than in 5.3. (Ballpark: +1hr.) One might speculate that this stems from high contention for CI resources, additional tests, and/or generic changes in the framework/process. While the first two may also be factors, I've run the test suites in isolation and found a clear across-the-board slowdown (Jul 24 dataset) causing many test runtimes to grow by double (or more). For example, when running api_v3_SyntaxConformanceTest
on an isolated system (no CPU/RAM contention), the time increased from 251s (v5.3) to 936s (v5.4.beta1) even while the #test-functions and #entities remained flat.
Focusing on one specific example (api_v3_UtilsTest
; increased from 1s to 1s).7s), I used 5s), although there's also observable impact from git bisect
to track the cause to 5aac553c from 12331 and #174 (closed) -- in particular, the biggest impact comes from adding CRM_Extension_System::singleton()->getCache()->flush()
(+Civi::cache('settings')->flush();
(+
In discussing 12331, one of the major concerns was "How do we make CRM_Utils_System::flushCache()
behave the same as before?" 5aac553c tried to do this, but it actually was a bit more aggressive. To see this, it helps to fully describe the behavior -- in each configuration, what's the impact on each cache?
-
(OLD) Behavior circa 5.3.x --
CRM_Utils_System::flushCache()
callsCRM_Utils_Cache::singleton()->flush()
which has a cascading effect on other caches.-
(DEFAULT) Default configuration using ArrayCache+SQL table
-
CRM_Utils_Cache
- Destroys the generalArrayCache
, which is used as the front-cache for several others. -
CRM_Core_BAO_Cache
: Due toCRM_Utils_Cache
flush, the front-caches are flushed, but the underlying SQL caches are preserved. -
settings
: Same asCRM_Core_BAO_Cache
. (Flush ArrayCache at front, but keep underlying SQL cache.) -
js_strings
: Same asCRM_Core_BAO_Cache
. (Flush ArrayCache at front, but keep underlying SQL cache.) -
community_messages
: Same asCRM_Core_BAO_Cache
. (Flush ArrayCache at front, but keep underlying SQL cache.) -
CRM_Extension_System::$cache
: Same asCRM_Core_BAO_Cache
. (Flush ArrayCache at front, but keep underlying SQL cache.) -
CiviCxnHttp
: Same asCRM_Core_BAO_Cache
. (Flush ArrayCache at front, but keep underlying SQL.)
-
-
(MEM) Alternative Memcache/Redis configuration
-
CRM_Utils_Cache
- Flushes everything in memory-backed cache. -
CRM_Core_BAO_Cache
: Destroys the memory-backed front-cache but preserves the underlying SQL cache. -
settings
: Completely flushed. (Due toCRM_Utils_Cache
flush.) -
js_strings
: Completely flushed. (Due toCRM_Utils_Cache
flush.) -
community_messages
: Completely flushed. (Due toCRM_Utils_Cache
flush.) -
CRM_Extension_System::$cache
: Completely flushed. (Due toCRM_Utils_Cache
flush.) -
CiviCxnHttp
: Same asCRM_Core_BAO_Cache
. (Flush memory-backed front-cache, but keep underlying SQL cache.)
-
-
(DEFAULT) Default configuration using ArrayCache+SQL table
-
(NEW) Behavior circa 5.4.beta1/5.5.alpha1 --
CRM_Utils_System::flushCache()
calls several individual flush functions-
(DEFAULT) Default configuration using ArrayCache+SQL table
-
CRM_Core_BAO_Cache
: Due toCRM_Utils_Cache
flush, the front-caches are flushed, but the underlying SQL caches are preserved. -
settings
: Completely flushed. (Due to explicit call.) -
js_strings
: Completely flushed. (Due to explicit call.) -
community_messages
: Completely flushed. (Due to explicit call.) -
CRM_Extension_System::$cache
: Completely flushed. (Due to explicit call.) -
CiviCxnHttp
: Completely flushed. (Due to explicit call.)
-
-
(MEM) Alternative Memcache/Redis configuration
-
CRM_Core_BAO_Cache
: Due toCRM_Utils_Cache
flush, the front-caches are flushed, but the underlying SQL caches are preserved. -
settings
: Completely flushed. (Due to explicit call.) -
js_strings
: Completely flushed. (Due to explicit call.) -
community_messages
: Completely flushed. (Due to explicit call.) -
CRM_Extension_System::$cache
: Completely flushed. (Due to explicit call.) -
CiviCxnHttp
: Completely flushed. (Due to explicit call.)
-
-
(DEFAULT) Default configuration using ArrayCache+SQL table
With that data, how we would describe both the original intent of 5aac553c ... and it's unintended consequence in this issue?
- During the development of 5.4.alpha1, we looked at the behavior of OLD-MEM and intended to keep the behavior the same. Thus, we have NEW-MEM which is basically the same as OLD-MEM.
- NEW-DEFAULT, NEW-MEM, and OLD-MEM are basically the same. But NEW-DEFAULT and OLD-DEFAULT are different.
- In running unit-tests, one flushes the caches frequently. The behavior of OLD-DEFAULT (where it only flushes front-caches... but generally preserves underlying caches) performs better for unit-tests. The behavior of NEW-DEFAULT/NEW-MEM/OLD-MEM does not perform as well.