CiviCRM Core issueshttps://lab.civicrm.org/dev/core/-/issues2023-10-20T07:57:25Zhttps://lab.civicrm.org/dev/core/-/issues/3740Set default lock level to READ COMMITTED2023-10-20T07:57:25ZJoeMurraySet default lock level to READ COMMITTEDDrupal recently changed the default lock level slightly down from REPEATABLE READ to READ COMMITTED, which has the effect of avoiding some db locks. See https://www.drupal.org/node/3269885 We have seen over recent years some issues with ...Drupal recently changed the default lock level slightly down from REPEATABLE READ to READ COMMITTED, which has the effect of avoiding some db locks. See https://www.drupal.org/node/3269885 We have seen over recent years some issues with deadlocks, primarily on rebuilding group contact cache when there are hierarchies of smart groups. Should CiviCRM be more explicit about what isolation level we expect in order to get more repeatable results, and should we follow Drupal in its move to READ COMMITTED? I tend to think so.
See responses in MM chat from @demeritcowboy
https://chat.civicrm.org/civicrm/pl/gnmhfdxes7rgb8k7ppoaezo45ahttps://lab.civicrm.org/dev/core/-/issues/3490Upgrader - Apply extension updates after core updates2023-02-08T19:08:30ZtottenUpgrader - Apply extension updates after core updatesOverview
----------------------------------------
Functionality in `civicrm-core` is increasingly organized with internal extensions (`civicrm-core:ext/*`). The main upgrade workflow should incorporate the upgrade-steps for extensions.
...Overview
----------------------------------------
Functionality in `civicrm-core` is increasingly organized with internal extensions (`civicrm-core:ext/*`). The main upgrade workflow should incorporate the upgrade-steps for extensions.
Example use-case
----------------------------------------
1. Extract tarball with a new version of `civicrm-core`, including updated `ext/*`
1. Run the main upgrader (`civicrm/upgrade?reset=1`, `cv core:upgrade`, `drush civicrm-upgrade-db`, or `wp civicrm upgrade-db`)
Current behaviour
----------------------------------------
It updates the schema for things like `civicrm_contact` (eg `CRM/Upgrade/*`) but not things like `civicrm_search_segment` (eg `ext/search_kit/CRM/Search/Upgrader.php`).
As a sysadmin, you must run the upgrades separately.
Most are not habituated to running these together. This leads to confusing user-stories like https://civicrm.stackexchange.com/questions/42094/civimail-issue-db-error-no-such-table-when-sending-mailing (where the first upgrade seemed to work - but you get random errors because the second upgrade hasn't run).
Proposed behaviour
----------------------------------------
Run both.
The core-upgrade should delegate/incorporate the ext-upgrade, and it should preserve essential elements of the ext-upgrade contract. Specifically: ext-upgrades can use features/services/APIs from core. This implies that (1) ext-upgrades run after core-upgrades and (2) when it gets to ext-upgrades, it should follow a normal/open dispatch-policy.
Comments
----------------------------------------
Filing issue after some Mattermost discussion with rudanpal, @colemanw, @bgm, @totten: https://chat.civicrm.org/civicrm/pl/b76y4gy517yxdet331ueqrxgrw
I posted a draft at https://github.com/civicrm/civicrm-core/compare/5.50...totten:run-ext-upgrades?expand=1
As mentioned in PHP comments+Mattermost, there's an issue with the current draft -- when it gets to ext-upgrade tasks, it would (unexpectedly) continue to apply the limited dispatch-policy used for core-upgrades (`upgrade.main` vs `upgrade.finish`). Some ways to address that:
* __Runner state__: While the upgrader goes through phases (*ex: 1-core upgrade, 2-ext upgrade, 3-new exts*), keep a stored value naming the phase. this stored value determines the active dispatch-policy
* ex: `Civi::settings()->set('upgrade_phase', '...')`
* __Task wrapping/task tagging__: When taking ext-upgrade tasks and putting them into the core-upgrade queue, put some wrapper or tag on each one to indicate the phase/policy that it relies on
* wrapper example: `CRM_Upgrade_Form::wrappedTask(string $dispatchPolicy, array $realCallback)
* tag example: `$queue->createItem($task, ['dispatchPolicy' => string])`