CiviCRM Core issueshttps://lab.civicrm.org/dev/core/-/issues2023-11-20T05:03:18Zhttps://lab.civicrm.org/dev/core/-/issues/3053Usability issue reported by user: All Reports page overrides report results s...2023-11-20T05:03:18ZDevAppUsability issue reported by user: All Reports page overrides report results settingWhen clicking on a report name on the All Reports page located at /wp-admin/admin.php?page=CiviCRM&q=civicrm%2Freport%2Flist&reset=1
The resulting link loads the report with output=criteria on the end. This stops the default parameters ...When clicking on a report name on the All Reports page located at /wp-admin/admin.php?page=CiviCRM&q=civicrm%2Freport%2Flist&reset=1
The resulting link loads the report with output=criteria on the end. This stops the default parameters of the results from loading, which was set to view results. The user expected results to appear when loading the report per the saved report setting. The result to the user is the report is broken.
The output=criteria is overriding the saved report behaviour. Whilst there is a view results button the right hand side of the All Reports page, it does create some confusion to the user with consistency. Perhaps if the setting is already set on the report to view results, the output=criteria could be removed from that link to create consistency... Or the GUI clearer about why the report wasn't run.
This isn't a bug, but more a usability issue.https://lab.civicrm.org/dev/core/-/issues/1843Url-tracking in mass sms2020-08-03T00:07:11ZdmunioUrl-tracking in mass smsOverview
----------------------------------------
If a mass sms is sent with a url in the message, when sending the sms a url-tracking is added. This affects the length of the sms and also, there are no sms tracking reports.
Reproductio...Overview
----------------------------------------
If a mass sms is sent with a url in the message, when sending the sms a url-tracking is added. This affects the length of the sms and also, there are no sms tracking reports.
Reproduction steps
----------------------------------------
1. New mass sms
2. Add url in the sms message.
3. Preview sms with url-tracking
![image](/uploads/8124f67212f608522c7b260185fa306b/image.png)5.29.0https://lab.civicrm.org/dev/core/-/issues/2898URL Tokens - CiviMail BAO and FlexMailer disagree over duplicate `http://`2022-12-19T22:45:31ZtottenURL Tokens - CiviMail BAO and FlexMailer disagree over duplicate `http://`Overview
--------
CiviMail BAO and Flexmailer provide two mechanisms to rendering/delivering mail-blasts. To phase out BAO-rendering, we should eventually migrate existing sites to Flexmailer. However, small discrepancies between them c...Overview
--------
CiviMail BAO and Flexmailer provide two mechanisms to rendering/delivering mail-blasts. To phase out BAO-rendering, we should eventually migrate existing sites to Flexmailer. However, small discrepancies between them can be an obstacle. This issue asks how to deal with a discrepancy when handling certain URL tokens.
(The issue is an off-shoot from the review discussion on https://github.com/civicrm/civicrm-core/pull/21522. However, they are somewhat distinct as 21522 applies to new-sites and this problem is more pressing on upgrade-sites.)
Background
-----------
Suppose you have an email token which generates a URL (eg `{action.forward}` or `{action.unsubscribeUrl}`) . The most correct way to use this token in an HTML-email is to place it in a hyperlink (`<a href="{action.forward}>`).
However, if you use CKEditor to compose a message, the hyperlink dialog strongly encourages you to put `http://` at the front of any hyperlink. Thus, you can organically produce expressions like `<a href="http://{action.forward}">`.
![CkeditorHttpToken](/uploads/e2d9bbf4ab6ba12a6cdcd936955277a9/CkeditorHttpToken.mov)
CiviMail BAO has a feature which mitigates this - in effect, both notations give the same output. However, Flexmailer (TokenProcessor) does not have a similar mitigation. Thus:
| HTML Email Expression | BAO Output | Flexmailer Output |
| -- | -- | -- |
| `<a href="{action.forward}>` | `<a href="https://example.com/civicrm/mailing/forward?...` | `<a href="https://example.com/civicrm/mailing/forward?...` |
| `<a href="http://{action.forward}>` | `<a href="https://example.com/civicrm/mailing/forward?...` | `<a href="http://https://example.com/civicrm/mailing/forward?...` |
Note the duplicate URL scheme (`http://http://`).
Scenarios: Good
---------
There are several scenarios wherein this discrepancy does not matter:
* You use CKEditor + BAO-renderer. (*The BAO renderer mitigates extraneous `http://`.*)
* You use Mosaico + Flexmailer. (*Mosaico's UI doesn't create extraneous `http://`.*)
(I suspect these two cohorts are the largest.)
Scenarios: Bad
---------
The potential for difficulty arises when mixing CKEditor with Flexmailer -- you take some HTML that was composed with CKEditor, run it through Flexmailer, and now you have invalid URLs with `http://https://`.
If we flip over more existing sites to Flexmailer, then it is foreseeable that more sites will be in this mixed scenario.
There's a secondary consideration in who may experience problems -- templates. Most users are not in the habit of typing `{action.forward}` or `{action.unsubscribeUrl}` regularly. Anecdotally, the typical practice is to put these kinds of tokens into a template, eg
* (A) Add a "User-Defined Template" (edited w/CKEditor in web; stored in `civicrm_msg_template`), or...
* (B) Reuse/copy a previous "Mailing" (edited w/CKEditor in web; stored in `civicrm_mailing`), or...
* (C) Configure default header/footer (no CKEditor; stored in `civicrm_mailing_component`)
Practical recap: today, if you enable Flexmailer on a site that uses CKEditor for mailings, then either:
* Your mailings and templates will have broken links -- because they use `<a href="http://{action.unsubscribeUrl}">`.
* This seems more likely if you use (A)/(B) - because the web UI uses CKEditor.
* Your mailings and templates will be cleanly interoperable -- because they use `<a href="{action.unsubscribeUrl}">`.
* This seems more likely if you use (C) header/footer because the web UI uses textarea.
Approaches
----------
1. Add a mitigation to Flexmailer or TokenProcessor which removes extraneous schema during the rendering phase.
* _Strength_: Best interoperability
* _Criticism_: Adds quirky bits to rendering.
* _Variations_: Patch Flexmailer vs patch TokenProcessor.
* _Variations_: Apply cleanup before rendering (`http://{action.*}` => `{action.*}`) or after rendering (`http://https?://` => `https?://`).
2. Add an upgrade-task to remove extra `http://` in front of tokens.
* _Strength_: No quirky bits in rendering.
* _Criticism_: CKEditor is still there. Can only upgrade core tables. Rewrites history (`civicrm_mailing.body_html` of past mailings).
3. Do nothing - Downstream users should manually convert their mailings/templates.
* _Strength_: Look, ma, no hands!
* _Criticism_: We cannot measure how many users will be impacted by break. For users with a "Reuse/Copy" workflow, they cannot fix historical mailings in UI.5.57.0https://lab.civicrm.org/dev/core/-/issues/1414Url search with `civicrm/case/search?case_owner=2&force=1` gives an E_NOTICE2019-11-22T02:47:10ZDaveDUrl search with `civicrm/case/search?case_owner=2&force=1` gives an E_NOTICEThis just started in master recently and I can't reproduce in 5.20 but I can on the public demo which is 5.21alpha1.
Running a url search `civicrm/case/search?reset=1&case_owner=2&force=1` gives
`Notice: Undefined index: html in CRM_Co...This just started in master recently and I can't reproduce in 5.20 but I can on the public demo which is 5.21alpha1.
Running a url search `civicrm/case/search?reset=1&case_owner=2&force=1` gives
`Notice: Undefined index: html in CRM_Core_Form_Search->getEntityDefaults() (line 291 of ...\CRM\Core\Form\Search.php)`
ping @seamuslee5.21.0seamusleeseamusleehttps://lab.civicrm.org/dev/core/-/issues/975URL for "New Activity" in the breadcrumb of the "Find Activities" page is inc...2019-05-30T01:27:11Zjustinfreeman (Agileware)URL for "New Activity" in the breadcrumb of the "Find Activities" page is incorrect - uses comma (,) instead of an ampersand (&)URL for "New Activity" in the breadcrumb of the "Find Activities" page is incorrect - uses comma (,) instead of ampersand (&).
The incorrect URL is:
https://dmaster.demo.civicrm.org/civicrm/activity?reset=1&action=add,context=standalone...URL for "New Activity" in the breadcrumb of the "Find Activities" page is incorrect - uses comma (,) instead of ampersand (&).
The incorrect URL is:
https://dmaster.demo.civicrm.org/civicrm/activity?reset=1&action=add,context=standalone
But the correct URL is:
https://dmaster.demo.civicrm.org/civicrm/activity?reset=1&action=add&context=standalone
Clicking the "New Activity" link causes this error.
`Notice: Undefined index: Basic in HTML_QuickForm_Controller->exportValues() (line 495 of /srv/buildkit/build/dmaster/sites/all/modules/civicrm/packages/HTML/QuickForm/Controller.php).
Warning: Invalid argument supplied for foreach() in HTML_QuickForm_Controller->exportValues() (line 495 of /srv/buildkit/build/dmaster/sites/all/modules/civicrm/packages/HTML/QuickForm/Controller.php).`
Here's a screenshot from dmaster CiviCRM
![Find_Activities___CiviCRM_Sandbox_on_Drupal](/uploads/2cd663447e49fd6e19e2b8e60d39a4ba/Find_Activities___CiviCRM_Sandbox_on_Drupal.png)
Agileware Ref: CIVICRM-12045.14.0https://lab.civicrm.org/dev/core/-/issues/1554URL Encoded values for event pages work but cause style changes2023-02-03T05:04:08Zluke.stewartURL Encoded values for event pages work but cause style changesOverview
----------------------------------------
In Drupal 7
Both: (A) civicrm/event/info%3Freset%3D1%26id%3D7 and (B) civicrm/event/info?reset=1&id=7 appear to show the same page however for anon viewers the custom-civicrm.css is loa...Overview
----------------------------------------
In Drupal 7
Both: (A) civicrm/event/info%3Freset%3D1%26id%3D7 and (B) civicrm/event/info?reset=1&id=7 appear to show the same page however for anon viewers the custom-civicrm.css is loaded on (A) but not (B) and for logged in users if a different civicrm admin theme is set this is used to render the page instead for (A).
The version (A) is what is provided by default when using drupal views to display a link to events.
Reproduction steps
----------------------------------------
1. Change the civicrm admin theme to be different to the default theme.
2. Add a civicrm custom stylesheet via /civicrm/admin/setting/url?reset=1 or install the shoreditch extension.
3. Create an event
4. visit the event info page via the URL encoded link and compare to the non url encoded link.
Current behaviour
----------------------------------------
![EventPage](/uploads/993d08057dd7662865b79c096dcb4e10/EventPage.png)
When viewed logged in with permission to view civicrm admin theme (A) displays using the admin theme (B) using the default theme.
When viewed as anon - or logged in civicrm-custom.css is loaded on (A) and not (B)
Expected behaviour
----------------------------------------
Both pages should be identical.
Environment information
----------------------------------------
* __CiviCRM:__ 5.21.1
* __CMS:__ Drupal 7.x
* __Web Server:__ Nginx
Comments
----------------------------------------
Of relevance:
https://lab.civicrm.org/dev/core/issues/378
https://github.com/civicrm/civicrm-core/pull/14876
https://chat.civicrm.org/civicrm/pl/iqg97j674fbn5d5jsf1n8qjnbyhttps://lab.civicrm.org/dev/core/-/issues/2961Upgrading to CiviCRM 5.43.1 fails2023-11-11T05:03:23ZspalmstromUpgrading to CiviCRM 5.43.1 failsOverview
----------------------------------------
_Please describe your problem or bug in detail._
_If you have already posted on https://civicrm.stackexchange.com or https://chat.civicrm.org, please include the link to that conversatio...Overview
----------------------------------------
_Please describe your problem or bug in detail._
_If you have already posted on https://civicrm.stackexchange.com or https://chat.civicrm.org, please include the link to that conversation._
Reproduction steps
----------------------------------------
1. Update the code
1. Run the update.
1. Get 'unknown column' error.
1. Recover database backup and add the missing columns manually.
1. Rerun the backup.
1. Get the invalid use of NULL error.
Current behaviour
----------------------------------------
_What happens currently. Please provide error messages, screenshots or gifs ([LICEcap](http://www.cockos.com/licecap/), [SilentCast](https://github.com/colinkeenan/silentcast)) where appropriate._
![image](/uploads/cd94ad564f0436275dc62b956467b679/image.png)
![image](/uploads/903629f47d1a9f7bcb07a218c47a807e/image.png)
Expected behaviour
----------------------------------------
The update should run.
Environment information
----------------------------------------
<!-- Some of the items below may not be relevant for every bug - if in doubt please include more information than you think is necessary. -->
* __Browser:__ _Edge_ but probably no relevant
* __CiviCRM:__ _5.41.2/5.43.1_ <!-- If this problem relates to an upgrade, then specify both old and new versions -->
* __PHP:__ _7.4.25__ but probably not relevant
* __CMS:__ _Drupal 9.2.8_ but probably not relevant
* __Database:__ _MySQL 8.0.27_
* __Web Server:__ _IIS_ but probably not relevant.
Comments
----------------------------------------
_Anything else you would like the reviewer to note._
Removing the NOT from the query in <Drupal Root>\vendor\/civicrm/civicrm-core/CRM\Upgrade/Incremental/php/FiveFortyThree.php line 205 cures the latter error.https://lab.civicrm.org/dev/core/-/issues/4754Upgrading to 5.66+ still has old copy of crm.menubar.js2023-11-15T01:19:55ZjonathandhnUpgrading to 5.66+ still has old copy of crm.menubar.jscrm.menubar.js l.293 return a console error for ```TypeError: undefined is not an object (evaluating 'result.values.length')``` when text is typed on the search box and live search from the menu bar stopped working.
```
if (result.v...crm.menubar.js l.293 return a console error for ```TypeError: undefined is not an object (evaluating 'result.values.length')``` when text is typed on the search box and live search from the menu bar stopped working.
```
if (result.values.length > 0) {
$('#crm-qsearch-input').autocomplete('widget').menu('option', 'disabled', false);
$.each(result.values, function(k, v) {
ret.push({value: v.id, label: v.data});
});
}
```
Civicrm 5.66.0, then 5.66.1, 5.66.2 and 5.67.0, Drupal 10.1.6 PHP 8.1.24, MariaDB 10.6.15https://lab.civicrm.org/dev/core/-/issues/3653Upgrading to 5.51 RC fails through the web interface, but works with `cv` and...2022-07-21T00:54:20ZjhungerfordUpgrading to 5.51 RC fails through the web interface, but works with `cv` and `drush`.Overview
----------------------------------------
I commented on this issue in the Town Square:
https://chat.civicrm.org/civicrm/pl/kpcoyh7hx3nzunx34ejbe8h8aw
Running the database upgrade to 5.51 RC by visiting `/civicrm/upgrade?reset=1...Overview
----------------------------------------
I commented on this issue in the Town Square:
https://chat.civicrm.org/civicrm/pl/kpcoyh7hx3nzunx34ejbe8h8aw
Running the database upgrade to 5.51 RC by visiting `/civicrm/upgrade?reset=1` fails with this message:
"DB Error: no such field"
`UPDATE civicrm_queue SET status = NULL WHERE name = 'CRM_Upgrade' [nativecode=1054 ** Unknown column 'status' in 'field list'`
Running the same upgrade with `cv` or `drush` works.
Reproduction steps
----------------------------------------
1. Install an older version of Civi (e.g. 5.49.5) on Drupal 7.88
2. Replace the Civi module with 5.51 RC (civicrm-RC-drupal.tar.gz ( 5.51.beta1-202206120040 ))
3. Visit /civicrm/upgrade?reset=1 in the web interface, and run the upgrade
4. Observe that it crashes with "DB Error: no such field"
5. Try again, to confirm that it consistently fails in the web interface
6. Run the upgrade with `cv upgrade:db` or drush, and observe that it works
Current behaviour
----------------------------------------
After the failure in the web interface, the Drupal log of our production clone contained a message with this string:
`UPDATE civicrm_queue SET status = NULL WHERE name = 'CRM_Upgrade' [nativecode=1054 ** Unknown column 'status' in 'field list'`
I didn't find any upgrade messages in the Drupal log of the clean install, I'm not sure why.
Expected behaviour
----------------------------------------
The web interface upgrade normally works the same as the command line process.
Environment information
----------------------------------------
* __Browser:__ Firefox 91.9.1esr (64-bit)
* __CiviCRM:__ from 5.49.5 to 5.51.beta1 (5.51.beta1-202206120040)
* __PHP:__ 7.3.33-1+0~20211119.91+debian10~1.gbp618351
* __CMS:__ Drupal 7.88
* __Database:__ mariadb Ver 15.1 Distrib 10.3.34-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2
* __Web Server:__ apache2 2.4.38-3+deb10u75.51.0https://lab.civicrm.org/dev/core/-/issues/1846Upgrading to 5.27 hits fatal errors getting to the upgrade screen2020-08-20T10:34:20ZeileenUpgrading to 5.27 hits fatal errors getting to the upgrade screenIn deploying the rc on a wordpress site just now I hit 3 places where DB queries failed before I could get to the upgrade screen (through the UI). One of these brought down the Wordpress site (a woo_commerce-civi api call) The problem i...In deploying the rc on a wordpress site just now I hit 3 places where DB queries failed before I could get to the upgrade screen (through the UI). One of these brought down the Wordpress site (a woo_commerce-civi api call) The problem is the new field civicrm_custom_field.serialize gets included in queries before it is in th DB
This patch got me there https://github.com/civicrm/civicrm-core/pull/17722 but drupal views is likely also an issue5.27.1https://lab.civicrm.org/dev/core/-/issues/1695Upgrading to 5.24.1 custom field not saved for Case when submitted from webform2020-08-07T03:09:34ZPradeep Nayakpradpnayak@gmail.comUpgrading to 5.24.1 custom field not saved for Case when submitted from webformTo Replicate:
Create a webform and enable civicase with custom field. After submitting the webform a new case is created but the custom fields are blank.
This was working until 5.22.x and than update to DB/ packages in 5.23 broke this f...To Replicate:
Create a webform and enable civicase with custom field. After submitting the webform a new case is created but the custom fields are blank.
This was working until 5.22.x and than update to DB/ packages in 5.23 broke this functionality. This cannot be replicated when case is created from UI but can be replicated by writing code eg
```
$result = civicrm_api3('Case', 'create', [
'contact_id' => 202,
'case_type_id' => "housing_support",
'subject' => "test",
])['id'];
$cust = array(
'19' => array(
'-1' => array(
'id' => '',
'value' => '344',
'type' => 'Money',
'custom_field_id' => '19',
'custom_group_id' => '13',
'table_name' => 'civicrm_value_reportable_co_13',
'column_name' => 'reportable_cost_19',
'file_id' => '',
'is_multiple' => '0',
)
)
);
CRM_Core_BAO_CustomValueTable::store($cust, 'civicrm_case', $result);
```
Digging into the code found that the entry civicrm_value_reportable_co_13 table does get created but isn't shown. I guess this is because the transaction is started but not committed. Below patch fixes my problem (which removes Transaction code) but I guess its not ideal solution.
```
diff --git a/Civi/CCase/Events.php b/Civi/CCase/Events.php
index 114875f17e..f76c92c267 100644
--- a/Civi/CCase/Events.php
+++ b/Civi/CCase/Events.php
@@ -78,12 +78,10 @@ class Events {
public static function fireCaseChangeForRealz($caseIds) {
foreach ((array) $caseIds as $caseId) {
if (!isset(self::$isActive[$caseId])) {
- $tx = new \CRM_Core_Transaction();
self::$isActive[$caseId] = 1;
$analyzer = new \Civi\CCase\Analyzer($caseId);
\CRM_Utils_Hook::caseChange($analyzer);
unset(self::$isActive[$caseId]);
- unset($tx);
}
}
}
```5.24.2https://lab.civicrm.org/dev/core/-/issues/3840Upgrading the database from 5.52.3 to 5.53.0 generates "The wrong dispatch po...2022-10-01T13:51:21ZJonGoldUpgrading the database from 5.52.3 to 5.53.0 generates "The wrong dispatch policy is active".See Mattermost at https://chat.civicrm.org/civicrm/pl/je8a8jqhfby8ibw99ie1k9wywr.
We're still tracking down the cause, but here is a verbose log.
```
Box Requirements Checker
========================
> Using PHP 7.4.30
> PHP is using t...See Mattermost at https://chat.civicrm.org/civicrm/pl/je8a8jqhfby8ibw99ie1k9wywr.
We're still tracking down the cause, but here is a verbose log.
```
Box Requirements Checker
========================
> Using PHP 7.4.30
> PHP is using the following php.ini file:
/etc/php/7.4/cli/php.ini
> Checking Box requirements:
.....
[OK] Your system is ready to run the application.
Found CiviCRM database version 5.52.3.
Found CiviCRM code version 5.53.0.
Checking pre-upgrade messages...
The default copies of the message templates listed below will be updated to
handle new features or correct a problem. Your installation has customized
versions of these message templates, and you will need to apply the updates
manually after running this upgrade. Click here [1] for detailed
instructions.
* _Contributions - Receipt (off-line)_ - Update to new smarty variables
for line items, tax
Links:
------
[1] https://docs.civicrm.org/user/en/latest/email/message-templates/#modifying-system-workflow-message-templates
Press ENTER to continue
Dropping SQL triggers...
Preparing upgrade...
Executing upgrade...
Cleanup old files (CRM_Upgrade_Form::doFileCleanup(/home/jon/.cv/upgrade/57a5be292438b5304f5ec9865e0f0d29.dat))
Cleanup old upgrade snapshots (CRM_Upgrade_Snapshot::cleanupTask(civicrm))
Checking extensions (CRM_Upgrade_Form::disableOldExtensions(/home/jon/.cv/upgrade/57a5be292438b5304f5ec9865e0f0d29.dat))
Begin Upgrade to 5.53.alpha1 (CRM_Upgrade_Form::doIncrementalUpgradeStart(5.53.alpha1))
Upgrade DB to 5.53.alpha1 (CRM_Upgrade_Form::doIncrementalUpgradeStep(5.53.alpha1,5.52.3,5.53.0,/home/jon/.cv/upgrade/57a5be292438b5304f5ec9865e0f0d29.dat))
Upgrade DB to 5.53.alpha1: SQL (CRM_Upgrade_Incremental_php_FiveFiftyThree::runSql(5.53.alpha1))
Replace %A specifier in date settings. (CRM_Upgrade_Incremental_php_FiveFiftyThree::replacePercentA())
Add invoice pdf format (CRM_Upgrade_Incremental_php_FiveFiftyThree::addInvoicePDFFormat())
Add Recent Items Providers (CRM_Upgrade_Incremental_php_FiveFiftyThree::addRecentItemsProviders())
Finish Upgrade DB to 5.53.alpha1 (CRM_Upgrade_Form::doIncrementalUpgradeFinish(5.53.alpha1,5.52.3,5.53.0,/home/jon/.cv/upgrade/57a5be292438b5304f5ec9865e0f0d29.dat))
Begin Upgrade to 5.53.beta1 (CRM_Upgrade_Form::doIncrementalUpgradeStart(5.53.beta1))
Upgrade DB to 5.53.beta1 (CRM_Upgrade_Form::doIncrementalUpgradeStep(5.53.beta1,5.52.3,5.53.0,/home/jon/.cv/upgrade/57a5be292438b5304f5ec9865e0f0d29.dat))
Upgrade DB to 5.53.beta1: SQL (CRM_Upgrade_Incremental_php_FiveFiftyThree::runSql(5.53.beta1))
Finish Upgrade DB to 5.53.beta1 (CRM_Upgrade_Form::doIncrementalUpgradeFinish(5.53.beta1,5.52.3,5.53.0,/home/jon/.cv/upgrade/57a5be292438b5304f5ec9865e0f0d29.dat))
Finish Upgrade DB to 5.53.0 (CRM_Upgrade_Form::doIncrementalUpgradeFinish(5.53.beta1,5.53.0,5.53.0,/home/jon/.cv/upgrade/57a5be292438b5304f5ec9865e0f0d29.dat))
Finish core DB updates 5.53.0 (CRM_Upgrade_Form::doCoreFinish(5.53.beta1,5.53.0,5.53.0,/home/jon/.cv/upgrade/57a5be292438b5304f5ec9865e0f0d29.dat))
Assess extension upgrades (CRM_Upgrade_Form::enqueueExtUpgrades(5.53.beta1,5.53.0,5.53.0,/home/jon/.cv/upgrade/57a5be292438b5304f5ec9865e0f0d29.dat))
Error executing task: %s
[RuntimeException]
Task can not execute correctly. The wrong dispatch policy is active. Expected to find "upgrade.finish".
Exception trace:
() at /var/www/mysite.megaphonetech.com/htdocs/sites/all/modules/civicrm/CRM/Upgrade/DispatchPolicy.php:164
CRM_Upgrade_DispatchPolicy::assertActive() at /var/www/mysite.megaphonetech.com/htdocs/sites/all/modules/civicrm/CRM/Upgrade/Form.php:856
CRM_Upgrade_Form::enqueueExtUpgrades() at /var/www/mysite.megaphonetech.com/htdocs/sites/all/modules/civicrm/CRM/Queue/Task.php:101
CRM_Queue_Task->run() at phar:///usr/local/bin/cv/vendor/composer/../../src/Util/ConsoleQueueRunner.php:78
Civi\Cv\Util\ConsoleQueueRunner->runAll() at phar:///usr/local/bin/cv/src/Command/UpgradeDbCommand.php:133
Civi\Cv\Command\UpgradeDbCommand->execute() at phar:///usr/local/bin/cv/vendor/symfony/console/Command/Command.php:257
Symfony\Component\Console\Command\Command->run() at phar:///usr/local/bin/cv/vendor/symfony/console/Application.php:850
Symfony\Component\Console\Application->doRunCommand() at phar:///usr/local/bin/cv/vendor/symfony/console/Application.php:193
Symfony\Component\Console\Application->doRun() at phar:///usr/local/bin/cv/src/Application.php:46
Civi\Cv\Application->doRun() at phar:///usr/local/bin/cv/vendor/symfony/console/Application.php:124
Symfony\Component\Console\Application->run() at phar:///usr/local/bin/cv/src/Application.php:15
Civi\Cv\Application::main() at phar:///usr/local/bin/cv/bin/cv:27
require() at /usr/local/bin/cv:14
upgrade:db [--out OUT] [--dry-run] [--retry] [--skip] [--level LEVEL] [-t|--test] [-U|--user USER]
```https://lab.civicrm.org/dev/core/-/issues/311Upgrading multilingual site causes DB Error2018-10-22T19:13:55ZDon WijesooriyaUpgrading multilingual site causes DB ErrorUpgrading from CiviCRM 4.4.7 to 5.4.0 (Drupal), cause a DB error. Upgrade process works fine but when loading any civi page, JavaScript error is displayed on the language file (backtrace displayed after enabling).
![javascript](/upload...Upgrading from CiviCRM 4.4.7 to 5.4.0 (Drupal), cause a DB error. Upgrade process works fine but when loading any civi page, JavaScript error is displayed on the language file (backtrace displayed after enabling).
![javascript](/uploads/cc865f9f1870d12a4b79309623de8147/javascript.PNG)
Checking the error log, few fields relating to uf_group are not getting created during the upgrade.
![errorlog](/uploads/8424555a1c8f657bbab2d6f05d5f4c08/errorlog.PNG)
Further investigation showed that in CRM/Core/I18n, there have been schema changes between the latest SchemaStructure.php and SchemaStructure_4_7_alpha1.php thus we had to copy and rename the file as SchemaStructure_5_4_alpha1.php as mentioned [here.](https://docs.civicrm.org/dev/en/latest/translation/database/#localised-fields-schema-changes)
![schemastructure](/uploads/c05dd2b1213cad872366dc0b5f9a43be/schemastructure.PNG)5.6https://lab.civicrm.org/dev/core/-/issues/423Upgrading gives error "Incorrect datetime value: '0000-00-00 00:00:00'" in My...2018-10-08T14:48:26ZkenUpgrading gives error "Incorrect datetime value: '0000-00-00 00:00:00'" in MySQL 5.7When upgrading CiviCRM from 4.7.12 to 5.3.2 I got the following error ...
>>>
PEAR_Exception: "DB Error: unknown error"
* ERROR TYPE: DB_Error
* ERROR CODE: -1
* ERROR MESSAGE: DB Error: unknown error
* ERROR MODE: 16
* ERROR USERI...When upgrading CiviCRM from 4.7.12 to 5.3.2 I got the following error ...
>>>
PEAR_Exception: "DB Error: unknown error"
* ERROR TYPE: DB_Error
* ERROR CODE: -1
* ERROR MESSAGE: DB Error: unknown error
* ERROR MODE: 16
* ERROR USERINFO: UPDATE civicrm_financial_trxn SET trxn_date = NULL WHERE trxn_date = '0000-00-00 00:00:00' [nativecode=1292 ** Incorrect datetime value: '0000-00-00 00:00:00' for column 'trxn_date' at row 1]
>>>
The error is raised by 2 lines of SQL in civicrm/CRM/Upgrade/Incremental/sql/4.7.19.mysql.tpl ...
```sql
UPDATE civicrm_financial_trxn SET trxn_date = NULL WHERE trxn_date = '0000-00-00 00:00:00';
UPDATE civicrm_contribution SET receive_date = NULL WHERE receive_date = '0000-00-00 00:00:00';
```
I'm using MySQL 5.7.23. I found a solution which is to cast the DATETIME columns in the WHERE clause to CHAR(20) before comparing to '0000-00-00 00:00:00'.5.7https://lab.civicrm.org/dev/core/-/issues/3107Upgrading from pre-5.47 to 5.48+ isn't re-runnable in certain situations2022-06-29T02:33:30ZDaveDUpgrading from pre-5.47 to 5.48+ isn't re-runnable in certain situations
Since sketching out the below, I think I can see there is a quick solution, but now that I've written the description already I'll still post the whole thing. Short version: This line should have a check that `is_autorun` exists in the ...
Since sketching out the below, I think I can see there is a quick solution, but now that I've written the description already I'll still post the whole thing. Short version: This line should have a check that `is_autorun` exists in the table before running the query: https://github.com/civicrm/civicrm-core/blob/88e3bfdfc52aa1e117160b6592597510af446d1a/CRM/Upgrade/Incremental/php/FiveFortyEight.php#L60
Problem
-------
Typical sysadmin: "Goin' to upgrade now. Let me make a backup... ok, done. Upgrade, and ... oh that failed. Ok let me reload my sql file and apply this patch and try again... Oh that failed - something about queues. Repeat. Fail again. Hmm."
For a dev doing testing this might not come up, because they might "start from scratch" when repeating (or use trickery like devs do).
The difference is the reload db, and in particular when upgrading pre-5.47 to 5.48+. And given the history to-date, that reload-and-try-again cycle is likely to come up more with 5.47 than usual.
In 5.47, the `civicrm_queue` table is created, but [ONLY if it doesn't exist](https://github.com/civicrm/civicrm-core/blob/88e3bfdfc52aa1e117160b6592597510af446d1a/CRM/Upgrade/Incremental/sql/5.47.alpha1.mysql.tpl#L3). I.e. as opposed to `DROP TABLE IF EXISTS civicrm_queue`, followed by the creation. Since this table might have long-term data, you don't want to drop it if an upgrade is being re-run some time in the future.
(Aside: That create table statement probably shouldn't enforce utf8mb4 yet, but that's separate.)
So now let's think about the sequence for the sysadmin:
* Before first attempt, civicrm_queue didn't exist.
* After first attempt, it now exists.
* Reload db. Note something important: The backup sql file does NOT have anything about civicrm_queue. There is no DROP TABLE statement for it. So after reloading the sql file, civicrm_queue DOES exist, which is different from before the first run.
* If you're the type of sysadmin who drops the whole db before reloading the sql file, you won't run into this. I don't know what percentage that covers.
So now, there's [an upgrade step](https://github.com/civicrm/civicrm-core/blob/88e3bfdfc52aa1e117160b6592597510af446d1a/CRM/Upgrade/Incremental/php/FiveFortyEight.php#L60) in 5.48 that assumes that the field `is_autorun` is going to exist. But it doesn't because civicrm_queue at this point is the already upgraded table.
SECONDARY ISSUE
---------------
There's another issue here too for similar reasons, but I haven't confirmed the specific problem, but at the moment I think there is one.
Something new in 5.47 is that civi will auto-install search kit and afform. These both create database tables. So you have a similar situation where after reloading the db, you have an already upgraded table present when it shouldn't be, so you get a similar problem when something tries to act on it in a way that isn't re-runnable.https://lab.civicrm.org/dev/core/-/issues/2326Upgrading an old site with spaces in the case type names and external xml fil...2021-01-26T21:10:30ZDaveDUpgrading an old site with spaces in the case type names and external xml files leads to difficult to resolve status messagesWhat happens is you get a status check message telling you the name of the file should be "Basket Weaving.xml" instead of "BasketWeaving.xml", but then when you change it you then get a status check message telling you the file is missin...What happens is you get a status check message telling you the name of the file should be "Basket Weaving.xml" instead of "BasketWeaving.xml", but then when you change it you then get a status check message telling you the file is missing. It's not easy to figure out what it wants, which is that you need to remove the space from the `name` field in the database and then also from the filename.5.35.0https://lab.civicrm.org/dev/core/-/issues/1937Upgrading a site that still has "mysql" in the dsn string breaks in latest ma...2020-08-28T01:22:17ZDaveDUpgrading a site that still has "mysql" in the dsn string breaks in latest masterI think it's partly because of https://github.com/civicrm/civicrm-packages/pull/302 but more because I remember once seeing something somewhere that civi silently converted mysql to mysqli for you, and maybe that has also been removed so...I think it's partly because of https://github.com/civicrm/civicrm-packages/pull/302 but more because I remember once seeing something somewhere that civi silently converted mysql to mysqli for you, and maybe that has also been removed somewhere. While reviewing it maybe I missed it.
The fix is easy, just update your DSN in civicrm.settings.php to explicitly say "mysqli".
The technical issue is that without that silent conversion, it ends up trying to load php-mysql. It's just confusing at first because the error is `extension not found`, which makes you think "Civi Extension", but it means php extension.
Going to label it regression, but it's not the usual type of regression and it just started yesterday.5.30.0https://lab.civicrm.org/dev/core/-/issues/3976Upgrades slow as a result of snapshot code2022-11-10T06:58:52ZeileenUpgrades slow as a result of snapshot codeThe addition of the snapshot code causes queries to run that are slow on a large database - eg. `SELECT COUNT(*) FROM civicrm_contact` - this is for a database update with NO changes to any of the large tables it is doing slow queries on...The addition of the snapshot code causes queries to run that are slow on a large database - eg. `SELECT COUNT(*) FROM civicrm_contact` - this is for a database update with NO changes to any of the large tables it is doing slow queries on.
All the messaging about the snapshot is about how to turn it ON - not about how to stop it from doing it's 'activation test'
Note the test is an arbitrary check on the size of some tables thought to possibly signal a large database - but then it turns it ALL off or ALL on after some slow queries. This means the sorts of updates we would have found this useful on (scheduled reminders) may be unnecessarily de-activated.
@tottenhttps://lab.civicrm.org/dev/core/-/issues/3490Upgrader - Apply extension updates after core updates2023-02-08T19:08:30ZtottenUpgrader - Apply extension updates after core updatesOverview
----------------------------------------
Functionality in `civicrm-core` is increasingly organized with internal extensions (`civicrm-core:ext/*`). The main upgrade workflow should incorporate the upgrade-steps for extensions.
...Overview
----------------------------------------
Functionality in `civicrm-core` is increasingly organized with internal extensions (`civicrm-core:ext/*`). The main upgrade workflow should incorporate the upgrade-steps for extensions.
Example use-case
----------------------------------------
1. Extract tarball with a new version of `civicrm-core`, including updated `ext/*`
1. Run the main upgrader (`civicrm/upgrade?reset=1`, `cv core:upgrade`, `drush civicrm-upgrade-db`, or `wp civicrm upgrade-db`)
Current behaviour
----------------------------------------
It updates the schema for things like `civicrm_contact` (eg `CRM/Upgrade/*`) but not things like `civicrm_search_segment` (eg `ext/search_kit/CRM/Search/Upgrader.php`).
As a sysadmin, you must run the upgrades separately.
Most are not habituated to running these together. This leads to confusing user-stories like https://civicrm.stackexchange.com/questions/42094/civimail-issue-db-error-no-such-table-when-sending-mailing (where the first upgrade seemed to work - but you get random errors because the second upgrade hasn't run).
Proposed behaviour
----------------------------------------
Run both.
The core-upgrade should delegate/incorporate the ext-upgrade, and it should preserve essential elements of the ext-upgrade contract. Specifically: ext-upgrades can use features/services/APIs from core. This implies that (1) ext-upgrades run after core-upgrades and (2) when it gets to ext-upgrades, it should follow a normal/open dispatch-policy.
Comments
----------------------------------------
Filing issue after some Mattermost discussion with rudanpal, @colemanw, @bgm, @totten: https://chat.civicrm.org/civicrm/pl/b76y4gy517yxdet331ueqrxgrw
I posted a draft at https://github.com/civicrm/civicrm-core/compare/5.50...totten:run-ext-upgrades?expand=1
As mentioned in PHP comments+Mattermost, there's an issue with the current draft -- when it gets to ext-upgrade tasks, it would (unexpectedly) continue to apply the limited dispatch-policy used for core-upgrades (`upgrade.main` vs `upgrade.finish`). Some ways to address that:
* __Runner state__: While the upgrader goes through phases (*ex: 1-core upgrade, 2-ext upgrade, 3-new exts*), keep a stored value naming the phase. this stored value determines the active dispatch-policy
* ex: `Civi::settings()->set('upgrade_phase', '...')`
* __Task wrapping/task tagging__: When taking ext-upgrade tasks and putting them into the core-upgrade queue, put some wrapper or tag on each one to indicate the phase/policy that it relies on
* wrapper example: `CRM_Upgrade_Form::wrappedTask(string $dispatchPolicy, array $realCallback)
* tag example: `$queue->createItem($task, ['dispatchPolicy' => string])`https://lab.civicrm.org/dev/core/-/issues/1454Upgrade: Upgrade to 5.20.0 freeze on2019-12-06T20:29:51ZmarcineqUpgrade: Upgrade to 5.20.0 freeze onOverview
----------------------------------------
When i start my CiviCRM upgrade, upgrader freeze on task: [Executed: Change direction of autoassignees in case type xml].
I have CiviCase component configured with custom case type, wit...Overview
----------------------------------------
When i start my CiviCRM upgrade, upgrader freeze on task: [Executed: Change direction of autoassignees in case type xml].
I have CiviCase component configured with custom case type, with custom roles autoassign to case creator.
After refresh updater page - updater not run.
In Drupal log can i see this issue after notice about executed job.
`Error: Class 'CRM_Documents_Entity_DocumentRepository' not found w documents_civicrm_pre() (linia 172 z /path/to/crm/sites/default/files/civicrm/ext/org.civicoop.documents/documents.php).`
I tried turn off Document extension and restore database from backup. After this i see this error in log:
Error: Class 'CRM_CiviMobileAPI_PushNotification_Utils_NotificationFactory' not found w civimobileapi_civicrm_pre() (linia 342 z /path/to/civicrm/sites/default/files/civicrm/ext/com.agiliway.civimobileapi/civimobileapi.php).
In CiviCRM log i don't have anything of this errors.
Reproduction steps
----------------------------------------
1. Try to upgrade CiviCRM from 5.19.3 to 5.20.0 with enable CiviCase
Current behaviour
----------------------------------------
![image](/uploads/79b14274de224db4f551476d4588c453/image.png)
Expected behaviour
----------------------------------------
Upgrade is finished.
Environment information
----------------------------------------
<!-- Some of the items below may not be relevant for every bug - if in doubt please include more information than you think is neccessary. -->
* __Browser:__ _Chrome 78.0.3904.108_
* __CiviCRM:__ _5.19.2 to 5.20.0_
* __PHP:__ _7.2__
* __CMS:__ _Drupal 7.67_
* __Database:__ _10.3.20-MariaDB_
* __Web Server:__ _Apache 2.4_
Comments
----------------------------------------
My extensions:
![image](/uploads/2101f41e3155a0098890bbe323e40752/image.png)