Development issueshttps://lab.civicrm.org/groups/dev/-/issues2021-11-15T07:59:28Zhttps://lab.civicrm.org/dev/core/-/issues/2942Displaying map for contact address causes WP error2021-11-15T07:59:28Ztreasurer@tricitiesprideDisplaying map for contact address causes WP errorOverview
----------------------------------------
Selecting "map" from the contact detail causes a Wordpress error.
![image](/uploads/6da1e74e2278d36dd879c7cf4a106605/image.png)
![image](/uploads/86c348df39b887caa6d56a9f92756bee/image....Overview
----------------------------------------
Selecting "map" from the contact detail causes a Wordpress error.
![image](/uploads/6da1e74e2278d36dd879c7cf4a106605/image.png)
![image](/uploads/86c348df39b887caa6d56a9f92756bee/image.png)
Reproduction steps
----------------------------------------
1. Click on **Contacts -> Map**.
Current behaviour
----------------------------------------
Selecting "map" from the contact detail causes a Wordpress error.
![image](/uploads/6da1e74e2278d36dd879c7cf4a106605/image.png)
![image](/uploads/86c348df39b887caa6d56a9f92756bee/image.png)
```
Error Details
=============
An error of type E_ERROR was caused in line 128 of the file /wp-content/plugins/civicrm/civicrm/CRM/Contact/Form/Task/Map.php. Error message: Uncaught Error: Using $this when not in object context in /wp-content/plugins/civicrm/civicrm/CRM/Contact/Form/Task/Map.php:128
Stack trace:
#0 /wp-content/plugins/civicrm/civicrm/CRM/Contact/Form/Task/Map.php(92): CRM_Contact_Form_Task_Map::createMapXML(Array, 1, Object(CRM_Contact_Form_Task_Map), true, 'Contact')
#1 /wp-content/plugins/civicrm/civicrm/CRM/Core/Form.php(629): CRM_Contact_Form_Task_Map->preProcess()
#2 /wp-content/plugins/civicrm/civicrm/CRM/Core/QuickForm/Action/Display.php(76): CRM_Core_Form->buildForm()
#3 /wp-content/plugins/civicrm/civicrm/packages/HTML/QuickForm/Controller.php(203): CRM_Core_QuickForm_Action_Display->perform(Object(CRM_Contact_Form_Task_Map), 'display')
#4 /wp-content/plugins/civicrm/civicrm/packages/HTML/QuickForm/Page.php(103): HTML_QuickForm_Controller->handle(Object(CRM_C
```
Expected behaviour
----------------------------------------
Map should display
Environment information
----------------------------------------
* __Browser:__ _Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36_
* __CiviCRM:__ Version 5.43.0
* __PHP:__ _7.4.22__
* __CMS:__ _Wordpress Version 5.8.1_
* __Database:__ MariaDB 10.3.31-MariaDB-log-cll-lve - MariaDB Server..._
* __Web Server:__ Apache 2.4.48_5.43.1https://lab.civicrm.org/dev/core/-/issues/2939getFieldValue failed on empty multiple-contact reference field2022-07-02T16:45:27ZAdam WoodgetFieldValue failed on empty multiple-contact reference fieldWhen submitting a record that includes a custom field of multiple contact reference type, if that field is empty (i.e. no contacts), an error message "getFieldValue failed" sometimes occurs and the screen freezes and needs to be reloaded...When submitting a record that includes a custom field of multiple contact reference type, if that field is empty (i.e. no contacts), an error message "getFieldValue failed" sometimes occurs and the screen freezes and needs to be reloaded. The record update seems to go ahead nonetheless.
This seems to be due to a change introduced in https://github.com/civicrm/civicrm-core/pull/18941 that adds support to CRM/Core/BAO/CustomField.php for multiple contact fields. The new logic checks that the value is of the correct type (array or numeric) but omits the check that the value is not empty. (Before this change, one of the conditions was that the value does not equate to false).
Before change (note the `&& $value`):
```php
if ($field['data_type'] == 'ContactReference' && $value) {
if (is_numeric($value)) {
$display = CRM_Core_DAO::getFieldValue('CRM_Contact_DAO_Contact', $value, 'display_name');
}
else {
$display = $value;
}
}
```
After change:
```php
if ($field['data_type'] == 'ContactReference' && (is_array($value) || is_numeric($value))) {
$displayNames = [];
foreach ((array) $value as $contactId) {
$displayNames[] = CRM_Core_DAO::getFieldValue('CRM_Contact_DAO_Contact', $contactId, 'display_name');
}
$display = implode(', ', $displayNames);
}
elseif ($field['data_type'] == 'ContactReference') {
$display = $value;
}
```
Suggested fix - remove vulnerability to empty scalar values being passed:
```php
if ($field['data_type'] == 'ContactReference' && (is_array($value) || is_numeric($value))) {
if (empty($value))
{
$display = '';
}
else
{
$displayNames = [];
foreach ((array) $value as $contactId) {
$displayNames[] = CRM_Core_DAO::getFieldValue('CRM_Contact_DAO_Contact', $contactId, 'display_name');
}
$display = implode(', ', $displayNames);
}
}
elseif ($field['data_type'] == 'ContactReference') {
$display = $value;
}
```5.45.0https://lab.civicrm.org/dev/drupal/-/issues/168Notice: Undefined variable: custom_link in civicrm_handler_field_link_contact...2022-12-01T01:13:06ZAndrew WassonNotice: Undefined variable: custom_link in civicrm_handler_field_link_contact->render_link() (line 118 of civicrm/drupal/modules/views/civicrm/civicrm_handler_field_link_contact.inc).Hi Guys,
It's been a while (years) since I've contributed in the issue queue so I'm a little rusty.
This is just an annoyance where CiviCRM tries to render the Contact Link in a view for visitors who don't have the permissions to see i...Hi Guys,
It's been a while (years) since I've contributed in the issue queue so I'm a little rusty.
This is just an annoyance where CiviCRM tries to render the Contact Link in a view for visitors who don't have the permissions to see it. The result is a PHP Warning:
```Notice: Undefined variable: custom_link in civicrm_handler_field_link_contact->render_link() (line 118 of /sites/all/modules/civicrm/drupal/modules/views/civicrm/civicrm_handler_field_link_contact.inc).```
In order to squelch the notice I've added a conditional isset($custom_link). I can't see any adverse consequences from adding it and I'll be happy to make a patch if someone can point me in the direction of some docs for doing so.
Cheers,
Andrew5.57.0https://lab.civicrm.org/dev/core/-/issues/2927Warning: is_dir(): open_basedir restriction in effect2022-02-16T14:47:09ZherbdoolWarning: is_dir(): open_basedir restriction in effectOverview
----------------------------------------
See https://github.com/civicrm/civicrm-core/pulls?q=basedir+is%3Aopen (saw a bunch of PRs as highlighted in the dev digest but no associated issue).
Happens with APIv3 Explorer (https:/...Overview
----------------------------------------
See https://github.com/civicrm/civicrm-core/pulls?q=basedir+is%3Aopen (saw a bunch of PRs as highlighted in the dev digest but no associated issue).
Happens with APIv3 Explorer (https://github.com/civicrm/civicrm-core/pull/21593), /CRM/Core/Smarty/plugins/ (https://github.com/civicrm/civicrm-core/pull/21589), etc.
Reproduction steps
----------------------------------------
When a server has open_basedir restrictions, error message logged repeatedly when using CiviCRM backend, especially with APIv3. PHP message: PHP Warning: is_dir(): open_basedir restriction in effect. File ... is not within the allowed path(s)
Comments
----------------------------------------
Also see https://lab.civicrm.org/dev/core/-/issues/2795 - the solution should not simply suppress the errors.5.48.0https://lab.civicrm.org/dev/core/-/issues/2925`%%dbSSL%%` variables left behind in civicrm.setting.php after install2023-10-09T18:45:44ZDaveD`%%dbSSL%%` variables left behind in civicrm.setting.php after installThis doesn't come up with `cv core:install` or any method that uses civicrm-setup, but does come up with drush and the legacy installer (e.g. sites/all/modules/civicrm/install/index.php). It will be fixed going forward but anyone who use...This doesn't come up with `cv core:install` or any method that uses civicrm-setup, but does come up with drush and the legacy installer (e.g. sites/all/modules/civicrm/install/index.php). It will be fixed going forward but anyone who used e.g. drush, wp-cli, or the legacy installer to install civi since 5.31 will likely have the `%%dbSSL%%` and `%%CMSdbSSL%%` variables left behind in CIVICRM_DSN and CIVICRM_UF_DSN in civicrm.settings.php.
Since it gets ignored during parsing/usage, it's mostly cosmetic at the moment. So:
1. Having an upgrade message seems like it's asking people to do something that's optional, which should maybe be handled a different way.
1. Having a status message will just get ignored by most staffers since they can't fix it anyway. And the tech people who deal with civicrm.settings.php might never see it. In some orgs it's the same person, but it will miss many.
1. Maybe in the docs in the version-specific steps, but again many people will miss that, but then at least it's not bothering anyone.
1. Dev-digest?
1. Blog?
1. Stackexchange - e.g. `Why do I have %%dbSSL%% in my civicrm.settings.php and can I remove it?`5.62.0https://lab.civicrm.org/dev/core/-/issues/2919Hidden submit buttons are confussing for keyboard users2021-11-17T19:56:33ZBradley TaylorHidden submit buttons are confussing for keyboard usersModals are used in various parts of the CiviCRM. For example:
![Screenshot_2021-10-17_at_16.39.33](/uploads/a0f38e37e9ecf5423826d1e8dd06973c/Screenshot_2021-10-17_at_16.39.33.png)
The two buttons in the bottom-right are actually genera...Modals are used in various parts of the CiviCRM. For example:
![Screenshot_2021-10-17_at_16.39.33](/uploads/a0f38e37e9ecf5423826d1e8dd06973c/Screenshot_2021-10-17_at_16.39.33.png)
The two buttons in the bottom-right are actually generated based on buttons within the modal content, which are hidden with CSS. With this CSS removed, the modal actually looks like this:
![Screenshot_2021-10-17_at_16.41.51](/uploads/365c9c0577f3e89a12451f36c059e828/Screenshot_2021-10-17_at_16.41.51.png)
The problem is that these buttons are still keyboard focussable when tabbing. If a user hits their enter key whilst one of these keys are focused, then the modal may be saved or cancelled without any visual indication of what they triggered.
The code contains this comment, which is fair enough if a field is actually focussed but not when a non-visible field is focussed:
`// display:none causes the form to not submit when pressing "enter"`
I'd like to propose that the hidden buttons are made non-focussable with `tabindex="-1"`. I think this should be a safe change, but it would be good to hear if anyone sees any potential issues with this change.
For accessibility, ideally the focussable submit button should be within the form. However, as there is consistency across how modals work within CiviCRM I think the proposed change should be ok from an accessibility point of view.5.45.0https://lab.civicrm.org/dev/core/-/issues/2918Users cannot be created if no unsupervised deduping rule exists2022-09-01T21:00:57ZBradley TaylorUsers cannot be created if no unsupervised deduping rule existsWe have recently had an issue where users could not be added via the WordPress admin add new user screen. The following exception was logged: `Unsupervised rule for Individual does not exist`
**Steps to reproduce:**
- Go to the CiviCRM ...We have recently had an issue where users could not be added via the WordPress admin add new user screen. The following exception was logged: `Unsupervised rule for Individual does not exist`
**Steps to reproduce:**
- Go to the CiviCRM dedup rules config screen (/wp-admin/admin.php?page=CiviCRM&q=civicrm%2Fcontact%2Fdeduperules&reset=1)
- For any unsupervised rules under the "Individual Rules" table, either delete the rule, or change the usage type to "General".
- Try adding a new WordPress user at /wp-admin/user-new.php
I've not tested this, but I expect the easiest course of action is to wrap the call to `synchronizeUFMatch` in `CiviCRM_For_WordPress_Users` in a try-catch block. I've not checked to see if this issue affects Drupal/ Joomla.
There is currently nothing in the documentation to suggest having an unsupervised rule is a requirement, but if not having an unsupervised rule will cause other problems then this should be clearly stated in https://docs.civicrm.org/user/en/latest/common-workflows/deduping-and-merging/5.49.0https://lab.civicrm.org/dev/core/-/issues/2917Replace "Oops" with more appropriate language2021-10-18T17:27:35ZhaystackReplace "Oops" with more appropriate languageOverview
----------------------------------------
The word "Oops" appears throughout CiviCRM. I suggest changing the strings in which it appears to be more appropriate to their context.
Example changes...
#### Already registered for a...Overview
----------------------------------------
The word "Oops" appears throughout CiviCRM. I suggest changing the strings in which it appears to be more appropriate to their context.
Example changes...
#### Already registered for an Event:
* "Oops. It looks like you are already registered for this event. [...]"
* "It looks like you are already registered for this event. [...]"
The "Oops" is jarring. It's not a mistake when I revisit the form to register someone else.
#### Event full:
* "Oops, it looks like there are currently no available spaces for [...]"
* "Unfortunately there are currently no available spaces for [...]"
It's not a mistake that "there are currently no available spaces". "Oops" suggests that it is.
Etc etc.5.44.0haystackhaystackhttps://lab.civicrm.org/dev/core/-/issues/2916Alter default frequency for scheduled reminders job2021-11-05T17:46:04ZwmortadaAlter default frequency for scheduled reminders jobAs [discussed on Mattermost](https://chat.civicrm.org/civicrm/pl/hcnos88x3pdzdgfkqbjbdktggh) with @bgm, the default for sending scheduled reminders is daily. This seems too infrequent, particularly if you have event reminders to be sent ...As [discussed on Mattermost](https://chat.civicrm.org/civicrm/pl/hcnos88x3pdzdgfkqbjbdktggh) with @bgm, the default for sending scheduled reminders is daily. This seems too infrequent, particularly if you have event reminders to be sent and want to send a reminder e.g. 2 hours before the event starts.
I propose that the default is changed to hourly.
The default is set here: https://lab.civicrm.org/dev/core/-/blob/ca09f9073ec01ba0a67e1c435213dea878069299/xml/templates/civicrm_data.tpl#L16835.45.0https://lab.civicrm.org/dev/core/-/issues/2914Count of sent messages can be wrong in status popup for email form2021-10-18T14:28:03ZDaveDCount of sent messages can be wrong in status popup for email formMaster and 5.43 are broken right at this moment and you can't send any email, but in 5.43+ if you do this:
1. Contacts - New Email.
1. Add one of more valid recipients.
1. Fill it out and send.
1. The status popup will say 0 messages se...Master and 5.43 are broken right at this moment and you can't send any email, but in 5.43+ if you do this:
1. Contacts - New Email.
1. Add one of more valid recipients.
1. Fill it out and send.
1. The status popup will say 0 messages sent, even though they were sent. The number it shows is however many were prepopulated on the form when it first opened, and doesn't account for any you added or subtracted.
(This isn't about emails that were for deceased or on-hold etc.)
It worked in 5.42.5.43.0https://lab.civicrm.org/dev/core/-/issues/2912Can't save extra custom fields when inline and required2021-11-11T19:25:39Zaydunsaidan.saunders@squiffle.ukCan't save extra custom fields when inline and requiredOverview
----------------------------------------
If you have a custom field set that allows multiple records with a required field and uses the inline style, then one record can be created but a second fails with ' ... is a required fie...Overview
----------------------------------------
If you have a custom field set that allows multiple records with a required field and uses the inline style, then one record can be created but a second fails with ' ... is a required field' for all required fields.
Reproduction steps
----------------------------------------
Setup: Create a custom field set:
1. Click on **Administer -> Customize Data and Screens > Custom Fields**.
1. Click **Add Set of Custom Fields**
1. Set Name: 'Test fields', Used for: Contacts, Does this Custom Field Set allow multiple records?: ticked, Display Style: Inline
1. Click **Save**
1. Field Label: Test, Data Type: Alphanumeric, Field input type: single-line input, Required: ticked [NB seems to affect all types, key thing is this is Required]
1. Click **Save**
Create first record:
1. Choose a contact
1. In 'Test fields' custom fields section, click **Edit**, enter a value for 'Test' field, click **Save**
[Should succeed]
Try to create second record - need to use 'Edit Contact'
1. On contact view, click **Edit**
1. In 'Test fields' section, click **Another Test fields record**
1. Enter value, click **Save**
1. Get ```Please correct the following errors in the form fields below: Test is a required field.```
Current behaviour
----------------------------------------
Second field produces error
Expected behaviour
----------------------------------------
Second field saves without error
Environment information
----------------------------------------
* __CiviCRM:__ _5.41.0_, _Master._ This used to work seems to have changed somewhere between 5.36.1 and 5.41.0
Comments
----------------------------------------
* The field saves correctly if not 'required'.
* The field saves correctly if displayed as 'Tab' or 'Tab with Table'. Problem occurs only when both 'Required' and 'Inline'.5.45.0https://lab.civicrm.org/dev/core/-/issues/2910Remove the CiviCRM Connections functionality (Task 1 of 2) - Remove the menu ...2022-08-09T14:44:38Zjustinfreeman (Agileware)Remove the CiviCRM Connections functionality (Task 1 of 2) - Remove the menu item**CiviCRM Connections** is available under the System Settings menu. From what I can tell this feature is an artefact that is no longer being used. Happy to be educated on the subject if this is not the case.
Cannot find any references ...**CiviCRM Connections** is available under the System Settings menu. From what I can tell this feature is an artefact that is no longer being used. Happy to be educated on the subject if this is not the case.
Cannot find any references to it in the documentation:
- https://docs.civicrm.org/user/en/latest/
- https://docs.civicrm.org/sysadmin/en/latest/
Googling doesn't reveal anything relevant either.
So can we remove this code from the system?
![Screenshot_20211013_091315](/uploads/724bb11ee0a345f31be5b369b22af959/Screenshot_20211013_091315.png)
Agileware Ref: CIVIBLD-2865.44.0https://lab.civicrm.org/dev/core/-/issues/2909Repair "Norwegian Bokmål"2021-11-04T17:48:03ZhaystackRepair "Norwegian Bokmål"Overview
----------------------------------------
"Norwegian Bokmål" is corrupted in [this file](https://github.com/civicrm/civicrm-core/blob/5db0bc3c1f54eaca4307f103a73bda596ae914d6/install/langs.php#L32) and [this one](https://github.c...Overview
----------------------------------------
"Norwegian Bokmål" is corrupted in [this file](https://github.com/civicrm/civicrm-core/blob/5db0bc3c1f54eaca4307f103a73bda596ae914d6/install/langs.php#L32) and [this one](https://github.com/civicrm/civicrm-core/blob/5db0bc3c1f54eaca4307f103a73bda596ae914d6/xml/templates/languages.tpl#L136).
Current behaviour
----------------------------------------
"Norwegian Bokmål" renders as "Norwegian BokmÃ¥l"
Expected behaviour
----------------------------------------
"Norwegian Bokmål" FTW.5.44.0haystackhaystackhttps://lab.civicrm.org/dev/core/-/issues/2904Date fields are empty on memberships created with Order API Pending2021-10-14T06:25:33ZFrancis (Agileware)Date fields are empty on memberships created with Order API PendingSame problem as #2902.
Example Order.create API Call to reproduce:
```json
{
"contact_id": 202,
"total_amount": 100.00,
"financial_type_id": "Member Dues",
"contribution_status_id": "Pending",
"line_items" : [
{
"p...Same problem as #2902.
Example Order.create API Call to reproduce:
```json
{
"contact_id": 202,
"total_amount": 100.00,
"financial_type_id": "Member Dues",
"contribution_status_id": "Pending",
"line_items" : [
{
"params": {
"membership_type_id": "General",
"contact_id": 202,
"status_id": "Pending",
"skipStatusCal":1
},
"line_item": [
{
"entity_table":"civicrm_membership",
"price_field_id":"4",
"price_field_value_id":"7",
"qty":"1",
"unit_price":"100.00",
"line_total":"100.00"
}
]
}
]
}
```
Result on 5.39 is a pending Contribution and Membership with join_date, start_date, and end_date
Result on 5.40+ is a pending Contribution and Membership with *NO* join_date, start_date, or end_date
Including 5.43alpha with @eileen's [patch for #2902](https://github.com/civicrm/civicrm-core/pull/21784) applied
5.42+ works correctly *without* skipStatusCal set, however for backwards compatibility it should work with it also.5.42.1https://lab.civicrm.org/dev/drupal/-/issues/167Drupal service 'site.path' is deprecated in drupal 9 and will be removed in d...2022-06-11T19:26:30ZDaveDDrupal service 'site.path' is deprecated in drupal 9 and will be removed in drupal 10In https://github.com/civicrm/civicrm-core/blob/master/setup/src/Setup/DrupalUtil.php#L23 it calls `\Drupal::service('site.path');`.
But I'm wondering why that function getDrupalSiteDir() even exists and why it's not part of CRM_Utils_S...In https://github.com/civicrm/civicrm-core/blob/master/setup/src/Setup/DrupalUtil.php#L23 it calls `\Drupal::service('site.path');`.
But I'm wondering why that function getDrupalSiteDir() even exists and why it's not part of CRM_Utils_System_X. Maybe a timing issue since civi isn't installed yet at that point?
Anyway the relevant drupal issue is https://www.drupal.org/project/drupal/issues/3074585#comment-13220446.
Because of the (bleep) `@trigger_error` usage that drupal uses though it only comes up when running unit tests in drupal's own testing environment (until drupal 10 when it will hard fail, effectively nulling the effect of outputting a deprecation for most people).5.52.0https://lab.civicrm.org/dev/core/-/issues/2902Date fields are empty on memberships (created with pay later OR with an API c...2021-10-14T04:08:04ZjitendraDate fields are empty on memberships (created with pay later OR with an API call)To replicate -
- Create a contribution page with Membership + Pay later.
- Submit the page in live mode.
- Date fields (join, start & end) are empty for the membership.
Found while writing this test - https://github.com/colemanw/webfor...To replicate -
- Create a contribution page with Membership + Pay later.
- Submit the page in live mode.
- Date fields (join, start & end) are empty for the membership.
Found while writing this test - https://github.com/colemanw/webform_civicrm/pull/622
Probably related to https://github.com/civicrm/civicrm-core/pull/20759 merge where api/v3 code used to calculate the dates for all NEW memberships. https://github.com/civicrm/civicrm-core/pull/20759/files#diff-74f06c64d06b6b6c191ff86ab31113f7198e73c0a1f8c5487116e7b3faef26fcL95
Now, in BAO, the membership date calculation is avoided if `skipStatusCal` is set in the params. See https://github.com/civicrm/civicrm-core/blob/master/CRM/Member/BAO/Membership.php#L260
I think a test with below API should be able to replicate it in the test
```
$membership = civicrm_api3('Membership', 'create', [
'membership_type_id' => "General",
'contact_id' => 2,
'status_id' => "Current", (or Pending, New, etc)
'skipStatusCal' => 1,
]);
// Verify if membership dates are correctly set. Currently, they're not.
```
I've confirmed that CiviCRM v5.35 sets the date on the above API call, but >5.40 does NOT. Does that seem to be an unintentional change of behavior?
Not sure what should be the desired result - Does the API call need to set start_date separately when `skipStatusCal` is enabled?
OR
`skipStatusCal` should only skip the calculation of "status" on the membership and continue to set the dates as it used to do in previous versions?
fyi - @KarinG @eileen5.42.1https://lab.civicrm.org/dev/core/-/issues/2901Find Activities + Send Email task always says the contact is on hold or decea...2021-10-10T12:48:05ZDaveDFind Activities + Send Email task always says the contact is on hold or deceased etcEven if all the communication prefs are good and the email is valid it still gives the warning message and doesn't let you send.
1. Find Activities.
2. Select one that has a valid With Contact who isn't deceased and doesn't have email o...Even if all the communication prefs are good and the email is valid it still gives the warning message and doesn't let you send.
1. Find Activities.
2. Select one that has a valid With Contact who isn't deceased and doesn't have email on hold etc.
3. From the actions dropdown choose send email
4. Choose With Contact.
5. Warning comes up and redirects you back to search. `Selected contact(s) do not have a valid email address, or communication preferences specify DO NOT EMAIL, or they are deceased or Primary email address is On Hold.`
I'm putting regression but not 100% sure it is recent.5.43.0https://lab.civicrm.org/dev/core/-/issues/2898URL Tokens - CiviMail BAO and FlexMailer disagree over duplicate `http://`2022-12-19T22:45:31ZtottenURL Tokens - CiviMail BAO and FlexMailer disagree over duplicate `http://`Overview
--------
CiviMail BAO and Flexmailer provide two mechanisms to rendering/delivering mail-blasts. To phase out BAO-rendering, we should eventually migrate existing sites to Flexmailer. However, small discrepancies between them c...Overview
--------
CiviMail BAO and Flexmailer provide two mechanisms to rendering/delivering mail-blasts. To phase out BAO-rendering, we should eventually migrate existing sites to Flexmailer. However, small discrepancies between them can be an obstacle. This issue asks how to deal with a discrepancy when handling certain URL tokens.
(The issue is an off-shoot from the review discussion on https://github.com/civicrm/civicrm-core/pull/21522. However, they are somewhat distinct as 21522 applies to new-sites and this problem is more pressing on upgrade-sites.)
Background
-----------
Suppose you have an email token which generates a URL (eg `{action.forward}` or `{action.unsubscribeUrl}`) . The most correct way to use this token in an HTML-email is to place it in a hyperlink (`<a href="{action.forward}>`).
However, if you use CKEditor to compose a message, the hyperlink dialog strongly encourages you to put `http://` at the front of any hyperlink. Thus, you can organically produce expressions like `<a href="http://{action.forward}">`.
![CkeditorHttpToken](/uploads/e2d9bbf4ab6ba12a6cdcd936955277a9/CkeditorHttpToken.mov)
CiviMail BAO has a feature which mitigates this - in effect, both notations give the same output. However, Flexmailer (TokenProcessor) does not have a similar mitigation. Thus:
| HTML Email Expression | BAO Output | Flexmailer Output |
| -- | -- | -- |
| `<a href="{action.forward}>` | `<a href="https://example.com/civicrm/mailing/forward?...` | `<a href="https://example.com/civicrm/mailing/forward?...` |
| `<a href="http://{action.forward}>` | `<a href="https://example.com/civicrm/mailing/forward?...` | `<a href="http://https://example.com/civicrm/mailing/forward?...` |
Note the duplicate URL scheme (`http://http://`).
Scenarios: Good
---------
There are several scenarios wherein this discrepancy does not matter:
* You use CKEditor + BAO-renderer. (*The BAO renderer mitigates extraneous `http://`.*)
* You use Mosaico + Flexmailer. (*Mosaico's UI doesn't create extraneous `http://`.*)
(I suspect these two cohorts are the largest.)
Scenarios: Bad
---------
The potential for difficulty arises when mixing CKEditor with Flexmailer -- you take some HTML that was composed with CKEditor, run it through Flexmailer, and now you have invalid URLs with `http://https://`.
If we flip over more existing sites to Flexmailer, then it is foreseeable that more sites will be in this mixed scenario.
There's a secondary consideration in who may experience problems -- templates. Most users are not in the habit of typing `{action.forward}` or `{action.unsubscribeUrl}` regularly. Anecdotally, the typical practice is to put these kinds of tokens into a template, eg
* (A) Add a "User-Defined Template" (edited w/CKEditor in web; stored in `civicrm_msg_template`), or...
* (B) Reuse/copy a previous "Mailing" (edited w/CKEditor in web; stored in `civicrm_mailing`), or...
* (C) Configure default header/footer (no CKEditor; stored in `civicrm_mailing_component`)
Practical recap: today, if you enable Flexmailer on a site that uses CKEditor for mailings, then either:
* Your mailings and templates will have broken links -- because they use `<a href="http://{action.unsubscribeUrl}">`.
* This seems more likely if you use (A)/(B) - because the web UI uses CKEditor.
* Your mailings and templates will be cleanly interoperable -- because they use `<a href="{action.unsubscribeUrl}">`.
* This seems more likely if you use (C) header/footer because the web UI uses textarea.
Approaches
----------
1. Add a mitigation to Flexmailer or TokenProcessor which removes extraneous schema during the rendering phase.
* _Strength_: Best interoperability
* _Criticism_: Adds quirky bits to rendering.
* _Variations_: Patch Flexmailer vs patch TokenProcessor.
* _Variations_: Apply cleanup before rendering (`http://{action.*}` => `{action.*}`) or after rendering (`http://https?://` => `https?://`).
2. Add an upgrade-task to remove extra `http://` in front of tokens.
* _Strength_: No quirky bits in rendering.
* _Criticism_: CKEditor is still there. Can only upgrade core tables. Rewrites history (`civicrm_mailing.body_html` of past mailings).
3. Do nothing - Downstream users should manually convert their mailings/templates.
* _Strength_: Look, ma, no hands!
* _Criticism_: We cannot measure how many users will be impacted by break. For users with a "Reuse/Copy" workflow, they cannot fix historical mailings in UI.5.57.0https://lab.civicrm.org/dev/core/-/issues/2895Emails can't resolve tokens for non-contact components for recipients added t...2021-10-13T20:23:45ZDaveDEmails can't resolve tokens for non-contact components for recipients added to the list after email form was openedhttps://github.com/civicrm/civicrm-core/pull/21745#discussion_r724286717
Affects 5.43 only.
e.g.
1. Create a case email, e.g. clicking on the mail icon in the roles section.
1. Add a recipient to the recipient field.
1. In the body pu...https://github.com/civicrm/civicrm-core/pull/21745#discussion_r724286717
Affects 5.43 only.
e.g.
1. Create a case email, e.g. clicking on the mail icon in the roles section.
1. Add a recipient to the recipient field.
1. In the body put something like `Dear {contact.display_name} Case {case.id}`
1. The email sent to the original recipient will resolve case.id. The added recipient will have it blank.
See below this also applies to other non-contact components, e.g. event tokens.5.43.0https://lab.civicrm.org/dev/core/-/issues/2892Replace event type token in action schedule changes event.event_type_id to pa...2021-10-06T08:34:02Zjustinfreeman (Agileware)Replace event type token in action schedule changes event.event_type_id to participant.event_type_id:labelReplace event type token in action schedule changes event.event_type_id to participant.event_type_id:label. But should be event.event_type_id to event.event_type_id:label
As identified by @magnolia61 here https://github.com/civicrm/civi...Replace event type token in action schedule changes event.event_type_id to participant.event_type_id:label. But should be event.event_type_id to event.event_type_id:label
As identified by @magnolia61 here https://github.com/civicrm/civicrm-core/pull/21666/files#r7200592855.43.0