CiviCRM Core issueshttps://lab.civicrm.org/dev/core/-/issues2022-06-27T21:54:03Zhttps://lab.civicrm.org/dev/core/-/issues/3677API thoughts: Global switch to turn off all permission checking, not just to ...2022-06-27T21:54:03ZDaveDAPI thoughts: Global switch to turn off all permission checking, not just to the "top-level" api call that was madeI've been thinking about https://lab.civicrm.org/dev/core/-/issues/3671, and my thinking went something like:
* This worked before, so it should go back to the way it was.
* But wait, isn't this really exposing a bug in webform because ...I've been thinking about https://lab.civicrm.org/dev/core/-/issues/3671, and my thinking went something like:
* This worked before, so it should go back to the way it was.
* But wait, isn't this really exposing a bug in webform because how did the person update a case if they don't have permissions to cases?
* But wait, what's this? Every api call in webform purposely turns off permissions? https://github.com/colemanw/webform_civicrm/blob/0bedf9411bf3f0a682ce7a1b3fad9a682093b240/src/Utils.php#L654
```php
function wf_civicrm_api($entity, $operation, $params) {
...
$params += [
'check_permissions' => FALSE,
];
```
* So webform is saying: "Core, I don't want you to check any permissions. Anyone who has (drupal) permission to use this form should have permission to do whatever the form does."
* I don't remember where but I think I saw a similar type of thing come up with afform, or it seems likely to, where "I've only given certain people permission to reach this form, but if they have it, they should be able to do whatever the form does."
So one way to address this use-case, in a way that doesn't require core to lower permission checking for everyone else, is to provide a way for webform and others to set a global kill-switch that turns off all permission checking. That way secondary permission checks that get called from within the api itself like the one in Utils_Recent would still be on for everyone else, but could be turned off for forms that don't want it.
I'm not confident this is the _best_ way, but am just putting it out there.https://lab.civicrm.org/dev/core/-/issues/3646Support tracking URLs with tokens in query strings2022-06-11T14:58:11ZlarsssandergreenSupport tracking URLs with tokens in query stringsURLs with tokens in mailings do not have click tracking. This is an important metric for fundraising (e.g. to see how many and even which contacts click through to contribution pages). @artfulrobot put together some code that worked well...URLs with tokens in mailings do not have click tracking. This is an important metric for fundraising (e.g. to see how many and even which contacts click through to contribution pages). @artfulrobot put together some code that worked well to track these clicks in Flexmailer, but that code was not merged before Flexmailer was merged into core.
https://github.com/civicrm/org.civicrm.flexmailer/issues/30
We used this without issue for four months and found it worked very well. @artfulrobot was using it since March. Would it be possible to merge this into core. Rich, would you be willing to copy your code to a patch against core?RichRichhttps://lab.civicrm.org/dev/core/-/issues/3638Provide a fullPage property for message templates2024-02-22T05:03:26ZmfbProvide a fullPage property for message templatesIn various parts of the UI, CiviCRM allows users to send a message template either on its own, or wrapped in a header and footer.
e.g. you can send a message template to a contact.
Or, you can create a mailing based on a message templa...In various parts of the UI, CiviCRM allows users to send a message template either on its own, or wrapped in a header and footer.
e.g. you can send a message template to a contact.
Or, you can create a mailing based on a message template, with additional header and footer.
CKEditor needs different settings at civicrm/admin/ckeditor?preset=civimail re: allowed HTML tags to support these two scenarios. With fullPage = true, the HTML is forcibly wrapped with html/head/body tags, and so a message template that includes its header and footer can be composed. With fullPage = false, html/head/body are silently stripped, so this is useful for message templates that should be used with a header and footer.
However, there is only one CKEditor setting for both of these scenarios.
I would propose a new database column where we can indicate if a message template is "fullPage" - i.e. includes header and footer, or not fullPage, i.e. designed to be used with header and footer.
This would allow us to toggle the fullPage setting appropriately. It would also allow us to prevent a user from sending a message template that needs a header and footer without one, or wrapping a message template that includes its header and footer with an additional header and footer.https://lab.civicrm.org/dev/core/-/issues/3636CiviCRM, Find Mailings. If there are no results, the same message "There are ...2022-06-11T14:57:49Zjustinfreeman (Agileware)CiviCRM, Find Mailings. If there are no results, the same message "There are no Unscheduled Mailings" is shownCiviCRM, Find Mailings. If there are no results, the same message "There are no Unscheduled Mailings" is shown regardless of the search criteria. If a different mailing status was used for search, it does not make sense to refer to "Unsc...CiviCRM, Find Mailings. If there are no results, the same message "There are no Unscheduled Mailings" is shown regardless of the search criteria. If a different mailing status was used for search, it does not make sense to refer to "Unscheduled Mailings".
This text should be changed to something else like: "There are no mailings matching this criteria".
Using CiviCRM 5.7.2
Agileware Ref: CIVICRM-1121
![Find_Mailings-2](/uploads/34e6b1996338721a1a2a333a3a16cd2a/Find_Mailings-2.png)https://lab.civicrm.org/dev/core/-/issues/3635Emails marked as bounce when quota is full2024-02-22T05:03:25ZEdselopezEmails marked as bounce when quota is fullThere are cases (usually when using Gmail SMTP) when the Outbound email fails to deliver a mailing due to the user's daily quota being exhausted. In this case, if the mailing is re-used unknowingly, attempts to re-send the mailing adds t...There are cases (usually when using Gmail SMTP) when the Outbound email fails to deliver a mailing due to the user's daily quota being exhausted. In this case, if the mailing is re-used unknowingly, attempts to re-send the mailing adds to the bounce threshold, finally surpassing the limit of three bounces, marking the email as "on hold". To solve this, I'm thinking bounce type of "Quota" should be excluded from the calculation of marking the email as on hold. Please let me know if this is reasonable and I will provide a patch.https://lab.civicrm.org/dev/core/-/issues/3634When using a smart group as a mailing list, users who unsubscribe from the sm...2022-06-11T14:57:45Ztom.mWhen using a smart group as a mailing list, users who unsubscribe from the smart group are still included in the mailing.It looks like in CRM/MailingBAO/Mailing.php, is not taking into account those who have been removed from the smart group. Civi version 5.3.
```
if (count($includeSmartGroupIDs)) {
$query = CRM_Utils_SQL_Select::from($contact)
...It looks like in CRM/MailingBAO/Mailing.php, is not taking into account those who have been removed from the smart group. Civi version 5.3.
```
if (count($includeSmartGroupIDs)) {
$query = CRM_Utils_SQL_Select::from($contact)
->select("$contact.id as contact_id, $entityTable.id as $entityColumn")
->join($entityTable, " INNER JOIN $entityTable ON $entityTable.contact_id = $contact.id ")
->join('gc', " INNER JOIN civicrm_group_contact_cache gc ON $contact.id = gc.contact_id ")
->join('mg', " INNER JOIN civicrm_mailing_group mg ON gc.group_id = mg.entity_id AND mg.search_id IS NULL ")
->join('temp', " LEFT JOIN $excludeTempTablename temp ON $contact.id = temp.contact_id ")
->where('gc.group_id IN (#groups)')
->merge($criteria)
->replaceInto($includedTempTablename, array('contact_id', $entityColumn))
->param('#groups', $includeSmartGroupIDs)
->param('#mailingID', $mailingID)
->execute();
}
```https://lab.civicrm.org/dev/core/-/issues/3631Unexpected behavior from api.MailingEventSubscribe.create2024-02-21T05:03:31ZginkgofjgUnexpected behavior from api.MailingEventSubscribe.createIn creating a custom e-newsletter sign-up form for a client, I encountered the following unexpected behaviors. I'm happy to submit one or more corrective pull requests but would first like to:
* confirm that this is not an unexpected us...In creating a custom e-newsletter sign-up form for a client, I encountered the following unexpected behaviors. I'm happy to submit one or more corrective pull requests but would first like to:
* confirm that this is not an unexpected use of this API,
* confirm that these behaviors are unexpected,
* get two cents from others on the approach, and
* get acceptance criteria for the PR(s) -- i.e., what tests do I need to write/pass?
The unexpecteds:
1. This is cosmetic, but the API metadata are wrong. The API Explorer describes the params as though they were for unsubscription.
2. Suppose contact X is already a member of a mailing group Y. If their IDs are passed to this API, the contact's status in the group is "downgraded" to "Pending" and the user must reconfirm membership by clicking the link emailed to them.
3. The contact's DO NOT MAIL and NO BULK EMAILS Communication Preferences are not updated. These changes should not be effected by the initial API call, but after the user clicks the confirmation link emailed to her.
The approaches:
1. This should be a trivial update to the API's `_spec()` function.
2. The API delegates to `CRM_Mailing_Event_BAO_Subscribe::subscribe()`. I would fix the problem in the delegate by checking for group membership and bailing out early if the record already exists with the appropriate status.
3. Confirmation is handled by `CRM_Mailing_Page_Confirm` which delegates to `CRM_Mailing_Event_BAO_Confirm::confirm()`. I would unset the communication preferences in question in the delegate.ginkgofjgginkgofjghttps://lab.civicrm.org/dev/core/-/issues/3620"View in Browser" does not feed mailing statistics2024-02-19T05:03:28Ztotten"View in Browser" does not feed mailing statisticsI just want to log the existence of the issue generally... suppose you send a newsletter and it has a "View in Browser" link.
Suppose the recipient uses that link. The view the mailing and then drill-down, clicking through some more lin...I just want to log the existence of the issue generally... suppose you send a newsletter and it has a "View in Browser" link.
Suppose the recipient uses that link. The view the mailing and then drill-down, clicking through some more links.
What happens to the stats?
* The web-based view does not count as an "Open". The MUA may or may not report back the "Open" on the original message.
* You will not have any "Click Through" events logged.
This is because the "View in browser" variant of the newsletter is not instrumented with tracking codes (while the MUA variant of the newsletter does have tracking codes).https://lab.civicrm.org/dev/core/-/issues/3616Should tracking pixel be placed at start of email for when Gmail clips messages?2024-02-18T05:03:26ZlarsssandergreenShould tracking pixel be placed at start of email for when Gmail clips messages?With Mosaico templates, it isn't that difficult to reach Gmail's limit of 96-102kB with an email newsletter, after which your email message is clipped and not displayed in full. This is unfortunate in itself, but it also breaks open trac...With Mosaico templates, it isn't that difficult to reach Gmail's limit of 96-102kB with an email newsletter, after which your email message is clipped and not displayed in full. This is unfortunate in itself, but it also breaks open tracking as the tracking pixel is only embedded at the very end of the message and thus isn't fetched unless the reader clicks "View entire message".
Would it be worthwhile to consider putting the tracking pixel at the start of the message to avoid this issue? Obviously, less than ideal for those who don't display images, but I'm not sure how these two might balance out or if there is a clever way to have the best of both worlds.
I'd be happy to provide a patch, but not sure what the best way forward is here.
(Was a former Flexmailer issue, reposted here as I think this is still important.)https://lab.civicrm.org/dev/core/-/issues/3615<link> URLs are tracked and shouldn't be2022-06-13T09:15:05Zlarsssandergreen<link> URLs are tracked and shouldn't beIf you use <link href=URL> in a mailing template, for example to load a webfont, that URL is converted to a tracked URL. Presumably it shouldn't be as this makes a mess of the clickthrough statistics (every one of these links will be cou...If you use <link href=URL> in a mailing template, for example to load a webfont, that URL is converted to a tracked URL. Presumably it shouldn't be as this makes a mess of the clickthrough statistics (every one of these links will be counted as clicked as long as the user's mail client fetches the resource). In other words, a link being fetched in a not a click and shouldn't be counted as one.
Do we need to track any URLs that aren't inside an <a> tag? It might be easier to limit to <a... rather than trying to exclude <link.... The only valid way to write an <a> is with the tag right after the <, so I think we could just look for <a instead of <.
I can provide a patch if this approach is supported.https://lab.civicrm.org/dev/core/-/issues/3607Feature: Provide an administrative option to make unsubscribe affect all matc...2024-02-17T05:03:32ZlolcodeFeature: Provide an administrative option to make unsubscribe affect all matching email addressesOur organization would like the unsubscribe action to unsubscribe all contacts with the same email from the group. In cases of duplicate contacts they unsubscribe once but one of the duplicates will still get an email and then we get CAS...Our organization would like the unsubscribe action to unsubscribe all contacts with the same email from the group. In cases of duplicate contacts they unsubscribe once but one of the duplicates will still get an email and then we get CASL complaints. Due to CASL we are trying to be cautious by default here.
We suggest an administrative setting for Civimail: "Unsubscribe contact (radio) or unsubscribe email address (radio)"
There was some support for this option expressed in a discussion here: https://chat.civicrm.org/civicrm/pl/sxd4jpwxpbynx8y15994ua7jea
A good point is that this needs to be guarded by token access to stop attackers being able to unsubscribe prominent emails from organisational lists (for example).
A further suggestion was made to make this a user selection option on the unsubscribe page. I feel that this would be a good but separate feature request.https://lab.civicrm.org/dev/core/-/issues/3606send test email may create duplicate contacts2022-06-11T14:55:49Zlcdwebsend test email may create duplicate contactsto reproduce:
1. construct a new email
2. in the "send test email to:" field list two email addresses that match existing contacts. separate them with a comma and space
3. trigger the test email
the first email will match the existi...to reproduce:
1. construct a new email
2. in the "send test email to:" field list two email addresses that match existing contacts. separate them with a comma and space
3. trigger the test email
the first email will match the existing contact. the second email will result in a new duplicate contact getting created. the issue is if you have a space after the comma separating the email addresses. the values are not trimmed when parsed, causing a failed match and the creation of a duplicate.5.6lcdweblcdwebhttps://lab.civicrm.org/dev/core/-/issues/3605"Tracking Click-Throughs" option in mailings generates 404 links when using m...2024-02-17T05:03:31Zjensschuppe"Tracking Click-Throughs" option in mailings generates 404 links when using multi-language with path prefixThis occurred in a Drupal environment with multiple languages and the following configuration:
- Languages in Drupal:
- English (activated)
- German (activated, default)
- Languages in CiviCRM:
- Default language: English
- avai...This occurred in a Drupal environment with multiple languages and the following configuration:
- Languages in Drupal:
- English (activated)
- German (activated, default)
- Languages in CiviCRM:
- Default language: English
- available: German, English
- Inherit CMS language: yes
Conditions under which the erroneous behavior can be reproduced:
- Mailing language: English
- UI language: German
- language prefix in the URL: none or "de"
or:
- Mailing language: German
- UI language: English
- language prefix in the URL: none or "en"
ergo: Using a UI language different from the mailing language and having path prefixes for language detection.
The result is links to the tracking script being prefixed with the language code of the UI language, like so:
https://example.org/de/sites/all/modules/civicrm/extern/url.php?u=123&qid=12345
which is producing a 404 as obviously the path to the script file is not valid due to the language prefix.
Apparently, this code in CRM/Mailing/BAO/TrackableURL.php:93 is where the URL is constructed, but with `userFrameworkResourceURL` being prefixed with the language code:
```
$redirect = $config->userFrameworkResourceURL . "extern/url.php?u=$id";
```https://lab.civicrm.org/dev/core/-/issues/3598Proposal: Scheduled Reminders should not (by default) use Smarty on new installs2023-01-27T15:27:33ZJonGoldProposal: Scheduled Reminders should not (by default) use Smarty on new installsWhen token rendering is enabled, and the `TokenProcessor` context says to use Smarty, the TokenRow rendering barfs on any curly brace that isn't a token or Smarty. Which includes, for instance, [inline CSS](https://civicrm.stackexchange...When token rendering is enabled, and the `TokenProcessor` context says to use Smarty, the TokenRow rendering barfs on any curly brace that isn't a token or Smarty. Which includes, for instance, [inline CSS](https://civicrm.stackexchange.com/q/17643/12).
At the time that SE post was made, Smarty stripped out the CSS. It no longer does, and instead generates large numbers of errors. Rather than fight that battle, I'm curious whether Smarty on scheduled reminders is necessary and serves a purpose in a post-5.43 TokenProcessor world.
If we decided that Scheduled Reminder TokenProcessors had [Smarty set to false](https://github.com/civicrm/civicrm-core/blob/ce0d67d6a686fcfae1d7046e615107f25cec501a/CRM/Core/BAO/ActionSchedule.php#L647) this problem would go away.
Does anyone use Smarty here? Should they?JonGoldJonGoldhttps://lab.civicrm.org/dev/core/-/issues/3595Mailing priority2024-02-16T05:03:23ZmfbMailing priorityOrganizations that send mailings to large lists, alongside more important messages sent to smaller lists, may need a mailing priority feature.
For example, a newsletter might go out to subscribers announcing a new campaign, while simult...Organizations that send mailings to large lists, alongside more important messages sent to smaller lists, may need a mailing priority feature.
For example, a newsletter might go out to subscribers announcing a new campaign, while simultaneously a press release is sent to press contacts. The latter would have higher priority.
CiviMail would send messages for mailings with a high priority before those with a lower priority.https://lab.civicrm.org/dev/core/-/issues/3594Core should provide bounce handling (and queueing) for transactional email2024-02-15T05:03:25ZmfbCore should provide bounce handling (and queueing) for transactional emailCurrently CiviCRM core does not provide bounce handling for transactional email. This can result in automated processes sending out a large volume of scheduled reminders, etc. (as well as user-triggered forms sending out subscription co...Currently CiviCRM core does not provide bounce handling for transactional email. This can result in automated processes sending out a large volume of scheduled reminders, etc. (as well as user-triggered forms sending out subscription confirmations, etc.) with only manual bounce handling by humans.
An extension exists to provide bounce handling and other features for transactional emails: https://civicrm.org/extensions/transactional-emails
Arguably, however, this feature should be provided by core itself, so all civi instances have bounce handling for any email message they send.
Assuming we continue the architecture where it is the CiviMail component that provides bounce handling, a solution could be that enabling CiviMail overrides how transactional email is sent, adding the extra bounce processing and other features.
Ideally, transactional email could also be queued by CiviMail, so failures are not fatal and delivery attempts can be retried. Currently, if you are over-quota or your mail delivery provider temporarily suspended your account, then you have real problems when doing some critical thing that sends a transactional email message.https://lab.civicrm.org/dev/core/-/issues/3579Add pause/resume functionality to civicrm bulk mailing.2022-06-11T14:54:43ZjitendraAdd pause/resume functionality to civicrm bulk mailing.Scenario
Site admin is sending out a big blast but want to send out a press release without waiting for first job to end.Scenario
Site admin is sending out a big blast but want to send out a press release without waiting for first job to end.5.4.0https://lab.civicrm.org/dev/core/-/issues/3578Send order when A/B testing2024-02-15T05:03:25ZufundoSend order when A/B testingAs far as I can see current behaviour when processing an A/B test mailing is that CiviMail sends out to all A recipients then all B recipients.
If you have rate limiting on outbound email that is significant relative to your list size, ...As far as I can see current behaviour when processing an A/B test mailing is that CiviMail sends out to all A recipients then all B recipients.
If you have rate limiting on outbound email that is significant relative to your list size, this can result in a significant difference in (average) send time between the A group and the B group, which may then be confounding your A/B test results.
I wonder how hard it would be to implement some kind of interpolation (?) for the sends for an A/B mailing -- something like firing off equal numbers of As and Bs in each mailing batch as they are processed?https://lab.civicrm.org/dev/core/-/issues/3574Password for mail accounts should not be stored in plain text2024-02-14T05:03:29ZscardiniusPassword for mail accounts should not be stored in plain textMySQL field civicrm_mail_settings.password contains password in plain text. Should it? I think not
This issue could by linked with https://lab.civicrm.org/dev/core/issues/236MySQL field civicrm_mail_settings.password contains password in plain text. Should it? I think not
This issue could by linked with https://lab.civicrm.org/dev/core/issues/236https://lab.civicrm.org/dev/core/-/issues/3573Unsubscribe fails but appears to have worked for resending to previous mailin...2024-02-14T05:03:29ZRichUnsubscribe fails but appears to have worked for resending to previous mailing to search resultsRecreate with 5.2.2 (on Dupal 7)
1. create mailing group A
2. create other group B
4. add contact into group: B
5. do mailing to contacts in group B, using group A as unsubscribe. (i.e. Find » In Group B » Action: create CiviMail mailin...Recreate with 5.2.2 (on Dupal 7)
1. create mailing group A
2. create other group B
4. add contact into group: B
5. do mailing to contacts in group B, using group A as unsubscribe. (i.e. Find » In Group B » Action: create CiviMail mailing)
7. Create and send mailing: sent to recipients of mlg1
8. click unsubscribe.
## Expect
to see confirm form saying "you will be unsubscribed from A" because A was the base group used for the original mailing. (When sending to previous mailing recipients there is no option to select an unsubscribe group)
## Actual
User sees "yes you've been unsubscribed from the mailing group/list" [code ref](https://lab.civicrm.org/dev/core/blob/master/CRM/Mailing/Form/Unsubscribe.php#L71) but nothing has actually happened!