Mailing issueshttps://lab.civicrm.org/dev/mail/-/issues2022-06-11T14:55:48Zhttps://lab.civicrm.org/dev/mail/-/issues/17"Tracking Click-Throughs" option in mailings generates 404 links when using m...2022-06-11T14:55:48Zjensschuppe"Tracking Click-Throughs" option in mailings generates 404 links when using multi-language with path prefixThis occurred in a Drupal environment with multiple languages and the following configuration:
- Languages in Drupal:
- English (activated)
- German (activated, default)
- Languages in CiviCRM:
- Default language: English
- avai...This occurred in a Drupal environment with multiple languages and the following configuration:
- Languages in Drupal:
- English (activated)
- German (activated, default)
- Languages in CiviCRM:
- Default language: English
- available: German, English
- Inherit CMS language: yes
Conditions under which the erroneous behavior can be reproduced:
- Mailing language: English
- UI language: German
- language prefix in the URL: none or "de"
or:
- Mailing language: German
- UI language: English
- language prefix in the URL: none or "en"
ergo: Using a UI language different from the mailing language and having path prefixes for language detection.
The result is links to the tracking script being prefixed with the language code of the UI language, like so:
https://example.org/de/sites/all/modules/civicrm/extern/url.php?u=123&qid=12345
which is producing a 404 as obviously the path to the script file is not valid due to the language prefix.
Apparently, this code in CRM/Mailing/BAO/TrackableURL.php:93 is where the URL is constructed, but with `userFrameworkResourceURL` being prefixed with the language code:
```
$redirect = $config->userFrameworkResourceURL . "extern/url.php?u=$id";
```https://lab.civicrm.org/dev/mail/-/issues/64"View in Browser" does not feed mailing statistics2022-06-11T14:57:07Ztotten"View in Browser" does not feed mailing statisticsI just want to log the existence of the issue generally... suppose you send a newsletter and it has a "View in Browser" link.
Suppose the recipient uses that link. The view the mailing and then drill-down, clicking through some more lin...I just want to log the existence of the issue generally... suppose you send a newsletter and it has a "View in Browser" link.
Suppose the recipient uses that link. The view the mailing and then drill-down, clicking through some more links.
What happens to the stats?
* The web-based view does not count as an "Open". The MUA may or may not report back the "Open" on the original message.
* You will not have any "Click Through" events logged.
This is because the "View in browser" variant of the newsletter is not instrumented with tracking codes (while the MUA variant of the newsletter does have tracking codes).https://lab.civicrm.org/dev/mail/-/issues/77(MOVED) [Feature] Please give us the option to disable subtotals for Soft Cre...2022-06-11T14:55:07Zswebervna(MOVED) [Feature] Please give us the option to disable subtotals for Soft Credit Contribution ReportsIssue moved to: https://lab.civicrm.org/dev/report/-/issues/51Issue moved to: https://lab.civicrm.org/dev/report/-/issues/51https://lab.civicrm.org/dev/mail/-/issues/95<link> URLs are tracked and shouldn't be2022-06-11T14:56:58Zlarsssandergreen<link> URLs are tracked and shouldn't beIf you use <link href=URL> in a mailing template, for example to load a webfont, that URL is converted to a tracked URL. Presumably it shouldn't be as this makes a mess of the clickthrough statistics (every one of these links will be cou...If you use <link href=URL> in a mailing template, for example to load a webfont, that URL is converted to a tracked URL. Presumably it shouldn't be as this makes a mess of the clickthrough statistics (every one of these links will be counted as clicked as long as the user's mail client fetches the resource). In other words, a link being fetched in a not a click and shouldn't be counted as one.
Do we need to track any URLs that aren't inside an <a> tag? It might be easier to limit to <a... rather than trying to exclude <link.... The only valid way to write an <a> is with the tag right after the <, so I think we could just look for <a instead of <.
I can provide a patch if this approach is supported.https://lab.civicrm.org/dev/mail/-/issues/50[Feature] More (privacy) options for click tracking2022-06-11T14:57:58Zmarcelklehr[Feature] More (privacy) options for click trackingHello,
I would like to have more options regarding click tracking in mailings.
1. Currently, every user can decide for themselves if they want to use it. I would like to be able override this with a setting, so i can e.g. disable click...Hello,
I would like to have more options regarding click tracking in mailings.
1. Currently, every user can decide for themselves if they want to use it. I would like to be able override this with a setting, so i can e.g. disable click tracking entirely.
2. Click-throughs are currently tracked by mail-adress, which is not conformant to the privacy laws in Germany, or at least requires additional consent. It would be nice to be able to limit this to just the numbers per link, without personal data. I would like to have this as an admin option, too.
Thanks for your consideration and this awesome software!https://lab.civicrm.org/dev/mail/-/issues/1Add pause/resume functionality to civicrm bulk mailing.2022-06-11T14:54:45ZjitendraAdd pause/resume functionality to civicrm bulk mailing.Scenario
Site admin is sending out a big blast but want to send out a press release without waiting for first job to end.Scenario
Site admin is sending out a big blast but want to send out a press release without waiting for first job to end.5.4.0https://lab.civicrm.org/dev/mail/-/issues/13All unsubscribe group members are excluded from a mailing with a manually spe...2022-06-11T14:54:13ZlolcodeAll unsubscribe group members are excluded from a mailing with a manually specified unsubscribe group (group_type=Base) if the unsubscribe group is a Smart groupAll unsubscribe group members are excluded from a mailing with a manually specified unsubscribe group (group_type=Base) if the unsubscribe group is a Smart group.
**To reproduce:** Create a smart group with some members in it. Some mem...All unsubscribe group members are excluded from a mailing with a manually specified unsubscribe group (group_type=Base) if the unsubscribe group is a Smart group.
**To reproduce:** Create a smart group with some members in it. Some members may also be added and removed manually as usual. Search for some contacts including some who are in that smart group and choose the action to send/schedule a bulk email. When creating the mail select the Smart group as your unsubscribe group.
**The expected result:** The mailing is not sent to any contacts in the search who are in the smart un-subscribe group with the status "Removed".
**The result:** The mailing is not sent to any contacts in the search who are in the smart un-subscribe group at all.5.2.0https://lab.civicrm.org/dev/mail/-/issues/23Bounce processing doesn't catch pattern "user doesn't exist"2022-06-11T14:50:34ZDetlev SieberBounce processing doesn't catch pattern "user doesn't exist"CiviCRM bounce processing analyzes the bounce text patterns, and decides what kind of bounce type it is. This is a fragile process, but there seems to be no better solution, since there is no general standardization of bounce patterns.
...CiviCRM bounce processing analyzes the bounce text patterns, and decides what kind of bounce type it is. This is a fragile process, but there seems to be no better solution, since there is no general standardization of bounce patterns.
Some email providers throw the bounce pattern "user doesn't exist", which is not found, and therefore results in bounce_type "syntax". However, this bounce pattern should result in immediately switching the email address to "inactive".
Solution is:
> insert into civicrm_mailing_bounce_pattern (bounce_type_id, pattern) values (6, 'user doesn\'t exist');
5.9Monish DebMonish Debhttps://lab.civicrm.org/dev/mail/-/issues/37Bounce processing fails for 4-byte unicode characters2022-06-11T14:55:28ZtschuettlerBounce processing fails for 4-byte unicode charactersWhen the bounced message contains any 4-byte unicode character, it cannot be saved to the database due to the currently missing support for utf8mb4 in CiviCRM.
Error Log:
```
[info] $backTrace = #0 /opt/buildkit/build/dmaster/sites/all...When the bounced message contains any 4-byte unicode character, it cannot be saved to the database due to the currently missing support for utf8mb4 in CiviCRM.
Error Log:
```
[info] $backTrace = #0 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/CRM/Core/Error.php(948): CRM_Core_Error::backtrace("backTrace", TRUE)
#1 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/packages/PEAR.php(921): CRM_Core_Error::exceptionHandler(Object(DB_Error))
#2 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/packages/DB.php(985): PEAR_Error->__construct("DB Error: unknown error", -1, 16, (Array:2), "INSERT INTO civicrm_mailing_event_bounce (event_queue_id , bounce_type_id , b...")
#3 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/packages/PEAR.php(575): DB_Error->__construct(-1, 16, (Array:2), "INSERT INTO civicrm_mailing_event_bounce (event_queue_id , bounce_type_id , b...")
#4 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/packages/PEAR.php(223): PEAR->_raiseError(Object(DB_mysqli), NULL, -1, 16, (Array:2), "INSERT INTO civicrm_mailing_event_bounce (event_queue_id , bounce_type_id , b...", "DB_Error", TRUE)
#5 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/packages/DB/common.php(1907): PEAR->__call("raiseError", (Array:7))
#6 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/packages/DB/mysqli.php(933): DB_common->raiseError(-1, NULL, NULL, "INSERT INTO civicrm_mailing_event_bounce (event_queue_id , bounce_type_id , b...", "1366 ** Incorrect string value: '\xF0\x9F\x8C\xB4' for column 'bounce_reason'...")
#7 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/packages/DB/mysqli.php(403): DB_mysqli->mysqliRaiseError()
#8 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/packages/DB/common.php(1216): DB_mysqli->simpleQuery("INSERT INTO civicrm_mailing_event_bounce (event_queue_id , bounce_type_id , b...")
#9 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/packages/DB/DataObject.php(2415): DB_common->query("INSERT INTO civicrm_mailing_event_bounce (event_queue_id , bounce_type_id , b...")
#10 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/packages/DB/DataObject.php(1040): DB_DataObject->_query("INSERT INTO civicrm_mailing_event_bounce (event_queue_id , bounce_type_id , b...")
#11 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/CRM/Core/DAO.php(571): DB_DataObject->insert()
#12 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/CRM/Mailing/Event/BAO/Bounce.php(87): CRM_Core_DAO->save()
#13 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/api/v3/Mailing.php(349): CRM_Mailing_Event_BAO_Bounce::create((Array:7))
#14 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/Civi/API/Provider/MagicFunctionProvider.php(100): civicrm_api3_mailing_event_bounce((Array:7))
#15 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/Civi/API/Kernel.php(169): Civi\API\Provider\MagicFunctionProvider->invoke((Array:9))
#16 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/Civi/API/Kernel.php(100): Civi\API\Kernel->runRequest((Array:9))
#17 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/api/api.php(23): Civi\API\Kernel->runSafe("Mailing", "event_bounce", (Array:6), NULL)
#18 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/CRM/Utils/Mail/EmailProcessor.php(341): civicrm_api("Mailing", "event_bounce", (Array:6))
#19 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/CRM/Utils/Mail/EmailProcessor.php(59): CRM_Utils_Mail_EmailProcessor::_process(TRUE, Object(CRM_Core_DAO_MailSettings), 0)
#20 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/api/v3/Job.php(403): CRM_Utils_Mail_EmailProcessor::processBounces(0)
#21 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/Civi/API/Provider/MagicFunctionProvider.php(100): civicrm_api3_job_fetch_bounces((Array:3))
#22 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/Civi/API/Kernel.php(169): Civi\API\Provider\MagicFunctionProvider->invoke((Array:9))
#23 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/Civi/API/Kernel.php(100): Civi\API\Kernel->runRequest((Array:9))
#24 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/api/api.php(23): Civi\API\Kernel->runSafe("job", "fetch_bounces", (Array:2), NULL)
#25 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/Civi/Test/Api3TestTrait.php(262): civicrm_api("job", "fetch_bounces", (Array:2))
#26 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/Civi/Test/Api3TestTrait.php(151): CiviUnitTestCase->civicrm_api("job", "fetch_bounces", (Array:2))
#27 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/tests/phpunit/CRM/Utils/Mail/EmailProcessorTest.php(78): CiviUnitTestCase->callAPISuccess("job", "fetch_bounces", (Array:2))
#28 [internal function](): CRM_Utils_Mail_EmailProcessorTest->testBounceProcessingUTF8mb4()
#29 phar:///opt/buildkit/extern/phpunit5/phpunit5.phar/phpunit/Framework/TestCase.php(1062): ReflectionMethod->invokeArgs(Object(CRM_Utils_Mail_EmailProcessorTest), (Array:0))
#30 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/tests/phpunit/CiviTest/CiviUnitTestCase.php(196): PHPUnit_Framework_TestCase->runTest()
#31 phar:///opt/buildkit/extern/phpunit5/phpunit5.phar/phpunit/Framework/TestCase.php(913): CiviUnitTestCase->runTest()
#32 phar:///opt/buildkit/extern/phpunit5/phpunit5.phar/phpunit/Framework/TestResult.php(686): PHPUnit_Framework_TestCase->runBare()
#33 phar:///opt/buildkit/extern/phpunit5/phpunit5.phar/phpunit/Framework/TestCase.php(868): PHPUnit_Framework_TestResult->run(Object(CRM_Utils_Mail_EmailProcessorTest))
#34 phar:///opt/buildkit/extern/phpunit5/phpunit5.phar/phpunit/Framework/TestSuite.php(733): PHPUnit_Framework_TestCase->run(Object(PHPUnit_Framework_TestResult))
#35 phar:///opt/buildkit/extern/phpunit5/phpunit5.phar/phpunit/TextUI/TestRunner.php(517): PHPUnit_Framework_TestSuite->run(Object(PHPUnit_Framework_TestResult))
#36 phar:///opt/buildkit/extern/phpunit5/phpunit5.phar/phpunit/TextUI/Command.php(186): PHPUnit_TextUI_TestRunner->doRun(Object(PHPUnit_Framework_TestSuite), (Array:54), TRUE)
#37 phar:///opt/buildkit/extern/phpunit5/phpunit5.phar/phpunit/TextUI/Command.php(116): PHPUnit_TextUI_Command->run((Array:11), TRUE)
#38 /opt/buildkit/extern/phpunit5/phpunit5.phar(598): PHPUnit_TextUI_Command::main()
#39 {main}
```5.11https://lab.civicrm.org/dev/mail/-/issues/78Can't save Message Template - authorization failed2022-06-11T14:55:45ZkenCan't save Message Template - authorization failedThis problem has been [reported on StackExchange](https://civicrm.stackexchange.com/questions/36233/problem-with-permissions-to-edit-message-templates) with version 5.27.2 and I have experienced it with 5.28.3.
Unless you have permissio...This problem has been [reported on StackExchange](https://civicrm.stackexchange.com/questions/36233/problem-with-permissions-to-edit-message-templates) with version 5.27.2 and I have experienced it with 5.28.3.
Unless you have permission "Administer CiviCRM" you can't edit a Message Template. This fails with an "Authorization failure" message.
I don't understand the code well enough to suggest a fix, but here's a hack to get around it.
The saving of the Message Template is done by this code on line 311 of CRM/Admin/Form/MessageTemplate.php
```
$messageTemplate = MessageTemplate::save()->setDefaults($params)->setRecords([['id' => $this->_id]])->execute()->first();
```
At the point when it comes to checking permissions, there are permissions on 'message_template' for 'get', 'create' and 'update' but not for 'save'. So the default permission is used (Administer CiviCRM).
The following patch to CRM/Core/Permission.php works around the problem ...
```
@@ -1483,6 +1483,7 @@ class CRM_Core_Permission {
$permissions['message_template'] = [
'get' => ['access CiviCRM'],
+ 'save' => [['edit message templates', 'edit user-driven message templates', 'edit system workflow message templates']],
'create' => [['edit message templates', 'edit user-driven message templates', 'edit system workflow message templates']],
'update' => [['edit message templates', 'edit user-driven message templates', 'edit system workflow message templates']],
];
```https://lab.civicrm.org/dev/mail/-/issues/69Civi not automatically imploding a dropdown Array for use as a token in a mes...2022-06-11T14:56:56ZswebervnaCivi not automatically imploding a dropdown Array for use as a token in a message templateHello!
I have a custom field, which is a multi-select dropdown, and I've inserted a token for that field to be used in a Message Template. Unfortunately, anytime I do the mailing using that Message Template, instead of the token inserti...Hello!
I have a custom field, which is a multi-select dropdown, and I've inserted a token for that field to be used in a Message Template. Unfortunately, anytime I do the mailing using that Message Template, instead of the token inserting what you'd expect (Option1,Option2, etc as selected in the dropdown), it just says "Array". I've found that it's just referencing the underlying infrastructure, in this case it's the Array entity *(Drupal 7)* for the dropdown value, but Civi won't implode it and list them like it should for that token:
**Mailing Result:**
![image](/uploads/84a86b7e9c68dc2a91b0a3170f430078/image.png)
**Message Template:**
![image](/uploads/63fbcdd549bd43a924c459fd31b9c6e6/image.png)
**Custom Field:**
![image](/uploads/1739d14f524a4eb599e4ad0e5846f952/image.png)https://lab.civicrm.org/dev/mail/-/issues/110CiviCRM bounce processing, processed emails are never deleted from the mailbo...2022-06-11T14:51:06Zjustinfreeman (Agileware)CiviCRM bounce processing, processed emails are never deleted from the mailbox which uses up disk space and provides very little valueCiviCRM bounce processing, processed emails are never deleted from the mailbox which uses up disk space and provides very little value.
Possible solutions:
1. Once a bounce email has been processed it should be deleted from the mailbox,...CiviCRM bounce processing, processed emails are never deleted from the mailbox which uses up disk space and provides very little value.
Possible solutions:
1. Once a bounce email has been processed it should be deleted from the mailbox, or
1. A new Scheduled Job process should be implemented which deletes all processed email which is older than 30 days
Relates to https://lab.civicrm.org/dev/mail/-/issues/108
Agileware Ref: CIVICRM-1977https://lab.civicrm.org/dev/mail/-/issues/30CiviCRM, Find Mailings by Mailing Status, select Draft/Unscheduled will show ...2022-06-11T14:56:02Zjustinfreeman (Agileware)CiviCRM, Find Mailings by Mailing Status, select Draft/Unscheduled will show "Complete" and "Not scheduled" mailings in the Mailing resultsCiviCRM, Find Mailings by Mailing Status, select Draft/Unscheduled will show "Complete" and "Not scheduled" mailings in the Mailing results.
Using CiviCRM 5.7.2
![find-mailings](/uploads/279c313811bfffbfc06232a191ec770b/find-mailings.p...CiviCRM, Find Mailings by Mailing Status, select Draft/Unscheduled will show "Complete" and "Not scheduled" mailings in the Mailing results.
Using CiviCRM 5.7.2
![find-mailings](/uploads/279c313811bfffbfc06232a191ec770b/find-mailings.png)
Agileware Ref: CIVICRM-1120https://lab.civicrm.org/dev/mail/-/issues/31CiviCRM, Find Mailings. If there are no results, the same message "There are ...2022-06-11T14:57:50Zjustinfreeman (Agileware)CiviCRM, Find Mailings. If there are no results, the same message "There are no Unscheduled Mailings" is shownCiviCRM, Find Mailings. If there are no results, the same message "There are no Unscheduled Mailings" is shown regardless of the search criteria. If a different mailing status was used for search, it does not make sense to refer to "Unsc...CiviCRM, Find Mailings. If there are no results, the same message "There are no Unscheduled Mailings" is shown regardless of the search criteria. If a different mailing status was used for search, it does not make sense to refer to "Unscheduled Mailings".
This text should be changed to something else like: "There are no mailings matching this criteria".
Using CiviCRM 5.7.2
Agileware Ref: CIVICRM-1121
![Find_Mailings-2](/uploads/2abe0973a4395a1e1fb50d2d4f71e1eb/Find_Mailings-2.png)https://lab.civicrm.org/dev/mail/-/issues/76CiviMail flagged URI_WP_HACKED_2 by SpamAssassin2022-06-11T14:51:12ZbrendanspaarCiviMail flagged URI_WP_HACKED_2 by SpamAssassinOverview
----------------------------------------
CiviCRM emails were reported as going to the Spam folder in Gmail. I did some digging with mail-tester.com and found that SpamAssassin is dinging my mailings -1.629 points for the code UR...Overview
----------------------------------------
CiviCRM emails were reported as going to the Spam folder in Gmail. I did some digging with mail-tester.com and found that SpamAssassin is dinging my mailings -1.629 points for the code URI_WP_HACKED_2.
I did some more digging and found this is likely caused by the WP REST API that was integrated into Civi in 5.25. I'm currently running 5.27.0.
Things I have done that have not resolved the issue:
- I have installed the cv-rest-mailing plugin located here: https://develop.tadpole.cc/plugins/cv-rest-mailing
- I have also ran my site against MalCare which returned a Clean rating.
- I have disabled the plugin WP Mail SMTP.
- I have disabled tracing of opens and click throughs.
I have confirmed that this is occurring on 3 of my clients sites all running Civi 5.27.0 with Mosaico.
I have confirmed this on sites that are running gSuite as their email provider as well as sites that are running a local mail exchanger.
The following two Stack Exchange questions are open with no resolution. I confirmed with Heather O. that she is still facing difficulty.
- https://civicrm.stackexchange.com/questions/35064/civimail-tracked-links-in-wordpress-site-are-marked-down-by-mail-tester?r=SearchResults&s=3|14.7395
- https://civicrm.stackexchange.com/questions/36244/spamassassin-marking-mailings-as-compromised-wordpress-site?r=SearchResults&s=1|42.5634
Reproduction steps
----------------------------------------
1. Create a new group called SpamTesting
2. Create a new individual named Spam Tester
3. Add Spam Tester to the Group SpamTesting
4. Give the individual Spam Tester the email address corresponding to https://www.mail-tester.com/
5. Create a new mailing or reuse an existing mailing.
6. Add SpamTesting as a recipient group.
7. Send the mailing immediately
8. Click check your score on Mail-Tester.com
Current behaviour
----------------------------------------
Many CiviCRM mailings are going to spam after upgrading to 5.27.0. Please see attached image.
![SpamAssassin Score](/uploads/f5b1c7397916bc4c45ad2d10d89fcd2f/KrY6G_1_.png)
Expected behaviour
----------------------------------------
Emails should not be getting flagged as a hacked WordPress site.
Environment information
----------------------------------------
<!-- Some of the items below may not be relevant for every bug - if in doubt please include more information than you think is neccessary. -->
* __Mail Server__ : gSuite
* __Browser:__ Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.105 Safari/537.36
* __CiviCRM:__ _Master/5.27.1
* __PHP:__ 7.3.14
* __CMS:__ WordPress 5.4.2
* __Database:__ _MySQL 8.0.18
* __Web Server:__ _Apache 2.4.41
Comments
----------------------------------------https://lab.civicrm.org/dev/mail/-/issues/51CiviMail: Provide transparency about (non)recipient details2022-06-11T14:54:19ZtottenCiviMail: Provide transparency about (non)recipient detailsTwo stories which are not-uncommon:
## Story 1
Someone starts using CiviMail. They spend a bunch of time arranging for the list of contacts (e.g. migrating from a previous system) and then go to send a mailing. They start composing a m...Two stories which are not-uncommon:
## Story 1
Someone starts using CiviMail. They spend a bunch of time arranging for the list of contacts (e.g. migrating from a previous system) and then go to send a mailing. They start composing a message and realize that the number of recipients is way off. What's going on? They start searching Google, Stackexchange, etc trying to understand why. Eventually, they learn that there are a half-dozen rules about who gets included or excluded from a mailing. (*Does the contact have an email address? Are they deceased? Have they opted-out of the specific mailing list? Have they opted-out of email generally? Have they had bounces? Does an extension - like GDPR - add more constraints?*) But they haven't solved the issue yet: now they need to figure out which criteria are affecting them, to what extent, and how to adapt.
And of course people have this problem - the "New Mailing" UX only presents the affirmative list of active recipients. It doesn't give any hint or detail about the omitted recipients.
![Screen_Shot_2019-09-03_at_12.58.40_PM](/uploads/5a4333487cef3b3df60c6ddc5bd76f65/Screen_Shot_2019-09-03_at_12.58.40_PM.png)
## Story 2
A new user composes an email blast and discovers this incredible drop-down button with a list of tokens. They start throwing in tokens like there's no tomorrow. The blast goes out, and recipients complain: "Why are you sending me these weird messages with skipped words?"
And, again, of course people have this problem - the "New Mailing" UX lets you choose tokens without knowing if your recipient-data is up to the job.
## Discussion
These two stories are thematically similar: in both cases, we have a long list of potential recipients, and we have obscure/hidden criteria which determines whether the mailing goes out successfully to the target audience.
In both cases, there needs to be more transparency - adding UI-elements/UX-mechanisms to communicate what's going on. For example, the data-table in the pop-up might be updated with more columns/icons/color-coding.
I've logged the issue because I think, at some point, there should be discussion around design/resourcing. I don't want to prejudge that by saying the UX mechanisms to resolve these two issues must or must-not be the same - they might be totally different, or they might be one data-table. Either way, I think it's good to have a record of the issue.
## Related Links
* https://chat.civicrm.org/civicrm/pl/nn5rjzwiz3fc78pgojmhubx89a
* https://civicrm.org/blog/simonparkervitiligosocietyorguk/solution-all-recipients-not-showing-up-for-sending-mass-email
* https://civicrm.stackexchange.com/questions/5086/civimail-does-not-send-to-a-whole-group
* https://civicrm.stackexchange.com/questions/6346/after-import-to-group-individuals-in-that-group-not-showing-up-as-recipients-to
* https://civicrm.stackexchange.com/questions/26320/contacts-visible-in-mailing-list-group-not-on-hold-or-otherwise-dnd-but-skippe
* https://civicrm.stackexchange.com/questions/18199/in-mailing-all-recipients-are-not-showing/18201#18201
* https://civicrm.org/extensions/zombie-check
https://lab.civicrm.org/dev/mail/-/issues/46Contribution tokens with option values display values, not label2022-06-11T14:57:03ZJonGoldContribution tokens with option values display values, not labelTo replicate on dmaster:
* Edit any contribution to add a value for the custom field *How long have you been a donor?*.
* With **Find Contributions**, search for that contribution and create either an email or thank-you letter.
* Use a t...To replicate on dmaster:
* Edit any contribution to add a value for the custom field *How long have you been a donor?*.
* With **Find Contributions**, search for that contribution and create either an email or thank-you letter.
* Use a token to display the value of *How long have you been a donor?*.
* Print/send.
### Expected result
The token should render the label of the selected option, e.g. "Less than 1 year", "1-3 years".
### Observed result
The token renders the value, e.g., "1", "2".
Contact tokens behave in the expected way.
Note: This is not, AFAICT, a regression. While I can't find other user reports on this, I also don't see any indication in the code that this ever was accounted for.JonGoldJonGoldhttps://lab.civicrm.org/dev/mail/-/issues/100Copying a mailing results in thousands of entries in civicrm_mailing_group, r...2022-06-11T14:58:18ZJonGoldCopying a mailing results in thousands of entries in civicrm_mailing_group, rendering page unresponsiveI'll start by saying I haven't been able to replicate this - but I've seen it happen multiple times on Civi 5.39 to 5.41. I'm hoping someone else finds this and it rings a bell.
Sometimes, when a client goes to edit a mailing, the page...I'll start by saying I haven't been able to replicate this - but I've seen it happen multiple times on Civi 5.39 to 5.41. I'm hoping someone else finds this and it rings a bell.
Sometimes, when a client goes to edit a mailing, the page is either a) unresponsive, or b) doesn't allow saving. The "calculating recipients" SQL is running indefinitely. When investigating, there are tens of thousands of entries in `civicrm_mailing_group` for that mailing - the `entity_id`s repeat.
While I'm unable to replicate, this seems to happen when:
* Users copy an existing mailing rather than starting from a new mailing;
* In both cases, mailings were going out to 17-24 groups.https://lab.civicrm.org/dev/mail/-/issues/32Deduping test email is case sensitive2022-06-11T14:54:29ZMartinDeduping test email is case sensitiveTested on 4.6.38 and 5.6.1.
When composing a mailing, the email address used in the "send test email to" field at the bottom is deduped. This is case sensitive, so fails to match emails that are the same but have different upper/lower c...Tested on 4.6.38 and 5.6.1.
When composing a mailing, the email address used in the "send test email to" field at the bottom is deduped. This is case sensitive, so fails to match emails that are the same but have different upper/lower case combinations. In this case, it will create a new contact with this interesting property: the value in the "email" field shows with the email address flattened to all lowercase, but the display name for the contact shows with the upper/lower case mix used when it was entered to test the mailing.
The upshot of all this is that if I'm putting together a mailing and send 6 test to the same email address (with an uppercase letter), I get 6 new duplicate contacts because it fails to match it each time.
Request to make the dedupe test for this mailing field case insensitive, since in general civi seems to want email addresses to be lowercase when they're used.
I can potentially make this change if someone can point me in the right direction.
Thanks!5.11https://lab.civicrm.org/dev/mail/-/issues/19Deleting a sent mailing leaves activities with invalid links to a mailing rep...2022-06-11T14:51:01ZlolcodeDeleting a sent mailing leaves activities with invalid links to a mailing report that doesn't workAssuming: Civicrm uses the default setting "Enable CiviMail to create activities on delivery"
If a mailing is created, sent and then deleted then the recipients of that mailing will have activities that link to a mailing report using an...Assuming: Civicrm uses the default setting "Enable CiviMail to create activities on delivery"
If a mailing is created, sent and then deleted then the recipients of that mailing will have activities that link to a mailing report using an invalid id. If the link is followed the error will be:
> Sorry, due to an error, we are unable to fulfill your request at the moment. You may want to contact your administrator or service provider with more details about what action you were performing when this occurred.
> Expected one Mailing but found 0
If clicked from the popup activity view it will just be a network error.
The body of the activity will also be blank after the mailing is deleted.
I am not convinced that allowing sent mailings to be deleted at all is a good design.
Some kind of an archive function might be better.
In the short term however the link should probably be removed from these activities.