Mailing issueshttps://lab.civicrm.org/dev/mail/-/issues2022-06-11T14:55:56Zhttps://lab.civicrm.org/dev/mail/-/issues/26getRecipients() appears to use a WHERE clause as an ORDER BY clause2022-06-11T14:55:56ZkengetRecipients() appears to use a WHERE clause as an ORDER BY clauseSee https://github.com/civicrm/civicrm-core/commit/906298d3f63825adbee0e85963d89354edf8acf1#r30370554See https://github.com/civicrm/civicrm-core/commit/906298d3f63825adbee0e85963d89354edf8acf1#r303705545.8https://lab.civicrm.org/dev/mail/-/issues/37Bounce processing fails for 4-byte unicode characters2022-06-11T14:55:28ZtschuettlerBounce processing fails for 4-byte unicode charactersWhen the bounced message contains any 4-byte unicode character, it cannot be saved to the database due to the currently missing support for utf8mb4 in CiviCRM.
Error Log:
```
[info] $backTrace = #0 /opt/buildkit/build/dmaster/sites/all...When the bounced message contains any 4-byte unicode character, it cannot be saved to the database due to the currently missing support for utf8mb4 in CiviCRM.
Error Log:
```
[info] $backTrace = #0 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/CRM/Core/Error.php(948): CRM_Core_Error::backtrace("backTrace", TRUE)
#1 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/packages/PEAR.php(921): CRM_Core_Error::exceptionHandler(Object(DB_Error))
#2 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/packages/DB.php(985): PEAR_Error->__construct("DB Error: unknown error", -1, 16, (Array:2), "INSERT INTO civicrm_mailing_event_bounce (event_queue_id , bounce_type_id , b...")
#3 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/packages/PEAR.php(575): DB_Error->__construct(-1, 16, (Array:2), "INSERT INTO civicrm_mailing_event_bounce (event_queue_id , bounce_type_id , b...")
#4 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/packages/PEAR.php(223): PEAR->_raiseError(Object(DB_mysqli), NULL, -1, 16, (Array:2), "INSERT INTO civicrm_mailing_event_bounce (event_queue_id , bounce_type_id , b...", "DB_Error", TRUE)
#5 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/packages/DB/common.php(1907): PEAR->__call("raiseError", (Array:7))
#6 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/packages/DB/mysqli.php(933): DB_common->raiseError(-1, NULL, NULL, "INSERT INTO civicrm_mailing_event_bounce (event_queue_id , bounce_type_id , b...", "1366 ** Incorrect string value: '\xF0\x9F\x8C\xB4' for column 'bounce_reason'...")
#7 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/packages/DB/mysqli.php(403): DB_mysqli->mysqliRaiseError()
#8 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/packages/DB/common.php(1216): DB_mysqli->simpleQuery("INSERT INTO civicrm_mailing_event_bounce (event_queue_id , bounce_type_id , b...")
#9 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/packages/DB/DataObject.php(2415): DB_common->query("INSERT INTO civicrm_mailing_event_bounce (event_queue_id , bounce_type_id , b...")
#10 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/packages/DB/DataObject.php(1040): DB_DataObject->_query("INSERT INTO civicrm_mailing_event_bounce (event_queue_id , bounce_type_id , b...")
#11 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/CRM/Core/DAO.php(571): DB_DataObject->insert()
#12 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/CRM/Mailing/Event/BAO/Bounce.php(87): CRM_Core_DAO->save()
#13 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/api/v3/Mailing.php(349): CRM_Mailing_Event_BAO_Bounce::create((Array:7))
#14 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/Civi/API/Provider/MagicFunctionProvider.php(100): civicrm_api3_mailing_event_bounce((Array:7))
#15 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/Civi/API/Kernel.php(169): Civi\API\Provider\MagicFunctionProvider->invoke((Array:9))
#16 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/Civi/API/Kernel.php(100): Civi\API\Kernel->runRequest((Array:9))
#17 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/api/api.php(23): Civi\API\Kernel->runSafe("Mailing", "event_bounce", (Array:6), NULL)
#18 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/CRM/Utils/Mail/EmailProcessor.php(341): civicrm_api("Mailing", "event_bounce", (Array:6))
#19 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/CRM/Utils/Mail/EmailProcessor.php(59): CRM_Utils_Mail_EmailProcessor::_process(TRUE, Object(CRM_Core_DAO_MailSettings), 0)
#20 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/api/v3/Job.php(403): CRM_Utils_Mail_EmailProcessor::processBounces(0)
#21 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/Civi/API/Provider/MagicFunctionProvider.php(100): civicrm_api3_job_fetch_bounces((Array:3))
#22 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/Civi/API/Kernel.php(169): Civi\API\Provider\MagicFunctionProvider->invoke((Array:9))
#23 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/Civi/API/Kernel.php(100): Civi\API\Kernel->runRequest((Array:9))
#24 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/api/api.php(23): Civi\API\Kernel->runSafe("job", "fetch_bounces", (Array:2), NULL)
#25 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/Civi/Test/Api3TestTrait.php(262): civicrm_api("job", "fetch_bounces", (Array:2))
#26 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/Civi/Test/Api3TestTrait.php(151): CiviUnitTestCase->civicrm_api("job", "fetch_bounces", (Array:2))
#27 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/tests/phpunit/CRM/Utils/Mail/EmailProcessorTest.php(78): CiviUnitTestCase->callAPISuccess("job", "fetch_bounces", (Array:2))
#28 [internal function](): CRM_Utils_Mail_EmailProcessorTest->testBounceProcessingUTF8mb4()
#29 phar:///opt/buildkit/extern/phpunit5/phpunit5.phar/phpunit/Framework/TestCase.php(1062): ReflectionMethod->invokeArgs(Object(CRM_Utils_Mail_EmailProcessorTest), (Array:0))
#30 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/tests/phpunit/CiviTest/CiviUnitTestCase.php(196): PHPUnit_Framework_TestCase->runTest()
#31 phar:///opt/buildkit/extern/phpunit5/phpunit5.phar/phpunit/Framework/TestCase.php(913): CiviUnitTestCase->runTest()
#32 phar:///opt/buildkit/extern/phpunit5/phpunit5.phar/phpunit/Framework/TestResult.php(686): PHPUnit_Framework_TestCase->runBare()
#33 phar:///opt/buildkit/extern/phpunit5/phpunit5.phar/phpunit/Framework/TestCase.php(868): PHPUnit_Framework_TestResult->run(Object(CRM_Utils_Mail_EmailProcessorTest))
#34 phar:///opt/buildkit/extern/phpunit5/phpunit5.phar/phpunit/Framework/TestSuite.php(733): PHPUnit_Framework_TestCase->run(Object(PHPUnit_Framework_TestResult))
#35 phar:///opt/buildkit/extern/phpunit5/phpunit5.phar/phpunit/TextUI/TestRunner.php(517): PHPUnit_Framework_TestSuite->run(Object(PHPUnit_Framework_TestResult))
#36 phar:///opt/buildkit/extern/phpunit5/phpunit5.phar/phpunit/TextUI/Command.php(186): PHPUnit_TextUI_TestRunner->doRun(Object(PHPUnit_Framework_TestSuite), (Array:54), TRUE)
#37 phar:///opt/buildkit/extern/phpunit5/phpunit5.phar/phpunit/TextUI/Command.php(116): PHPUnit_TextUI_Command->run((Array:11), TRUE)
#38 /opt/buildkit/extern/phpunit5/phpunit5.phar(598): PHPUnit_TextUI_Command::main()
#39 {main}
```5.11https://lab.civicrm.org/dev/mail/-/issues/32Deduping test email is case sensitive2022-06-11T14:54:29ZMartinDeduping test email is case sensitiveTested on 4.6.38 and 5.6.1.
When composing a mailing, the email address used in the "send test email to" field at the bottom is deduped. This is case sensitive, so fails to match emails that are the same but have different upper/lower c...Tested on 4.6.38 and 5.6.1.
When composing a mailing, the email address used in the "send test email to" field at the bottom is deduped. This is case sensitive, so fails to match emails that are the same but have different upper/lower case combinations. In this case, it will create a new contact with this interesting property: the value in the "email" field shows with the email address flattened to all lowercase, but the display name for the contact shows with the upper/lower case mix used when it was entered to test the mailing.
The upshot of all this is that if I'm putting together a mailing and send 6 test to the same email address (with an uppercase letter), I get 6 new duplicate contacts because it fails to match it each time.
Request to make the dedupe test for this mailing field case insensitive, since in general civi seems to want email addresses to be lowercase when they're used.
I can potentially make this change if someone can point me in the right direction.
Thanks!5.11https://lab.civicrm.org/dev/mail/-/issues/110CiviCRM bounce processing, processed emails are never deleted from the mailbo...2022-06-11T14:51:06Zjustinfreeman (Agileware)CiviCRM bounce processing, processed emails are never deleted from the mailbox which uses up disk space and provides very little valueCiviCRM bounce processing, processed emails are never deleted from the mailbox which uses up disk space and provides very little value.
Possible solutions:
1. Once a bounce email has been processed it should be deleted from the mailbox,...CiviCRM bounce processing, processed emails are never deleted from the mailbox which uses up disk space and provides very little value.
Possible solutions:
1. Once a bounce email has been processed it should be deleted from the mailbox, or
1. A new Scheduled Job process should be implemented which deletes all processed email which is older than 30 days
Relates to https://lab.civicrm.org/dev/mail/-/issues/108
Agileware Ref: CIVICRM-1977https://lab.civicrm.org/dev/mail/-/issues/109Denial of service - CiviCRM Mailing Job, if there are 5 errors sequentially w...2022-06-11T14:54:54Zjustinfreeman (Agileware)Denial of service - CiviCRM Mailing Job, if there are 5 errors sequentially which are caused by an invalid email domain. The Mailing Job will abort and mailing will not sendCiviCRM Mailing Job, if there are 5 errors sequentially which are caused by an invalid email domain. The Mailing Job will abort and the mailing will not send.
An invalid email domain returns an SMTP error 451: _SMTP error 451 Unable to ...CiviCRM Mailing Job, if there are 5 errors sequentially which are caused by an invalid email domain. The Mailing Job will abort and the mailing will not send.
An invalid email domain returns an SMTP error 451: _SMTP error 451 Unable to complete command, DNS not available or timed out 451 Domain of sender address does not resolve_
When the CiviCRM Mailing Job executes again, the process repeats, the same invalid email addresses are used, the same errors are returned and the mailing again will not send.
This process will repeat until someone intervenes, cancelling the mailing, locating and removing the contacts which have the invalid email addresses.
The CiviCRM Mailing Job does not currently check the actual SMTP error code and just treats all SMTP errors as "SMTP Connection Errors". Counting up to 5 and then aborting the job when the threshold is reached.
I think it's fair to call this a "denial of service" because a bad actor could sign up a bunch of fake Contacts to a CiviCRM mailing list with either invalid email domains or valid email domains which are then rendered invalid - and thus cause that mailing list to cease processing.
**Possible solutions**
Change the CiviCRM Mailing Job to check the return SMTP error code and only increment the count for valid "SMTP Connection Errors".
Change the CiviCRM Mailing Job to continue with the mailing, regardless of the 5 errors encountered, which will also skip sending the emails to those Contacts with an invalid email domain. As shown in the patch below.
```
Index: CRM/Mailing/BAO/MailingJob.php
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/CRM/Mailing/BAO/MailingJob.php b/CRM/Mailing/BAO/MailingJob.php
--- a/CRM/Mailing/BAO/MailingJob.php (revision 20a0376709a4616e32689a821a7e95ca255ed85f)
+++ b/CRM/Mailing/BAO/MailingJob.php (date 1620794367592)
@@ -672,20 +672,9 @@
if ($smtpConnectionErrors <= 5) {
$mailer->disconnect();
$retryGroup = TRUE;
+ CRM_Core_Error::debug_log_message("More than 5 consecutive SMTP Socket Errors. Re-starting mailer.");
continue;
}
-
- // seems like we have too many of them in a row, we should
- // write stuff to disk and abort the cron job
- $this->writeToDB(
- $deliveredParams,
- $targetParams,
- $mailing,
- $job_date
- );
-
- CRM_Core_Error::debug_log_message("Too many SMTP Socket Errors. Exiting");
- CRM_Utils_System::civiExit();
}
// Register the bounce event.
Index: ext/flexmailer/src/Listener/DefaultSender.php
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/ext/flexmailer/src/Listener/DefaultSender.php b/ext/flexmailer/src/Listener/DefaultSender.php
--- a/ext/flexmailer/src/Listener/DefaultSender.php (revision 20a0376709a4616e32689a821a7e95ca255ed85f)
+++ b/ext/flexmailer/src/Listener/DefaultSender.php (date 1620794390628)
@@ -76,15 +76,9 @@
if ($smtpConnectionErrors <= 5) {
$mailer->disconnect();
$retryBatch = TRUE;
+ \CRM_Core_Error::debug_log_message("More than 5 consecutive SMTP Socket Errors. Re-starting mailer.");
continue;
}
-
- // seems like we have too many of them in a row, we should
- // write stuff to disk and abort the cron job
- $job->writeToDB($deliveredParams, $targetParams, $mailing, $job_date);
-
- \CRM_Core_Error::debug_log_message("Too many SMTP Socket Errors. Exiting");
- \CRM_Utils_System::civiExit();
}
else {
$this->recordBounce($job, $task, $result->getMessage());
```
Agileware Ref: CIVICRM-1728https://lab.civicrm.org/dev/mail/-/issues/108Denial of service - CiviCRM Fetch Bounces scheduled job will fail to process ...2022-06-11T14:54:57Zjustinfreeman (Agileware)Denial of service - CiviCRM Fetch Bounces scheduled job will fail to process any emails if a single email is sent to the bounce mailbox with an invalid returnPathCiviCRM bounce processing will fail to process any emails if a single email ("Denial of service email") is sent to the bounce mailbox with an invalid returnPath.
CiviCRM will repeatedly try and fail to process the "Denial of service ema...CiviCRM bounce processing will fail to process any emails if a single email ("Denial of service email") is sent to the bounce mailbox with an invalid returnPath.
CiviCRM will repeatedly try and fail to process the "Denial of service email" causing all other emails to remain unprocessed. CiviCRM mail reports are therefore incorrect and ultimately the CiviCRM Fetch Bounces scheduled job fails to perform the job it was designed to do.
Example invalid returnPaths as observed in production:
```
x2108-arfsfufwyfgkvni2x7ww5wvac3eifc2wx3vr1fcwbbysieqppis4ytnyip09xcohck9s1vrm77nbg4xh431tthdsmtmfvfpa@rspf.mxthunderstruck.net
```
```
SRS0=3Ypr33=QM=crm.nothappyjohn.org.au=bounce+b.25211.6039846.eb7e7e8ba282f9d0@airquotes.com.au
```
This problem has been highlighted by the introduction of "Scheduled Job Failures" feature which raises the visibility of this type of problem.
Bounce mailboxes have been observed with 10k to 50k unprocessed emails, depending on the time when the "Denial of service email" was received.
Agileware Ref: CIVICRM-1970https://lab.civicrm.org/dev/mail/-/issues/102Undefined index: error-codes in recaptcha_check_answer()2022-06-11T14:55:43Zchris_bluejacUndefined index: error-codes in recaptcha_check_answer()## Overview
When non-authenticated user subscribes to mailing list at /civicrm/mailing/subscribe, checks the "I am not a robot" reCAPTCHA, and clicks subscribe. The subscription request is processed properly, but an error is thrown:
Not...## Overview
When non-authenticated user subscribes to mailing list at /civicrm/mailing/subscribe, checks the "I am not a robot" reCAPTCHA, and clicks subscribe. The subscription request is processed properly, but an error is thrown:
Notice: Undefined index: error-codes in recaptcha_check_answer() (line 159 of /bitnami/drupal/modules/contrib/civicrm/ext/recaptcha/lib/recaptcha/recaptchalib.php).
## Reproduction steps
1. /civicrm/mailing/subscribe
2. complete email address, check relevant mailing list (in this case, "General Newsletter")
3. Check "I'm not a robot" in reCAPTCHA
4. Click "Subscribe"
5. Subscription is processed properly.
6. Error reported on confirmation page.
## Current behaviour
Error message presented.
Functionality of reCAPTCHA and subscription is okay.
~~~
Notice: Undefined index: error-codes in recaptcha_check_answer() (line 159 of /bitnami/drupal/modules/contrib/civicrm/ext/recaptcha/lib/recaptcha/recaptchalib.php).
~~~
## Expected behaviour
No error message
## Environment Information
- CiviCRM: 5.40.2
- PHP: 7.3.27
- CMS: Drupal 7.82
- Database: MySQL 5.7.33
- Web Server: Apache 2.4.46
- reCAPTCHA: Version 5.40.2
![Screenshot_2021-10-31_084608](/uploads/b53cc12d8c04b7ec56da353510254b8b/Screenshot_2021-10-31_084608.png)https://lab.civicrm.org/dev/mail/-/issues/101High Database Load INSERT IGNORE INTO `civicrm_tmp_e_gccache_2022-06-11T14:56:44ZMickCHigh Database Load INSERT IGNORE INTO `civicrm_tmp_e_gccache_I thought I'd bring this to attention of the Civi Mail team as it's causing high load.
This has happened on more than 1 occasion, running many of these queries for about 3 hours.
![Screen_Shot_2021-09-17_at_3.44.33_pm](/uploads/f86b6f89b...I thought I'd bring this to attention of the Civi Mail team as it's causing high load.
This has happened on more than 1 occasion, running many of these queries for about 3 hours.
![Screen_Shot_2021-09-17_at_3.44.33_pm](/uploads/f86b6f89b44caf7dbc7bcab05609a88d/Screen_Shot_2021-09-17_at_3.44.33_pm.png)
I get notified of it by an AWS Cloudwatch alarm on RDS.
The database was not running a mailing at the time.
Contacts approx 100,000 so quite large.
However database load is usually very low, even when running a mailing.
I haven't been able to identify what this process is - if anyone knows, then is there something that can be done to address the load?
The database joins in the query make is a large query - is there another way to do this? e.g. break the query up into temp tables, fewer joins, use indexes etc
Thankshttps://lab.civicrm.org/dev/mail/-/issues/100Copying a mailing results in thousands of entries in civicrm_mailing_group, r...2022-06-11T14:58:18ZJonGoldCopying a mailing results in thousands of entries in civicrm_mailing_group, rendering page unresponsiveI'll start by saying I haven't been able to replicate this - but I've seen it happen multiple times on Civi 5.39 to 5.41. I'm hoping someone else finds this and it rings a bell.
Sometimes, when a client goes to edit a mailing, the page...I'll start by saying I haven't been able to replicate this - but I've seen it happen multiple times on Civi 5.39 to 5.41. I'm hoping someone else finds this and it rings a bell.
Sometimes, when a client goes to edit a mailing, the page is either a) unresponsive, or b) doesn't allow saving. The "calculating recipients" SQL is running indefinitely. When investigating, there are tens of thousands of entries in `civicrm_mailing_group` for that mailing - the `entity_id`s repeat.
While I'm unable to replicate, this seems to happen when:
* Users copy an existing mailing rather than starting from a new mailing;
* In both cases, mailings were going out to 17-24 groups.https://lab.civicrm.org/dev/mail/-/issues/99Tokens broken2022-06-11T14:56:45ZJoeMurrayTokens brokenThe contact source and created date tokens are exposed in CiviMail/Mosaico. They do work when doing one off emails and print/merge.The contact source and created date tokens are exposed in CiviMail/Mosaico. They do work when doing one off emails and print/merge.https://lab.civicrm.org/dev/mail/-/issues/98Set time limit for bounce types with hold_threshold > 12022-06-11T14:50:53ZMichael McAndrewSet time limit for bounce types with hold_threshold > 1Bounce types can be split into those with a hold_threshold = 1 where a single bounce should cause this contact to be put on hold (e.g. mailbox non existent), and those where more than one bounce is required (mailbox full).
We don't curr...Bounce types can be split into those with a hold_threshold = 1 where a single bounce should cause this contact to be put on hold (e.g. mailbox non existent), and those where more than one bounce is required (mailbox full).
We don't currently set a time limit on those where more than one bounce is required, which is an oversight. For example, the bounce threshold for Mailbox full is 3. This means that as soon as we have received more than 3 mailbox full messages for a contact, their email will be put on hold. This might make sense if it was 3 in the last year, but doesn't really make sense if it was three in the last 10 years.
Similar arguments can be made for other bounce types, like Away, which is set to 30. If I send a fair amount of email to people that are out of the office quite a lot, over time a significant chunk of these will send me > 30 out of the office messages.
Hence we should only consider messages received in a certain time period. I am going to suggest one year. I would be up for making it user configurable (number of days) if people feel strongly about it.
Let me know what you think.https://lab.civicrm.org/dev/mail/-/issues/97Send order when A/B testing2022-06-11T14:54:42ZufundoSend order when A/B testingAs far as I can see current behaviour when processing an A/B test mailing is that CiviMail sends out to all A recipients then all B recipients.
If you have rate limiting on outbound email that is significant relative to your list size, ...As far as I can see current behaviour when processing an A/B test mailing is that CiviMail sends out to all A recipients then all B recipients.
If you have rate limiting on outbound email that is significant relative to your list size, this can result in a significant difference in (average) send time between the A group and the B group, which may then be confounding your A/B test results.
I wonder how hard it would be to implement some kind of interpolation (?) for the sends for an A/B mailing -- something like firing off equal numbers of As and Bs in each mailing batch as they are processed?https://lab.civicrm.org/dev/mail/-/issues/96Specifying a "Selection Method" without a location type gives a DB error2022-06-11T14:54:23ZJonGoldSpecifying a "Selection Method" without a location type gives a DB errorSteps to replicate (demo server is fine):
* Create a new mailing.
* Click the wrench next to "Recipients".
* Set a *Selection Method* other than "Automatic" but do not touch *Location Type*.
* Attempt to submit the mailing (or press the ...Steps to replicate (demo server is fine):
* Create a new mailing.
* Click the wrench next to "Recipients".
* Set a *Selection Method* other than "Automatic" but do not touch *Location Type*.
* Attempt to submit the mailing (or press the "X Recipients" to recalculate immediately).
Expected Result:
You should be warned that this is an invalid combination.
Actual Result:
You will get a "Mailing content is out of date, please refresh" error. When you refresh, it turns into a "DB Error: Syntax Error".
This combination generates invalid SQL - but when I went to fix it, I realized it made no sense to have this particular combination. So we should warn the user what exactly went wrong.
![Selection_1115](/uploads/c373c331a713ec9b73f7ef8c17842306/Selection_1115.png)5.40.0JonGoldJonGoldhttps://lab.civicrm.org/dev/mail/-/issues/95<link> URLs are tracked and shouldn't be2022-06-11T14:56:58Zlarsssandergreen<link> URLs are tracked and shouldn't beIf you use <link href=URL> in a mailing template, for example to load a webfont, that URL is converted to a tracked URL. Presumably it shouldn't be as this makes a mess of the clickthrough statistics (every one of these links will be cou...If you use <link href=URL> in a mailing template, for example to load a webfont, that URL is converted to a tracked URL. Presumably it shouldn't be as this makes a mess of the clickthrough statistics (every one of these links will be counted as clicked as long as the user's mail client fetches the resource). In other words, a link being fetched in a not a click and shouldn't be counted as one.
Do we need to track any URLs that aren't inside an <a> tag? It might be easier to limit to <a... rather than trying to exclude <link.... The only valid way to write an <a> is with the tag right after the <, so I think we could just look for <a instead of <.
I can provide a patch if this approach is supported.https://lab.civicrm.org/dev/mail/-/issues/94Should tracking pixel be placed at start of email for when Gmail clips messages?2022-06-11T14:57:01ZlarsssandergreenShould tracking pixel be placed at start of email for when Gmail clips messages?With Mosaico templates, it isn't that difficult to reach Gmail's limit of 96-102kB with an email newsletter, after which your email message is clipped and not displayed in full. This is unfortunate in itself, but it also breaks open trac...With Mosaico templates, it isn't that difficult to reach Gmail's limit of 96-102kB with an email newsletter, after which your email message is clipped and not displayed in full. This is unfortunate in itself, but it also breaks open tracking as the tracking pixel is only embedded at the very end of the message and thus isn't fetched unless the reader clicks "View entire message".
Would it be worthwhile to consider putting the tracking pixel at the start of the message to avoid this issue? Obviously, less than ideal for those who don't display images, but I'm not sure how these two might balance out or if there is a clever way to have the best of both worlds.
I'd be happy to provide a patch, but not sure what the best way forward is here.
(Was a former Flexmailer issue, reposted here as I think this is still important.)https://lab.civicrm.org/dev/mail/-/issues/93Include optional unsubscribe group field on regular mailings2022-06-11T14:54:10ZlarsssandergreenInclude optional unsubscribe group field on regular mailingsWhen you create a mailing from search results, you are required to add an unsubscribe group. When you create a regular mailing and add groups to the recipients, you can't specify the unsubscribe group. Sometimes, it would be helpful to b...When you create a mailing from search results, you are required to add an unsubscribe group. When you create a regular mailing and add groups to the recipients, you can't specify the unsubscribe group. Sometimes, it would be helpful to be able to specify an unsubscribe group that is different than the group the contact is a member of. For instance, if you send a mailing to a few groups based on your main mailing list, perhaps a set of smart group that are subsets of the main list, you may want contacts to unsubscribe from your main list instead of the smart group.
Proposal: Add an optional unsubscribe group to all mailings. If it is blank, unsubscribes work as they do now. If it is populated, all recipients are unsubscribed from the unsubscribe group (exactly as this works for a mailing based on a search now).
EDIT: On @JoeMurray's suggestion, contacts would be unsubscribed from both the original group that are recipients of the mailing and the specified unsubscribe group.
I can give this a go if there is support for the concept.https://lab.civicrm.org/dev/mail/-/issues/92Token renders in message preview and test message, not in final sent message2022-06-11T14:58:17ZChrisHardieToken renders in message preview and test message, not in final sent messageI'm on CiviCRM 5.35.1 with WordPress 5.7. Today I used CiviMail for the first time to send a mailing to a group of contacts. I used some tokens (custom contact fields) that rendered fine in my "preview HTML" view and in the generated mes...I'm on CiviCRM 5.35.1 with WordPress 5.7. Today I used CiviMail for the first time to send a mailing to a group of contacts. I used some tokens (custom contact fields) that rendered fine in my "preview HTML" view and in the generated message when I sent a test email to myself. However, in the final sent message that went out to recipients, they were not rendered and instead displayed as `{contact.custom_10}` and `{contact.custom_9}`. Other tokens were rendered just fine. The only thing that seems different about these two is that they are custom fields, and that they were in a bolded line of text.
Is this something I've missed in our configuration? Let me know if you need any other information. Thank you.https://lab.civicrm.org/dev/mail/-/issues/90Make it possible to make a copy of a draft mailing2022-06-11T14:50:38ZlarsssandergreenMake it possible to make a copy of a draft mailingIt would be quite useful to be able to make a copy of a draft mailing. As it stands currently, if you want to duplicate a draft email (for instance, if you want to send two slightly different mailings), you have to send it first and then...It would be quite useful to be able to make a copy of a draft mailing. As it stands currently, if you want to duplicate a draft email (for instance, if you want to send two slightly different mailings), you have to send it first and then reuse it.
I'm not sure if this would be complicated or simple to implement (essentially the same as re-using a mailing or something more complex). Does anyone have thoughts?5.38.0https://lab.civicrm.org/dev/mail/-/issues/88SMS limit is hardcoded at 460 but should be changeable somewhere2022-06-11T14:58:16ZjasonobrownSMS limit is hardcoded at 460 but should be changeable somewhereTwillio (and probably other sms providers) allows SMS messages up to 1600 characters, but Civi has the limit hardcoded at 460. I've verified that by simply going to CRM/SMS/Provider.php and changing MAX_SMS_CHAR from 460 to 1600 the sys...Twillio (and probably other sms providers) allows SMS messages up to 1600 characters, but Civi has the limit hardcoded at 460. I've verified that by simply going to CRM/SMS/Provider.php and changing MAX_SMS_CHAR from 460 to 1600 the system now accepts (and sends) messages up to 1600 characters. It seems logical that the character limit should able to be adjusted and stored somewhere rather than set as a static value in the code. We are currently using the larger limit, and would really like to see this implemented asap so that we can stop manually updating the code each time we update civi.https://lab.civicrm.org/dev/mail/-/issues/87Multilingual and Mailings: the interface only displays the groups selected in...2022-06-11T14:51:05ZbgmMultilingual and Mailings: the interface only displays the groups selected in a given languageThis is a weird bug:
* Enable multi-lingual, and enable a second language (ex: English & French)
* Create a new mailing while in English
* Add a group to the list of recipients.
* Save the draft
Now switch the interface language to the...This is a weird bug:
* Enable multi-lingual, and enable a second language (ex: English & French)
* Create a new mailing while in English
* Add a group to the list of recipients.
* Save the draft
Now switch the interface language to the other language (ex: French), and edit the mailing. Notice that the group has disappeared. It's not gone, but hidden.
In my case, the mailing was re-used from EN to FR, and since the groups were not visible, and there are different mailing groups for each language, the mailing ended up being sent to both language recipients.