CiviCRM Core issueshttps://lab.civicrm.org/dev/core/-/issues2022-11-22T09:40:51Zhttps://lab.civicrm.org/dev/core/-/issues/3908Permission to view Subscription History in SearchKit2022-11-22T09:40:51ZwmortadaPermission to view Subscription History in SearchKitOverview
----------------------------------------
I've created a saved search using SearchKit to show contacts that have recently signed up to a mailing list. This search works fine for users with full administer permissions to CiviCRM ...Overview
----------------------------------------
I've created a saved search using SearchKit to show contacts that have recently signed up to a mailing list. This search works fine for users with full administer permissions to CiviCRM but doesn't show any results for users with more limited permissions to CiviCRM.
Reproduction steps
----------------------------------------
1. Create a search using SearchKit that uses the Contact Subscription Histories entity - see below
2. View the search as a user with access to CiviCRM but not the 'administer CiviCRM' permission
![image](/uploads/7ec66c8b8ae197ebb4c6d0c5a847cb36/image.png)
Current behaviour
----------------------------------------
The results don't load for the user with limited permissions. Note that the column headings are also missing.
![image](/uploads/aa2eadeac7872d9ff9432d0c6fcb0d4f/image.png)
Expected behaviour
----------------------------------------
The results should load
![image](/uploads/6a430f08519863c1cf05fb8f19b06bf8/image.png)
Environment information
----------------------------------------
CiviCRM 5.56.alpha1https://lab.civicrm.org/dev/core/-/issues/3905Need to increase data size for 'data' column on 'civicrm_job_log' table2022-10-12T13:27:08ZyashodhaNeed to increase data size for 'data' column on 'civicrm_job_log' tableOverview
----------------------------------------
Currently _data_ column of the _civicrm_job_log_ table is of data type TEXT. This is sufficient for running most jobs. However we have a some comprehensive log for mailchimp sync extensio...Overview
----------------------------------------
Currently _data_ column of the _civicrm_job_log_ table is of data type TEXT. This is sufficient for running most jobs. However we have a some comprehensive log for mailchimp sync extension and it would throw errors when running the job.
`INSERT INTO civicrm_job_log (domain_id , job_id , name , command , ...", "1406 ** Data too long for column 'data' at row 1`
Proposed behaviour
----------------------------------------
The Proposal is to change the data type of the _data_ column of the _civicrm_job_log_ table to LONGTEXT instead of TEXT.5.56.0yashodhayashodhahttps://lab.civicrm.org/dev/core/-/issues/3904SearchKit: Active Relationship filter doesn't work in Form Builder2024-02-01T11:35:21ZGhost UserSearchKit: Active Relationship filter doesn't work in Form BuilderOverview
----------------------------------------
When you create a report with Search Kit and Form Builder to view the current Active Relationships, when you filter by "Is Active: Yes" it messes up and shows you the inactive Relationsh...Overview
----------------------------------------
When you create a report with Search Kit and Form Builder to view the current Active Relationships, when you filter by "Is Active: Yes" it messes up and shows you the inactive Relationships.
Reproduction steps
----------------------------------------
1. Create a Search Kit with Related Contacts such as Below
![image](/uploads/76051b5efdcbe35c7ae8f8d0f06ef021/image.png)
2. In the Form Builder add the "Relationship is Active" Field to be able to filter by it.
![image](/uploads/973c5d23ca11fece04f5eb86e957bbbe/image.png)
3. In the Report just filter by active Relationhsips. It will show you the inactive relationships.
![image](/uploads/bca81addc8e401ebf691152bfe62b44f/image.png)
Current behaviour
----------------------------------------
When you filter in the Search Kit Form it filters correctly.
![image](/uploads/089efa35f6f7bbb42836608e7e9e6ad2/image.png)
But when you filter in the Form, it shows you the inactive relationships, as showed above.
When you filter by Inactive relationships they are showed correctly.
![image](/uploads/226ea4dd88937432a00b3db138d5ff69/image.png)
But when you filter by both, it shows only Inactive Relationships.
![image](/uploads/e06ba811de15c17b6db2439a6243727d/image.png)
When you leave the filtering empy it works fine. It shows Active and Inactive Relationships.
Expected behaviour
----------------------------------------
Filter by Active relationships when you input "Yes".
Filter by Inactive relationships when you input "No".
Environment information
----------------------------------------
* __CiviCRM:__ dmaster v.5.56https://lab.civicrm.org/dev/core/-/issues/3893Searchkit: Change Rewrite field to textarea2023-11-15T14:51:59ZshaneonabikeSearchkit: Change Rewrite field to textareaHey there,
After realizing that we can use SMARTY conditions in Searchkit this really took one of our clients systems to an entire new level. Thanks so much for all the hard work on Searchkit!
## Problem
Presently, the Searchkit Rewri...Hey there,
After realizing that we can use SMARTY conditions in Searchkit this really took one of our clients systems to an entire new level. Thanks so much for all the hard work on Searchkit!
## Problem
Presently, the Searchkit Rewrite field is a textfield, which is fine if you want to append a few characters to the end or before a value. It gets extremely hard to read and also modify (I have to use a texteditor beside) to modify if you start using SMARTY attributes.
In the situation below, we are generating one column that contains one of two values using a SMARTY ```if```.
![Selection_001](/uploads/57432093e023009c9b311a77f1fe00eb/Selection_001.png)
to generate
![Selection_002](/uploads/1a87132c9a0e9ce080c8e9bd2418c5ea/Selection_002.png)
## Proposed solution
Would it be difficult to modify this field to a textarea instead? It would make it much easier to modify and read. I really don't know if this would be a massive change but I think it would really help. Also, we could provide a help icon to inform people that they can use SMARTY in these fields as long as they put quotes around the different values being retrieved (this was the element I was missing when I first tried and didn't get it working.
![Selection_003](/uploads/10973172b4588429613fbc87616ea879/Selection_003.png)5.56.0https://lab.civicrm.org/dev/core/-/issues/3866API - CustomGroup - extends_entity_column_value2022-09-26T23:06:37ZtschuettlerAPI - CustomGroup - extends_entity_column_valueOverview
----------------------------------------
Filtering custom field groupy by `extends_entity_column_value` is not possible via API. It is also not possible to create a custom field group via API explorer when trying to limit it by ...Overview
----------------------------------------
Filtering custom field groupy by `extends_entity_column_value` is not possible via API. It is also not possible to create a custom field group via API explorer when trying to limit it by `extends_entity_column_value`, because you have no option value to select.
Reproduction steps
----------------------------------------
1. Create a new Custom Field Group via UI.
* Let it extend `Activity` and limit it to `Phone Call`
1. Try to retrieve that Custom Field Group via APIv3 or APIv4 by adding a filter on `extends_entity_column_value`
1. The field for the Option list will remain empty with a loading gif in the API Explorer
Current behaviour
----------------------------------------
``` php
\Civi\Api4\CustomGroup::get()
->addWhere('extends', '=', 'Activity')
->addWhere('extends_entity_column_value:name', '=', 'Phone Call')
->execute();
```
Returns an empty result. The value for `extends_entity_column_value` seems to get dropped in the process of executing the query.
Creating the Custom Group via API works via code or browser, by creating the request manually.
https://dmaster.demo.civicrm.org/civicrm/api4#/explorer/CustomGroup/create?values=%5B%5B%22title%22,%22Phone%20Call%20Fields%22%5D,%5B%22extends%22,%22Activity%22%5D,%5B%22extends_entity_column_value:name%22,%22Phone%20Call%22%5D%5D
There is no drop down option in the API Explorer.
``` php
\Civi\Api4\CustomGroup::create()
->addValue('title', 'Phone Call Fields')
->addValue('extends', 'Activity')
->addValue('extends_entity_column_value:name', 'Phone Call')
->execute();
```
Expected behaviour
----------------------------------------
Existing custom field groups should be filterable by `extends_entity_column_value` via API.
Environment information
----------------------------------------
* __Browser:__ _Firefox 102.2.0
* https://dmaster.demo.civicrm.org
Comments
----------------------------------------
This seems to fail both with APIv3 and APIv4.https://lab.civicrm.org/dev/core/-/issues/3844Dummy payment processor should be flagged as such on LIVE page2023-01-31T09:53:32ZyashodhaDummy payment processor should be flagged as such on LIVE pageWhen dummy payment processor is configured for contribution pages/ event, there is no indication that is NOT a real transaction. This could be confusing and we should flag any LIVE page that is configured with dummy processor.
![dummy](...When dummy payment processor is configured for contribution pages/ event, there is no indication that is NOT a real transaction. This could be confusing and we should flag any LIVE page that is configured with dummy processor.
![dummy](/uploads/7cbf8624f40a97ab1c89d123ab4b4941/dummy.png)
![con_page](/uploads/1ea201d5b3c96e90e5385b5f82596996/con_page.png)
No indication that this is a Dummy payment processor
![contr+page](/uploads/456a92cbdafa407721c1ac4545cbb22e/contr+page.png)
This would similar to what we already do for TEST transactions.5.59.0yashodhayashodhahttps://lab.civicrm.org/dev/core/-/issues/3802Symfony 6 can't find the EventDispatcher when adding hooks that are defined i...2022-08-15T02:11:04ZDaveDSymfony 6 can't find the EventDispatcher when adding hooks that are defined in getSubscribedEventsThey did this, and also a little higher up you can see they removed the constructor: https://github.com/symfony/event-dispatcher/commit/0bc2e0d8ba8a6eb353a42d19890f57a3dee410a5#diff-f3f413e17bbd20e345b3721f9c4556ed5e319d133c32bf7492dffd5...They did this, and also a little higher up you can see they removed the constructor: https://github.com/symfony/event-dispatcher/commit/0bc2e0d8ba8a6eb353a42d19890f57a3dee410a5#diff-f3f413e17bbd20e345b3721f9c4556ed5e319d133c32bf7492dffd55a024d0efR57
So what happens is this https://github.com/civicrm/civicrm-core/blob/da21dc84950b013a03d3fe5e72e12e7d652cce7f/Civi/Core/Container.php#L93 is expecting it to be called `dispatcher`, but this https://github.com/symfony/event-dispatcher/commit/0bc2e0d8ba8a6eb353a42d19890f57a3dee410a5#diff-f3f413e17bbd20e345b3721f9c4556ed5e319d133c32bf7492dffd55a024d0efR57 is expecting it to be called `event_dispatcher`, so that returns early and never calls `$container->findTaggedServiceIds('kernel.event_listener'`, so doesn't pick up any of the getSubscribedEvents hooks, e.g. civi.token.eval and others.
The naive fix of calling `$container->setAlias('event_dispatcher', 'dispatcher')` doesn't seem to be enough and causes a crash later which I haven't tracked down yet.5.54.0https://lab.civicrm.org/dev/core/-/issues/3634When using a smart group as a mailing list, users who unsubscribe from the sm...2022-06-11T14:57:45Ztom.mWhen using a smart group as a mailing list, users who unsubscribe from the smart group are still included in the mailing.It looks like in CRM/MailingBAO/Mailing.php, is not taking into account those who have been removed from the smart group. Civi version 5.3.
```
if (count($includeSmartGroupIDs)) {
$query = CRM_Utils_SQL_Select::from($contact)
...It looks like in CRM/MailingBAO/Mailing.php, is not taking into account those who have been removed from the smart group. Civi version 5.3.
```
if (count($includeSmartGroupIDs)) {
$query = CRM_Utils_SQL_Select::from($contact)
->select("$contact.id as contact_id, $entityTable.id as $entityColumn")
->join($entityTable, " INNER JOIN $entityTable ON $entityTable.contact_id = $contact.id ")
->join('gc', " INNER JOIN civicrm_group_contact_cache gc ON $contact.id = gc.contact_id ")
->join('mg', " INNER JOIN civicrm_mailing_group mg ON gc.group_id = mg.entity_id AND mg.search_id IS NULL ")
->join('temp', " LEFT JOIN $excludeTempTablename temp ON $contact.id = temp.contact_id ")
->where('gc.group_id IN (#groups)')
->merge($criteria)
->replaceInto($includedTempTablename, array('contact_id', $entityColumn))
->param('#groups', $includeSmartGroupIDs)
->param('#mailingID', $mailingID)
->execute();
}
```https://lab.civicrm.org/dev/core/-/issues/3632Mass SMS not sent to contacts whose mobile phone is not their Primary phone2022-06-12T21:25:03ZkenMass SMS not sent to contacts whose mobile phone is not their Primary phoneI have just upgraded to CiviCRM 5.3.2 and have found that Mass SMS are not being delivered to contacts where the mobile phone is not the Primary phone.
The change was made in this commit https://github.com/civicrm/civicrm-core/commit/6f...I have just upgraded to CiviCRM 5.3.2 and have found that Mass SMS are not being delivered to contacts where the mobile phone is not the Primary phone.
The change was made in this commit https://github.com/civicrm/civicrm-core/commit/6f3a35e0986bc21ffbf730c7d3f20f2894c1bb58#diff-8c026e80a2f25f46826dd9568bc301b0 in this PR https://github.com/civicrm/civicrm-core/pull/11558 as a result of this refactoring https://issues.civicrm.org/jira/browse/CRM-21316.
It seems from the unit test case that the code handles the case where a contact has multiple Mobile phones and chooses the Primary phone over the non-Primary phone.
But the code fails if the Primary phone is not the Mobile one.
What is needed is something similar to the Mass Email case. For emails, CRM_Mailing_BAO_Mailing::getLocationFilterAndOrderBy() generates a WHERE clause and an ORDER BY clause that ensures the right emails are selected and preferred.https://lab.civicrm.org/dev/core/-/issues/3628Add pre/post hook for CRM_Mailing_BAO_MailingJob2022-06-15T11:19:25ZMonish DebAdd pre/post hook for CRM_Mailing_BAO_MailingJobThis ticket is about adding pre, post hook to MailingJob and it involve:
1. Call the pre/post hook inside `CRM_Mailing_BAO_MailingJob::create()`
2. Add `CRM_Mailing_BAO_MailingJob::deleteMailingJob` to delete Mailing Jobs and add pre/pos...This ticket is about adding pre, post hook to MailingJob and it involve:
1. Call the pre/post hook inside `CRM_Mailing_BAO_MailingJob::create()`
2. Add `CRM_Mailing_BAO_MailingJob::deleteMailingJob` to delete Mailing Jobs and add pre/post delete hook
3. Replace all the DAO call with corresponding create and delete fn in the codebase.Monish DebMonish Debhttps://lab.civicrm.org/dev/core/-/issues/3608getRecipients() appears to use a WHERE clause as an ORDER BY clause2022-06-11T14:55:55ZkengetRecipients() appears to use a WHERE clause as an ORDER BY clauseSee https://github.com/civicrm/civicrm-core/commit/906298d3f63825adbee0e85963d89354edf8acf1#r30370554See https://github.com/civicrm/civicrm-core/commit/906298d3f63825adbee0e85963d89354edf8acf1#r303705545.8https://lab.civicrm.org/dev/core/-/issues/3600Bounce processing fails for 4-byte unicode characters2022-06-11T14:55:27ZtschuettlerBounce processing fails for 4-byte unicode charactersWhen the bounced message contains any 4-byte unicode character, it cannot be saved to the database due to the currently missing support for utf8mb4 in CiviCRM.
Error Log:
```
[info] $backTrace = #0 /opt/buildkit/build/dmaster/sites/all...When the bounced message contains any 4-byte unicode character, it cannot be saved to the database due to the currently missing support for utf8mb4 in CiviCRM.
Error Log:
```
[info] $backTrace = #0 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/CRM/Core/Error.php(948): CRM_Core_Error::backtrace("backTrace", TRUE)
#1 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/packages/PEAR.php(921): CRM_Core_Error::exceptionHandler(Object(DB_Error))
#2 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/packages/DB.php(985): PEAR_Error->__construct("DB Error: unknown error", -1, 16, (Array:2), "INSERT INTO civicrm_mailing_event_bounce (event_queue_id , bounce_type_id , b...")
#3 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/packages/PEAR.php(575): DB_Error->__construct(-1, 16, (Array:2), "INSERT INTO civicrm_mailing_event_bounce (event_queue_id , bounce_type_id , b...")
#4 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/packages/PEAR.php(223): PEAR->_raiseError(Object(DB_mysqli), NULL, -1, 16, (Array:2), "INSERT INTO civicrm_mailing_event_bounce (event_queue_id , bounce_type_id , b...", "DB_Error", TRUE)
#5 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/packages/DB/common.php(1907): PEAR->__call("raiseError", (Array:7))
#6 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/packages/DB/mysqli.php(933): DB_common->raiseError(-1, NULL, NULL, "INSERT INTO civicrm_mailing_event_bounce (event_queue_id , bounce_type_id , b...", "1366 ** Incorrect string value: '\xF0\x9F\x8C\xB4' for column 'bounce_reason'...")
#7 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/packages/DB/mysqli.php(403): DB_mysqli->mysqliRaiseError()
#8 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/packages/DB/common.php(1216): DB_mysqli->simpleQuery("INSERT INTO civicrm_mailing_event_bounce (event_queue_id , bounce_type_id , b...")
#9 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/packages/DB/DataObject.php(2415): DB_common->query("INSERT INTO civicrm_mailing_event_bounce (event_queue_id , bounce_type_id , b...")
#10 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/packages/DB/DataObject.php(1040): DB_DataObject->_query("INSERT INTO civicrm_mailing_event_bounce (event_queue_id , bounce_type_id , b...")
#11 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/CRM/Core/DAO.php(571): DB_DataObject->insert()
#12 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/CRM/Mailing/Event/BAO/Bounce.php(87): CRM_Core_DAO->save()
#13 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/api/v3/Mailing.php(349): CRM_Mailing_Event_BAO_Bounce::create((Array:7))
#14 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/Civi/API/Provider/MagicFunctionProvider.php(100): civicrm_api3_mailing_event_bounce((Array:7))
#15 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/Civi/API/Kernel.php(169): Civi\API\Provider\MagicFunctionProvider->invoke((Array:9))
#16 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/Civi/API/Kernel.php(100): Civi\API\Kernel->runRequest((Array:9))
#17 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/api/api.php(23): Civi\API\Kernel->runSafe("Mailing", "event_bounce", (Array:6), NULL)
#18 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/CRM/Utils/Mail/EmailProcessor.php(341): civicrm_api("Mailing", "event_bounce", (Array:6))
#19 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/CRM/Utils/Mail/EmailProcessor.php(59): CRM_Utils_Mail_EmailProcessor::_process(TRUE, Object(CRM_Core_DAO_MailSettings), 0)
#20 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/api/v3/Job.php(403): CRM_Utils_Mail_EmailProcessor::processBounces(0)
#21 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/Civi/API/Provider/MagicFunctionProvider.php(100): civicrm_api3_job_fetch_bounces((Array:3))
#22 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/Civi/API/Kernel.php(169): Civi\API\Provider\MagicFunctionProvider->invoke((Array:9))
#23 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/Civi/API/Kernel.php(100): Civi\API\Kernel->runRequest((Array:9))
#24 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/api/api.php(23): Civi\API\Kernel->runSafe("job", "fetch_bounces", (Array:2), NULL)
#25 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/Civi/Test/Api3TestTrait.php(262): civicrm_api("job", "fetch_bounces", (Array:2))
#26 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/Civi/Test/Api3TestTrait.php(151): CiviUnitTestCase->civicrm_api("job", "fetch_bounces", (Array:2))
#27 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/tests/phpunit/CRM/Utils/Mail/EmailProcessorTest.php(78): CiviUnitTestCase->callAPISuccess("job", "fetch_bounces", (Array:2))
#28 [internal function](): CRM_Utils_Mail_EmailProcessorTest->testBounceProcessingUTF8mb4()
#29 phar:///opt/buildkit/extern/phpunit5/phpunit5.phar/phpunit/Framework/TestCase.php(1062): ReflectionMethod->invokeArgs(Object(CRM_Utils_Mail_EmailProcessorTest), (Array:0))
#30 /opt/buildkit/build/dmaster/sites/all/modules/civicrm/tests/phpunit/CiviTest/CiviUnitTestCase.php(196): PHPUnit_Framework_TestCase->runTest()
#31 phar:///opt/buildkit/extern/phpunit5/phpunit5.phar/phpunit/Framework/TestCase.php(913): CiviUnitTestCase->runTest()
#32 phar:///opt/buildkit/extern/phpunit5/phpunit5.phar/phpunit/Framework/TestResult.php(686): PHPUnit_Framework_TestCase->runBare()
#33 phar:///opt/buildkit/extern/phpunit5/phpunit5.phar/phpunit/Framework/TestCase.php(868): PHPUnit_Framework_TestResult->run(Object(CRM_Utils_Mail_EmailProcessorTest))
#34 phar:///opt/buildkit/extern/phpunit5/phpunit5.phar/phpunit/Framework/TestSuite.php(733): PHPUnit_Framework_TestCase->run(Object(PHPUnit_Framework_TestResult))
#35 phar:///opt/buildkit/extern/phpunit5/phpunit5.phar/phpunit/TextUI/TestRunner.php(517): PHPUnit_Framework_TestSuite->run(Object(PHPUnit_Framework_TestResult))
#36 phar:///opt/buildkit/extern/phpunit5/phpunit5.phar/phpunit/TextUI/Command.php(186): PHPUnit_TextUI_TestRunner->doRun(Object(PHPUnit_Framework_TestSuite), (Array:54), TRUE)
#37 phar:///opt/buildkit/extern/phpunit5/phpunit5.phar/phpunit/TextUI/Command.php(116): PHPUnit_TextUI_Command->run((Array:11), TRUE)
#38 /opt/buildkit/extern/phpunit5/phpunit5.phar(598): PHPUnit_TextUI_Command::main()
#39 {main}
```5.11https://lab.civicrm.org/dev/core/-/issues/3599Test mailings create new contacts even when "Add Contacts" permission is not ...2022-06-11T14:55:17ZJonGoldTest mailings create new contacts even when "Add Contacts" permission is not present.### Overview
When sending a test mail, CiviCRM will check if the email matches an existing contact in the database. If it does, it uses that contact ID; if not, it creates a contact. However, it creates a contact without regard for wh...### Overview
When sending a test mail, CiviCRM will check if the email matches an existing contact in the database. If it does, it uses that contact ID; if not, it creates a contact. However, it creates a contact without regard for whether a user has permission to add contacts.
### Steps to replicate
* Create a user that does not have the "Add Contacts" permission.
* With that user, create a new mailing (traditional or Mosaico, doesn't matter).
* Send a test ("draft") mail to an email address that doesn't exist in the database.
### Expected behavior
No new contact is created.
### Actual behavior
A new contact is created.5.29.0JonGoldJonGoldhttps://lab.civicrm.org/dev/core/-/issues/3581Email to activity processing: New feature to skip emails which do not have a ...2022-06-11T14:54:48ZJamie Novick - CompucoEmail to activity processing: New feature to skip emails which do not have a Case ID or Case token- As a CiviCRM administrator
- I would like to configure whether CiviCRM will process emails without a case ID (or case “token”) in the subject line
- so that I can ensure that emails which do not have a case ID are not filed on the con...- As a CiviCRM administrator
- I would like to configure whether CiviCRM will process emails without a case ID (or case “token”) in the subject line
- so that I can ensure that emails which do not have a case ID are not filed on the contact record outside the case by accident.
**How it works currently**
For those a little less familiar with email to activity processing:
CiviCRM will connect to a users MS exchange mailbox and create the following folder structure:
- Inbox
- /CiviCRM
- //CiviMail
- ///ignored
- ///processed
Notes:
- Users simply copy or move emails into the /civicrm folder in their inbox. CiviCRM has a scheduled job that can be configured to run periodically (say every hour) and poll the mailbox folder (Civimail) by IMAP in order to read and process the emails.
- By default CiviCRM will match any email from, to, cc fields to contacts in the CRM and file the email as an activity against those contacts (including recording any attachments as files).
- If however there is a case ID in the subject line (or a case ID "token"*) then CiviCRM will instead file the email straight onto the case itself. The format is: [case #1234] (see: https://issues.civicrm.org/jira/browse/CRM-21446)
- If the email is processed successfully it will be moved to the processed folder.
- If for any reason CiviCRM cannot file the email it will be moved to the ignored folder. This normally happens if the email address is invalid for some reason (please note: emails that are sent internally between staff on exchange server can sometimes have this problem as exchange doesn't always use the external email address but instead uses some local username/domain combination - this maybe something to test and see if it maybe a problem for NEU).
- Note: *When sending out emails from CiviCRM from CiviCase it appends a case ID token - which is a string of characters and not the exact Case ID. This is done to obscure the case ID number in the email. In effect you can have either this token or the case id in the subject line and CiviCRM will file the email correctly.
- More background: https://docs.civicrm.org/sysadmin/en/latest/setup/civimail/inbound/#autofiling-email-activities-via-emailprocessor
**Problem**
For multiple clients they want to use email to activity processing for their casework teams. However sometimes they forget to add the case ID to the subject line of the email and the system incorrectly files the email outside the case - which is a "security" risk as the emails can be sensitive.
As such they would like to be able to specify that the emails being filed from the casework team inbox will only be filed if there is a valid Case ID in the subject line and are skipped if not, and hence there is no risk of the email being filed outside of the case.
**Proposed improvement**
Approach:
When "Used For?" = Email to activity processing
Show an additional option:
- Skip emails which do not have a Case ID or Case token:
- Checkbox
- Help text:
- CiviCRM has functionality to file emails which contain the Case ID or Case Hash in the subject line in the format [case #1234] against a case record.
- Where the Case ID or Case Hash is not included CiviCRM will file the email against the contact record, by matching the email addresses on the email with any email addresses of Contact records in CiviCRM.
- Enabling this option will have CiviCRM skip any emails that do not have the Case ID or Case Hash so that the system will only process emails that can be placed on case records.
- Any emails that are not processed will be moved the ignored folder.
- Default null
- If checked:
- Emails which do not have a valid case ID or case token should be moved into the “ignored” folder. (See folders above) after processing and no Activity should be created.
Would be great to know if we can get the magical "concept approved" flag.
We need to work on this quite urgently so if there are no great concerns that would be much appreciated...5.31.0https://lab.civicrm.org/dev/core/-/issues/3579Add pause/resume functionality to civicrm bulk mailing.2022-06-11T14:54:43ZjitendraAdd pause/resume functionality to civicrm bulk mailing.Scenario
Site admin is sending out a big blast but want to send out a press release without waiting for first job to end.Scenario
Site admin is sending out a big blast but want to send out a press release without waiting for first job to end.5.4.0https://lab.civicrm.org/dev/core/-/issues/3572Deduping test email is case sensitive2022-06-11T14:54:27ZMartinDeduping test email is case sensitiveTested on 4.6.38 and 5.6.1.
When composing a mailing, the email address used in the "send test email to" field at the bottom is deduped. This is case sensitive, so fails to match emails that are the same but have different upper/lower c...Tested on 4.6.38 and 5.6.1.
When composing a mailing, the email address used in the "send test email to" field at the bottom is deduped. This is case sensitive, so fails to match emails that are the same but have different upper/lower case combinations. In this case, it will create a new contact with this interesting property: the value in the "email" field shows with the email address flattened to all lowercase, but the display name for the contact shows with the upper/lower case mix used when it was entered to test the mailing.
The upshot of all this is that if I'm putting together a mailing and send 6 test to the same email address (with an uppercase letter), I get 6 new duplicate contacts because it fails to match it each time.
Request to make the dedupe test for this mailing field case insensitive, since in general civi seems to want email addresses to be lowercase when they're used.
I can potentially make this change if someone can point me in the right direction.
Thanks!5.11https://lab.civicrm.org/dev/core/-/issues/3566Public View link does not show from Scheduled and Sent screen2022-06-11T14:54:15ZseamusleePublic View link does not show from Scheduled and Sent screenWhen you view a list of scheduled and sent mailings you cannot access the Public View url link even tho the code suggests it should be availableWhen you view a list of scheduled and sent mailings you cannot access the Public View url link even tho the code suggests it should be available5.10https://lab.civicrm.org/dev/core/-/issues/3557Preview screen doesn't open until recipients list is built on mail compose sc...2022-06-11T14:51:02ZMonish DebPreview screen doesn't open until recipients list is built on mail compose screenTo replicate the issue:
1. Create a new mailing and fill out everything but not the Recipients field.
2. Now select a recipients group that is either very large or is based on a complex smart group. after selecting it -- click preview...To replicate the issue:
1. Create a new mailing and fill out everything but not the Recipients field.
2. Now select a recipients group that is either very large or is based on a complex smart group. after selecting it -- click preview HTML/Text.
The preview window will not open until the recipient count query has completed. This can also be replicated by setting up a mailing with a large/complex group, saving it, then continuing it and immediately clicking preview. the window will not open until the query completes.Monish DebMonish Debhttps://lab.civicrm.org/dev/core/-/issues/3550Email to activity processing: New feature to make the creation of new contact...2022-06-11T14:50:44ZJamie Novick - CompucoEmail to activity processing: New feature to make the creation of new contacts "optional"**User story:**
- As an administrator
- When filing inbound emails
- I would like to configure whether CiviCRM will create a new contact where a contact does not already exist within the system
- So that I can ensure that new contacts a...**User story:**
- As an administrator
- When filing inbound emails
- I would like to configure whether CiviCRM will create a new contact where a contact does not already exist within the system
- So that I can ensure that new contacts are not created in the system when filing an email.
- In the situation of a case email where the email has a valid Case ID or Case token in the subject line the email should still be filed to a case, even if no contacts are matched.
**How it works currently:**
Currently, when CiviCRM performs email to activity processing, Civi will search CiviCRM for a contact with the relevant email address in the From / To / CC fields and then assign the created activity accordingly.
When CiviCRM cannot find a contact with a relevant email address, it simply creates a new "stub" contact with the email address.
**The problem**
For multiple reasons this is might not be desirable behaviour:
This is a security risk, as in most cases CiviCRM email-activity processing works off a mailbox, that might need to be public. As such it is possible that an unscrupulous third party may chose to spam your mailbox and thus spam your CRM with unwanted emails and content. Whilst this can be mitigated by having your automated filing on a separate mailbox, it might be preferable to simply only file the emails for contacts who already exist in your system, and to selectively add the rest.
Also, from a GDPR perspective it may not be desirable to hold the details of the persons who sent an email, whilst the contents of the email might be important (i.e. the attachments) especially in a casework context.
**Proposed solution:**
Add an additional option to the email to activity processing configurations (civicrm/admin/mailSettings?action=add&reset=1)
- When "Used For?" = Email to activity processing
- Show an additional option:
- "Do not create new contacts when filing emails"
- Checkbox
- Default null
- Help:
- If this option is enabled, CiviCRM will not create new contacts when filing emails.
- The email should also always be filed as an activity.
If checked:
- When CiviCRM checks for a matching contact, if no matching contact is found it will not create one and the email is skipped
- unless the email has a valid Case ID or Case token in the subject line, in which it will still be filed against the case, but no contacts will be linked to the email.5.31.0https://lab.civicrm.org/dev/core/-/issues/3544Bounce processing doesn't catch pattern "user doesn't exist"2022-06-11T14:50:31ZDetlev SieberBounce processing doesn't catch pattern "user doesn't exist"CiviCRM bounce processing analyzes the bounce text patterns, and decides what kind of bounce type it is. This is a fragile process, but there seems to be no better solution, since there is no general standardization of bounce patterns.
...CiviCRM bounce processing analyzes the bounce text patterns, and decides what kind of bounce type it is. This is a fragile process, but there seems to be no better solution, since there is no general standardization of bounce patterns.
Some email providers throw the bounce pattern "user doesn't exist", which is not found, and therefore results in bounce_type "syntax". However, this bounce pattern should result in immediately switching the email address to "inactive".
Solution is:
> insert into civicrm_mailing_bounce_pattern (bounce_type_id, pattern) values (6, 'user doesn\'t exist');
5.9Monish DebMonish Deb