Development issueshttps://lab.civicrm.org/groups/dev/-/issues2022-06-21T08:07:08Zhttps://lab.civicrm.org/dev/core/-/issues/3668Lack of hooks to detect when a "CustomOption" is deleted2022-06-21T08:07:08ZhaystackLack of hooks to detect when a "CustomOption" is deletedOverview
----------------------------------------
When a "CustomOption" is deleted, no hooks fire because all database operations are done via the `CRM_Core_DAO::executeQuery` method.
Reproduction steps
--------------------------------...Overview
----------------------------------------
When a "CustomOption" is deleted, no hooks fire because all database operations are done via the `CRM_Core_DAO::executeQuery` method.
Reproduction steps
----------------------------------------
1. Delete an Option on, for example, a Custom Field of Data Type "Alphanumeric" and Field Type "CheckBox".
1. No hooks fire.
Expected behaviour
----------------------------------------
At minimum `civicrm_pre` and `civicrm_post` should fire.
I realise that this papers over the way in which Options are handled for Custom Fields (i.e. they should probably be handled using the same logic as other OptionValues) but rebuilding the CustomOption system isn't on my radar at the moment - sorry!
Comments
----------------------------------------
PR that introduces `civicrm_pre` and `civicrm_post` hooks to follow.haystackhaystackhttps://lab.civicrm.org/dev/core/-/issues/36675.51 import - cannot add a contact to an existing relationship2022-06-20T21:28:36ZAndy Clark5.51 import - cannot add a contact to an existing relationshipI added 3 contacts each with a relationship to another contact. One of the related contacts existed, the other two did not. The new related contacts were added, but the relationship to the existing contact was rejected - see screen prin...I added 3 contacts each with a relationship to another contact. One of the related contacts existed, the other two did not. The new related contacts were added, but the relationship to the existing contact was rejected - see screen print. This is a regression as you should be able to import contacts that relate to existing contacts. Using nightly from 17/6.![screenshot_20220617_175039](/uploads/d358d81d5c1b05674befa4c6b6e34bcd/screenshot_20220617_175039.png)5.51.0https://lab.civicrm.org/dev/core/-/issues/36665.51 import - crash when input date format not selected correctly2022-06-20T21:23:15ZAndy Clark5.51 import - crash when input date format not selected correctlyWhen importing contacts with a date column, I used UK format of dd/mm/yyyy in the input file - but did not select that format on the first page (it defaults to US format). When I hit 'Continue' on the next screen the result was that the ...When importing contacts with a date column, I used UK format of dd/mm/yyyy in the input file - but did not select that format on the first page (it defaults to US format). When I hit 'Continue' on the next screen the result was that the imports were rejected but when I click on the 'Download Errors' link I get the error screen with 'DB error: syntax error'. The link that I followed was 'http://example.com/civicrm/import/outcome?user_job_id=6&status=4&reset=1' Log Viewer screen print attached. Using the nightly from 17/6.![screenshot_20220617_172242](/uploads/926883e3cf2908005f5950a2f9e41436/screenshot_20220617_172242.png)5.51.0https://lab.civicrm.org/dev/core/-/issues/36655.51 import with relationships - count of inserts incorrect2022-06-22T13:38:20ZAndy Clark5.51 import with relationships - count of inserts incorrectWhen inserting contacts with relationships where both the contact and the related contact are new, the count of 'Total number of contact records created or modified during the import.' is incorrect. The related contacts that were correc...When inserting contacts with relationships where both the contact and the related contact are new, the count of 'Total number of contact records created or modified during the import.' is incorrect. The related contacts that were correctly added were not counted. In my test case I had 3 rows, where each contact had a relationship to a contact that didn't previously exist so this was inserted by the upload as well. The count as above was 3 but should have been 6. (Using nightly tarball from 17/6)5.51.0https://lab.civicrm.org/dev/core/-/issues/36635.51 import error when CSV header contains spaces in column names2022-08-04T15:56:32ZJonGold5.51 import error when CSV header contains spaces in column namesMy client attempted to import a CSV with this as the header row:
```
ID,IND/ORG,SALUTATION,FIRSTNAME,LASTNAME,Status,Country,EMAILADDRESS,AGBUHIGHLIGHTSRECIPIENT,AGBUARMENIA,YOUNGPROFESSIONALS,Do Not Mail,Valid Address
```
That yielded ...My client attempted to import a CSV with this as the header row:
```
ID,IND/ORG,SALUTATION,FIRSTNAME,LASTNAME,Status,Country,EMAILADDRESS,AGBUHIGHLIGHTSRECIPIENT,AGBUARMENIA,YOUNGPROFESSIONALS,Do Not Mail,Valid Address
```
That yielded this error/backtrace, which seems connected to the field names. Note that I looked at `civicrm_tmp_d_dflt_53f990c8f2166dca1b59a22af4096233` and the column names were snake-cased (e.g. `do_not_mail`).
```
[error] $Fatal Error Details = Array
(
[callback] => Array
(
[0] => CRM_Core_Error
[1] => exceptionHandler
)
[code] => -2
[message] => DB Error: syntax error
[mode] => 16
[debug_info] => SELECT _id, _status_message, ID, IND/ORG, SALUTATION, FIRSTNAME, LASTNAME, Status, Country, EMAILADDRESS, AGBUHIGHLIGHTSRECIPIENT, AGBUARMENIA, YOUNGPROFESSIONALS, Do Not Mail, Valid Address FROM civicrm_tmp_d_dflt_53f990c8f2166dca1b59a22af4096233 WHERE _status IN ("error","invalid","soft_credit_error","pledge_payment_error") [nativecode=1064 ** You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'Mail, Valid Address FROM civicrm_tmp_d_dflt_53f990c8f2166dca1b59a22af4096233 ...' at line 1]
[type] => DB_Error
[user_info] => SELECT _id, _status_message, ID, IND/ORG, SALUTATION, FIRSTNAME, LASTNAME, Status, Country, EMAILADDRESS, AGBUHIGHLIGHTSRECIPIENT, AGBUARMENIA, YOUNGPROFESSIONALS, Do Not Mail, Valid Address FROM civicrm_tmp_d_dflt_53f990c8f2166dca1b59a22af4096233 WHERE _status IN ("error","invalid","soft_credit_error","pledge_payment_error") [nativecode=1064 ** You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'Mail, Valid Address FROM civicrm_tmp_d_dflt_53f990c8f2166dca1b59a22af4096233 ...' at line 1]
[to_string] => [db_error: message="DB Error: syntax error" code=-2 mode=callback callback=CRM_Core_Error::exceptionHandler prefix="" info="SELECT _id, _status_message, ID, IND/ORG, SALUTATION, FIRSTNAME, LASTNAME, Status, Country, EMAILADDRESS, AGBUHIGHLIGHTSRECIPIENT, AGBUARMENIA, YOUNGPROFESSIONALS, Do Not Mail, Valid Address FROM civicrm_tmp_d_dflt_53f990c8f2166dca1b59a22af4096233 WHERE _status IN ("error","invalid","soft_credit_error","pledge_payment_error") [nativecode=1064 ** You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'Mail, Valid Address FROM civicrm_tmp_d_dflt_53f990c8f2166dca1b59a22af4096233 ...' at line 1]"]
)
[debug] $backTrace = #0 /code/vendor/civicrm/civicrm-core/CRM/Core/Error.php(954): CRM_Core_Error::backtrace("backTrace", TRUE)
#1 /code/vendor/pear/pear-core-minimal/src/PEAR.php(944): CRM_Core_Error::exceptionHandler(Object(DB_Error))
#2 /code/vendor/pear/db/DB.php(997): PEAR_Error->__construct("DB Error: syntax error", -2, 16, (Array:2), "SELECT _id, _status_message, ID, IND/ORG, SALUTATION, FIRSTNAME, LASTNAME, S...")
#3 /code/vendor/pear/pear-core-minimal/src/PEAR.php(575): DB_Error->__construct(-2, 16, (Array:2), "SELECT _id, _status_message, ID, IND/ORG, SALUTATION, FIRSTNAME, LASTNAME, S...")
#4 /code/vendor/pear/pear-core-minimal/src/PEAR.php(223): PEAR::_raiseError(Object(DB_mysqli), NULL, -2, 16, (Array:2), "SELECT _id, _status_message, ID, IND/ORG, SALUTATION, FIRSTNAME, LASTNAME, S...", "DB_Error", TRUE)
#5 /code/vendor/pear/db/DB/common.php(1928): PEAR->__call("raiseError", (Array:7))
#6 /code/vendor/pear/db/DB/mysqli.php(936): DB_common->raiseError(-2, NULL, NULL, "SELECT _id, _status_message, ID, IND/ORG, SALUTATION, FIRSTNAME, LASTNAME, S...", "1064 ** You have an error in your SQL syntax; check the manual that correspon...")
#7 /code/vendor/pear/db/DB/mysqli.php(406): DB_mysqli->mysqliRaiseError()
#8 /code/vendor/pear/db/DB/common.php(1234): DB_mysqli->simpleQuery("SELECT _id, _status_message, ID, IND/ORG, SALUTATION, FIRSTNAME, LASTNAME, S...")
#9 /code/vendor/civicrm/civicrm-packages/DB/DataObject.php(2696): DB_common->query("SELECT _id, _status_message, ID, IND/ORG, SALUTATION, FIRSTNAME, LASTNAME, S...")
#10 /code/vendor/civicrm/civicrm-packages/DB/DataObject.php(1829): DB_DataObject->_query("SELECT _id, _status_message, ID, IND/ORG, SALUTATION, FIRSTNAME, LASTNAME, S...")
#11 /code/vendor/civicrm/civicrm-core/CRM/Core/DAO.php(472): DB_DataObject->query("SELECT _id, _status_message, ID, IND/ORG, SALUTATION, FIRSTNAME, LASTNAME, S...")
#12 /code/vendor/civicrm/civicrm-core/CRM/Core/DAO.php(1637): CRM_Core_DAO->query("SELECT _id, _status_message, ID, IND/ORG, SALUTATION, FIRSTNAME, LASTNAME, S...", TRUE)
#13 /code/vendor/civicrm/civicrm-core/CRM/Import/DataSource.php(568): CRM_Core_DAO::executeQuery("SELECT _id, _status_message, ID, IND/ORG, SALUTATION, FIRSTNAME, LASTNAME, S...")
#14 /code/vendor/civicrm/civicrm-core/CRM/Import/DataSource.php(229): CRM_Import_DataSource->instantiateQueryObject()
#15 /code/vendor/civicrm/civicrm-core/CRM/Import/DataSource.php(212): CRM_Import_DataSource->getRow()
#16 /code/vendor/civicrm/civicrm-core/CRM/Import/Forms.php(473): CRM_Import_DataSource->getRows()
#17 /code/vendor/civicrm/civicrm-core/CRM/Import/Forms.php(528): CRM_Import_Forms->getOutputRows((Array:1))
#18 /code/vendor/civicrm/civicrm-core/CRM/Core/Invoke.php(285): CRM_Import_Forms::outputCSV()
#19 /code/vendor/civicrm/civicrm-core/CRM/Core/Invoke.php(69): CRM_Core_Invoke::runItem((Array:17))
#20 /code/vendor/civicrm/civicrm-core/CRM/Core/Invoke.php(36): CRM_Core_Invoke::_invoke((Array:3))
#21 /code/modules/contrib/civicrm/src/Civicrm.php(88): CRM_Core_Invoke::invoke((Array:3))
#22 /code/modules/contrib/civicrm/src/Controller/CivicrmController.php(80): Drupal\civicrm\Civicrm->invoke((Array:3))
#23 [internal function](): Drupal\civicrm\Controller\CivicrmController->main((Array:3), "")
#24 /code/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(123): call_user_func_array((Array:2), (Array:2))
#25 /code/core/lib/Drupal/Core/Render/Renderer.php(564): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}()
#26 /code/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(124): Drupal\Core\Render\Renderer->executeInRenderContext(Object(Drupal\Core\Render\RenderContext), Object(Closure))
#27 /code/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(97): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->wrapControllerExecutionInRenderContext((Array:2), (Array:2))
#28 /code/vendor/symfony/http-kernel/HttpKernel.php(158): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}()
#29 /code/vendor/symfony/http-kernel/HttpKernel.php(80): Symfony\Component\HttpKernel\HttpKernel->handleRaw(Object(Symfony\Component\HttpFoundation\Request), 1)
#30 /code/core/lib/Drupal/Core/StackMiddleware/Session.php(58): Symfony\Component\HttpKernel\HttpKernel->handle(Object(Symfony\Component\HttpFoundation\Request), 1, TRUE)
#31 /code/core/lib/Drupal/Core/StackMiddleware/KernelPreHandle.php(48): Drupal\Core\StackMiddleware\Session->handle(Object(Symfony\Component\HttpFoundation\Request), 1, TRUE)
#32 /code/core/modules/page_cache/src/StackMiddleware/PageCache.php(106): Drupal\Core\StackMiddleware\KernelPreHandle->handle(Object(Symfony\Component\HttpFoundation\Request), 1, TRUE)
#33 /code/core/modules/page_cache/src/StackMiddleware/PageCache.php(85): Drupal\page_cache\StackMiddleware\PageCache->pass(Object(Symfony\Component\HttpFoundation\Request), 1, TRUE)
#34 /code/core/lib/Drupal/Core/StackMiddleware/ReverseProxyMiddleware.php(48): Drupal\page_cache\StackMiddleware\PageCache->handle(Object(Symfony\Component\HttpFoundation\Request), 1, TRUE)
#35 /code/core/lib/Drupal/Core/StackMiddleware/NegotiationMiddleware.php(51): Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, TRUE)
#36 /code/vendor/stack/builder/src/Stack/StackedHttpKernel.php(23): Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, TRUE)
#37 /code/core/lib/Drupal/Core/DrupalKernel.php(708): Stack\StackedHttpKernel->handle(Object(Symfony\Component\HttpFoundation\Request), 1, TRUE)
#38 /code/index.php(19): Drupal\Core\DrupalKernel->handle(Object(Symfony\Component\HttpFoundation\Request))
#39 {main}
```5.51.0https://lab.civicrm.org/dev/core/-/issues/36625.51: "data too long" error when recording duplicates multiple contacts.2022-06-20T21:26:07ZJonGold5.51: "data too long" error when recording duplicates multiple contacts.My client has been testing the 5.51 import some more, I see this backtrace today:
Now I'm not sure what led to this state of affairs, and that's probably a bug in itself we need to track down - but the "data too long" issue could concei...My client has been testing the 5.51 import some more, I see this backtrace today:
Now I'm not sure what led to this state of affairs, and that's probably a bug in itself we need to track down - but the "data too long" issue could conceivably come up even without the other bug manifesting.
```
[info] $CRM_Queue_Page_AJAX_runNext_error = PEAR_Exception: "DB Error: unknown error"
* ERROR TYPE: DB_Error
* ERROR CODE: -1
* ERROR MESSAGE: DB Error: unknown error
* ERROR MODE: 16
* ERROR USERINFO: UPDATE civicrm_tmp_d_dflt_92a94bde8a533a2019a537a7bf1e22f6 SET _status = 'ERROR', _status_message = 'Record duplicates multiple contacts: 2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,123,124,125,126,127,128,129,130,131,132,133,134,135,136,137,138,139,140,141,142,143,144,145,146,147,148,149,150,151,152,153,154,155,156,157,158,159,160,161,162,163,164,165,166,167,168,169,170,171,172,173,174,175,176,177,178,179,180,181,182,183,184,185,186,187,188,189,190,191,192,193,194,195,196,197,198,199,200,201,202,203,204,205,206,207,208,209,210,211,212,213,214,215,216,217,218,219,220,221,222,223,224,225,226,227,228,229,230,231,232,233,234,235,236,237,238,239,240,241,242,243,244,245,246,247,248,249,250,251,252,253,254,255,256,257,258,259,260,261,262,263,264,265,266,267,268,269,270,271,272,273,274,275,276,277,278,279,280,281,282,283,284,285,286,287,288,289,290,291,292,293,294,295,296,297,298,299,300,301,302,303,304,305,306,308,309,310,311,312,313,314,315,316,317,318,319,320,321,322,323,324,325,326,327,328,329,330,331,332,333,334,335,336,337,338,339,340,341,342,343,344,345,346,347,348,349,350,351,352,353,354,355,356,357,358,359,360,361,362,363,364,365,366,367,368,369,370,371,372,373,374,375,376,377,378,379,380,381,382,383,384,385,386,387,388,389,390,391,392,393,394,395,396,397,398,399,400,<...snipped a very long list here...>, 149491' WHERE _id = 1 [nativecode=1406 ** Data too long for column '_status_message' at row 1]
#0 /code/vendor/pear/pear-core-minimal/src/PEAR.php(944): CRM_Core_Error::exceptionHandler(Object(DB_Error))
#1 /code/vendor/pear/db/DB.php(997): PEAR_Error->__construct("DB Error: unknown error", -1, 16, (Array:2), "UPDATE civicrm_tmp_d_dflt_92a94bde8a533a2019a537a7bf1e22f6 SET _status = 'ERR...")
#2 /code/vendor/pear/pear-core-minimal/src/PEAR.php(575): DB_Error->__construct(-1, 16, (Array:2), "UPDATE civicrm_tmp_d_dflt_92a94bde8a533a2019a537a7bf1e22f6 SET _status = 'ERR...")
#3 /code/vendor/pear/pear-core-minimal/src/PEAR.php(223): PEAR::_raiseError(Object(DB_mysqli), NULL, -1, 16, (Array:2), "UPDATE civicrm_tmp_d_dflt_92a94bde8a533a2019a537a7bf1e22f6 SET _status = 'ERR...", "DB_Error", TRUE)
#4 /code/vendor/pear/db/DB/common.php(1928): PEAR->__call("raiseError", (Array:7))
#5 /code/vendor/pear/db/DB/mysqli.php(936): DB_common->raiseError(-1, NULL, NULL, "UPDATE civicrm_tmp_d_dflt_92a94bde8a533a2019a537a7bf1e22f6 SET _status = 'ERR...", "1406 ** Data too long for column '_status_message' at row 1")
#6 /code/vendor/pear/db/DB/mysqli.php(406): DB_mysqli->mysqliRaiseError()
#7 /code/vendor/pear/db/DB/common.php(1234): DB_mysqli->simpleQuery("UPDATE civicrm_tmp_d_dflt_92a94bde8a533a2019a537a7bf1e22f6 SET _status = 'ERR...")
#8 /code/vendor/civicrm/civicrm-packages/DB/DataObject.php(2696): DB_common->query("UPDATE civicrm_tmp_d_dflt_92a94bde8a533a2019a537a7bf1e22f6 SET _status = 'ERR...")
#9 /code/vendor/civicrm/civicrm-packages/DB/DataObject.php(1829): DB_DataObject->_query("UPDATE civicrm_tmp_d_dflt_92a94bde8a533a2019a537a7bf1e22f6 SET _status = 'ERR...")
#10 /code/vendor/civicrm/civicrm-core/CRM/Core/DAO.php(472): DB_DataObject->query("UPDATE civicrm_tmp_d_dflt_92a94bde8a533a2019a537a7bf1e22f6 SET _status = 'ERR...")
#11 /code/vendor/civicrm/civicrm-core/CRM/Core/DAO.php(1637): CRM_Core_DAO->query("UPDATE civicrm_tmp_d_dflt_92a94bde8a533a2019a537a7bf1e22f6 SET _status = 'ERR...", TRUE)
#12 /code/vendor/civicrm/civicrm-core/CRM/Import/DataSource.php(555): CRM_Core_DAO::executeQuery("UPDATE civicrm_tmp_d_dflt_92a94bde8a533a2019a537a7bf1e22f6 SET _status = %1, ...", (Array:2))
#13 /code/vendor/civicrm/civicrm-core/CRM/Import/Parser.php(2013): CRM_Import_DataSource->updateStatus(1, "ERROR", "Record duplicates multiple contacts: 2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,...", NULL, (Array:0))
#14 /code/vendor/civicrm/civicrm-core/CRM/Contact/Import/Parser/Contact.php(183): CRM_Import_Parser->setImportStatus(1, "ERROR", "Record duplicates multiple contacts: 2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,...")
#15 /code/vendor/civicrm/civicrm-core/CRM/Import/Parser.php(1911): CRM_Contact_Import_Parser_Contact->import((Array:10))
#16 /code/vendor/civicrm/civicrm-core/CRM/Queue/Task.php(95): CRM_Import_Parser::runImport(Object(CRM_Queue_TaskContext), 5, 5)
#17 /code/vendor/civicrm/civicrm-core/CRM/Queue/Runner.php(257): CRM_Queue_Task->run(Object(CRM_Queue_TaskContext))
#18 /code/vendor/civicrm/civicrm-core/CRM/Queue/Page/AJAX.php(36): CRM_Queue_Runner->runNext(TRUE)
#19 /code/vendor/civicrm/civicrm-core/CRM/Queue/ErrorPolicy.php(89): CRM_Queue_Page_AJAX::{closure}()
#20 /code/vendor/civicrm/civicrm-core/CRM/Queue/Page/AJAX.php(38): CRM_Queue_ErrorPolicy->call(Object(Closure))
#21 /code/vendor/civicrm/civicrm-core/CRM/Core/Invoke.php(285): CRM_Queue_Page_AJAX::runNext()
#22 /code/vendor/civicrm/civicrm-core/CRM/Core/Invoke.php(69): CRM_Core_Invoke::runItem((Array:17))
#23 /code/vendor/civicrm/civicrm-core/CRM/Core/Invoke.php(36): CRM_Core_Invoke::_invoke((Array:4))
#24 /code/modules/contrib/civicrm/src/Civicrm.php(88): CRM_Core_Invoke::invoke((Array:4))
#25 /code/modules/contrib/civicrm/src/Controller/CivicrmController.php(80): Drupal\civicrm\Civicrm->invoke((Array:4))
#26 [internal function](): Drupal\civicrm\Controller\CivicrmController->main((Array:4), "queue:ajax:runNext")
#27 /code/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(123): call_user_func_array((Array:2), (Array:2))
#28 /code/core/lib/Drupal/Core/Render/Renderer.php(564): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}()
#29 /code/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(124): Drupal\Core\Render\Renderer->executeInRenderContext(Object(Drupal\Core\Render\RenderContext), Object(Closure))
#30 /code/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(97): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->wrapControllerExecutionInRenderContext((Array:2), (Array:2))
#31 /code/vendor/symfony/http-kernel/HttpKernel.php(158): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}()
#32 /code/vendor/symfony/http-kernel/HttpKernel.php(80): Symfony\Component\HttpKernel\HttpKernel->handleRaw(Object(Symfony\Component\HttpFoundation\Request), 1)
#33 /code/core/lib/Drupal/Core/StackMiddleware/Session.php(58): Symfony\Component\HttpKernel\HttpKernel->handle(Object(Symfony\Component\HttpFoundation\Request), 1, TRUE)
#34 /code/core/lib/Drupal/Core/StackMiddleware/KernelPreHandle.php(48): Drupal\Core\StackMiddleware\Session->handle(Object(Symfony\Component\HttpFoundation\Request), 1, TRUE)
#35 /code/core/modules/page_cache/src/StackMiddleware/PageCache.php(106): Drupal\Core\StackMiddleware\KernelPreHandle->handle(Object(Symfony\Component\HttpFoundation\Request), 1, TRUE)
#36 /code/core/modules/page_cache/src/StackMiddleware/PageCache.php(85): Drupal\page_cache\StackMiddleware\PageCache->pass(Object(Symfony\Component\HttpFoundation\Request), 1, TRUE)
#37 /code/core/lib/Drupal/Core/StackMiddleware/ReverseProxyMiddleware.php(48): Drupal\page_cache\StackMiddleware\PageCache->handle(Object(Symfony\Component\HttpFoundation\Request), 1, TRUE)
#38 /code/core/lib/Drupal/Core/StackMiddleware/NegotiationMiddleware.php(51): Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, TRUE)
#39 /code/vendor/stack/builder/src/Stack/StackedHttpKernel.php(23): Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, TRUE)
#40 /code/core/lib/Drupal/Core/DrupalKernel.php(708): Stack\StackedHttpKernel->handle(Object(Symfony\Component\HttpFoundation\Request), 1, TRUE)
#41 /code/index.php(19): Drupal\Core\DrupalKernel->handle(Object(Symfony\Component\HttpFoundation\Request))
#42 {main}
```5.51.0https://lab.civicrm.org/dev/core/-/issues/36615.51 regression: Permission issue on CRM_Mailing_BAO_Mailing::getGroupNames()2022-06-21T03:29:59ZJonGold5.51 regression: Permission issue on CRM_Mailing_BAO_Mailing::getGroupNames()I have an extension "unsubscribe group token". From the description:
> When using "ad-hoc" CiviMail, the "{mailing.group}" token evaluates to "Hidden Smart Group x". This evaluates identically to {mailing.group} on regular CiviMails, b...I have an extension "unsubscribe group token". From the description:
> When using "ad-hoc" CiviMail, the "{mailing.group}" token evaluates to "Hidden Smart Group x". This evaluates identically to {mailing.group} on regular CiviMails, but evaluates to the unsubscribe group name on ad-hoc mails. This extension provides a token to display your "Unsubscribe from {group}" text appropriately.
This token calls `CRM_Mailing_BAO_Mailing::getGroupNames()`, which @ufundo recently improved in #3463. However, this now requires the cron user to have the "Access CiviMail" permission.
Perhaps this should have always been the case. Perhaps I shouldn't be using `CRM_Mailing_BAO_Mailing::getGroupNames()` from an extension. I'm not convinced that https://github.com/civicrm/civicrm-core/pull/23570 is wrong to check API permissions - but it *is* a regression.
I'll submit a PR, and if it feels inappropriate to merge, that's cool.JonGoldJonGoldhttps://lab.civicrm.org/dev/core/-/issues/3660When upgrading an extension to use mixins, existing managed entities get deleted2022-07-05T20:39:56ZcolemanwWhen upgrading an extension to use mixins, existing managed entities get deletedI noticed while working on https://github.com/systopia/de.systopia.eck/pull/53/commits/66e1be252f3743aad35c3d028b628f3ea721948d that when switching from the master branch (without mixins) to the working branch (with mixins) that upon cle...I noticed while working on https://github.com/systopia/de.systopia.eck/pull/53/commits/66e1be252f3743aad35c3d028b628f3ea721948d that when switching from the master branch (without mixins) to the working branch (with mixins) that upon clearing caches all managed entities would be deleted and recreated.
In some cases that's not a big deal, but in this case it's quite a problem because of cascading deletes. The option group is a managed entity, so all user-created option values get destroyed in the process!
It must be an order-of-operations thing, where the managed entities get reconciled *before* the `info.xml` file gets scanned for mixins.tottentottenhttps://lab.civicrm.org/dev/core/-/issues/3658Custom data type grouping mismatch2022-07-08T04:55:03ZPradeep Nayakpradpnayak@gmail.comCustom data type grouping mismatchHi @colemanw
Have two instance, after running below api from the explorer get different output
$result = civicrm_api3('OptionValue', 'get', [
'sequential' => 1,
'return' => ["grouping", "label"],
'option_group_id' => "custom_dat...Hi @colemanw
Have two instance, after running below api from the explorer get different output
$result = civicrm_api3('OptionValue', 'get', [
'sequential' => 1,
'return' => ["grouping", "label"],
'option_group_id' => "custom_data_type",
]);
1. Fresh Install 5.50.2
![Screenshot_2022-06-16_at_13.21.01](/uploads/3c8bb4654229b36072759f1f31e11f25/Screenshot_2022-06-16_at_13.21.01.png)
2. Upgraded CiviCRM to 5.46 to 5.50.2
![Screenshot_2022-06-16_at_13.21.08](/uploads/deb4268e0f87d547d7eba38c2ce9d216/Screenshot_2022-06-16_at_13.21.08.png)
The grouping is missing on fresh install (can replicate on dmaster as well)
PR: https://github.com/civicrm/civicrm-core/pull/23336
The PR does have upgrade code but not generated mysql file for install.5.51.0colemanwcolemanwhttps://lab.civicrm.org/dev/core/-/issues/3657Crash importing contacts using 5.51 beta2022-06-17T02:37:42ZAndy ClarkCrash importing contacts using 5.51 betaUsing a simple test file on nightly 5.51 build of 16/6 contact import crashes when hitting 'Continue' on first screen. Screen shot attached. Same thing happened on previous nightly build. Test system was on 5.49.5 prior to that and went...Using a simple test file on nightly 5.51 build of 16/6 contact import crashes when hitting 'Continue' on first screen. Screen shot attached. Same thing happened on previous nightly build. Test system was on 5.49.5 prior to that and went through upgrade to 5.51 beta OK. System is Drupal 7 based. ![screenshot_20220616_094041](/uploads/6d06bb10cbb1a2827829e15fe96cc8a8/screenshot_20220616_094041.png)https://lab.civicrm.org/dev/core/-/issues/3656Unhide message admin UI core extension2024-03-02T05:03:27ZeileenUnhide message admin UI core extensionThe message admin UI core extension is intended as a replacement for the core Message templates screens - as part of our process of deprecating the core screens through extensions. Currently it ships hidden
I'm creating this to get the ...The message admin UI core extension is intended as a replacement for the core Message templates screens - as part of our process of deprecating the core screens through extensions. Currently it ships hidden
I'm creating this to get the steps to unhide the core message template admin UI.
The key reason it is still hidden is that it provides the opportunity to create translated versions of core workflow message templates - but core still does not send them out.
So we need to clarify what needs to be done to fill that gap - checklist first, then discussion
- [ ] Add a new `formatLocale` allowing this to be toggled separately to `tsLocale` - we already do this via a setting
- [ ] Add Subscriber for Translate to load a translation on `get` calls &, if available, change `formatLocale` and probably `tsLocale` (current use cases do not require the `tsLocale` change but that 'feels right')
- [ ] Language variant translations should load if available
- [ ] Add `setLanguage` to the `GenericTrait`
- [ ] Within `MessageTemplate.render` load the contact's preferred language & use `setLanguage` if not null.
- [ ] Within the message template UI ensure the `formattingLocale` is set when rendering
- [ ] Allow WorkflowMessageTemplates to be found from outside core - this is just a case of expanding the file search for them which we left tight to start with
- [ ] Message admin UI doesn't show the tokens available to it (even though they should be able to be determined.) It DOES show mailing ones - odd
**Rendering**
This is the key issue - when the `render` function is called in `sendMessageTemplate` (which is used to send out workflow emails within core) it needs to load the contact's language and call `setLanguage`. The responsibility for handling `setLanguage` is then on the `Translation` entity - which should ONLY take action if it can find a usable translation (this means we are not faffing with format localisation if we are not doing anything else) and it should change `formatLocale` (new) at that point and probably `tsLocale`.
Note that this means
- We are retrieving translations under `get` circumstances but they are not used in `where` or other clauses - the usage is for when the calling class does not know if a translation is in place
***Interaction with existing api code***
Currently the ` DAOActionTrait` implements `setLanguage`. When used the `I18nSubscriber` kicks in and calls `setLocale` which then
- if the site is NOT mutli-language it does NOTHING
- if the site IS multi-language then it either throws an exception if the language is not valid OR changes the `dbLocale` and the `tsLocale` if it is valid.
The only change that needs to be made here is to DO NOTHING rather than throw an exception if the language is not valid. There is otherwise no conflict with allowing the `I18nSubscriber` to make a decision based on the multilingual setting at the translation code from making a decision based on whether there is a translation for that entity availlable.
*** language variants***
We have some WMF code for this but basically if you speak Canadian French and only a French French translation exists you should get that, rather than the US English
***Format locale***
Format locale is a bit tricksy since what makes sense varies - eg. US site sends an email to NZ user would ideally
- format dates to `en_NZ`
- format currency to `en_US`
Why? well the US dates can be confusing to an NZ user so NZ dates are much better. However, if you choose `en_NZ` for the currency the refence to it being in NZD will be dropped - which is something an NZ user is likely to be looking for to make sure. By always using the `en_US` you can also predict in smarty if it is likely to be there and (eg. add `{if $currencyCode === 'USD'}$currencyCode{/if}
Since our date formatting is currently only marginally localised I think we can ignore it for now & understand that for 'language variants' then only load the translated one.https://lab.civicrm.org/dev/core/-/issues/3655Add pre/post hook for mailing trackable url2024-03-04T05:03:26ZyashodhaAdd pre/post hook for mailing trackable urlAdd pre/post hook support for mailing trackable urlAdd pre/post hook support for mailing trackable urlyashodhayashodhahttps://lab.civicrm.org/dev/financial/-/issues/199Additional Details section on Recurring Contribution template edit screen doe...2022-06-15T00:25:33ZDaveDAdditional Details section on Recurring Contribution template edit screen doesn't load anymoreSomewhat related to https://lab.civicrm.org/dev/financial/-/issues/197
If you look at screenshot 5 there, in the Additional Details section, note the spinning icon and "Loading...". This is coming from https://github.com/civicrm/civicrm...Somewhat related to https://lab.civicrm.org/dev/financial/-/issues/197
If you look at screenshot 5 there, in the Additional Details section, note the spinning icon and "Loading...". This is coming from https://github.com/civicrm/civicrm-core/pull/21471/files#diff-74e8144a14bfd4d8bf9cf4279c90cded28c43351c70a338d4502612d6e0f1c36R448 where it's trying to freeze the status field but there is no status field on the Additional Details section.5.51.0https://lab.civicrm.org/dev/core/-/issues/3654Feature request - import allow dedupe rule to be selected for contribution et...2023-01-31T04:30:02ZeileenFeature request - import allow dedupe rule to be selected for contribution etc importsPer last comment on https://lab.civicrm.org/dev/core/-/issues/3652Per last comment on https://lab.civicrm.org/dev/core/-/issues/3652https://lab.civicrm.org/dev/core/-/issues/3653Upgrading to 5.51 RC fails through the web interface, but works with `cv` and...2022-07-21T00:54:20ZjhungerfordUpgrading to 5.51 RC fails through the web interface, but works with `cv` and `drush`.Overview
----------------------------------------
I commented on this issue in the Town Square:
https://chat.civicrm.org/civicrm/pl/kpcoyh7hx3nzunx34ejbe8h8aw
Running the database upgrade to 5.51 RC by visiting `/civicrm/upgrade?reset=1...Overview
----------------------------------------
I commented on this issue in the Town Square:
https://chat.civicrm.org/civicrm/pl/kpcoyh7hx3nzunx34ejbe8h8aw
Running the database upgrade to 5.51 RC by visiting `/civicrm/upgrade?reset=1` fails with this message:
"DB Error: no such field"
`UPDATE civicrm_queue SET status = NULL WHERE name = 'CRM_Upgrade' [nativecode=1054 ** Unknown column 'status' in 'field list'`
Running the same upgrade with `cv` or `drush` works.
Reproduction steps
----------------------------------------
1. Install an older version of Civi (e.g. 5.49.5) on Drupal 7.88
2. Replace the Civi module with 5.51 RC (civicrm-RC-drupal.tar.gz ( 5.51.beta1-202206120040 ))
3. Visit /civicrm/upgrade?reset=1 in the web interface, and run the upgrade
4. Observe that it crashes with "DB Error: no such field"
5. Try again, to confirm that it consistently fails in the web interface
6. Run the upgrade with `cv upgrade:db` or drush, and observe that it works
Current behaviour
----------------------------------------
After the failure in the web interface, the Drupal log of our production clone contained a message with this string:
`UPDATE civicrm_queue SET status = NULL WHERE name = 'CRM_Upgrade' [nativecode=1054 ** Unknown column 'status' in 'field list'`
I didn't find any upgrade messages in the Drupal log of the clean install, I'm not sure why.
Expected behaviour
----------------------------------------
The web interface upgrade normally works the same as the command line process.
Environment information
----------------------------------------
* __Browser:__ Firefox 91.9.1esr (64-bit)
* __CiviCRM:__ from 5.49.5 to 5.51.beta1 (5.51.beta1-202206120040)
* __PHP:__ 7.3.33-1+0~20211119.91+debian10~1.gbp618351
* __CMS:__ Drupal 7.88
* __Database:__ mariadb Ver 15.1 Distrib 10.3.34-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2
* __Web Server:__ apache2 2.4.38-3+deb10u75.51.0https://lab.civicrm.org/dev/core/-/issues/3652I've just tested 5.51 (5.51.alpha1-202206100340), and "First Name" and "Last ...2023-01-31T04:29:42ZeileenI've just tested 5.51 (5.51.alpha1-202206100340), and "First Name" and "Last Name" seem to have disappeared from the import mapping options.Tracking for https://lab.civicrm.org/dev/core/-/issues/2858#note_75339
I've just tested 5.51 (5.51.alpha1-202206100340), and "First Name" and "Last Name" seem to have disappeared from the import mapping options. I note that in the code ...Tracking for https://lab.civicrm.org/dev/core/-/issues/2858#note_75339
I've just tested 5.51 (5.51.alpha1-202206100340), and "First Name" and "Last Name" seem to have disappeared from the import mapping options. I note that in the code (e.g. the mapFieldDataProvider function) they're now referred to as "first_name" and "last_name", but I can't find them in the field mapping dropdown (see screenshot).
![firstname_lastname_missing](/uploads/209a7788029206c0c7842bc5631aa854/firstname_lastname_missing.jpg)
The saved field mapping has reverted these fields to "- do not import -".
This prevents the import from running. I see the following error message when I try to click Continue to go past the Match Fields screen (step 2 of 4):
"Missing required contact matching fields. email(weight 15) first_name(weight 5) last_name(weight 10) (Sum of all weights should be greater than or equal to threshold: 30)."
I notice that "Organisation Name" is in the list, but Contact Type is definitely set to Individual in the first screen of the import process.5.51.0https://lab.civicrm.org/dev/core/-/issues/3651Activity import error output missing the column the error is about2022-06-14T03:15:51ZDaveDActivity import error output missing the column the error is aboute.g. Create an import file with an invalid date, something like this for Activities
```csv
Type,Subject,Contact,Date
2,something,13,2021-02-03 16
```
The error output will correctly flag the date column as invalid, but does not include...e.g. Create an import file with an invalid date, something like this for Activities
```csv
Type,Subject,Contact,Date
2,something,13,2021-02-03 16
```
The error output will correctly flag the date column as invalid, but does not include it in the output:
```csv
"Line Number",Reason,Type,Subject,Contact,Date
1,"Invalid value for field(s) : Activity Date",2,something,13
```5.51.0https://lab.civicrm.org/dev/core/-/issues/3650Copying a mailing results in thousands of entries in civicrm_mailing_group, r...2024-02-24T05:03:29ZJonGoldCopying a mailing results in thousands of entries in civicrm_mailing_group, rendering page unresponsiveI'll start by saying I haven't been able to replicate this - but I've seen it happen multiple times on Civi 5.39 to 5.41. I'm hoping someone else finds this and it rings a bell.
Sometimes, when a client goes to edit a mailing, the page...I'll start by saying I haven't been able to replicate this - but I've seen it happen multiple times on Civi 5.39 to 5.41. I'm hoping someone else finds this and it rings a bell.
Sometimes, when a client goes to edit a mailing, the page is either a) unresponsive, or b) doesn't allow saving. The "calculating recipients" SQL is running indefinitely. When investigating, there are tens of thousands of entries in `civicrm_mailing_group` for that mailing - the `entity_id`s repeat.
While I'm unable to replicate, this seems to happen when:
* Users copy an existing mailing rather than starting from a new mailing;
* In both cases, mailings were going out to 17-24 groups.https://lab.civicrm.org/dev/core/-/issues/3649Token renders in message preview and test message, not in final sent message2022-06-11T14:58:16ZChrisHardieToken renders in message preview and test message, not in final sent messageI'm on CiviCRM 5.35.1 with WordPress 5.7. Today I used CiviMail for the first time to send a mailing to a group of contacts. I used some tokens (custom contact fields) that rendered fine in my "preview HTML" view and in the generated mes...I'm on CiviCRM 5.35.1 with WordPress 5.7. Today I used CiviMail for the first time to send a mailing to a group of contacts. I used some tokens (custom contact fields) that rendered fine in my "preview HTML" view and in the generated message when I sent a test email to myself. However, in the final sent message that went out to recipients, they were not rendered and instead displayed as `{contact.custom_10}` and `{contact.custom_9}`. Other tokens were rendered just fine. The only thing that seems different about these two is that they are custom fields, and that they were in a bolded line of text.
Is this something I've missed in our configuration? Let me know if you need any other information. Thank you.https://lab.civicrm.org/dev/core/-/issues/3647Deleting C or Final mailing for A/B test makes report on A and B segments unv...2024-02-24T05:03:28ZlarsssandergreenDeleting C or Final mailing for A/B test makes report on A and B segments unviewableIf you create and A/B test, send the A & B segments and then delete the C or Final segment from Draft and Unscheduled Mailings, you can no longer view the mailing report for the A/B test. The mailing report URL just shows a blank page wi...If you create and A/B test, send the A & B segments and then delete the C or Final segment from Draft and Unscheduled Mailings, you can no longer view the mailing report for the A/B test. The mailing report URL just shows a blank page with header and footer.
I think it is probably not that uncommon an occurrence for someone to create an A/B test and later need to make a small change to the final version before sending it. That's easy enough to do by re-using one of the mailings and excluding the A and B mailings from your recipients. However, you will probably delete the C mailing from the unscheduled list at this point and then no longer be able to view the original A/B mailing report.
I'm testing on 5.28.4. This happens with both Mosaico and traditional mailings.