API call Contribution.repeattransaction fails with recurring contributions that have a one-to-one custom group record attached
Overview
Re: https://chat.civicrm.org/civicrm/pl/i8fht46isjn5uj4dbdosrqpqmr
The API call Contribution.repeattransaction appears to write a custom group record twice when completing a transaction. On the second attempt, a database error caused by the clashing unique key is generated. The transaction appears to end up being recorded correctly, however the fatal error means that processes which call this function repeatedly end up not completing. Our use case involves finding all Eway transactions whose next scheduled contribution date is today, and then calling Contribution.repeattransaction for each one.
Reproduction steps
- Ensure a one-to-one custom group exists against contributions.
- Ensure a recurring contribution exists with a record in that custom group against the transaction.
- Run
drush cvapi Contribution.repeattransaction contribution_status_id=Completed total_amount=<some value> contribution_recur_id=<contribution_recur_id in previous step> trxn_id=<random unique number>
Current behaviour
Failure message appears: DB Error: already exists
; process terminates. Record appears to be correctly produced in CiviCRM.
Log details:
Dec 07 12:40:11 [error] $Fatal Error Details = Array
(
[callback] => Array
(
[0] => CRM_Core_Error
[1] => exceptionHandler
)
[code] => -5
[message] => DB Error: already exists
[mode] => 16
[debug_info] => INSERT INTO civicrm_value_contribution_custom_group_1 ( `custom_field_1`,`custom_field_2`,`custom_field_3`,`custom_field_4`,`entity_id` ) VALUES ( 1,'Some Text','','22943426',123456 ) [nativecode=1062 ** Duplicate entry '123456' for key 'unique_entity_id']
[type] => DB_Error
[user_info] => INSERT INTO civicrm_value_contribution_custom_group_1 ( `custom_field_1`,`custom_field_2`,`custom_field_3`,`custom_field_4`,`entity_id` ) VALUES ( 1,'Some Text','','22943426',123456 ) [nativecode=1062 ** Duplicate entry '123456' for key 'unique_entity_id']
[to_string] => [db_error: message="DB Error: already exists" code=-5 mode=callback callback=CRM_Core_Error::exceptionHandler prefix="" info="INSERT INTO civicrm_value_contribution_custom_group_1 ( `custom_field_1`,`custom_field_2`,`custom_field_3`,`custom_field_4`,`entity_id` ) VALUES ( 1,'Some Text','','22943426',123456 ) [nativecode=1062 ** Duplicate entry '123456' for key 'unique_entity_id']"]
)
Dec 07 12:40:11 [debug] $backTrace = #0 /path/to/civi/root/civicrm/CRM/Core/Error.php(942): CRM_Core_Error::backtrace("backTrace", TRUE)
#1 /path/to/civi/root/civicrm/vendor/pear/pear-core-minimal/src/PEAR.php(922): CRM_Core_Error::exceptionHandler(Object(DB_Error))
#2 /path/to/civi/root/civicrm/vendor/pear/db/DB.php(997): PEAR_Error->__construct("DB Error: already exists", -5, 16, (Array:2), "INSERT INTO civicrm_value_contribution_custom_group_1 ( `hide_from_slider_182`,`custom_...")
#3 /path/to/civi/root/civicrm/vendor/pear/pear-core-minimal/src/PEAR.php(575): DB_Error->__construct(-5, 16, (Array:2), "INSERT INTO civicrm_value_contribution_custom_group_1 ( `custom_field_1`,`custom_...")
#4 /path/to/civi/root/civicrm/vendor/pear/pear-core-minimal/src/PEAR.php(223): PEAR->_raiseError(Object(DB_mysqli), NULL, -5, 16, (Array:2), "INSERT INTO civicrm_value_contribution_custom_group_1 ( `custom_field_1`,`custom_...", "DB_Error", TRUE)
#5 /path/to/civi/root/civicrm/vendor/pear/db/DB/common.php(1928): PEAR->__call("raiseError", (Array:7))
#6 /path/to/civi/root/civicrm/vendor/pear/db/DB/mysqli.php(936): DB_common->raiseError(-5, NULL, NULL, "INSERT INTO civicrm_value_contribution_custom_group_1 ( `custom_field_1`,`custom_...", "1062 ** Duplicate entry '123456' for key 'unique_entity_id'")
#7 /path/to/civi/root/civicrm/vendor/pear/db/DB/mysqli.php(406): DB_mysqli->mysqliRaiseError()
#8 /path/to/civi/root/civicrm/vendor/pear/db/DB/common.php(1234): DB_mysqli->simpleQuery("INSERT INTO civicrm_value_contribution_custom_group_1 ( `custom_field_1`,`custom_...")
#9 /path/to/civi/root/civicrm/packages/DB/DataObject.php(2696): DB_common->query("INSERT INTO civicrm_value_contribution_custom_group_1 ( `custom_field_1`,`custom_...")
#10 /path/to/civi/root/civicrm/packages/DB/DataObject.php(1829): DB_DataObject->_query("INSERT INTO civicrm_value_contribution_custom_group_1 ( `custom_field_1`,`custom_...")
#11 /path/to/civi/root/civicrm/CRM/Core/DAO.php(468): DB_DataObject->query("INSERT INTO civicrm_value_contribution_custom_group_1 ( `custom_field_1`,`custom_...")
#12 /path/to/civi/root/civicrm/CRM/Core/DAO.php(1621): CRM_Core_DAO->query("INSERT INTO civicrm_value_contribution_custom_group_1 ( `custom_field_1`,`custom_...", TRUE)
#13 /path/to/civi/root/civicrm/CRM/Core/BAO/CustomValueTable.php(271): CRM_Core_DAO::executeQuery("INSERT INTO civicrm_value_contribution_custom_group_1 ( `custom_field_1`,`custom_...", (Array:5))
#14 /path/to/civi/root/civicrm/CRM/Core/BAO/CustomValueTable.php(391): CRM_Core_BAO_CustomValueTable::create((Array:2), "create")
#15 /path/to/civi/root/civicrm/CRM/Core/BAO/CustomValueTable.php(411): CRM_Core_BAO_CustomValueTable::store((Array:5), "civicrm_contribution", 123456, "create")
#16 /path/to/civi/root/civicrm/CRM/Core/DAO.php(2022): CRM_Core_BAO_CustomValueTable::postProcess((Array:5), "civicrm_contribution", 123456, "Contribution", "create")
#17 /path/to/civi/root/civicrm/CRM/Contribute/BAO/Contribution.php(2394): CRM_Core_DAO->copyCustomFields(123455, 123456)
#18 /path/to/civi/root/civicrm/CRM/Contribute/BAO/Contribution.php(4000): CRM_Contribute_BAO_Contribution::repeatTransaction((Array:10), (Array:18))
#19 /path/to/civi/root/civicrm/api/v3/Contribution.php(687): CRM_Contribute_BAO_Contribution::completeOrder((Array:10), "2269", NULL, NULL)
#20 /path/to/civi/root/civicrm/api/v3/Contribution.php(635): _ipn_process_transaction((Array:7), Object(CRM_Contribute_BAO_Contribution), (Array:10), (Array:10))
#21 /path/to/civi/root/civicrm/Civi/API/Provider/MagicFunctionProvider.php(89): civicrm_api3_contribution_repeattransaction((Array:7))
#22 /path/to/civi/root/civicrm/Civi/API/Kernel.php(149): Civi\API\Provider\MagicFunctionProvider->invoke((Array:8))
#23 /path/to/civi/root/civicrm/Civi/API/Kernel.php(81): Civi\API\Kernel->runRequest((Array:8))
#24 /path/to/civi/root/civicrm/api/api.php(132): Civi\API\Kernel->runSafe("contribution", "repeattransaction", (Array:5))
#25 /path/to/civi/root/civicrm/drupal/drush/civicrm.drush.inc(1581): civicrm_api3("Contribution", "repeattransaction", Array:5))
#26 /usr/local/src/drush/includes/command.inc(422): drush_civicrm_api("Contribution.repeattransaction", "contribution_status_id=Completed", "total_amount=11.00", "contribution_recur_id=22663", "trxn_id=aa11bb22cc33dd47")
#27 /usr/local/src/drush/includes/command.inc(231): _drush_invoke_hooks((Array:38), (Array:5))
#28 /usr/local/src/drush/includes/command.inc(199): drush_command("Contribution.repeattransaction", "contribution_status_id=Completed", "total_amount=11.00", "contribution_recur_id=22663", "trxn_id=aa11bb22cc33dd47")
#29 /usr/local/src/drush/lib/Drush/Boot/BaseBoot.php(67): drush_dispatch((Array:38))
#30 /usr/local/src/drush/includes/preflight.inc(67): Drush\Boot\BaseBoot->bootstrap_and_dispatch()
#31 /usr/local/src/drush/drush.php(12): drush_main()
#32 {main}
Expected behaviour
Fatal error not raised; no attempt to write the exact same record twice.
I have mitigated this behaviour with the following patch, but this is clearly a temporary solution and the real root cause needs to be found:
diff --git a/CRM/Core/BAO/CustomValueTable.php b/CRM/Core/BAO/CustomValueTable.php
index 8556c4ec21..6773b994c4 100644
--- a/CRM/Core/BAO/CustomValueTable.php
+++ b/CRM/Core/BAO/CustomValueTable.php
@@ -55,7 +55,7 @@ class CRM_Core_BAO_CustomValueTable {
$hookOP = 'edit';
}
else {
- $sqlOP = "INSERT INTO $tableName ";
+ $sqlOP = "INSERT IGNORE INTO $tableName ";
$where = NULL;
$hookOP = 'create';
}
Environment information
- Browser: Firefox 94.0.2
- CiviCRM: 5.43.2
- PHP: 7.3
- CMS: Drupal 7.82
- Database: MariaDB 10.4.21
- Web Server: Nginx 1.10.3