CiviCRM Core issueshttps://lab.civicrm.org/dev/core/-/issues2024-02-06T05:45:17Zhttps://lab.civicrm.org/dev/core/-/issues/4798Online contribution flow - fix for php8.x, notices, smarty32024-02-06T05:45:17ZeileenOnline contribution flow - fix for php8.x, notices, smarty3This is an oversight issue for fixing up the online contribution form in php8.x with a view to
1) resolving all the smarty & php layer notices, including in the message templateb
2) fixing the places where it doesn't work with Smarty3 (...This is an oversight issue for fixing up the online contribution form in php8.x with a view to
1) resolving all the smarty & php layer notices, including in the message templateb
2) fixing the places where it doesn't work with Smarty3 (in the online membership receipt & I think I fixed one in Confirm.tpl maybe)
3) fixing it to work with php8.2 (no undeclared properties)
4) fixing the underlying line item handling mess (we have multiple different ways of setting & retrieving the line itesm + a whole lot of code that does stuff to the line items but ultimately alters variables that get discarded. The impact of that I found tax bugs as soon as I started looking at the code.
5) fixing the online membership receipt such that it is previewable in MessageAdmin Ui, and does not require complex code to call it & have it render correctly (ie. `composeMessageArray` & the form layer both do a lot of assigning that should be unnecessary)
Much of this work has been done in small steps with a few in 5.68 & the bulk in 5.69. The goal is to try to resolve all the most important parts in 5.69 & then not do any more & encourage lots of rc testing on 5.69.
Areas to think about when testing
- data input with different currency separators
- non deductible amount with tax
- checking tax in separate payment & non separate payment scenarios (this actually has modest test cover but should be extended)
- checking the receipts
- checking the contribution recur amount is correct - when there is an autorenew membership + a contribution & it is not separate payment the amount should be equal to the tax-inclusive amount for the membership part of the contribution
Issues
**Must fix regressions**
- [x] Form layer issue on other amount js - probably regression in rc or master https://lab.civicrm.org/dev/core/-/issues/4795
- [x] Disabled options showing up https://github.com/civicrm/civicrm-core/pull/28356
- [x] Membership type not selected as a default https://github.com/civicrm/civicrm-core/pull/28361
- [ ] Is something weird happening with pledge? I couldn't figure out how to enable & disable & see it there & not there - UPDATE - I think this is a demo site data issue - https://github.com/civicrm/civicrm-core/pull/28497
- [x] Is the other Amount js right when there is NOT a membership? Need to compare to an older version... This might also be a demo site data issue per ^^
**Regression?Improvement?**
- [x] Change in line items for membership + contribution https://lab.civicrm.org/dev/core/-/issues/4814
**Extension issues**
- [x] A couple of extensions are interacting with the no-longer-in-use `_lineItem` undeclared property. This undeclared property also causes php8 fails. Declaring it would give a false sense of security but an alternative is to add a magic `__GET` & `__SET`. In my digging so far taxcalculator would need some small changes to adapt to this as the undeclared property is passed-by-reference but other extensions are likely to be OK with it. PR to add magic methods https://github.com/civicrm/civicrm-core/pull/28276 PR on taxcalculator https://lab.civicrm.org/extensions/taxcalculator/-/merge_requests/9
**Pre-existing issues**
These are not necessarily going to be fixed but the process of verifying them provides a goodly amount of rc testing. In some cases they are suspected fully or partially fixed already
*Tax*
- [ ] Long term issue on tax not applying to separate membership payments, in my testing pre-cleanup the tax-not-applied seemed to go beyond separate payments. This needs re-testing & a list of issues confirmed https://lab.civicrm.org/dev/financial/-/issues/154
- [ ] Aha - here is the general tax issue I hit https://lab.civicrm.org/dev/core/-/issues/4524 - needs verification
- [x] How should tax apply to 'Other amount' - is the amount entered inclusive? https://lab.civicrm.org/dev/core/-/issues/4806
- [ ] possible bug - needs verification https://lab.civicrm.org/dev/financial/-/issues/182
- [ ] Now that we have standardised on publicly supported `getLineItems()` & `setLineItems()` how is our hook support https://lab.civicrm.org/dev/core/-/issues/2796
*Localisation*
- [x] Non-English currency formatting messes with other amount field https://lab.civicrm.org/dev/core/-/issues/4802
*Page flow*
- [ ] Membership amount + contribution issues (erm what did I mean when I wrote this?)
- [ ] Data issue when email fails https://lab.civicrm.org/dev/core/-/issues/4540
- [ ] Data issue on payment fail https://github.com/civicrm/civicrm-core/pull/26120
- [ ] ContributionRecurID not always passed to payment processor https://lab.civicrm.org/dev/core/-/issues/4019
- [ ] The stretch goal - create order before adding payment https://lab.civicrm.org/dev/financial/-/issues/76
- [ ] Also the stretch goal https://lab.civicrm.org/dev/financial/-/issues/53
- [ ] Another meta issue of unknown status https://lab.civicrm.org/dev/core/-/issues/928
*Validation*
- [ ] Contribution + membership + quick config https://lab.civicrm.org/dev/core/-/issues/3314
*On Behalf*
- [ ] Profile bug?? https://lab.civicrm.org/dev/financial/-/issues/124
- [ ] Onbehalf hide for orgs https://lab.civicrm.org/dev/core/-/issues/4725
*Presentation issues*
- [ ] Notices on form - undefined property https://lab.civicrm.org/dev/core/-/issues/4760
- [x] Currency incorrect on thank you / confirm pages https://lab.civicrm.org/dev/core/-/issues/3381
- [ ] Probably the same https://lab.civicrm.org/dev/core/-/issues/3917
- [ ] Probably the same https://lab.civicrm.org/dev/core/-/issues/411
- [ ] Problems at form layer with auto-renew handling https://lab.civicrm.org/dev/core/-/issues/3963
- [ ] Frequency units translation issues https://lab.civicrm.org/dev/translation/-/issues/53
- [ ] On behalf confusing https://lab.civicrm.org/dev/core/-/issues/2266
- [ ] PCP notices https://lab.civicrm.org/dev/drupal/-/issues/159
- [ ] Translation missin ghttps://lab.civicrm.org/dev/translation/-/issues/27
- [ ] Show hide billing doesn't work https://lab.civicrm.org/dev/financial/-/issues/129
- [ ] Email appears twice https://lab.civicrm.org/dev/core/-/issues/3858
- [ ] Translation issue https://lab.civicrm.org/dev/core/-/issues/4071
- [ ] Possible bug https://lab.civicrm.org/dev/core/-/issues/4293
- [ ] Bug? replicable https://lab.civicrm.org/dev/core/-/issues/4488
- [ ] Upload image does not show https://lab.civicrm.org/dev/core/-/issues/4817
- [x] Undefined array key "honoreeProfileFields" also "title" in ThankYou.tpl.php that appeared when payment incorrectly recorded as $0 https://lab.civicrm.org/dev/core/-/issues/4863
- [x] Edit contribution has about a dozen undefined array keys for defaultContribution, displayLineItemFinancialType, getTaxDetails, pricesetFieldsCount, hookDiscount https://lab.civicrm.org/dev/core/-/issues/4864
*Membership calculation*
- [ ] Membership term usage issue https://lab.civicrm.org/dev/core/-/issues/3344
- [ ] Probably the same thing https://lab.civicrm.org/dev/core/-/issues/3810
- [ ] Probably the same thing https://lab.civicrm.org/dev/core/-/issues/3339
- [ ] Autorenew deep dive https://lab.civicrm.org/dev/financial/-/issues/128
- [ ] My read of the code suggests that if 2 membership types are configured to offer auto-renew it will only work for one
*Premiums*
- [x] notices on manage premiums https://lab.civicrm.org/dev/core/-/issues/4793
- [x] need api support for premiums https://github.com/civicrm/civicrm-core/pull/28261
- [x] Smarty notices on premiums in Online form flow https://lab.civicrm.org/dev/core/-/issues/4794
- [ ] pre-existing brokenness on deductible amount https://lab.civicrm.org/dev/core/-/issues/1083
- [ ] More non-deductible amount issues https://lab.civicrm.org/dev/core/-/issues/2414
- [ ] non-deductible amount misconfiguration https://lab.civicrm.org/dev/core/-/issues/3083 (note we could add a system check as a low-risk step)
*Manage contribution pages*
- [x] issue saving renew options https://github.com/civicrm/civicrm-core/pull/24997
- [ ] Validation lacking https://lab.civicrm.org/dev/core/-/issues/2846
- [ ] Widget issue https://lab.civicrm.org/dev/core/-/issues/20
*Membership receipts*
- [x ] Remove text version of receipt - we are moving to relying on generated text version rather than maintaining both
- [ ] Membership data wrong https://lab.civicrm.org/dev/core/-/issues/2354
- [ ] Credit card info sometimes missing https://lab.civicrm.org/dev/core/-/issues/661
- [ ] Receipt text inconsistently handled https://lab.civicrm.org/dev/user-interface/-/issues/13
- [ ] Issue on upsell https://lab.civicrm.org/dev/core/-/issues/4332
- [ ] Proposal - remove trxn_id https://lab.civicrm.org/dev/financial/-/issues/217
- [ ] Missing recur links https://lab.civicrm.org/dev/financial/-/issues/127
*Contribution receipts*
- [ ] Somewhat obscure bug on on-behalf profiles (should be addressed if we get as far as moving the handling of that to the WorkflowMessage class) https://lab.civicrm.org/dev/core/-/issues/4779
- [x] Remove text version of receipt - we are moving to relying on generated text version rather than maintaining both
- [ ] Duplicate of donor name https://lab.civicrm.org/dev/core/-/issues/2021
- [ ] Not sent for paypal express https://lab.civicrm.org/dev/financial/-/issues/13
- [ ] PDF file name issue https://lab.civicrm.org/dev/core/-/issues/3068
- [ ] Broken links in contribution issue https://lab.civicrm.org/dev/core/-/issues/1195
- [ ] Likely fixed bug https://lab.civicrm.org/dev/core/-/issues/4185
- [ ] Checksum wiht on-behalf of https://lab.civicrm.org/dev/core/-/issues/4265
*Wordpress issue?*
- [ ] https://lab.civicrm.org/dev/wordpress/-/issues/120
Feature request
- [ ] Configurable confirm button - more than one request - like events - https://lab.civicrm.org/dev/user-interface/-/issues/10 https://lab.civicrm.org/dev/core/-/issues/4324https://lab.civicrm.org/dev/core/-/issues/3441When generating emails from Search results, activity is recorded as a Print P...2023-03-06T20:39:37ZStoobWhen generating emails from Search results, activity is recorded as a Print PDF DocumentSteps to reproduce:
1. use Search Kit to generate results as _Contributions_
2. choose to send emails receipts
3. emails are sent, but activity is recorded "Print/Merge Document" rather than Type "Email".
#searchkitSteps to reproduce:
1. use Search Kit to generate results as _Contributions_
2. choose to send emails receipts
3. emails are sent, but activity is recorded "Print/Merge Document" rather than Type "Email".
#searchkithttps://lab.civicrm.org/dev/core/-/issues/3440Check for matching contact on contact add form sends hardcoded fields to dupl...2023-07-06T07:25:40ZdarrickCheck for matching contact on contact add form sends hardcoded fields to duplicatecheck api callOverview
----------------------------------------
If a custom Supervised rule is created using any field not in ['first_name', 'last_name', 'nick_name', 'household_name', 'organization_name', 'email'] then clicking on the **Check for Mat...Overview
----------------------------------------
If a custom Supervised rule is created using any field not in ['first_name', 'last_name', 'nick_name', 'household_name', 'organization_name', 'email'] then clicking on the **Check for Matching Contact** button will return no results.
Reproduction steps
----------------------------------------
1. Click on **Contacts -> Find and Merge Duplicate Contacts**.
2. Click on **Add Individual Rule**
3. Add a rule with field phone, weight 10, threshold 10
4. Click on **Change rule** and set to **Supervised**
5. Click on **Contacts -> New Individual**
6. Add a contact with First Name: Bob, Last Name: Dobbs and phone 666.666.6666
7. Save the contact.
8. Click on **Contacts -> New Individual**
9. Add a contact with First Name: Bob, Last Name: Dobbs and phone 666.666.6666
10. Click on **Check for Matching Contact**
Current behaviour
----------------------------------------
A popup displays "Similar contact if found" after entering the First Name and after entering the Last Name in step 9.
After entering the duplicate phone number nothing happens.
After clicking on **Check for Matching Contact** nothing happens.
Expected behavior
----------------------------------------
After entering either First Name or Last Name nothing should happen unless the entered field is included in the Supervised rule.
A popup displaying "Similar contact if found" should happen after the phone number is entered and also after clicking on **Check for Matching Contact**
Comments
----------------------------------------
I ran across this while looking to see if I could fix any other outstanding bugs related dedupe. Was working on this one: (https://lab.civicrm.org/dev/core/-/issues/2966) I wasn't able to reproduce their issue.
It may still be useful to hard code those fields so by default the form always matches on name or email when entering those fields but then the additional fields will also be searched when added using the custom rule. As any fields not needed for the custom rule will just be ignored.https://lab.civicrm.org/dev/core/-/issues/4950Acivity tab timing out after 5.70 upgrade on large DB2024-02-08T19:14:09ZElliott EgglestonAcivity tab timing out after 5.70 upgrade on large DBThe admin_ui version of the activity tab is generating queries for some CIDs that don't return in a reasonable amount of time. EXPLAIN shows that filesort is being used for the first subquery.
One example query, generated for a contact ...The admin_ui version of the activity tab is generating queries for some CIDs that don't return in a reasonable amount of time. EXPLAIN shows that filesort is being used for the first subquery.
One example query, generated for a contact with only 18 activities: https://phabricator.wikimedia.org/P55961
Wikimedia's tracking task, with the SQL query explain: https://phabricator.wikimedia.org/T356269
Oddly, I WAS able to load the activities tab for my own user which has 700+ activities, so it's not just related to the number of activities on a contact.
If I just roll back 04078d359e298a5a02ea274404dcdb8af83225dd it restores the previous activity tab and I can load the CID's activity tab fine with no runaway query generated.5.70.0colemanwcolemanwhttps://lab.civicrm.org/dev/core/-/issues/4745Joomla: Fails to download extension feed2023-11-09T23:56:48ZtottenJoomla: Fails to download extension feedOverview
----------------------------------------
There appears to be a Guzzle conflict when fetching the extension feed on Joomla.
Reproduction steps
----------------------------------------
1. Install CiviCRM on Joomla
1. Navigate to...Overview
----------------------------------------
There appears to be a Guzzle conflict when fetching the extension feed on Joomla.
Reproduction steps
----------------------------------------
1. Install CiviCRM on Joomla
1. Navigate to dashboard
Current behaviour
----------------------------------------
The first pageload fails with:
![Screen_Shot_2023-11-01_at_4.20.15_PM](/uploads/bfab953fd770a8c09af0658b11529f04/Screen_Shot_2023-11-01_at_4.20.15_PM.png)
The NACK is cached for a while, so you won't immediately see the problem again. But we should expect it to recur.
Expected behaviour
----------------------------------------
No crash.
Environment information
----------------------------------------
The "hydra sites" (https://test.civicrm.org/view/Sites/job/hydra-sites/) are clean CMS installations. (They have the Civi source code preloaded -- but the installer hasn't run.) The error was observed on the hydra site for Joomla with CiviCRM `5.67.beta2` circa Nov 1, 2023.
* __Browser:__ Firefox
* __CiviCRM:__ 5.67.beta2
* __PHP:__ 7.4
* __CMS:__ Joomla
* __Database:__ 5.7
* __Web Server:__ Apachehttps://lab.civicrm.org/dev/core/-/issues/4452Message Admin doesn't show which message templates have been edited2023-09-18T12:15:28ZlarsssandergreenMessage Admin doesn't show which message templates have been edited~~It looks like Message Admin is now enabled by default, but~~ I've noticed that there is no indication in Message Admin of which templates have been edited or not. This is pretty important info for admins and should be available in the ...~~It looks like Message Admin is now enabled by default, but~~ I've noticed that there is no indication in Message Admin of which templates have been edited or not. This is pretty important info for admins and should be available in the UI.
Compare the old - very clear which has been edited:
![image](/uploads/43963f9cbbf0e08c5f1679056e4a2c35/image.png)
to the new - no indication which has been edited:
![image](/uploads/4a394422a26c4a1536306e96d3c77a7c/image.png)
I can't see anything on the edit page to indicate this either.
Edit: There is also no longer any way to revert a message to the default, other than by manually copying and pasting the template content. I think having the "Revert to default" link is useful, if you made a small change and have decided you no longer need it because there has been an upgrade.
~~Also, I guess the drafting process isn't implemented yet (or not accessible from the UI), so maybe it would make sense to hide the Draft column until that's ready.~~https://lab.civicrm.org/dev/core/-/issues/4287PHP 8 - Undefined variable warnings from Smarty appear in email notifications2023-11-14T01:40:49ZjasonhildebrandPHP 8 - Undefined variable warnings from Smarty appear in email notificationsOverview
----------------------------------------
We recently upgraded Civi to 5.60 and PHP 8 (was previously PHP 7.x). Our site uses Drupal 7.
Since the upgrade, we are seeing errors such as ```Undefined array key "phone_type"``` appe...Overview
----------------------------------------
We recently upgraded Civi to 5.60 and PHP 8 (was previously PHP 7.x). Our site uses Drupal 7.
Since the upgrade, we are seeing errors such as ```Undefined array key "phone_type"``` appearing in our email notifications to event participants.
![screenshot](/uploads/4cee73442124f9c2a3b1158e8925c1e7/screenshot.png)
We have taken action to suppress errors and warnings in Drupal, but this does not appear to help when smarty is used to render email notifications.
Expected behaviour
----------------------------------------
I would expect these warnings not to appear, or to be suppressible with setting, so that they can be turned off in production.
Workaround
-----------
As a workaround, I added the 3 lines marked below to civicrm/CRM/Core/TokenSmarty.php in order to suppress the warnings before rendering, then restore the error_reporting to the original setting after rendering.
Perhaps this kind of approach could be added to Civi with a configuration setting to turn these messages on/off.
```
// Evaluate/render templates
try {
if ($useSmarty) {
$orig_reporting = error_reporting(); // ADDED
error_reporting(0); // ADDED
CRM_Core_Smarty::singleton()->pushScope($smartyAssigns);
}
$tokenProcessor->evaluate();
foreach ($messages as $messageId => $ign) {
foreach ($tokenProcessor->getRows() as $row) {
$result[$messageId] = $row->render($messageId);
}
}
}
finally {
if ($useSmarty) {
CRM_Core_Smarty::singleton()->popScope();
error_reporting($orig_reporting); // ADDED
}
}
```https://lab.civicrm.org/dev/core/-/issues/4191Export with a field that has a small DB size but long labels results in fatal2023-06-15T21:16:14ZbgmExport with a field that has a small DB size but long labels results in fatalTo reproduce:
- Create a new custom field of type=alphanum, widget=select list, size=2 (on dmaster, you can edit "Constituent Information > Marital Status" to have a width of 2)
- Enter some field options that are more than 2 characters...To reproduce:
- Create a new custom field of type=alphanum, widget=select list, size=2 (on dmaster, you can edit "Constituent Information > Marital Status" to have a width of 2)
- Enter some field options that are more than 2 characters
Then go to Advanced Search:
- click submit to search,
- export all results
- select fields to export: marital status (or the field you created)
It will cause CiviCRM to fatal because it creates a temp table with a `varchar(4)` but it tries to store the field labels.
It works on CiviCRM 5.55 but not on CiviCRM 5.60 RC.https://lab.civicrm.org/dev/core/-/issues/3824Possible for historic scheduled reminder to be sent multiple times if cron jo...2022-11-28T08:14:03ZAdam WoodPossible for historic scheduled reminder to be sent multiple times if cron job does not complete properlyWhen the `Send Scheduled Reminders` job runs, the process fails with errors like the following (reported by our Cron Daemon):
```
PHP Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 245760 bytes) in /ho...When the `Send Scheduled Reminders` job runs, the process fails with errors like the following (reported by our Cron Daemon):
```
PHP Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 245760 bytes) in /home/cses_org_uk/public_html/administrator/components/com_civicrm/civicrm/CRM/Utils/Cache/SqlGroup.php on line 184
```
If attempting to invoke the job manually from the scheduled jobs admin page, we get an HTTP 500 error and the following entries in the error_log:
```
[Thu Aug 25 19:51:26.801469 2022] [fcgid:warn] [pid 27274] [client 79.69.229.183:51994] mod_fcgid: read data timeout in 301 seconds, referer: https://cses.org.uk/administrator/?option=com_civicrm&task=civicrm/admin/job&action=view&reset=1&context=joblog&id=9
[Thu Aug 25 19:51:26.819905 2022] [core:error] [pid 27274] [client 79.69.229.183:51994] End of script output before headers: index.php, referer: https://cses.org.uk/administrator/?option=com_civicrm&task=civicrm/admin/job&action=view&reset=1&context=joblog&id=9
```
We can see that the job is starting but never finishing from the log:
![image](/uploads/f30bd5bc2f0df34579eb3fa9247545b9/image.png)
_**The scheduled reminders do seem to be sending successfully, however. Other cron jobs also seem to complete successfully.**_
This issue seemed to start recently with an unplanned, emergency server migration. It's a virtual server so notionally everything is the same, but there may be some other factor at play here (CiviCRM upgrade, change to one of the reminder templates etc). I have double-checked PHP resource limits (256M memory, 300 second timeout i.e. more than enough) and MySQL settings and can't find anything amiss.
The error occurs at the following line in `SQLGroup.php`:
```php
private function reobjectify($value) {
return is_object($value) ? unserialize(serialize($value)) : $value; // This line fails
}
```
It seems as though `serialize()` is failing: whatever value is at fault, it was previously `unserialize()`d successfully to read it out of the cache in the first place. What does this function actually do - why the round trip??
I attempted to debug with some judicious `error_log()` immediately before and after this line to see which cache value was causing the memory leak, but to no avail. No error_log was written before the crash (although many previous successful calls to the function were logged).
This is on CiviCRM 5.50.4. I will attempt an upgrade soon to see if this fixes it, and will continue to investigate / try to narrow down / work out how to reproduce.