CiviCRM Core issueshttps://lab.civicrm.org/dev/core/-/issues2023-09-24T22:50:10Zhttps://lab.civicrm.org/dev/core/-/issues/2890Contribution api - get paid, balance2023-09-24T22:50:10ZeileenContribution api - get paid, balance@colemanw just wanted to put this on your radar - being able to get
contribution.paid_amount and
contribution.balance
would be hugely use at the api level , search kit AND the template / token level.
We currently have a php function ...@colemanw just wanted to put this on your radar - being able to get
contribution.paid_amount and
contribution.balance
would be hugely use at the api level , search kit AND the template / token level.
We currently have a php function & what it does to get paid_amount is this...
I suspect there are some challenges in that it would have to be a subquery which would have performance limits but it would have a lot of application
```
$sql = "SELECT SUM(ft.total_amount) FROM civicrm_financial_trxn ft
INNER JOIN civicrm_entity_financial_trxn eft ON (eft.financial_trxn_id = ft.id AND eft.entity_table = 'civicrm_contribution')
WHERE eft.entity_id = %1 AND ft.is_payment = 1 AND ft.status_id IN (%2) ";
return (float) CRM_Core_DAO::singleValueQuery($sql, [
1 => [$contributionID, 'Integer'],
// status IDs are 'completed' & 'refunded'
2 => [implode(',', $statusIDs), 'CommaSeparatedIntegers'],
]);
```https://lab.civicrm.org/dev/core/-/issues/587Permissions on GroupContact API calls seem wrong2023-09-24T16:09:28ZJonGoldPermissions on GroupContact API calls seem wrongI got a support request from a user who didn't have "Edit All Contacts" permissions stating that they couldn't remove someone from a group. Sure enough, [edit all contacts](https://lab.civicrm.org/dev/core/blob/master/CRM/Core/Permissio...I got a support request from a user who didn't have "Edit All Contacts" permissions stating that they couldn't remove someone from a group. Sure enough, [edit all contacts](https://lab.civicrm.org/dev/core/blob/master/CRM/Core/Permission.php#L1229) is the necessary permission. However, editing/removing tags just requires "access CiviCRM".
Does this seem correct to folks? Is it to prevent someone escalating their ACL permissions? If so, it feels like we need a different permission, and predates more nuanced solutions such as [Group Protect](https://github.com/CiviCooP/org.civicoop.groupprotect). The only other entity that needs such high permissions is Relationship - that also seems wrong.
I propose that we add both GroupContact and Relationship entities to the `_civicrm_api3_check_edit_permissions()` function. If you can edit the contact, you can edit their groups/relationships. If someone gives this a "Concept: Approved" I'll work on the PR.JonGoldJonGoldhttps://lab.civicrm.org/dev/core/-/issues/2841Undefined index isCaseActivity when creating an activity2023-09-24T05:03:25ZDaveDUndefined index isCaseActivity when creating an activityThis is probably from one of the message template rearrangings. For the email it sends to the activity assignee.This is probably from one of the message template rearrangings. For the email it sends to the activity assignee.https://lab.civicrm.org/dev/core/-/issues/2818Context gets lost when sending an email from manage case so it redirects some...2023-09-24T05:03:25ZDaveDContext gets lost when sending an email from manage case so it redirects somewhere else afterI think this is the same cause as https://lab.civicrm.org/dev/core/-/issues/2318 where something about `$this->_single` went missing. It should redirect you back to manage case.
1. Turn off popups at administer - customize - display pre...I think this is the same cause as https://lab.civicrm.org/dev/core/-/issues/2318 where something about `$this->_single` went missing. It should redirect you back to manage case.
1. Turn off popups at administer - customize - display prefs (or when opening the link in step 3 open in a new tab).
1. Create case
1. In the roles section use the email icon to send an email to e.g. the client
1. After sending, it goes somewhere else instead of manage case. Sometimes it will take you to the contact record which at least is related, sometimes it goes to /civicrm/dashboard, sometimes somewhere else.https://lab.civicrm.org/dev/core/-/issues/4621Some links don't work in Firefox with uBlock Origin2023-09-23T10:15:05ZresgaSome links don't work in Firefox with uBlock Origin## Overview
If I surf my CiviCRM installation with the Firefox extension [uBlock Origin](https://github.com/gorhill/uBlock) enabled and try to view a member, I get redirected to the front page, and this error message in the upper right ...## Overview
If I surf my CiviCRM installation with the Firefox extension [uBlock Origin](https://github.com/gorhill/uBlock) enabled and try to view a member, I get redirected to the front page, and this error message in the upper right corner:
> Error
> We could not find a contact id.
I managed to get the member to show via another method (I have a custom Drupal list) but when I click on "Contributions" or "Memberships" I get these errors:
> Error
> Could not find valid value for cid
> Network Error
> Unable to reach the server. Please refresh this page in your browser and try again.
If I disable uBlock Origin, the problem goes away.
In the browser address field, slashes and question marks are transformed, to look like this:
https://mysite.ddev.site/civicrm/contact/view%3Fcid%3D5773%26reset%3D1?selectedChild=summary
Could that be the cause? I believe this behaviour started after updating from version 5.43.2 to 5.51.1.
## Reproduction steps
1. Install and enable uBlock Origin in Firefox
1. Click on **Search -> Find Members**
1. Click on **Search** button
1. Click on a member
1. Get an error "**Error: > We could not find a contact id.**"
## Environment information
* __Browser:__ _Firefox 117.0.1_
* __Browser extension:__ _uBlock Origin_
* __CiviCRM:__ _Master/5.60.0_
* __PHP:__ _7.4_
* __CMS:__ _Drupal 7.97_
* __Database:__ _MariaDB 10.4_
* __Web Server:__ _Apache 2.4 AND Nginx 1.16_
## Possible solution
Would not HTML-encoding the links allow it to work with uBlock Origin?https://lab.civicrm.org/dev/core/-/issues/4501Redis performance issue on delete contact2023-09-23T05:08:34ZeileenRedis performance issue on delete contactWe have a process where we merge contacts and then delete the deleted contacts after a period of time.
However, we have more or less never run the delete deleted contacts script because it is too slow. I dug into it today and found that...We have a process where we merge contacts and then delete the deleted contacts after a period of time.
However, we have more or less never run the delete deleted contacts script because it is too slow. I dug into it today and found that
- on staging it takes 15 seconds to delete 500 contacts
- on production it takes 6-10 minutes to delete 500 contacts
I've spent most of the day digging into why & determined that the queries run are identical & timings are similary. However, on production each time this line of code runs ` Civi::service('prevnext')->deleteItem($id);` it takes a bit over 1 second. This is not the case on staging because there are no users populating the prevnext cache with searches. Hence I have diagnosed that the problem is having users & the solution is to lock their accounts.
More specifically the issue is that the code is going through all the Redis keys to remove the contact - which seems to be inefficient.
![image](/uploads/701ae2d41d47d0eae49ee118f80d4dc4/image.png)
I did wonder if a quick-fix would be to only call `deleteItem` if the contact is not already deleted (which they are in our use case) - I would need to change [the find to fetch here](https://github.com/civicrm/civicrm-core/blob/aef17937a6d1bf00d2ca446bf3c5fc81644b3b92/CRM/Contact/BAO/Contact.php#L907-L911) I think...
Alternatively there is probably some option around queuing the cache clear to happen at the end. However, I think that 1 second + delay is actually not great for users either - e.g when deduping a bunch of contacts than having each form submit take that bit longer would add up.
We are on the cusp of getting `coworker` going so pushing something to a queue to clear out caches might be an option.https://lab.civicrm.org/dev/core/-/issues/2831It is possible to break Scheduled Reminder by adding too may entity types2023-09-23T05:03:26ZDavid HayesIt is possible to break Scheduled Reminder by adding too may entity types# How to reproduce
1. Add about 100 activity types
1. Create a Scheduled Reminder that triggers on Activity
1. Select most or all activity types
1. Save Scheduled Reminder
1. Edit the recently created Scheduled Reminder, notice not all A...# How to reproduce
1. Add about 100 activity types
1. Create a Scheduled Reminder that triggers on Activity
1. Select most or all activity types
1. Save Scheduled Reminder
1. Edit the recently created Scheduled Reminder, notice not all Activity Types are selected anymore
1. Scheduled Reminders will begin failing
-Checking the Scheduled Job logs, the error will be `Finished execution of Send Scheduled Reminders with result: Failure, Error message: Failed encoding non-numeric value (#selectedValues)`
# The cause
The selected activity types are stored in the `civicrm_action_schedule` table in the `entity_value` column. This column only supports 255 characters. Adding any more characters than this causes the value to be truncated with an elipsis "..." at the end. This error originates from `CRM_Activity_ActionMapping::createQuery` where the code places the trucated list of Activity Types into a where clause in a query. The "..." after the last value cause an integer check to fail.
# How to fix
Convert this column to something longer? TEXT maybe? Happy to make a PR, but would like to know the preferred path.https://lab.civicrm.org/dev/core/-/issues/645Use site email domain in place of bounce email domain for automated messages2023-09-23T05:03:26ZnishantBUse site email domain in place of bounce email domain for automated messagesHello there!!
We encountered a scenario where we can whitelist only one email domain to send emails. We want to use email domain (eg: bouncedomain.com) for bounce processing and a different one (sitedomain.com) for rest of the emails wh...Hello there!!
We encountered a scenario where we can whitelist only one email domain to send emails. We want to use email domain (eg: bouncedomain.com) for bounce processing and a different one (sitedomain.com) for rest of the emails which works fine except that do-not-reply emails use the email domain set for bounce processing (bouncedomain.com) because of which AWS SES prevent those emails to be sent.
**Current behaviour:**
1. Regular emails: example@sitedomain.com
2. Bounce emails: bounce@bouncedomain.com
3. Do-not-reply emails: do-not-reply@bouncedomain.com
**Expected behaviour:**
1. Regular emails: example@sitedomain.com
2. Bounce emails: bounce@bouncedomain.com
3. Do-not-reply emails: do-not-reply@sitedomain.com
Is there already a configuration to set that or would it be a good idea to add a field where we can set the email domain for do-not-reply emails ?
Thanks!https://lab.civicrm.org/dev/core/-/issues/2813"Soft Credit Only" Contribution export screen and exported data don't match2023-09-22T05:03:27Zalicefrumin"Soft Credit Only" Contribution export screen and exported data don't matchOverview
----------------------------------------
When Exporting Contributions from a search with the "Soft Credits Only" filter, the export screen shows the contributor information but the exported file shows the soft credited contact.
...Overview
----------------------------------------
When Exporting Contributions from a search with the "Soft Credits Only" filter, the export screen shows the contributor information but the exported file shows the soft credited contact.
Reproduction steps
----------------------------------------
1. Go to the "Find Contributions" search form (CiviCRM Admin menu -> Search -> Find Contributions)
2. For the "Contributions OR Soft Credits?" field select "Soft Credits Only"
3. Click Search
4. Select All records
5. Select the Action "export contributions"
6. pick the "Select fields for Export" option and click "Continue"
7. Add a "First Name" Field
8. Click "Download File"
Current behaviour
----------------------------------------
The export screen shows the first name of the Contributor
The export file shows the first name of the person who is soft credited
Expected behaviour
----------------------------------------
The export screen and exported file should display the same information. Because the filter is for "Soft Credit Only" I would expect they would both show the soft credit contact.
Environment information
----------------------------------------
I was able to recreate this on https://dmaster.demo.civicrm.org/
* __CiviCRM: 5.43.alpha1
* __CMS:drupalhttps://lab.civicrm.org/dev/core/-/issues/2827Groups do not show on multisite except in search builder2023-09-22T05:03:26ZandyburnsGroups do not show on multisite except in search builderEver since Civi 5.35.2, group search field does not populate for non-admin users. It either has no dropdown options or is entirely missing. No groups showing under 'Manage Groups'. See discussion here: https://github.com/civicrm/civicrm-...Ever since Civi 5.35.2, group search field does not populate for non-admin users. It either has no dropdown options or is entirely missing. No groups showing under 'Manage Groups'. See discussion here: https://github.com/civicrm/civicrm-core/pull/19372. Before all groups would show. On multisite. Users have `CiviCRM Multisite: list all groups in domain permission`. Groups do show in search builder but outputs the ID of group and not the name.
I've ruled out extensions (renamed directory) and the WP plugins. Replicated on another multisite as well.
A non-admin user is able to see groups on domain ID 1 but not on any other domain. That is as far as I have debugged.
See: https://civicrm.stackexchange.com/questions/40065/group-search-not-available-on-all-search-screens-except-for-search-builderhttps://lab.civicrm.org/dev/core/-/issues/4616js error on contribution pages - also prevents honoree section toggling from ...2023-09-22T01:12:32ZDaveDjs error on contribution pages - also prevents honoree section toggling from working`var quickConfig = ;`
templates/CRM/Contribute/Form/Contribution/Main.tpl is expecting the $quickConfig variable to be a 0 or 1 but it's a bool. Note there's also a second spot in the tpl where the smarty var is used.
I think it might ...`var quickConfig = ;`
templates/CRM/Contribute/Form/Contribution/Main.tpl is expecting the $quickConfig variable to be a 0 or 1 but it's a bool. Note there's also a second spot in the tpl where the smarty var is used.
I think it might be from here https://github.com/civicrm/civicrm-core/pull/27234/files.
Technically this affects events with pricesets too, except you don't notice because the end result is the same.5.67.0https://lab.civicrm.org/dev/core/-/issues/1251Missing Activity from case if source contact is deleted.2023-09-21T05:03:18ZjitendraMissing Activity from case if source contact is deleted.Looks like when a source contact attached to a case activity is deleted, the activity is not displayed under the case.
Scenario -
- Case has some set of activities with target = XYZ and source contact set to "ABC".
- ABC contact is per...Looks like when a source contact attached to a case activity is deleted, the activity is not displayed under the case.
Scenario -
- Case has some set of activities with target = XYZ and source contact set to "ABC".
- ABC contact is permanently deleted from civicrm.
- Activities with source set to ABC is not displayed on the case.
- Viewing the activity from the URL leads to a fatal error.
Approach to fix?
1. Add a validation error/warning before permanently deleting a contact so that user knows this contact is related to activity as "source" and can lead to some missing data on case management screen.
2. Load activity under case even if source contact is not present. https://github.com/civicrm/civicrm-core/commit/cbcb312b3e023d8205bb261067d54c79ac79d971 can provide a fix for this approach and a fatal error on view activity page.https://lab.civicrm.org/dev/core/-/issues/2823Extend 'managed' api to provide information about 'status'2023-09-21T05:03:17ZeileenExtend 'managed' api to provide information about 'status'We've had some discussion on chat about making the 'Managed' api more flexible - ie
- being able to retrieve a list of outstanding actions
- being able to selectively action them
- having the api figure out dependencies - either 'explic...We've had some discussion on chat about making the 'Managed' api more flexible - ie
- being able to retrieve a list of outstanding actions
- being able to selectively action them
- having the api figure out dependencies - either 'explicit' because the managed description declares them or 'implicit' using logic like 'create option groups before option values
- full support for apiv4
I've dug around in the code a little & my thiking is a first helpful refactor would be to make the Managed class build a full array of 'managed actions' rather than passing around 'todos' in a way that makes things unavailable outside the specific flow.
I think the number of rows in the 'managed' actions is not likely to be so great it would cause memory issues so I'm imaginging the property looking like
```
$this->managedActions = [
[$module-$name-$entity] => [
/// all the details from the module +
'id' => x, // if it exists in civicrm_managed
'required_action' => 'update'// or delete, create
],
]
```
Then we could start to add functions like `getEntitiesToDelete` & iterate through them, rather than passing around `$todos`https://lab.civicrm.org/dev/core/-/issues/2005Token Sub Type improvements2023-09-20T05:03:28ZseamusleeToken Sub Type improvementsGitlab for PR https://github.com/civicrm/civicrm-core/pull/16982Gitlab for PR https://github.com/civicrm/civicrm-core/pull/16982https://lab.civicrm.org/dev/core/-/issues/171Personal information can be leaked by omitting capture token in mailing2023-09-20T05:03:28ZIan KellingPersonal information can be leaked by omitting capture token in mailingI've tested this on 5.0. I don't see any changelog entry that would make
this not happen in newer versions.
I'm sure there is a more minimal repro if I understood the root cause.
Repro:
Create a group with 3 contacts, I'll call them ...I've tested this on 5.0. I don't see any changelog entry that would make
this not happen in newer versions.
I'm sure there is a more minimal repro if I understood the root cause.
Repro:
Create a group with 3 contacts, I'll call them contact1 contact2 contact3.
Do a mailing to that group, where the html and plain text email version is this:
{capture assign=addressee}{contact.addressee}{/capture}
Dear {if $addressee }{ $addressee }{else}Supporter{/if},
{domain.address}
{action.optOutUrl}
Have contact3 opt out using the link in the email.
Reuse that mailing to create a new one. Name it different, and change
just the text version of the email to be this:
Dear {if $addressee }{ $addressee }{else}Supporter{/if},
{domain.address}
{action.optOutUrl}
Expected result:
Text version of second email says "Dear Supporter" to two contacts.
Actual result:
contact1 gets an email saying "Dear Supporter". contact2 gets an email
saying "Dear contact1".
Now contact1's name and the fact that they are on this list has been
revealed to contact2. This is private information which should not
have been revealed to contact2.
This should be considered a very high priority issue, because in a
larger group, almost all contacts would receive the private information
of someone other than themselves, which is a serious data breach /
privacy problem.
If this bug is confirmed, as part of the fix I recommend creating a
tool which can analyze civi users past mailings to see if any of them
were affected by this bug.https://lab.civicrm.org/dev/core/-/issues/4605Don't try to append breadcrumbs while in upgrade mode2023-09-20T02:57:38ZDaveDDon't try to append breadcrumbs while in upgrade modeOn a drupal 7 site with civicrm_entity and views, the upgrade crashes with the symptom people have been reporting about core extensions not getting enabled, so it can't find `Api4\EntityFinancialAccount`. At least on this site, it happen...On a drupal 7 site with civicrm_entity and views, the upgrade crashes with the symptom people have been reporting about core extensions not getting enabled, so it can't find `Api4\EntityFinancialAccount`. At least on this site, it happens while it's trying to rebuild the theme registry while adding a breadcrumb. This ends up calling hook_views_data, which ends up calling civicrm_entity_entity_property_info_alter, which tries to getOptions for various civi things, which then hits this missing class since it hasn't been installed yet.
But breadcrumbs seem pointless during upgrade. Doing this works to get around the problem:
```diff
--- a/CRM/Core/Invoke.php
+++ b/CRM/Core/Invoke.php
@@ -252,7 +252,7 @@ class CRM_Core_Invoke {
CRM_Utils_System::setTitle($item['title']);
}
- if (isset($item['breadcrumb']) && empty($item['is_public'])) {
+ if (!CRM_Core_Config::isUpgradeMode() && isset($item['breadcrumb']) && empty($item['is_public'])) {
CRM_Utils_System::appendBreadCrumb($item['breadcrumb']);
}
```
Might need some testing on other cmses.5.65.2https://lab.civicrm.org/dev/core/-/issues/4551Single quote in "public" event page title throws smarty error on receipt2023-09-19T14:02:11ZelilisseckSingle quote in "public" event page title throws smarty error on receiptOverview
----------------------------------------
Reproduced on a blank 5.64.2 buildkit site. If a contribution page has a single quote in the title, is set to "is_share", and has receipts on, a user registering will receive this gross m...Overview
----------------------------------------
Reproduced on a blank 5.64.2 buildkit site. If a contribution page has a single quote in the title, is set to "is_share", and has receipts on, a user registering will receive this gross message during the registration process:
![image](/uploads/e105ff57ae8d9ea64bf4d30b81f08b35/image.png)
Reproduction steps
----------------------------------------
1. On 5.64.0+ add (or edit) an event page to have a single quote in the page title such as `What's a quote doing here`
2. Check off "is_share" and turn on receipt emails.
3. Run through a registration on the front-end to observe the issue.
Comments
----------------------------------------
It looks like maybe this commit introduced this but it's hard to follow: https://github.com/civicrm/civicrm-core/commit/e90b263a9ccd7c8106c98f5f7f82d4b267820f43#diff-2652c5ed241e0c2a68dc79d97ce2b6056ad2bc8ce37aee9bce8dd69678a160a8L167
Either way the issue is coming from this line in the workflow message template `{include file="CRM/common/SocialNetwork.tpl" emailMode=true url=$eventUrl title=$event.title pageURL=$eventUrl}` where the event title needs to have single quotes escaped or something of that nature to make it through smarty.5.65.0https://lab.civicrm.org/dev/core/-/issues/4611Drupal 7 SmartMenu not appearing2023-09-19T13:02:50ZshaneonabikeDrupal 7 SmartMenu not appearingOverview
----------------------------------------
When viewing CiviCRM on a Mobile device (or via Mobile testing) the SmartMenu implemented in #487 is not appearing. I can see this working just fine on both Wordpress and Drupal 8, but f...Overview
----------------------------------------
When viewing CiviCRM on a Mobile device (or via Mobile testing) the SmartMenu implemented in #487 is not appearing. I can see this working just fine on both Wordpress and Drupal 8, but for some reason it stopped working on Drupal7. It flashes on initial load (mobile), but then disappears.
Reproduction steps
----------------------------------------
1. Login to a site via mobile (Drupal 7)
2. Go to CiviCRM dashboard
3. Notice top menu flash Smartmenu and then disappears
Expected behaviour
----------------------------------------
SmartMenu would appear like all other instances.
Environment information
----------------------------------------
<!-- Some of the items below may not be relevant for every bug - if in doubt please include more information than you think is neccessary. -->
* __Browser:__ _Firefox 117_
* __CiviCRM:__ _Master_
* __PHP:__ _7.3_
* __CMS:__ _Drupal 7.98_
* __Web Server:__ _Nginx_
* JS not being aggregated by Drupal
Additional notes
----------------------------------------
+ I can see that on Wordpress it is creating a file ```/civicrm/persist/contribute/dyn/crm-menubar.9befe9224bf410ee58ffe3edd216b485.css``` which is being used to handle the display of the SmartMenu from what I can tell
+ This file isn't loaded or created on a Drupal7 instance in question - we tried it also on our dmaster (demo) and it is also not workinghttps://lab.civicrm.org/dev/core/-/issues/2748Remove unused token assigns2023-09-19T05:03:18ZeileenRemove unused token assignsWe have a bunch of smarty variables that we spend a lot of code assigning to the the template but which have not been used in the shipped template for some time. We've been scared to stop assigning them 'in case' but we now have code to ...We have a bunch of smarty variables that we spend a lot of code assigning to the the template but which have not been used in the shipped template for some time. We've been scared to stop assigning them 'in case' but we now have code to warn on upgrade about these tokens so we can start removing
Notably the 'relatedContact' tokens removal will also allow some weird code to gohttps://lab.civicrm.org/dev/core/-/issues/182Proposal - add testSetup hook2023-09-19T05:03:18ZeileenProposal - add testSetup hookI am having a problem with test interoperability. I have an extension that other extensions can hook into (by virtue of having an api with a particular action).
The issue appears in unit tests. On the main extension test it will call t...I am having a problem with test interoperability. I have an extension that other extensions can hook into (by virtue of having an api with a particular action).
The issue appears in unit tests. On the main extension test it will call the code in the other extension as it would in production. However, there are some specific set up actions required for testing on other extension that happen in it's own tests but not, currently, in the main extension's test runs.
My best idea for this is to have a hook that sits in core called civicrm_testSetup & which could be called from the setup function in a unit test.
I did think about using the existing Config hook - but we run these tests under the Drupal CMS rather than under the Unit tests CMS so I'm not sure how we would identify it's a test. There is also something a bit odd about putting test concerns in the main config function.
I also thought about just putting the hook in my extension - but it seems that I would quickly want to add it to more than one extension - hence I think core maybe makes sense