Development issueshttps://lab.civicrm.org/groups/dev/-/issues2023-09-24T22:55:47Zhttps://lab.civicrm.org/dev/core/-/issues/3444Contribution balance token2023-09-24T22:55:47Zmagnolia61Contribution balance tokenOverview
----------------------------------------
Would it be technically easy and functional desired to have contribution balance token?
Example use-case
----------------------------------------
We would like to advocate for an contrib...Overview
----------------------------------------
Would it be technically easy and functional desired to have contribution balance token?
Example use-case
----------------------------------------
We would like to advocate for an contribution balance token
Current behaviour
----------------------------------------
1. We want to send our customers an email with the amount due, or the amount that is pending refund.
2. There is a balance field in the event participant context. But this is absent in the contribution context.
Proposed behaviour
----------------------------------------
A custom token field like {contribution.balance} is available in the contribution context
Comments
----------------------------------------
Eileen opened an issue to enable a balance field for api4 (and an total paid field): https://lab.civicrm.org/dev/core/-/issues/2890https://lab.civicrm.org/dev/core/-/issues/2638Money - create new Civi:: facade - now format helper2023-09-24T22:53:25ZeileenMoney - create new Civi:: facade - now format helperOur CRM_Utils_Money class wound up a mess so we talked about creating a new supported/tested/readable class to put our money functions in and this class would be supported for use by core & external code.
@seamuslee did a first cut here...Our CRM_Utils_Money class wound up a mess so we talked about creating a new supported/tested/readable class to put our money functions in and this class would be supported for use by core & external code.
@seamuslee did a first cut here https://github.com/civicrm/civicrm-core/pull/20296 which highlighted some of the issues / complexity
I made comments there but I've moved them to this gitlab
Useful framing by @jaapjansma
CiviCRM has the following ways of displaying data:
- On the screen in back office (this is the language and locale of the logged in user)
- On the screen in front office screens (such as registering for an event, this should be the sites locale and language)
- In communications with the contact, such as letters and pdf. We should use the language and locale of the contact
Ideally we should also have a setting for the locale and not only the language.
Looking at @jaapjansma's list it seems that at the end of the day we want (these don't have to be all working / added now but I think we might want to comment in the intent into the class)
```
Money()->formatMachine($amount, $currency);
Money()->formatSiteLocale($amount, $currency);
Money()->formatUserLocale($amount, $currency, $contactID); // defaults to logged in user
Money()->formatSpecifiedLocale($amount, $currency, $locale);
```
We also have 2 other variables in the mix - precision & noCurrencySymbol
I tend to prefer to use separate functions for noCurrencySymbol - eg
```
Money()->formatNumericOnlyMachine($amount, $currency);
Money()->formatNumericOnlySiteLocale($amount, $currency);
Money()->formatNumericOnlyUserLocale($amount, $currency, $contactID); // defaults to logged in user
Money()->formatNumericOnlySpecifiedLocale($amount, $currency, $locale);
```
but it could be a consistent parameter in the other functions
On the precision question we have
- fixed precision
- no rounding
- currency specific precision
Perhaps this is a parameter on all the above functions ('fixed', NULL or 'by_currency' ) - I personally prefer documenting that in the comment block than using a constant although some people like constants
This is quickly getting us back into the confusion of CRM_Core_Money which this PR was intended to get us away from but hopefully we can figure out the set of signatures we want this time AND remember what they actually mean!
Also - we probably want functions that returns the actual Money object - not just the output
Note that where we want to be is that the CRM_Utils_Money class can be an internal class and the Civi::Money is an external class - so how we deal with the thousand separator in CRM_Utils_Money can change over timehttps://lab.civicrm.org/dev/core/-/issues/28715.43 will need an upgrade message about the token changes - with a link to th...2023-09-24T22:50:51Zeileen5.43 will need an upgrade message about the token changes - with a link to the docshttps://lab.civicrm.org/dev/core/-/issues/2890Contribution api - get paid, balance2023-09-24T22:50:10ZeileenContribution api - get paid, balance@colemanw just wanted to put this on your radar - being able to get
contribution.paid_amount and
contribution.balance
would be hugely use at the api level , search kit AND the template / token level.
We currently have a php function ...@colemanw just wanted to put this on your radar - being able to get
contribution.paid_amount and
contribution.balance
would be hugely use at the api level , search kit AND the template / token level.
We currently have a php function & what it does to get paid_amount is this...
I suspect there are some challenges in that it would have to be a subquery which would have performance limits but it would have a lot of application
```
$sql = "SELECT SUM(ft.total_amount) FROM civicrm_financial_trxn ft
INNER JOIN civicrm_entity_financial_trxn eft ON (eft.financial_trxn_id = ft.id AND eft.entity_table = 'civicrm_contribution')
WHERE eft.entity_id = %1 AND ft.is_payment = 1 AND ft.status_id IN (%2) ";
return (float) CRM_Core_DAO::singleValueQuery($sql, [
1 => [$contributionID, 'Integer'],
// status IDs are 'completed' & 'refunded'
2 => [implode(',', $statusIDs), 'CommaSeparatedIntegers'],
]);
```https://lab.civicrm.org/dev/core/-/issues/587Permissions on GroupContact API calls seem wrong2023-09-24T16:09:28ZJonGoldPermissions on GroupContact API calls seem wrongI got a support request from a user who didn't have "Edit All Contacts" permissions stating that they couldn't remove someone from a group. Sure enough, [edit all contacts](https://lab.civicrm.org/dev/core/blob/master/CRM/Core/Permissio...I got a support request from a user who didn't have "Edit All Contacts" permissions stating that they couldn't remove someone from a group. Sure enough, [edit all contacts](https://lab.civicrm.org/dev/core/blob/master/CRM/Core/Permission.php#L1229) is the necessary permission. However, editing/removing tags just requires "access CiviCRM".
Does this seem correct to folks? Is it to prevent someone escalating their ACL permissions? If so, it feels like we need a different permission, and predates more nuanced solutions such as [Group Protect](https://github.com/CiviCooP/org.civicoop.groupprotect). The only other entity that needs such high permissions is Relationship - that also seems wrong.
I propose that we add both GroupContact and Relationship entities to the `_civicrm_api3_check_edit_permissions()` function. If you can edit the contact, you can edit their groups/relationships. If someone gives this a "Concept: Approved" I'll work on the PR.JonGoldJonGoldhttps://lab.civicrm.org/dev/core/-/issues/2841Undefined index isCaseActivity when creating an activity2023-09-24T05:03:25ZDaveDUndefined index isCaseActivity when creating an activityThis is probably from one of the message template rearrangings. For the email it sends to the activity assignee.This is probably from one of the message template rearrangings. For the email it sends to the activity assignee.https://lab.civicrm.org/dev/core/-/issues/2818Context gets lost when sending an email from manage case so it redirects some...2023-09-24T05:03:25ZDaveDContext gets lost when sending an email from manage case so it redirects somewhere else afterI think this is the same cause as https://lab.civicrm.org/dev/core/-/issues/2318 where something about `$this->_single` went missing. It should redirect you back to manage case.
1. Turn off popups at administer - customize - display pre...I think this is the same cause as https://lab.civicrm.org/dev/core/-/issues/2318 where something about `$this->_single` went missing. It should redirect you back to manage case.
1. Turn off popups at administer - customize - display prefs (or when opening the link in step 3 open in a new tab).
1. Create case
1. In the roles section use the email icon to send an email to e.g. the client
1. After sending, it goes somewhere else instead of manage case. Sometimes it will take you to the contact record which at least is related, sometimes it goes to /civicrm/dashboard, sometimes somewhere else.https://lab.civicrm.org/dev/core/-/issues/4621Some links don't work in Firefox with uBlock Origin2023-09-23T10:15:05ZresgaSome links don't work in Firefox with uBlock Origin## Overview
If I surf my CiviCRM installation with the Firefox extension [uBlock Origin](https://github.com/gorhill/uBlock) enabled and try to view a member, I get redirected to the front page, and this error message in the upper right ...## Overview
If I surf my CiviCRM installation with the Firefox extension [uBlock Origin](https://github.com/gorhill/uBlock) enabled and try to view a member, I get redirected to the front page, and this error message in the upper right corner:
> Error
> We could not find a contact id.
I managed to get the member to show via another method (I have a custom Drupal list) but when I click on "Contributions" or "Memberships" I get these errors:
> Error
> Could not find valid value for cid
> Network Error
> Unable to reach the server. Please refresh this page in your browser and try again.
If I disable uBlock Origin, the problem goes away.
In the browser address field, slashes and question marks are transformed, to look like this:
https://mysite.ddev.site/civicrm/contact/view%3Fcid%3D5773%26reset%3D1?selectedChild=summary
Could that be the cause? I believe this behaviour started after updating from version 5.43.2 to 5.51.1.
## Reproduction steps
1. Install and enable uBlock Origin in Firefox
1. Click on **Search -> Find Members**
1. Click on **Search** button
1. Click on a member
1. Get an error "**Error: > We could not find a contact id.**"
## Environment information
* __Browser:__ _Firefox 117.0.1_
* __Browser extension:__ _uBlock Origin_
* __CiviCRM:__ _Master/5.60.0_
* __PHP:__ _7.4_
* __CMS:__ _Drupal 7.97_
* __Database:__ _MariaDB 10.4_
* __Web Server:__ _Apache 2.4 AND Nginx 1.16_
## Possible solution
Would not HTML-encoding the links allow it to work with uBlock Origin?https://lab.civicrm.org/dev/core/-/issues/4501Redis performance issue on delete contact2023-09-23T05:08:34ZeileenRedis performance issue on delete contactWe have a process where we merge contacts and then delete the deleted contacts after a period of time.
However, we have more or less never run the delete deleted contacts script because it is too slow. I dug into it today and found that...We have a process where we merge contacts and then delete the deleted contacts after a period of time.
However, we have more or less never run the delete deleted contacts script because it is too slow. I dug into it today and found that
- on staging it takes 15 seconds to delete 500 contacts
- on production it takes 6-10 minutes to delete 500 contacts
I've spent most of the day digging into why & determined that the queries run are identical & timings are similary. However, on production each time this line of code runs ` Civi::service('prevnext')->deleteItem($id);` it takes a bit over 1 second. This is not the case on staging because there are no users populating the prevnext cache with searches. Hence I have diagnosed that the problem is having users & the solution is to lock their accounts.
More specifically the issue is that the code is going through all the Redis keys to remove the contact - which seems to be inefficient.
![image](/uploads/701ae2d41d47d0eae49ee118f80d4dc4/image.png)
I did wonder if a quick-fix would be to only call `deleteItem` if the contact is not already deleted (which they are in our use case) - I would need to change [the find to fetch here](https://github.com/civicrm/civicrm-core/blob/aef17937a6d1bf00d2ca446bf3c5fc81644b3b92/CRM/Contact/BAO/Contact.php#L907-L911) I think...
Alternatively there is probably some option around queuing the cache clear to happen at the end. However, I think that 1 second + delay is actually not great for users either - e.g when deduping a bunch of contacts than having each form submit take that bit longer would add up.
We are on the cusp of getting `coworker` going so pushing something to a queue to clear out caches might be an option.https://lab.civicrm.org/dev/core/-/issues/2831It is possible to break Scheduled Reminder by adding too may entity types2023-09-23T05:03:26ZDavid HayesIt is possible to break Scheduled Reminder by adding too may entity types# How to reproduce
1. Add about 100 activity types
1. Create a Scheduled Reminder that triggers on Activity
1. Select most or all activity types
1. Save Scheduled Reminder
1. Edit the recently created Scheduled Reminder, notice not all A...# How to reproduce
1. Add about 100 activity types
1. Create a Scheduled Reminder that triggers on Activity
1. Select most or all activity types
1. Save Scheduled Reminder
1. Edit the recently created Scheduled Reminder, notice not all Activity Types are selected anymore
1. Scheduled Reminders will begin failing
-Checking the Scheduled Job logs, the error will be `Finished execution of Send Scheduled Reminders with result: Failure, Error message: Failed encoding non-numeric value (#selectedValues)`
# The cause
The selected activity types are stored in the `civicrm_action_schedule` table in the `entity_value` column. This column only supports 255 characters. Adding any more characters than this causes the value to be truncated with an elipsis "..." at the end. This error originates from `CRM_Activity_ActionMapping::createQuery` where the code places the trucated list of Activity Types into a where clause in a query. The "..." after the last value cause an integer check to fail.
# How to fix
Convert this column to something longer? TEXT maybe? Happy to make a PR, but would like to know the preferred path.https://lab.civicrm.org/dev/core/-/issues/645Use site email domain in place of bounce email domain for automated messages2023-09-23T05:03:26ZnishantBUse site email domain in place of bounce email domain for automated messagesHello there!!
We encountered a scenario where we can whitelist only one email domain to send emails. We want to use email domain (eg: bouncedomain.com) for bounce processing and a different one (sitedomain.com) for rest of the emails wh...Hello there!!
We encountered a scenario where we can whitelist only one email domain to send emails. We want to use email domain (eg: bouncedomain.com) for bounce processing and a different one (sitedomain.com) for rest of the emails which works fine except that do-not-reply emails use the email domain set for bounce processing (bouncedomain.com) because of which AWS SES prevent those emails to be sent.
**Current behaviour:**
1. Regular emails: example@sitedomain.com
2. Bounce emails: bounce@bouncedomain.com
3. Do-not-reply emails: do-not-reply@bouncedomain.com
**Expected behaviour:**
1. Regular emails: example@sitedomain.com
2. Bounce emails: bounce@bouncedomain.com
3. Do-not-reply emails: do-not-reply@sitedomain.com
Is there already a configuration to set that or would it be a good idea to add a field where we can set the email domain for do-not-reply emails ?
Thanks!https://lab.civicrm.org/dev/core/-/issues/2813"Soft Credit Only" Contribution export screen and exported data don't match2023-09-22T05:03:27Zalicefrumin"Soft Credit Only" Contribution export screen and exported data don't matchOverview
----------------------------------------
When Exporting Contributions from a search with the "Soft Credits Only" filter, the export screen shows the contributor information but the exported file shows the soft credited contact.
...Overview
----------------------------------------
When Exporting Contributions from a search with the "Soft Credits Only" filter, the export screen shows the contributor information but the exported file shows the soft credited contact.
Reproduction steps
----------------------------------------
1. Go to the "Find Contributions" search form (CiviCRM Admin menu -> Search -> Find Contributions)
2. For the "Contributions OR Soft Credits?" field select "Soft Credits Only"
3. Click Search
4. Select All records
5. Select the Action "export contributions"
6. pick the "Select fields for Export" option and click "Continue"
7. Add a "First Name" Field
8. Click "Download File"
Current behaviour
----------------------------------------
The export screen shows the first name of the Contributor
The export file shows the first name of the person who is soft credited
Expected behaviour
----------------------------------------
The export screen and exported file should display the same information. Because the filter is for "Soft Credit Only" I would expect they would both show the soft credit contact.
Environment information
----------------------------------------
I was able to recreate this on https://dmaster.demo.civicrm.org/
* __CiviCRM: 5.43.alpha1
* __CMS:drupalhttps://lab.civicrm.org/dev/core/-/issues/2827Groups do not show on multisite except in search builder2023-09-22T05:03:26ZandyburnsGroups do not show on multisite except in search builderEver since Civi 5.35.2, group search field does not populate for non-admin users. It either has no dropdown options or is entirely missing. No groups showing under 'Manage Groups'. See discussion here: https://github.com/civicrm/civicrm-...Ever since Civi 5.35.2, group search field does not populate for non-admin users. It either has no dropdown options or is entirely missing. No groups showing under 'Manage Groups'. See discussion here: https://github.com/civicrm/civicrm-core/pull/19372. Before all groups would show. On multisite. Users have `CiviCRM Multisite: list all groups in domain permission`. Groups do show in search builder but outputs the ID of group and not the name.
I've ruled out extensions (renamed directory) and the WP plugins. Replicated on another multisite as well.
A non-admin user is able to see groups on domain ID 1 but not on any other domain. That is as far as I have debugged.
See: https://civicrm.stackexchange.com/questions/40065/group-search-not-available-on-all-search-screens-except-for-search-builderhttps://lab.civicrm.org/dev/core/-/issues/4616js error on contribution pages - also prevents honoree section toggling from ...2023-09-22T01:12:32ZDaveDjs error on contribution pages - also prevents honoree section toggling from working`var quickConfig = ;`
templates/CRM/Contribute/Form/Contribution/Main.tpl is expecting the $quickConfig variable to be a 0 or 1 but it's a bool. Note there's also a second spot in the tpl where the smarty var is used.
I think it might ...`var quickConfig = ;`
templates/CRM/Contribute/Form/Contribution/Main.tpl is expecting the $quickConfig variable to be a 0 or 1 but it's a bool. Note there's also a second spot in the tpl where the smarty var is used.
I think it might be from here https://github.com/civicrm/civicrm-core/pull/27234/files.
Technically this affects events with pricesets too, except you don't notice because the end result is the same.5.67.0https://lab.civicrm.org/dev/core/-/issues/1251Missing Activity from case if source contact is deleted.2023-09-21T05:03:18ZjitendraMissing Activity from case if source contact is deleted.Looks like when a source contact attached to a case activity is deleted, the activity is not displayed under the case.
Scenario -
- Case has some set of activities with target = XYZ and source contact set to "ABC".
- ABC contact is per...Looks like when a source contact attached to a case activity is deleted, the activity is not displayed under the case.
Scenario -
- Case has some set of activities with target = XYZ and source contact set to "ABC".
- ABC contact is permanently deleted from civicrm.
- Activities with source set to ABC is not displayed on the case.
- Viewing the activity from the URL leads to a fatal error.
Approach to fix?
1. Add a validation error/warning before permanently deleting a contact so that user knows this contact is related to activity as "source" and can lead to some missing data on case management screen.
2. Load activity under case even if source contact is not present. https://github.com/civicrm/civicrm-core/commit/cbcb312b3e023d8205bb261067d54c79ac79d971 can provide a fix for this approach and a fatal error on view activity page.https://lab.civicrm.org/dev/core/-/issues/2823Extend 'managed' api to provide information about 'status'2023-09-21T05:03:17ZeileenExtend 'managed' api to provide information about 'status'We've had some discussion on chat about making the 'Managed' api more flexible - ie
- being able to retrieve a list of outstanding actions
- being able to selectively action them
- having the api figure out dependencies - either 'explic...We've had some discussion on chat about making the 'Managed' api more flexible - ie
- being able to retrieve a list of outstanding actions
- being able to selectively action them
- having the api figure out dependencies - either 'explicit' because the managed description declares them or 'implicit' using logic like 'create option groups before option values
- full support for apiv4
I've dug around in the code a little & my thiking is a first helpful refactor would be to make the Managed class build a full array of 'managed actions' rather than passing around 'todos' in a way that makes things unavailable outside the specific flow.
I think the number of rows in the 'managed' actions is not likely to be so great it would cause memory issues so I'm imaginging the property looking like
```
$this->managedActions = [
[$module-$name-$entity] => [
/// all the details from the module +
'id' => x, // if it exists in civicrm_managed
'required_action' => 'update'// or delete, create
],
]
```
Then we could start to add functions like `getEntitiesToDelete` & iterate through them, rather than passing around `$todos`https://lab.civicrm.org/dev/core/-/issues/2005Token Sub Type improvements2023-09-20T05:03:28ZseamusleeToken Sub Type improvementsGitlab for PR https://github.com/civicrm/civicrm-core/pull/16982Gitlab for PR https://github.com/civicrm/civicrm-core/pull/16982https://lab.civicrm.org/dev/core/-/issues/171Personal information can be leaked by omitting capture token in mailing2023-09-20T05:03:28ZIan KellingPersonal information can be leaked by omitting capture token in mailingI've tested this on 5.0. I don't see any changelog entry that would make
this not happen in newer versions.
I'm sure there is a more minimal repro if I understood the root cause.
Repro:
Create a group with 3 contacts, I'll call them ...I've tested this on 5.0. I don't see any changelog entry that would make
this not happen in newer versions.
I'm sure there is a more minimal repro if I understood the root cause.
Repro:
Create a group with 3 contacts, I'll call them contact1 contact2 contact3.
Do a mailing to that group, where the html and plain text email version is this:
{capture assign=addressee}{contact.addressee}{/capture}
Dear {if $addressee }{ $addressee }{else}Supporter{/if},
{domain.address}
{action.optOutUrl}
Have contact3 opt out using the link in the email.
Reuse that mailing to create a new one. Name it different, and change
just the text version of the email to be this:
Dear {if $addressee }{ $addressee }{else}Supporter{/if},
{domain.address}
{action.optOutUrl}
Expected result:
Text version of second email says "Dear Supporter" to two contacts.
Actual result:
contact1 gets an email saying "Dear Supporter". contact2 gets an email
saying "Dear contact1".
Now contact1's name and the fact that they are on this list has been
revealed to contact2. This is private information which should not
have been revealed to contact2.
This should be considered a very high priority issue, because in a
larger group, almost all contacts would receive the private information
of someone other than themselves, which is a serious data breach /
privacy problem.
If this bug is confirmed, as part of the fix I recommend creating a
tool which can analyze civi users past mailings to see if any of them
were affected by this bug.https://lab.civicrm.org/dev/core/-/issues/4605Don't try to append breadcrumbs while in upgrade mode2023-09-20T02:57:38ZDaveDDon't try to append breadcrumbs while in upgrade modeOn a drupal 7 site with civicrm_entity and views, the upgrade crashes with the symptom people have been reporting about core extensions not getting enabled, so it can't find `Api4\EntityFinancialAccount`. At least on this site, it happen...On a drupal 7 site with civicrm_entity and views, the upgrade crashes with the symptom people have been reporting about core extensions not getting enabled, so it can't find `Api4\EntityFinancialAccount`. At least on this site, it happens while it's trying to rebuild the theme registry while adding a breadcrumb. This ends up calling hook_views_data, which ends up calling civicrm_entity_entity_property_info_alter, which tries to getOptions for various civi things, which then hits this missing class since it hasn't been installed yet.
But breadcrumbs seem pointless during upgrade. Doing this works to get around the problem:
```diff
--- a/CRM/Core/Invoke.php
+++ b/CRM/Core/Invoke.php
@@ -252,7 +252,7 @@ class CRM_Core_Invoke {
CRM_Utils_System::setTitle($item['title']);
}
- if (isset($item['breadcrumb']) && empty($item['is_public'])) {
+ if (!CRM_Core_Config::isUpgradeMode() && isset($item['breadcrumb']) && empty($item['is_public'])) {
CRM_Utils_System::appendBreadCrumb($item['breadcrumb']);
}
```
Might need some testing on other cmses.5.65.2https://lab.civicrm.org/dev/core/-/issues/4551Single quote in "public" event page title throws smarty error on receipt2023-09-19T14:02:11ZelilisseckSingle quote in "public" event page title throws smarty error on receiptOverview
----------------------------------------
Reproduced on a blank 5.64.2 buildkit site. If a contribution page has a single quote in the title, is set to "is_share", and has receipts on, a user registering will receive this gross m...Overview
----------------------------------------
Reproduced on a blank 5.64.2 buildkit site. If a contribution page has a single quote in the title, is set to "is_share", and has receipts on, a user registering will receive this gross message during the registration process:
![image](/uploads/e105ff57ae8d9ea64bf4d30b81f08b35/image.png)
Reproduction steps
----------------------------------------
1. On 5.64.0+ add (or edit) an event page to have a single quote in the page title such as `What's a quote doing here`
2. Check off "is_share" and turn on receipt emails.
3. Run through a registration on the front-end to observe the issue.
Comments
----------------------------------------
It looks like maybe this commit introduced this but it's hard to follow: https://github.com/civicrm/civicrm-core/commit/e90b263a9ccd7c8106c98f5f7f82d4b267820f43#diff-2652c5ed241e0c2a68dc79d97ce2b6056ad2bc8ce37aee9bce8dd69678a160a8L167
Either way the issue is coming from this line in the workflow message template `{include file="CRM/common/SocialNetwork.tpl" emailMode=true url=$eventUrl title=$event.title pageURL=$eventUrl}` where the event title needs to have single quotes escaped or something of that nature to make it through smarty.5.65.0