CiviCRM Core issueshttps://lab.civicrm.org/dev/core/-/issues2023-11-08T05:03:19Zhttps://lab.civicrm.org/dev/core/-/issues/2752Financial entity permissions2023-11-08T05:03:19ZeileenFinancial entity permissionsI hit an issue where a contact without 'Administer CiviCRM' but WITH 'view all contributions' cannot access a payment search display - [as described](https://civicrm.org/blog/eileen/searching-payments-civi)
I think the VIEW permissions ...I hit an issue where a contact without 'Administer CiviCRM' but WITH 'view all contributions' cannot access a payment search display - [as described](https://civicrm.org/blog/eileen/searching-payments-civi)
I think the VIEW permissions on all financial entities should be 'view all contributions' because otherwise we will keep hitting issues where part but not all of a contribution can be retrieved
@JoeMurray @monish.deb @seamusleehttps://lab.civicrm.org/dev/core/-/issues/2746Tokens - formatting syntax2021-11-03T09:24:42ZeileenTokens - formatting syntaxOne of the issues with our tokens is around formatting - or the lack of options around it - consider the following requirements (these are a little artificial but are representative of various requests)
1) I want to insert a birth date ...One of the issues with our tokens is around formatting - or the lack of options around it - consider the following requirements (these are a little artificial but are representative of various requests)
1) I want to insert a birth date in the standard formatting (I think that is 'long date'
2) I want to insert a birth date using short format
1) I want to insert Month of birth into an email, in the language in use
1) I want to add text if someone was born before a certain date
2) I want to use conditional that checks whether the month of birth is higher than today's month
We can visualise these as
```
Hey
Your birthday was on {contact.birth_date}, or for short {contact.birth_date|short}.
You were born in the month of {contact.birth_date|F}
{if {now|m} > {contact.birth_date|m}sorry I missed your birthday {/if}
{if {contact.birth_date|raw} < '1960-01-01'}You are pretty old ya know{/if}
```
Note that similar concerns apply to 'amount' fields where the need for 'raw' amounts is quite common.
Options as I see them are
1) never do anything to support format flexibility
2) define a pseudotoken formatting syntax for them - somewhat like above
3) define a pseudofield formatting syntax for them - this is basically 2 but with dots instead of pipes - we could also use double underscores
4) add a process (I thought we might be able to do this in token smarty but I'm not quite sure) that assigns a smarty token for every resolved token. ie if {contact.birth_date} is resolved then ALSO assign {$contact__birth_date}. Always assign in raw format. This would give template users the option of using smarty variables as interchangeable, more flexible versions. Over time we could alter what is advertised & promote smarty as preferred, if we choose.
**Some notes**
1. I haven't dug into how hard 4 would be - my gut says it is doable but is likely to wind up in the too hard basket if we extend that conversation to other templating engines (eg. twig) - which might be a good reason to look at 2 or 3 instead
1. It turns out that where smarty is enabled 'now' can already be printed out - ie `{$smarty.now|date_format:'%d %b %Y %H:%M:%S'}`. this is interesting because there are at least 3 extensions that make it available as a custom token....
2. Formatting thoughts - smarty uses [strftime formatting](https://www.php.net/strftime) eg. '%d %b %Y %H:%M:%S'. I feel like most of us would be more comfortable with strtotime formatting however, my efforts to understand the difference suggest that it is strftime not strtotime that handles formatting by locale so I feel pretty sure we should use strftime formats + our own standins - ie `dateformatDatetime`, `dateformatFull` `dateformatPartial` `dateformatYear` (we have a few more that we are currently storing strtotime style - ie `dateformatFinancialBatch` `dateformatshortdate` and `dateInputFormat` (brain explodes)
.
@totten @colemanw @seamuslee5.43.0https://lab.civicrm.org/dev/core/-/issues/2745Tokens - contributions - could we show them all?2021-09-29T04:58:14ZeileenTokens - contributions - could we show them all?The tokens available in the contribution tokens is a list of tokens people thought would be handy and wrote in - the list differed between the action schedule & the send letter and in some cases advertised tokens didn't work on one of th...The tokens available in the contribution tokens is a list of tokens people thought would be handy and wrote in - the list differed between the action schedule & the send letter and in some cases advertised tokens didn't work on one of those places. That reconcilliation is complete with https://github.com/civicrm/civicrm-core/pull/21046 merged.
However, we wind up with a list of available tokens that is 'all the fields on the contribution record except for 9 - which are in the screen shot below. I could make a weak case for excluding 'is_template_contribution' and a stronger case for excluding 'contact_id' but not for the others.
![image](/uploads/a8f29423e15b6c5e631da08b780fbcb7/image.png)
Options
1) include all tokens
2) include all tokens but hard-code out contact_id since it is likely to be confused with {contact.id} & potentially {membership.id}
3) include all tokens but add some form of metadata to Contribution.xml like <token>FALSE</token> and add that to contact_id & is_template_contribution (defaults to TRUE if not set)
@totten @colemanw @seamuslee5.43.0https://lab.civicrm.org/dev/core/-/issues/2730Consider replacing fopen() call in CRM_Utils_File::isIncludable with stream_r...2021-08-09T12:58:54ZDaveDConsider replacing fopen() call in CRM_Utils_File::isIncludable with stream_resolve_include_path()The problem is depending on how strict your setup is, some code that uses the error suppression operator (e.g. `@unlink('file_which_may_not_exist')`) behaves differently in php 8 and doesn't get supressed, either flooding your logs or ma...The problem is depending on how strict your setup is, some code that uses the error suppression operator (e.g. `@unlink('file_which_may_not_exist')`) behaves differently in php 8 and doesn't get supressed, either flooding your logs or making civi unusable in some cases.
In particular the api magic function provider uses an algorithm for e.g. getfields, where often the first guess is wrong, leading to an error.
isIncludable was added in 2011, and while stream_resolve_include_path was available at that time, it was introduced into php in 2010, so it's possible not a lot of sites had it.5.41.0https://lab.civicrm.org/dev/core/-/issues/2650Use consistent token syntax for pseudoconstants2022-09-01T07:55:13ZeileenUse consistent token syntax for pseudoconstantsAt some point we want to have consolidated token code. One thing that is missing at the moment is agreement as to what tokens we are working towards. (this came out of discussion fixing for php8 in https://github.com/civicrm/civicrm-core...At some point we want to have consolidated token code. One thing that is missing at the moment is agreement as to what tokens we are working towards. (this came out of discussion fixing for php8 in https://github.com/civicrm/civicrm-core/pull/20591) I want to constrain this issue to the very narrow question of syntax for pseudoconstants & unique names in tokens.
If we look at [the test](https://github.com/civicrm/civicrm-core/blob/275686a34ca7bf5e9c4d6653cd8cca5cf15e36cf/tests/phpunit/CRM/Contribute/Form/Task/PDFLetterCommonTest.php#L264) we can see the current tokens supported by the existing code. A somewhat different set is supported by the [TokenProcessor class for contributions](https://github.com/civicrm/civicrm-core/blob/c2a33d9c890f81bf2287351f0a830ef58b1dc98c/CRM/Contribute/Tokens.php#L26)
So we can see that in one place you would pass
{contribution.contribution_status_id} to get 'Pending'
and in the other it would be {contribution.status}
We are also 'promoting' (by presenting in the UI selector) {contribution.contribution_source} over {contribution.source} in how we present tokens, even though in this case I think both work.
I would propose that our preferred tokens adhere to apiv4 style conventions as much as possible. Preferring means that we work towards the following (on an ad hoc basis).
1) the tokens that are selectable in the UI on the form return the preferred variants
2) we support the preferred variants in tested code on both token methods
3) we run db update scripts to replace the less preferred with the preferred in the msg_template and action_schedule tables as we feel safe doing so
4) we deprecate less preferred variants by adding deprecation notices to places where they are used, when we feel safe doing so.
Proposed conventions
1) deprecate unique name prefixing - ie prefer {contribution.source} over {contribution.contribution_source}
2) actual field name returns the value of that field - ie {contribution.contribution_status_id} would return a number rather than a string
3) we use apiv4 style tokens for labels - ie {contribution.contribution_status_id:label}
(out of scope for this issue = apiv4 style custom field names or custom fields in general, returning calculated values for eg. contribution.check_number)
If we go with the above the specific action I will add to my (long) todo list is to engage with fixing contribution_status_id to make it consistent between the 2 classes and the selector - ie
1) Replace the 'advertised' token from
``` 'contribution.contribution_status_id:label' => 'Contribution Status'```
'contribution.contribution_status_id:label' => 'Contribution Status'
2) add tested support for ```'contribution.contribution_status_id:label'``` to both classes
3) fix any core message templates to use 'contribution.contribution_status_id:label'
4) fix the upgrade script to str_replace 'contribution.contribution_status_id' in the message template table (only)
The other fields in this space that are currently inconsistent are
- financial_type_id, campaign, payment instrument id, cancel_date, source
@colemanw @totten @seamuslee @mattwire5.43.0https://lab.civicrm.org/dev/core/-/issues/2639Money: Plan to migrate from currency separators from locale2023-08-16T05:03:18ZeileenMoney: Plan to migrate from currency separators from localeThere is a goal to switch to sites specifying their money_locale from specifying currency separators
The original discussion on how to do this was in the dev-digest and a PR. @mattwire has since suggested an alternate https://github.com...There is a goal to switch to sites specifying their money_locale from specifying currency separators
The original discussion on how to do this was in the dev-digest and a PR. @mattwire has since suggested an alternate https://github.com/civicrm/civicrm-core/pull/20296#issuecomment-855377189
"Override currency separators" - "By default CiviCRM will format currency in accordance with the locale for the user that is viewing the information"
I've moved the text below from a comment on the github PR to here ....
In terms of adding a setting I went hunting for where it was discussed before & it is here
https://github.com/civicrm/civicrm-core/pull/19753
![image](https://user-images.githubusercontent.com/336308/120940640-63d47800-c772-11eb-8b46-302f4d311ef3.png)
Further down the same PR is the content that was pasted into the dev-digest
![image](https://user-images.githubusercontent.com/336308/120940681-92eae980-c772-11eb-9ff9-588d1e3ca663.png)
Compared to @mattwire's suggestion this is
1) implicitly opt in at the start - ie let early adopters set it & when we are confident be more aggressive
2) a different setting name / description - I think setting money_locale might be less intuitive short term (since it magically disables the other settings) and more intuitive longer term since the other settings should become meaningless & disappear over time.
Revisiting @jaapjansma's comment
![image](https://user-images.githubusercontent.com/336308/120940808-33410e00-c773-11eb-9bae-9910a3ff7583.png)
It DOES seem like site locale is going to be wanted as a setting - although it could be that this is not the point where we add ithttps://lab.civicrm.org/dev/core/-/issues/2634Membership api for v42021-08-25T22:27:29ZeileenMembership api for v4Apiv4 doesn't have a membership api yet - mostly because of issues with the membership.create process.
I caught up with @kcristiano this week & we fleshed out what the 3 issues are
1) Membership api does financial stuff including creat...Apiv4 doesn't have a membership api yet - mostly because of issues with the membership.create process.
I caught up with @kcristiano this week & we fleshed out what the 3 issues are
1) Membership api does financial stuff including creating line items and in some cases creating contribution. I worked through getting rid of one of these bits of magic, converting the back office form to use Order api when setting up a new recurring membership. Getting review on this was slow going so it will take a long time to convert other places. (I have started on the batch membership form & Kevin is going to help with review there).
However, I made a proposal that Kevin agreed with that we simply don't touch ANY of that financial stuff if $params['version'] === 4. The idea is that the handling (which may or may not be right) for apiv3 and for those calls that still call the BAO directly will be unchanged but for v4 callers need to either
a) call the order api (which we will fix to call the v4 membership api internally) or
b) create their own line items. Note that memberships without contributions actually should have line items according to the model.
I will work on this once we have fixed some issues with the order api's integrity - https://github.com/civicrm/civicrm-core/pull/20656 is the most obvious but the intent is to get the extra cover that deprecating our random magic will provide https://github.com/civicrm/civicrm-core/pull/20678 too. (This is not a big piece of work but it requires fairly experienced review )
2) Membership date handling for new memberships is in the v3 api (and the form layer). I did a patch for this. https://github.com/civicrm/civicrm-core/pull/20759 -
3) Membership date handling for renewals. Apiv3 has some magic around this that is not in the BAO. I have come to the conclusion we can handle this by
-- using the Order->payment flow. In other words the dates are calculated at the point when Payment.create is added, using the membership_num_terms as stored in https://github.com/civicrm/civicrm-core/pull/20672
-- OR - if that flow won't work for people for any reason, they can pass in all the date values.5.42.0https://lab.civicrm.org/dev/core/-/issues/2629Participant Status: pending refund can have two conflicting meanings2024-01-17T05:03:29Zmagnolia61Participant Status: pending refund can have two conflicting meaningsOverview
----------------------------------------
With partial payments and partial refunds becoming mainstream in CiviCRM, I think it is time to reconsider the connection between contribution status and participant status. I think there...Overview
----------------------------------------
With partial payments and partial refunds becoming mainstream in CiviCRM, I think it is time to reconsider the connection between contribution status and participant status. I think there are more than one aspects of this, but I will focus on 'pending refund'
Example use-cases
----------------------------------------
1. change the payment or price-set to make a partial refund necessary
2. cancel the participant to make a (partial) refund necessary
Current behaviour
----------------------------------------
Currently a participant status: Pending Refund defaults as a positive count.
But there are two scenario's that can cause this participant status:
1. there is a contribution total that is higher than the total event fee amount
2. there has been a cancellation for the event and the fee is pending to be refunded
In scenario 1: The participant is still expected to attend the event
In scenario 2: The participant will not attend the event
Proposed behaviour
----------------------------------------
I think it is becoming troublesome to relate the contribution status with the event status.
The default is that 'Pending refund' counts positive towards attendance.
Two proposals could be
a) to remove participant statuses connected with the contribution
- may cause other difficulties, might me the best in the long end
b) to add a new status and to differentiate between two types of pending refund scenarios:
- Pending refund (cancelled)
- Pending refund (attending)
First one would have count status 0, second one count status 1
Comments
----------------------------------------
Any thoughts on this one? Can I find agreement on this proposal?https://lab.civicrm.org/dev/core/-/issues/2624Line items not visible on recurring contribution2021-07-28T14:48:48ZjaapjansmaLine items not visible on recurring contributionWhen creating a new contribution recur with a contribution page and a price set with multiple line items. The line items are not stored within the contribution recur entity.
**Steps to reproduce**
1. Create a new price set (Financial t...When creating a new contribution recur with a contribution page and a price set with multiple line items. The line items are not stored within the contribution recur entity.
**Steps to reproduce**
1. Create a new price set (Financial type donation)
2. Create a price set field (Financial type Campaign donation)
3. Create a second price set field (Financial type Products)
4. Create a contribution page:
Financial type: donation
Amounts tab: Payment Processor: test processor
Amounts tab: Contributions amount section enabled: checked
Amounts tab: Price set: _select the created price set_
Amounts tab: Recurring contributions: check
5. Create a new contribution recur through the contribution page
6. Lookup the contribution recur on the Contact Summary
**Expected result**
Showing the line items. The total amount is correct.
**Actual result**
Line items are not visible at the contribution recur.
Below a screenshot of the contribution
![Screenshot_20210520_211830](/uploads/2447b95df431002e4405559a72fec4b5/Screenshot_20210520_211830.png)
And of the recurring contribution
![Screenshot_20210520_211918](/uploads/4036a932904649bbaa7c0d93b13879fb/Screenshot_20210520_211918.png)
**Possible solution**
I have looked up the line items in the database and there are no line items linked to the recurring contribution.
So the possible solution would exists of two steps:
1. Connect the line items to contribution recur
2. Make the line items visible in a similar way as on the contribution.
**Remarks**
Creating a new contribution with the `contribution.repeattransaction` api based on the first contribution and the recurring contribution correctly creates the line items at the contribution level.
**Funding**
I have some funding to work on this. So PR's might follow soon.
**PRs**
* ~~Fix for saving line items at the recurring contribution: https://github.com/civicrm/civicrm-core/pull/20398~~
* Fix for showing the line items from the template contribution: https://github.com/civicrm/civicrm-core/pull/20399
* Fix for excluding template contributions from reports: https://github.com/civicrm/civicrm-core/pull/20450
* Fix for searching template contributions: https://github.com/civicrm/civicrm-core/pull/20451
* Fix for excluding template contributions on the contact summary tab: https://github.com/civicrm/civicrm-core/pull/20452
* Fix for adding a button view template on the recurring contributions tab: https://github.com/civicrm/civicrm-core/pull/20685
* ~~Fix for creation of template contributions: https://github.com/civicrm/civicrm-core/pull/20453~~
**To do**
* Update user documentation
* Make sure an upgrade path exists
See also financial#65.41.0jaapjansmajaapjansmahttps://lab.civicrm.org/dev/core/-/issues/2622APiv4 does not handled display name when only email is provided2021-05-27T22:27:30ZeileenAPiv4 does not handled display name when only email is providedDemonstrated in the test https://github.com/civicrm/civicrm-core/pull/18865
Options are
1) accept that it's OK to pass 'email' to contact.create bao even though it is not a 'contact field' or
2) do an extra check on every primary email...Demonstrated in the test https://github.com/civicrm/civicrm-core/pull/18865
Options are
1) accept that it's OK to pass 'email' to contact.create bao even though it is not a 'contact field' or
2) do an extra check on every primary email save - note that historically parsing the smarty greeting tpl for the display name is expensive so we'd need to only set if empty.
I'm on the fence about the performance of 2 - most of our db locks these days are actually db insert locks on the email table for some reason (now that we have eliminated the acl related locks)
@colemanw - but to gitlab as promisedhttps://lab.civicrm.org/dev/core/-/issues/2557Add hook support for Report tabs.2023-08-06T05:03:24ZyashodhaAdd hook support for Report tabs.Add hook support for Report tabs.Add hook support for Report tabs.yashodhayashodhahttps://lab.civicrm.org/dev/core/-/issues/2551Proposal - add support for defining 'fields' within json blobs2023-07-29T05:03:23ZeileenProposal - add support for defining 'fields' within json blobsThis covers a discussion at https://chat.civicrm.org/civicrm/pl/pt4nizctz7db8xecrq1triy38e
whereby we have a few entities with configuration options that can't be adequately captured in fields. We sort of make this work with payment_pro...This covers a discussion at https://chat.civicrm.org/civicrm/pl/pt4nizctz7db8xecrq1triy38e
whereby we have a few entities with configuration options that can't be adequately captured in fields. We sort of make this work with payment_processor by having fields that we kinda rename via the payment_processor_type table and as long as there are only 4 fields it's fine. However, it's not a great structure.
I'm working on a monolog extension and like payment processor and geocoders the configuration options desired by the service varies - but it's not 100% open ended - ie we don't want to just pass through 'whatever'. The best data model to store this is a json blob- however, we then have no UI knowledge of what would go in there and afform is unable to present this field in a UI-friendly way
We looked at the idea of having something like
```
<field>
<name>contact_type</name>
</field>
<field>
<name>contact_sub_type</name>
</field>
<field>
<name>options</name>
<type>text</type>
<serialize>json</serialize>
<schema-callback>CRM_My_Schema::getFields</schema-callback>
<schema-watch>contact_type,contact_sub_type</schema-watch>
</field>
```
Where the UI layer would need to implement the ability to watch the fields defined as schema-watch and update the UI based on it (presumably only for fields that are ALSO already displayed)
A simple xml definition of subfields might look like https://gist.github.com/totten/84c140310d4adc9b5d3842b5d3364a9e - which could be rendered by a generic function.
@totten @seamuslee - anything else you think is important to transfer over here?https://lab.civicrm.org/dev/core/-/issues/2545Free memberschip sign up (eg. prayerteam) assumes contribution involved2023-07-29T05:03:22Zmagnolia61Free memberschip sign up (eg. prayerteam) assumes contribution involvedOverview
----------------------------------------
Managing memberships in civicrm currently pretty much assumes a contribution is involved. Which is not the case for many forms of memberships. It is even not possible to create a membersh...Overview
----------------------------------------
Managing memberships in civicrm currently pretty much assumes a contribution is involved. Which is not the case for many forms of memberships. It is even not possible to create a membership sign up page without using a contribution page!
Right now for our summercamps it's pretty strange for people to sign up to become a member of the prayerteam to "review their contribution". Although technically they give their contribution in the form of prayer one could argue.
Example use-case
----------------------------------------
1. Create a membership sign up page in CiviCRM for a free membership
2. Look at the wording of the sign-up button
Current behaviour
----------------------------------------
The current button to sign up for a membership says 'review your contribution'.
Proposed behaviour
----------------------------------------
The button text here should be editable, or for signing up for free memberships a different default wording should be used.
Comments
----------------------------------------
Consistent UI would be I guess to be able to create a membership sign up page from the membership menu. Underwater this could be a contribution page with or without fees involved.https://lab.civicrm.org/dev/core/-/issues/2544Move Contact Layout Editor to a core extension2023-07-27T05:03:20ZcolemanwMove Contact Layout Editor to a core extensionAt some point, it would be good to get rid of the hard-coded contact summary screen, and just let the Contact Layout Editor manage the display.
As an intermediary step, it probably makes sense to move Contact Layout Editor into the core...At some point, it would be good to get rid of the hard-coded contact summary screen, and just let the Contact Layout Editor manage the display.
As an intermediary step, it probably makes sense to move Contact Layout Editor into the core tarball (via distmaker import), or the `ext/` directory of the core repo.
Today might not be the day for that, but logging some notes on it anyway:
- Today, the only benefit of moving the extension into core is it would be more visible to end-users.
However, in the future, benefits might include:
- Easier simultaneous development of extension & core with no version conflicts.
- Possible to enable the extension by default.
- Possible to rip out redundant functionality from core.
Challenges
- There isn't an easy upgrade path when an extension is moved into core; you end up with 2 copies of the extension
- Currently, the extension scanner does not give the core `ext/` directory top priority, which means older versions of the extension outside core would take precedence.
Considerations
- As handy as it is, Contact Layout Editor still uses smarty to render the page.
- Maybe the end-game is to make that page an Afform instead. However, that day is a long way off given Afform's current capabilities.https://lab.civicrm.org/dev/core/-/issues/2540Print report permissions2022-06-02T18:01:45ZDevAppPrint report permissionshttps://civicrm.stackexchange.com/questions/22261/how-to-hide-configuration-tabs-of-a-report-with-the-print-to-pdf-action-remain
But when disabling the “access Report Criteria” permission, the "Print report" and "Print to PDF" functions...https://civicrm.stackexchange.com/questions/22261/how-to-hide-configuration-tabs-of-a-report-with-the-print-to-pdf-action-remain
But when disabling the “access Report Criteria” permission, the "Print report" and "Print to PDF" functions on a report appear, but do not function.
The user is just redirected back to the same report.
The functionality appears broken, as the report gives the option to print but it does not work without the additional permission.
Possible solutions:
- Give permissions to print if access report permissions are already granted and the user can view a report
- Add a new print permission and only show the print option if enabled for user
- Hide existing print options if view report criteria not selected, as the user can't print5.51.0Monish DebMonish Debhttps://lab.civicrm.org/dev/core/-/issues/2508Improve unit tests by checking if session::setStatus sets errors2023-07-20T05:03:20ZDaveDImprove unit tests by checking if session::setStatus sets errorsThe alert popups that come up in civi happen two ways: One is via javascript-only, which is difficult to test in the regular phpunit tests, and the other is via CRM_Core_Session::setStatus().
One approach is as in https://github.com/civ...The alert popups that come up in civi happen two ways: One is via javascript-only, which is difficult to test in the regular phpunit tests, and the other is via CRM_Core_Session::setStatus().
One approach is as in https://github.com/civicrm/civicrm-core/pull/19939 but it's not workable in practice because it generates too much extra visual noise on screen. However it pointed out some tests which could be improved. See also https://github.com/civicrm/civicrm-core/pull/19939#issuecomment-809683859 for some additional details. The last one here about financialtype I think I saw a variation of on-screen once and was just that the type of popup was wrong (i.e. using error instead of success).
```
not ok 599 - Error: CRM_Contact_Form_Task_EmailCommonTest::testPostProcessWithSignature
not ok 603 - Error: CRM_Contact_Form_Task_UseraddTest::testUserCreateFail
not ok 653 - Error: CRM_Contact_Page_View_UserDashBoardTest::testDashboardContentEmptyContact
not ok 654 - Error: CRM_Contact_Page_View_UserDashBoardTest::testDashboardContentContributionsWithInvoicingEnabled
not ok 655 - Error: CRM_Contact_Page_View_UserDashBoardTest::testDashboardContentContributions
not ok 698 - Failure: CRM_Contribute_BAO_ContributionRecurTest::testAutoRenewalWhenOneMemberIsDeceased
not ok 944 - Error: CRM_Core_BAO_AddressTest::testSharedAddressChaining3
not ok 1115 - Error: CRM_Core_BAO_SettingTest::testGetItemGroup_Override
not ok 1116 - Error: CRM_Core_BAO_SettingTest::testDefaults
not ok 1117 - Error: CRM_Core_BAO_SettingTest::testOnChange
not ok 1118 - Error: CRM_Core_BAO_SettingTest::testSetCivicrmEnvironment
not ok 1119 - Error: CRM_Core_BAO_SettingTest::testPseudoConstants
not ok 507 - Error: api_v3_ContactTest::testMergedGetWithPermanentlyDeletedContact
not ok 1382 - Failure: api_v3_JobTest::testProcessMembershipDeceased
not ok 1418 - Failure: api_v3_LoggingTest::testEnableDisableLogging
not ok 1522 - Failure: api_v3_MembershipStatusTest::testDeceasedMembershipInline
not ok 280 - Error: Civi\Test\ExampleHookTest::testPageOutput
fatal error - ext/oauth-client/tests/phpunit/CRM/OAuth/MailSetupTest.php:18
not ok 2 - Failure: Civi\Financialacls\FinancialTypeTest::testChangeFinancialTypeName
```https://lab.civicrm.org/dev/core/-/issues/2505CiviReport does not localize custom fields of type Number2023-02-16T13:30:23ZjaapjansmaCiviReport does not localize custom fields of type Number**How to reproduce**
1. Set localization settings: **decimal separator** to `,` and **thousand separator** to `.`
2. Add a custom group for cases
3. Add a custom field of **Type** `Number` and **Field type** `Text`
4. Enter a value such...**How to reproduce**
1. Set localization settings: **decimal separator** to `,` and **thousand separator** to `.`
2. Add a custom group for cases
3. Add a custom field of **Type** `Number` and **Field type** `Text`
4. Enter a value such as `1,234.56` which is localized `1.234,56`
5. Create a new CiviCase detail report. Check the column to display the custom field.
**Expected result**
Value of the custom field displayed as `1.234,56`
**Actual result**
Value of custom field is displayed as `1234.56` (note the decimal separator and the missing thousand separator).
**Comment**
The display value of custom number fields is also broken on the manage case screen.
We have a function `CRM_Utils_Money::formatLocaleNumericRoundedForDefaultCurrency` to format money fields. However we do not have such a function for number fields with a decimal. So basically nowhere in CiviCRM a number field is formatted according to localization settings. Unless the smarty modifier `|crmNumberFormat` is used in the template.
A fix should probably happen in `CRM_Core_BAO_CustomField::formatDisplayValue`, see https://github.com/civicrm/civicrm-core/blob/master/CRM/Core/BAO/CustomField.php#L1187
However I am not exactly sure how to fix this as I could not find a number formatting function such as we have for the money field. Does such a function exists and if so where do I find it?5.38.0https://lab.civicrm.org/dev/core/-/issues/2504CiviReport does not localize custom date fields when exporting to CSV2021-04-09T13:32:59ZjaapjansmaCiviReport does not localize custom date fields when exporting to CSVCiviReport does not localize custom fields of type Date
**How to reproduce**
1. Set localization settings: **Date Format Complete date** to `%d-%m-%Y`
2. Add a custom group for cases
3. Add a custom field of Type Date and no time.
4. E...CiviReport does not localize custom fields of type Date
**How to reproduce**
1. Set localization settings: **Date Format Complete date** to `%d-%m-%Y`
2. Add a custom group for cases
3. Add a custom field of Type Date and no time.
4. Enter a value such as 31-04-2021 (1st of april 2021)
5. Create a new CiviCase detail report. Check the column to display the custom field.
6. Export the report to CSV
**Expected result**
Value of the custom field displayed as `01-04-2021` as it is shown in the report and according to the localization setting.
**Actual result**
Value of custom field is displayed as `2021-04-01` which is ISO format.
**Comment**
The format for iso format is hard coded in: https://github.com/civicrm/civicrm-core/blob/master/CRM/Report/Utils/Report.php#L270
I am not sure if that is the desired behavior. I dont think so as the custom fields of type money are localized into a CSV export. So I would argue that the date should also be formatted according to the localization settings.https://lab.civicrm.org/dev/core/-/issues/2496Api support for afform contribution pages2023-07-17T05:03:23ZeileenApi support for afform contribution pagesAfform what do we need to do to offer Contribution form
Contribution entity - but just basic fields
Payment block
To do the latter we need an api like
PaymentForm with the actions
```
->submit
->getfields
->validate
```
```payment_pro...Afform what do we need to do to offer Contribution form
Contribution entity - but just basic fields
Payment block
To do the latter we need an api like
PaymentForm with the actions
```
->submit
->getfields
->validate
```
```payment_processor_id ``` would be required for each of these (getfields would reply with metadata that would indicate what additional fields are required)
That should be the minimum for a totally basic processor (like paypal std, paypalpro, dummy, payment express, Mollie, Sagepay recurring in Omnipay, some IATS)
All the complexity comes in when we start adding in the js - e.g Stripe, Paypal checkout, some IATS. I think it makes much more sense to get the easy ones working first - but once they are then for phase 2 we will need at minimum
- An api function to get the JS to add with some details about it
- A js object on the pay to return the total_amount from the form - this is something Matt has been working on & would need to be thought through across forms - eg. CRM.PaymentForm.getAmount() should do the same on the angular form and on the core civicrm forms. We probably need to articulate what other functions it may or may not need
On submission we would do
```
Order.create (this can just be the same params as Contribution.create in the initial submit. At minimum it is basically contribution.create with a forced ‘Pending’ status
PaymentProcessor.pay
Payment.create
```
These exist in apiv3 & I think we would use them for nowhttps://lab.civicrm.org/dev/core/-/issues/2495Proposal - populate all requested smart groups at once2023-07-17T05:03:24ZeileenProposal - populate all requested smart groups at onceThis is tied to https://lab.civicrm.org/dev/core/-/issues/2480 but I wanted to split it off as it is related specifically to the existing load() mechanism and where I think we could tweak existing code slightly to make a function that wo...This is tied to https://lab.civicrm.org/dev/core/-/issues/2480 but I wanted to split it off as it is related specifically to the existing load() mechanism and where I think we could tweak existing code slightly to make a function that would be more efficient to support APIv4
What currently happens
load() is called for each **requested** group in turn.
If you want contacts in 8 different groups is is called 8 times.
(take a moment to realise I'm not talking about populating all groups in this gl - only those groups specifically required for the action)
it follows these steps
1) checks if it has already been loaded within this php process
2) checks if the cache date on the civicrm_group is NULL or longer ago than the smart group cache timeout
3) creates a temp table
4) populates the temp table with the contents of the saved search
5) acquires a mysql lock for the group - if not aquired it returns at this point
6) removes records for those groups from the group_contact_cache table
7) inserts rows from the temp table into the group_contact_cache table
8) drops the temp table
9) updates the group contact cache table.
10) releases the lock
What I think we could do is have a new function
```
public function loadGroups($groupIds) {
}
```
which would start with ids as a list of groups to resolve but
1) filter out any ids already loaded in this php process
2) filter out any groups with expired caches
3) attempt to acquire locks on each group, filter out any groups for which a lock cannot be aquired.
3) create a temp table
4) populate the temp table with the contents of each of the filtered groups (one by one still - we could test UNION later on if we want - out of scope)
5) remove records from the group contact table for all the groups
6) insert rows from the temp table for all the groups
7) drop the temp table
8) update the group contact cache table for all groups
9) release all the locks
While this is potentially less php cycles the thing I'm really trying to get to is less inserts into the civicrm_group_contact cache table as these can clash with each other or with delete actions. There is a risk that the number of rows inserted at once would be too many (although this already exists & it's unclear whether this WOULD actually make it worse) - we might iterate through the temporary table only inserting say 50k rows per query if this turns out to be an issue in testing.
I think @mattwire developed the process whereby we build a temp table first and then insert and I believe it has been a big improvement. I think this would get us further. I would want to focus on testing / implementing it in apiv4 rather than everywhere but api v4 could then figure out what groups it wants and request that they be loaded before joining onto the group_contact_cache table (in practice
LEFT JOIN civicrm_group_contact_cache UNION civicrm_group_contact if not all groups are smart groups).
I've also been pondering the (out of scope!!!) idea of really splitting out the temporary table building from the insert into smart group contact cache - perhaps the table could be durable & the query could use the temporary table itself and we could store the temporary table name & timestamp and somehow fire off a non-browser process to reap it into the group_contact_cache and drop the table.
@pfigel @colemanw @seamuslee @mattwire @totten @bgm @sluc23 @BjoernE