Development issueshttps://lab.civicrm.org/groups/dev/-/issues2022-11-11T02:07:08Zhttps://lab.civicrm.org/dev/core/-/issues/3977Payment processor handling of `billing-country-5` inconsistent2022-11-11T02:07:08ZeileenPayment processor handling of `billing-country-5` inconsistentThis is a follow on from https://lab.civicrm.org/dev/core/-/issues/3918 / https://github.com/civicrm/civicrm-core/pull/24232 / https://github.com/civicrm/civicrm-core/pull/24903
The original PR https://github.com/civicrm/civicrm-core/pu...This is a follow on from https://lab.civicrm.org/dev/core/-/issues/3918 / https://github.com/civicrm/civicrm-core/pull/24232 / https://github.com/civicrm/civicrm-core/pull/24903
The original PR https://github.com/civicrm/civicrm-core/pull/24903 did not explain what problem was being solved & it didn't come out in the review process.... so I think in the first instance we should establish that.
This is my best guess to reverse engineer the point of the original change ...
All calls to `doPayment` are expected to pass through the documented parameters https://docs.civicrm.org/dev/en/latest/extensions/payment-processors/create/#billing-fields - in this case the relevant parameter is `billing_country-5`. That parameter is a bit of a pig so some/ most code ALSO passes `country` - and this seems to be done consistently enough that `Paypal` and `Authorize.net` (our oldest processors) are using `country`.
I do think it is important that when payment processors are sending out an array the contract of what keys they should pass out is clear. If we want to deprecate `billing_country-5` in favour of 'country' that seems OK (we would need to update the docs) - but at the moment there seems to be some code that passes `country` in a format that is not correct - which led to the regression.
I feel like in the first instance we should find & fix (with tests) the reported instances of that that came up in https://lab.civicrm.org/dev/core/-/issues/3918
Looking at the code for `setBillingCountry` and the validation - I have a feeling that code may only be hit from the unit tests? In which case the case for reducing the validation seems strong enough.
One thing we COULD do in the test suite is register a hook on all unit tests for `alterPaymentProcessor` & validate the `country` field in it - that might track down any errant ones
I should note that `PropertyBag` isn't a subsitute for us fixing core code to pass the contract parameters5.56.0https://lab.civicrm.org/dev/core/-/issues/3975Search kit does not refresh correctly on delete2022-11-27T23:27:08ZeileenSearch kit does not refresh correctly on deleteTo reproduce - start with the below search (I don't think the search is 'special' - just where I hit it)
- go to the LAST page
- Note the number of rows in the total
- ![image](/uploads/a4ad196fbacefe1b79a7d4cb6aa1ede9/image.png)
- use ...To reproduce - start with the below search (I don't think the search is 'special' - just where I hit it)
- go to the LAST page
- Note the number of rows in the total
- ![image](/uploads/a4ad196fbacefe1b79a7d4cb6aa1ede9/image.png)
- use an action to delete the last record
- Note the new count
- ![image](/uploads/26b3429dab37d1c4baf02923464b3b4a/image.png)
```
[
[
"SavedSearch",
"save",
{
"records": [
{
"name": "languages",
"label": "languages",
"form_values": null,
"mapping_id": null,
"search_custom_id": null,
"api_entity": "OptionValue",
"api_params": {
"version": 4,
"select": [
"id",
"label",
"value",
"name"
],
"orderBy": [],
"where": [
[
"option_group_id:name",
"=",
"languages"
]
],
"groupBy": [],
"join": [],
"having": []
},
"expires_date": null,
"description": null
}
],
"match": [
"name"
]
}
]
]
```colemanwcolemanwhttps://lab.civicrm.org/dev/core/-/issues/3974PHP8: fatal error when viewing event info page with OpenStreetMaps as mapping...2022-11-07T23:25:39ZJonGoldPHP8: fatal error when viewing event info page with OpenStreetMaps as mapping provider and a location with no geocode### Replication Steps
* On PHP 8.1, configure Civi with OpenStreetMaps as the Mapping Provider, but no (working) geocoding provider. On a civibuild site, setting *Geocoding Provider* to **Google** will do the trick.
* Edit an event to c...### Replication Steps
* On PHP 8.1, configure Civi with OpenStreetMaps as the Mapping Provider, but no (working) geocoding provider. On a civibuild site, setting *Geocoding Provider* to **Google** will do the trick.
* Edit an event to create a new address (which won't have a geocode since geocoding is disabled).
* Configure the event to show a map.
* Go to the Event Info page.
### Expected Result
No map, because no geocode.
### Actual Result
Fatal Error.
### Comments
This happens because of this line in `<civiroot>/templates/CRM/Contact/Form/Task/Map/OpenStreetMaps.tpl`:
```
{if count($locations) gt 1}
```
It expects `$locations` to always be set.
I imagine an alternative solution is to add an `isset()` to the template. I can do the other fix if preferred though.5.57.0JonGoldJonGoldhttps://lab.civicrm.org/dev/core/-/issues/3973SearchKit: be able to choose *which* Actions are available on a display2023-03-02T16:50:05ZherbdoolSearchKit: be able to choose *which* Actions are available on a displayThe Actions for SearchKit displays seem to be automatically chosen based on the entities. And it's an all or nothing deal. In my case, I was hoping to allow staff to *only* download the results and nothing else. But there a bunch that sh...The Actions for SearchKit displays seem to be automatically chosen based on the entities. And it's an all or nothing deal. In my case, I was hoping to allow staff to *only* download the results and nothing else. But there a bunch that show up, including delete, update. Even if the search is using a GROUP BY, which doesn't make much sense I think.https://lab.civicrm.org/dev/release/-/issues/19ESR Workflow: Post full suite of resources at start of cycle2023-01-13T18:46:04ZtottenESR Workflow: Post full suite of resources at start of cycleBackground / Motivation
-----------------------
Every 6 months, we designate a version for extended security support. When there's a new security release, the ESR version is posted to Gitlab (`esr/core`, `esr/packages`, `esr/drupal-8`, ...Background / Motivation
-----------------------
Every 6 months, we designate a version for extended security support. When there's a new security release, the ESR version is posted to Gitlab (`esr/core`, `esr/packages`, `esr/drupal-8`, and so on -- collectively, the `esr/*` projects).
If you are sysadmin who wants to be _prepared_ for future security updates, then you would look to `esr/*` and do a trial-run. Make sure you have access/credentials. Make sure you can find the spot for the releases. Make sure you're targeting the right version. Make sure your tools (like `composer`) are configured to pull from there.
However, if there hasn't been a security release, then there may not be anything published on `esr/*`. This interim period invites some confusion. On one hand, there's no security-release to publish -- so the regular security-release scripts don't apply. On the other hand, if you want to be ready, then you need something to look at. The result is that we improvise during the interim.
Goal
----
Whenever we start new ESR cycle, immediately publish to `esr/*`. Include sufficient resources that enable the sysadmins to do a "dress rehearsal" (downloading/install using the exact same procedure as for a security fix). For example, suppose we're discussing v5.51 which started its extended support circa Aug 3. Here are some materials that you might expect to find (circa Aug 3):
* In the main repo, the tarballs for the `5.51.x` releases
* In each repo, the branch (`5.51-esr`).
* In each repo, the tags for specific patch releases (`5.51.x` or `5.51.x+esr`).
* In each repo, package-registry entries for each release (`5.51.x` or `5.51.x+esr`).
* In the main repo, links for the documentation interim changes (release announcements/notes for 5.51, 5.50, 5.49, 5.48, 5.47, 5.46)
Questions
----------
Are there other resources that you expect in this period?
If 5.51 starts public-support in July and starts extended-support in August... then... when should 5.51 become available in `esr/*`? Start of July? Start of August?
(If 5.51 is published to `esr/*` in July, then it will overlap with the tail-end of 5.45. Would it be confusing to see both during July?)https://lab.civicrm.org/dev/core/-/issues/3967Deduping with multiple fields doesn't work with more than 999 contacts using ...2022-11-04T19:17:46ZlarsssandergreenDeduping with multiple fields doesn't work with more than 999 contacts using MariaDB 10.3+After upgrading to MariaDB 10.3 or later, finding dupes with a rule than uses multiple fields creates queries that runs for hours when checking 1000 contacts, but works fine for 999 contacts.
Here are the steps to replicate:
- MariaDB 1...After upgrading to MariaDB 10.3 or later, finding dupes with a rule than uses multiple fields creates queries that runs for hours when checking 1000 contacts, but works fine for 999 contacts.
Here are the steps to replicate:
- MariaDB 10.3+
- Create a dedupe rule with First Name and Last Name
- Use the rule on a group with 1000 or more contacts (or use the Deduper extension to limit to 1000 contacts)
- Result: query runs for a very long time
- Use the rule on a group with 999 or fewer contacts
- Result: results returned quickly as expected
Turns out this is due to a setting that was added to MariaDB in 10.3: [In Predicate Conversion Threshold](https://mariadb.com/docs/reference/mdb/system-variables/in_predicate_conversion_threshold/), which converts the long list of ids in the query that CiviCRM builds to a temp table when the length of the list of ids exceeds the setting, which is 1000 by default. Changing the setting to 0 fixes the issue by disabling the conversion.
Not clear why the temp table is so slow. I'm going to try creating temp tables with indexes explicitly to see if that helps. If not and nothing else suggests itself as a solution, will at least add some documentation unless others have thoughts on how to resolve this so it doesn't require adjusting settings for everyone using MariaDB.https://lab.civicrm.org/dev/backdrop/-/issues/76The control buttons in crm pop ups are all X's2022-11-10T17:27:08ZalicefruminThe control buttons in crm pop ups are all X'sThe icons in crm-pop-up title bars in backdrop all display as X's see screenshot below:
![allexes](/uploads/f29652a34e9b7d1048ea33507031615c/allexes.png)
They should display as whatever font awesome icon is assigned to them usually so...The icons in crm-pop-up title bars in backdrop all display as X's see screenshot below:
![allexes](/uploads/f29652a34e9b7d1048ea33507031615c/allexes.png)
They should display as whatever font awesome icon is assigned to them usually something like print, minimize, maximize or close see screenshot below:
![rightIcons](/uploads/1de63b46ded997951c0a0b5e8f7e49b5/rightIcons.png)5.55.2https://lab.civicrm.org/dev/core/-/issues/3965Adding mailing events (unsub, open, clicks, etc) to API42023-10-16T00:28:23ZlarsssandergreenAdding mailing events (unsub, open, clicks, etc) to API4I'd like to add mailing events to API4 so we can use them in SearchKit, to improve the A/B Mailing report page, and so on.
This would be all the [Mailing Events here.](https://github.com/civicrm/civicrm-core/tree/master/CRM/Mailing/Even...I'd like to add mailing events to API4 so we can use them in SearchKit, to improve the A/B Mailing report page, and so on.
This would be all the [Mailing Events here.](https://github.com/civicrm/civicrm-core/tree/master/CRM/Mailing/Event/DAO) Not sure they are all essential, but might as well do them all at once, as they will all be very similar.
Having poked around at this, I see a few issues that I think I need some help with before starting on this.
We have, for example, TrackableURLOpen, which links to a TrackableURL and also to the Queue entity, which links to the Job entity, which links to the Mailing entity. I don't think we want to have users building queries with joins on all of these entities in SearchKit in order to get useful information back. Ideally, we'd have an entity that has a get action that would return:
- id from TrackableURLOpen
- timestamp from TrackableURLOpen
- URL from TrackableURL
- contact id from Queue
- mailing id from Job
Can we do this by adding all these entities to the API, adding entity bridges to connect them all and then adding an abstract entity that wraps everything up together, with a get function that gets all the fields above from the API (and a getfields function as well). Or is there a less manual way to do this?
Also, I see that the Queue entity already exists in API4, but it isn't the Mailing_Event_Queue entity. Is there a way to specify the full class for an entity so that we don't get a collision? It looks like it just expects the last word from the classname as the API class and that won't work in this case. This might be helpful in general here, because adding an entity called just Opened is going to be confusing (versus naming it something like MailingEventOpened).https://lab.civicrm.org/dev/core/-/issues/3963Mix of auto renewing membership types and non-auto renewing membership types ...2024-03-15T21:08:28ZEdselopezMix of auto renewing membership types and non-auto renewing membership types not handled properly in pricesets.Overview
----------------------------------------
On a membership price set, it is possible to specify a mix of auto renewing memberships as well as non auto renewing memberships, but allow the selection of only one at a time (by virtue ...Overview
----------------------------------------
On a membership price set, it is possible to specify a mix of auto renewing memberships as well as non auto renewing memberships, but allow the selection of only one at a time (by virtue of it being a radio select). It seems CiviCRM doesn't know how to handle this properly, as it assumes that multiple options can be selected.
The code here https://github.com/civicrm/civicrm-core/blob/master/CRM/Price/BAO/PriceSet.php#L1290 seems to fetch the HTML type (which should be the deciding factor on if the payment processor is compatible with handling multiple concurrent transactions) but doesn't really do anything with it.
Reproduction steps
----------------------------------------
1. Create a membership price set
2. Add a price field of type radio
3. Proceed to create selections having membership types selected, but be sure to include a mix of auto renewing membership types and non-auto renewing membership types
4. Add the price field on a contribution page and click save.
Current behaviour
----------------------------------------
(I have tested this for PayPal - Website Payments Standard, but it should be replicable for any payment processor which doesn't support "MultipleConcurrentPayments".
Error shown when trying to save
"Price Set
The membership price set associated with this online contribution allows a user to select BOTH an auto-renew AND a non-auto-renew membership. This requires submitting multiple processor transactions, and is not supported for one or more of the payment processors enabled under the Amounts tab."
Expected behaviour
----------------------------------------
The contribution form should recognize that the price field is a single select field, and therefore allow me to save the configuration.
Environment information
----------------------------------------
* __Browser:__ _Firefox 59.0.1/Chrome 78.0.3904/Safari 13_
* __CiviCRM:__ _Master/5.20.0/5.19.1/5.18.2/..._ <!-- If this problem relates to an upgrade, then specify both old and new versions -->
* __PHP:__ _7.0/7.1/7.2/7.3/...__
* __CMS:__ _Backdrop 1.5/Drupal 7.30/Joomla 3.3/WordPress 4.5/..._
* __Database:__ _MySQL 5.7.7/MariaDB 10.4/..._
* __Web Server:__ _Apache 2.4/Nginx 1.16/..._5.69.5seamusleeseamusleehttps://lab.civicrm.org/dev/core/-/issues/3962Bool token formatting2022-11-18T14:27:07ZeileenBool token formattingI hit an issue today where we were using tokens in smarty. We already have this format in several message templates - e.g
`{if '{contact.current_employer}'} ({contact.current_employer}){/if}`
The single quotes are necessary as `{if } ...I hit an issue today where we were using tokens in smarty. We already have this format in several message templates - e.g
`{if '{contact.current_employer}'} ({contact.current_employer}){/if}`
The single quotes are necessary as `{if } {/if} ` will fail - so it renders to `{if ''} {/if} `
However, the problem we hit is if the employer is `O'Malley Construction'` which renders as `{if 'O'Malley Construction'} {/if} `
I think the solution is to add a bool modifier & allow any token to be passed through it - it would also simplify the smarty a little - ie
`{if {contact.current_employer|bool}} ({contact.current_employer}){/if}`5.57.0https://lab.civicrm.org/dev/core/-/issues/3956SearchKit: Tagging a saved search adds strange "Include tags used for Saved S...2022-10-30T13:47:08ZlarsssandergreenSearchKit: Tagging a saved search adds strange "Include tags used for Saved Searches" search option to Advanced Search![image](/uploads/7472f315ef96fdc3d5c3c0ef829eb893/image.png)
Steps to reproduce on dmaster:
- Save a SK search with a tag
- This odd search option shows up in Advanced Search![image](/uploads/7472f315ef96fdc3d5c3c0ef829eb893/image.png)
Steps to reproduce on dmaster:
- Save a SK search with a tag
- This odd search option shows up in Advanced Searchhttps://lab.civicrm.org/dev/core/-/issues/3954sybunt/lybunt default for "this year" is 20122022-10-28T22:40:31ZDaveDsybunt/lybunt default for "this year" is 2012As much as we all want to forget the last few years happened, defaulting back to 2012 might be too far.
Note that in sample data it used to be hardcoded in a saved instance as 2011 but now that the 10 year window in the dropdown has pas...As much as we all want to forget the last few years happened, defaulting back to 2012 might be too far.
Note that in sample data it used to be hardcoded in a saved instance as 2011 but now that the 10 year window in the dropdown has passed that's now effectively ignored, so can close https://lab.civicrm.org/dev/core/-/issues/1091
But in regular installs it's a bit silly.
The problem though is the 'default' parameter doesn't seem to get used. It's correctly set to date('Y'). I dunno why it doesn't work yet.https://lab.civicrm.org/dev/core/-/issues/3953Fatal error after upgrade to 5.54.0 and enabling Authx2022-11-01T22:19:44ZkcristianoFatal error after upgrade to 5.54.0 and enabling AuthxI want to log this as it was very puzzling, but could be site specific.
Upgraded from CiviCRM 5.51.3 to 5.54.0 on WP 5.9.x and php 7.4 on a Multi-Site.
Also updated Mosaico from 2.9.x to 2.10
After Upgrade CiviCRM said Authx needed to...I want to log this as it was very puzzling, but could be site specific.
Upgraded from CiviCRM 5.51.3 to 5.54.0 on WP 5.9.x and php 7.4 on a Multi-Site.
Also updated Mosaico from 2.9.x to 2.10
After Upgrade CiviCRM said Authx needed to be enabled, so I did that.
Once I did that on the 'sub-sites' all non admins could not load New Mailings, Contribution Dashboard, Advanced search.
Error was
```
$Fatal Error Details = array:3 [
"message" => "The "cache.decendantGroups" service or alias has been removed or inlined when the container was compiled. You should either make it public, or stop using the container directly and use dependency injection instead."
"code" => null
"exception" => Symfony\Component\DependencyInjection\Exception\ServiceNotFoundException {#3618
-id: "cache.decendantGroups"
-sourceId: null
-alternatives: []
#message: "The "cache.decendantGroups" service or alias has been removed or inlined when the container was compiled. You should either make it public, or stop using the container directly and use dependency injection instead."
```
I was able to grant two new poermissions to the user's role
```
AuthX: Authenticate to services with password
AuthX: Authenticate to services with API key
```
That seems to have fixed the issue - but I don't understand what really happened here.
Will try and reproduce and report back.https://lab.civicrm.org/dev/core/-/issues/3951AdminUI: Edit and Delete buttons don't work for Financial Types2022-10-28T02:27:08ZlarsssandergreenAdminUI: Edit and Delete buttons don't work for Financial TypesIn the new AdminUI screen for managing Financial Types, the Edit and Delete buttons don't work - they just bring up the same screen in a pop-up.
This was just [merged recently.](https://github.com/civicrm/civicrm-core/pull/24715)
Teste...In the new AdminUI screen for managing Financial Types, the Edit and Delete buttons don't work - they just bring up the same screen in a pop-up.
This was just [merged recently.](https://github.com/civicrm/civicrm-core/pull/24715)
Tested on dmaster (5.56).https://lab.civicrm.org/dev/core/-/issues/3950SearchKit: Campaign search doesn't work correctly2022-11-16T20:07:08ZlarsssandergreenSearchKit: Campaign search doesn't work correctlyIf you search for a contribution or a participant by campaign in SK, there are two options: Campaign Campaign Title and Campaign ID. If you search by Campaign Campaign Title, you can enter the campaign title by typing it in manually, but...If you search for a contribution or a participant by campaign in SK, there are two options: Campaign Campaign Title and Campaign ID. If you search by Campaign Campaign Title, you can enter the campaign title by typing it in manually, but there is no autocomplete. If you search by Campaign ID, then there is autocomplete, but the search does not actually work (shows no results because it is doing campaign_id:name when it should be doing campaign_id).
I think there should be just one Campaign search option, with autocomplete that also shows you the most recent active campaigns in the select for quickly picking a recent campaign.
Tested on dmaster (5.56).https://lab.civicrm.org/dev/core/-/issues/3949SearchKit: no JOIN available for ContactReference custom fields2023-05-11T04:35:35ZherbdoolSearchKit: no JOIN available for ContactReference custom fieldsI have a ContactReference custom field and an EntityRef field defined in XML schema on an entity. [Update: both are storing a single contact id] In SK I'd like to add a join to the Contact entity via the custom field. There is one availa...I have a ContactReference custom field and an EntityRef field defined in XML schema on an entity. [Update: both are storing a single contact id] In SK I'd like to add a join to the Contact entity via the custom field. There is one available for the EntityRef field, but not ContactReference custom field. I can make the join in API4 just fine.
This looks like a missing piece of functionality. Or I'm just not looking in the right place.https://lab.civicrm.org/dev/core/-/issues/3944SearchKit: Show count on both top and bottom of display2022-10-27T20:47:07ZlarsssandergreenSearchKit: Show count on both top and bottom of displayCurrently, the count of results in a SearchKit display is only shown on the bottom of the page. In core, totals are usually shown at the top and bottom of the page in search results. I think it would be more usable to also show the total...Currently, the count of results in a SearchKit display is only shown on the bottom of the page. In core, totals are usually shown at the top and bottom of the page in search results. I think it would be more usable to also show the total count in SearchKit displays at the top of the page, beside the Actions drop down menu. It's a good sanity check for users to be able to see how many entities they are doing an action on before doing so - they shouldn't have to scroll to the bottom of the page and then back up to see the count. It's also not always easy to see how many are selected or if you've selected just one page of results or the whole list.
Something simple like this, which would also show NNN selected of NNN results when some are selected.
![image](/uploads/0587e86d7faed05a9ecf441a64f93378/image.png)
This could be disabled by disabling show count for the display.https://lab.civicrm.org/dev/core/-/issues/3942Fatal Error in PHP 8.0 when passing an empty array to a custom checkboxes/sel...2023-01-29T06:17:46ZherbdoolFatal Error in PHP 8.0 when passing an empty array to a custom checkboxes/select field.When passing an empty array to a custom Checkboxes (or Select) field, under PHP 8.0 it gives a fatal error: `TypeError: mysqli_real_escape_string(): Argument #2 ($string) must be of type string, array given in mysqli_real_escape_string()...When passing an empty array to a custom Checkboxes (or Select) field, under PHP 8.0 it gives a fatal error: `TypeError: mysqli_real_escape_string(): Argument #2 ($string) must be of type string, array given in mysqli_real_escape_string() (line 880 of /app/vendor/pear/db/DB/mysqli.php).` There is no error in PHP 7.4.
Using CiviCRM 5.54.0.
For example:
```
$results = Address::update()
->addValue(Address_status.Fruit, [])
->addWhere('id', '=', 1)
->execture();
```
Where Address_status is the custom field group and Fruit is the checkboxes or select field with options.
I am assuming that either the error should be caught earlier by API4 or DAO and warn that empty arrays cannot be passed to this field type, or that it doesn't complain at all and converts the empty array to a serialized string, much like it does with populated arrays.https://lab.civicrm.org/dev/core/-/issues/3941CiviCRM fatal error on enabling `civiimport`2022-11-11T02:30:30ZeileenCiviCRM fatal error on enabling `civiimport`On master I firstly did an import and then enabled `civiimport` through the UI (on an otherwise freshly built site). On refresh it loads...
Note we should fix in 5.54 to ensure sites that enable the extension do not hit issues
![image]...On master I firstly did an import and then enabled `civiimport` through the UI (on an otherwise freshly built site). On refresh it loads...
Note we should fix in 5.54 to ensure sites that enable the extension do not hit issues
![image](/uploads/ca13d998ee3830566d7282ba55f79baa/image.png)
Note the backtrace
{{image.png}}5.55.0https://lab.civicrm.org/dev/core/-/issues/3938SearchKit - arithmetic not working2022-10-30T16:27:08ZvitiusSearchKit - arithmetic not workingWhen you have custom field that is number or integer, arithmetic will not work.
How to reproduce this issue:
1) Create custom set on contact
2) Create custom field type integer name "testinteger"
3) Create search kit on Contact
4) Add "t...When you have custom field that is number or integer, arithmetic will not work.
How to reproduce this issue:
1) Create custom set on contact
2) Create custom field type integer name "testinteger"
3) Create search kit on Contact
4) Add "testinteger" field
5) In field transformation change "testinteger" to arithmetic like on picture bellow
6) Now when you click on search, you will not get any result
![image](/uploads/2861db81768ab80206b30ed1b20db46d/image.png)