Development issueshttps://lab.civicrm.org/groups/dev/-/issues2023-10-24T05:03:26Zhttps://lab.civicrm.org/dev/core/-/issues/2045Error: "Could not find valid value for u"2023-10-24T05:03:26ZTony Maynard-SmithError: "Could not find valid value for u"Overview
----------------------------------------
Since upgrading to 5.28.2 have been seeing a number of Error messages in the log
![image](/uploads/e3502b4c0b5c03442b9692850f15ce08/image.png)
Reproduction steps
-----------------------...Overview
----------------------------------------
Since upgrading to 5.28.2 have been seeing a number of Error messages in the log
![image](/uploads/e3502b4c0b5c03442b9692850f15ce08/image.png)
Reproduction steps
----------------------------------------
Go to https://mysite.com/civicrm/mailing/url
Current behaviour
----------------------------------------
On 5.28.2:
- If Anonymous User get the error message on a blank white screen.
- If logged in as Administrator get the error in an Error box.
- If switch on Trace on Error get:
![image](/uploads/91bdda40068438722d74fc6d59f6c917/image.png)
Expected behaviour
----------------------------------------
Not quite sure, but 5.24.4 gives:
- If Anonymous get standard Access Denied message - seems correct.
- If Admin get a list of Mailings. Not sure what should happen here.
Environment information
----------------------------------------
- CiviCRM 5.28.2
- Drupal 7.72
- PHP 7.2.33
Comments
----------------------------------------
Also on StachExchange https://civicrm.stackexchange.com/questions/37657/what-is-error-could-not-find-valid-value-for-uhttps://lab.civicrm.org/dev/core/-/issues/2044Proposal apiv4 - revisit required parameters on location entities2020-10-02T01:41:10ZeileenProposal apiv4 - revisit required parameters on location entitiesI propose we adjust the location api required fields inn v4api
- Address - don't require contact_id, make location_type_id required in xml
- Email - don't require contact_id, make location_type_id required in xml
- Phone - don't require...I propose we adjust the location api required fields inn v4api
- Address - don't require contact_id, make location_type_id required in xml
- Email - don't require contact_id, make location_type_id required in xml
- Phone - don't require contact_id, make location_type_id required in xml
For all these entities we have a valid use case for no contact_id in the context of location blocks for addresses. I wanted to cleanup the code in ManageLocation around addresses https://github.com/civicrm/civicrm-core/pull/18488 & found I can't use Address::save to do in a better way due to the above
@colemanw how far did you go down this path?5.31.0https://lab.civicrm.org/dev/core/-/issues/2043[cq] Do not pass by reference where avoidable2023-05-24T05:03:17Zeileen[cq] Do not pass by reference where avoidableOne of our nastier code legacies is the prevalence of pass-by-reference - many places in code look something like this
```
function doIt(&$params) {
// don't alter $params
}
doIt($params);
```
In order to understand what is happenin...One of our nastier code legacies is the prevalence of pass-by-reference - many places in code look something like this
```
function doIt(&$params) {
// don't alter $params
}
doIt($params);
```
In order to understand what is happening wit $params it is necessary to dig into the function to see if $params is altered in the process.
On an ongoing basis we need to identify places where a variable is passed-by-reference unnecessarily and remove the &
Note that these were added originally for 2 reasons
1) retrofitting php4 compatibility
2) the incorrect belief it was more efficient on memoryhttps://lab.civicrm.org/dev/core/-/issues/2042CiviCRM is creating a lot of orphaned records in civicrm_address with contact...2020-11-02T00:59:55Zjustinfreeman (Agileware)CiviCRM is creating a lot of orphaned records in civicrm_address with contact_id NULL and master_id NULLCiviCRM is creating a lot of orphaned records in civicrm_address with contact_id NULL and master_id NULL. For CiviCRM sites which typically accept **donations**. There have been instances where thousands of orphaned records exist in the ...CiviCRM is creating a lot of orphaned records in civicrm_address with contact_id NULL and master_id NULL. For CiviCRM sites which typically accept **donations**. There have been instances where thousands of orphaned records exist in the database.
**Steps to reproduce**
1. Check the civicrm_address table that no orphaned records exist (contact_id NULL and master_id NULL)
1. Visit a Donation form (Contribution page) as Anonymous user
1. Complete the form and address fields
1. Submit the Donation form to complete the Contribution
1. Check the civicrm_address table for orphaned records
1. Again, repeat the process
1. Check the civicrm_address table for orphaned records
**How to locate the orphaned records**
You can easily locate them by querying:
`SELECT COUNT(*) FROM civicrm_address WHERE contact_id IS NULL`;
And also you can find these orphaned records by excluding addresses created by Event locations.
`SELECT * FROM civicrm_address a
WHERE a.contact_id is null and a.is_billing=1
AND NOT EXISTS
(SELECT id FROM civicrm_loc_block l
WHERE l.address_id=a.id);
`
And also by checking contributions...
`SELECT COUNT(*) FROM civicrm_address a LEFT JOIN civicrm_contribution c ON c.address_id = a.id WHERE a.contact_id IS NULL AND c.id IS NULL;`
Agileware Ref: CIVICRM-1562https://lab.civicrm.org/dev/core/-/issues/2041Plaintext version of email not replacing tokens in some situations2021-09-29T04:57:25ZDaveDPlaintext version of email not replacing tokens in some situationsI'm still trying to nail down the exact situations but here's one way to reproduce:
1. Create an outbound email.
1. Send it to two people in the To field, and one in the Cc.
1. In the html body put {contact.first_name} somewhere.
1. In ...I'm still trying to nail down the exact situations but here's one way to reproduce:
1. Create an outbound email.
1. Send it to two people in the To field, and one in the Cc.
1. In the html body put {contact.first_name} somewhere.
1. In the text body put {contact.last_name} somewhere.
Everything is correct except the text body inserts blank for the token.5.43.0https://lab.civicrm.org/dev/core/-/issues/2039Address extraneous location queries2022-02-01T08:31:54ZeileenAddress extraneous location queriesWhen importing a 'contact with a donation' 18% of the total queries come from [this line of code](https://github.com/civicrm/civicrm-core/blob/d357f225f281a0862b67a60d027c703ed2292f8b/CRM/Core/BAO/Location.php#L73) for [this issue](https...When importing a 'contact with a donation' 18% of the total queries come from [this line of code](https://github.com/civicrm/civicrm-core/blob/d357f225f281a0862b67a60d027c703ed2292f8b/CRM/Core/BAO/Location.php#L73) for [this issue](https://issues.civicrm.org/jira/browse/CRM-5051). I believe the handling in the create classes likely makes it obsolete.
I'm going to dig a bit further on that & put up a PR with the case for removing the line (unless I find a reason not to)
Per https://lab.civicrm.org/dev/core/-/issues/2033
examples - openid queries make up 4% of the total queries! and the line is 18% when including all location entities. This would be higher if it were only Contact.create that were being called
|timestamp|query|seconds|rows found|columns requested|
|----------|-------|-------|-------|-------|
| Location queries | 15/09/20 2:54 | SELECT * FROM `civicrm_openid` WHERE ( ( is_primary = 0 OR is_primary IS NULL ) ) AND ( `civicrm_openid`.`contact_id` = 46011839 ) | 0.000643 | 0 | 0 |
| Location queries | 15/09/20 2:54 | SELECT * FROM `civicrm_openid` WHERE ( ( is_primary = 0 OR is_primary IS NULL ) ) AND ( `civicrm_openid`.`contact_id` = 46011840 ) | 0.000517 | 0 | 0 |
| Location queries | 15/09/20 2:54 | SELECT * FROM `civicrm_openid` WHERE ( ( is_primary = 1 ) ) AND ( `civicrm_openid`.`contact_id` = 46011834 ) | 0.001377 | 0 | 0 |5.31.0https://lab.civicrm.org/dev/core/-/issues/2038When assigning a type to a smart group, empty spaces and commas appear when l...2023-05-06T05:03:21Zfreeform.stephWhen assigning a type to a smart group, empty spaces and commas appear when listed on the manage groups pageOverview
----------------------------------------
Unwanted commas and spaces are appearing under the group type column on the manage groups page.
Example use-case
----------------------------------------
1. Click on **Contacts -> Manage...Overview
----------------------------------------
Unwanted commas and spaces are appearing under the group type column on the manage groups page.
Example use-case
----------------------------------------
1. Click on **Contacts -> Manage Groups**.
1. Create a group and select type "Mailing List", save
Current behaviour
----------------------------------------
- On the manage groups page you now see under group type, "**, Mailing List,**"
- This is only for newly created Smart Groups (it doesn't happen when creating a new static group)
- Occurs even if there's only 1 option available
Proposed behaviour
----------------------------------------
It should simply say "**Mailing List**" without commas or spaces
![screenshot-staging.icvanetwork.org-2020.09.15-17_14_38](/uploads/0132d4de2435e39384383ec9d1fd990a/screenshot-staging.icvanetwork.org-2020.09.15-17_14_38.png)https://lab.civicrm.org/dev/core/-/issues/2037CachedCiviContainer generated with Symfony 4 removes private services2021-11-10T19:48:00ZcantrellnmCachedCiviContainer generated with Symfony 4 removes private servicesOverview
----------------------------------------
When using Symfony 4.4, private services & aliases are removed from the generated `CachedCiviContainer.*.php` file making them missing/unavailable.
Reproduction steps
-------------------...Overview
----------------------------------------
When using Symfony 4.4, private services & aliases are removed from the generated `CachedCiviContainer.*.php` file making them missing/unavailable.
Reproduction steps
----------------------------------------
1. Set up a Drupal 9 site using composer (requires `symfony/dependency-injection` and other Symfony packages with version ^4.4). (I know Drupal 9 isn't officially supported yet, but civicrm-core does claim it can use Symfony ~3.0 || ~4.4 and I think Symfony is the problem here)
1. Follow instructions from "Install CiviCRM on Drupal 8" documentation to add/install CiviCRM.
3. If you look at the `CachedCiviContainer.*.php` file generated in the `templates_c` folder, you can see `$this->privates` and `$this->aliases` are empty arrays, whereas with Symfony 3 there was a 'cache.short' alias and a number of private services that are now missing.
4. Visit `.../civicrm/admin/extensions?reset=1` page and it will break.
Current behaviour
----------------------------------------
When visiting `.../civicrm/admin/extensions?reset=1` page:
![Screen_Shot_2020-09-14_at_4.03.57_PM](/uploads/01249e57312bead90ebeff3cabe1bf76/Screen_Shot_2020-09-14_at_4.03.57_PM.png)
```
The "cache.short" service or alias has been removed or inlined when the container was compiled. You should either make it public, or stop using the container directly and use dependency injection instead.
```
There are probably other pages/actions that try & fail to use one of the services/alias that were removed, but this is the one I found.
Expected behaviour
----------------------------------------
Page should load without error!
Environment information
----------------------------------------
* __CiviCRM:__ _5.29.0_
* __PHP:__ _7.3_
* __CMS:__ _Drupal 9.0.5_
* __Database:__ _MariaDB 10.4_
* __Web Server:__ _Nginx_
Comments
----------------------------------------
I believe Symfony 4 may remove these private services/alias with `RemoveUnusedDefinitionsPass` because they look "unused"? The error message offers a solution, make these services/alias public or "stop using the container directly and use dependency injection instead". I'm not sure what the implications of making them public are, or how much work would be involved in using dependency injection.https://lab.civicrm.org/dev/core/-/issues/2034PayPal payment processor uses deprecated methods, breaking functionality2022-11-26T13:19:24ZAdam WoodPayPal payment processor uses deprecated methods, breaking functionalitySymptoms per issues #760, #1195 and #1007
When a payment is processed via PayPal standard, inappropriate "Cancel Subscription" and "Update Billing Details" links are shown in the confirmation emails and front-end contact dashboard. The...Symptoms per issues #760, #1195 and #1007
When a payment is processed via PayPal standard, inappropriate "Cancel Subscription" and "Update Billing Details" links are shown in the confirmation emails and front-end contact dashboard. These links either do not work or present a form that does nothing.
I can see that in https://github.com/civicrm/civicrm-core/blob/master/CRM/Core/Payment/PayPalImpl.php various fixes have been made for this (e.g. line 651 in cancelSubscriptionURL() and line 669 in isSupported(), and subsequent callback methods themselves) however these appear to have been deprecated in the base class https://github.com/civicrm/civicrm-core/blob/1ff2405992ac45df88425ff400a2c37f6056d938/CRM/Core/Payment.php:
- isSupported() replaced by supports() (line 306)
- cancelSubscriptionURL() replaced by subscriptionURL(... $action) (line 1693)
- Similar for other actions e.g. change billing details
These methods are now used in other areas of the application such as email and form generation, but PayPalImpl.php appears to have not been updated, which causes the broken behaviour.
Side issues observed while testing this (for a new installation of 5.29.0 on Joomla 3.9.8):
- Link to instalments for recurring contributions in front-end user dashboard (if present) is broken - gives permissions / access error
- Instalments text has spurious trailing slash (?)
- Formatting errors when displaying recurrence frequency for recurring contributions in front-end user dashboard
(Perhaps these are separate issue(s)).https://lab.civicrm.org/dev/core/-/issues/2033Performance - query analysis2020-10-13T02:06:15ZeileenPerformance - query analysisUsing the methodology described [here](https://lab.civicrm.org/extensions/systemtools/-/blob/master/README.md) I have analysed the queries to pipe 7 contributions into CiviCRM and analyse all the queries to see if we can eliminate some. ...Using the methodology described [here](https://lab.civicrm.org/extensions/systemtools/-/blob/master/README.md) I have analysed the queries to pipe 7 contributions into CiviCRM and analyse all the queries to see if we can eliminate some. Analysis follows (I will look to address some)
1. This is on a server that uses Redis with associated reduction of queries
2. Main civi api calls called are Contact.create & contribution.create.
3. None of the queries are slow - the focus here is on removing unnecessary queries from high volume code paths
4. All 7 contributions were inserted within the total time of one second. We normally process 2000 donations per code run so reducing 1 query per row saves 2000 per run
5. 18% of the total queries come from [this line of code](https://github.com/civicrm/civicrm-core/blob/d357f225f281a0862b67a60d027c703ed2292f8b/CRM/Core/BAO/Location.php#L73) for [this issue](https://issues.civicrm.org/jira/browse/CRM-5051). I believe the handling in the create classes likely makes it obsolete - https://lab.civicrm.org/dev/core/-/issues/2039
| Type of query | Count |proportion queries that are unnecessary|
|--------------------|-------|-------|
| Activity wrangling #2057 | 42 |high|
| CiviCRM log | 14 |medium|
| greeting queries | 6 |undetermined|
| Initialisation | 24 |low|
| Insert actions | 97 |Low|
| location queries #2039 | 119 |very high|
| metadata | 51 |low|
| pcp #2056 | 14 |high|
| uf match queries #2087 | 14 |medium|
| wmf | 14 |low|
| **Grand Total** | **395** |
Visual https://docs.google.com/document/d/1Min5WxmC8O0_MF4nK3LMTXcVZ-WQU_4k1WSJtMeqLl8/edit?usp=sharing
**Activity wrangling 42**
I haven't dug very far but my sense is this is likely about checking for existing activities linked to the contributions. As these are new contributions these checks might be largely avoidable
Examples
|timestamp|query|seconds|rows found|columns requested|
|----------|-------|-------|-------|-------|
| 15/09/20 2:54 | SELECT * FROM `civicrm_activity_contact` WHERE ( `civicrm_activity_contact`.`activity_id` = 112183025 ) AND ( `civicrm_activity_contact`.`contact_id` = 46011835 ) AND ( `civicrm_activity_contact`.`record_type_id` = 2 ) | 0.000549 | 0 | 0 |
| 15/09/20 2:54 | SELECT * FROM `civicrm_activity_contact` WHERE ( `civicrm_activity_contact`.`activity_id` = 112183025 ) AND ( `civicrm_activity_contact`.`record_type_id` = 1 ) | 0.000606 | 0 | 0 |
| 15/09/20 2:54 | SELECT * FROM `civicrm_activity_contact` WHERE ( `civicrm_activity_contact`.`activity_id` = 112183025 ) AND ( `civicrm_activity_contact`.`record_type_id` = 2 ) | 0.000725 | 0 | 0 |
| 15/09/20 2:54 | SELECT * FROM `civicrm_activity_contact` WHERE ( `civicrm_activity_contact`.`activity_id` = 112183025 ) AND ( `civicrm_activity_contact`.`record_type_id` = 2 ) | 0.000605 | 1 | 1 |
| 15/09/20 2:54 | SELECT * FROM `civicrm_activity_contact` WHERE ( `civicrm_activity_contact`.`activity_id` = 112183025 ) AND ( `civicrm_activity_contact`.`record_type_id` = 3 ) | 0.000636 | 0 | 0 |
**CiviCRM Log = 14**
I split these out from the main insert queries as my feeling is that we don't need to add to civicrm log when logging is enabled - I'm pretty sure there is another GL on this & it's not quite that black and white
Examples
|timestamp|query|seconds|rows found|columns requested|
|----------|-------|-------|-------|-------|
|15/09/20 2:54 |INSERT INTO `civicrm_log` (`entity_table` , `entity_id` , `data` , `modified_id` , `modified_date` ) VALUES ('civicrm_activity' , 112183024 , 'Activity created for source=46011834' , 46011834 , 20200909141546 )| |
**greeting queries = 6**
I haven't done any analysis on these - I suspect we have skipGreeting enabled in our api call so it's only 6
|timestamp|query|seconds|rows found|columns requested|
|----------|-------|-------|-------|-------|
| 15/09/20 2:54 | SELECT contact_a.id as contact_id, contact_a.email_greeting_id as email_greeting_id, contact_a.postal_greeting_id as postal_greeting_id, contact_a.addressee_id as addressee_id, contact_a.addressee_display as addressee_display, contact_a.addressee_custom as addressee_custom, contact_a.email_greeting_display as email_greeting_display, contact_a.email_greeting_custom as email_greeting_custom, contact_a.postal_greeting_display as postal_greeting_display, contact_a.postal_greeting_custom as postal_greeting_custom FROM civicrm_contact contact_a WHERE ( contact_a.id = '46011839' ) LIMIT 0, 25 | 0.000734
| 15/09/20 2:54 | SELECT contact_a.id as contact_id, contact_a.email_greeting_id as email_greeting_id, contact_a.postal_greeting_id as postal_greeting_id, contact_a.addressee_id as addressee_id, contact_a.addressee_display as addressee_display, contact_a.addressee_custom as addressee_custom, contact_a.email_greeting_display as email_greeting_display, contact_a.email_greeting_custom as email_greeting_custom, contact_a.postal_greeting_display as postal_greeting_display, contact_a.postal_greeting_custom as postal_greeting_custom FROM civicrm_contact contact_a WHERE ( contact_a.id = '46011840' ) LIMIT 0, 25 | 0.000913 | 1 | 1 |
**Initialisation queries 24**
These are all pretty necessary
examples
|timestamp|query|seconds|rows found|columns requested|
|----------|-------|-------|-------|-------|
| 15/09/20 2:53 | ( SELECT * FROM civicrm_menu WHERE path in ( 'civicrm' ) AND domain_id = 1 ORDER BY length(path) DESC LIMIT 1 ) UNION ( SELECT * FROM civicrm_menu WHERE path IN ( 'navigation' ) AND domain_id = 1 ) | 0.001571 | 1 | 1 |
| 15/09/20 2:53 | /*!40101 SET NAMES utf8 */ | | | |
| 15/09/20 2:53 | /*!50503 SET NAMES utf8mb4 */ | | | |
| 15/09/20 2:53 | BEGIN | | | |
**Insert actions 97**
This is where the work is done - nothing extraneous in the query list here
example
|timestamp|query|seconds|rows found|columns requested|
|----------|-------|-------|-------|-------|
| 15/09/20 2:54 | INSERT INTO `civicrm_entity_financial_trxn` (`entity_table` , `entity_id` , `financial_trxn_id` , `amount` ) VALUES ('civicrm_financial_item' , 67488679 , 67491096 , 2.35 ) |
**Location queries 119**
These seem to be mainly extraneous. 70 of them come from what appears to be an outdated effort to handle is_primary (including 10 on open id which we don't support). At least some of the remaining 39 appear to be variants of this - this is the lowest hanging fruit
examples
|timestamp|query|seconds|rows found|columns requested|
|----------|-------|-------|-------|-------|
| Location queries | 15/09/20 2:54 | SELECT * FROM `civicrm_openid` WHERE ( ( is_primary = 0 OR is_primary IS NULL ) ) AND ( `civicrm_openid`.`contact_id` = 46011839 ) | 0.000643 | 0 | 0 |
| Location queries | 15/09/20 2:54 | SELECT * FROM `civicrm_openid` WHERE ( ( is_primary = 0 OR is_primary IS NULL ) ) AND ( `civicrm_openid`.`contact_id` = 46011840 ) | 0.000517 | 0 | 0 |
| Location queries | 15/09/20 2:54 | SELECT * FROM `civicrm_openid` WHERE ( ( is_primary = 1 ) ) AND ( `civicrm_openid`.`contact_id` = 46011834 ) | 0.001377 | 0 | 0 |
**Metadata queries 51**
This is less of a concern for us in this context as these (almost) all only ran once and none ran once per contribution. There is scope to improve them though & UI users would probably be helped. We would prefer they used the Redis-backed metadata cache
examples
|timestamp|query|seconds|rows found|columns requested|
|----------|-------|-------|-------|-------|
| 15/09/20 2:54 | SELECT a.id as `id`, a.custom_group_id as `custom_group_id`, a.name as `name`, a.label as `label`, a.data_type as `data_type`, a.html_type as `html_type`, a.default_value as `default_value`, a.is_required as `is_required`, a.is_searchable as `is_searchable`, a.is_search_range as `is_search_range`, a.weight as `weight`, a.help_pre as `help_pre`, a.help_post as `help_post`, a.mask as `mask`, a.attributes as `attributes`, a.javascript as `javascript`, a.is_active as `is_active`, a.is_view as `is_view`, a.options_per_line as `options_per_line`, a.text_length as `text_length`, a.start_date_years as `start_date_years`, a.end_date_years as `end_date_years`, a.date_format as `date_format`, a.time_format as `time_format`, a.note_columns as `note_columns`, a.note_rows as `note_rows`, a.column_name as `column_name`, a.option_group_id as `option_group_id`, a.serialize as `serialize`, a.filter as `filter`, a.in_selector as `in_selector` FROM civicrm_custom_field a INNER JOIN `civicrm_custom_group` `custom_group_id_to_civicrm_custom_group` ON a.custom_group_id = custom_group_id_to_civicrm_custom_group.id WHERE (a.name = "original_currency" OR a.label = "original_currency") AND (custom_group_id_to_civicrm_custom_group.name = "contribution_extra" OR custom_group_id_to_civicrm_custom_group.title = "contribution_extra") LIMIT 25 OFFSET 0 | 0.001292 | 1 | 1 |
| 15/09/20 2:54 | SELECT a.id as `id`, a.custom_group_id as `custom_group_id`, a.name as `name`, a.label as `label`, a.data_type as `data_type`, a.html_type as `html_type`, a.default_value as `default_value`, a.is_required as `is_required`, a.is_searchable as `is_searchable`, a.is_search_range as `is_search_range`, a.weight as `weight`, a.help_pre as `help_pre`, a.help_post as `help_post`, a.mask as `mask`, a.attributes as `attributes`, a.javascript as `javascript`, a.is_active as `is_active`, a.is_view as `is_view`, a.options_per_line as `options_per_line`, a.text_length as `text_length`, a.start_date_years as `start_date_years`, a.end_date_years as `end_date_years`, a.date_format as `date_format`, a.time_format as `time_format`, a.note_columns as `note_columns`, a.note_rows as `note_rows`, a.column_name as `column_name`, a.option_group_id as `option_group_id`, a.serialize as `serialize`, a.filter as `filter`, a.in_selector as `in_selector` FROM civicrm_custom_field a INNER JOIN `civicrm_custom_group` `custom_group_id_to_civicrm_custom_group` ON a.custom_group_id = custom_group_id_to_civicrm_custom_group.id WHERE (a.name = "original_amount" OR a.label = "original_amount") AND (custom_group_id_to_civicrm_custom_group.name = "contribution_extra" OR custom_group_id_to_civicrm_custom_group.title = "contribution_extra") LIMIT 25 OFFSET 0 | 0.001085 | 1 | 1 |
| 15/09/20 2:54 | SELECT a.id as `id`, a.custom_group_id as `custom_group_id`, a.name as `name`, a.label as `label`, a.data_type as `data_type`, a.html_type as `html_type`, a.default_value as `default_value`, a.is_required as `is_required`, a.is_searchable as `is_searchable`, a.is_search_range as `is_search_range`, a.weight as `weight`, a.help_pre as `help_pre`, a.help_post as `help_post`, a.mask as `mask`, a.attributes as `attributes`, a.javascript as `javascript`, a.is_active as `is_active`, a.is_view as `is_view`, a.options_per_line as `options_per_line`, a.text_length as `text_length`, a.start_date_years as `start_date_years`, a.end_date_years as `end_date_years`, a.date_format as `date_format`, a.time_format as `time_format`, a.note_columns as `note_columns`, a.note_rows as `note_rows`, a.column_name as `column_name`, a.option_group_id as `option_group_id`, a.serialize as `serialize`, a.filter as `filter`, a.in_selector as `in_selector` FROM civicrm_custom_field a WHERE (a.name = "opt_in" OR a.label = "opt_in") LIMIT 25 OFFSET 0 | 0.001043 | 1 | 1 |
| 15/09/20 2:54 | SELECT a.id as `id`, a.entity_table as `entity_table`, a.entity_id as `entity_id`, a.account_relationship as `account_relationship`, a.financial_account_id as `financial_account_id` FROM civicrm_entity_financial_account a WHERE (a.entity_id = "9") AND (a.entity_table = "civicrm_financial_type") LIMIT 25 OFFSET 0 | 0.000718 | 6 | 6 |
| 15/09/20 2:54 | SELECT a.id as `id`, a.financial_account_id as `financial_account_id` FROM civicrm_entity_financial_account a WHERE (a.entity_table = "civicrm_option_value") AND (a.entity_id = "6342") LIMIT 1 OFFSET 0 | 0.001156 | 1 | 1 |
| 15/09/20 2:54 | SELECT a.id as `id`, a.financial_account_id as `financial_account_id` FROM civicrm_entity_financial_account a WHERE (a.entity_table = "civicrm_option_value") AND (a.entity_id = "844") LIMIT 1 OFFSET 0 | 0.000697 | 1 | 1 |
**PCP 14 queries**
I haven't done much analysis but I think these should be avoidable on sites without pcps - maybe the same way we no longer have product queries
examples
|timestamp|query|seconds|rows found|columns requested|
|----------|-------|-------|-------|-------|
| 15/09/20 2:54 | SELECT id FROM civicrm_contribution_soft WHERE contribution_id = 49769031 AND pcp_id IS NOT NULL | 0.000624 | 0 | 0 |
| 15/09/20 2:54 | SELECT id FROM civicrm_contribution_soft WHERE contribution_id = 49769031 AND pcp_id IS NULL | 0.000616 | 0 | 0 |
| 15/09/20 2:54 | SELECT id FROM civicrm_contribution_soft WHERE contribution_id = 49769032 AND pcp_id IS NOT NULL | 0.000503 | 0 | 0 |
| 15/09/20 2:54 | SELECT id FROM civicrm_contribution_soft WHERE contribution_id = 49769032 AND pcp_id IS NULL | 0.000562 | 0 | 0 |
**uf match queries - 14**
These are not relevant to the incoming but not sure if they are avoidable - the duplication might be though
|timestamp|query|seconds|rows found|columns requested|
|----------|-------|-------|-------|-------|
| 15/09/20 2:54 | SELECT * FROM `civicrm_uf_match` WHERE ( `civicrm_uf_match`.`domain_id` = 1 ) AND ( `civicrm_uf_match`.`contact_id` = 46011840 ) | 0.000567 | 0 | 0 |
| 15/09/20 2:54 | SELECT * FROM `civicrm_uf_match` WHERE ( `civicrm_uf_match`.`domain_id` = 1 ) AND ( `civicrm_uf_match`.`contact_id` = 46011840 ) | 0.00053 | 0 | 0 |
**wmf queries - 14**
these are specific to our script5.32.0https://lab.civicrm.org/dev/core/-/issues/2032Proposal - add ability to segment query logs2020-10-08T00:28:45ZeileenProposal - add ability to segment query logsI'm just working on documenting how to use query logs per process & it occurs to me that it is a bit hard getting the process specific part out of a longer query log. I have a possible way we could break it out into files / give back mor...I'm just working on documenting how to use query logs per process & it occurs to me that it is a bit hard getting the process specific part out of a longer query log. I have a possible way we could break it out into files / give back more control which I will put up as part of a PR - but it is a bit hacky....5.31.0https://lab.civicrm.org/dev/core/-/issues/2031Proposal - add exclude dirs to default civicrm.settings.php or in buildkit fo...2021-01-15T05:31:11ZeileenProposal - add exclude dirs to default civicrm.settings.php or in buildkit for performanceI think we should consider building this recommendation into buildkit defaults, if not default on install.
https://docs.civicrm.org/sysadmin/en/latest/setup/optimizations/#exclude-dirs-that-do-not-need-to-be-scanned
If would give
1) be...I think we should consider building this recommendation into buildkit defaults, if not default on install.
https://docs.civicrm.org/sysadmin/en/latest/setup/optimizations/#exclude-dirs-that-do-not-need-to-be-scanned
If would give
1) better dev experience (less slowness) and
2) if there are any issues with that recommendation we'd notice them & hopefully update it.https://lab.civicrm.org/dev/core/-/issues/2028Obsolete wkhtmltopdfPath causes hard fail in event registration due to intern...2023-07-14T05:03:22ZMonish DebObsolete wkhtmltopdfPath causes hard fail in event registration due to internal fatal error for the missing packageThis occurred to us after migration, as on new server the wkhtmltopdfPath is not installed/or it might be changed and as result of which alternate executable path of wkhtmltopdfPath becomes obsolete. This causes hard fail on any of the f...This occurred to us after migration, as on new server the wkhtmltopdfPath is not installed/or it might be changed and as result of which alternate executable path of wkhtmltopdfPath becomes obsolete. This causes hard fail on any of the following operations which involves PDF generation from html content:
1. attaching confirmation mail to participant and attaching PDF of the content on it.
2. same with #1 but in case of online donation.
3. Manual print document for contacts
ETC.
In our case its #1 which prevents the event registration process with an internal error
```
Sep 02 14:03:16 [error]
$Fatal Error Details = array(3) {
["message"]=>
string(424) "The exit status code '127' says something went wrong:
stderr: "sh: /old/path/wkhtmltox/bin/wkhtmltopdf: not found
"
stdout: ""
command: /old/path/wkhtmltox/bin/wkhtmltopdf --lowquality --margin-bottom '0.75in' --margin-left '0.75in' --margin-right '0.75in' --margin-top '0.75in' --orientation 'portrait' --page-height '792pt' --p
age-width '612pt' '/tmp//knp_snappy5f4fec74637de.html' '/tmp//knp_snappy5f4fec74638cf.pdf’.
“
```
To resolve this I would propose the following approach:
1. In case, if the wkhtmltopdf executable path is not working for various reason, throw an error message on civicrm status page.
2. Prevent the code to use the wkhtmltopdf executable path and fallback to use the default pdf package tcpdf which is bundled with CiviCRM
ping @eileen @JoeMurray @seamuslee5.34.0seamusleeseamusleehttps://lab.civicrm.org/dev/core/-/issues/2027Northern Ireland / Wales counties are out of date2021-03-14T21:47:07ZJKingsnorthNorthern Ireland / Wales counties are out of dateWe had some feedback...
```
Comments: My address is actually in the County of Pembrokeshire, not Dyfed, but this option wasn't available on the drop down menu. Dyfed was abolished over 20 years ago when the original Counties of Pembrok...We had some feedback...
```
Comments: My address is actually in the County of Pembrokeshire, not Dyfed, but this option wasn't available on the drop down menu. Dyfed was abolished over 20 years ago when the original Counties of Pembrokeshire, Cardiganshire and Carmarthenshire were reinstated. I would be grateful if you could amend your records!
```
The updated list of ISO counties for Wales can be found here: https://en.wikipedia.org/wiki/ISO_3166-2:GB
I started to write the updates into `xml/templates/civicrm_state_province.tpl`. And an update script to update the name for 'Gwent' to 'Bleanau Gwent'.
But then I saw in the ISO list, for some counties, the spelling is there twice, as are the abbreviations, depending on the language used.
I was not sure how to proceed...https://lab.civicrm.org/dev/core/-/issues/2026Feature Request - CiviCRM Profile, change field order method to drag'n drop t...2020-09-17T09:48:03Zjustinfreeman (Agileware)Feature Request - CiviCRM Profile, change field order method to drag'n drop to reduce errors and increase speedCiviCRM Profile, change field order method to drag'n drop to reduce errors and increase speed when re-organising the order of fields. When a lot of fields are included on a Profile, the current method is very time consuming and prone to ...CiviCRM Profile, change field order method to drag'n drop to reduce errors and increase speed when re-organising the order of fields. When a lot of fields are included on a Profile, the current method is very time consuming and prone to error.
Agileware Ref: CIVICRM-1560https://lab.civicrm.org/dev/core/-/issues/2025Proposal update PR template to add a section on testing2023-05-01T05:03:22ZeileenProposal update PR template to add a section on testingI think in addition to our current sections on the PR template we should add a section something like
"Explain how this PR adds relevant test cover. If test cover has not been added please explain why not, or which existing tests cover...I think in addition to our current sections on the PR template we should add a section something like
"Explain how this PR adds relevant test cover. If test cover has not been added please explain why not, or which existing tests cover the relevant code, with reference to discussion at https://github.com/civicrm/civicrm-dev-docs/issues/832"
@MikeyMJCO @totten @seamuslee @colemanwhttps://lab.civicrm.org/dev/financial/-/issues/149Contributions fail with "Authorization failed" (in master only)2020-09-14T01:06:52ZDaveDContributions fail with "Authorization failed" (in master only)You can see this at https://dmaster.demo.civicrm.org/civicrm/contribute/transact?id=1&reset=1 just complete the transaction to the end (as an anonymous user) and you get `Payment Processor Error message :Failed to update contribution in ...You can see this at https://dmaster.demo.civicrm.org/civicrm/contribute/transact?id=1&reset=1 just complete the transaction to the end (as an anonymous user) and you get `Payment Processor Error message :Failed to update contribution in database`, but the actual error "Authorization Failed" is coming from https://github.com/civicrm/civicrm-core/commit/62a721eda743ba248cf3b1a571a83f5fdbc72a56#diff-7a5b0e2d131dabc49178460fc63328c0R4430
----
Why you don't see the actual error is a [separate weirdness](https://github.com/civicrm/civicrm-core/blob/a31cdda9e9acda454f02790e3e8d1aca1f02ea2c/CRM/Contribute/Form/Contribution/Confirm.php#L2475-L2490), i.e. why does the api throw an exception on successful transactions, and even if that's ok, then why throw a new exception when caught - just rethrow the actual $e so that you can see the actual error.5.31.0https://lab.civicrm.org/dev/core/-/issues/2024Membership renewal with 0 tax creating extra line item2020-12-10T00:07:08Zvakeesan26Membership renewal with 0 tax creating extra line itemWhen we renewing a membership which is having 0 tax financial type creating a extra line item in contribution
![image](/uploads/e3e81457f94951f77f7922b82ed025f9/image.png)
![image](/uploads/41d18c2fda344f922361f66d5130d7eb/image.png)
...When we renewing a membership which is having 0 tax financial type creating a extra line item in contribution
![image](/uploads/e3e81457f94951f77f7922b82ed025f9/image.png)
![image](/uploads/41d18c2fda344f922361f66d5130d7eb/image.png)
![image](/uploads/4b69cd8deaad3cbfbef86062f8a45322/image.png)
![image](/uploads/8f3999d9705ee6922d29a4fbaab3d6e4/image.png)
![image](/uploads/3b968cb573efb40c331496ffae17dd68/image.png)
**CiviCRM - 5.29.0**5.34.0https://lab.civicrm.org/dev/translation/-/issues/51inheritLocale regression2020-09-26T00:27:07ZbgminheritLocale regressionTo reproduce:
* Tested on Drupal7
* Enable locale/i18n and two languages (ex: English and French)
* Enable language-prefix language detection
* In CiviCRM, enable multilingual, and two languages
Then use the Drupal language switcher to...To reproduce:
* Tested on Drupal7
* Enable locale/i18n and two languages (ex: English and French)
* Enable language-prefix language detection
* In CiviCRM, enable multilingual, and two languages
Then use the Drupal language switcher to change the language.
Result: the language only changes if we flush the CiviCRM cache. This is because the cache-flush removes data from `civicrm_cache`, including some session data.
Bug happens since 5.29, also in 5.30 and master.
cc @haystack @sluc23 @samuelsov (fyi)5.31.0https://lab.civicrm.org/dev/core/-/issues/2023Apiv4 Membership api2023-05-03T07:00:40ZeileenApiv4 Membership apiThis is missing at the moment due to complexity around the create action - which does some stuff with line items.
I expect quite a bit of code cleanup around membership functions before I get to the bottom of itThis is missing at the moment due to complexity around the create action - which does some stuff with line items.
I expect quite a bit of code cleanup around membership functions before I get to the bottom of it