CiviCRM Core issueshttps://lab.civicrm.org/dev/core/-/issues2023-05-03T07:00:40Zhttps://lab.civicrm.org/dev/core/-/issues/2023Apiv4 Membership api2023-05-03T07:00:40ZeileenApiv4 Membership apiThis is missing at the moment due to complexity around the create action - which does some stuff with line items.
I expect quite a bit of code cleanup around membership functions before I get to the bottom of itThis is missing at the moment due to complexity around the create action - which does some stuff with line items.
I expect quite a bit of code cleanup around membership functions before I get to the bottom of ithttps://lab.civicrm.org/dev/core/-/issues/1968Can't display System Status page - Guzzle error2024-03-22T05:03:28Zaydunsaidan.saunders@squiffle.ukCan't display System Status page - Guzzle errorOverview
----------------------------------------
Accessing the System Status page produces the boilerplate but no content.
Reproduction steps
----------------------------------------
1. Go to **Administer > Administration Console > Sy...Overview
----------------------------------------
Accessing the System Status page produces the boilerplate but no content.
Reproduction steps
----------------------------------------
1. Go to **Administer > Administration Console > System Status**
Current behaviour
----------------------------------------
The Drupal log shows:
```
| Location | https://example.org/civicrm/ajax/rest |
| Referrer | https://example.org/civicrm/a/ |
| Message | Error: Call to undefined function GuzzleHttp\Psr7\get_message_body_summary() in GuzzleHttp\Exception\RequestException::getResponseBodySummary() (line 127 of /var/www/sites/all/modules/civicrm/vendor/guzzlehttp/guzzle/src/Exception/RequestException.php). |
```
This is a failure in handling an exception, so unless there is some other failure this function is not called.
It looks to be an internal error in the Guzzle. A quick search, shows reports of the problem in other software using this Guzzle.
- https://github.com/nextcloud/twofactor_gateway/issues/312
- https://wordpress.org/support/topic/internal-server-error-620/
- https://stackoverflow.com/questions/61363997/guzzlehttp-gives-call-to-undefined-function-guzzlehttp-psr7-get-message-body-su
This is only happening on one server. Others are functioning normally.
Commenting out the referenced line lets the status be produced.
Dumping the `$response` object shows a status of 403, but does not contain the URL.
Environment information
----------------------------------------
* __CiviCRM:__ _5.28.2_
* __PHP:__ _7.3_
* __CMS:__ _Drupal 7.30_
* __Web Server:__ _Apache 2.4_https://lab.civicrm.org/dev/core/-/issues/1849Smart Group search criteria cannot be saved if the pre-existing Smart Group s...2020-09-17T09:46:10Zjustinfreeman (Agileware)Smart Group search criteria cannot be saved if the pre-existing Smart Group search criteria returns no resultsSmart Group search criteria cannot be saved if the pre-existing Smart Group search criteria returns no results.
This is increasingly common since CiviCRM will now warn users about disabled/deleted custom fields and the related Smart Gro...Smart Group search criteria cannot be saved if the pre-existing Smart Group search criteria returns no results.
This is increasingly common since CiviCRM will now warn users about disabled/deleted custom fields and the related Smart Group that need to be updated. The user is then stuck and unable to action the warning notice displayed by CiviCRM.
![chrome_aphUhSKcqb](/uploads/8c9ec177e1eacbb9ca6183e1e15beb2f/chrome_aphUhSKcqb.png)
A workaround to the problem is to manually add a contact to the Smart Group and then the ability to update the Smart Group via Actions becomes available. Or to change the search criteria so that it does return results - even if those results do not match the purpose of the Smart Group. These workarounds are not obvious to end-users.
It would be preferable for the action to update the Smart Group criteria was available even if no search results were returned by the current search, allowing the user to save the search criteria. This gives the future ability for contacts that match the search criteria to populate the group.
![chrome_DIjHo5MOpZ](/uploads/fac7cbc7901cf1088ed9f3adb98ca860/chrome_DIjHo5MOpZ.png)
Relates to https://lab.civicrm.org/dev/core/-/issues/1471
Agileware Ref: CIVICRM-1509https://lab.civicrm.org/dev/core/-/issues/1796Better message for missing module files2023-03-24T05:03:20ZMichael McAndrewBetter message for missing module filesI have come across messages like this
![image](/uploads/930fbdf1d723a523f78b72cad93f8cc1/image.png)
enough times for it to annoy me enough to want to submit a patch along the lines of
```diff
diff --git a/src/civicrm/civicrm/CRM/Util...I have come across messages like this
![image](/uploads/930fbdf1d723a523f78b72cad93f8cc1/image.png)
enough times for it to annoy me enough to want to submit a patch along the lines of
```diff
diff --git a/src/civicrm/civicrm/CRM/Utils/Hook.php b/src/civicrm/civicrm/CRM/Utils/Hook.php
index 656238f..b169f79 100644
--- a/src/civicrm/civicrm/CRM/Utils/Hook.php
+++ b/src/civicrm/civicrm/CRM/Utils/Hook.php
@@ -329,8 +329,11 @@ abstract class CRM_Utils_Hook {
foreach ($civiModules as $civiModule) {
if (!file_exists($civiModule['filePath'])) {
CRM_Core_Session::setStatus(
- ts('Error loading module file (%1). Please restore the file or disable the module.',
- [1 => $civiModule['filePath']]),
+ ts("Error loading file '%1' for module '%2'. Please restore the file or disable the module.",
+ [
+ 1 => $civiModule['filePath'],
+ 2 => $civiModule['prefix'],
+ ]),
ts('Warning'), 'error');
continue;
}
```
I'm testing the patch on a client site as we speak. Thought I would create this issue as a placeholder.https://lab.civicrm.org/dev/core/-/issues/1795Using Parent tag in search form doesn't pull contacts marked with child tag i...2021-04-01T11:40:23ZMonish DebUsing Parent tag in search form doesn't pull contacts marked with child tag in search form resultReproduction steps
----------------------------------------
Steps to replicate:
1. Create a parent tag A and child tag A1
2. Create/update contact and tag with A1
3. Go to 'Find Contacts' or 'Find Contributions' and search by Tag - A
...Reproduction steps
----------------------------------------
Steps to replicate:
1. Create a parent tag A and child tag A1
2. Create/update contact and tag with A1
3. Go to 'Find Contacts' or 'Find Contributions' and search by Tag - A
Current behaviour
----------------------------------------
```
1. In 'Find Contact' result, it doesn't show contact which is tagged with child tag A1
2. In 'Find Contributions' result, it doesn't pull contribution(s) of contact tagged with child tag A1
```
Expected behaviour
----------------------------------------
Searching using a parent tag should automatically include the contacts which are in parent tag A and in its n child tag(s) - An. In this use-case it should show contact tagged with child A1 tag.
Comments
----------------------------------------
ping @JoeMurray @eileen @seamuslee @lcdweb5.29.0Monish DebMonish Debhttps://lab.civicrm.org/dev/core/-/issues/1778Workaround for contribution.payment_instrument_id2023-03-19T05:03:31ZJoeMurrayWorkaround for contribution.payment_instrument_idSummary: when a contribution is created as Pay Later, the payment method on the first payment should replace the contribution's payment method. In the long term, a better solution to handling the many payment methods per contribution pro...Summary: when a contribution is created as Pay Later, the payment method on the first payment should replace the contribution's payment method. In the long term, a better solution to handling the many payment methods per contribution problem is needed.
Background
A contribution's payment method (civicrm_contribution.payment_instrument_id) is a crufty historical artifact from the era before partial payments, and it's not currently editable when a contribution is edited.
By contrast, the payment method on payments (civicrm_financial_trxn.payment_instrument_id) is editable in the browser when editing a payment.
From an accounting perspective, it isn't clear what it would mean if the contribution's payment method was changed after one or more payments are received - should it replace the payment method on all of the payments?? Reports and search have not been refactored to use civicrm_financial_trxn.payment_instrument_id, unfortunately, so there is a motivation to make the contribution's payment method better reflect the reality of what happens.
One important, common scenario that causes a lot of grief could be better handled with a small change. When a Pay Later contribution is created, a payment method needs to be specified, but it may not turn out to be accurate. When a Pay Later contribution status is transitioned to Completed or Partially Paid due to a Record Payment action, the payment method for the payment could be used to update the contribution's. No other change in what goes in to the database is anticipated, and more particularly, no change in the financial accounting.
--------------
FWIW, this is a similar problem of many-to-one as the financial types of line items compared to the contribution's single financial type. It also developed because the introduction of the many-to-one conceptual model was not matched with the courage and resources to refactor the original one-to-one model.
A proper solution would be to get rid of the field on contribution records. Changes would be needed in view contribution, search, and reporting so that the actual payment methods used for the payments on a contribution were used consistently.
With the new Search Builder about to be released we might be on verge of a new era of accuracy and flexibilty. In a few short months, its results will be the basis of the data for a new era of reports. Let's aim to get accurate data on payment methods out in those two areas and then come back to address this long-standing problem.JoeMurrayJoeMurrayhttps://lab.civicrm.org/dev/core/-/issues/1727Allow sites to not have Billing Address Location Type2023-03-09T05:03:25ZJoeMurrayAllow sites to not have Billing Address Location TypeOverview
----------------------------------------
Currently CiviCRM has a confusing array of default location types and flags. This issue proposes changes that would allow the Is Billing flag to be the only indication that a location is ...Overview
----------------------------------------
Currently CiviCRM has a confusing array of default location types and flags. This issue proposes changes that would allow the Is Billing flag to be the only indication that a location is the billing location rather than also having a Location Type of Billing. The intent is to allow instances to experiment with simpler approaches to Location Types in order to improve the usability of the product.
Example use-case
Add a hook to allow billing location to be determined by an alternative implementation. Sample LExIM extension approach would return the address which has isBilling==True if there is no LocationType of 'Billing' or if there is no address passed to function with LocationType of 'Billing' from https://github.com/civicrm/civicrm-core/blob/master/CRM/Core/Payment/BaseIPN.php#L491
In current places where code creates a 'Billing' type address one potential approach would assume it should be home location type for individual and household contacts and work for organizational contacts. In all cases the Is Billing would still be set. The objective of this proposal is to refactor and introduce hooks in order to allow a LExIM extension to work. More details about specific places in code that need to be changed and ideas on what exactly to do in a simpler approach to be provided if there is potential for this proposal to go forward.
Comments
----------------------------------------
Just testing water for potential support before developing more detailed proposal. Otherwise we'll likely just override core for this particular client.JoeMurrayJoeMurrayhttps://lab.civicrm.org/dev/core/-/issues/1709Better localised address handling.2023-04-30T05:03:22ZhomotechsualBetter localised address handling.Overview
----------------------------------------
_Currently, address forms in CiviCRM are built from a US-centric default set of form fields, currently we're using the translation system to alter the titles of these for different region...Overview
----------------------------------------
_Currently, address forms in CiviCRM are built from a US-centric default set of form fields, currently we're using the translation system to alter the titles of these for different regions (e.g a string translation exists for State/Province to County for EN_GB)._
_This ends up giving us a bad DX and bad UX we end up in a situation where even US address fields end up with a "County" field if we're using the EN_GB translation, not to mention that we're violating a rather fundamental aim of translation in that we're not actually translating - we're substituting a different string for a localised requirement that's unconnected to the language in use._
Proposed Solution
----------------------------------------
_We should investigate using a library e.g: [CommerceGuys/Addressing](https://github.com/commerceguys/addressing) (or another mechanism) to provide country-aware address forms/fields which work for CiviCRM's global community._https://lab.civicrm.org/dev/core/-/issues/1638Introduce "civi.dao.preUpdate" and "civi.dao.preInsert" events2020-05-18T17:53:27ZhaystackIntroduce "civi.dao.preUpdate" and "civi.dao.preInsert" eventsOverview
----------------------------------------
Following on from the action taken some time ago in [Add civi.dao.preDelete event](https://issues.civicrm.org/jira/browse/CRM-20458) (and the proposal raised in #161 to an extent) it woul...Overview
----------------------------------------
Following on from the action taken some time ago in [Add civi.dao.preDelete event](https://issues.civicrm.org/jira/browse/CRM-20458) (and the proposal raised in #161 to an extent) it would be helpful to apply the same `hook_civicrm_pre` + `hook_civicrm_post` pattern to the Symfony events in `CRM_Core_DAO::save()`. I propose CiviCRM introduces `civi.dao.preUpdate` and `civi.dao.preInsert` events to complement the existing `civi.dao.preDelete` event.
Symfony events have advantages over "old-style" hooks of the form `hook_civicrm_preSave`. The old-style hook format is likely to lead some devs to write callbacks in the format `extensionname_civicrm_preSave_table_name` which are difficult to unhook or prevent from running endlessly. Symfony events _can_ be unhooked and therefore make it relatively easy to avoid endless loops such as @colemanw raises concerns about in [his comment](https://lab.civicrm.org/dev/core/issues/161#note_9549). Also, they are largely undocumented and therefore avoid the effort of having to add to the Docs :-)
Example use-case
----------------------------------------
Let's look at what can be done with operations on "Option Value" data:
In both the [CiviCRM Event Organiser](https://github.com/christianwach/civicrm-event-organiser) and the [CiviCRM ACF Integration](https://github.com/christianwach/civicrm-acf-integration) plugins, I'd like to be able to inspect an Option Value _before_ it is saved to the database so that I know what the full state of the Option Value was before the update is applied. Once the edit has happened, there's (obviously) no way of retrieving the required data.
(FWIW, this is to perform actions on synced entities in WordPress rather than modifying the Option Value before it is saved, but the same applies should Option Value data need to be modified.)
Current behaviour
----------------------------------------
As far as I can tell, there isn't a hook available for me to inspect an Option Value data prior to it being created or updated programatically. I _partially_ solve this for changes made via the CiviCRM UI using the equivalent pre-post combination of `hook_civicrm_preProcess` and `hook_civicrm_postProcess`. However changes made via AJAX actions on the main "Event Type Options" listing screen can only be detected _after_ the edit has been made.
Proposed behaviour
----------------------------------------
To mirror the pattern of `hook_civicrm_pre` + `hook_civicrm_post` and `hook_civicrm_preProcess` + `hook_civicrm_postProcess` and the existing pattern of `civi.dao.preDelete` + `civi.dao.postDelete` to complete the set with:
* `civi.dao.preInsert` + `civi.dao.postInsert`
* `civi.dao.preUpdate` + `civi.dao.postUpdate`
Comments
----------------------------------------
PR to follow.5.26.0haystackhaystackhttps://lab.civicrm.org/dev/core/-/issues/1625IDN-Domain Emails - dont pass email check.2022-09-29T14:42:46ZRar9IDN-Domain Emails - dont pass email check.See Issue https://issues.civicrm.org/jira/browse/CRM-15975 and
Email validation is too restrictive (rejects IDN addresses) https://issues.civicrm.org/jira/browse/CRM-16313
Emails with these Characters should pass
https://www.denic.de/...See Issue https://issues.civicrm.org/jira/browse/CRM-15975 and
Email validation is too restrictive (rejects IDN addresses) https://issues.civicrm.org/jira/browse/CRM-16313
Emails with these Characters should pass
https://www.denic.de/en/know-how/idn-domains/idn-character-list/
Last tested with Civicrm 5.22.1 + Drupal 7
Php 7.2x Intl enabled.https://lab.civicrm.org/dev/core/-/issues/1527Improve log framework2023-01-28T05:03:21ZeileenImprove log frameworkI just tried to use Civi::log() & hit 1 limitation & one thing that I think is intended to be a feature that is missing.
The limitation is that debug_log_message() accepts a value 'prefix' that causes a separate file to be used. It's no...I just tried to use Civi::log() & hit 1 limitation & one thing that I think is intended to be a feature that is missing.
The limitation is that debug_log_message() accepts a value 'prefix' that causes a separate file to be used. It's not possible to pass that using Civi::log(). I think ideally it would be possible to do
Civi::log('my thing') & have that passed through to 'prefix'
I'm not quite sure how that would look at the container set up level - but I think it should be 'that easy' for the calling function.
The second thing is I think there is an intent these could be altered via hook/symfony so that, eg. a syslog logger can be used instead
@totten @seamusleehttps://lab.civicrm.org/dev/core/-/issues/1486Deadlocks on acl_cache2023-09-12T18:50:06ZeileenDeadlocks on acl_cacheOne of the prevalent deadlocks we see is on the acl_contact_cache. I've found that I can replicate this locally fairly consistently just by running 2 sets of tests at once.
The thing that is somewhat odd about this is that we don't use ...One of the prevalent deadlocks we see is on the acl_contact_cache. I've found that I can replicate this locally fairly consistently just by running 2 sets of tests at once.
The thing that is somewhat odd about this is that we don't use ACLs at all so in fact the table is always empty. Notably the table is also extremely slow to truncate - despite being empty.
**What happens**
The flow we have is
1) api call to create a contact
2) when the contact is created CRM_Contact_BAO_Contact_Utils::clearContactCaches() is called. It hits a deadlock because
a) it has an FK to contact.id (& in our case it has an FK and an index seemingly)
b) it is called a lot
3) it retries and succeeds
4) we then try to add an email - the add function fails on a foreign key constraint because the earlier deadlock rolled the contact create back
**Proposals**
I think there are a number of things we can do & we should do some combo:
1) Make the delete function less expensive & less locky
a) Add an index on modified date - we are constantly deleting where modified_date < x so let's index it
b) remove the index on acl_id - we ALSO have an FK so having both is probably harmless-but-confusing
c) alter the FK on contact ID to be an index not a foreign key. This is perhaps a confusing suggestion but it is also the one that will make a real dent in these deadlocks. Remember this is a cache table not a 'persistent' table. The impact of the rows outliving the existence of the contact for a little bit is basically none because we don't do ACL lookups on non-existent contacts (this is also potentially helpful for better handling 0 or anonymous contact_id where we've had to work around this before
2) Protect against conflict through mysql locks. I think we can use a mysql lock to avoid 2 processes attempting this flush at once. I'll do a PR for this
3) How often do we REALLY need to flush those caches. Maybe we could cache a 'last flushed' key & flush no more than every 30 seconds.
4) Consider checking if the table is empty before attempting to empty it.
5) Review the places where acl cache is called from & decide how necessary there are - here is the list:
**CRM_ACL_BAO_Cache::resetCache**
CRM/Contact/BAO/Contact/Utils.php: CRM_ACL_BAO_Cache::resetCache();
CRM/Core/BAO/Cache.php: CRM_ACL_BAO_Cache::resetCache();
CRM/Utils/System.php: CRM_ACL_BAO_Cache::resetCache();
CRM/ACL/Page/EntityRole.php: CRM_ACL_BAO_Cache::resetCache();
CRM/ACL/Form/ACLBasic.php: CRM_ACL_BAO_Cache::resetCache();
CRM/ACL/Form/EntityRole.php: CRM_ACL_BAO_Cache::resetCache();
CRM/ACL/BAO/ACL.php: CRM_ACL_BAO_Cache::resetCache();
**CRM_Contact_BAO_Contact_Utils::clearContactCaches()**
CRM/Contact/BAO/Contact/Utils.php: public static function clearContactCaches($isEmptyPrevNextTable = FALSE) {
CRM/Contact/BAO/GroupContact.php: CRM_Contact_BAO_Contact_Utils::clearContactCaches();
CRM/Contact/BAO/GroupContact.php: CRM_Contact_BAO_Contact_Utils::clearContactCaches();
CRM/Contact/BAO/Contact.php: CRM_Contact_BAO_Contact_Utils::clearContactCaches();
CRM/Contact/Import/Form/Preview.php: CRM_Contact_BAO_Contact_Utils::clearContactCaches(TRUE);
bin/cli.class.php: CRM_Contact_BAO_Contact_Utils::clearContactCaches();
6) A bigger task perhaps - but permit opportunistic flushing to be turned off in favour of a cron (per group contact cache)
**Deadlock output**
LATEST DETECTED DEADLOCK
------------------------
2019-12-19 11:29:45 0x700010cd2000
*** (1) TRANSACTION:
TRANSACTION 34518715, ACTIVE 1 sec starting index read
mysql tables in use 1, locked 1
LOCK WAIT 11 lock struct(s), heap size 1136, 4 row lock(s), undo log entries 7
MySQL thread id 571, OS thread handle 123145583353856, query id 709677 localhost 127.0.0.1 drupalcivi_lkznr updating
DELETE
FROM civicrm_acl_cache
WHERE modified_date IS NULL
OR (modified_date <= '20191218222444')
*** (1) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 143182 page no 3 n bits 80 index PRIMARY of table `drupalcivi_lkznr`.`civicrm_acl_cache` trx id 34518715 lock_mode X waiting
Record lock, heap no 2 PHYSICAL RECORD: n_fields 6; compact format; info bits 32
0: len 4; hex 00003c14; asc < ;;
1: len 6; hex 0000020eb6ba; asc ;;
2: len 7; hex 3b000001442ac5; asc ; D* ;;
3: SQL NULL;
4: len 4; hex 00000002; asc ;;
5: SQL NULL;
*** (2) TRANSACTION:
TRANSACTION 34518714, ACTIVE 1 sec starting index read
mysql tables in use 2, locked 2
66 lock struct(s), heap size 8400, 71 row lock(s), undo log entries 74
MySQL thread id 573, OS thread handle 123145584189440, query id 710022 localhost 127.0.0.1 drupalcivi_lkznr updating
UPDATE civicrm_contact SET contact_type = 'Organization' , contact_sub_type = NULL , email_greeting_id = NULL , postal_greeting_id = NULL , addressee_id = 3 , employer_id = 15055 WHERE ( civicrm_contact.id = 15055 )
*** (2) HOLDS THE LOCK(S):
RECORD LOCKS space id 143182 page no 3 n bits 80 index PRIMARY of table `drupalcivi_lkznr`.`civicrm_acl_cache` trx id 34518714 lock_mode X
Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0
0: len 8; hex 73757072656d756d; asc supremum;;seamusleeseamusleehttps://lab.civicrm.org/dev/core/-/issues/1482Remove code to create a subscription history entry when a contact is created2019-12-22T22:53:31ZeileenRemove code to create a subscription history entry when a contact is createdI feel like these lines are part of an idea in the early days of Civi that faded out & we should remove them
```
// make a civicrm_subscription_history entry only on contact create (CRM-777)
if (empty($params['contact_id'])) {
...I feel like these lines are part of an idea in the early days of Civi that faded out & we should remove them
```
// make a civicrm_subscription_history entry only on contact create (CRM-777)
if (empty($params['contact_id'])) {
$subscriptionParams = [
'contact_id' => $contact->id,
'status' => 'Added',
'method' => 'Admin',
];
CRM_Contact_BAO_SubscriptionHistory::create($subscriptionParams);
}
```
The referenced JIRA is https://issues.civicrm.org/jira/browse/CRM-777
(I only noticed them because they failed due to a deadlock - https://lab.civicrm.org/dev/core/issues/1481)5.22.0https://lab.civicrm.org/dev/core/-/issues/1315Ensure exceptions / failures are visible to site admins2023-04-13T22:43:06ZxurizaemonEnsure exceptions / failures are visible to site adminsExample use case: including `<style>.foo { color: red }</style>` in a CiviCRM mailing template can cause the scheduled reminders action to fail with a fatal error (#58, [PR#15436 on Github](https://github.com/civicrm/civicrm-core/pull/15...Example use case: including `<style>.foo { color: red }</style>` in a CiviCRM mailing template can cause the scheduled reminders action to fail with a fatal error (#58, [PR#15436 on Github](https://github.com/civicrm/civicrm-core/pull/15436)).
If #1256 is implemented then Smarty should throw an exception instead. Once that's the case, CiviCRM can trap the exceptions and record them in some central log (akin to Drupal dblog), and from there we can show a status notification to site admins which informs them of the fact that some component may not be functioning as expected.
This is useful because it closes the loop from problems like "anonymous user could not complete contribution" or "users are no longer getting membership reminders after theme updates" and some actionable response which starts with "site admin knows there's a problem".
I don't see this on existing issue lists, but please point me in the right direction if there is such a thing.
We don't need #1256 resolved to add this improvement.https://lab.civicrm.org/dev/core/-/issues/1211Add option to search by date not set2023-01-16T05:03:34ZyashodhaAdd option to search by date not setCurrently in CiviCRM, for date fields you can either search by date range or for some fields explictly there is an option to filter by NULL/NOT NULL.
![contrib_search](/uploads/f6d1e0474fb4fcceefac4f0fdd5d58e2/contrib_search.png)
The op...Currently in CiviCRM, for date fields you can either search by date range or for some fields explictly there is an option to filter by NULL/NOT NULL.
![contrib_search](/uploads/f6d1e0474fb4fcceefac4f0fdd5d58e2/contrib_search.png)
The option to filter by NULL/NOT NULL should be available for all date fields in the filter drop down fields.
I have already added this option in reports and it looks like it could helpful for other date search screens as well.![contri_date+null](/uploads/dd4cfc311620ae952ab237e4abee7767/contri_date+null.png)yashodhayashodhahttps://lab.civicrm.org/dev/core/-/issues/1073Performance : Expose some important smarty configuration variables outside th...2021-03-29T14:14:27ZVangelisPPerformance : Expose some important smarty configuration variables outside the smarty classWhile working with multiple "complex" civicrm sites on dedicated server instances, I've noticed that even though the database was highly optimized, all sites were having some sort of overhead of nearly 1.5-2 seconds before rendering the...While working with multiple "complex" civicrm sites on dedicated server instances, I've noticed that even though the database was highly optimized, all sites were having some sort of overhead of nearly 1.5-2 seconds before rendering the results. That is on the contact display page, on search, advanced search and so on.
Added Redis caching to CiviCRM, helped with a bit of small improvement but again, the delay was there.
Digging deeper with XHProf, I've found out that for every page that was loading, Smarty was checking if the templates needed recompiling.
Long story short, inside the /packages/Smarty/Smarty.class.php there's a variable that is `true` by default:
`var $compile_check = true;`
Description is the following:
> This tells Smarty whether to check for recompiling or not. Recompiling does not need to happen unless a template or config file is changed. Typically you enable this during development, and disable for production.
Disabling this in production, i did see a big improvement in terms of speed, as Smarty is now being instructed not scan every time if it needs to recompile the templates.
My suggestion would be to expose that variable (along with `$caching`) as a constant so that for example we can add it to `settings.php` as 'false' in production environments.
I could provide a patch (if needed) that does that.
PS. "complex" = a CiviCRM instance with many extensions and many tpl injections through custom extensions.5.17.0https://lab.civicrm.org/dev/core/-/issues/991CiviCRM 5.13.4 - smart groups with contact subtypes: subtype is not in Contac...2023-03-16T05:03:21ZscardiniusCiviCRM 5.13.4 - smart groups with contact subtypes: subtype is not in Contact Type field on edit criteria modeHow to reproduce:
* open demo site https://demo.circle-interactive.co.uk/
* open Search > Find contacts
* Choose "is..." equal "Student"
* after click "Search" button we have 3 records
* Select all rows and create new smart group "tes...How to reproduce:
* open demo site https://demo.circle-interactive.co.uk/
* open Search > Find contacts
* Choose "is..." equal "Student"
* after click "Search" button we have 3 records
* Select all rows and create new smart group "test-smart-students" from "Actions" menu...
* Open Contacts > Manage groups
* Find "test-smart-students" on the list and click "Contacts" link for this group
* after click we have 3 records, nice
* Click button "Edit Smart Group Search Criteria for test-smart-students"
* you see "Contact Type In Individual ...AND... Contact Subtype Like Student" - nice :)
* Expand section "Edit test-smart-students Smart Group Criteria"
* in Contact Type(s) field you see only "Individual" - ups :(
* Click on "Search" button
* you see 175 contact - not good :-[
where clause: ( contact_a.contact_type IN ("Individual") AND ( (contact_a.contact_sub_type LIKE '%Student%') ) )https://lab.civicrm.org/dev/core/-/issues/878Remove copyright & years from all code, except LICENSE.md etc2020-04-15T06:55:32ZxurizaemonRemove copyright & years from all code, except LICENSE.md etcI'm browsing for a small change made to a template, and I'm navigating through a bunch of NFC changes like 2a73d3b and 6b83d5b. (Clicking on those commits may even WSoD Gitlab, so here's [2a73d3b on Github](https://github.com/civicrm/civ...I'm browsing for a small change made to a template, and I'm navigating through a bunch of NFC changes like 2a73d3b and 6b83d5b. (Clicking on those commits may even WSoD Gitlab, so here's [2a73d3b on Github](https://github.com/civicrm/civicrm-core/commit/2a73d3b).)
Credit to those doing the busywork, but those changes contribute little to the software while adding noise to the commit history of every file in CiviCRM.
The commit messages are fairly inconsistent over time, so someone browsing history can't be 100% sure that any given commit is safe to ignore.
I say it's time to stop changing hundreds of^W^W [thousands of](https://github.com/civicrm/civicrm-core/commit/6b83d5) (?!) files over and over just to update the copyright & CiviCRM version.
Let's make one final commit and remove them all, removing those comments in favour of references to licensing via COPYRIGHT.md (copyright year and owner), LICENSE.md (contents of the AGPL), and https://civicrm.org/licensing instead.
No shortage of other projects demonstrating this - I don't think it's legally contentious based on the number of projects using that approach.https://lab.civicrm.org/dev/core/-/issues/5068Do we need the qfkey in group urls?2024-03-07T13:49:27ZAndrew WestDo we need the qfkey in group urls?My users are forever emailing each other broken links to Civi groups, due to the qfkey in the URL. This doesn't seem to be needed in most of the rest of Civi - is it worth my investigating how to remove it? Or would we aim to replace tha...My users are forever emailing each other broken links to Civi groups, due to the qfkey in the URL. This doesn't seem to be needed in most of the rest of Civi - is it worth my investigating how to remove it? Or would we aim to replace that page with a search kit version soon anyway?https://lab.civicrm.org/dev/core/-/issues/4923Possible performance improvement: When a custom field is disabled, automatica...2024-02-08T14:11:58ZDaveDPossible performance improvement: When a custom field is disabled, automatically uncheck the searchable box and remove the db indexSuppose you have about 20 fields in the custom group and they were almost all searchable (had the searchable box checked). Over time let's say half get disabled. The indexes are still in the db, and the searchable checkbox is meaningless...Suppose you have about 20 fields in the custom group and they were almost all searchable (had the searchable box checked). Over time let's say half get disabled. The indexes are still in the db, and the searchable checkbox is meaningless because they don't appear anywhere anyway.
If the table is large, those indexes potentially cause trouble with no benefit.
If you go to re-enable, it's true it won't automatically re-check the searchable box and recreate the index, but there's no real harm, you just go back in and check the box.
An alternative to automatically doing this would be a status check: "The following fields are disabled but marked searchable. Consider making them unsearchable to improve performance on large databases."