CiviCRM Core issueshttps://lab.civicrm.org/dev/core/-/issues2023-10-05T11:56:00Zhttps://lab.civicrm.org/dev/core/-/issues/4636Deleting group invalidates ACL2023-10-05T11:56:00Zaydunsaidan.saunders@squiffle.ukDeleting group invalidates ACLOverview
----------------------------------------
Groups used in ACL's can be deleted without warning resulting in invalid ACL's.
Setup
----------------------------------------
1. Create a group: TestGroup
1. Create an ACL: Role: Every...Overview
----------------------------------------
Groups used in ACL's can be deleted without warning resulting in invalid ACL's.
Setup
----------------------------------------
1. Create a group: TestGroup
1. Create an ACL: Role: Everyone, Operation: View, Type: Group, Group: TestGroup (just created)
1. Note that the ACL display will shows `Type` as 'Group' and shows the group name
1. In SQL, look at the `civicrm_acl` table and note the `object_table` is 'civicrm_group', and `object_id` is the new group id.
Now delete the TestGroup:
- There is no warning that this group is used in an ACL
- In the ACL display the group name is now blank - a situation which cannot be created through the Add or Edit ACL screens.
- In SQL, the `civicrm_acl` table still show the `object_id` as the now non-existent group id.
Prior to https://github.com/civicrm/civicrm-core/pull/27679 this causes DB Syntax Errors when calling `CRM_Contact_BAO_Contact_Permission::allow()`
Expected behaviour
----------------------------------------
Good question! Maybe a warning that the group is being used in an ACL, but what should then happen to the ACL? Disable it?
Environment information
----------------------------------------
* __CiviCRM:__ _Master but may be longstanding_ <!-- If this problem relates to an upgrade, then specify both old and new versions -->https://lab.civicrm.org/dev/core/-/issues/4625Inconsistent search result when "Modified By" field and apply an action2023-09-26T12:28:52Zthomas_SYSTOPIAInconsistent search result when "Modified By" field and apply an action**Description:**
When apply an action (like "Tag - add to contacts") on a search result using "Change Log->Modified By" field the result scope is lost. Instead the action would be applied to all contacts.
**Steps to reproduce** (tested...**Description:**
When apply an action (like "Tag - add to contacts") on a search result using "Change Log->Modified By" field the result scope is lost. Instead the action would be applied to all contacts.
**Steps to reproduce** (tested on https://d9-master.demo.civicrm.org/):
1. Use the advanced search form with "Change Log->Modified By" = _admin_. The result includes exactly one contact.
2. Choose "Tag - add to contacts" from the action menu.
3. You'll see "Number of selected contacts: 204" (204 are the sum of all contacts.)https://lab.civicrm.org/dev/core/-/issues/4622Running queues following the docs' example might lead to infinite loops2023-10-27T08:36:37ZjensschuppeRunning queues following the docs' example might lead to infinite loopsFollowing the [_Queue Reference_ example](https://docs.civicrm.org/dev/en/latest/framework/queues/#3-run-the-queue) for running a queue in `CRM_Queue_Runner::ERROR_CONTINUE` mode with a `while` loop that only `break`s for `is_continue!=1...Following the [_Queue Reference_ example](https://docs.civicrm.org/dev/en/latest/framework/queues/#3-run-the-queue) for running a queue in `CRM_Queue_Runner::ERROR_CONTINUE` mode with a `while` loop that only `break`s for `is_continue!=1` will lead to infinite loops when a queue item has not been released yet (e.g. due to the script terminating without an exception being caught and the queue being re-run before release of the next item).
That's because `CRM_Queue_Queue_Sql::claimItem()` fetches the _first_ queue item and filters for `release_time` _afterwards_ - yielding no result, but reporting more items to be in the queue (`is_continue=1`). The only way to catch these cases with code calling the `CRM_Queue_Runner::runNext()` method is to check for `is_error=1` and the `exception` message being `Failed to claim next task`.
So, I've got some questions regarding Queues:
* Is there any best practice for handling errors during `claimItem()` any better?
* Would you agree there should be another indicator in the task result for those cases, or maybe that exception actually being thrown?
* Alternatively, should `is_continue` be `0` instead for those cases (i.e. its meaning would change from _queue finished_ to _queue processing can continue_)?
* Can the `$lease_time` parameter for SQL queues' `claimItem()` be exposed to all queues, so that the caller can modify it?
I see that the query was altered with https://github.com/civicrm/civicrm-core/pull/15421 by @artfulrobothttps://lab.civicrm.org/dev/core/-/issues/4617Running Job.update_greeting without updating customized greetings2023-10-18T08:06:00ZCharlotteRunning Job.update_greeting without updating customized greetingsOverview
----------------------------------------
For the scheduled job `update_greeting` there are two options for the parameter `force` which are explained by the [documentation](https://docs.civicrm.org/user/ca/latest/initial-set-up/...Overview
----------------------------------------
For the scheduled job `update_greeting` there are two options for the parameter `force` which are explained by the [documentation](https://docs.civicrm.org/user/ca/latest/initial-set-up/scheduled-jobs/#job_update_greeting) as follows:
- `0` only contacts with null values are updated (default)
- `1` update all contacts
There is no explanation about what happens with customized greetings when using `force = 1`.
It turns out that all customized greetings will be overridden by the standard greeting. I did not expect this.
It would be great to have another option
- `2` update all contacts excluding customized greetings
If this is not possible, then a more detailed documentation could be helpful.
This issue has also been described on [stackexchange](https://civicrm.stackexchange.com/questions/41709/running-job-update-greeting-without-updating-customized-greetings).
Work around
----------------------------------------
Use API v4 to set all greetings that are not customized greetings to NULL.
Here an example for individuals and email greeting:
```
$contacts = civicrm_api4('Contact', 'get', [
'where' => [
['email_greeting_custom', 'IS EMPTY'],
['contact_type', '=', 'Individual'],
],
'chain' => [
'name_me_0' => ['Contact', 'update', ['values' => ['id' => '$id', 'custom_greeting_display' => NULL]]],
],
]);
```
Afterwards, the greetings are not NULL but already contain the new standard greeting. There is no need to run the job `update_greeting` with `force = 0`.
Environment information
----------------------------------------
* __CiviCRM:__ 5.63.2
* __PHP:__ 8.1
* __CMS:__ Drupal 9https://lab.civicrm.org/dev/core/-/issues/4613Can't uncheck "display in table" for custom fields2023-09-26T12:18:06ZDaveDCan't uncheck "display in table" for custom fields1. Create a multivalued custom field group.
2. Create a custom field with "display in table" checked.
3. Edit the field and uncheck display in table.
4. Go to edit it again and note that it's checked.
This sounds familiar.1. Create a multivalued custom field group.
2. Create a custom field with "display in table" checked.
3. Edit the field and uncheck display in table.
4. Go to edit it again and note that it's checked.
This sounds familiar.https://lab.civicrm.org/dev/core/-/issues/4610checkAccess() or the Contact get api does not return contacts allowed by rela...2023-09-26T12:16:37ZjitendracheckAccess() or the Contact get api does not return contacts allowed by relationship.**Usecase**: Contact loaded via checksum on a webform does not load the permitted related contact.
Checksum in the URL adds the contact id in the session. Contact get api with `checkPermission = true` should allow the access for contact...**Usecase**: Contact loaded via checksum on a webform does not load the permitted related contact.
Checksum in the URL adds the contact id in the session. Contact get api with `checkPermission = true` should allow the access for contacts which has permission enabled in the relationship table.
Current code only checks for the contacts allowed via ACL https://github.com/civicrm/civicrm-core/blob/master/CRM/Contact/BAO/Contact/Permission.php#L361
Possible fixes:
- Add related contacts ids in the ACL cache table OR.
- Add another code to `cacheSubQuery` for the relationship_cache table.https://lab.civicrm.org/dev/core/-/issues/4581Define re-usable idiom for deferrable upgrade steps2023-09-19T13:05:13ZtottenDefine re-usable idiom for deferrable upgrade stepsOverview
----------------------------------------
In managing upgrade-steps, there is some tension between:
* Making the upgrades automatic for typical sysadmins
* Allowing very large deployments to schedule/manage expensive upgrade-st...Overview
----------------------------------------
In managing upgrade-steps, there is some tension between:
* Making the upgrades automatic for typical sysadmins
* Allowing very large deployments to schedule/manage expensive upgrade-steps
This proposes to balance those expectations by defining a _deferrable upgrade task_. Sysadmins can choose to approach them a few ways:
* "Just run the task like every other upgrade step" (*e.g. default; for average and small systems*)
* "Let me decide when to run it... but please don't make me figure out obscure incantations" (*for larger systems*)
* "Let me find a highly-optimized/bespoke way to accomplish the change" (*for extra-large systems with a team of admins*)
Example use-case
----------------------------------------
1. Take a table (like `civicrm_contact`, `civicrm_activity`, `civicrm_mailing_event_queue`) which can be very large on some deployments.
1. Define a schema change (e.g. add a column and an index)
1. Executing this change be quite slow.
1. In the long-term, all systems need the schema-change. If people skip it, then it will lead to problems/confusion/bug-reports/etc.
1. In the near-future, the schema change is not quite mandatory. (The new column isn't actively used; or its usages are limited+guarded.)
A sketch
----------------------------------------
* Designate a place to store a list of deferred upgrade steps (e.g. `civicrm_deferred_upgrade` or `civicrm_queue_item`)
* In the `CRM/Upgrade/` system, add some helpers to conditionally execute or defer a step. Ex:
```php
$this->addDeferrableTask(ts('Update CiviMail tracking'), 'doCiviMailQueueConversion');
```
```php
function addDeferrableTask($title, $funcName, $args) {
if (deferred upgrades enabled) {
// Add $title, $action, $args to a persistent TODO list
}
else {
$this->addTask($title, $funcName, $args);
}
}
```
* Add a system-status-check and post-upgrade message to warn if there are any of these things that need to run
* Add an API/job/command/UI to list/execute/skip/delete deferred stepshttps://lab.civicrm.org/dev/core/-/issues/4564Idea: Smart groups without the cache2023-09-13T12:20:47ZAndrew WestIdea: Smart groups without the cacheJust throwing this out there as something we could maybe fund:
Current behaviour
----------------------------------------
Smart groups always write to the civicrm_group_contact_cache table
Proposed behaviour
---------------------------...Just throwing this out there as something we could maybe fund:
Current behaviour
----------------------------------------
Smart groups always write to the civicrm_group_contact_cache table
Proposed behaviour
----------------------------------------
Smart groups optionally write to the civicrm_group_contact_cache table (default: yes)
Overview
----------------------------------------
As I understand it, whenever you request a smart group it stores its contacts in civicrm_group_contact_cache. This is for good reason: it's a cache that means the smart groups aren't regenerated every time they're requested, which can be slow.
It also simplifies the code: you can effectively treat smart groups the same as non-smart groups - as it's just grabbing a list of contacts from a table.
But it's possible to set the cache to expire after 0 seconds, if you need your smart groups to be up to date: ie. they are regenerated each time they're requested. I think a fair few orgs do this.
If you have a 0s cache, you sometimes don't need the data written to a table at all. You just need a transient list of contacts that doesn't need to persist past the end of the request. And it's pretty slow to use the cache table: the 'INSERT INTO' command is slow relative to the original SELECT query. This time can add up if you're querying a bunch of groups at once, or have nested smart groups, or have a lot of contacts in the groups.
So I was tentatively wondering if there could be an API4 'no-smart-group-cache' parameter, that just runs the underlying query and stores it all in memory. I imagine this would only work for API4 smart groups, as it's super-easy to get their underlying search criteria.
The default would remain to use the cache, for full backwards-compatibility. But the parameter could be used in custom code.
Two questions:
1. Anyone else interested in this?
2. How hard would it be / how much of a can of worms is it?https://lab.civicrm.org/dev/core/-/issues/4558Proposal: setting to define alternative frontend URLs for when using a proxy2023-09-13T12:12:51ZbgmProposal: setting to define alternative frontend URLs for when using a proxyWhen CiviCRM runs behind another system, such as a reverse-proxy running on a separate domain that only exposes specific features, we do not have a way to define that URL.
For example:
- Staff access crm.example.org - the service is ru...When CiviCRM runs behind another system, such as a reverse-proxy running on a separate domain that only exposes specific features, we do not have a way to define that URL.
For example:
- Staff access crm.example.org - the service is running behind a VPN and not available to the general public
- However, they send mailings, which include unsubscribe and opt-out links. This feature is provided by a reverse-proxy that forwards those requests specifically.
[CRM/Mailing/BAO/Mailing.php](https://github.com/civicrm/civicrm-core/blob/master/CRM/Mailing/BAO/Mailing.php#L1020) defines a few URLs and uses `CRM_Utils_System::url` to generate the links. In the example I found, they were overriding the core PHP file, in order to hardcode the domain.
~~I'm thinking mailing [hook_civicrm_links](https://docs.civicrm.org/dev/en/latest/hooks/hook_civicrm_links/) might be a good semantic fit, even if it's a bit of an odd case, but it is used is weird places, such as in the Extension Manager.~~
- `op` = `mailing.content.actions` ? (mailing.actions could be confusing with actions in the UI, such as edit/delete)
- `objectID` = `job_id` ?
~~Where it gets weirder, is that the array of `$urls` would have to be packaged into the format that the hook expects, then unpackaged back into the expected format from the parent function (unless we just send it as-is, because the fluff is not necessary, and it's up to the hook to check for the `$op`).~~
An alternative could be to add a special `CIVICRM_UF_BASEURL`, but then it would mean more params for `CRM_Utils_System::url` and it seems already messy enough.
Edit: based on Jaap's comment, it probably makes more sense to find a solution for a "reverse-proxy URL".https://lab.civicrm.org/dev/core/-/issues/4557Proposal - Allow empty profile field labels instead of null2023-09-08T15:11:53ZshaneonabikeProposal - Allow empty profile field labels instead of nullPresently, when you create a Custom Profile and add fields you need add a label. In some cases, having a large question makes the label kind of awkward and crammed. It makes for a bad UX experience I believe.
So the workaround by most o...Presently, when you create a Custom Profile and add fields you need add a label. In some cases, having a large question makes the label kind of awkward and crammed. It makes for a bad UX experience I believe.
So the workaround by most of the designers I work with is to simply remove the ```label``` and then drop the question in the Pre text. This works just dandy, except that CiviCRM is adding ```null``` for empty texts. The only way to prevent this is to modify the label in the DB.
I would like to propose that we remove the code that is setting this text to null. I could make a quick patch. I realize the label is being used to create the actual field name, but in the case of the Profile the field name is already created.
![Selection_007](/uploads/48ab9decd2a6b1c3bb5a380a8278f12d/Selection_007.png)https://lab.civicrm.org/dev/core/-/issues/4552Add scheduled job form doesn't go to the right place after save or delete2023-09-06T09:36:22ZDaveDAdd scheduled job form doesn't go to the right place after save or deleteI'm not sure how often people use the UI to add scheduled jobs - I was just testing something and came across this. But what happens is it will create the job but then just go to a blank form. After deleting a scheduled job, it does dele...I'm not sure how often people use the UI to add scheduled jobs - I was just testing something and came across this. But what happens is it will create the job but then just go to a blank form. After deleting a scheduled job, it does delete it but just reloads the confirmation screen.https://lab.civicrm.org/dev/core/-/issues/4544Older sites missing index for `is_deceased` on `civicrm_contact`2023-09-06T13:56:12ZwmortadaOlder sites missing index for `is_deceased` on `civicrm_contact`This is a follow up from Eileen's comment about a missing index in https://lab.civicrm.org/dev/core/-/issues/4539#note_150479.
The index for `is_deceased` was added in CiviCRM 4.7 and is defined in [xml/schema/Contact/Contact.xml](https...This is a follow up from Eileen's comment about a missing index in https://lab.civicrm.org/dev/core/-/issues/4539#note_150479.
The index for `is_deceased` was added in CiviCRM 4.7 and is defined in [xml/schema/Contact/Contact.xml](https://github.com/civicrm/civicrm-core/blob/f068222ac25a5a393e9238b617ae91a4d9d29797/xml/schema/Contact/Contact.xml#L722-L726). However, there doesn't appear to be an upgrade task to add this index to existing sites. As a result any site installed before CiviCRM 4.7 is likely to be missing this index. Checking our client sites, I can see a mix of sites that do and don't have it.
The MR where this index was added is: https://github.com/civicrm/civicrm-core/pull/10489
There was a question from Tim about whether there should be an upgrader. There was a later comment from Eileen about whether this index should be removed.
So I guess the question is whether we want this index or not:
- if we **do** want this index we should add an upgrade task so that older sites have this index
- if we **don't** want this index we should add an upgrade task to remove it from sites that have ithttps://lab.civicrm.org/dev/core/-/issues/4539Very slow query when sending mailings via CiviMail2023-09-12T05:08:13ZwmortadaVery slow query when sending mailings via CiviMailOverview
----------------------------------------
We have come across an issue where the database connection was timing out due to slow, long-running queries when the Send Scheduled Mailings job was running. We became aware of this when...Overview
----------------------------------------
We have come across an issue where the database connection was timing out due to slow, long-running queries when the Send Scheduled Mailings job was running. We became aware of this when we set a maximum execution time limit of 3 minutes for MySQL queries. Mailings for one of our clients were not being sent because the database connection was timing out.
As a workaround, we have increased the maximum execution time to 10 minutes and reduced the Mailer Batch Limit and Mailer Job Size to 1,000. However, we'd like to fix the underlying issue that the queries are running so slowly.
When sending a mailing CiviCRM runs a query of the form:
```sql
SELECT r.contact_id, r.email_id, r.phone_id
FROM civicrm_mailing_recipients r
INNER JOIN civicrm_contact c on
(c.id = r.contact_id
AND c.is_deleted = 0
AND c.is_deceased = 0
AND c.do_not_email = 0
AND c.is_opt_out = 0
)
INNER JOIN civicrm_email e ON (r.email_id = e.id AND e.on_hold = 0)
WHERE r.mailing_id = 3271
LIMIT 18000, 1000
```
There are no indices for `is_deceased`, `do_not_email` or `is_opt_out` so this query is very slow. (There is an index for `is_deleted`. There should be an index for `is_deceased` but this appears to be missing from this site.)
We have tested adding these indices in a development environment to see what difference it would make and this increase the speed of the query by a factor of one thousand:
- 20,000 rows with no indices = 80 seconds
- 20,000 rows with indices = 0.085 seconds
Reproduction steps
----------------------------------------
This is likely to only affect sites with a large number of contacts and large number of rows in the `civicrm_mailing_recipients` table (in this case 46,000 contacts and 9 million rows respectively).
1. Set a maximum execution time for MySQL of 3 minutes (say)
2. Create a mailing to a large group of contacts (>20,000)
3. Schedule the mailing to send
4. Send Scheduled Mailings job fails with error: "Finished execution of Send Scheduled Mailings with result: Failure, Error message: DB Error: unknown error"
Current behaviour
----------------------------------------
No indices ~~`is_deceased`~~, `do_not_email` or `is_opt_out` in `civicrm_contact` so queries run very slowly.
Expected behaviour
----------------------------------------
Indices for the above fields so that the query runs quickly.
Environment information
----------------------------------------
* __CiviCRM:__ _5.62_
* __PHP:__ _8.1__
* __CMS:__ _N/A_
* __Database:__ _MySQL 5.7.43_
Comments
----------------------------------------
There are other speed improvements in #4045 but these relate to the speed of the user interface rather than the speed of sending emails.
Perhaps this can also help to [reduce our collective CO2 emissions](https://civicrm.org/blog/systopia/beginning-green-how-can-we-make-civicrm-more-sustainable)?https://lab.civicrm.org/dev/core/-/issues/4528Entity Reference custom fields: Configurable delete behavior2024-01-17T22:28:23ZjensschuppeEntity Reference custom fields: Configurable delete behavior## Overview
Currently, when entities referenced in a custom field of the type _Entity Reference_ are deleted, the entity reference field gets cleared (set to `NULL`). This is easy and prevents us from database inconsistencies. But this ...## Overview
Currently, when entities referenced in a custom field of the type _Entity Reference_ are deleted, the entity reference field gets cleared (set to `NULL`). This is easy and prevents us from database inconsistencies. But this can lead to problems:
* inconsistencies when the entity reference field is set to be required
* stale/orphaned entities the entity reference field is attached to
* `civicrm_value_*` records with all-empty columns (or are they being taken care of already)
## Example use-case
Imagine a custom entity named _Qualification_ with (among others) an entity reference custom field referencing a _Contact_ entity, which is set to be required; read: A _Contact_ has multiple _Qualifications_.
## Current behaviour
When deleting the contact, the _Qualification_ entity will not be deleted until done so manually. The reference is just being set to `NULL` instead. Those entities will be meaningless without a referenced contact, the required entity reference custom field will be empty, which breaks the semantic contract.
Question: Is the `SET NULL` behavior a FK constraint on the DB table or part of some logic in Core code?
## Proposed behaviour
When deleting the contact, the _Qualification_ entity should be deleted. This is what we'd have configured the entity reference field to behave, as in other scenarios, the current behavior will be totally fine.
So, we'll need a configuration option for _Entity Reference_ custom fields for selecting what should happen upon deletion of the referenced entity. Available options could be (basically MySQL foreign key constraints):
* Remove Reference (`SET NULL`) - current behavior, maybe also the default option
* Cascade Delete (`CASCADE`) - removes the _`civicrm_value_*`_ record and the entity the custom group is attached to
* Restrict Delete (`RESTRICT`) - just as you can't e. g. delete contacts with financial transactions
* Set Default (`SET DEFAULT`) - to be configured on the custom field
* ~~Keep Reference (`NO ACTION`)~~ - might not be a good idea
## Comments
@colemanw and @michaelmcandrew might be interested.
Currently, this can be partly achieved by implementing a `preDelete` event for retrieving and deleting referencing entities.https://lab.civicrm.org/dev/core/-/issues/4516Rethinking 20 minutes QF session timeouts2023-10-05T15:27:38ZJonGoldRethinking 20 minutes QF session timeoutsThe 20 minute timeout has been a part of Civi for as long as I can remember. It's also caused lost donations, frustrated applicants, and so on. Reading an [article about why short session timeouts aren't helpful](https://www.sjoerdlang...The 20 minute timeout has been a part of Civi for as long as I can remember. It's also caused lost donations, frustrated applicants, and so on. Reading an [article about why short session timeouts aren't helpful](https://www.sjoerdlangkemper.nl/2023/08/16/session-timeout/) made me think about this for Civi.
* What is the original reason? Is it still valid?
* If we lengthen the timeout, do we need to make special arrangements around event registration when there's a maximum number of seats available?https://lab.civicrm.org/dev/core/-/issues/4504Create new campaign "on the fly" when entering new contribution: custom field...2023-09-18T15:18:01ZDetlev SieberCreate new campaign "on the fly" when entering new contribution: custom fields are not supported## Overview
When I enter a new contribution as a CiviCRM Admin user, I can select a campaign - and, when the campaign is not found, I can add a new campaign.
However, the pop up screen for entering the new campaign does not include cam...## Overview
When I enter a new contribution as a CiviCRM Admin user, I can select a campaign - and, when the campaign is not found, I can add a new campaign.
However, the pop up screen for entering the new campaign does not include campaign custom fields - what makes this feature in certain use cases dysfunctional.
## Current behaviour
1. Create a custom field for a campaign.
2. Click on Campaigns -\> New Campaign: As you see on the screenshot, you can enter something in that custom field
![grafik.png](/uploads/82dd2d0b7dcf927ddef04e6cc6a60564/grafik.png)
3. Now, click on **Contributions -\> New Contributions**
4. Click on Campaigns and (1) enter some non-existent value. Now, you can click on "+" to enter a new campaign (2)
![grafik.png](/uploads/8aaf3446170f02b6fe7a0d9af8177431/grafik.png)
5. After Click on **New Campaign** a pop up opens where you can enter a new campaign - but here, the campaign's custom field is missing:
![grafik.png](/uploads/44069feb22709c809764a1db80d5d919/grafik.png)
## Proposed behaviour
It should be possible to access the custom fields in the same way as with **Campaign -\> New Campaign**
_What should happen? How is this better? If appropriate/available, include any wireframes or mockups._
##https://lab.civicrm.org/dev/core/-/issues/4500Ongoing duplicate contact creation if mismatch in civicrm_uf_match2024-01-24T21:04:46ZAllenShawOngoing duplicate contact creation if mismatch in civicrm_uf_match(I've only seen this on WordPress, but the relevant file is in core, so I'm filing under dev/core).
**Summary:**
Under certain circumstances involving "crossed wires" in the `civicrm_uf_match` table, a logged-in user's mere activity in...(I've only seen this on WordPress, but the relevant file is in core, so I'm filing under dev/core).
**Summary:**
Under certain circumstances involving "crossed wires" in the `civicrm_uf_match` table, a logged-in user's mere activity in CiviCRM will create a new contact (containing only the user's email address) for each CiviCRM page load (or ajax call, etc.). Left un-checked, this can lead to thousands of duplicate contacts containing only an identical email address.
**Scope of impact:**
I've seen this on a handful of WordPress sites over the past 5-7 years. I've not heard others in the community mention it, nor seen an existing issue on l.c.o.
**Example steps to reproduce:**
This is only one example; there are surely other steps that will get us there. See "Requisite data conditions" below.
These steps are for repro under WordPress. I haven't tried this under other CMSs.
1. Create WP user _a_ with email address _a@example.com_. Observe this creates a corresponding civicrm contact; we'll call this contact C1. Observe that the entry in `civicrm_uf_match` is correct (i.e. Summary tab for contact C1 shows the user ID of user _a_).
2. Change the email address for user _a_ to _a2@example.com_. Observe that the `civicrm_uf_match` link is preserved (but also that the email address in `civicrm_uf_match.uf_name` is unchanged).
3. Delete contact C1 (permanently or to trash). Observe entry in `civicrm_uf_match` is deleted, and user _a_ still exists.
4. Create a new contact (which we'll call C2) with another email address, e.g. _aFoo@example.com_.
5. Use CiviCRM's "Create User Record" feature to create a new WP user for contact C2; specify any username you like, but we'll assume username _a2_. Observe that the entry in `civicrm_uf_match` is correct (i.e. Summary tab for contact C2 shows the user ID of user _a2_), and that user _a2_ has email address _aFoo@example.com_.
6. Change contact C2's email address to _a2@example.com_. Observe that the email address in `civicrm_uf_match.uf_name` is updated, and the emmail address for WP user _a2_ is unchanged.
At this point, you'll have this state of data:
| wp_users.ID* | wp_users.user_login | civicrm_uf_match.id* | civicrm_uf_match.contact_id* | civicrm contact primary email | civicrm_uf_match.uf_name | wp_users.user_email |
|-------------|---------------------|---------------------|-----------------------------|-------------------------------|--------------------------|---------------------|
| 64 | a | NULL | NULL | NULL | NULL | a2@example.com |
| 65 | a2 | 162 | 75512 | a2@example.com | a2@example.com | aFoo@example.com |
_\* IDs in this table are from my real data; yours will differ of course._
7. As for permissions, I granted user _a_ the WP Administrator role, which has _Administer CiviCRM_; you could probably repro with narrower permissions.
8. Log in as user _a_. Perform a CiviCRM search for contacts having email address _a2@example.com_. Observe the result count N.
9. **Bad Behavior:** Do just about anything in CiviCRM. For example, refresh the search just performed. Observe the result count is >N.
10. Repeat step 9 and observe the increasing number of contacts with email address _a2@example.com_.
**Requisite data conditions**:
As mentioned above, there are probably many possible repro recipes, but the key is to arrive at this requisite state of data:
Given a specific WP user e.g. username _a_, email address _a2@example.com_, new contacts are created containing only this email address, each time this logged-in user takes any action (or presumably almost any action) in CiviCRM, as long as:
1. The WP user ID is not represented in `civicrm_uf_match.uf_id`; and
2. The WP user email (_a2@example.com_) is associated with one or more CiviCRM contacts; in a list of these contacts, sorted by `is_primary DESC, contact_id`, the first contact in that list is represented in `civicrm_uf_match.contact_id`; and
3. The WP user email (_a2@example.com_) is represented in `civicrm_uf_match.uf_name`.
**Relevant code:**
This all seems to be primarily handled by `CRM_Core_DAO_UFMatch::synchronizeUFMatch()`. The above "requisite data conditions" are a summary of the logical path in that method that leads to the stated problem.
Interestingly, the code seems to be aware of this problem, as in [line 284](https://github.com/civicrm/civicrm-core/blob/master/CRM/Core/BAO/UFMatch.php#L284) it defines a variable `$msg` which would read along the lines of (using values from my data-state above) "Contact ID 162 is a match for WordPress user 64 but has already been matched to 65" -- but that message is never used or logged anywhere.
**Workarounds:**
My simplest fix has been simply to update civicrm_uf_match via SQL to link the right user with the right contact, like so (again using values from my real data above): `update civicrm_uf_match set uf_id = 64 where contact_id = 75512;`. And then of course to remove all of the duplicate contacts.
I thought _Synchronize Users to Contacts_ might do the trick, but it has no affect on the relevant data, and the bad behavior persists.
(Joinery reference: F#1180, F#1339)https://lab.civicrm.org/dev/core/-/issues/4490Feature request - Queue api should respect maintenance mode2023-09-20T02:39:21ZeileenFeature request - Queue api should respect maintenance modeProposal to add a maintenance mode setting that is respected by the Queue api such that it would not permit `claimItems` while this setting is enabled - except for possibly upgrade tasks?
Note that when coworker respects this it should ...Proposal to add a maintenance mode setting that is respected by the Queue api such that it would not permit `claimItems` while this setting is enabled - except for possibly upgrade tasks?
Note that when coworker respects this it should log a message with a priority of `warning` so that a sysadmin can see that although the process is running no tasks are.
Chat discussion to date https://chat.civicrm.org/civicrm/pl/9h93yabz33g17qa3apcet8irdy
The summary of the chat is that `claimItem` api should be the place where maintenance mode is checked & heeded. This would be for 'persistent queues' - ie `CRM_Queue_Queue_Memory` would not check it.
- [ ] The claim item/s functions should call a symfony listener - returning false in this piece of code if an external listener (threshold check) sets the status to false
- [ ] we should add a setting for Maintenance mode (keep it simple) - call it `queue_maintenance_mode`? ClaimItem should return FALSE if this is TRUE
**Notes**
1) the following query will determine whether all queues have finished once maintenance mode is enabled:
`SELECT * FROM civicrm_queue_item where release_time > NOW()`
If future scheduling is in use (civi-rules?) then this might not suffice and if that ever arises as a feature someone wants to work on they could add a suitable field to the `civicrm_queue_item` table (`is_future_scheduled` ?)
2) I can't recall what we decided re should claimItems support an override - ie if you took everything else down in order to keep one queue running & want that queue to still claim items.... perhaps I'll find ithttps://lab.civicrm.org/dev/core/-/issues/4482Action schedule effective start and end dates ignored when using an absolute ...2023-08-09T13:56:33ZjamieAction schedule effective start and end dates ignored when using an absolute trigger dateHere's an action schedule that did not go as expected:
![image](/uploads/8205f1e5dbba0c9f0b056b72d4f6c261/image.png)
Notice how there is an absolute date as the "When (trigger) date" *that does not allow you to enter a time* and also a...Here's an action schedule that did not go as expected:
![image](/uploads/8205f1e5dbba0c9f0b056b72d4f6c261/image.png)
Notice how there is an absolute date as the "When (trigger) date" *that does not allow you to enter a time* and also an "Effective Start Date" that *does allow you to enter a time.*
In the [code](https://github.com/civicrm/civicrm-core/blob/4e4de633c452fd2e9a36df61bdbdb258b9f88ed7/Civi/ActionSchedule/RecipientBuilder.php#L418) we seem to ignore the effective start and end dates when an absolute date is specified.
So, this email was sent at 12:05 am on August 5th instead of 8:05 am on August 5th.
I get it. If you are specifying an absolute start date, why would you need an effective start date?
Well, in this case it's because you are fine-tuning with a time.
I see three options:
1. Change the UI so that the effective start and end dates are greyed out if you enter an absolute date. This seems like the easiest to implement.
2. Change the code so that we don't ignore the effective start and end date when using an absolute date. I'm not crazy about messing with the Action Schedule code. But, this seems like it could be pulled off without a lot of work or risk.
3. Allow setting an absolute date with a time attached to it (and also doing number two). I would rather have a root canal than be responsible for this one. But, it does seem like the best optionhttps://lab.civicrm.org/dev/core/-/issues/4464Authentication tokens: session already active - different user2023-08-04T12:51:21Zaydunsaidan.saunders@squiffle.ukAuthentication tokens: session already active - different userOverview
----------------------------------------
See #4463 and https://lab.civicrm.org/dev/core/-/issues/4462 This improvement issue relates to handling the case where the new user is different to that of the existing session.
From [t...Overview
----------------------------------------
See #4463 and https://lab.civicrm.org/dev/core/-/issues/4462 This improvement issue relates to handling the case where the new user is different to that of the existing session.
From [this chat](https://chat.civicrm.org/civicrm/pl/n4cr3jownifexnjda3k45qhura):
@totten says:
- if the active session is same user, then no problems. just proceed.
- if the active session user differs from the requested user, then you kinda have to choose between:
- killing the old session and starting a new one
- changing the userID of the live session. (but then you're liable to leak session-data in hard-to-predict ways)
- aborting the request
For the already active session with different user - I think it would be ideal to show a prompt and confirm that they want to logout/switch-user.