CiviCRM Core issueshttps://lab.civicrm.org/dev/core/-/issues2024-01-02T21:02:33Zhttps://lab.civicrm.org/dev/core/-/issues/4251Track Contact `image_URL` files in the `civicrm_file` table2024-01-02T21:02:33ZcolemanwTrack Contact `image_URL` files in the `civicrm_file` table# History
**Note: CiviCRM is a large, old project with many contributors, which makes "big picture" perspectives cumbersome to gather. Often a contributor simply wants to fix a bug or add a small feature, not dive into decades-long hist...# History
**Note: CiviCRM is a large, old project with many contributors, which makes "big picture" perspectives cumbersome to gather. Often a contributor simply wants to fix a bug or add a small feature, not dive into decades-long history and contemplate massive refactoring. The story of the `civicrm_contact.image_URL` field is a microcosm of the complicated world of CiviCRM.**
- The `civicrm_contact.image_URL` field (added v1.1) predates the `civicrm_file` table (added v1.5), which may explain why it was originally designed as a simple textfield with no file-management. It is a simple varchar and can store the url to any image file on the web.
- Originally, the UI allowed contact image files to be uploaded to a publicly-readable directory on the webserver, and `image_URL` stored an absolute url to that file.
- In 2014, [security hardening](https://issues.civicrm.org/jira/browse/CRM-14499) led to the addition of an `.htaccess` rule which blocked the contents of that directory from public visibility.
- This accidentally broke contact images, leading to [a rushed fix](https://github.com/civicrm/civicrm-core/pull/3058) which created `CRM_Contact_Page_ImageFile` at `civicrm/contact/imagefile`. This path allows open access to all contact images, but not other files, by querying `civicrm_contact.image_URL` for a matching filename before outputting the contents. An upgrade script rewrote all local `image_URL` fields to point to this path, using an absolute URL. This solution works but is very slow on big databases due to the unindexed query.
- During this rushed fix, it doesn't appear that consideration was given to html escaping of `&` characters. The default behavior of `CRM_Utils_System::url` is to escape `&` to `&` which is (IMHO) a bad default and certainly a poor choice for storing url strings in the database, but it was the default and no one changed it, and then Tim inadvertently cemented it with 065034510d48b722844b2007f8016ea644bd0cbd so now in order to safely read the urls you must pass them through the aptly-named `CRM_Utils_String::unstupifyUrl()` function.
- In 2016 there was an [attempt to fix the absolute URL and performance issues](https://github.com/civicrm/civicrm-core/pull/9237) which updated all the `image_URL` fields to relative paths.
- But this would have broken Drupal Views and other tools that rely on the convenience of querying the field from the db and outputting the url directly (in the future, SearchKit would rely on this too).
- A [compromise solution was reached](https://github.com/civicrm/civicrm-core/pull/9241) which rewrites the url at runtime on Civi pages. A `image_URL` like `http://wp.demo/civicrm/contact/imagefile/?photo=abc.jpeg` would get rewritten to `http://wp.demo/civicrm/file?reset=1&filename=abc.jpeg&mime-type=image/jpeg`. For reasons not entirely clear, this goes through a different internal path (`civicrm/file` instead of `civicrm/contact/imagefile`).
- In 2019, thinking it was unused, [Eileen removed support](https://github.com/civicrm/civicrm-core/commit/2762985c11e01ede473d86a33af279bc341f9a46) for passing `filename=` into the `civicrm/file` path, since that endpoint is typically supplied with a file id from the `civicrm_file` table plus a security hash.
- This accidentally broke contact images, leading to [a rushed fix](https://github.com/civicrm/civicrm-core/pull/13663) which added back the ability to get files by name from the `civicrm/file` path (since contact images are not tracked in `civicrm_file` and don't have an `id`). With the benefit of the history laid out above, a better fix might have been to switch to using the `civicrm/contact/imagefile` path and keep the `civicrm/file` path secure.
- In 2021 [I proposed adding](https://lab.civicrm.org/dev/core/-/issues/2755) an `image_file_id` FK field to the `civicrm_contact` table to track uploaded files. This proposal was met with approval, but when I recently tried to implement it I realized that the `civicrm_file` table already has an FK to `civicrm_contact` and circular references are not allowed.
- In 2023 [an option group was added](https://github.com/civicrm/civicrm-core/pull/25904) for the previously unused column `civicrm_file.file_type_id`. One possible use for that field would be to designate a file type of `"contact_image"`.
# Current Situation
The `civicrm_contact.image_URL` field can still store any url string pointing to a file on or off the server. It could point to any image on the internet, and would work fine. But if it's a file uploaded via the Civi UI, it will be an absolute link pointing to the `civicrm/contact/imagefile` path with a `photo=filename` argument. If CiviCRM recognizes this pattern it will rewrite it on core pages to the other path at `civicrm/file`, otherwise it will leave it alone.
For the confused, yes contact images are accessible at two paths, and neither is a direct link to the file on disk:
| Path | `civicrm/contact/imagefile` | `civicrm/file` |
| ------ | ------ | ------ |
| Class | `CRM_Contact_Page_ImageFile` | `CRM_Core_Page_File` |
| Permission | _none_ | "access uploaded files" |
| Args | `photo` | `filename`, `mime-type` |
| Uses | Stored in `image_URL` field as absolute URL. Output by Views & SearchKit | `image_URL` rewritten to this path on Civi pages for logged-in users |
**This situation leads to the following quirks and problems**
1. The absolute url to `civicrm/contact/imagefile` works great in Views and SearchKit... as long as the site name never changes! Otherwise, absolute URLs are a pain.
2. The url is still stored with html-escaped `&` characters that must be unstupified.
3. Anyone can access a contact image via the 1st path if they know the filename, however, the 2nd has a permission check which means logged-in users without "access uploaded files" cannot see contact images even though anonymous users can!
4. The security hash usually required by `civicrm/file` can be circumvented if you know the filename and mime-type. But the risk is mitigated by that path requiring "access uploaded files" permission.
5. There is still no file-management of contact images. Deleting a contact does not delete their image file. Deleting or changing a contact image also doesn't delete the old one.
# Proposal for File Management
1. Stop using `civicrm/file` for all contact images and [restore the patch to remove support for `filename`](https://github.com/civicrm/civicrm-core/commit/2762985c11e01ede473d86a33af279bc341f9a46).
2. Include `cid` as an argument to `civicrm/contact/imagefile` (and update stored paths accordingly) to fix the unindexed query. Also add `is_deleted = 0` to the query.
3. Add an option_value `"contact_image"` to the option group for `civicrm_file.file_type_id`.
4. When uploading a new contact image, create a record in `civicrm_file` table, and designate it `file_type_id` = `"contact_image"`.
5. Also create a record in `civicrm_entity_file` for contact images.
6. Add a virtual APIv4 field for the contact entity `image_file_id` which would allow getting/setting the file id. When setting a new file id, regenerate the `image_URL` with a post hook.
# Thoughts on Absolute URLs
All of these changes would result in better file management, but still doesn't solve the absolute url issue. This is tricky to solve because Views and SearchKit still rely on being able to output the image url directly from a query. Here are a few ideas for that one:
1. Bite the bullet and update all `image_URL` fields pointing to a local file to use a relative URL. SearchKit will still work. Views and other SQL-based tools will still work unless embedded on a remote site. Random offsite images will be unaffected.
2. Keep `image_URL` absolute but add an APIv4 virtual field like `Contact.image` which calculates the url at runtime. This satisfies moree use-cases but at the expense of adding complexity to an already overcomplicated situation.https://lab.civicrm.org/dev/core/-/issues/4250Expose the mailing/mosaico template when viewing mailing report2023-06-19T23:42:32ZyashodhaExpose the mailing/mosaico template when viewing mailing reportCurrently, we don't show which mosaico template was used for a mailing in mailing reports.
It could be helpful as sometimes the template could be deleted and when the same mailing is re-used then mosaico is broken in the copied mailing....Currently, we don't show which mosaico template was used for a mailing in mailing reports.
It could be helpful as sometimes the template could be deleted and when the same mailing is re-used then mosaico is broken in the copied mailing. We could show name/id and deleted if not present.https://lab.civicrm.org/dev/core/-/issues/4245FormBuilder/SearchKit: add 'enabled' field2023-04-20T06:55:21Zaydunsaidan.saunders@squiffle.ukFormBuilder/SearchKit: add 'enabled' fieldOverview
----------------------------------------
Suggestion: add an 'enabled' field for both Searches and Forms (maybe Segments and related entities as well).
Why?: quite often I clone something, make some changes, switch to that - but...Overview
----------------------------------------
Suggestion: add an 'enabled' field for both Searches and Forms (maybe Segments and related entities as well).
Why?: quite often I clone something, make some changes, switch to that - but don't want to delete the original yet until I'm sure I don't want to revert to it. Marking it as 'not enabled' helps search & form management particularly as the numbers increase.https://lab.civicrm.org/dev/core/-/issues/4241Refresh custom fields names in "Reuse an existing set"2023-04-18T06:50:39ZkristinecRefresh custom fields names in "Reuse an existing set"Overview
----------------------------------------
When adding a new custom field and selecting "reuse an existing set" of custom fields, the dropdown list for some of the multiple choice option sets do not pick up the custom field set na...Overview
----------------------------------------
When adding a new custom field and selecting "reuse an existing set" of custom fields, the dropdown list for some of the multiple choice option sets do not pick up the custom field set names. For example, you have a multiple choice question called "Educational Status" in Custom Field Set A and the same name "Educational Status" in Custom Field B.
Current behaviour
----------------------------------------
In CiviCRM 5.59, the custom field set labeling are missing for some old custom fields (made before a certain Civi update?).
![Screenshot_2023-04-15_111421](/uploads/dc83d16e1c6fcf8a2e9cce1961260909/Screenshot_2023-04-15_111421.png)
Proposed behaviour
----------------------------------------
The word "refresh" or "reset" the dropdown list probably isn't the right word. But the expected behavior should appear as:
Custom Field Set A:: Educational Status
Custom Field Set B:: Educational Status
P.S. And yes, technically if the options in "Educational Status" are exactly the same, this field set should have been shared rather than exist in two separate custom field sets.https://lab.civicrm.org/dev/core/-/issues/4235Simplify cron setup2023-04-18T20:10:24Zaydunsaidan.saunders@squiffle.ukSimplify cron setupOverview
----------------------------------------
A recurring problem for new users is how to get cron running for scheduled jobs.
We have documentation but it's still a stumbling block for many new users as shown by recurring questions...Overview
----------------------------------------
A recurring problem for new users is how to get cron running for scheduled jobs.
We have documentation but it's still a stumbling block for many new users as shown by recurring questions on Stack Exchange and chat, and (just now) discussed on the FormBuilder call.
Part of the problem is that it involves the OS and/or the hosting platform (cPanel etc) and we generally shy away from documenting anything in those areas but the collective brains of the Civi world surely can come up a way to make this work more easily.
Advanced users can obviously do their own thing, but the aim here is to lower the barrier for new installations to having a functioning cron.https://lab.civicrm.org/dev/core/-/issues/4203add end date to petition2023-03-27T06:52:21Zjamieadd end date to petitionOverview
----------------------------------------
This proposed feature would add a new field to petitions called: End date. If someone tries to sign a petition that has an end date in the past, they would get a friendly error warning t...Overview
----------------------------------------
This proposed feature would add a new field to petitions called: End date. If someone tries to sign a petition that has an end date in the past, they would get a friendly error warning them that the petition is no longer active.
Current behaviour
----------------------------------------
Now, we can create petitions and manually disable them. However there are two problems:
1. If you disable a petition it no longer shows up in advanced search so you can no longer find signatories.
2. It's easy to forget to disable a petition. That means these potentially spam-collecting forms can live on for too long
3.
Proposed behaviour
----------------------------------------
By adding an end date when you create the petition, you can set it up and forget about it.
Comments
----------------------------------------
I'm open to taking a crack at implementing, but wanted to first get feedback to see if this is the best way to go about it or if there are any potential pitfalls.https://lab.civicrm.org/dev/core/-/issues/4194Proposal - remove some fields from dedupe field options2023-03-27T02:42:05ZeileenProposal - remove some fields from dedupe field optionsWhen creating dedupe rules there are some fields in the drop down that don't make sense to me
![image](/uploads/87e8fe511a5e795ce362dd7027705505/image.png)
Specifically
**Irrational (technical term)**
(if these are present the dedupe...When creating dedupe rules there are some fields in the drop down that don't make sense to me
![image](/uploads/87e8fe511a5e795ce362dd7027705505/image.png)
Specifically
**Irrational (technical term)**
(if these are present the dedupe rule functionality won't apply)
- 'id' => 'Contact ID',
- 'external_identifier' => 'External Identifier',
**Dumb (technical term)**
- Contact.is_deceased
- 'addressee_id' => 'Addressee',
- 'addressee_custom' => 'Addressee Custom',
- 'do_not_email' => 'Do Not Email',
- 'do_not_mail' => 'Do Not Mail',
- 'do_not_phone' => 'Do Not Phone',
- 'do_not_sms' => 'Do Not Sms',
- 'do_not_trade' => 'Do Not Trade',
- 'email_greeting_id' => 'Email Greeting',
- 'email_greeting_custom' => 'Email Greeting Custom',
- 'external_identifier' => 'External Identifier',
- 'image_URL' => 'Image Url',
- 'is_opt_out' => 'No Bulk Emails (User Opt Out)',
- 'postal_greeting_id' => 'Postal Greeting',
- 'postal_greeting_custom' => 'Postal Greeting Custom',
- 'preferred_communication_method' => 'Preferred Communication Method',
- 'communication_style_id' => 'Communication Style',
- 'signature_html' => 'Signature Html',
- 'signature_text' => 'Signature Text',
**Marginal (laymans' term) **
- 'geo_code_1' => 'Latitude',
- 'geo_code_2' => 'Longitude',
- 'master_id' => 'Master Address ID',
<details>
<summary>Full list - excluding custom fields</summary>
```
/**
* Get the list of supportedFields to test against.
*
* This is a statically maintained (in this test list).
*
*/
public function getSupportedFields() {
return [
'civicrm_address' =>
[
'name' => 'Address Name',
'city' => 'City',
'country_id' => 'Country',
'county_id' => 'County',
'geo_code_1' => 'Latitude',
'geo_code_2' => 'Longitude',
'master_id' => 'Master Address ID',
'postal_code' => 'Postal Code',
'postal_code_suffix' => 'Postal Code Suffix',
'state_province_id' => 'State',
'street_address' => 'Street Address',
'supplemental_address_1' => 'Supplemental Address 1',
'supplemental_address_2' => 'Supplemental Address 2',
'supplemental_address_3' => 'Supplemental Address 3',
],
'civicrm_contact' =>
[
'addressee_id' => 'Addressee',
'addressee_custom' => 'Addressee Custom',
'id' => 'Contact ID',
'source' => 'Contact Source',
'contact_sub_type' => 'Contact Subtype',
'do_not_email' => 'Do Not Email',
'do_not_mail' => 'Do Not Mail',
'do_not_phone' => 'Do Not Phone',
'do_not_sms' => 'Do Not Sms',
'do_not_trade' => 'Do Not Trade',
'email_greeting_id' => 'Email Greeting',
'email_greeting_custom' => 'Email Greeting Custom',
'external_identifier' => 'External Identifier',
'image_URL' => 'Image Url',
'legal_identifier' => 'Legal Identifier',
'legal_name' => 'Legal Name',
'nick_name' => 'Nickname',
'is_opt_out' => 'No Bulk Emails (User Opt Out)',
'organization_name' => 'Organization Name',
'postal_greeting_id' => 'Postal Greeting',
'postal_greeting_custom' => 'Postal Greeting Custom',
'preferred_communication_method' => 'Preferred Communication Method',
'preferred_language' => 'Preferred Language',
'sic_code' => 'Sic Code',
'user_unique_id' => 'Unique ID (OpenID)',
'sort_name' => 'Sort Name',
'communication_style_id' => 'Communication Style',
],
'civicrm_email' =>
[
'email' => 'Email',
'signature_html' => 'Signature Html',
'signature_text' => 'Signature Text',
],
'civicrm_im' =>
[
'name' => 'IM Screen Name',
],
'civicrm_note' =>
[
'note' => 'Note',
],
'civicrm_openid' =>
[
'openid' => 'OpenID',
],
'civicrm_phone' =>
[
'phone_numeric' => 'Phone',
'phone_ext' => 'Phone Extension',
],
'civicrm_website' =>
[
'url' => 'Website',
],
];
}
```
</details>
'sic_code' => 'Sic Code',
'user_unique_id' => 'Unique ID (OpenID)',https://lab.civicrm.org/dev/core/-/issues/4192Improvement: Required? when creating Custom Field2023-03-23T08:14:19ZStoobImprovement: Required? when creating Custom FieldAfter many years in my position working with clients, and even explicit training on this issue, clients continue to flummox themselves by not reading the 'help text' and clicking **Required? [ ]** in a Custom Data field ... when that in ...After many years in my position working with clients, and even explicit training on this issue, clients continue to flummox themselves by not reading the 'help text' and clicking **Required? [ ]** in a Custom Data field ... when that in fact isn't their intent.
Proposal Pt 1: Expanding the caution to the label itself like so: **[ ] Always require a value?**
Proposal Pt 2: Clarifying the help text like so. **This forces a value in this field to create or edit this type of record. To make a field required on a specific form, use a Profile.**
In a comparison other software:
Salesforce:
`[ ] Always require a value in this field in order to save a record`
Network for Good:
`[ ] Required?` same as Civi but I like their help text a bit better than Civi's `If "Yes" this custom field will be required when manually adding/editing contacts.`https://lab.civicrm.org/dev/core/-/issues/4181Custom fields on multiple entities2023-10-13T15:42:00Zaydunsaidan.saunders@squiffle.ukCustom fields on multiple entitiesOverview
----------------------------------------
Currently Custom Fields 'extend' one type of entity, possibly limited to one or more subtypes(*) of that entity.
Sometimes it would be useful to allow a set of Custom Fields to be linked...Overview
----------------------------------------
Currently Custom Fields 'extend' one type of entity, possibly limited to one or more subtypes(*) of that entity.
Sometimes it would be useful to allow a set of Custom Fields to be linked to multiple entities.
[(*) - "subtypes" in a broad sense. In the case of Contacts, the fields can be linked to sub-types. For entities like Activities, fields can be linked to say Phone Calls and Meetings which are not strictly subtypes]
Example use-case
----------------------------------------
1) Suppose we want to create custom fields for 'service providers'. Some 'service providers' are Organizations (subtype SP-O), some are Individuals (subtype SP-I).
Current options are:
- link the fields to Contacts - but the fields are only relevant to small proportion of Contacts.
- duplicate the custom fields for each subtype - need to search multiple similar fields that logically should be one.
2) Suppose we want to create a field common to multiple entities (eg 'is_validated' on Memberships, Contributions & Participants). This would require separate field sets per entity.
Current behaviour
----------------------------------------
The current model comes from a single inheritance background: this set of fields 'extends' a particular entity.
Proposed behaviour
----------------------------------------
A more flexible model is to think of a set of fields 'attached' to a selection of entities.
(Vaguely like PHP traits vs class inheritance.)
Comments
----------------------------------------
Looking for feedback on this in two main areas:
1) Is this desirable?
2) How practical is it? Custom fields are used very widely so figuring out a transition plan is key. If APIv4 can bridge the gap between the current situation and a new world then it becomes feasible. @colemanw - I know you've already done a load of work with APIv4 and Custom Fields and probably best placed to answer this bit.https://lab.civicrm.org/dev/core/-/issues/4169Let "Number" and "Money"-type custom fields be nullable2023-03-13T07:39:11ZnoahLet "Number" and "Money"-type custom fields be nullableOverview
----------------------------------------
There are many times when NULL would be a meaningful/useful value in a custom field of type "Number" (float) or "Money" (decimal). However, it is not currently possible, via the form laye...Overview
----------------------------------------
There are many times when NULL would be a meaningful/useful value in a custom field of type "Number" (float) or "Money" (decimal). However, it is not currently possible, via the form layer, to set these fields to NULL. Blank values are currently turned into zero.
Example use-case
----------------------------------------
Let's say we need to track the elevation (meters above or below sea level) of the addresses in our database.
1. Create a custom field "Geographic Elevation" of type "Number", extending the Address entity. Leave the "Default value" blank.
1. Go to a contact and edit one of their addresses, or create a new one.
1. On the create/edit form, leave the "Geographic Elevation" field blank -- because let's say we don't know the elevation at this address.
1. Submit the form and view the saved value.
Current behaviour
----------------------------------------
"Geographic Elevation" is set to zero. That means sea level. But that's not correct -- we actually don't know the elevation.
Proposed behaviour
----------------------------------------
When the field submitted with a blank value (empty string), the database field should be set to NULL, and displayed as blank.
Comments
----------------------------------------
NULL is also a meaningful value for Money fields. Say we have a field on individuals called "Net Worth". Zero (the person is destitute) and NULL (we don't know how much money they have) are quite different.
It _is_ possible to set the field to NULL through the API, but only by passing the string 'null' (due to a PEAR DB limitation).
<details><summary>Show API example</summary>
````php
// NULL will be saved as 0
Civi\Api4\Address::update()
->addValue('Geography.Elevation', NULL)
->addWhere('id', '=', 27)
->execute();
$address = Civi\Api4\Address::get()
->addSelect('Geography.Elevation')
->addWhere('id', '=', 27)
->execute()->single();
// [
// 'id' => 27,
// 'Geography.Elevation' => 0,
// ]
// but 'null' will be saved as NULL
Civi\Api4\Address::update()
->addValue('Geography.Elevation', 'null')
->addWhere('id', '=', 27)
->execute();
$address = Civi\Api4\Address::get()
->addSelect('Geography.Elevation')
->addWhere('id', '=', 27)
->execute()->single();
// [
// 'id' => 27,
// 'Geography.Elevation' => null,
// ]
````
</details>https://lab.civicrm.org/dev/core/-/issues/4164CiviMail send mails only to certain location type2023-03-13T11:54:35ZjmargrafCiviMail send mails only to certain location typeI have the following Usecase:
A big organization is using CiviCRM for mailing marketing of different departments. Their Contacts can have several email addresses (with different location types).
A Contact has an email address that is use...I have the following Usecase:
A big organization is using CiviCRM for mailing marketing of different departments. Their Contacts can have several email addresses (with different location types).
A Contact has an email address that is used by department A for the communication with the customer via mass mailing.
But department B wants to communicate with the customer via another email address via mass mailing.
Currently there seems to be no possibility to select which email address location type should be preferred for an specific mass mailing (of department A - while department B creates another mass mailing and wants to select their preferred location type for another mass mailing).
Possibilities in CiviCRM:
I can create different location types for the different purposes.
I can add email addresses with the corresponding location type.
I can select one email address as the primary email address.
I can select one email address for mass mailing.
Problem:
But I can not select different email addresses for different mass mailings
Possible Workaround:
I could keep the two contacts as duplicates in the system. But the customer still still want to have an overall overview over their communication with this contact - so simply having duplicates is not an attractive option.
Feature Request:
What I would need is the feature of selecting the preferred location type to be used for a specific mailing.
If the contact has no email address of this communication type the fallback option could be the primary / bulk-e-mail-address
What effort would it take to create such a feature? I guess it would make most sense to implement it into the civicrm core - do you aggree?https://lab.civicrm.org/dev/core/-/issues/4142Add Activity Target consistently2023-04-24T19:25:30ZSandor SemseyAdd Activity Target consistentlyOverview
----------------------------------------
When creating related activities for other entities (e.g. Contribution, Participant (event registration), Membership) source and target contact IDs holds different contacts and have diffe...Overview
----------------------------------------
When creating related activities for other entities (e.g. Contribution, Participant (event registration), Membership) source and target contact IDs holds different contacts and have different meanings if a logged-in user added the entity or an anonymous user (through an online form).
This creates inconsistency and makes any downstream processing harder: searching, creating extensions & automations based on these activities.
Example use-case
----------------------------------------
If a contact get registered for an event by a logged-in user (someone showed interest in email and a backoffice user adding the registration to Civi) "Event Registration" activity's source (Added by) would be the CRM user and target (With) is the contact who wants to attend the event. This makes sense and the roles match the label: registration was *added by* the user and the *target* is the contact. Also works well as audit trails.
However, if the contact is using the online registration form source will be the contact and target will be empty.
Proposed behaviour
----------------------------------------
In my opinion best case would be: if there's no logged in user then source is the system user and target is the contact. This presents exactly what happened: a contact has registered and registration was added by the system, not by a real user manually.
If that's too much of a change or system user determination is hard across all CMS then just add contact to target (in this case source and target would be the same contact_id). This way also has the benefits of the same results for consumers.
In the latter case the downside is there would be redundancy in `civicrm_activity_contact` table (https://lab.civicrm.org/dev/core/-/issues/2450#note_55527). Regarding this, I argue that consistency worths more than disk space.
Comments
----------------------------------------
I've found some issues that mention this, but not as the main topic (https://lab.civicrm.org/dev/core/-/issues/2450) or hasn't any progress (https://lab.civicrm.org/dev/core/-/issues/2669). As it would be nice to change this through all of Civi.
PR at https://github.com/civicrm/civicrm-core/pull/256505.60.0https://lab.civicrm.org/dev/core/-/issues/4136Saving Repeating Events causes Registration date to be copied and/or overwritten2023-02-20T06:29:43ZshaneonabikeSaving Repeating Events causes Registration date to be copied and/or overwrittenOverview
----------------------------------------
Repeating Events are amazing for structuring recurring events. For a client, we had about 12 events scheduled and I wanted to add a new payment processor to those events. Upon adding tha...Overview
----------------------------------------
Repeating Events are amazing for structuring recurring events. For a client, we had about 12 events scheduled and I wanted to add a new payment processor to those events. Upon adding that to the Fees , and applying it to all the Events it changed all the ```Registration dates``` to match the first event :face_palm:
Reproduction steps
----------------------------------------
1. Create an Event
1. Set some fees and one payment processor
1. Set a Start date and Registration date a week before the start date
1. Set the repeat tab to ```Repeats every```: 1 month on the Third Thursday
1. Set an End after 11 occurrences
1. Save the Event choosing ```Every Event``` in the popup
1. Edit one of the Events
1. Check off another payment processor
1. Save the Event
1. The Registration date has changed on all the events
Expected behaviour
----------------------------------------
Basically, I was thinking that either A) we detect this is a modification and don't propagate OR B) we just don't propagate that field since it doesn't seem to make sense (at least to me) to do that on repeating events.https://lab.civicrm.org/dev/core/-/issues/4134SearchKit: Add TIMESTAMPDIFF to calculate between two dates in years, months,...2023-08-30T15:49:10ZfrancescbassasSearchKit: Add TIMESTAMPDIFF to calculate between two dates in years, months, weeks, etc.Overview
----------------------------------------
Allow to calculate number of days, weeks, months, years, etc. between two dates.
[DATEDIFF](https://github.com/civicrm/civicrm-core/pull/24875) only allows to calculate difference betwee...Overview
----------------------------------------
Allow to calculate number of days, weeks, months, years, etc. between two dates.
[DATEDIFF](https://github.com/civicrm/civicrm-core/pull/24875) only allows to calculate difference between two days in days. Instead, TIMESTAMPDIFF allows set the unit (years, months, weeks, etc.) units in which you want to get the difference between two dates.
Eg. to calculate age of a contact at the time they registered for an event between contact birthday and registration event date.
Current behaviour
----------------------------------------
Can't calculate years between two dates.
![datediff](/uploads/b7a37ccd3a0c048750934fadc5cf5763/imatge.png)
Proposed behaviour
----------------------------------------
Can calculate years (and FRAC_SECOND (microseconds), SECOND, MINUTE, HOUR, DAY, WEEK, MONTH, QUARTER) between two dates.
![timestampdiff1](/uploads/6dcc9610fd2fc195d2157007f3b71fba/Captura_de_pantalla_de_2023-02-17_09-36-56.png)
![timestampdiff2](/uploads/b6e7b2822f7a55eee279ea251536f913/imatge.png)
Comments
----------------------------------------
TIMESTAMPDIFF can in principle reproduce the same behavior as DATEDIFF. If so, current _Days between two dates_ field transformation could be replaced with _Diff between two dates_ ensuring that it continues to work for old cases.https://lab.civicrm.org/dev/core/-/issues/4098Feature Request: Ability to hide FormBuilder Search Forms when there are no r...2023-01-30T08:30:41ZbrienneFeature Request: Ability to hide FormBuilder Search Forms when there are no results foundOverview
----------------------------------------
By the combined powers of SearchKit and FormBuilder, you can create Search Forms and display them on the Contact Summary Page. While you can limit the type of contact on which to display ...Overview
----------------------------------------
By the combined powers of SearchKit and FormBuilder, you can create Search Forms and display them on the Contact Summary Page. While you can limit the type of contact on which to display it, i.e. Organization or Individual, it is currently not possible to hide the display when there are no results returned from the SearchKit search.
Example use-case
----------------------------------------
Creating a display block on Organizations to display the "Primary Contact". If the Organization has a primary contact, then the user would want to see that displayed in the block. If they do not have a primary contact, then a user may want to hide that display altogether.
1. Make a SearchKit search with the following criteria:
* Search for *Contacts*
* With (required) *Contact Related Contacts* if Relationship to contact = *Primary Contact of*
* Contact Type = *Organization*
1. Make a Table display for that SearchKit and configure as desired.
2. Make a Form from that display, check the *Add to Contact Summary Page* option, and select **Organization** next to the *For* label.
Current behaviour
----------------------------------------
Currently, it is not possible to hide the display block when there are no results found. Instead, you get a bright blue box on the Contact Summary Page that states "None found".
![Selection_202](/uploads/d03094d0e269f5e8529084f9e3e91ca6/Selection_202.png)
Proposed behaviour
----------------------------------------
There should a variety of options for users to configure how these blocks are displayed/not displayed when there are no options (such as the "no results options with Drupal Views).
For example, there could be a checkbox on the FormBuilder section that adds the display to the Contact Summary Page that gives the option to hide the display when there are no results. This type of checkbox could also be on the SearchKit display options.
![Selection_204](/uploads/cf167a5469addcff5093897b850b790e/Selection_204.png)
![Selection_205](/uploads/004f81cdb74c390c5568a25cfec1ec5c/Selection_205.png)
Comments
----------------------------------------
This is an unfunded feature request of suggestions for UX improvements.https://lab.civicrm.org/dev/core/-/issues/4073Problem with navigation keys stored in civicrm_settings table / proposal to m...2023-01-12T09:58:57ZJKingsnorthProblem with navigation keys stored in civicrm_settings table / proposal to move itWe've had a huge bloat in our log_civicrm_settings logging table - millions of rows. The culprit is the navigation key, which is stored in civicrm_settings with a name of 'navigation' for many contact IDs.
Every time caches are flushed,...We've had a huge bloat in our log_civicrm_settings logging table - millions of rows. The culprit is the navigation key, which is stored in civicrm_settings with a name of 'navigation' for many contact IDs.
Every time caches are flushed, the navigation cache strings are 'NULLed' by `\CRM_Core_Config::clearDBCache();` and then rewritten by
`CRM_Core_BAO_Navigation::resetNavigation();`
In practice, if I truncate log_civicrm_settings, and then run I get:
- 2785 distinct contact IDs in log_civicrm_settings
- 39270 new entries for 'navigation' in log_civicrm_settings!
My contact ID's navigation value is first NULLed and updated 13 times to different values with the same log_conn_id ! Multiply that by our 4 domains, and that's 56 entries per contact per cache flush...
---
- [ ] 1 - Why is it being updated so many times? That's a performance issue even without logging.
- [ ] 2 - Can we move the 'navigation' key out of civicrm_settings, and into a cache table? Probably its own cache table, like civicrm_navigation_cachehttps://lab.civicrm.org/dev/core/-/issues/4066Can the `CRM_Event_Badge` classes be deprecated and removed?2023-07-03T14:20:54ZBradley TaylorCan the `CRM_Event_Badge` classes be deprecated and removed?I recently came across these classes:
- `CRM_Event_Badge`
- `CRM_Event_Badge_Logo`
- `CRM_Event_Badge_Logo5395`
- `CRM_Event_Badge_NameTent`
- `CRM_Event_Badge_Simple`
Stylistically they're not great, and I noticed they don't seem to b...I recently came across these classes:
- `CRM_Event_Badge`
- `CRM_Event_Badge_Logo`
- `CRM_Event_Badge_Logo5395`
- `CRM_Event_Badge_NameTent`
- `CRM_Event_Badge_Simple`
Stylistically they're not great, and I noticed they don't seem to be in use.
The last 4 of these classes extend `CRM_Event_Badge`, and are referenced in the `eventBadge` option group. However, I can't see any scenario where this `eventBadge` option group is actually used.
The `Event Name Badge Layouts` configuration screen gets stored in the `civicrm_print_label` table, and `CRM_Badge_BAO_Badge` is what actually get's used when you print a selection of badges for an event. `CRM_Badge_BAO_Badge` does not use `CRM_Event_Badge` (although they are very similar. I suspect one started as a copy of the other.)
I think it would make sense to:
1. Check nobody is using the 5 classes listed above through a universe search.
2. Noisily deprecated the 5 classes, specifying `CRM_Badge_BAO_Badge` should be used instead.
3. After a period of time remove the 5 classes.
4. Tidy up (remove all trace of) the `event_badge` option group.
Of these, the last step is the one I'm most unsure of the process for. I guess it'd be removed through an upgrade step.https://lab.civicrm.org/dev/core/-/issues/4045Ideas for (funded) speed improvements in mailings2023-08-28T14:44:15ZAndrew WestIdeas for (funded) speed improvements in mailingsWe'd be interested in funding some speed improvements when building mailing recipients in CiviMail. They'd inherently feed through to Mosaico, I think.
We find the mailings screens pretty snappy apart from selecting recipients. There a...We'd be interested in funding some speed improvements when building mailing recipients in CiviMail. They'd inherently feed through to Mosaico, I think.
We find the mailings screens pretty snappy apart from selecting recipients. There are few bottlenecks that it'd be nice to get rid of:
**Improve the dropdown**
When you have lots of groups and mailings the dropdown doesn't work very well. It stalls, with multiple spinners appearing, or wipes out your text while typing, or says 'none found' for a bit before updating, or alternates between 'Searching' and 'Loading', and generally takes an achingly long time to find the groups in its own dropdown.
![image](/uploads/91dcd7a0a2c2e7d3230c4016d1d49db0/image.png)
Compare this to the smart group dropdown in Search Kit, which is blindingly fast.
Maybe this could be replaced with the new API4-based entityref? Possibly split into two (or even four?): Groups to include / Groups to Exclude / Mailings to include / Mailings to exclude?
**Improve generating the list of recipients**
A complicated selection of groups can easily take 30 seconds to update and save, for us. This is very frustrating when adding one smart group at a time in the dropdown. Possible fixes:
- Refactor CRM_Mailing_BAO_Mailing::getRecipients(). At the moment this function seems to create a lot of temp tables, and uses a bunch of heavy queries to compare them against each other. Again, Search Kit can do complicated includes/excludes way faster. So I figure this could be modernised.
- Possibly the slow part here is emptying/filling civicrm_mailing_recipients each time the criteria are changed. Maybe add a 'getRecipientCountPreview' function to get a count of recipients without actually populating the table? This would let people experiment in the mailing screen quickly. Then only generate the actual recipient list when the mailing is scheduled or saved as a draft?
Any other ideas? Or thoughts on whether the above makes sense?
We could fund maybe 15hrs on this atm. If anyone wants to join us that'd be great too :-)https://lab.civicrm.org/dev/core/-/issues/4041Should we hard-code messages that event registration cancellations are not re...2022-12-20T10:01:32ZlarsssandergreenShould we hard-code messages that event registration cancellations are not refundable?Self-service cancellation of event registrations does not support refunds, so there are at least two places where the registrants are told that there are no refunds (in the registration confirmation email if self-service cancellation is ...Self-service cancellation of event registrations does not support refunds, so there are at least two places where the registrants are told that there are no refunds (in the registration confirmation email if self-service cancellation is enabled and when they actually try to cancel, though this has been broken since 5.43).
I wonder if it makes sense for CiviCRM to put this expectation that there are no refunds into code. Even though refunds are not supported via the self-service form, organizations may have policies that are different and may manually process refunds in some cases. I think it would make more sense to simply not say anything about refunds and allow organizations to define their own policies in the event registration.
So my proposal: Remove the warning about no refunds from self-service cancellation (which doesn't currently work anyways), remove the text about no refunds from event registration confirmation templates and remove the help text that mentions no refunds from the self-service cancellation option on the event registration set up.
It's far from ideal either way, but I think leaving it open is better than specifying a policy that may not work for all.https://lab.civicrm.org/dev/core/-/issues/4034CRM Builder - Profiles allow multiple Formatting Free HTML fields in forms2023-01-03T14:46:14ZJonny ToomeyCRM Builder - Profiles allow multiple Formatting Free HTML fields in formsOverview
----------------------------------------
When building a conribution form, or altering the profile for a form, the Javascript rightly detects duplciate fields and prevents saving. Would like to make an exception for Free HTML fi...Overview
----------------------------------------
When building a conribution form, or altering the profile for a form, the Javascript rightly detects duplciate fields and prevents saving. Would like to make an exception for Free HTML fields (could be expanded to anything from the Formatting section if that section should expand)
Current workaround
----------------------------------------
Saving after each change gets around this for now
Proposed behaviour
----------------------------------------
alter the markDuplicates() method so that is_duplicate class is only applied when the ufFieldModel does not have the label 'Free HTML'. This will allow multiple Free HTML fields for formatting forms
Proposed solution
----------------------------------------
civicrm/js/model/crm.uf.js
![markDuplicates](/uploads/fe6b7f9c7da464418d2bd6a27f75639a/markDuplicates.PNG)