CiviCRM Core issueshttps://lab.civicrm.org/dev/core/-/issues2022-11-30T08:39:36Zhttps://lab.civicrm.org/dev/core/-/issues/4009Proposal: SearchKit show Calculated Field entity refs as joins2022-11-30T08:39:36Zaydunsaidan.saunders@squiffle.ukProposal: SearchKit show Calculated Field entity refs as joinsOverview
----------------------------------------
If you join an entity using 'With' then it appears in the 'Add' dropdown as an expandable entity: - see 'Contact Contributions' in this image:
![image](/uploads/78451866fa4f155fb32594879e...Overview
----------------------------------------
If you join an entity using 'With' then it appears in the 'Add' dropdown as an expandable entity: - see 'Contact Contributions' in this image:
![image](/uploads/78451866fa4f155fb32594879efb754d/image.png)
However entities that are joined via Calculated Fields show differently so we get fields like 'Address (billing) Country' rather than a dropdown for Address providing access to all the address fields.
With the recently merged https://github.com/civicrm/civicrm-core/pull/25056, the `master_id` field appears in the list but none of its fields.
So can we make those entities show in the same way as those joined via 'With' ? (APIv4 Explorer does this).https://lab.civicrm.org/dev/core/-/issues/4007Proposal: SearchKit Templates2022-12-05T23:15:19ZcolemanwProposal: SearchKit TemplatesBackground
-------------
Olly is working on a new Extension called [SearchKit Reports](https://lab.civicrm.org/extensions/search_kit_reports) which is a collection of SavedSearches. The idea is to reproduce most of CiviReport using Searc...Background
-------------
Olly is working on a new Extension called [SearchKit Reports](https://lab.civicrm.org/extensions/search_kit_reports) which is a collection of SavedSearches. The idea is to reproduce most of CiviReport using SearchKit, and people can then use those packaged searches to get a jump-start on building their own searches.
Since these searches are intended to be used as templates, why not actually make them templates.
Rationale
-------------
I think this would benefit the SK UI because because so far, the "Packaged Searches" tab contains stuff the average user shouldn't mess with unless they know what they're doing; searches like "Administer Custom Fields" provide critical functionality to various CiviCRM screens that could potentially be broken if messed with.
Now the [SearchKit Reports](https://lab.civicrm.org/extensions/search_kit_reports) extension is introducing a bunch of packaged searches with the opposite intention. They provide *no* functionality out of the box and we *want* the user to mess with them & experiment as much as they want. We also want to encourage a workflow where they don't directly edit the packaged version but save a copy.
Design
-----------
I'm thinking that SavedSearchTemplates would:
1. Appear on their own tab in SearchKit
2. Not be runnable outside the SearchKit Admin UI - clicking on one would pull up a new pre-configured search (the same as clicking the "Clone" button on a SavedSearch)
3. Stored in their own table `civicrm_saved_search_template`
4. Can be created, updated & packaged like regular saved searches
To point 3, I thought about adding an `is_template` column like we do in the `civicrm_event` table, and then thought about what a PITA that column is, and is it really so hard to create a new table? No, not hard at all.https://lab.civicrm.org/dev/core/-/issues/3998Add phpunit-bridge to unit testing2022-11-22T07:23:46ZDaveDAdd phpunit-bridge to unit testingSee https://github.com/civicrm/civicrm-core/pull/25009 for the results of what currently happens. The idea is that e.g. some php 8.x deprecations get hidden, but this package helps flush them out.
Some things that would need doing:
* A...See https://github.com/civicrm/civicrm-core/pull/25009 for the results of what currently happens. The idea is that e.g. some php 8.x deprecations get hidden, but this package helps flush them out.
Some things that would need doing:
* Adjusting the two propertybag tests so that they aren't trying to do the same thing. Or whatever the issue is there.
* A test listener(? or something) so that the deprecations become more visible rather than buried in the run output.
* What is "THE ERROR HANDLER HAS CHANGED!" about, and why is it in GIANT LETTERS? It appears after some test suite runs instead of where you would expect to see the deprecations. Possibly the suites where it appears somehow have their own error handlers in a way where it conflicts with phpunit-bridge.
FYI @totten @seamuslee. Not asking you to do anything just something you might be interested in.
P.S. To get a stack trace showing exactly where the deprecation is happening, you need to run locally and set SYMFONY_DEPRECATIONS_HELPER to a regex that matches the deprecation notice, as per https://symfony.com/doc/current/components/phpunit_bridge.html#display-the-full-stack-trace.
P.P.S. On windows depending on which of the 3^4 options you chose when you installed git, using `/` as the regex terminator confuses it and it gets rewritten as `C:/path/to/git/<the regex>`https://lab.civicrm.org/dev/core/-/issues/3961Proposal: Create stub extensions for civicrm core + all components2023-07-13T10:41:05ZcolemanwProposal: Create stub extensions for civicrm core + all components*Edited to reflect the current state of things, some comments reply to an earlier revision*
**Motivation 1:**
Require SearchKit in core. This has been [solved in a different way](https://github.com/civicrm/civicrm-core/pull/24739) so i...*Edited to reflect the current state of things, some comments reply to an earlier revision*
**Motivation 1:**
Require SearchKit in core. This has been [solved in a different way](https://github.com/civicrm/civicrm-core/pull/24739) so is no longer relevant to this issue.
**Motivation 2**
We are starting to package SearchKit displays as replacements for Smarty/DataTables in the UI. This has worked out great for CiviGrant: now that it's an extension we can easily package Afforms, SavedSearches and other managed entities with it. But other components are not extensions and so have no mechanism for including their own managed entities, afforms, etc. Components are less modular and more monolithic than extensions.
**Motivation 3**
Currently an extension has no way to require a component, e.g. an extension cannot declare a dependency on CiviEvent.
**Motivation 4**
Extensions and Components are similar but not the same, and this is confusing to new users and new developers. In most respects Extensions are better than Components, and it would be great to "some day" have all components simply be extensions.
**Proposal**
Given the experience of converting CiviGrant to an extension was a "harder than expected" undertaking, I think converting all other components to extensions at once is not a realistic goal. It would need to be a slow, iterative process, and I propose this as the first iteration:
1. Create a small stub extension for each component (CiviEvent, CiviContribute, etc).
2. Use hooks to sync between the `civicrm_extension` table and the `civicrm_component` table, so that enabling the "CiviEvent" extension also enables the "CiviEvent" component, and vice-versa.
3. Get rid of the "Manage Components" screen. Admins can enable/disable components via the "Manage Extensions" screen.https://lab.civicrm.org/dev/core/-/issues/3934Settings - Deprecate "serialize" flag2022-12-16T01:05:12ZtottenSettings - Deprecate "serialize" flag# Background: Basic settings
Settings are "key-value" pairs. The "value" part allows type `mixed` (*ie `string` or `int` or `float` or `array` or whatever*).
Support for `mixed` is consistent between `Civi::settings()` and `$civicrm_se...# Background: Basic settings
Settings are "key-value" pairs. The "value" part allows type `mixed` (*ie `string` or `int` or `float` or `array` or whatever*).
Support for `mixed` is consistent between `Civi::settings()` and `$civicrm_setting`.
```php
// Consistent access to a scalar
Civi::setting()->set('my_string', 'abc');
$civicrm_setting['domain']['my_string'] = 'abc';
assert Civi::setting()->get('my_string') === 'abc';
```
```php
// Consistent access to an array
Civi::setting()->set('my_array', [1,2,3]);
$civicrm_setting['domain']['my_array'] = [1,2,3];
assert Civi::setting()->get('my_string') === [1,2,3];
```
Under the hood, values are stored in MySQL using `serialize()`. You can see this in both recent and ancient code:
* 5.54's `setDb()`: https://github.com/civicrm/civicrm-core/blob/5.54/Civi/Core/SettingsBag.php#L413
* 5.54's `loadValues()`: https://github.com/civicrm/civicrm-core/blob/5.54/Civi/Core/SettingsBag.php#L137
* 4.3's `setItem()`: https://github.com/civicrm/civicrm-core/blob/4.3/CRM/Core/BAO/Setting.php#L325
* 4.3's `getItems()`: https://github.com/civicrm/civicrm-core/blob/4.3/CRM/Core/BAO/Setting.php#L216
# Background: `serialize` metadata
There is an additional option in `*.setting.php` metadata called `serialize`. Superficially, this resembles `serialize` metadata for DAO fields.
This option is defined for 8 settings: `contact_view_options`, `contact_edit_options`, `advanced_search_options`, `user_dashboard_options`, `address_options`, `contact_autocomplete_options`, `contact_reference_options`, and `quicksearch_options`.
In all cases, the setting has a serialization format of `SERIALIZE_SEPARATOR_BOOKEND`.
APIv4's `Setting` is a wrapper for `Civi::settings()`. Notably, it interprets the `serialize` metadata and applies an (additional/secondary) serialization.
# What's good about this?
It makes it easier for REST/AJAX agents to read and write those 8 historical settings (*the pre-existing settings with the annoying BOOKEND format*).
At the time of introduction, it probably had no effects on compatibility and required no upgrade steps.
# What's bad about this?
It encourages people to add unnecessary serialization layers. If you are writing a _new_ setting with structured data, then you'll see the `serialize` metadata and think: "I should add that!" Here are some docs that suggest `JSON` for a setting:
https://docs.civicrm.org/dev/en/latest/framework/setting/#settings-definition
But you shouldn't do that on a new setting! Here's why:
* __It adds complexity -- double-encoding.__ Every read and write of the data goes through both `serialize()` and some additional serialization method.
* __It adds complexity -- conflicted APIs.__ The setting can be viewed through various programmatic interfaces (e.g. local PHP array `$civicrm_setting`; local PHP method `Civi::setting()->get()`; e.g. remote REST method `Setting.get`). Some APIs present an `array` -- but others present an encoded string (like `^1^2^3^`). There is no decent reason why `$civicrm_setting`, `Civi::setting()`, and `Setting.get` should present it differently.
* __It distracts from the real data-modeling problem of serialized fields.__ We generally do simple settings (strings/ints) because they're easier. Complex settings (like `mailing_backend`) are possible, but we don't do them as much - because the tooling/validation/etc is not as good. Intuitively, complex settings could work better if we added more metadata. _But this is the wrong metadata._ Switching `mailing_backend` between `serialize`/JSON/CSV/YAML/etc doesn't make this any better. What you need is a sharper description of the data inside of that serialized blob.
* __It becomes overall harder to change.__ Perhaps settings should be stored with `json_encode()` instead of `serialize()`? `serialize()` has additional kinds of vulnerabilities. JSON has better support in MySQL. Plot out the path for how to migrate the entirety of `civicrm_setting.value` -- and compare that to switching serialization on a setting-by-setting basis. It's easier to change if individual settings are _not_ committed to specific serializers.
# What to do
Simplest thing:
* Keep the mechanism
* Present it as deprecatedhttps://lab.civicrm.org/dev/core/-/issues/3921OptionValues & search kit - missing metadata?2022-11-11T03:26:35ZeileenOptionValues & search kit - missing metadata?We want to use search kit to create a UI to cleanup some of our languages. However, most of the fields are not available in the form-builder layer (where I would want to use them as filters).
I *think* they only show if they have `html...We want to use search kit to create a UI to cleanup some of our languages. However, most of the fields are not available in the form-builder layer (where I would want to use them as filters).
I *think* they only show if they have `html` defined in the xml - and that the right thing to do is to add `html` to `value`, `name` , `id`, `label` `is_active` and possibly others but to set most of them to `read-only` (which is a bit confusing but seems to deal with the two different use cases). Probably only `label` & `is_active` need to be editable
@colemanwhttps://lab.civicrm.org/dev/core/-/issues/3872Search on pledge payments on Find contributions screen2022-09-26T20:24:23ZyashodhaSearch on pledge payments on Find contributions screenIn _Find contributions_, if _CiviPledge_ is enabled, add search on pledge payments.
![se](/uploads/37de92cf4ea25288db09be94e60df203/se.png)
We do have bunch of options for search on contributions that are recurring or not, pay later o...In _Find contributions_, if _CiviPledge_ is enabled, add search on pledge payments.
![se](/uploads/37de92cf4ea25288db09be94e60df203/se.png)
We do have bunch of options for search on contributions that are recurring or not, pay later or not, would be helpful to check if the contribution is a pledge payment or not.
And based on the components enabled, we can have for event/membership as well.yashodhayashodhahttps://lab.civicrm.org/dev/core/-/issues/3871Write-off feature for pledges2022-10-11T13:07:54ZyashodhaWrite-off feature for pledges**Functional specifications:**
On a pledge, in _Edit Scheduled Payment_, we can delete a pledge payment. The total pledge amount is reduced by the amount of the scheduled payment and the balance due is recalculated.
But in this scenar...**Functional specifications:**
On a pledge, in _Edit Scheduled Payment_, we can delete a pledge payment. The total pledge amount is reduced by the amount of the scheduled payment and the balance due is recalculated.
But in this scenario, the pledge status is left as is.
We can also cancel a pledge. All scheduled payments are canceled but the pledge status is set to canceled and the pledge balance and the total pledge amount are not recalculated. So I propose a new feature for the write-off.
- Add the menu option “Write Off” before Cancel.
- All scheduled payments are canceled like for a pledge cancellation.
- Pledge status is changed to Completed.
- Total pledge amount is reduced by the existing pledge balance amount.
- Pledge balance is set to $0.
- An activity of type “Pledge write-off” is created and recorded on the contact.
- Activity Subject=total pledge amount
- Activity Date and time = date and time of the action
- Activity Status=Completed
- Actions available on the activity should be View and File on case only like for the activity type Contribution.
- The option _Write-Off_ is available only for a pledge that has at least one pledge payment. If no pledge payment has been recorded with a pledge, write-off doesn’t make any sense, so we should have only the options to delete or cancel a pledge.
- if pledge status =Pending or Overdue > Show only Cancel and Delete
- if pledge status = In progress > show only Write-Off and Delete
- if pledge status = Completed or Canceled > show only Deleteyashodhayashodhahttps://lab.civicrm.org/dev/core/-/issues/3868Link contribution view with asssociated entity2023-05-18T23:37:57ZyashodhaLink contribution view with asssociated entityWhen you view a contribution, there's no way to know whether this contribution was done as a part of membership, pledge or an event registration.
I propose provide this additional data. Display this under total amount
e,g
![Screens...When you view a contribution, there's no way to know whether this contribution was done as a part of membership, pledge or an event registration.
I propose provide this additional data. Display this under total amount
e,g
![Screenshot_from_2022-09-23_19-15-29](/uploads/24028770f5b17e20ab40334973061b05/Screenshot_from_2022-09-23_19-15-29.png)
Membership will be a link to membership view page.
(depending on the related entity we display the link)yashodhayashodhahttps://lab.civicrm.org/dev/core/-/issues/3861add message template id to sendmail function2022-09-23T02:19:02Zmagnolia61add message template id to sendmail functionOverview
----------------------------------------
In trying to use AlterMailParams I ran into this 'bug'. The MessageTemplateID is a field that is used in the email workflow functions, but when manually sending an email using a message t...Overview
----------------------------------------
In trying to use AlterMailParams I ran into this 'bug'. The MessageTemplateID is a field that is used in the email workflow functions, but when manually sending an email using a message template the ID is not passed on.
Reproduction steps
----------------------------------------
1. Send email manually to a contact using a message template
2. the message template ID is not passed from submit function to sendmail function in CRM/Contact/Form/Task/EmailTrait.php
3. AlterMailParams cannot use the message template ID for the kind of mails
Current behaviour
----------------------------------------
Message Template ID is not passed on to send email functions
Expected behaviour
----------------------------------------
Message Template ID is not passed on to send email functions
Environment information
----------------------------------------
- CiviCRM: 5.53.0
- CMS: Drupal 7.92
- PHP: 7.4.30 (fpm-fcgi)
- Database: 10.5.15-MariaDB-0+deb11u1-log engine: InnoDB 10 row format: Dynamic, Compressed
- Webserver: Apache/2.4.54 (Debian)
Comments
----------------------------------------
PR: https://github.com/civicrm/civicrm-core/pull/24583https://lab.civicrm.org/dev/core/-/issues/3841Core (etc.) extensions should be categorized and filterable2022-10-11T10:11:26ZJonGoldCore (etc.) extensions should be categorized and filterableThe extensions page is getting harder to navigate, and isn't alphabetized by description. Finding an extension can take a while.
We should take a cue from Drupal and add `category` to the `info.xml`, then put extensions into accordions ...The extensions page is getting harder to navigate, and isn't alphabetized by description. Finding an extension can take a while.
We should take a cue from Drupal and add `category` to the `info.xml`, then put extensions into accordions by category.
Like Drupal, I don't think we need a set taxonomy, and even if only core extensions had this, it would be a good UX improvement.
I also realize that we're probably near the point where the Extensions page is built in Form Builder, but the first step is buy-in on the concept, implementation can wait a bit.JonGoldJonGoldhttps://lab.civicrm.org/dev/core/-/issues/3835Proposal: Deprecate profile standalone listing mode2023-11-30T17:03:40ZJonGoldProposal: Deprecate profile standalone listing modeProfile standalone listing code is deeply entangled with the query BAOs, and produces weird and wonderful bugs like #3834.
I don't think they're widely used in modern CiviCRM, they seem to have been a stopgap before Views integration, a...Profile standalone listing code is deeply entangled with the query BAOs, and produces weird and wonderful bugs like #3834.
I don't think they're widely used in modern CiviCRM, they seem to have been a stopgap before Views integration, and later for non-Drupal users to get a Views-esque experience. But Search Kit has effectively replaced them - they have feature parity and then some.
I think we can improve the query code if we no longer needed to worry about standalone listing mode.
Proposal:
* Remove the ability to create standalone listings on new installations of CiviCRM.
* Create a check for profiles using standalone listing mode, warning admins if they use it.
We can deprecate this over a long period of time, so the impact should be gentle.
If this proposal is approved, we may also want to consider a function that converts profile listings to SK searches. This will have added utility as we deprecate other uses of profiles (e.g. once SK replaces Advanced Search, Search Profiles will go away).https://lab.civicrm.org/dev/core/-/issues/3740Set default lock level to READ COMMITTED2023-10-20T07:57:25ZJoeMurraySet default lock level to READ COMMITTEDDrupal recently changed the default lock level slightly down from REPEATABLE READ to READ COMMITTED, which has the effect of avoiding some db locks. See https://www.drupal.org/node/3269885 We have seen over recent years some issues with ...Drupal recently changed the default lock level slightly down from REPEATABLE READ to READ COMMITTED, which has the effect of avoiding some db locks. See https://www.drupal.org/node/3269885 We have seen over recent years some issues with deadlocks, primarily on rebuilding group contact cache when there are hierarchies of smart groups. Should CiviCRM be more explicit about what isolation level we expect in order to get more repeatable results, and should we follow Drupal in its move to READ COMMITTED? I tend to think so.
See responses in MM chat from @demeritcowboy
https://chat.civicrm.org/civicrm/pl/gnmhfdxes7rgb8k7ppoaezo45ahttps://lab.civicrm.org/dev/core/-/issues/3648SMS limit is hardcoded at 460 but should be changeable somewhere2024-01-29T09:44:22ZjasonobrownSMS limit is hardcoded at 460 but should be changeable somewhereTwillio (and probably other sms providers) allows SMS messages up to 1600 characters, but Civi has the limit hardcoded at 460. I've verified that by simply going to CRM/SMS/Provider.php and changing MAX_SMS_CHAR from 460 to 1600 the sys...Twillio (and probably other sms providers) allows SMS messages up to 1600 characters, but Civi has the limit hardcoded at 460. I've verified that by simply going to CRM/SMS/Provider.php and changing MAX_SMS_CHAR from 460 to 1600 the system now accepts (and sends) messages up to 1600 characters. It seems logical that the character limit should able to be adjusted and stored somewhere rather than set as a static value in the code. We are currently using the larger limit, and would really like to see this implemented asap so that we can stop manually updating the code each time we update civi.https://lab.civicrm.org/dev/core/-/issues/3645Eventually google will require OAUTH for bounce processing and may require it...2022-12-08T20:55:07ZDaveDEventually google will require OAUTH for bounce processing and may require it for outbound SMTP through gmail serversAs noted on [stackexchange](https://civicrm.stackexchange.com/questions/34142/sending-mail-via-gmail-google-announce-username-password-authentication-will-b), Google sent out an email outlining plans to require OAUTH and discontinue supp...As noted on [stackexchange](https://civicrm.stackexchange.com/questions/34142/sending-mail-via-gmail-google-announce-username-password-authentication-will-b), Google sent out an email outlining plans to require OAUTH and discontinue support for just username/password.
The way I read it, this is mostly applying to POP/IMAP clients for reading mail, and they are *hinting* that someday they might require it for outbound SMTP, i.e. at Administer - System Settings - Outbound Mail you choose SMTP and have it set to use gmail servers. The relevant quote is
> No change is required for scanners or other devices using simple mail transfer protocol (SMTP) or LSAs to send emails. If you replace your device, look for one that sends email using OAuth.
(LSA means less secure apps, which is google-speak for username/password)
They are targeting June 2020 - Feb 2021 for IMAP etc, but as noted above they have not specifically set a date for changes regarding sending outbound emails.
It occurs to me though that bounce processing and the email processor use POP/IMAP. For the email processor you can work around it by just forwarding it to a non-gmail address and polling that, but bounce processing might be harder depending on setup.https://lab.civicrm.org/dev/core/-/issues/3556Deleting a sent mailing leaves activities with invalid links to a mailing rep...2023-09-05T10:00:34ZlolcodeDeleting a sent mailing leaves activities with invalid links to a mailing report that doesn't workAssuming: Civicrm uses the default setting "Enable CiviMail to create activities on delivery"
If a mailing is created, sent and then deleted then the recipients of that mailing will have activities that link to a mailing report using an...Assuming: Civicrm uses the default setting "Enable CiviMail to create activities on delivery"
If a mailing is created, sent and then deleted then the recipients of that mailing will have activities that link to a mailing report using an invalid id. If the link is followed the error will be:
> Sorry, due to an error, we are unable to fulfill your request at the moment. You may want to contact your administrator or service provider with more details about what action you were performing when this occurred.
> Expected one Mailing but found 0
If clicked from the popup activity view it will just be a network error.
The body of the activity will also be blank after the mailing is deleted.
I am not convinced that allowing sent mailings to be deleted at all is a good design.
Some kind of an archive function might be better.
In the short term however the link should probably be removed from these activities.https://lab.civicrm.org/dev/core/-/issues/3552Set time limit for bounce types with hold_threshold > 12023-09-05T10:00:02ZMichael McAndrewSet time limit for bounce types with hold_threshold > 1Bounce types can be split into those with a hold_threshold = 1 where a single bounce should cause this contact to be put on hold (e.g. mailbox non existent), and those where more than one bounce is required (mailbox full).
We don't curr...Bounce types can be split into those with a hold_threshold = 1 where a single bounce should cause this contact to be put on hold (e.g. mailbox non existent), and those where more than one bounce is required (mailbox full).
We don't currently set a time limit on those where more than one bounce is required, which is an oversight. For example, the bounce threshold for Mailbox full is 3. This means that as soon as we have received more than 3 mailbox full messages for a contact, their email will be put on hold. This might make sense if it was 3 in the last year, but doesn't really make sense if it was three in the last 10 years.
Similar arguments can be made for other bounce types, like Away, which is set to 30. If I send a fair amount of email to people that are out of the office quite a lot, over time a significant chunk of these will send me > 30 out of the office messages.
Hence we should only consider messages received in a certain time period. I am going to suggest one year. I would be up for making it user configurable (number of days) if people feel strongly about it.
Let me know what you think.https://lab.civicrm.org/dev/core/-/issues/3539When communcating with external network services, detect transient failures a...2024-02-12T13:57:20ZmfbWhen communcating with external network services, detect transient failures and queue for retryFor improved resiliency, CiviCRM should detect transient failures when communicating with external network services and queue for retry. e.g.
* Transactional email: Queue for sending rather than sending during the request process?
* Bul...For improved resiliency, CiviCRM should detect transient failures when communicating with external network services and queue for retry. e.g.
* Transactional email: Queue for sending rather than sending during the request process?
* Bulk mail: Catch transient failures and ensure that they are retried. PRs: https://github.com/civicrm/civicrm-core/pull/11838 https://github.com/civicrm/civicrm-core/pull/11840 (and there are additional failure scenarios possible)
* ...https://lab.civicrm.org/dev/core/-/issues/3311Proposal - store metadata on membership renewal on line item2023-06-18T00:43:47ZeileenProposal - store metadata on membership renewal on line item~~UPDATE - discussion on this is converging on adding a new json field 'metdata' to the line item table, permitting data related to that line item to be stored at point of order, and used when confirming. (UPDATE - this is not really the...~~UPDATE - discussion on this is converging on adding a new json field 'metdata' to the line item table, permitting data related to that line item to be stored at point of order, and used when confirming. (UPDATE - this is not really the convergence now - proposal updated below)~~
We have a few issues open ( https://lab.civicrm.org/dev/membership/-/issues/28 , https://lab.civicrm.org/dev/core/-/issues/1402 , https://lab.civicrm.org/dev/financial/-/issues/53 ) that are to my mind blocked by us not having a data model that holds the user's intent when renewing. The focus of this gl is the data model and exposing it via the back office renewal form & doing what is needed in the apis.
**The problem**
When renewing an expired membership the back office user has a fairly blunt way of influencing the calculated dates - they can set the 'renewal date'. However, if the payment is not made in the same form submission that intent is not captured anywhere and cannot be respected when completing the transaction. Likewise the receipt text and number of renewal terms are not captured.
**The proposal data model**
1) add membership_num_terms to civicrm_line_item table.
2) add 'payment_completion_metadata' to the ~~civicrm_line_item~~ civicrm_contribution table for data that is only used in the order->payment flow. Limit what can be stored here to tested values used in that flow - likely to be the message details and effective date the renewal form uses but doesn't sotre.
~~1. New field added to civicrm_line_item called 'metadata' which stores information required to complete the membership renewal - e.g. the field might store ``json_encode(['start_date' => '2010-01-01', 'end_date' => '2020-01-01', 'membership_status_id' => 'Current', 'membership_num_terms' => 2']~~
~~- note this is the new proposal - but I'm wondering what happens if it's renewed in the meantime & then paid~~
~~- New status 'Pending renewal' used when there is a need to record the dates (is_current_member = 0, is_reserved = 1, is_admin = 1 too I think)~~
~~- When a contribution linked to a membership in 'Pending renewal' is completed the status is updated but the dates are not changed~~
~~- If a membership has 'Pending renewal' status and status_override_end_date is set then any processing of that membership after that date would result in it being reverted to previous dates and status - this information should be available in the membership_log table.~~
**Proposed form exposure**
No proposed form changes - this is just storing data already submitted in the renewal form.
~~- the logic could be used outside of existing core forms but within core I think this is something we are exposing to the back-office user doing a renewal - they already have considerable ability to edit dates. So I think this is a form layer thing not some site-wide setting.
- when renewing an expired membership the renewal form would present the user with what will happen to the start date, end date etc if payment is received on that day and offer a checkbox to specify dates - if that is checked then date fields would be exposed an on save the Pending Renewal status would be used with the selected dates.
- potentially the user can also enter the date the 'offer' is available until - ie status_override_end_date~~
**Edge cases**
1. Multiple terms
We would add an extra field membership_num_terms to the line items column
~~https://lab.civicrm.org/dev/membership/-/issues/28 throws up the edge case that more than one renewal term might be selected. Both @jptillman and I assumed that the right way to represent that would be setting qty = n and then picking that when confirming. @andrewhunt didn't think that was the right assumption https://github.com/civicrm/civicrm-core/pull/18618. If we come down on NOT setting qty then we ALSO need a way to record intent for non-expired renewals - this could be an extra membership_num_terms column on the line_item table. (I'm still inclined to my original assumption but more points of view needed).~~
2. About to expire rather than actually expired
Out of scope for now - just do what the form currently does
~~I think we would still manage this in the form ie - "the membership is due to to expire in 4 days, if payment is received after that then .... ' and present the same options as for an actually expired membership~~https://lab.civicrm.org/dev/core/-/issues/3292Upgrade select2 to stable version 4.0.4 and fix compatibility issues in Civi2024-02-23T17:58:01ZMonish DebUpgrade select2 to stable version 4.0.4 and fix compatibility issues in CiviCurrently, Civi is using select2 3.5 which we need to upgrade to latest stable 4.0.4 (https://github.com/select2/select2/releases) and also resolve compatibility issues with CiviCRM.
Major issues:
1. Auto appending values doesn't work a...Currently, Civi is using select2 3.5 which we need to upgrade to latest stable 4.0.4 (https://github.com/select2/select2/releases) and also resolve compatibility issues with CiviCRM.
Major issues:
1. Auto appending values doesn't work anymore
Error: Uncaught TypeError: Cannot read property 'prop' of undefined
LOC - https://github.com/civicrm/civicrm-core/blob/master/js/crm.searchForm.js#L9
Reference - https://select2.org/upgrading/migrating-from-35#select2-val
2. Placeholder broken
Error: Uncaught TypeError: Cannot read property 'id' of undefined
As per 3.5 - placeholderOption is used to set the placeholder value for select2 widget
As per 4.0.4 - placeholder option can now accept placeholder value both in string and object
Reference - https://select2.org/upgrading/migrating-from-35#more-flexible-placeholders
3. EntityRef select2 fields are broken
Error: Uncaught Error: No select2/compat/initSelection
As per 4.0.4 - Removed the requirement of 'initSelection'
Reference - https://select2.org/upgrading/migrating-from-35#removed-the-requirement-of-initselection
4. Navigation menu style is not applied
Error: Uncaught TypeError: Cannot read property 'prop' of undefined
As per 4.0.4 - $("select").val("1").trigger("change"); // instead of $("select").select2("val", "1");
Reference - https://select2.org/upgrading/migrating-from-35#select2-val
5. Broken entityRef field
There are multiple issues with an entityRef field which are listed below:
1. Placeholder doesn't show
2. Unable to populate data via REST API link
3. Create contact dialog box doesn't render below the search field
4. Escape text doesn't show up
6. Action-menu select2 doesn't work on selecting record(s) in a search list
It throws an error `Uncaught TypeError: Cannot read property 'apply' of undefined` after Search and also action-menu field doesn't behave on selecting a record.
7. The native crm CSS styling doesn't apply on select2/4.0+ widget
Minor issues
1. Multi select2 widget doesn't show downarrow placeholder icon
2. select2 css style aren't applied after upgrade