CiviCRM Core issueshttps://lab.civicrm.org/dev/core/-/issues2023-04-20T06:55:21Zhttps://lab.civicrm.org/dev/core/-/issues/4245FormBuilder/SearchKit: add 'enabled' field2023-04-20T06:55:21Zaydunsaidan.saunders@squiffle.ukFormBuilder/SearchKit: add 'enabled' fieldOverview
----------------------------------------
Suggestion: add an 'enabled' field for both Searches and Forms (maybe Segments and related entities as well).
Why?: quite often I clone something, make some changes, switch to that - but...Overview
----------------------------------------
Suggestion: add an 'enabled' field for both Searches and Forms (maybe Segments and related entities as well).
Why?: quite often I clone something, make some changes, switch to that - but don't want to delete the original yet until I'm sure I don't want to revert to it. Marking it as 'not enabled' helps search & form management particularly as the numbers increase.https://lab.civicrm.org/dev/core/-/issues/3868Link contribution view with asssociated entity2023-05-18T23:37:57ZyashodhaLink contribution view with asssociated entityWhen you view a contribution, there's no way to know whether this contribution was done as a part of membership, pledge or an event registration.
I propose provide this additional data. Display this under total amount
e,g
![Screens...When you view a contribution, there's no way to know whether this contribution was done as a part of membership, pledge or an event registration.
I propose provide this additional data. Display this under total amount
e,g
![Screenshot_from_2022-09-23_19-15-29](/uploads/24028770f5b17e20ab40334973061b05/Screenshot_from_2022-09-23_19-15-29.png)
Membership will be a link to membership view page.
(depending on the related entity we display the link)yashodhayashodhahttps://lab.civicrm.org/dev/core/-/issues/464Disabling "Search Primary Details Only" breaks export of primary address2023-06-16T21:12:50ZmfbDisabling "Search Primary Details Only" breaks export of primary addressMigrated from https://issues.civicrm.org/jira/browse/CRM-21423
Disabling the "Search Primary Details Only"* setting switch leads to wrong export of contact primary address.
Reason is that a requested location_type is not set and Export...Migrated from https://issues.civicrm.org/jira/browse/CRM-21423
Disabling the "Search Primary Details Only"* setting switch leads to wrong export of contact primary address.
Reason is that a requested location_type is not set and Export.php's call to CRM_Contact_BAO_Query does not set the default search to use primary location.
A fix was proposed at https://github.com/civicrm/civicrm-core/pull/11274 but has been closed as more tests would need to be written.
\* My anecdotal experience is that, if the user has been merging contacts w/ multiple email addresses such that secondary email addresses are fairly common, then there's a frequent need to search across all email addresses, so this setting does need to be disabled.https://lab.civicrm.org/dev/core/-/issues/3311Proposal - store metadata on membership renewal on line item2023-06-18T00:43:47ZeileenProposal - store metadata on membership renewal on line item~~UPDATE - discussion on this is converging on adding a new json field 'metdata' to the line item table, permitting data related to that line item to be stored at point of order, and used when confirming. (UPDATE - this is not really the...~~UPDATE - discussion on this is converging on adding a new json field 'metdata' to the line item table, permitting data related to that line item to be stored at point of order, and used when confirming. (UPDATE - this is not really the convergence now - proposal updated below)~~
We have a few issues open ( https://lab.civicrm.org/dev/membership/-/issues/28 , https://lab.civicrm.org/dev/core/-/issues/1402 , https://lab.civicrm.org/dev/financial/-/issues/53 ) that are to my mind blocked by us not having a data model that holds the user's intent when renewing. The focus of this gl is the data model and exposing it via the back office renewal form & doing what is needed in the apis.
**The problem**
When renewing an expired membership the back office user has a fairly blunt way of influencing the calculated dates - they can set the 'renewal date'. However, if the payment is not made in the same form submission that intent is not captured anywhere and cannot be respected when completing the transaction. Likewise the receipt text and number of renewal terms are not captured.
**The proposal data model**
1) add membership_num_terms to civicrm_line_item table.
2) add 'payment_completion_metadata' to the ~~civicrm_line_item~~ civicrm_contribution table for data that is only used in the order->payment flow. Limit what can be stored here to tested values used in that flow - likely to be the message details and effective date the renewal form uses but doesn't sotre.
~~1. New field added to civicrm_line_item called 'metadata' which stores information required to complete the membership renewal - e.g. the field might store ``json_encode(['start_date' => '2010-01-01', 'end_date' => '2020-01-01', 'membership_status_id' => 'Current', 'membership_num_terms' => 2']~~
~~- note this is the new proposal - but I'm wondering what happens if it's renewed in the meantime & then paid~~
~~- New status 'Pending renewal' used when there is a need to record the dates (is_current_member = 0, is_reserved = 1, is_admin = 1 too I think)~~
~~- When a contribution linked to a membership in 'Pending renewal' is completed the status is updated but the dates are not changed~~
~~- If a membership has 'Pending renewal' status and status_override_end_date is set then any processing of that membership after that date would result in it being reverted to previous dates and status - this information should be available in the membership_log table.~~
**Proposed form exposure**
No proposed form changes - this is just storing data already submitted in the renewal form.
~~- the logic could be used outside of existing core forms but within core I think this is something we are exposing to the back-office user doing a renewal - they already have considerable ability to edit dates. So I think this is a form layer thing not some site-wide setting.
- when renewing an expired membership the renewal form would present the user with what will happen to the start date, end date etc if payment is received on that day and offer a checkbox to specify dates - if that is checked then date fields would be exposed an on save the Pending Renewal status would be used with the selected dates.
- potentially the user can also enter the date the 'offer' is available until - ie status_override_end_date~~
**Edge cases**
1. Multiple terms
We would add an extra field membership_num_terms to the line items column
~~https://lab.civicrm.org/dev/membership/-/issues/28 throws up the edge case that more than one renewal term might be selected. Both @jptillman and I assumed that the right way to represent that would be setting qty = n and then picking that when confirming. @andrewhunt didn't think that was the right assumption https://github.com/civicrm/civicrm-core/pull/18618. If we come down on NOT setting qty then we ALSO need a way to record intent for non-expired renewals - this could be an extra membership_num_terms column on the line_item table. (I'm still inclined to my original assumption but more points of view needed).~~
2. About to expire rather than actually expired
Out of scope for now - just do what the form currently does
~~I think we would still manage this in the form ie - "the membership is due to to expire in 4 days, if payment is received after that then .... ' and present the same options as for an actually expired membership~~https://lab.civicrm.org/dev/core/-/issues/224Event confirmation emails do not populate guest details when payment is confi...2023-06-19T04:25:50ZJKingsnorthEvent confirmation emails do not populate guest details when payment is confirmed by IPN (API)This issue occurs with:
* Paid events, using an IPN payment gateway, or delayed payment
* On events that can have 'additional participants' (guests)
In this case, the event confirmation emails are missing:
* The names of additional part...This issue occurs with:
* Paid events, using an IPN payment gateway, or delayed payment
* On events that can have 'additional participants' (guests)
In this case, the event confirmation emails are missing:
* The names of additional participants
* Any custom 'participant' fields associated with the additional participants
Detailed steps to recreate:
* Create custom fields assigned to the 'Participant' entity, add them to a profile
* Create a free event with online registration, allowing multiple participants, and add the profile with custom fields to it, enable confirmation emails
* Register for the event, with an 'additional participant'
* Notice that the confirmation email contains the 'name' of the additional participant, and also the custom fields associated with their registration
* Create a paid event, with the 'pay later' option (otherwise the same options as above)
* Register for the event, with an 'additional participant' (pay later)
* Now use the Contribution - completetransaction API call to 'complete' the payment (this is the process that would happen if the IPN was triggered).
* Notice that the confirmation email does not contain the name of the participants, and any custom participant fields for the additional participants are NOT included.
Issue cause:
Information from the 'form' submission is passed into the email template when it is triggered for a free event. This information is not populated when the \CRM_Event_BAO_Event::sendMail method is called from the 'completetransaction' API call. Some information is assigned in \CRM_Event_Form_Registration_Confirm::buildQuickForm and includes the $part variable, which is assigned to the template, and subsequently used in the 'online event registration' email template. The custom fields for additional participants are also assigned to the template, based on the form (https://github.com/civicrm/civicrm-core/blob/694ccbf41adc31bce4d056fcc46930ba9c2e15e5/CRM/Event/Form/Registration/Confirm.php#L311), which is also missing from the 'completetransaction' route.
Our solution:
The code around this area of the system was quite spahettified, so we have solved this issue using a custom hook: https://gist.github.com/JKingsnorth/4d233df13a0f6493c44f69d81fa5f27a - although this isn't a great approach (in terms of being able to commit it back to core) - it does seem to resolve the issue.https://lab.civicrm.org/dev/core/-/issues/4250Expose the mailing/mosaico template when viewing mailing report2023-06-19T23:42:32ZyashodhaExpose the mailing/mosaico template when viewing mailing reportCurrently, we don't show which mosaico template was used for a mailing in mailing reports.
It could be helpful as sometimes the template could be deleted and when the same mailing is re-used then mosaico is broken in the copied mailing....Currently, we don't show which mosaico template was used for a mailing in mailing reports.
It could be helpful as sometimes the template could be deleted and when the same mailing is re-used then mosaico is broken in the copied mailing. We could show name/id and deleted if not present.https://lab.civicrm.org/dev/core/-/issues/5CiviMail Mailing Accounts allow duplicate names, but breaks processing.2023-06-23T17:54:21ZjohnffCiviMail Mailing Accounts allow duplicate names, but breaks processing.If two CiviMail Mailing Accounts have the same name, even if they have different roles, only the first will be used in either case.
The offending line of code is somewhere in the path of: CRM_Utils_Mail_EmailProcessor::_process -> CRM_M...If two CiviMail Mailing Accounts have the same name, even if they have different roles, only the first will be used in either case.
The offending line of code is somewhere in the path of: CRM_Utils_Mail_EmailProcessor::_process -> CRM_Mailing_MailStore::getStore.
To replicate (Drupal, 4.7.27):
1. Create two mailing accounts.
The first one should be "Email to Activity" and have no username / password settings
The second should be "Bounce processing" and have username / password settings
(order of creation is important, and give them both the same name)
2. Run the scheduled job "fetch bounces".
3. In watchdog, observe the error "could not connect" using the credentials of email to activity processor, even though it should use the Bounce Processing one.
4. Clear the watchdog logs.
5. Rename them to have different names, then retry
6. Check the watchdogs, and observe no error.https://lab.civicrm.org/dev/core/-/issues/24Passing an array for contact_id/client_id to Case.Create API when updating an...2023-06-23T17:54:22Zmattwiremjw@mjwconsult.co.ukPassing an array for contact_id/client_id to Case.Create API when updating an existing case causes case to be "reassigned"The contact_id/client_id for the case supports multiple ids. It can be passed to the case.create API either as a single value or as an array. However, the code is inconsistent in it's handling of contact_id and expects it to be both an...The contact_id/client_id for the case supports multiple ids. It can be passed to the case.create API either as a single value or as an array. However, the code is inconsistent in it's handling of contact_id and expects it to be both an array and a single value. The result is that if an array is passed in it will fail to detect that an existing case contact_id already matches and it will "reassign" the case to the default contact id (1). This issue was identified because of inconsistent behaviour when submitting case/activity updates via webform_civicrm.
The proposed solution is to ensure that contact_id is always an array, and then check the first element against the original contact id.https://lab.civicrm.org/dev/core/-/issues/4328Double opt-in requires traditional bounce handling to be enabled2023-06-28T06:55:29ZJonGoldDouble opt-in requires traditional bounce handling to be enabledOverview
----------------------------------------
The default double opt-in email says:
> You have a pending subscription to the {subscribe.group} mailing list. To confirm this subscription, reply to this email or click <a href="{subscri...Overview
----------------------------------------
The default double opt-in email says:
> You have a pending subscription to the {subscribe.group} mailing list. To confirm this subscription, reply to this email or click <a href="{subscribe.url}">here</a>.
However, "reply to this email" requires traditional bounce handling (with a bounce mailbox) to be set up. In this day and age that's less common compared to using a third party mailer that sends bounce notifications via API (to [Airmail](https://civicrm.org/extensions/airmail) or one of its many cousins).
Moreover, it's a usability issue to put "here" as the clickable text. It should be more like: "Please click here to <a href="{subscribe.url}">confirm your subscription.</a>.
I want to get concept approval - and also guidance on whether we should update existing templates or just new installs. Personally I think we should update any non-customized template.https://lab.civicrm.org/dev/core/-/issues/4361Add Pay now link to Invoice template2023-06-28T06:58:22ZlarsssandergreenAdd Pay now link to Invoice templateIt would be nice to have the link to pay online included in the Invoice template. Currently, a user would have to guess this is possible, find the correct link on SE and add it to the template (then maintain it as the template is updated...It would be nice to have the link to pay online included in the Invoice template. Currently, a user would have to guess this is possible, find the correct link on SE and add it to the template (then maintain it as the template is updated).
I think if default_invoice_page is set, we could simply include a Pay Now link in the Payment Advice section (which is only shown if the Contribution is pending and pay later). If an org doesn't want to include a link for online payment, they can simply leave default_invoice_page empty (Edit: They cannot currently do this for newer installs, will have to make it possible). I think it would be very rare for an org to want to have a default_invoice_page for payments from the User Dashboard, but not to include a link for online payments in invoices (and in that case, they can edit the template).
Will do this and add docs if supported.https://lab.civicrm.org/dev/core/-/issues/4066Can the `CRM_Event_Badge` classes be deprecated and removed?2023-07-03T14:20:54ZBradley TaylorCan the `CRM_Event_Badge` classes be deprecated and removed?I recently came across these classes:
- `CRM_Event_Badge`
- `CRM_Event_Badge_Logo`
- `CRM_Event_Badge_Logo5395`
- `CRM_Event_Badge_NameTent`
- `CRM_Event_Badge_Simple`
Stylistically they're not great, and I noticed they don't seem to b...I recently came across these classes:
- `CRM_Event_Badge`
- `CRM_Event_Badge_Logo`
- `CRM_Event_Badge_Logo5395`
- `CRM_Event_Badge_NameTent`
- `CRM_Event_Badge_Simple`
Stylistically they're not great, and I noticed they don't seem to be in use.
The last 4 of these classes extend `CRM_Event_Badge`, and are referenced in the `eventBadge` option group. However, I can't see any scenario where this `eventBadge` option group is actually used.
The `Event Name Badge Layouts` configuration screen gets stored in the `civicrm_print_label` table, and `CRM_Badge_BAO_Badge` is what actually get's used when you print a selection of badges for an event. `CRM_Badge_BAO_Badge` does not use `CRM_Event_Badge` (although they are very similar. I suspect one started as a copy of the other.)
I think it would make sense to:
1. Check nobody is using the 5 classes listed above through a universe search.
2. Noisily deprecated the 5 classes, specifying `CRM_Badge_BAO_Badge` should be used instead.
3. After a period of time remove the 5 classes.
4. Tidy up (remove all trace of) the `event_badge` option group.
Of these, the last step is the one I'm most unsure of the process for. I guess it'd be removed through an upgrade step.https://lab.civicrm.org/dev/core/-/issues/4401SearchKit - Report replacement - as a standard user without edit searchkit pe...2023-07-06T06:37:04ZsamuelsovSearchKit - Report replacement - as a standard user without edit searchkit permission, add a way to show/hide columns in table displayThere is already an [extension](https://lab.civicrm.org/extensions/search_kit_reports) that recreate the queries of core reports into SearchKit so admin/super user can customize and use them instead of report.
However, as a standard use...There is already an [extension](https://lab.civicrm.org/extensions/search_kit_reports) that recreate the queries of core reports into SearchKit so admin/super user can customize and use them instead of report.
However, as a standard user, edit a Searchkit is too complicated and reports is still the better option if we need to provide a lot of flexibility to users.
Currently, in Reports, standard user have access to customize :
- Columns - no way to do this in SK without editing the SK - hence the proposal
- Sorting - already there by clicking on the columns of the SK table
- Filters - already there by configuring them with FormBuilder
- Title and Format - don't think it is required
- Email delivery - missing, already documented in https://lab.civicrm.org/dev/core/-/issues/3478
So I think the main undocumented missing feature if we want to replace the reports is to have a way for a standard user to choose which columns to show/hide in a table display. It needs to apply to the screen and to the Download action.
As an example, I like the way new reports in Quickbooks works with the ability to switch each column on/off and as a bonus allow to change columns order :
![Rapport](/uploads/550ef9e7d7ffd1f19849b4034faa7765/Rapport.png)
As a first step, I don't think we need to save the user preferences but we will probably want to be able to do that / reset to default eventually.https://lab.civicrm.org/dev/core/-/issues/2484Create apiv4 payment api2023-07-13T08:07:37ZeileenCreate apiv4 payment apiIn order to be able to search for payments we need to be able to search for payment rows (rows with is_payment = TRUE) in the financial_trxn table and for the entity bridge to be able to find it's way from there to the contribution table...In order to be able to search for payments we need to be able to search for payment rows (rows with is_payment = TRUE) in the financial_trxn table and for the entity bridge to be able to find it's way from there to the contribution table.
This gives us the ability to say find all contributions with a payment with a payment instrument of 'Check' or find the contribution where the payment's trxn value is y
The scope I would be looking for is
1) can use any of the fields in financial_trxn as a filter
2) can logically link to the contribution (probably through bridge entities)
3) forces 'is_payment' as a filter (only gets payments)
4) the create action calls Payment.create
5) the update & delete actions are blocked
This effectively gives us parity with the v3 api albeit with more options provided through the apiv4 layer itself.
Note that I think this is likely blocked on other feature requests which @mattwire will add notes onhttps://lab.civicrm.org/dev/core/-/issues/3961Proposal: Create stub extensions for civicrm core + all components2023-07-13T10:41:05ZcolemanwProposal: Create stub extensions for civicrm core + all components*Edited to reflect the current state of things, some comments reply to an earlier revision*
**Motivation 1:**
Require SearchKit in core. This has been [solved in a different way](https://github.com/civicrm/civicrm-core/pull/24739) so i...*Edited to reflect the current state of things, some comments reply to an earlier revision*
**Motivation 1:**
Require SearchKit in core. This has been [solved in a different way](https://github.com/civicrm/civicrm-core/pull/24739) so is no longer relevant to this issue.
**Motivation 2**
We are starting to package SearchKit displays as replacements for Smarty/DataTables in the UI. This has worked out great for CiviGrant: now that it's an extension we can easily package Afforms, SavedSearches and other managed entities with it. But other components are not extensions and so have no mechanism for including their own managed entities, afforms, etc. Components are less modular and more monolithic than extensions.
**Motivation 3**
Currently an extension has no way to require a component, e.g. an extension cannot declare a dependency on CiviEvent.
**Motivation 4**
Extensions and Components are similar but not the same, and this is confusing to new users and new developers. In most respects Extensions are better than Components, and it would be great to "some day" have all components simply be extensions.
**Proposal**
Given the experience of converting CiviGrant to an extension was a "harder than expected" undertaking, I think converting all other components to extensions at once is not a realistic goal. It would need to be a slow, iterative process, and I propose this as the first iteration:
1. Create a small stub extension for each component (CiviEvent, CiviContribute, etc).
2. Use hooks to sync between the `civicrm_extension` table and the `civicrm_component` table, so that enabling the "CiviEvent" extension also enables the "CiviEvent" component, and vice-versa.
3. Get rid of the "Manage Components" screen. Admins can enable/disable components via the "Manage Extensions" screen.https://lab.civicrm.org/dev/core/-/issues/2565HTML editor for additional event confirmation email text2023-08-02T21:16:19ZlarsssandergreenHTML editor for additional event confirmation email textIt would be nice to be able to have an HTML editor for the additional event confirmation text entered on the Manage Event Page that gets inserted into the System Template email template. The benefit would be to be able to easily add link...It would be nice to be able to have an HTML editor for the additional event confirmation text entered on the Manage Event Page that gets inserted into the System Template email template. The benefit would be to be able to easily add links, etc. I understand this will be a complicated one, but it does seem worthwhile as almost every other similar field is HTML, so this one really stands out. [See previous issue.](https://lab.civicrm.org/dev/core/-/issues/852#note_58464)
How it works now: Text from the event confirmation email text is added into the event confirmation message template with some minimal tags added. I see a <p> wrapper around the text and <br /> inserted for newlines, at a cursory glance.
Here's my guess at what needs to happen:
1. Add HTML editor to Event Configuration
2. Add HTML editor on the New Event Registration page (which is also Add Participant from events or Add Event Registration for a contact), so that the message can be edited before sending.
3. Inject content as HTML into Message Templates, i.e. remove code that adds tags, wrap code in div or something.
4. Convert all existing content in the DB with the same process now taken to inject the text into the Message Template or handle the content injection into the Message Template with two cases based on text/HTML.
The first three should be relatively simple, but the last one there seems like the tricky part. How has this has been handled in the past?https://lab.civicrm.org/dev/core/-/issues/4470Allow disabling household contact type2023-08-04T03:59:56ZlarsssandergreenAllow disabling household contact typeCurrent proposal:
Make it possible to disable household contact type.
Initial: These cannot be disabled. It would be nice to be able to disable them, especially households as many orgs do not use them. I know Spark has households disab...Current proposal:
Make it possible to disable household contact type.
Initial: These cannot be disabled. It would be nice to be able to disable them, especially households as many orgs do not use them. I know Spark has households disabled, so at least disabling households shouldn't break anything. Organizations might be different though and less important to allow disabling. Maybe worth trying to allow disabling households first and then see.https://lab.civicrm.org/dev/core/-/issues/852Provide wysiwyg editor for additional confirmation email text2023-08-05T04:15:04ZyashodhaProvide wysiwyg editor for additional confirmation email textProvide wysiwyg editor for additional confirmation email text for online and offline event registrations. (In other words, this is the text from the text area widget on the Manage Event page, not the event template itself.)
https://civi...Provide wysiwyg editor for additional confirmation email text for online and offline event registrations. (In other words, this is the text from the text area widget on the Manage Event page, not the event template itself.)
https://civicrm.stackexchange.com/questions/21255/confirmation-email-providing-a-wysiwyg-so-users-can-add-html-ified-contentyashodhayashodhahttps://lab.civicrm.org/dev/core/-/issues/2597Tabset for manage contribution pages or manage events requires Classname to e...2023-08-07T15:13:57ZlarsssandergreenTabset for manage contribution pages or manage events requires Classname to equal pathIf you add a tab to a tabset for Configure Event, it will work fine with the last word of the class name different from the last word of the path, except for trying to save, which will save, but redirect to Manage Events instead of stayi...If you add a tab to a tabset for Configure Event, it will work fine with the last word of the class name different from the last word of the path, except for trying to save, which will save, but redirect to Manage Events instead of staying on the same tab. This is because [code here](https://github.com/civicrm/civicrm-core/blob/5db0bc3c1f54eaca4307f103a73bda596ae914d6/CRM/Event/Form/ManageEvent.php#L377) expects lastWord from CRM_myExtension_Form_lastWord to be the same as the path, i.e. /civicrm/event/manage/lastword. Everything else works as expected if these are different, except Save.
Looks like this same issue exists for [Contribution Pages](https://github.com/civicrm/civicrm-core/blob/5db0bc3c1f54eaca4307f103a73bda596ae914d6/CRM/Contribute/Form/ContributionPage.php#L427
) and probably elsewhere.
For now, I've added a warning to the docs for [the tabset hook](https://lab.civicrm.org/documentation/docs/dev/-/merge_requests/908). It would be good to fix this so the code linked above uses the path for the url instead of the class, at some point.https://lab.civicrm.org/dev/core/-/issues/2562Simplify when scheduled reminders are sent or not sent for events2023-08-07T15:14:25ZlarsssandergreenSimplify when scheduled reminders are sent or not sent for eventsFor event scheduled reminders, you can set emails to be sent on specific days or N hours or days before or after an event (or before or after registration ends or starts). For emails sent on a specific day, this is straightforward. They ...For event scheduled reminders, you can set emails to be sent on specific days or N hours or days before or after an event (or before or after registration ends or starts). For emails sent on a specific day, this is straightforward. They are sent on that day and anyone who registers after that day does not receive the email.
But for emails scheduled N hours or days before or after an event, the behaviour is less clear. My testing indicates contacts registered soon after an event has ended can still receive emails scheduled to go out 24 hours before the event start or 30 hours before the event end (but contacts registered somewhat later won't receive these emails). Similarly, contacts registered a day after an event ends will receive an email scheduled to go out an hour after the event start. I can't find any documentation that tells users what will happen when they schedule these reminders, though there are a few questions on Stack Exchange wondering about what actually does happen (opinions vary, which is a bad sign).
I think the clearest and best option would be to simply send scheduled reminders at the time they are scheduled and not afterwards. For one, this is simple and probably what users assume happens. It's certainly what our users think. Secondly, let's say you have an event with a confirmation email, one scheduled reminder a day before the event and one 2 hours before the event: with the current system a person who registers at the last minute will receive three emails in short order, which doesn't make sense. Thirdly, if a user registers someone in the backend for an event after the event has ended, just to record their attendance, they probably don't want that person to automatically receive emails previously scheduled to go out right after the event start or similar (this could be avoided by using a different status for the post-event registration and the scheduled reminder, but that's a pretty fine point that users won't think about).
In short, I think there are so many potential pitfalls with a system that sends scheduled reminders after the time set on the reminder has passed, especially when none of this behaviour is apparent to users when they set up the reminders. To me, this outweighs the potential convenience of having reminders sent to late registrants, so we should simply not send scheduled reminders whose time has passed.
I haven't dug into the code yet to see how this all works, but I'll do that and submit a PR as long as this is supported.
EDIT: removed aside on timing of scheduled reminders.https://lab.civicrm.org/dev/core/-/issues/2595Change Log Tab Excrutiatingly Slow - Poorly Performing Query and Fix (from 8 ...2023-08-14T13:09:44ZgordanChange Log Tab Excrutiatingly Slow - Poorly Performing Query and Fix (from 8 minutes down to 4 seconds)The page takes about 8 minutes to return which is absurdly slow for anything expected to be remotely interactive. It is so slow that my client is referring to it as the "Triangle of Doom".
I tracked it down to this query:
```
INSERT IG...The page takes about 8 minutes to return which is absurdly slow for anything expected to be remotely interactive. It is so slow that my client is referring to it as the "Triangle of Doom".
I tracked it down to this query:
```
INSERT IGNORE INTO civicrm_tmp_e_logsummary_ffa3ac146126d178ade367f2a5d17bf5
SELECT activity_id, IF (entity_log_civireport.log_action = 'Insert' AND extra_table.activity_type_id = 51 , GROUP_CONCAT(entity_log_civireport.contact_id), 1) , entity_log_civireport.log_action as log_civicrm_entity_log_action, 'log_civicrm_activity_contact' as log_civicrm_entity_log_type, entity_log_civireport.log_user_id as log_civicrm_entity_log_user_id, entity_log_civireport.log_date as log_civicrm_entity_log_date, modified_contact_civireport.display_name as log_civicrm_entity_altered_contact, modified_contact_civireport.id as log_civicrm_entity_altered_contact_id, entity_log_civireport.log_conn_id as log_civicrm_entity_log_conn_id, modified_contact_civireport.is_deleted as log_civicrm_entity_is_deleted, altered_by_contact_civireport.display_name as altered_by_contact_display_name
FROM staging_civicrm.log_civicrm_activity_contact entity_log_civireport
JOIN civicrm_contact modified_contact_civireport ON (entity_log_civireport.contact_id = modified_contact_civireport.id )
JOIN staging_civicrm.log_civicrm_activity extra_table ON extra_table.id = entity_log_civireport.activity_id
LEFT JOIN civicrm_contact altered_by_contact_civireport ON (entity_log_civireport.log_user_id = altered_by_contact_civireport.id)
WHERE modified_contact_civireport.id = 338520 AND
entity_log_civireport.log_action != 'Initialization'
GROUP BY entity_log_civireport.log_conn_id,
entity_log_civireport.log_user_id,
EXTRACT(DAY_MICROSECOND FROM entity_log_civireport.log_date),
entity_log_civireport.id
ORDER BY entity_log_civireport.log_date DESC;
EXPLAIN shows:
```
```
+------+-------------+-------------------------------+--------+---------------+---------+---------+---------------------------------------------------+---------+-------------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+------+-------------+-------------------------------+--------+---------------+---------+---------+---------------------------------------------------+---------+-------------------------------------------------+
| 1 | SIMPLE | modified_contact_civireport | const | PRIMARY | PRIMARY | 4 | const | 1 | Using temporary; Using filesort |
| 1 | SIMPLE | extra_table | ALL | NULL | NULL | NULL | NULL | 3014020 | |
| 1 | SIMPLE | entity_log_civireport | ALL | NULL | NULL | NULL | NULL | 5518537 | Using where; Using join buffer (flat, BNL join) |
| 1 | SIMPLE | altered_by_contact_civireport | eq_ref | PRIMARY | PRIMARY | 4 | staging_civicrm.entity_log_civireport.log_user_id | 1 | Using where |
+------+-------------+-------------------------------+--------+---------------+---------+---------+---------------------------------------------------+---------+-------------------------------------------------+
```
The fix passes:
First pass:
```
ALTER TABLE log_civicrm_activity_contact ADD INDEX index_activity_id (activity_id);
```
With no further changes, this alone makes the above query go from 8 minutes to 1m45s.
Explain plain:
```
+------+-------------+-------------------------------+--------+-------------------+-------------------+---------+---------------------------------------------------+---------+---------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+------+-------------+-------------------------------+--------+-------------------+-------------------+---------+---------------------------------------------------+---------+---------------------------------+
| 1 | SIMPLE | modified_contact_civireport | const | PRIMARY | PRIMARY | 4 | const | 1 | Using temporary; Using filesort |
| 1 | SIMPLE | extra_table | ALL | NULL | NULL | NULL | NULL | 3014020 | Using where |
| 1 | SIMPLE | entity_log_civireport | ref | index_activity_id | index_activity_id | 5 | staging_civicrm.extra_table.id | 1 | Using where |
| 1 | SIMPLE | altered_by_contact_civireport | eq_ref | PRIMARY | PRIMARY | 4 | staging_civicrm.entity_log_civireport.log_user_id | 1 | Using where |
+------+-------------+-------------------------------+--------+-------------------+-------------------+---------+---------------------------------------------------+---------+---------------------------------+
```
Second pass:
```
ALTER TABLE log_civicrm_activity ADD INDEX index_id (id);
```
Change the JOIN order explicitly and add a hint for the query optimizer to not re-order the JOINs:
```
SELECT STRAIGHT_JOIN activity_id, IF (entity_log_civireport.log_action = 'Insert' AND extra_table.activity_type_id = 51 , GROUP_CONCAT(entity_log_civireport.contact_id), 1) , entity_log_civireport.log_action as log_civicrm_entity_log_action, 'log_civicrm_activity_contact' as log_civicrm_entity_log_type, entity_log_civireport.log_user_id as log_civicrm_entity_log_user_id, entity_log_civireport.log_date as log_civicrm_entity_log_date, modified_contact_civireport.display_name as log_civicrm_entity_altered_contact, modified_contact_civireport.id as log_civicrm_entity_altered_contact_id, entity_log_civireport.log_conn_id as log_civicrm_entity_log_conn_id, modified_contact_civireport.is_deleted as log_civicrm_entity_is_deleted, altered_by_contact_civireport.display_name as altered_by_contact_display_name
FROM civicrm_contact modified_contact_civireport
JOIN staging_civicrm.log_civicrm_activity_contact entity_log_civireport ON entity_log_civireport.contact_id = modified_contact_civireport.id
JOIN staging_civicrm.log_civicrm_activity extra_table ON extra_table.id = entity_log_civireport.activity_id
LEFT JOIN civicrm_contact altered_by_contact_civireport ON entity_log_civireport.log_user_id = altered_by_contact_civireport.id
WHERE modified_contact_civireport.id = 338520 AND
entity_log_civireport.log_action != 'Initialization'
GROUP BY entity_log_civireport.log_conn_id,
entity_log_civireport.log_user_id,
EXTRACT(DAY_MICROSECOND FROM entity_log_civireport.log_date),
entity_log_civireport.id
ORDER BY entity_log_civireport.log_date DESC;
```
New EXPLAIN:
```
+------+-------------+-------------------------------+--------+-------------------+----------+---------+---------------------------------------------------+---------+---------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+------+-------------+-------------------------------+--------+-------------------+----------+---------+---------------------------------------------------+---------+---------------------------------+
| 1 | SIMPLE | modified_contact_civireport | const | PRIMARY | PRIMARY | 4 | const | 1 | Using temporary; Using filesort |
| 1 | SIMPLE | entity_log_civireport | ALL | index_activity_id | NULL | NULL | NULL | 5518537 | Using where; Using filesort |
| 1 | SIMPLE | extra_table | ref | index_id | index_id | 5 | staging_civicrm.entity_log_civireport.activity_id | 1 | |
| 1 | SIMPLE | altered_by_contact_civireport | eq_ref | PRIMARY | PRIMARY | 4 | staging_civicrm.entity_log_civireport.log_user_id | 1 | Using where |
+------+-------------+-------------------------------+--------+-------------------+----------+---------+---------------------------------------------------+---------+---------------------------------+
```
This gets it down to 4 seconds!
Since the first index we started with is no longer getting used in the final variant, we can just not add it.
Summary:
To fix "Change Log" tab taking forever to load, the following fix is needed:
Add index:
```
ALTER TABLE log_civicrm_activity ADD INDEX index_id (id);
```
Make the code emit the query modified as follows:
```
INSERT IGNORE INTO civicrm_tmp_e_logsummary_ffa3ac146126d178ade367f2a5d17bf5
SELECT STRAIGHT_JOIN activity_id, IF (entity_log_civireport.log_action = 'Insert' AND extra_table.activity_type_id = 51 , GROUP_CONCAT(entity_log_civireport.contact_id), 1) , entity_log_civireport.log_action as log_civicrm_entity_log_action, 'log_civicrm_activity_contact' as log_civicrm_entity_log_type, entity_log_civireport.log_user_id as log_civicrm_entity_log_user_id, entity_log_civireport.log_date as log_civicrm_entity_log_date, modified_contact_civireport.display_name as log_civicrm_entity_altered_contact, modified_contact_civireport.id as log_civicrm_entity_altered_contact_id, entity_log_civireport.log_conn_id as log_civicrm_entity_log_conn_id, modified_contact_civireport.is_deleted as log_civicrm_entity_is_deleted, altered_by_contact_civireport.display_name as altered_by_contact_display_name
FROM civicrm_contact modified_contact_civireport
JOIN staging_civicrm.log_civicrm_activity_contact entity_log_civireport ON entity_log_civireport.contact_id = modified_contact_civireport.id
JOIN staging_civicrm.log_civicrm_activity extra_table ON extra_table.id = entity_log_civireport.activity_id
LEFT JOIN civicrm_contact altered_by_contact_civireport ON entity_log_civireport.log_user_id = altered_by_contact_civireport.id
WHERE modified_contact_civireport.id = 338520 AND
entity_log_civireport.log_action != 'Initialization'
GROUP BY entity_log_civireport.log_conn_id,
entity_log_civireport.log_user_id,
EXTRACT(DAY_MICROSECOND FROM entity_log_civireport.log_date),
entity_log_civireport.id
ORDER BY entity_log_civireport.log_date DESC;
```
Not only does this speed it up from 8 minutes down to 4 seconds, it also doesn't wreak havoc with row locking where every row scanned gets locked by the transaction engine, potentially resulting in a massive query pile-up in the database.