Development issueshttps://lab.civicrm.org/groups/dev/-/issues2023-07-20T09:34:15Zhttps://lab.civicrm.org/dev/core/-/issues/4177Auto-generation of recurring contribution template ignores recurring amount2023-07-20T09:34:15ZAlanDixonAuto-generation of recurring contribution template ignores recurring amountOverview
----------------------------------------
When a recurring contribution does not have a corresponding template, sometimes one is generated implicitly (e.g. when viewing the 'template' and clicking 'done').
That 'implicit generat...Overview
----------------------------------------
When a recurring contribution does not have a corresponding template, sometimes one is generated implicitly (e.g. when viewing the 'template' and clicking 'done').
That 'implicit generation' does not pay attention to the amount in the recurring contribution record.
Reproduction steps
----------------------------------------
1. As an administrator, create a new recurring contribution for $1.
1. Observe that only the recurring contribution and contribution record are created, no recurring template contribution (requires mysql access to see this!).
1. Edit the recurring amount to $2. Note that there is still no recurring template.
1. View the recurring 'template' and click done. This implicitly creates the recurring record based on the $1 contribution, ignoring the $2 recurring contribution record amount, and as a side effect, modifies the recurring contribution record amount back to $1.
Current behaviour
----------------------------------------
The implicit generation of the recurring template does not respect the amount in the recurring record total.
Expected behaviour
----------------------------------------
It should! Of course, it's complicated. e.g. maybe that recurring template should get generated when the original recurring contribution is created?
I would note that I've only described one way that the template gets created, it appears there may be others. So regardless of whether we avoid this specific way of implicitly-generating the template, it should behave better than it does ...
Environment information
----------------------------------------
CiviCRM version: likely all versions since last June 2022.
Relevant docs link https://docs.civicrm.org/dev/en/latest/financial/recurring-contributions/https://lab.civicrm.org/dev/core/-/issues/4415FormBuilder: Add 'is empty' filter2023-07-20T22:00:28Zaydunsaidan.saunders@squiffle.ukFormBuilder: Add 'is empty' filterOverview
----------------------------------------
In FormBuilder, add `Is Empty` / `Is Not Empty` for filter fields.
Booleans can be represented as Yes/No radios. This would let other fields be represented in the same way based on whet...Overview
----------------------------------------
In FormBuilder, add `Is Empty` / `Is Not Empty` for filter fields.
Booleans can be represented as Yes/No radios. This would let other fields be represented in the same way based on whether they are empty or not.
Example use-case
----------------------------------------
1. On the current Contribution Search form there is a 'Contribution is Recurring?' Yes/No which is based on whether the `recur_id` is empty or not.
Current behaviour
----------------------------------------
FormBuilder can add a filter for a specific value of recur_id but not 'is empty'/'is not empty'
Proposed behaviour
----------------------------------------
Allow 'is empty'/'is not empty' to be used with any filter field and be configured as booleans.
Comments
----------------------------------------
General 'is empty'/'is not empty' is available in Apiv4 'where' clauses and SearchKit, so this is about extending to FormBuilder.
See https://chat.civicrm.org/civicrm/pl/js843u3qz3f6pgc8ay183bhuzocolemanwcolemanwhttps://lab.civicrm.org/dev/core/-/issues/4440SearchUI: Find Contributions issues2023-07-21T09:54:27Zaydunsaidan.saunders@squiffle.ukSearchUI: Find Contributions issuesIssues that need addressing for the SearchUI Find Contributions page to match core:
- [ ] Figure out how to handle payments:
- In current version, clicking on amount expands to show payment details and buttons
- [ ] Figure out ho...Issues that need addressing for the SearchUI Find Contributions page to match core:
- [ ] Figure out how to handle payments:
- In current version, clicking on amount expands to show payment details and buttons
- [ ] Figure out how to handle Soft Credits
- Current version has several filter options
- [ ] Figure out how to handle Recurring Contributions
- Current version also searches for these. Maybe better as a separate search?
- [ ] Figure out how to handle PCPs
- (Does anyone use PCPs? If so, what is useful in this search?) Not showing correctly.
- [x] Add contribution totals
- [ ] Some actions missing
- [x] Filters - convert some to Yes/No ([#4415](https://lab.civicrm.org/dev/core/-/issues/4415))
- done by adding 'is null' fields to search. Would be nice to rework if/when 4415 is implemented.
- but this doesn't interact properly with 'Group by' used in the Summary section
- [ ] Filter layout
- Current labels and groupings are for dev purposes, need improvement
- [ ] Add more here ...https://lab.civicrm.org/dev/core/-/issues/4438Test Contributions cannot be selected by id in SearchKit2023-07-21T09:56:24ZbrienneTest Contributions cannot be selected by id in SearchKitOverview
----------------------------------------
In SearchKit, a test Contribution cannot be selected by id, such as in the `WHERE` clause, as the test does not appear as an option in the autocomplete drop down list. However, this is no...Overview
----------------------------------------
In SearchKit, a test Contribution cannot be selected by id, such as in the `WHERE` clause, as the test does not appear as an option in the autocomplete drop down list. However, this is not a problem in APIv4, where you can type in the id directly into the available text box.
Reproduction steps
----------------------------------------
1. Find or create a test Contribution
* the quickest way to create one would be by using APIv4 with `is_test = Yes` in the `WHERE` clause.
1. Create a SearchKit with Contributions as the primary entity.
1. In the `WHERE` section, select `Contribution ID =` and then type in the id of the test contribution
1. The drop down list will display 'None found'.
Current behaviour
----------------------------------------
A test Contribution cannot be selected by its id from SearchKit, however, it is available in APIv4.
![Selection_160](/uploads/2547cfd47d94f8e48dce669a391c44a8/Selection_160.png)
![Selection_161](/uploads/88a8cf0da71c4114ab4ca47d1f38e6e7/Selection_161.png)
Expected behaviour
----------------------------------------
A test Contribution should be select-able by its id from SearchKit.
Environment information
----------------------------------------
* __CiviCRM:__ 5.63.alpha1https://lab.civicrm.org/dev/core/-/issues/4434Don't allow uploading attachments to bulk mailings if they exceed the size limit2023-07-21T09:57:50ZDaveDDon't allow uploading attachments to bulk mailings if they exceed the size limitThere's been a couple times on multiple sites where the mailing bounces and it's not clear to the average office staffer why, but it's because they've uploaded a 30MB pdf to the mailing.
It should just not allow doing that in the first ...There's been a couple times on multiple sites where the mailing bounces and it's not clear to the average office staffer why, but it's because they've uploaded a 30MB pdf to the mailing.
It should just not allow doing that in the first place if the config settings have a smaller limit.https://lab.civicrm.org/dev/core/-/issues/4432Permission system can be bypassed from the search results action menu2023-07-21T09:59:05ZschorschiiPermission system can be bypassed from the search results action menuOverview
----------------------------------------
I created a simple permission structure where a group of CiviCRM users ("Group A") has write access to a group of contacts ("Group B"). (Every logged in user can read all contacts in our ...Overview
----------------------------------------
I created a simple permission structure where a group of CiviCRM users ("Group A") has write access to a group of contacts ("Group B"). (Every logged in user can read all contacts in our system.)
When a contact is now added to "Group B", users of "Group A" see the edit button on the contact and can add/remove the contact to/from groups on the contact detail page. When removing the contact from "Group B", the edit button disappears. So far, everything as expected.
But when using the actions menu from the search results, users can add/remove group assignments of a contact which is not in "Group B".
Isn't this an inconsistency in the permission system? Or am I missing something? How to avoid group membership changes of contacts which are not in "Group B" by users which are in "Group A"?
Reproduction steps
----------------------------------------
1. Create a role, assign it to "Group A" and create ACL "edit" for "Group B".
2. Log in with a non-admin user which is member of "Group A".
3. Search a contact which is not member of "Group B". Select it in the search results and choose "Group - add contact" or "Group - remove contact".
Current behaviour
----------------------------------------
I can change the group membership of this contact from within the action menu of the search result list.
Expected behaviour
----------------------------------------
Group membership changes should be refused since the contact is not member of "Group B".
When opening the contact's detail view, it works as expected, which means I'm not able to change the group memberships there.
Environment information
----------------------------------------
CiviCRM version 5.63.1 under WordPress
[Corresponding question on StackExchange](https://civicrm.stackexchange.com/questions/45266/permission-system-not-working-as-expected)https://lab.civicrm.org/dev/core/-/issues/4431API4: "!=" has unexpected results on NULL fields2023-07-21T09:59:23ZJonGoldAPI4: "!=" has unexpected results on NULL fieldsOverview
----------------------------------------
Using the `!=` operator when some of the values are `NULL` does not return the `NULL` records.
Reproduction steps
----------------------------------------
1. On a demo site (which by def...Overview
----------------------------------------
Using the `!=` operator when some of the values are `NULL` does not return the `NULL` records.
Reproduction steps
----------------------------------------
1. On a demo site (which by default has no contacts with a subtype), search for contacts that are not of subtype "Parent".
E.g.:
```php
\Civi\Api4\Contact::get(TRUE)
->addWhere('contact_sub_type', '!=', 'Parent')
->execute();
```
Current behaviour
----------------------------------------
No results are returned.
Expected behaviour
----------------------------------------
All results are returned, since no record is a Parent.
Comments
----------------------------------------
I thought at some point, `!=` internally generated `!= and IS NOT NULL` but maybe that was somewhere else.
I just learned about the MySQL null-safe equals operator, which would also solve this problem: https://stackoverflow.com/a/44723097/2832108https://lab.civicrm.org/dev/financial/-/issues/177Batch Transactions: Search cannot filter by Soft-Credit2023-07-21T17:13:51ZmhowisonBatch Transactions: Search cannot filter by Soft-CreditI'm trying to create a batch of existing contacts who have all made a contribution via Donor-Advised Fund (DAF).
When I use the Find Contributions search feature, I'm able to produce a list of contacts who gave through Donor-Advised Fun...I'm trying to create a batch of existing contacts who have all made a contribution via Donor-Advised Fund (DAF).
When I use the Find Contributions search feature, I'm able to produce a list of contacts who gave through Donor-Advised Funds in FY21 (Date Received=Previous Fiscal Year, Contributions AND Their Soft Credits, Soft Credit Type=Donor-Advised Fund).
However, when I run this same query through the search feature in the 'add existing transactions to a batch' screen, the results are incorrect. It filters for previous fiscal year contributions, but it returns all contributions rather than just contributions with a soft credit type of DAF.
The Soft Credit Type criteria doesn't seem to be registering. This shows you the 'add existing transactions to a batch' screen I'm referring to: (https://docs.civicrm.org/user/en/latest/contributions/accounting-integration/#create-a-new-batch-from-existing-transactions.)https://lab.civicrm.org/dev/core/-/issues/1926Finish allowing use of SSL to connect to database2023-07-24T03:44:31ZDaveDFinish allowing use of SSL to connect to databaseFollowing on to complete what's left for https://lab.civicrm.org/dev/core/-/issues/1137, there are a couple confusing things so I'm listing them to try to get a handle on them:
| thing | comment
| ---- | ----
| [CRM_Utils_File](https://...Following on to complete what's left for https://lab.civicrm.org/dev/core/-/issues/1137, there are a couple confusing things so I'm listing them to try to get a handle on them:
| thing | comment
| ---- | ----
| [CRM_Utils_File](https://github.com/civicrm/civicrm-core/blob/6eced9e6cedd66769757a30415a4c6c99fd0d231/CRM/Utils/File.php#L327) | This block seems to do almost the same thing as CRM_Core_DAO::init(), but not completely, but I think also has a "bug" where it wipes the sql_mode instead of updating it. This is called from e.g. the upgrade process, but it's not clear yet why that would require its own thing, since civi is already installed. Will test this out and see.
| [CRM_Utils_System::authenticate](https://github.com/civicrm/civicrm-core/blob/da6c68c61ab13ba434a5061701c3acfd8676cc29/CRM/Utils/System.php#L727) | It calls DB::connect assuming it's already available, and then does nothing with it, and then a couple milliseconds later the UF class does it again. Is this just leftover from refactoring, maybe as suggested in the comment a couple lines above. Additionally, only drupal 7 seems to use it, but it's not clear if it's really necessary and why can't it just use cms functions in the cms class, which it does in drupal 8 and the others.
| install/ | Would like to leave this out if the future plan is to move to a civicrm-setup variant. Otherwise it means adding fields to the form for example.https://lab.civicrm.org/dev/core/-/issues/4433E2E_Core_PathUrlTest::testGetUrl_WpAdmin() fails because CiviCRM routing is c...2023-07-24T20:52:34ZtottenE2E_Core_PathUrlTest::testGetUrl_WpAdmin() fails because CiviCRM routing is confusingThe gist of the test: it calls `cv url civicrm/contribute?reset=1` and asserts that the URL will open in the WordPress backend UI (aka `/wp-admin/`).
([Full source](https://github.com/civicrm/civicrm-core/blob/8e0dabd20bebe55d5dd725f301...The gist of the test: it calls `cv url civicrm/contribute?reset=1` and asserts that the URL will open in the WordPress backend UI (aka `/wp-admin/`).
([Full source](https://github.com/civicrm/civicrm-core/blob/8e0dabd20bebe55d5dd725f3015c41019cb8dbed/tests/phpunit/E2E/Core/PathUrlTest.php#L94-L117))
The test is failing. There are currently two patches:
* [25476](https://github.com/civicrm/civicrm-core/pull/25476): Automatically choose frontend/backend based on the route metadata
* [26772](https://github.com/civicrm/civicrm-core/pull/26772): Change the test to specifically request backend
Both have issues. So here's a deep-dive into the context.
Background
----------
* CiviCRM runs on Drupal/Backdrop and WordPress/Joomla.
* On WordPress/Joomla, the "frontend" and "backend" are different sub-applications (e.g. `/` vs `/wp-admin/`). The applications have very different URL structures. To make a hyperlink, you first decide which sub-application to target -- and then pick a page within it.
* On Drupal/Backdrop, it's one application, and all URLs have similar structure. To make a hyperlink, you simply identify the page.
* (*Drupal/Backdrop UX can sometimes distinguish frontend/backend -- but it's a visual choice based on local configuration. It's not a structural part of the URL.*)
* CiviCRM's routing system integrates into every UF/CMS, but (internally) it is closer to Drupal's. There is one `civicrm_menu` with all frontend+backend pages.
* On Drupal/Backdrop, CiviCRM's routes pass directly to CMS routes.
* On WordPress/Joomla, CiviCRM's routing integrates with both subapplications (ie "frontend" and "backend").
* CiviCRM stores a flag `bool is_public` for each route.
* CiviCRM includes routes which can be classified as:
* Purely backend (ex: `civicrm/dashboard`)
* Purely frontend (ex: `civicrm/event/register`)
* Purely web-service (ex: `civicrm/payment/ipn`)
* Multi-homed
* Ex: `civicrm/profile/view` has use-cases for frontend and backend
* Ex: `civicrm/ajax/api4/%` has use-cases for frontend, backend, and web-service
* CiviCRM has a function `CRM_Utils_System::url()`
* At first glance, it resembles Drupal's `\url()`. You typically just give the page (e.g. `url('civicrm/foo/bar')`).
* Over time, several additional parameters were added -- notably, the 6th parameter `bool $frontend` and the 7th parameter `bool $forceBackend`.
Problems
--------
* The status-quo invites bugs between CMS's.
* A developer working on Drupal will have trouble recognizing the importance of the 6th and 7th parameters. (*Those params do nothing on Drupal - and they're buried at the end of method.*)
* The status-quo invites bugs between interactive and automatic processes.
* A developer defines a custom token and tests it interactively; then at runtime, the token is sent by an automatic process. But the interactive/automatic split is orthogonal to frontend/backend split. Sometimes, interactive/automatic agree with each other (*both frontend or both backend*), and sometimes they disagree (*one frontend, one backend*).
* Overall, what tends to happen is:
* Developer writes a call like `CRM_Utils_System::url('civicrm/foo/bar')`, and it looks pretty.
* They test, and it works beautifully on their system.
* They publish, and it fails on other systems.
* Someone writes a patch to add 5 more parameters (`url('civicrm/foo/bar', '', FALSE, NULL, TRUE, TRUE)`). And then it's OK.
* The DX is awkward and invites bugs (in core and contrib) -- but it becomes more annoying for `cv` UX.
* If you need a 5th/6th param in PHP code, then you'll eventually figure it out and commit the update to your codebase. Then you forget about it.
* `cv` has some commands to facilitate manual testing and E2E testing (ie `cv url`, `cv http`, `cv open`). These are things that you improvise. Mismatched URLs can be annoying anytime you use these subcommands.
What to do
----------
There are two PRs to fix the test, and honestly - I don't really like either.
* [26772](https://github.com/civicrm/civicrm-core/pull/26772) makes the red mark go away, but it leaves the underlying issue (poor DX for PHP and poor UX for cv).
* [25476](https://github.com/civicrm/civicrm-core/pull/25476) aims to fix the underlying issue, but it's probably too facile. The current metadata can distinguish "purely frontend" pages from "purely backend" pages, but it cannot recognize "purely web-service" or "multi-homed". It probably messes-up some scenarios for those.
* Fixing this probably requires some more aggressive transition in the contract.
* Ex: Improve the routing metadata so that we can distinguish web-service routes and multi-home routes.
* Ex: Define a different class or function for generating URLs (*with a more usable signature*).
* Another option is to leave `civicrm-core` as-is -- and only update `cv`.
* Ex: Give `cv` the mechanism to resolve a frontend/backend based on metadata.
* Ex: Change `cv` to complain if you don't a specify frontend/backend flag.
* Either way, it feels like a bit of a wasted opportunity to only patch `cv` when we know that other users of `CRM_Utils_System::url()` get confused about the frontend/backend flags.https://lab.civicrm.org/dev/core/-/issues/2935Print/Merge produces blank results when using a docx or odt template file2023-07-27T10:57:01ZChrisHardiePrint/Merge produces blank results when using a docx or odt template fileOverview
----------------------------------------
When trying to generate thank you letters or other printed/merged documents, and when uploading a .docx or .odt template file to use, CiviCRM generates blank output (e.g. a blank PDF file...Overview
----------------------------------------
When trying to generate thank you letters or other printed/merged documents, and when uploading a .docx or .odt template file to use, CiviCRM generates blank output (e.g. a blank PDF file, a blank .docx file). When generating the same document without uploading a template file, CiviCRM successfully generates output that contains the desired document body.
Reproduction steps
----------------------------------------
1. Find one or more contacts, or contributions, or other entities that can be printed/merged
1. Select an action that involves print/merge (e.g. thank you letters)
1. Upload a template document to use for the printed/merged output, in .odt or .docx format
1. Generate the merged document in any format (pdf, docx, html, odt)
1. Observe that the resulting document is blank
Expected behaviour
----------------------------------------
When a merged/generated document is produced, it should use the uploaded template and the specified document body, and not be blank.
Environment information
----------------------------------------
* __Browser:__ Attempted with Safari 15.1 and Chrome 96.0.4664.27 on macOS
* __CiviCRM:__ 5.42.0
* __PHP:__ 7.4
* __CMS:__ WordPress 5.8.1https://lab.civicrm.org/dev/core/-/issues/4356(Exploration) Replace AngularJS with `$XXX`2023-07-28T00:09:29Ztotten(Exploration) Replace AngularJS with `$XXX`(*This document was written for a technical audience -- people who are familiar with many technologies, who want to bring them to Civi, and who find themselves asking, "Why can't we have nice things?" It was initially drafted as private ...(*This document was written for a technical audience -- people who are familiar with many technologies, who want to bring them to Civi, and who find themselves asking, "Why can't we have nice things?" It was initially drafted as private document;
then shared during the Manchester 2022 sprint as a [gist](https://gist.github.com/totten/5c34e3885a4fe7002f990e09395b4294). By request, it's now copied to the issue-tracker so that we have a more familiar discussion-space and reference-id. I've made
a few small edits and documented one more of the challenges, but the substance is largely the same.*)
# Table of Contents
[[_TOC_]]
# Preliminaries
## Why replace AngularJS?
* The upstream "AngularJS" project (*a Javascript framework*) re-made itself as "Angular" (*a Typescript framework*).
* This reflects important changes in the platform, and it is not a straight-forward upgrade. Adopting newer "Angular" versions should be approached as a *migration*.
* The upstream "AngularJS" project has been proclaiming: "We don't like our JS product anymore. Don't use that."
* One original goal of adopting "AngularJS" was to get some bandwagon benefits. Those benefits are no longer there.
## What is "$XXX"?
`$XXX` is a stand-in for "Angular 14 or ReactJS or some other client-side page-rendering system". The specific value is a matter for consideration.
Some migration issues are generally applicable to any alternative, and we may use `$XXX` as stand-in (because it gets tiresome to write "Angular 14 or ReactJS or some other client-side page-rendering system").
## What is CiviCRM?
For purposes of this evaluation, CiviCRM is an ecosystem -- a collection of packages developed and maintained by many people. It includes a significant core package and a significant set of extensions. If we adopt `$XXX`, then we ae encouraging this ecosystem to write their screens and UI components with `$XXX`.
Today, CiviCRM's ecosystem is quite varied:
* You have `civicrm-core` -- which includes the framework, several common business modules, APIs, and various UIs for admins+staff+consumers.
* You have extensions like CiviVolunteer or CiviGrant -- which define new business modules (including their APIs and UIs).
* You have extensions like CiviRules or Search Kit -- which build generic functionality (that works with many business modules).
* You have extensions like Mosaico or CiviDiscount -- which significantly improve the features of an existing business-module.
* You have extensions like AngularProfiles or Email API -- which add extra building-blocks or fill-in gaps.
* You have bespoke extensions for specific sites -- extensions which fine-tune screens, workflows, settings, for a specific organizational context.
Each package may define some mix of:
* Redistributed artifacts (like CSS files, APIv3 files, APIv4 files, Smarty templates, etc) curated by developers.
* Local artifacts (like Profiles and SavedSearches and Webforms and Afforms) curated by administrators for a specific site (with support from the extensions).
These packages are distributed in various ways, eg:
* Published as part of `civicrm-core`
* Published in a public directory for impromptu download/installation
* Organized as in-house collections
Is this the optimal shape for the ecosystem? I'm not sure how you would answer that.
Instead, for this discussion, I simply accept this shape for the ecosystem. CiviCRM's ecosystem should continue to have similar kinds of packages and similar kinds of distribution -- even if some specific packages change, and even if it adopts another tool like `$XXX`. This shape is not perfect or static - but it is realistic and representative.
# Challenges
Perhaps we just drop-in some new files for the latest version of Angular? Or, failing that, drop-in ReactJS? Surely, at the end of the day, these are JS files used by the browser. You just need to put them on a web-server. The browser will run them, and that's it. What's wrong with that?
Let's break-down a few specific challenges.
* Linker-Loader and Modularity
* Angular 14 and TypeScript
* ReactJS and JSX
The challenges can be *addressed* -- the question is what *trade-offs, costs, or compromises* we accept.
## Linker-Loader and Modularity
### Definition
Sane software is built in pieces. To run the software, you have to put the pieces together. This requires a linking or loading mechanism.
Just to clarify what I mean, let's use a basic example. Here we have some classes for an application where the user draws rectangles on a canvas:
```js
// FILE: Canvas.js
class Canvas {
constructor() {
// A list of visual objects to display in this presentation
this.objects = [];
}
addRectangle(x, y, width, height) {
this.objects.push({
pos: new Coordinate(x, y),
obj: new Rectangle(width, height)
});
}
}
```
```js
// FILE: Rectangle.js
class Rectangle {
constructor(width, height) {
this.width = width;
this.height = height;
}
}
```
```js
// FILE: Coordinate.js
class Coordinate {
constructor(x, y) {
this.x = x;
this.y = y;
}
}
```
N.B. To use `Canvas`, you must also use `Rectangle` and `Coordinate`. There must be a mechanism to say: "*We are loading `Canvas.js` and also `Rectangle.js` plus `Coordinate.js`.*"
If you're reading this, then I'm sure you're already familiar with several such mechanisms, though they go by many names. Perhaps you've encountered PHP's "autoloader"; Javascript's "webpack"; bash's "PATH"; C's "static-linking" and "libtool". All of these address the need to combine/resolve related pieces of software.
The design of the link-load mechanism can have effects on:
* Performance - Load times, download sizes, cache efficacy, update frequency, etc. (*A link-load mechanism that performs nicely for small apps might perform badly for large apps, and vice-versa.*)
* Workflow - The ease or difficulty of installing, developing, customizing, upgrading, etc.
* Modularity - How you design/frame/distribute the packages of the system.
In the phrase "link-load", I wish to evoke the general mechanism of "getting the stuff we need" -- with an understanding that different systems have different phrases, tools, and flows.
### What kind of linking/loading does CiviCRM require?
Recall that the CiviCRM ecosystem has a mix of extensions providing different kinds of value (*business modules, APIs, UIs, etc*) with various artifacts (*CSS files, APIv4 files, Quickform files*). If we adopt `$XXX`, then `$XXX` will provide more kinds of artifacts (*JS-based pages; Angular components; React components; etc*). You should be able to define these in Civi extensions. Thus, you might create files like:
```
## Angular component defined in core on Drupal 9
/var/www/vendor/civicrm/civicrm-core/ang/*.component.ts
## React component defined in an extension on Drupal 7
/var/www/sites/default/files/ext/myextension/react/*.jsx
```
The linker/loader mechanism used for `$XXX` in CiviCRM should be able to:
* Load screens defined by core+extensions.
* Load components defined by core+extensions (across extension boundaries).
* Load transitive dependencies required by the screens+components in core+extensions (across extension boundaries).
### What does linking/loading look like in Javascript?
Javascript is a big ecosystem! And it's been around for 25 years. We will not try to discuss all the techniques of the many applications/frameworks/tools. Rather, let's note a couple relevant practices - the baseline; the current mechanisms in CiviCRM; and the mechanisms favored in contemporary Angular+React ecosystems.
First, as a baseline, the traditional mechanism (*available to all web developers since 1995*) is to write HTML: you create a webpage with a list of `<script>` tags containing URLs of all your JS files.
```html
<head>
<script type="text/javascript" src="./js/Rectangle.js"></script>
<script type="text/javascript" src="./js/Coordinate.js"></script>
<script type="text/javascript" src="./js/Canvas.js"></script>
</head>
```
This is (roughly) how JS files get into CiviCRM's PHP screens - except that the `<script>` tags are written as PHP statements (`addScriptFile($ext, $file)`). But it's similar: it's a top-level list of JS files, and they're printed as a list of `<script>` tags in the main HTML document. On a case-by-case basis, some screens make extra calls to `addScriptFile`/`<script>`.
CiviCRM's AngularJS screens use another mechanism -- a *dependency-graph*. It works as follows:
* Each major Angular module has an explicit metadata file (`*.ang.php`) with a list of inputs and outputs (components/tags/etc).
* If a site-builder/developer makes a custom form (`*.aff.html`), then we scan the HTML and infer its dependencies (based on the tags and attributes).
* When a user requests an Angular-based screen (eg `civicrm/admin/search`), then we use this metadata to identify dependencies and send them all as a bundle.
In contemporary Angular and ReactJS ecosystems, it is also conventional to use a dependency-graph -- but now based on *ES6 modules*. ECMAScript 6 defines *a language-level standard* with `import`/`export` statements. For example, `Canvas.js` could be updated to:
```js
// FILE: Canvas.js
import { Coordinate } from 'Coordinate.js';
import { Rectangle } from 'Rectangle.js';
export class Canvas {
constructor() {
// A list of visual objects to display in this presentation
this.objects = [];
}
addRectangle(x, y, width, height) {
this.objects.push({
pos: new Coordinate(x, y),
obj: new Rectangle(width, height)
});
}
}
```
For pure JS development, this represents a significant improvement over the baseline -- you do not need to maintain a global list of scripts. You don't need custom metadata (like `*.ang.php`). You describe the JS dependencies within the JS files. And unlike Civi's AngularJS loader, this mechanism is a widely known standard. ES6 modules have become nearly ubiquitous in documentation for contemporary JS libraries.
### Challenges of linking/loading for Javascript
ES6 `import`s are part of the language standard, but you cannot take this mechanism for granted. There are multiple implementations with important differences. To see this, let's do an example.
Suppose you are writing an extension like "Mosaico" which builds on top of CiviMail. You need to use a (Angular/React/etc) component from CiviMail. So you `import` it:
```js
import ReceipientSelector from '../../../sites/all/modules/civicrm/components/Mailing/RecipientSelector.js';
const MosaicoEditor = () => <div>....<ReceipientSelector />...</div>;
export default MosaicoEditor;
```
This import (*with a physical path*) is compatible with most implementations of `import`, but the path itself is untenable -- it will only work on (say) 50% of CiviCRM deployments. For broad support, you need to hide some of these physical details and use a simpler (logical) path -- such as:
```diff
-import ReceipientSelector from '../../../sites/all/modules/civicrm/components/Mailing/RecipientSelector.js';
+import ReceipientSelector from 'civicrm/components/Mailing/RecipientSelector.js';
```
This can be done - and I suspect that most pure-JS apps benefit from simpler expressions. However, this depends on *the particular implementation* of `import`. Implementations fall into a few major groups:
* _Browsers_: Since [~2019](https://caniuse.com/mdn-javascript_operators_import), most major web-browsers have had support for `import`s with physical paths. However, support for [logical paths ("import maps")](https://github.com/WICG/import-maps) is [gradually rolling out](https://caniuse.com/import-maps). At time of writing, Chrome has support; Firefox requires an experimental opt-in (`about:config`); and Safari has no support. ([There may be a polyfill](https://github.com/systemjs/systemjs/blob/main/docs/import-maps.md).)
* _Bundlers_: Bundlers (like `webpack` or `rollup`) have provided support for a longer period, and they appear in many web-app project-templates. Bundlers scan JS code, examine `import`s, and build aggregated JS files. They have more features -- such as customized path resolution and build-optimizations ("minification", "tree shaking"). However, most (all?) popular bundlers are designed for NodeJS.
(*Question: Is the choice of `import` mechanism all-or-nothing? Or do they interoperate? For example, suppose you use `webpack` to make a base-bundle `$XXX + common widgets` -- and then you load a few extra JS modules in the browser. Will `import`s in the browser find resources from `webpack`'s bundle?*)
### Approaches to linking/loading ES6 for CiviCRM
1. Don't support ES6 modules
* _Pro_: No new code!
* _Con_: You will find it confusing to read documentation+examples in the `$XXX` ecosystem. Authors tend to assume you have figured out ES6. (*If you read `import foo from bar.js`, then you will need to mentally substitute calls like `addScriptFile('bar.js')`.*)
* _Con_: Some libraries may be difficult/impossible to assimilate (*because the code is sprinkled with unresolvable `import`s*).
1. Rely on browser support. All `import`s in Civi-land use physical paths. Rearrange Civi.
* _How_: Rearrange the file-tree (for every Civi-CMS integration) to make physical paths cleaner and more predictable.
* _Pro_: Clean, predictable paths are pleasant.
* _Pro_: Standards-based `import` mechanism. No special tools/libraries.
* _Con_: Major compatibility break with existing CiviCRM deployments/hosts/scripts.
* _Con_: Doesn't support bundling/aggregation.
1. Rely on browser support. All `import`s in Civi-land use physical paths. Sync files during installation.
* _How_: Define a data folder like `[civicrm.files]/js`. Whenever you install/upgrade/remove an extension, synchronize the JS resources to this folder -- and ensure that each file lands in a clean/predictable subdir. The web-browser requests JS files from `[civicrm.files]/js`.
* _Pro_: No major compatibility break.
* _Pro_: Clean, predictable paths are pleasant.
* _Con_: Large file sync's are unpleasant (especially during development).
* _Con_: Doesn't support bundling/aggregation.
1. Rely on browser support. All `import`s in Civi-land use import-maps.
* _How_: In PHP, define a registry of import-paths. Fill this based on the list of installed extensions. Output the `import-map` to the HTML `<HEAD>`.
* _Pro_: Works with existing file-tree.
* _Pro_: Standards-based `import` mechanism. No special tools/libraries.
* _Con_: Firefox+Safari support is... eh... "pending". *There may be a polyfill (not yet assessed).*
* _Con_: Doesn't support bundling/aggregation.
* _Comment_: If we think that Firefox+Safari will have good support in (say) 2023, then we could proceed down this path and simply keep this functionality in "alpha" until 2023.
1. Rely on a bundler. Every extension calls the bundler on its own.
* _How_: In each extension that uses `$XXX` for a UI, define `package.json` (et al) to download a bundler (eg `webpack`) and build a custom bundle. To include resources from another extension, ask Civi for the paths (eg `cv path -x $EXT`). Publish the compiled bundle as part of the extension.
* _Pro_: Use a popular bundler with advanced features (like tree-shaking).
* _Pro_: Each extension developer can choose their own bundler. (Freeedom!!)
* _Con_: All such extensions will have independent versions of `$XXX` (and independent versions of common utilities/widgets/etc).
* _Con_: Upgrades/bugfixes are onerous (*must upgrade every extension; they're all statically-linked*).
* _Con_: If you have a screen (e.g. "The Dashboard") that mixes elements from multiple extensions, then you will get multiple copies of `$XXX`.
* _Con_: Impossible for one extension to add to another extension (*eg CiviDiscount adding elements to CiviEvent*).
* _Con_: Strange bootstrap process (*similar to `civicrm_generated.mysql`; you need a working Civi deployment with the extension enabled before you can write the `*.js` file that will be redistributed.*)
1. Rely on a bundler. Integrate a popular bundler (like `webpack`/`rollup`) into Civi. Run it whenever you install/upgrade/remove extensions.
* _How_: Make a JS script which calls `webpack` (etc). Pass key configuration-data (e.g. the list of bundles and JS files) from Civi to `webpack`.
* _Pro_: Use a popular bundler with advanced features (like tree-shaking).
* _Pro_: Works with existing file-tree.
* _Con_: Requires JS runtime -- eg `node` subprocess, `node` daemon, or browser-based queue-worker.
* _Con_: Vendor-specific `import` mechanism.
* _Con_: No dynamic bundling (*based on user-defined forms, settings, etc*)
1. Rely on a bundler. Use/improve a PHP-based bundler for JS files.
* _How_: There's an alpha-stage JS bundler at https://bitbucket.org/fugu-fuman-library/es6bundler/src/main/. Add it to core. As necessary, contribute to it.
* _Pro_: Works with existing source-tree.
* _Pro_: Supports bundling
* _Pro_: Supports dynamic bundling (*based on user-defined forms, settings, etc*)
* _Con_: It's alpha-stage with no major user-base. It likely needs work.
* _Con_: There is not much of an ecosystem for parsing JS in PHP. You're likely to use approximations (regex) or something slow.
* _Con_: Limited optimizations (Never as good as `webpack`. Ex: You can do minification but not tree-shaking.*)
* _Con_: Vendor-specific `import` mechanism.
1. Rely on 2-step bundler. Use JS to build dependency graph. Use PHP to concatenate final bundle(s).
* _How_: If an extension publishes JS files, then it should also publish a JSON index (listing the files+dependencies). Provide JS devtool to generate the index. In Civi, use the PHP bundler.
* _Pro_: Works with existing source-tree.
* _Pro_: Supports bundling
* _Pro_: Supports dynamic bundling (*based on user-defined forms, settings, etc*)
* _Pro_: JS scanning is done by JS code (*should be able to find a robust JS parser*)
* _Con_: No one else is doing it.
* _Con_: Need to include JS indexer as part of extension-development workflow.
* _Con_: Limited optimizations (Never as good as `webpack`. Ex: You can do minification but not tree-shaking.*)
* _Con_: Vendor-specific `import` mechanism.
## Angular, TypeScript, ng
Angular 14 appears to parallel the original AngularJS in that it has two main file-types:
| Version | Logic/Behavior-Implementation | Layout/Component-Graph |
| -- | -- | -- |
| AngularJS | Javascript (`*.js`) | XHTML (`*.html`) |
| Angular | Typescript (`*.ts`) | XHTML (`*.html`) |
The good news is that the layout/component-graph is broadly the same format - HTML provides the canvas for organizing the choice of components that you want on the screen. This canvas still uses a well-known format which is amenable to reading/filtering/writing with _many_ tools. This gives us a good prognosis for migrating data (*immediately*) and maintaining it (*future updates*).
But... the logic/behavior files changed to Typescript. This requires a TypeScript compiler. Like with linker-loader, the compiler needs to be taught to load classes from different extensions (*ie the type-checker needs full visibility on the list of types - and types can originate in different places*).
I haven't examined of how Angular 14 implements its compiler/linker/loader mechanisms, so the list of approaches here is nowhere near as detailed as the link-loader discussion above. But loosely, the documented/supported mechanisms are built around a CLI tool named `ng` (*implemented with `node`*). The general options appear to be:
* Integrate with `ng`. (This will have similar pro/con as the integrating with the webpack bundler.)
* Examine+decompose the functionality of `ng`. It may be possible to combine this with one of our linker-loader approaches.
* Don't support TypeScript at all. (This will likely be very difficult. It's true that earlier versions of Angular presented JS options, and it's true that Angular compiles down to JS, but contemporary documentation is exclusively geared toward TS. Even if we figure out how, they're liable to change it.)
The adoption of `ng` represents another significant change between AngularJS and contemporary Angular. This is likely very nice for people who write pure-Angular applications -- the `ng` tool provides more functionality and hides more details. Thus, they can make big changes in the mechanics (eg [switching from "Just in Time" compilation to "Ahead of Time" compilation](https://angular.io/guide/aot-compiler)) without significantly changing the DX. But I worry about the breadth of `ng`'s scope -- that your choice is to either (a) do everything "the ng way" or (b) continuously reengineer your process to stay aligned with `ng`.
I have a theory (*but can't fully back it up, as I haven't been following along with Angular 2+ closely*) -- by introducing `ng`, Angular began asserting more ownership over the stack+workflow. This has probably helped pure-Angular development work (because lots of changes can be encapsulated within `ng`) -- but it's has made it harder to integrate with other applications (like Civi and the CMSs). It doesn't seem like a coincidence that AngularJS (*JS library*) had a fairly long/stable 1.x series -- while Angular (*Typescript platform*) has done frequent major-version bumps.
## ReactJS and JSX
(*Sketch/Notes*)
* The graph of ReactJS components is defined as a series of JS functions. These functions can be written in pure JS, but they are typically written in "JSX" (JS with inline HTML).
* From POV of an HTML developer, the extra power of JSX is that it provides a full (Turing complete) language -- where you can mingle function calls, loops, etc.
* From POV of a JS developer, the extra ease of JSX is that embeds HTML (rather than using verbose DOM calls).
* These things are exactly what makes it hard to process in other ways -- eg if the component-graph is defined with JS function-calls, then you're tooling must model "JS function-calls".
* To provide __tooling__ around JSX, you need the ability to compile/parse/generate JSX (aka "encode/decode"; aka "use the abstract syntax tree (AST)").
* Here are some of the things that would require AST/encode-decode capability:
* Drag/drop editor (let a web-user compose the graph of widgets)
* Incremental-upgrades (automatically swap deprecated code with supported code)
* Make client+server logic match-up (use the same document to determine entity-bindings and validations)
* The standard tooling for JSX (compilers/parsers/ASTs) are implemented in JS themselves. But if the server and dev-workflow are in PHP, then you have to either (a) live with constrained callouts between PHP+JS or (b) reimplement JSX tooling in PHP.
* Generally, to support more tooling+workflows, you want a standardized notation such as XML or JSON.
* It is possible to use ReactJS *only* as a runtime (*in the same sense that Civi Profiles (`UFGroup`s) use Quickform as the runtime*).
* You might do this if you want to get some part of ReactJS (e.g. pure-functional code-style; e.g. branding) but don't need "native feel".
* ReactJS without JSX is not the "ReactJS experience". For a developer who's experienced in ReactJS, it will _feel_ alien.
* One example: ["HTM"](https://github.com/developit/htm) is a different templating language that also produces React components. HTM relaxes some of workflow
requirements (e.g. no need for transpiler). However, HTM (specifically) still has similar tooling+AST issues as JSX.
## Walk and/or Chew Gum
Broadly speaking, there is a tension between:
* (a) Providing value to users -- i.e. improving utility/functionality of SearchKit and Form Builder. We have some momentum and regular investment, and we're in the middle of converting existing screens to the newer stack (e.g. `civicrm_admin_ui`). We don't want to kill this.
* (b) Replacing AngularJS -- i.e. migrating to Angular(TS) or ReactJS or similar. This could be as a difficult as a rewrite. If you do a straight-conversion, that's a high-bar -- you may be stuck with a project that cannot be released for 12-18 months. Or you may be stuck with two parallel versions, where the new one never reaches completion.
The challenge is to navigate that tension in a realistic way. It seems unrealistic to stop all improvements for a year. It also seems unrealistic to stay on AngularJS permanently. So how do you address both goals?
To my mind, the most promising concept so far looks roughly like this:
* Take our current data-formats of SK/FB. Formalize them -- define schemas with version-numbers, add enforcement mechanisms (warnings/migrations).
* The data-formats require runtime-interpreters. We currently have one based on AngularJS. We seek to add one based on `$XXX`.
* Gradually phase-out the AngularJS interpreter; gradually phase-in the `$XXX` interpreter. Site-builders and extension-developers use
this data-format throughout the transition.
Here's an example of an incremental transition:
* __Definitions__:
* SKFB-A is the SearchKit/FormBuilder runtime on Angular.
* SKFB-X is the SearchKit/FormBuilder runtime on `$XXX`.
* __Incubation Period__:
* SKFB-A is default. All stable/released functionality is based on SKFB-A.
* SKFB-X is undergoing primary development. You can opt into it (e.g. toggle a flag or extension).
* The format may still undergo small revisions which are implemented/supported by SKFB-A.
* __Sync-Up Period (Soft Freeze)__
* The format is in soft-freeze. Features shouldn't be added (unless doing so helps to achieve the sync-up).
* SKFB-A is default. It may get small bug-fixes.
* SKFB-X is the focus of development attention. The aim is to achieve compatibility with current FMT and most screens.
* __Wind-Down Period__:
* SKFB-X is used for all user-generated screens and many built-in screens.
* SKFB-A is still available - and is used transitionally for screens that require it.
* The format is unfrozen. It may continue to evolve on SKFB-X.
* __Other Considerations__:
* Throughout the process, both SKFB-A and SKFB-X are developed in `civicrm-core.git`.
* Throughout the process, downstream site-builders and developers (of other extensions) may continue to use FMT define screens.https://lab.civicrm.org/dev/core/-/issues/4006PHP 8 - Undefined variable warnings in compiled Smarty templates2023-07-28T18:28:40ZAdam WoodPHP 8 - Undefined variable warnings in compiled Smarty templatesHave just upgraded to PHP 8.1 and CiviCRM 5.55.2.
We can now see a bunch of warnings about undeclared variables / array keys in the `error_log`, all of which seem to come from compiled Smarty template files in `templates_c`.
Below are ...Have just upgraded to PHP 8.1 and CiviCRM 5.55.2.
We can now see a bunch of warnings about undeclared variables / array keys in the `error_log`, all of which seem to come from compiled Smarty template files in `templates_c`.
Below are the most common repeat offenders (from a quick scan of the log), but there are probably more:
```
[Sun Nov 27 08:35:34.862647 2022] [fcgid:warn] [pid 62763] [client 92.2.56.169:60387] mod_fcgid: stderr: PHP Warning: Undefined array key "class" in /home/cses_org_uk/public_html/media/civicrm/templates_c/en_GB/%%11/11A/11A32C94%%UserDashboard.tpl.php on line 13, referer: https://cses.org.uk/membership/login
[Sun Nov 27 08:35:34.862655 2022] [fcgid:warn] [pid 62763] [client 92.2.56.169:60387] mod_fcgid: stderr: PHP Warning: Undefined array key "fields" in /home/cses_org_uk/public_html/media/civicrm/templates_c/en_GB/%%D2/D27/D2748E2E%%GrantApplicationDashboard.tpl.php on line 27, referer: https://cses.org.uk/membership/login
[Sun Nov 27 09:21:17.286322 2022] [fcgid:warn] [pid 64736] [client 66.249.76.73:54724] mod_fcgid: stderr: PHP Warning: Undefined array key "sidebarLeft" in /home/cses_org_uk/public_html/media/civicrm/templates_c/en_GB/%%EA/EAA/EAA96A89%%joomla.tpl.php on line 19
[Sun Nov 27 09:21:17.294758 2022] [fcgid:warn] [pid 64736] [client 66.249.76.73:54724] mod_fcgid: stderr: PHP Warning: Undefined array key "localTasks" in /home/cses_org_uk/public_html/media/civicrm/templates_c/en_GB/%%EA/EAA/EAA96A89%%joomla.tpl.php on line 56
[Sun Nov 27 09:21:17.294768 2022] [fcgid:warn] [pid 64736] [client 66.249.76.73:54724] mod_fcgid: stderr: PHP Warning: Undefined array key "registerClosed" in /home/cses_org_uk/public_html/media/civicrm/templates_c/en_GB/%%8E/8EE/8EE20E00%%EventInfo.tpl.php on line 6
[Sun Nov 27 09:21:17.294772 2022] [fcgid:warn] [pid 64736] [client 66.249.76.73:54724] mod_fcgid: stderr: PHP Warning: Undefined array key "summary" in /home/cses_org_uk/public_html/media/civicrm/templates_c/en_GB/%%8E/8EE/8EE20E00%%EventInfo.tpl.php on line 82
[Sun Nov 27 09:21:17.294774 2022] [fcgid:warn] [pid 64736] [client 66.249.76.73:54724] mod_fcgid: stderr: PHP Warning: Undefined array key "summary" in /home/cses_org_uk/public_html/media/civicrm/templates_c/en_GB/%%8E/8EE/8EE20E00%%EventInfo.tpl.php on line 97
[Sun Nov 27 09:21:17.294777 2022] [fcgid:warn] [pid 64736] [client 66.249.76.73:54724] mod_fcgid: stderr: PHP Warning: Undefined array key "profileGID" in /home/cses_org_uk/public_html/media/civicrm/templates_c/en_GB/%%77/771/771F796E%%Google.tpl.php on line 36
[Sun Nov 27 09:21:17.294780 2022] [fcgid:warn] [pid 64736] [client 66.249.76.73:54724] mod_fcgid: stderr: PHP Warning: Undefined array key 1 in /home/cses_org_uk/public_html/media/civicrm/templates_c/en_GB/%%8E/8EE/8EE20E00%%EventInfo.tpl.php on line 148
[Sun Nov 27 09:21:17.294783 2022] [fcgid:warn] [pid 64736] [client 66.249.76.73:54724] mod_fcgid: stderr: PHP Warning: Trying to access array offset on value of type null in /home/cses_org_uk/public_html/media/civicrm/templates_c/en_GB/%%8E/8EE/8EE20E00%%EventInfo.tpl.php on line 148
[Sun Nov 27 09:21:17.294786 2022] [fcgid:warn] [pid 64736] [client 66.249.76.73:54724] mod_fcgid: stderr: PHP Warning: Undefined array key 1 in /home/cses_org_uk/public_html/media/civicrm/templates_c/en_GB/%%8E/8EE/8EE20E00%%EventInfo.tpl.php on line 148
[Sun Nov 27 09:21:17.294789 2022] [fcgid:warn] [pid 64736] [client 66.249.76.73:54724] mod_fcgid: stderr: PHP Warning: Trying to access array offset on value of type null in /home/cses_org_uk/public_html/media/civicrm/templates_c/en_GB/%%8E/8EE/8EE20E00%%EventInfo.tpl.php on line 148
[Sun Nov 27 09:21:17.294791 2022] [fcgid:warn] [pid 64736] [client 66.249.76.73:54724] mod_fcgid: stderr: PHP Warning: Undefined array key "groupId" in /home/cses_org_uk/public_html/media/civicrm/templates_c/en_GB/%%FD/FD4/FD481315%%CustomDataView.tpl.php on line 176
[Sun Nov 27 09:21:17.294794 2022] [fcgid:warn] [pid 64736] [client 66.249.76.73:54724] mod_fcgid: stderr: PHP Warning: Undefined array key "emailMode" in /home/cses_org_uk/public_html/media/civicrm/templates_c/en_GB/%%79/79B/79BBF2EC%%SocialNetwork.tpl.php on line 11
```
Looking into one case in further detail (missing "summary" on line 82 of `EventInfo.tpl.php`), we see that the following Smarty code:
```smarty
<div class="vevent crm-event-id-{$event.id} crm-block crm-event-info-form-block">
<div class="event-info">
{* Display top buttons only if the page is long enough to merit duplicate buttons *}
{if $event.summary or $event.description}
```
Compiles into the following PHP:
```php
<div class="vevent crm-event-id-<?php echo $this->_tpl_vars['event']['id']; ?>
crm-block crm-event-info-form-block">
<div class="event-info">
<?php if ($this->_tpl_vars['event']['summary'] || $this->_tpl_vars['event']['description']): ?>
```
(i.e. there is no protection that the array index actually exists.)
So it looks like there may be an overall effort required here to tighten up the templates and/or corresponding page class PHP code to ensure that array indices are always declared correctly - and I'm happy to tackle some of these in the background - but is there any update from Smarty planned or pending that would affect this?
In this instance (and I suspect many of the others), I would suggest we use the `empty()` function to test for the variables in question:
```smarty
<div class="vevent crm-event-id-{$event.id} crm-block crm-event-info-form-block">
<div class="event-info">
{* Display top buttons only if the page is long enough to merit duplicate buttons *}
{if !empty($event.summary) or !empty($event.description)}
```https://lab.civicrm.org/dev/core/-/issues/2509Search kit: Links to case (and other) activities go to the wrong form2023-08-01T01:50:48ZDaveDSearch kit: Links to case (and other) activities go to the wrong formIf you do a search either purely on activities and the results happen to include case activities, or a search on cases and include the related contact activities, the links to the activities in the results list go to the regular activity...If you do a search either purely on activities and the results happen to include case activities, or a search on cases and include the related contact activities, the links to the activities in the results list go to the regular activity form not the case activity form. This matters because:
1. It's not all the right fields on the form.
2. Form hooks won't work properly.
3. You get errors on save.
@colemanwhttps://lab.civicrm.org/dev/core/-/issues/4299Send contribution receipt when contribution completed by recording payment on...2023-08-01T14:42:08ZlarsssandergreenSend contribution receipt when contribution completed by recording payment on the backendThrough discussion in this [PR](https://github.com/civicrm/civicrm-core/pull/26247), it has become clear that there is an inconsistency in which message templates are sent when completing a pending contribution with a payment, depending ...Through discussion in this [PR](https://github.com/civicrm/civicrm-core/pull/26247), it has become clear that there is an inconsistency in which message templates are sent when completing a pending contribution with a payment, depending on where the payment is recorded.
If the payment is recorded via API and the receipt has not yet been set, a contribution receipt template is sent.
If the payment is recorded via a Search Action and the receipt has not been sent, a contribution receipt template is sent.
If the payment is recorded via Record Payment in the UI, there is an option to send to send a receipt, but if selected a "Additional Payment Receipt or Refund Notification" template is sent.
I think that in the third case, for consistency, we should send a contribution receipt template as long as the contribution is now completed. The same logic works with membership receipt templates.https://lab.civicrm.org/dev/core/-/issues/4113Afform - Can't display default value for deceased field2023-08-01T20:14:31ZGhost UserAfform - Can't display default value for deceased fieldOverview
----------------------------------------
When you create a Search Kit and want to set a default value to the deceased field in the Form Builder, it doesn't show.
Reproduction steps
----------------------------------------
1. C...Overview
----------------------------------------
When you create a Search Kit and want to set a default value to the deceased field in the Form Builder, it doesn't show.
Reproduction steps
----------------------------------------
1. Create a Search Kit based on Contact and create a Form.
2. In the Form add the 'Is deceased' field into the Form Layout and select a default value. Also add a route to be able to see the Form. See image below:
![image](/uploads/c19e8c240e4c53e1ad662cb30953fbc2/image.png)
3. Go to the Page that you created and you see that the default value is not selected.
![image](/uploads/5c85d8a8dcd614967a9ab67210561f7b/image.png)
Expected behaviour
----------------------------------------
In the created Page the default value should be already selected, which it isnt't the case.
Environment information
----------------------------------------
<!-- Some of the items below may not be relevant for every bug - if in doubt please include more information than you think is neccessary. -->
* __Browser:__ Firefox 109.0.1/Chrome 109.0.5414.120
* __CiviCRM:__ 5.60.alpha1
* __Site:__ [dmaster](https://dmaster.demo.civicrm.org/)https://lab.civicrm.org/dev/user-interface/-/issues/52Consistent markup/classes for status messages2023-08-02T18:26:34ZbgmConsistent markup/classes for status messagesWhile looking at the code of a popular theme, grepping for `\.status`, it made more realize how many different uses there are for the `status` class.
- Footer status
- Contribution Dashboard: when there are no events, there is `status n...While looking at the code of a popular theme, grepping for `\.status`, it made more realize how many different uses there are for the `status` class.
- Footer status
- Contribution Dashboard: when there are no events, there is `status no-popup`, without success/warning
![image](/uploads/7740b1f673e27365551dd18b13ea2230/image.png)
- Find Participants \> Cancel registrations, uses `messages status no-popup`, and the theme has to guess it's a warning:
![image](/uploads/c972b46b3c337ee8e4d2327b17a5d398/image.png)
- Upgrade messages (https://github.com/civicrm/civicrm-core/pull/25961)
- Standalone: https://github.com/civicrm/civicrm-core/pull/26965/fileshttps://lab.civicrm.org/dev/core/-/issues/2565HTML editor for additional event confirmation email text2023-08-02T21:16:19ZlarsssandergreenHTML editor for additional event confirmation email textIt would be nice to be able to have an HTML editor for the additional event confirmation text entered on the Manage Event Page that gets inserted into the System Template email template. The benefit would be to be able to easily add link...It would be nice to be able to have an HTML editor for the additional event confirmation text entered on the Manage Event Page that gets inserted into the System Template email template. The benefit would be to be able to easily add links, etc. I understand this will be a complicated one, but it does seem worthwhile as almost every other similar field is HTML, so this one really stands out. [See previous issue.](https://lab.civicrm.org/dev/core/-/issues/852#note_58464)
How it works now: Text from the event confirmation email text is added into the event confirmation message template with some minimal tags added. I see a <p> wrapper around the text and <br /> inserted for newlines, at a cursory glance.
Here's my guess at what needs to happen:
1. Add HTML editor to Event Configuration
2. Add HTML editor on the New Event Registration page (which is also Add Participant from events or Add Event Registration for a contact), so that the message can be edited before sending.
3. Inject content as HTML into Message Templates, i.e. remove code that adds tags, wrap code in div or something.
4. Convert all existing content in the DB with the same process now taken to inject the text into the Message Template or handle the content injection into the Message Template with two cases based on text/HTML.
The first three should be relatively simple, but the last one there seems like the tricky part. How has this has been handled in the past?https://lab.civicrm.org/dev/core/-/issues/4354Activities created via API should notify Assignees2023-08-03T12:38:52Zwil_SRQActivities created via API should notify AssigneesOverview
----------------------------------------
If the GUI (https://cc.unidosnow.org/civicrm/activity?action=add) would notify assignees when an activity is created. It'd be useful for the API to do so too. The GUI respects the setting...Overview
----------------------------------------
If the GUI (https://cc.unidosnow.org/civicrm/activity?action=add) would notify assignees when an activity is created. It'd be useful for the API to do so too. The GUI respects the setting in Administer > Customize Data and Screens > Display Preferences > Notify Activity Assignees and notification rules by Activity type. It'd be useful for the API to do so too.
Example use-case
----------------------------------------
1. Invoke civicrm_api3('Activity', 'create', []) or civicrm_api4('Activity', 'create', [])
2. Include assignees
Current behaviour
----------------------------------------
The Activity is created but the assignees are not notified, even in situations where creating the equivalent activity via the GUI would have issued notifications.
Proposed behaviour
----------------------------------------
Notify assignees using the same rules and notification format as the GUI.
Comments
----------------------------------------
See https://civicrm.stackexchange.com/q/45078/5446
Workaround is to call CRM_Activity_BAO_Activity::sendToAssignee() separatelyhttps://lab.civicrm.org/dev/core/-/issues/4389Can't import using contact id2023-08-03T12:40:09Zaydunsaidan.saunders@squiffle.ukCan't import using contact idOverview
----------------------------------------
`Contact id` cannot be selected as a field to match on when importing.
(I'm fairly certain this used to work, but don't know when it stopped.)
Reproduction steps
-----------------------...Overview
----------------------------------------
`Contact id` cannot be selected as a field to match on when importing.
(I'm fairly certain this used to work, but don't know when it stopped.)
Reproduction steps
----------------------------------------
1. Click on **Contacts -> Import Contacts**.
1. Upload a CSV with a 'contact id' column
1. Note that there is no option to select 'Contact ID' in the field mapping options.
Current behaviour
----------------------------------------
Can't match to existing contact using contact id.
Expected behaviour
----------------------------------------
Can match to existing contact using contact id.
Environment information
----------------------------------------
* __CiviCRM:__ master_ <!-- If this problem relates to an upgrade, then specify both old and new versions -->
Comments
----------------------------------------