Development issueshttps://lab.civicrm.org/groups/dev/-/issues2023-09-26T12:18:30Zhttps://lab.civicrm.org/dev/core/-/issues/4614Search Displays: Let booleans be displayed as "instant-save" checkboxes2023-09-26T12:18:30ZnoahSearch Displays: Let booleans be displayed as "instant-save" checkboxesIn SearchKit, turning on "inline edit" for a boolean field results in a UI that involves three clicks to toggle the field value:
1. click the field to enter edit mode
2. click "Yes" or "No"
3. click the submit button
A much smoother UI...In SearchKit, turning on "inline edit" for a boolean field results in a UI that involves three clicks to toggle the field value:
1. click the field to enter edit mode
2. click "Yes" or "No"
3. click the submit button
A much smoother UI (one click instead of three) would be a single checkbox that saves immediately when toggled.https://lab.civicrm.org/dev/core/-/issues/4576Proposal: Unique Identifier/Field references for input fields in Form Builder2023-09-14T13:23:58Zsimon.hermannProposal: Unique Identifier/Field references for input fields in Form Builder## Overview
We propose to add a way to reference fields via an unique identifier in order to use the field value elsewhere on the field, e.g. the success page proposed in issue #4569 or on another page of a multi-page form.
Proposed use...## Overview
We propose to add a way to reference fields via an unique identifier in order to use the field value elsewhere on the field, e.g. the success page proposed in issue #4569 or on another page of a multi-page form.
Proposed uses of this feature are:
* setting the value of another field
* setting input fields in a form processor
The field reference should be automatically generated when creating a new field but be editable, if needed.
## Use case
Donation form:
* set default value for account holder based on field values “first name”, “last name”
* show chosen amount of donation on the next page
* use “first name” and “last name” as well as “donation amount” for success page to confirm donation and say thank you
## Current behavior
Not implemented
## Proposed behavior
Every field has an auto-generated field reference. Using the field reference, the field value can be shown on the form.https://lab.civicrm.org/dev/core/-/issues/4574Proposal: Suggestions for multi-page forms with Form Builder2023-09-14T12:59:56Zsimon.hermannProposal: Suggestions for multi-page forms with Form Builder## Overview
We are happy to see that there is a plan to implement multi-page forms. To enable the users to navigate the form efficiently, it would be great if the following two features are included
- Users should be allowed to navigat...## Overview
We are happy to see that there is a plan to implement multi-page forms. To enable the users to navigate the form efficiently, it would be great if the following two features are included
- Users should be allowed to navigate between pages, even if not all required fields on the current page are filled. This allows users to get an overview of the overall (multi-page) form.
- There should be an option to move to the next or previous page, but also an additonal menu that allows the users to move directly to a specific page of the form. It would also be great if the names of the pages could be set.
- (Optional) Have a representation of the current progress of the form. This should show on which page of the current form I am, in e.g. page 1 of 4 pages (maybe as a progress indicator as welll).
## Example use-case
1. Have additional buttons that allow moving forward or backward on the form
2. Have an option to add a navigation menu to a multi-page form where the names of the pages can be set as well.
3. Have an option for multi-page form, to add a progress indicator, with options to either show the current page number compared to the overall number of pages or similar to a graphical progress bar.
## Current behaviour
The option for multi page forms is not implemented yet.
## Proposed behaviour
Be able to separate the form into multiple pages. Have buttons to navigate to the next or previous page, even if not all required fields are filled and have an additional menu, which allows the user to navigate to one specific page. Have an option to add a progress indicator.https://lab.civicrm.org/dev/core/-/issues/4573Proposal: Add preliminary submission to the Form Builder2024-03-12T13:28:32Zsimon.hermannProposal: Add preliminary submission to the Form Builder## Overview
For longer and more complex forms, users may want to extend the process of filling out a submission form across several browser sessions. Therefore, it would be great to be able to submit/save a partially completed form as a...## Overview
For longer and more complex forms, users may want to extend the process of filling out a submission form across several browser sessions. Therefore, it would be great to be able to submit/save a partially completed form as a preliminary submission and finalise the data later. Complex forms, such as grant applications, could be started and then completed when all the required information is collected. We would suggest adding a "Continue Later" or "Preliminary Submission" button to allow submissions even if not all required fields are filled in.
There would be 2 options for handling the pre-submitted data.
Option 1: The data will be saved in a submission log, but not yet processed in CiviCRM. Using a unique identifier for the submission entry as an URL parameter will load the data from the submission log into the form.
Option 2: Data is processed and stored directly in CiviCRM, meaning contacts, cases or other entities are already created or updated. The form can be edited later using URL parameters and retrieval of defaults.
The user receives a personalised link that allows them to continue with the form. This means that fields such as an email address must be marked as 'required for preliminary submission'. We suggest that this additional option for any input field is only displayed if a "continue later" button is added to the form itself.
At our CiviSprint in Zeitz, Germany, there was a preference for the second option by clients who wanted to use this functionality for their grant application process. It is helpful for clients to be able to see a partially submitted application in order to support the applicant during the application process.
## Example use-case
1. Add an additional button called 'Resume later'.
2. This adds the option on each field to be 'required for preliminary submission'.
## Current behaviour
- When a form is submitted, all required fields have to be filled.
## Proposed behaviour
- There is a new button that allows a (preliminary) submission even if not all required fields are filled.
- Fields can be marked as be required when the new button is triggered.
- A personalised link is sent to the user, which allows to reopen the form with the pre-filled data.https://lab.civicrm.org/dev/core/-/issues/4572Proposal: Retrieval of Defaults w/ Form Builder and Form Processor2023-09-18T12:24:16ZMariaVProposal: Retrieval of Defaults w/ Form Builder and Form ProcessorOverview
----------------------------------------
At the CiviSprint in Germany we found out that Submission forms (Form Builder) currently do not support use cases such as self-service forms that allow users to check and change their exi...Overview
----------------------------------------
At the CiviSprint in Germany we found out that Submission forms (Form Builder) currently do not support use cases such as self-service forms that allow users to check and change their existing contact data.
A lot of Wordpress projects use CalderaForms together with Form Processor to transfer the data to CiviCRM. In CalderaForms there is an option to activate Retrieval of Defaults:
![grafik](/uploads/912921154b32fbf34052c77dcf79b597/grafik.png)
As well as in Form Processor:
![grafik](/uploads/8802a13e8aa75798be4a25bfa9126fcf/grafik.png)
Caldera Forms allows passing URL parameters to the form processor – e.g. a contact CiviCRM ID along with a checksum to prove that the user is allowed to see and edit data for this particular contact, as a way of authentication without requiring a login. These URLs look as follows:
_https//link.org/example-page/cid={contact.contact_id}&{contact.checksum}_
With this information the form processor can retrieve data from within CiviCRM when the form is loaded and prefill fields in the form.
What works in the Form Builder already?
----------------------------------------
- Adding Form Processor as an entity
- Using Form Processor fields for a form
- Submitting/Creating data entered in submission form
- Using the entity id as an URL parameter (requires login)
What is missing?
----------------------------------------
- Option to enable retrieval of defaults (like in CalderaForms) to fill forms with existing data.
- Forms that work for anonymous users, using a checksum for authentication
Comments
----------------------------------------
If you need any further information, please let me know.
I could assist in setting up an example in a test environment since I am not able to implement this function myself.https://lab.civicrm.org/dev/core/-/issues/4498Non-administrators can't select mailing groups on 5.642023-09-02T05:11:56ZJonGoldNon-administrators can't select mailing groups on 5.64Non-administrators get a "loading failed" when attempting to use the new mailing widgets in 5.64. Network tab shows a 403 error, backtrace is below.
As a secondary issue - I assume that the `{error_id}` in the first line is a token tha...Non-administrators get a "loading failed" when attempting to use the new mailing widgets in 5.64. Network tab shows a 403 error, backtrace is below.
As a secondary issue - I assume that the `{error_id}` in the first line is a token that's not resolving.
You can replicate this by removing "Administer CiviCRM" from a demo site's `civicrm_webtest_user` role, since they have most other permissions. Create a new user, assign it that role.
```
Aug 14 13:52:03 [debug] AJAX Error ({error_id}): failed with exception
Array
(
[error_id] => 4tEY-xebR-Pf4L
[exception] => Civi\API\Exception\UnauthorizedException: "Authorization failed"
#0 /var/www/mysite.org/web/sites/all/modules/civicrm/Civi/API/Kernel.php(149): Civi\API\Kernel->authorize()
#1 /var/www/mysite.org/web/sites/all/modules/civicrm/Civi/Api4/Generic/AbstractAction.php(249): Civi\API\Kernel->runRequest()
#2 /var/www/mysite.org/web/sites/all/modules/civicrm/api/api.php(85): Civi\Api4\Generic\AbstractAction->execute()
#3 /var/www/mysite.org/web/sites/all/modules/civicrm/ext/search_kit/Civi/Api4/Action/SearchDisplay/Run.php(107): civicrm_api4()
#4 /var/www/mysite.org/web/sites/all/modules/civicrm/ext/search_kit/Civi/Api4/Action/SearchDisplay/AbstractRunAction.php(107): Civi\Api4\Action\SearchDisplay\Run->processResult()
#5 /var/www/mysite.org/web/sites/all/modules/civicrm/Civi/Api4/Provider/ActionObjectProvider.php(72): Civi\Api4\Action\SearchDisplay\AbstractRunAction->_run()
#6 /var/www/mysite.org/web/sites/all/modules/civicrm/Civi/API/Kernel.php(158): Civi\Api4\Provider\ActionObjectProvider->invoke()
#7 /var/www/mysite.org/web/sites/all/modules/civicrm/Civi/Api4/Generic/AbstractAction.php(249): Civi\API\Kernel->runRequest()
#8 /var/www/mysite.org/web/sites/all/modules/civicrm/ext/search_kit/Civi/Api4/Action/SearchDisplay/AbstractRunAction.php(83): Civi\Api4\Generic\AbstractAction->execute()
#9 /var/www/mysite.org/web/sites/all/modules/civicrm/Civi/Api4/Generic/AutocompleteAction.php(183): Civi\Api4\Action\SearchDisplay\AbstractRunAction->execute()
#10 /var/www/mysite.org/web/sites/all/modules/civicrm/Civi/Api4/Provider/ActionObjectProvider.php(72): Civi\Api4\Generic\AutocompleteAction->_run()
#11 /var/www/mysite.org/web/sites/all/modules/civicrm/Civi/API/Kernel.php(158): Civi\Api4\Provider\ActionObjectProvider->invoke()
#12 /var/www/mysite.org/web/sites/all/modules/civicrm/Civi/Api4/Generic/AbstractAction.php(249): Civi\API\Kernel->runRequest()
#13 /var/www/mysite.org/web/sites/all/modules/civicrm/api/api.php(85): Civi\Api4\Generic\AbstractAction->execute()
#14 /var/www/mysite.org/web/sites/all/modules/civicrm/CRM/Api4/Page/AJAX.php(116): civicrm_api4()
#15 /var/www/mysite.org/web/sites/all/modules/civicrm/CRM/Api4/Page/AJAX.php(55): CRM_Api4_Page_AJAX->execute()
#16 /var/www/mysite.org/web/sites/all/modules/civicrm/CRM/Core/Invoke.php(319): CRM_Api4_Page_AJAX->run()
#17 /var/www/mysite.org/web/sites/all/modules/civicrm/CRM/Core/Invoke.php(69): CRM_Core_Invoke::runItem()
#18 /var/www/mysite.org/web/sites/all/modules/civicrm/CRM/Core/Invoke.php(36): CRM_Core_Invoke::_invoke()
#19 /var/www/mysite.org/web/sites/all/modules/civicrm/drupal/civicrm.module(471): CRM_Core_Invoke::invoke()
#20 /var/www/mysite.org/web/includes/menu.inc(527): civicrm_invoke()
#21 /var/www/mysite.org/web/index.php(21): menu_execute_active_handler()
#22 {main}
```5.64.1https://lab.civicrm.org/dev/core/-/issues/4442Add action to convert smart group to regular group2023-08-31T16:33:25ZyashodhaAdd action to convert smart group to regular groupThere are cases when smart groups have actually run their course. The list is not going to change after a while so no need keeping the group smart esp when the criteria is quite complex and increases the load time. The idea here is keep ...There are cases when smart groups have actually run their course. The list is not going to change after a while so no need keeping the group smart esp when the criteria is quite complex and increases the load time. The idea here is keep only those groups that are needed as smart/dynamic so that it prevents unnecessary process of re-calculating the groups that just shouldn't be if deletion of the group is not an option.
In such cases, provide the ability to convert smart group to regular group.
Proposal : Provide action link for smart groups to convert to regular group.
- refresh the smart group
- move from the group contact cache to group contacts
- unset the saved search on the groupyashodhayashodhahttps://lab.civicrm.org/dev/core/-/issues/4377Don't allow scheduled reminders for events to also include groups2023-09-02T05:11:52ZlarsssandergreenDon't allow scheduled reminders for events to also include groupsEdit: See discussion below, this doesn't actually work anyways.
If a user is able to edit events and has access to a particular group, they can add that group to a scheduled reminder for an event with Also include. It doesn't seem like ...Edit: See discussion below, this doesn't actually work anyways.
If a user is able to edit events and has access to a particular group, they can add that group to a scheduled reminder for an event with Also include. It doesn't seem like a good idea to let users scheduled a reminder to potentially a very large number of contacts via an event scheduled reminder, which is intended to email participants for an event.
I can see the case for also including manually selected contacts, but also including groups seems dangerous and an invitation for mistakes. If a user needs to send a scheduled reminder to a group, they would probably be better off using a mailing or a separate scheduled reminder.
Proposal: Disable the Also include > Select Group option in scheduled reminders for events.https://lab.civicrm.org/dev/core/-/issues/4364Afform: Adding forms to menu is not compatible with Customize Navigation Menu2023-10-19T23:44:23ZlarsssandergreenAfform: Adding forms to menu is not compatible with Customize Navigation MenuIf you add a menu item for a form directly in the form, it shows up sort of where you want it (though the interface to set the order is pretty unhelpful, because you basically are guessing what the weight of existing items in the menu mi...If you add a menu item for a form directly in the form, it shows up sort of where you want it (though the interface to set the order is pretty unhelpful, because you basically are guessing what the weight of existing items in the menu might be). However, if you later go to Customize Navigation Menu, you can move the menu item you created around and it looks like it works and it will work for a while, but then later, it will move back to the location and weight set in the form.
This is confusing for users and frustrating if you don't know what's going on. Seems like we need to have just one way to edit the menu. Maybe it makes sense to simply remove the add to menu option from forms and let users add the menu item manually? Or alternately, we need a way for the menu location and weight to only be used on inserting the menu item and to be uneditable in the form afterwards, maybe with a help text that tells you to edit this directly in the menu.https://lab.civicrm.org/dev/core/-/issues/4354Activities created via API should notify Assignees2023-08-03T12:38:52Zwil_SRQActivities created via API should notify AssigneesOverview
----------------------------------------
If the GUI (https://cc.unidosnow.org/civicrm/activity?action=add) would notify assignees when an activity is created. It'd be useful for the API to do so too. The GUI respects the setting...Overview
----------------------------------------
If the GUI (https://cc.unidosnow.org/civicrm/activity?action=add) would notify assignees when an activity is created. It'd be useful for the API to do so too. The GUI respects the setting in Administer > Customize Data and Screens > Display Preferences > Notify Activity Assignees and notification rules by Activity type. It'd be useful for the API to do so too.
Example use-case
----------------------------------------
1. Invoke civicrm_api3('Activity', 'create', []) or civicrm_api4('Activity', 'create', [])
2. Include assignees
Current behaviour
----------------------------------------
The Activity is created but the assignees are not notified, even in situations where creating the equivalent activity via the GUI would have issued notifications.
Proposed behaviour
----------------------------------------
Notify assignees using the same rules and notification format as the GUI.
Comments
----------------------------------------
See https://civicrm.stackexchange.com/q/45078/5446
Workaround is to call CRM_Activity_BAO_Activity::sendToAssignee() separatelyhttps://lab.civicrm.org/dev/core/-/issues/4349Migrate "Edit Profile" popup to SearchKit/FormBuilder, kill BackBone2023-06-09T09:14:03ZcolemanwMigrate "Edit Profile" popup to SearchKit/FormBuilder, kill BackBoneCiviCRM includes an entire javascript framework stack, Backbone + Marionette, and only uses it to do one thing: the "Edit Profile" popup.
It wouldn't be quite the same, but I think we could make something equivalent *enough* using Search...CiviCRM includes an entire javascript framework stack, Backbone + Marionette, and only uses it to do one thing: the "Edit Profile" popup.
It wouldn't be quite the same, but I think we could make something equivalent *enough* using SearchKit and Afform and kill off Backbone once and for all.https://lab.civicrm.org/dev/core/-/issues/4334Remove Event Info link from Manage Event (and same from Manage Contribution P...2023-06-21T14:55:10ZlarsssandergreenRemove Event Info link from Manage Event (and same from Manage Contribution Page)![image](/uploads/dd822bae24450b2001a604f392198a89/image.png)
I'm not sure this is needed, since it just replicates the link available in the Event Links above, except less completely, as it doesn't include Online Registration, so users...![image](/uploads/dd822bae24450b2001a604f392198a89/image.png)
I'm not sure this is needed, since it just replicates the link available in the Event Links above, except less completely, as it doesn't include Online Registration, so users tend to think this is the only option (some organizations do not use Event Info, preferring to link straight to Registration).
There is a similar thing at the bottom of the first Manage Contribution Page tab that could also be removed, in my opinion. These look kind of weird and aren't really necessary, so I think they can go without losing anything, but am looking for further opinions on this.https://lab.civicrm.org/dev/core/-/issues/4273Allow double opt-in email (and other emails that you don't want a reply from)...2023-07-05T23:48:40ZJamie Novick - CompucoAllow double opt-in email (and other emails that you don't want a reply from) to use a user configured "mail from" address**How it works currently:**
Currently CiviCRM forces the double opt in email to an email address which is:
the words "do not reply" + the domain of the site from your default mail account configured here: https://dmaster.demo.civicrm.o...**How it works currently:**
Currently CiviCRM forces the double opt in email to an email address which is:
the words "do not reply" + the domain of the site from your default mail account configured here: https://dmaster.demo.civicrm.org/civicrm/admin/mailSettings?reset=1
https://github.com/civicrm/civicrm-core/blob/6f14847526226279076f63cc472afc18f2ce27e6/CRM/Core/BAO/Domain.php#L364
**What is the issue?**
The problem with this is that the default mail account email domain (used for bounce handling), may not be the same domain that you want to use as the from address for your double opt in emails (or for any other email that you don't want a reply from).
**Proposed solution**
We already have a place to configure the available from addresses in the system. This is the option group here: https://dmaster.demo.civicrm.org/civicrm/admin/options/from_email_address?reset=1
As such I would suggest that:
1. We create a new setting on the CiviMail component settings (https://dmaster.demo.civicrm.org/civicrm/admin/setting/preferences/mailing?reset=1) (suggest this is below "Enable Double Opt-in for Profiles which use the "Add to Group" setting":
- called "Email From Address to use where a reply is not expected".
- Field type - single select,
- Options to show from available "Email From Addresses" here: https://dmaster.demo.civicrm.org/civicrm/admin/options/from_email_address?reset=1.
- Not required
- Help: "Specify an Email From Address to use when the system sends an email but a reply is not expected, for example when a user is sent an email for a double opt-in. Leaving this blank will use the default which will be do-not-reply@default_domain where the default_domain will be the email domain address of your default mailing account also used for bounce handling. You can add additional Email From Addresses here [link to admin/options/from_email_address?reset=1].
2. If an email address is specified in the setting above it should be used instead of the current hardcoded default value.
**Next steps**
We're happy to submit a PR for this so if we can get this concept approved we will submit a core PR asap.
Thankshttps://lab.civicrm.org/dev/core/-/issues/4262Auto detect line endings Deprecated2023-05-02T21:57:19ZTony Maynard-SmithAuto detect line endings DeprecatedThe civicrm.settings.php file, about line 581, tries to set the PHP ini function auto_detect_line_endings, which is now Deprecated (in PHP 8.1) and no longer required.
Remove this from the settings file.
(This is on my v5.60.0 system...The civicrm.settings.php file, about line 581, tries to set the PHP ini function auto_detect_line_endings, which is now Deprecated (in PHP 8.1) and no longer required.
Remove this from the settings file.
(This is on my v5.60.0 system, but this has been upgraded from earlier versions. Even if fixed in new installs, an upgrade should now fix it.)5.62.0https://lab.civicrm.org/dev/core/-/issues/4245FormBuilder/SearchKit: add 'enabled' field2023-04-20T06:55:21Zaydunsaidan.saunders@squiffle.ukFormBuilder/SearchKit: add 'enabled' fieldOverview
----------------------------------------
Suggestion: add an 'enabled' field for both Searches and Forms (maybe Segments and related entities as well).
Why?: quite often I clone something, make some changes, switch to that - but...Overview
----------------------------------------
Suggestion: add an 'enabled' field for both Searches and Forms (maybe Segments and related entities as well).
Why?: quite often I clone something, make some changes, switch to that - but don't want to delete the original yet until I'm sure I don't want to revert to it. Marking it as 'not enabled' helps search & form management particularly as the numbers increase.https://lab.civicrm.org/dev/core/-/issues/4149User editable Message Templates, the Contact Action: Send an Email and the Co...2023-03-29T21:09:22Zjustinfreeman (Agileware)User editable Message Templates, the Contact Action: Send an Email and the Contact Action: Print Merge/Document cannot use any Smarty Tokens with the crmDate function because CKEditor 4 unnecessarily HTML encodes single quotes (') and double quotes (")User editable Message Templates, the Contact Action: Send an Email and the Contact Action: Print Merge/Document cannot use any Smarty Tokens with the crmDate function because CKEditor 4 unnecessarily HTML encodes single quotes (') and do...User editable Message Templates, the Contact Action: Send an Email and the Contact Action: Print Merge/Document cannot use any Smarty Tokens with the crmDate function because CKEditor 4 unnecessarily HTML encodes single quotes (') and double quotes (").
So if you use a Smarty Token like:
```{contribution.receive_date|crmDate:"%E%f %B %Y"}```
```{contribution.receive_date|crmDate:'%E%f %B %Y'}```
When CKEditor 4 parses the HTML it converts this into:
```{contribution.receive_date|crmDate:"%E%f %B %Y"}```
```{contribution.receive_date|crmDate:'%E%f %B %Y'}```
Effectively rendering the crmDate function unusable in these situations.
Using CKEditor 5 does solve this specific problem, however CKEditor 5 introduces new problems such as the inability for users to resize images, loss of some formatting controls and importantly prevents the ability to view/edit the source HTML.
Agileware Ref: CIVICRM-21035.61.0https://lab.civicrm.org/dev/core/-/issues/4146Upgrade Smarty to Smarty3+2024-01-03T19:48:39ZeileenUpgrade Smarty to Smarty3+This is a meta issue for looking at moving from Smarty2 to Smarty3+
**Status as of 5.68 - Smarty3 is optional, It is recommended in 5.69 as the preferred version**
to enable - add a define to `civicrm_settings.php` like this (adjust...This is a meta issue for looking at moving from Smarty2 to Smarty3+
**Status as of 5.68 - Smarty3 is optional, It is recommended in 5.69 as the preferred version**
to enable - add a define to `civicrm_settings.php` like this (adjust sample path as desired)
```
if (!defined('CIVICRM_SMARTY3_AUTOLOAD_PATH')) {
define('CIVICRM_SMARTY3_AUTOLOAD_PATH', $civicrm_root . '/packages/smarty3/vendor/autoload.php');
}
```
**Gaps as of 5.68**
- [x] resolve Sections section of civi-report https://github.com/civicrm/civicrm-core/pull/27777
- [x] the online membership receipt has an instance of 'crmMoney maths' - this should be the same as other templates - I'm just struggling to understand the double contribution thing with separate membership payment
- [ ] not all extensions are compatible - see below
**Things that cause compatibility issues** (resolve in core with the exception of the 2 listed)
- **maths** in Smartyv2 {$amount+$taxAmount|crmMoney} is implicitly `{($amount+$taxAmount)|crmMoney}` - whereas in Smarty3 it does not resolve correctly without the braces - which Smarty2 does not support. Generally the maths can be moved to the php layer
- Extensions with CiviX versions lower than 23.01 - mostly these only cause notices & smarty files will not load - although some very old civix versions will cause a hard fail. Running civix will remove these lines
![image](/uploads/6ec4b8e83ac6fa02c1ec6d0bec57bac8/image.png)
And add in the smarty mixin if smarty files are in use
- Accessing the smarty property `_tplVars` directly (in most cases running civix will fix)
- usage of `{php}` within tpl files
- incorrect variable assignments in templates for correct quotes and backticks, e.g.
```
{assign var="foo" value="bar-`$something.else`"}
```
if a variable is being concatenated into a string in a template, the string should be surrounded by double quotes, not single. If the variable name contains characters other than a-zA-Z0-9_ (e.g. a period), then it needs to be surrounded by backticks. If a variable is an int and you're doing math, don't use backticks (e.g. `value=$one.thing+1`).
**Why smarty3**
1. Accessing documentation is less confusing as Smarty documentation targets v3+
2. Smarty3 is more secure - in part because it has security releases, in part because escaping is on by default - which actually still seems disabled in the working version - but it makes sense to upgrade smarty first
3. Smarty3 is more efficient in handling one-off strings. We have kinda hacked this into Smartyv2 with a combination of the core function `CRM_Utils_String::parseOneOffStringThroughSmarty` and hacking some security into `fetch` but under volume this can result in #4143 and also reads & writes to disk more than is ideal
4. The work to get to Smarty4 is the same as the work to get to Smarty3 (mostly done) + an unknown extra amount of work. Not a bad thing to do but a big scope creep
5. In theory `Smarty3` is a relatively [seemless upgrade from Smartyv2](https://github.com/smarty-php/smarty/blob/v3.1.47/SMARTY_2_BC_NOTES.txt).
**Things that need to be done in extensions**
Other than the first item these are fairly uncommong
1) run `civix upgrade` on extensions where [the civix version of the extension is 23.01 or lower](https://github.com/civicrm/civicrm-core/pull/27565#issuecomment-1732155043). (it's worth making sure you have the [latest civix first](https://github.com/totten/civix/tree/master)
4) test any unusual smarty screens
**Challenges outstanding**
1) When we are happy from a QA point ov view we need to roll it out as part of our package. At that point we will need to decide if it is opt in & what that looks like or whether it is a hard-switch.
** Challenges that we are doing OK on **
- **Autoloading** - works OK if extensions are updated to recent civix.
- **Function Naming** - Smarty3 has renamed a bunch of functions - eg `register_resource` becomes `registerResource`. This seems to be manageable with [a compatibility class](https://github.com/civicrm/civicrm-core/pull/27585) - allowing us to switch to the forward compatible function names. Note it really is mostly (maybe only) core that interacts with these functions
- **Fetch override** - We have overridden the `fetch` function to try to mimic Smartyv3 security in v2 - this function has a different signature in `v3` so overriding it is a noisy experience. [Solution is to move our patches to Smartyv2 packages](https://github.com/civicrm/civicrm-core/pull/27588)
- **Escape by default** - so far this seems to have not been enabled on Smarty3 - I think it's fine to defer worrying about it until the upgrade is done if it is not hurting us
- **Invalid Smarty Syntax** - so far I have identified that the use of '`' in Smarty causes a hard crash - we don't do this much, mostly in older code... @larssg has done a [pretty solid clean up in core](https://github.com/civicrm/civicrm-core/pull/27547) but there could be some in extensions
- **Direct access of class properties** - we have a couple of places that do things like `$smarty->_tpl_vars` - those need replacing with `get_template_vars or, if we add compatibility `getTemplateVars`
- **Register String Resource** - this is not an issue for `v3` but is the current blocker on `v4` the parameter type in our function `civicrm_smarty_register_string_resource` is an `array` but that is not v4-compatible
**Related**
https://lab.civicrm.org/dev/core/-/issues/4618https://lab.civicrm.org/dev/core/-/issues/4128Custom tokens not working in CiviMail2024-01-29T10:04:07ZmartyCustom tokens not working in CiviMailOverview
----------------------------------------
Custom tokens are not evaluated during CiviMail mailings.
Reproduction steps
----------------------------------------
1. Add example code from [Defining tokens documentation](https://doc...Overview
----------------------------------------
Custom tokens are not evaluated during CiviMail mailings.
Reproduction steps
----------------------------------------
1. Add example code from [Defining tokens documentation](https://docs.civicrm.org/dev/en/latest/framework/token/#defining-tokens) to define two custom tokens: `{profile.viewUrl}` and `{profile.viewLink}`. Cleanup caches.
2. Create a CiviMail mailing and insert the new tokens then submit the mailing.
Current behaviour
----------------------------------------
Resulting email sent by the background mailing job does not include either of the custom tokens. Emails containing the custom tokens are evaluated properly when sent as a CiviMail test or via a send email action for a contact.
Expected behaviour
----------------------------------------
CiviMail mailings must evaluate custom tokens.
Environment information
----------------------------------------
* __CiviCRM:__ 5.58.0
* __PHP:__ 7.4
* __CMS:__ Both WordPress 6.1.1 and Drupal 7.94
* __Database:__ MySQL 5.7
* __Web Server:__ Apachehttps://lab.civicrm.org/dev/core/-/issues/4068Running a Contact Summary report as a limited access user granted gives DB Er...2023-03-18T05:12:26ZtiminaustRunning a Contact Summary report as a limited access user granted gives DB Error: no such fieldOverview
----------------------------------------
Running a report as a limited access user based on a group that has ACL permissions granted gives ...
```
Sorry, due to an error, we are unable (etc)... DB Error: no such field
```
My par...Overview
----------------------------------------
Running a report as a limited access user based on a group that has ACL permissions granted gives ...
```
Sorry, due to an error, we are unable (etc)... DB Error: no such field
```
My particular use case was a Contact Summary report based on a smartgroup being run by a locked-down user who has acl access to that group (ie not all groups).
Reproduction steps (simplified from my actual use case).
----------------------------------------
1. Create a smart-group and set up ACL for a specific user that doesn't have global access to all contacts/groups.
2. Login as that user, create the report, select a couple of fields AND the smart-group.
3. Attempt to run the report.
Current behaviour
----------------------------------------
Report fails with DB Error: no such field as limited access user, but runs fine for full access user.
The following SQL was generated for the limited access user ...
```
SELECT SQL_CALC_FOUND_ROWS contact_civireport.sort_name AS civicrm_contact_sort_name,
contact_civireport.id AS civicrm_contact_id,
email_civireport.email AS civicrm_email_email,
(address_civireport.street_number % 2) AS civicrm_address_address_odd_street_number,
address_civireport.postal_code AS civicrm_address_address_postal_code
FROM civicrm_contact contact_civireport
LEFT JOIN civicrm_address address_civireport ON
(contact_civireport.id = address_civireport.contact_id)
AND address_civireport.is_primary = 1
LEFT JOIN civicrm_email email_civireport ON
contact_civireport.id = email_civireport.contact_id
AND email_civireport.is_primary = 1
WHERE ((contact_civireport.contact_type IN ('Individual')))
AND contact_civireport.id IN
(SELECT DISTINCT cgroup_civireport.contact_id
FROM civicrm_group_contact cgroup_civireport
WHERE cgroup_civireport.group_id IN (11)
AND cgroup_civireport.status = 'Added'
UNION DISTINCT SELECT DISTINCT smartgroup_contact.contact_id
FROM civicrm_group_contact_cache smartgroup_contact
WHERE smartgroup_contact.group_id IN (11) )
AND (`contact_civireport`.`id` IS NULL
OR (`contact_civireport`.`id` IN
(SELECT contact_id FROM civicrm_acl_contact_cache WHERE user_id = 6)))
AND (`contact_civireport`.`is_deleted` IS NULL
OR (`contact_civireport`.`is_deleted` != 1))
AND (`cgroup_civireport`.`id` IS NULL
OR (`cgroup_civireport`.`id` IN (3, 7, 9, 11)))
ORDER BY contact_civireport.sort_name ASC
LIMIT 0, 50
```
The error is caused by the last where clause which is generated from buildPermissionClause(), which is trying to access 'contact_civireport' that only exists in the nested where/filter subquery.
The last 3 where clauses are not present for the full access user (ie buildPermissionsClause() return empty string).
Expected behaviour
----------------------------------------
The report should run correctly without the last where clause for limited access use.
Environment information
----------------------------------------
* __Browser: Firefox 108
* __CiviCRM: 5.57.0
* __PHP: 8.1
* __CMS: WordPress 6.1.1
* __Database: MariaDB 10.5
* __Web Server: Apache 2.4
Comments
----------------------------------------
The problem appears to be in civicrm/CRM/report/form.php.
From what I read ...
The last three where clauses in the above are generated by the function buildPermissionClause() which uses a BAO call to generate a user-specific clause for each table in the query where the user does not have full access.
This relies on function selectedTables() to generate the list of tables. The last part of selectedTables function searches all the filters an adds them to the list of tables, however for filters using $filterop 'in' or 'notin', the tables referenced by these clauses may not exist in the main query. It would seem likely that the selected tables list should not include tables where the filter uses these two constructs.
I am not sure apart from 'group' filter (which has specific handling), if any report filters would use 'in/not in (select...)' rather than 'in/out (value list)' so I am not sure the best way to fix this as I have only investigated my usage.
The solution may be to explicitly excluding contact group table (cgroup_civireport), or by updating to selecttables to exclude all 'in' and 'notin' filters as there is no guarantee the table exists in the main query for these types of filters.
Change in function selecttables (line ~4388) with
```
$this->_selectedTables[] = $tableName;"
```
to:
```
if ( ! in_array($filterOp, ["in","notin"] )) $this->_selectedTables[] = $tableName;
```
If the intent is to restrict access to acl enabled groups this would need the same clause added in function joinGroupTempTable(). There may be other places where this should apply.
(On a side issue as I am new here - in my sql query, civicrm_address_address_odd_street_number is not a column that is selectable - it seems to be preset as required and hidden for some reason and limited to a 'Walking Report' template. Having this would cause an unnecessary join to the address table if no address fields are being reported - only a little performance thing but should this be another problem or improvement request?).5.59.0https://lab.civicrm.org/dev/core/-/issues/4053Standalone: users and roles2023-08-11T16:23:42ZbgmStandalone: users and rolesGeneral issue to discuss user management, roles and permissions.
There was a discussion in Manchester about this. At first we explored the idea of using CiviCRM groups to manage permissions, but there was a lot of discomfort because of ...General issue to discuss user management, roles and permissions.
There was a discussion in Manchester about this. At first we explored the idea of using CiviCRM groups to manage permissions, but there was a lot of discomfort because of the (lack of) security around groups and how it could end up adding a lot of extra complexity. Of course, maybe an implement or another might prove one way or another.
So far one [WIP branch](https://github.com/civicrm/civicrm-core/compare/master...demeritcowboy:civicrm-core:user-storage?expand=1) by @DaveD proposes creating `civicrm_user` with an ID, username, password and maybe email. While testing, I managed to get it working by also adding a record in the `civicrm_uf_match` table (for authx http logins), to link that user to a contact.
Presumably we would also have `civicrm_user_role` (ex: admin, staff, member) and `civicrm_user_permission` (ex: "admin" has the "Administer CiviCRM" permission).
And then we would have the same permission grid similar to what CiviCRM has for WordPress role management (in that case, it adds WordPress capabilities, but in this case, it would add records in `civicrm_user_permission`).
cc @DaveD @artfulrobot @JoeMurray
Related meta: #2998