CiviCRM Core issueshttps://lab.civicrm.org/dev/core/-/issues2022-10-21T07:28:15Zhttps://lab.civicrm.org/dev/core/-/issues/3906FormBuilder URL Filter dropdown with multiple values are only filterd with fi...2022-10-21T07:28:15ZdavidFormBuilder URL Filter dropdown with multiple values are only filterd with first valueOverview
----------------------------------------
If a URL filter with multiple values is used in formbuilder, all values are picked up and shown in the multiselect, but the search is only filterd with the first value.
e.g. /civicrm/for...Overview
----------------------------------------
If a URL filter with multiple values is used in formbuilder, all values are picked up and shown in the multiselect, but the search is only filterd with the first value.
e.g. /civicrm/forms/fzt-suche/#?event_type_id=2,3,4
![Unbenannt](/uploads/10a0f5f22f4acd9aaa2a2912743ae71a/Unbenannt.JPG)
Environment information
----------------------------------------
* __Browser:__ Firefox 105
* __CiviCRM:__ 5.54.0
* __PHP:__ 7.3
* __CMS:__ Drupal 7https://lab.civicrm.org/dev/core/-/issues/3907Proposal: When excluding past mailing recipients, exclude past recipient emai...2022-10-21T07:27:20ZlarsssandergreenProposal: When excluding past mailing recipients, exclude past recipient email addresses, not just contactsSometimes, you want to send a few different versions of a mailing, so you segment your list and send different versions to different groups or to search results. Of course, you want to make sure no one receives two different versions of ...Sometimes, you want to send a few different versions of a mailing, so you segment your list and send different versions to different groups or to search results. Of course, you want to make sure no one receives two different versions of the same email, so you make sure to exclude the earlier mailings from the later ones. But if two contacts share an email address, one can receive the first mailing and the other the second mailing. Excluding recipients of the first mailing only excludes contacts who received the first mailing, not email addresses that received the first mailing.
Similarly, you might send a mailing, only to have someone realize a mistake was made when half the emails have been delivered. You cancel the mailing, make a copy and fix the error and then send the mailing to the same group minus the recipients of the first mailing. This doesn't work at all, because the _intended_ recipients of the excluded mailing are excluded, not the _actual_ recipients. But if we get past that problem, we'd also run into the same problem as above.
I propose that to fix this, when the dedupe emails setting is enabled in CiviMail, we do an additional check to see if any of the excluded contacts' primary/bulk email address is also used by a contact in the included list, then exclude those recipients as well. I'll need to look at the details of this to make sure performance isn't an issue. If someone wants multiple emails sent to the same address for multiple contacts, they would have the dedupe setting in Civimail turned off, so this change wouldn't affect them. Otherwise, I think this is line with what users would expect.
Also, when recipients of a mailing are excluded, we should check if the excluded mailing status is cancelled. If it is cancelled, then we should only exclude the actual recipients rather than the intended recipients. I think users would also expect excluding recipients of a mailing would exclude the actual recipients, rather than the intended recipients.https://lab.civicrm.org/dev/core/-/issues/3909No installation of extensions dependencies on upgrade might lead to errors2022-10-21T07:25:55Zsluc23No installation of extensions dependencies on upgrade might lead to errorsOverview
----------------------------------------
Current CiviCRM behavior installs dependencies on extension installation, but it does not on extension upgrade.
Having an installed extension and when in a new release this extension ad...Overview
----------------------------------------
Current CiviCRM behavior installs dependencies on extension installation, but it does not on extension upgrade.
Having an installed extension and when in a new release this extension adds dependencies in `info.xml`, there's might be a problem on the upgrade process (depending on the case)
A clear example is `mosaico` upgrade from 2.8 to 2.9.
Until version 2.8 `mosaico` has no dependencies apart from `flexmailer`, but in 2.9 it added `org.civicrm.search_kit` and `org.civicrm.afform`
Reproduction steps
----------------------------------------
1. Have a CiviCRM installation with `mosaico` 2.8 but not installed `org.civicrm.search_kit` and `org.civicrm.afform`
2. Upgrade `mosaico` codebase to 2.9.
3. CiviCRM will crash on bootstrap because of unmet dependencies
Current behaviour
----------------------------------------
If you don't install the dependencies before upgrading the codebase of the extension, you might have unexpected results (depending where the dependent classes are called, in case of mosaico at bootstrap)
Expected behaviour
----------------------------------------
Not sure what's the best solution.. @totten suggested on a automatic process to check for unmet dependencies and disable this problematic extensions to avoid further errors
Environment information
----------------------------------------
* __CiviCRM:__ _Master/5.51_
* __PHP__:__7.4/8.0__https://lab.civicrm.org/dev/core/-/issues/3910afform: Support url-params and other config options for Values in Submmission...2022-11-03T17:23:57ZAllenShawafform: Support url-params and other config options for Values in Submmission formsPer conversations with @kurund , @totten , and @colemanw at the 2022 Manchester sprint:
Submission forms currently offer configuration of Values as a way to ensure a certain value is submitted for a given property of the submitted entit...Per conversations with @kurund , @totten , and @colemanw at the 2022 Manchester sprint:
Submission forms currently offer configuration of Values as a way to ensure a certain value is submitted for a given property of the submitted entity, without displaying a field to the form user. Currently it's only possible to specify a static value for each Value. This issue covers work to increase this functionality by allowing the specification of more options for each value. As a start, we're aiming to support these: a default value; and a value defined by URL parameter (overriding the default value). Future efforts may allow for defining a mechanism for on-page calculation of the value; specifying that the submitted value should not be applied if updating an existing entity (could be useful e.g. for Source field); and more.
## New attributes:
At the start we aim to support only these:
- `default`: the default value
- `urlparam`: the name of a URL parameter from which the value can be taken (which value, if given, will override the `default` value)
We're also aiming to make decisions now which would facilitate adding other attributes in the future.
## UI changes:
- Move the Values section out of the "pallette" pane in the left, to a new "Values" tab in the "canvas" pane on the right:
![values](/uploads/4a5fb8bd7c29c136ddb18ebc191e7201/values.png)
- For each defined Value, support configuration of various attributes, probably through a pop-up accessed through a gear icon, similar to that already in use under for fields under the Fields tab:
![gear2](/uploads/6c850f34b123efd4651720ff50ef30e1/gear2.png)
Rationale:
- We currently have the "Values" configuration appearing at the top of the left-side "pallette" pane. This leaves limited space for configuration options on each Value, whereas there will be more room for that in the "canvas" pane.
- It would make for a more consisten user experience to treat the left-side "pallette" only as a pool of available entity properties (fields), and the right-side "canvas" as the place for configuring those properties which one has selected for use in the form.
## Form storage/schema changes:
Values are currently stored in the form markup within the `data` attribute of each `<af-entity>` tag; e.g. this line indicates that the do_not_email field should be forced to "yes":
```
<af-entity data="{contact_type: 'Individual', do_not_email: '1'}" type="Contact" name="Individual1" label="Individual 1" actions="{create: true, update: true}" security="RBAC" />
```
To store these additional config options, we make some changes to this schema:
- Values will be stored as an `<af-field>` element within the `<af-entity>` container, so that the above-mentioned `<af-entity>` tag would be represented like so:
```
<af-entity data="{contact_type: 'Individual'}" type="Contact" name="Individual1" label="Individual 1" actions="{create: true, update: true}" security="RBAC">
<af-field name="do_not_email" default="1" />
</af-entity>
```
Rationale:
- The `<af-field>` tag is appropriate here because these "entity properties" are in fact fields -- though they won't be presented as _form_ fields, they will be submitted as _API_ fields (consider the "getFields" api action).
- Storing as attributes of an `<af-field>` tag is an alternative to cramming arrays of increasing complexity into the `data` attribute of the `<af-entity>` tag, and this alternative facilitates easier validation.
- When parsing the stored form markup for display, it should be pretty easy to identify `<af-field>` tags within an `<af-entity>` container, then avoid formatting them as displayed form fields, and ensure they're attributed to the correct entity upon submission.https://lab.civicrm.org/dev/core/-/issues/3911civix upgrade fails on afform extension with mgd mixin incompatible version e...2022-10-21T07:23:20Zaydunsaidan.saunders@squiffle.ukcivix upgrade fails on afform extension with mgd mixin incompatible version errorOn master, go to `civicrm/ext/afform/core` run `civix upgrade`
Error:
```
General upgrade
===============
In Mixlib.php line 147:
Received incompat...On master, go to `civicrm/ext/afform/core` run `civix upgrade`
Error:
```
General upgrade
===============
In Mixlib.php line 147:
Received incompatible version (expected="mgd-php@1.1.0", actual="mgd-php@1.0.0")
```
`info.xml` includes:
```
<mixin>mgd-php@1.1.0</mixin>
```tottentottenhttps://lab.civicrm.org/dev/core/-/issues/3912Recreate core screens as Search Displays2024-03-01T22:32:39ZwmortadaRecreate core screens as Search DisplaysSee also dev/core#3761 for the SK/FB roadmap.
This is a meta issue for the work required to recreate many core screens using SearchKit and FormBuilder.
These new screens are in one of three locations:
1. Directly in core ([list](#core...See also dev/core#3761 for the SK/FB roadmap.
This is a meta issue for the work required to recreate many core screens using SearchKit and FormBuilder.
These new screens are in one of three locations:
1. Directly in core ([list](#core-replacements)): these screens replace the old screens, and often the old code is removed. There is no option to revert to the old versions.
2. In the [CiviCRM Administration UI](https://github.com/civicrm/civicrm-core/tree/master/ext/civicrm_admin_ui) extension ([list](#adminui)): these screens replace the old screens but administrators can either disable the extension, or disable individual screens (by removing the URL in FormBuilder). These will eventually become the standard screens and old code removed - but need a bit more testing and verification.
3. In the [CiviCRM Search UI](https://github.com/civicrm/civicrm-core/tree/master/ext/civicrm_search_ui) extension ([list](#searchui)): these screens don't have the full functionality of the screens they will eventually replace. In some cases, new features in SK are needed before they can fully replace existing screens. They are still useful and are a basis for further development. These may eventually migrate into AdminUI.
We welcome contributions from anyone who wants to get involved with this. If you need any help getting started, ask in the \~AdminUI channel on Mattermost.
It builds on the work started by Coleman to create the [CiviCRM Administration UI](https://github.com/civicrm/civicrm-core/tree/master/ext/civicrm_admin_ui) extension. Many more screens were created during the Manchester Sprint in October 2022.
Notes:
1. Join the \~AdminUI channel on Mattermost.
2. If you want to work on any of these, please add your name to the item and keep it updated as work progresses.
3. Look at the screens that have already been converted and try to follow a similar format.
4. The aim is to include all the functionality of the existing screens but without twisting SK in knots to exactly reproduce an existing screen.
5. Each screen will have at least three files: `.aff.html` and `.aff.json` copied from the `<civicrm.files>/ang` directory and `.mgd.php` exported from API Explorer.
6. If you can't reproduce some aspect of an existing screen, create a WIP (Work in Progress) PR and note it below. Someone else may know how to do that, or it may need a new feature adding to SK.
7. Some people found creating the PR harder than creating the new screen. We don't want that to be a barrier to getting these screens converted so if you don't want to create the PR, just [email me](mailto:aidan.saunders@squiffle.uk?subject=%22AdminUI%22) with the 3 files to and I'll do the PR.
8. There are more screens to do than those below. If you want to do something else, please add it to this page.
## AdminUI pages
| Form | URL | Status | PR | Who | Notes |
|------|-----|--------|----|-----|-------|
| Profiles | `civicrm/admin/uf/group` | Merged | [24715](https://github.com/civicrm/civicrm-core/pull/24725) | Alain | |
| Location Types | `civicrm/admin/locationType` | Merged | [24722](https://github.com/civicrm/civicrm-core/pull/24722) | @wmortada | |
| Financial Types | `civicrm/admin/financial/financialType` | Submitted | [24720](https://github.com/civicrm/civicrm-core/pull/24720) | Nicolas | |
| Financial Accounts | `civicrm/admin/financial/financialAccount` | Merged | [24730](https://github.com/civicrm/civicrm-core/pull/24730) | Dave J | |
| Relationship Types | `/civicrm/admin/reltype` | Merged | [24709](https://github.com/civicrm/civicrm-core/pull/24709) | Damilare | |
| Grant Status Options | `/civicrm/admin/options/grant_status` | Submitted | [24729](https://github.com/civicrm/civicrm-core/pull/24729) | Damilare | |
| Grant Type | `/civicrm/admin/options/grant_type` | Submitted | [24729](https://github.com/civicrm/civicrm-core/pull/24729) | Damilare | |
| Option Values | `civicrm/admin/options` | Blocked | | Patrick | To be used for all option values |
| Import/Export Mappings | civicrm/admin/mapping | Submitted | [28509](https://github.com/civicrm/civicrm-core/pull/28509) | @pradeep | |
| Label Formats | civicrm/admin/labelFormats | Done | [28513](https://github.com/civicrm/civicrm-core/pull/28513) | @dewymercerais | |
| Manage Premiums | civicrm/admin/contribute/managePremiums | Submitted | [28540](https://github.com/civicrm/civicrm-core/pull/28540) | @AllenShaw | |
| Personal Campaign Pages | civicrm/admin/pcp | Done | [29541](https://github.com/civicrm/civicrm-core/pull/29541) | @kcristiano | |
| Print Page (PDF) Formats | civicrm/admin/pdfFormats | done | [28520](https://github.com/civicrm/civicrm-core/pull/28520) | @usha.makoa | |
| Settings - SMS Provider | civicrm/admin/sms/provider | Submitted | [28512](https://github.com/civicrm/civicrm-core/pull/28512) | @pradeep | |
| Event Templates | civicrm/admin/eventTemplate | Submitted | [28514](https://github.com/civicrm/civicrm-core/pull/28514) | @pradeep | |
| Event Name Badge Layouts | civicrm/admin/badgelayout | Done | [28518](https://github.com/civicrm/civicrm-core/pull/28518) | @dewymercerais | |
| Settings - Payment Processor | civicrm/admin/paymentProcessor | done | | | |
| Headers, Footers, and Automated Messages | civicrm/admin/component | | [28484](https://github.com/civicrm/civicrm-core/pull/28484) | @kcristiano @19ATF72 | |
| Message Templates | civicrm/admin/messageTemplates | done | | | |
| Membership Types | civicrm/admin/member/membershipType | WIP | | @ayduns | |
| Membership Status Rules | civicrm/admin/member/membershipStatus | Done | [28480](https://github.com/civicrm/civicrm-core/pull/28480) | @usha.makoa | |
| Campaigns | the search doesn't exist in core (the dashboard does and is already in SearchUI) | abandonned | | | |
| Discounts | civicrm/cividiscount | | | Coleman | [CiviDiscount MR](https://lab.civicrm.org/extensions/cividiscount/-/merge_requests/284 "Convert all tables to SearchKit displays") |
| Dedupe Exceptions | civicrm/dedupe/exception | | | | see [comment](https://github.com/civicrm/civicrm-core/pull/25522#issue-1575266329) |
| Contact Types | civicrm/admin/option/subtype | done | | | |
| Joblogs | civicrm/admin/joblog | Merged | [26732](https://github.com/civicrm/civicrm-core/pull/26732) | @ayduns | |
| Mail Accounts | civicrm/admin/mailSettings | Merged | [26733](https://github.com/civicrm/civicrm-core/pull/26733) | @ayduns | Does not yet support oauth-client method to add mail accounts |
| Participant Status | civicrm/admin/participant_status | | | @ayduns | see [conditional in-place edit](https://lab.civicrm.org/dev/core/-/issues/4406 "SearchKit: make 'inplace edit' conditional") |
| Payment Processor Type | civicrm/admin/paymentProcessorType | | | | Doesn't exist in menus. Don't think this is needed as processor types added by extensions, not UI. |
| Date Preferences | civicrm/admin/setting/preferences/date | | [28483](https://github.com/civicrm/civicrm-core/pull/28483) | @pradpnayak | |
| ACL Assign Users to Roles | civicrm/admin/acl/entityrole | Merged | [26925](https://github.com/civicrm/civicrm-core/pull/26925) | @ayduns | |
More challenging:
| Form | URL | Status | PR | Who | Notes |
|------|-----|--------|----|-----|-------|
| Message Templates | civicrm/admin/messageTemplates | On hold | [24981](https://github.com/civicrm/civicrm-core/pull/24981) | @ayduns | There is a new Message Admin Core extension. See comments on PR |
| Option Groups/Values | civicrm/admin/options | In progress | [25387](https://github.com/civicrm/civicrm-core/pull/25387) | @colemanw | |
| Schedule Reminders | civicrm/admin/scheduleReminders | Merged | [26896](https://github.com/civicrm/civicrm-core/pull/26896) | @colemanw | |
| Settings - Scheduled Jobs | civicrm/admin/job | Merged | [26060](https://github.com/civicrm/civicrm-core/pull/26060) | @ayduns | |
| Price Sets | civicrm/admin/price | WIP | | @ayduns | |
| Price set fields | civicrm/admin/price/field | WIP | | @ayduns | |
| Manage Contribution Pages | civicrm/admin/contribute | Merged | [24937](https://github.com/civicrm/civicrm-core/pull/24937) | @ayduns | |
| Manage Events | civicrm/event/manage | Merged | [28476](https://github.com/civicrm/civicrm-core/pull/28476) | @alainb (Alain Benbassat) | |
| CiviCRM Reports | civicrm/report/list | WIP - need PR (see wip [28511](https://github.com/civicrm/civicrm-core/pull/28511)) to add a pseudo field with components list | [28480](https://github.com/civicrm/civicrm-core/pull/28480) | @usha.makoa | |
| Create New Report from Template | civicrm/admin/report/template/list | | | | The Report Templates Entity lacks in SK |
| Manage Groups | civicrm/group | Merged | [26261](https://github.com/civicrm/civicrm-core/pull/26261) | @samuelsov | |
Other Option Values (when main Option Values is done):
| Form | URL | Status | PR | Who | Notes |
|------|-----|--------|----|-----|-------|
| Accepted Credit Cards Options | civicrm/admin/options/accept_creditcard | | | | |
| Activity Type Options | civicrm/admin/options/activity_type | | | | |
| Addressee Type Options | civicrm/admin/options/addressee | | | | |
| Communication Style Options | civicrm/admin/options/communication_style | | | | |
| Custom Search Options | civicrm/admin/options/custom_search | | | | |
| Email Greeting Type Options | civicrm/admin/options/email_greeting | | | | |
| From Email Address Options | civicrm/admin/options/from_email_address | | | | |
| Gender Options | civicrm/admin/options/gender | | | | |
| Individual contact suffixes Options | civicrm/admin/options/individual_suffix | | | | |
| Instant Messenger (IM) screen-names Options | civicrm/admin/options/instant_messenger_service | | | | |
| Languages Options | civicrm/admin/options/languages | | | | |
| Mobile Phone Providers Options | civicrm/admin/options/mobile_provider | | | | |
| Payment Methods Options | civicrm/admin/options/payment_instrument | | | | |
| Phone Type Options | civicrm/admin/options/phone_type | | | | |
| Postal Greeting Type Options | civicrm/admin/options/postal_greeting | | | | |
| Preferred Communication Method Options | civicrm/admin/options/preferred_communication_method | | | | |
| Soft Credit Types Options | civicrm/admin/options/soft_credit_type | | | | |
| Website Type Options | civicrm/admin/options/website_type | | | | |
| Word Replacements | civicrm/admin/options/wordreplacements | | | | |
## SearchUI
These searches should be placed in the SearchUI extension rather than AdminUI. When the new search fully replaces the functionality of the core page it can override the existing URL. Otherwise, create an alternate URL and place a menu entry under Experimental.
<table>
<tr>
<th>Form</th>
<th>URL</th>
<th>Status</th>
<th>PR</th>
<th>Who</th>
<th>Notes</th>
</tr>
<tr>
<td>Find Contacts</td>
<td>civicrm/contact/search</td>
<td>Merged</td>
<td>
[26381](https://github.com/civicrm/civicrm-core/pull/26381)
</td>
<td>
@samuelsov
</td>
<td>
</td>
</tr>
<tr>
<td>Advanced Search</td>
<td>civicrm/contact/search/advanced</td>
<td>
</td>
<td>
</td>
<td>
</td>
<td>it doesn't make sense since we can create appropriate searches with SK</td>
</tr>
<tr>
<td>Find Contributions</td>
<td>civicrm/contribute/search</td>
<td>Merged, needs more work</td>
<td>
[26746](https://github.com/civicrm/civicrm-core/pull/26746)
</td>
<td>
@ayduns
</td>
<td>
</td>
</tr>
<tr>
<td>Find Mailings</td>
<td>civicrm/mailing civicrm/mailing/browse/scheduled civicrm/mailing/browse/archived civicrm/mailing/browse/unscheduled</td>
<td>WIP</td>
<td>
[26532](https://github.com/civicrm/civicrm-core/pull/26532)
</td>
<td>
@samuelsov
</td>
<td>
</td>
</tr>
<tr>
<td>Find Memberships</td>
<td>civicrm/membership/search</td>
<td>Submitted</td>
<td>
[29064](https://github.com/civicrm/civicrm-core/pull/29064)
</td>
<td>
@jitendra
</td>
<td>
</td>
</tr>
<tr>
<td>Find Participants</td>
<td>civicrm/event/search</td>
<td>Done</td>
<td>
[28477](https://github.com/civicrm/civicrm-core/pull/28477)
</td>
<td>
@breheret
</td>
<td>
it's staying :
- Add the 'fee level' field which I was unable to put
- Add custom group fields linked to an event/participant
- Add actions that are not native to Searchkit:
- Cancel registration
- Email - send now
- Group - add contacts
- Group - create smart group
- Name badges - print
- PDF letter - print for participants
- Participant status - change
- Print selected rows
- Update multiple participants
</td>
</tr>
<tr>
<td>Find Activities</td>
<td>civicrm/activity/search</td>
<td>Done</td>
<td>
[28480](https://github.com/civicrm/civicrm-core/pull/28480)
</td>
<td>
@usha.makoa
</td>
<td>
</td>
</tr>
<tr>
<td>Cases Dashboard</td>
<td>civicrm/case</td>
<td>Done</td>
<td>
</td>
<td>
@ayduns and @simon.hermann
</td>
<td>
https://lab.civicrm.org/extensions/casesummary/-/merge_requests/3
</td>
</tr>
<tr>
<td>Single Case View</td>
<td>civicrm/contact/view/case</td>
<td>WIP</td>
<td>
</td>
<td>
@simon.hermann
</td>
<td>
</td>
</tr>
</table>
## Core replacements
| Form | URL | Status | PR | Who | Notes |
|------|-----|--------|----|-----|-------|
| Contact summary: Notes tab | civicrm/note | Merged | [27610](https://github.com/civicrm/civicrm-core/pull/27610) | @colemanw | |
| Contact summary: Relationships tab | civicrm/contact/view/rel | Merged | [27701](https://github.com/civicrm/civicrm-core/pull/27701) | @colemanw | |
| Campaign Dashboard | civicrm/campaign, civicrm/petition, civicrm/survey | Merged | [27271](https://github.com/civicrm/civicrm-core/pull/27271) | @colemanw | |
| Contact summary: Membership tab | civicrm/ | Merged | [28810](https://github.com/civicrm/civicrm-core/pull/28810) | @jitendra | |
| Contact summary: Event tab | civicrm/ | Submitted | [29570](https://github.com/civicrm/civicrm-core/pull/29570)| @guyiac | |https://lab.civicrm.org/dev/core/-/issues/3913[Improvement] Ability to send only one single email confirmation after sign-u...2022-10-20T07:43:32ZJanOnline[Improvement] Ability to send only one single email confirmation after sign-up to several mailing listsOverview
----------------------------------------
CiviCRM allows people to sign up to several mailing lists / groups at once via `civicrm/mailing/subscribe`. Then they receive numerous confirmation emails to double opt-in. In our experie...Overview
----------------------------------------
CiviCRM allows people to sign up to several mailing lists / groups at once via `civicrm/mailing/subscribe`. Then they receive numerous confirmation emails to double opt-in. In our experience, since those emails all look very similar, many people mistake them for being the same; hence they click and confirm in one but not the others. As a result, you end up with many "pending" sign-ups (we have 10,000+ by now).
Example use-case
----------------------------------------
1. Create several mailing list groups
2. Go to `civicrm/mailing/subscribe` and sign up to more than one list
3. Receive several similar looking confirmation emails
4. Confirm only one and find yourself being subscribed to only one of the list you actually wanted to be on
Proposed behaviour
----------------------------------------
To remedy this situation, CiviCRM could give admins the option of whether CiviCRM should send a single double opt-in email for every sign-up event instead of individual ones per mailing list in order to avoid confusing people when signing up.https://lab.civicrm.org/dev/core/-/issues/3914Missing getRoleNames() method in WordPress System Utility2022-11-01T08:51:34ZBastien HoMissing getRoleNames() method in WordPress System UtilityOverview
----------------------------------------
Some extensions use the `CRM_Core_Config::singleton()->userSystem->getRoleNames()` method, which is not implemented for WordPressOverview
----------------------------------------
Some extensions use the `CRM_Core_Config::singleton()->userSystem->getRoleNames()` method, which is not implemented for WordPresshttps://lab.civicrm.org/dev/core/-/issues/3915Event registration sequence weirdness2022-10-20T07:36:12Zaydunsaidan.saunders@squiffle.ukEvent registration sequence weirdnessDocumenting some discussion at the #ManchesterSprint
We have a weird sequence in the event registration workflow whereby customers pay for the event before the total amount is determined. This causes a variety of problems:
- Customers ...Documenting some discussion at the #ManchesterSprint
We have a weird sequence in the event registration workflow whereby customers pay for the event before the total amount is determined. This causes a variety of problems:
- Customers are asked to pay before making choices which is contrary to almost any other payment workflow and therefore confuses customers.
- Payment processors (eg Stripe) need to jump through hoops to make this work which complicates the code and causes more opportunities for failure.
- Customers may be asked by their bank to confirm a payment of zero which is later updated. Those customers who read the amount assume something is broken.
Changing the registration sequence to put payment at the end would resolve all these problems, and nobody (so far) thinks the current arrangement is desirable.
Fyi @mattwirehttps://lab.civicrm.org/dev/core/-/issues/3916Improve colour contrast for SearchKit buttons when disabled2022-10-14T17:58:49ZwmortadaImprove colour contrast for SearchKit buttons when disabledOverview
----------------------------------------
When a row in a SearchKit display is styled as 'disabled' it changes the text colour to grey. Unfortunately, this also affects the buttons. For the buttons grey doesn't have sufficient co...Overview
----------------------------------------
When a row in a SearchKit display is styled as 'disabled' it changes the text colour to grey. Unfortunately, this also affects the buttons. For the buttons grey doesn't have sufficient colour contrast to the button colour. See screenshot below.
![Screenshot_from_2022-10-14_18-17-41](/uploads/28406fdaf97f3020d19778de1f0c1e4b/Screenshot_from_2022-10-14_18-17-41.png)
I think it would be better to keep the text colour white for buttons and instead change the button colour to a more muted shade.
Reproduction steps
----------------------------------------
1. Create a SearchKit display with buttons
2. Add a conditional row style of 'Disabled' - e.g. if a row is marked as disabled.
Current behaviour
----------------------------------------
The colour contrast is poor and it is difficult to read the text on the buttons.
Expected behaviour
----------------------------------------
There should be sufficient colour contrast that users should be able to read the text on the buttons and it meets accessibility best practice.
Environment information
----------------------------------------
CiviCRM 5.56.alpha1
Comments
----------------------------------------
To fix this properly we should consider all six button styles (e.g. default, warning, success etc.) and all nine row styles (e.g. disabled, strikethrough, italic etc.). These could occur in any combination and all of them should provide sufficient colour contrast.
For context see: https://github.com/civicrm/civicrm-core/pull/24722#discussion_r995974092https://lab.civicrm.org/dev/core/-/issues/3917Incorrect currency displayed for contribution2023-11-23T07:47:00ZKurund JalmiIncorrect currency displayed for contributionCreate a contribution page and set the currency other than default currency.
Confirm and thank you page displays default currency instead of one that's configured.Create a contribution page and set the currency other than default currency.
Confirm and thank you page displays default currency instead of one that's configured.Kurund JalmiKurund Jalmihttps://lab.civicrm.org/dev/core/-/issues/3919User-selected Field Transformations in Search Kit can cause a DB constraint v...2022-10-20T07:30:18ZbrienneUser-selected Field Transformations in Search Kit can cause a DB constraint violation errorOverview
----------------------------------------
Within Search Kit, a user is not always limited to what type of Field Transformation they can use on a particular column, which can result in a DB constraint violation error.
Reproductio...Overview
----------------------------------------
Within Search Kit, a user is not always limited to what type of Field Transformation they can use on a particular column, which can result in a DB constraint violation error.
Reproduction steps
----------------------------------------
1. Go to **Search > Search Kit** and click **NEW SEARCH**
1. Create a Search with *Contributions* as the primary Entity. Name and Save the search.
1. Click **Entity+** to add *Contribution Contacts* as a second Entity
1. Click **Group By** and select **Contact ID**
1. Expand *Field Transformations* and select **Sum** from the drop down menu on the Contact ID column
1. Click **+Add** underneath the highlighted *Compose Search* block, and select **Smart Group**
1. Give the group a title and make sure the *Contact Column* has **Contact ID** selected
1. Go to **Contacts > Manage Groups** and click on **Contacts** in the row of your test Smart Group
1. Got an error "**Fatal error: DB error**".
Current behaviour
----------------------------------------
This issue was discovered when a client set a transformation of "Sum" on the Contact ID field within a Search Kit search with Contributions as the primary entity, Grouped By Contact Id, and then created a Smart Group from that search. When the user went to **Manage Groups** and clicked on **Contacts** next to the Smart Group in question, they received an error screen with a DB constraint violation. Within the ConfigandLog, the error was as follows:
```json
+message: "DB Error: constraint violation"
+userinfo: """
INSERT INTO civicrm_group_contact_cache (contact_id, group_id)\n
SELECT contact_id, group_id FROM civicrm_tmp_e_gccache_bcc919fe8bd625701eed1c8b83638db1\n
GROUP BY contact_id,group_id\n
[nativecode=1452 ** Cannot add or update a child row: a foreign key constraint fails (`client_civi`.`civicrm_group_contact_cache`, CONSTRAINT `FK_civicrm_group_contact_cache_contact_id` FOREIGN KEY (`contact_id`) REFERENCES `civicrm_contact` (`id`) ON DELETE CASCADE)]
```
This happened because the the Sum transformation was selected on the Contact ID field, as shown within Search Kit below:
![Selection_068](/uploads/b4c5bfd830ccdcc6e719e698daa02a3e/Selection_068.png)
![Selection_069](/uploads/1e09d9de5bc15ba1a88ebed3b23bc2ce/Selection_069.png)
Expected behaviour
----------------------------------------
Field Transformations on columns that should only return a singe value (i.e. Contact ID) should be restricted so that a user does not inadvertently cause a constraint violation via Search Kit.
Environment information
----------------------------------------
* __CiviCRM:__ _Master/5.53.0https://lab.civicrm.org/dev/core/-/issues/3921OptionValues & search kit - missing metadata?2022-11-11T03:26:35ZeileenOptionValues & search kit - missing metadata?We want to use search kit to create a UI to cleanup some of our languages. However, most of the fields are not available in the form-builder layer (where I would want to use them as filters).
I *think* they only show if they have `html...We want to use search kit to create a UI to cleanup some of our languages. However, most of the fields are not available in the form-builder layer (where I would want to use them as filters).
I *think* they only show if they have `html` defined in the xml - and that the right thing to do is to add `html` to `value`, `name` , `id`, `label` `is_active` and possibly others but to set most of them to `read-only` (which is a bit confusing but seems to deal with the two different use cases). Probably only `label` & `is_active` need to be editable
@colemanwhttps://lab.civicrm.org/dev/core/-/issues/3924Afform linter has only been running on core-core2022-10-20T07:29:10ZtottenAfform linter has only been running on core-coreOverview
----------------------------------------
There is a linting test for Civi's Angular-HTML files (`tests/phpunit/Civi/Angular/PartialSyntaxTest.php`). However, the linting is not applied consistently, and several files are not in...Overview
----------------------------------------
There is a linting test for Civi's Angular-HTML files (`tests/phpunit/Civi/Angular/PartialSyntaxTest.php`). However, the linting is not applied consistently, and several files are not in compliance.
Having files which pass the linter ensures that:
* Modified/forked files can be easily compared to their originals.
* Files can be filtered/processed via `hook_alterAngular`.
Current behaviour
----------------------------------------
`PartialSyntaxTest` scans all available Angular modules at the time of test. In effect, it scans `civicrm-core:ang/**.html`.
When the test runs, `search_kit` and `afform` are not active, so not `civicrm-core:ext/search_kit/ang/**.html` and `civicrm-core:ext/afform/*/ang/**.html` are not scanned.
NOTE: This mechanism is built on top of PHP's `DOMDocument` (generally) and `phpQuery` (specifically). To test an HTML partial, we basically say:
```php
$doc = (new DOMDocument(...))->loadHTML($inputString);
$outputString = $doc->saveHTML();
assert $inputString === $outputString
```
So a document is considered good if it is stable/consistent across reads+writes.
Expected behaviour
----------------------------------------
All Angular HTML in `civicrm-core.git` should be scanned by a linter.
All files should pass.
Comments
----------------------------------------
There is a helper script `tools/scripts/check-angular.php` which can validate+reformat file. The script is not super-sophisticated -- but just to get a sense of it, one might call:
```
php tools/scripts/check-angular.php --colordiff $PWD/ext/search_kit/ang/crmSearchTasks/crmSearchTaskUpdate.html
```
In principal, we could run this on all the HTML files and simply commit the reformatted version. Then we might rework `PartialSyntaxTest` to plug the hole in the workflow.
But...
There's the issue of the actual changes in the formatting. It is important to note that PHP's `DOMDocument` has two formatting modes -- HTML and XML. The parser here has to use one or the other. These differ in several small ways -- eg some notations work the same in both, but some notations are rejected by one while accepted by the other. Going from memory (*based on testing yesterday*), here are some notable differences:
| Topic | Example | HTML Mode Support | XML Mode Support | Comment |
| -- | -- | -- | -- | -- |
| Standard tag, text | `<div>Hello</div>` | :white_check_mark: Read-write | :white_check_mark: Read-write | `div`, `span`, `p`, etc |
| Standard tag, empty-text | `<div></div>` | :white_check_mark: Read-write | :warning: Read-only<br/> (Prefer `<div/>`) | `div`, `span`, `p`, etc |
| Standard tag, self-closing | `<div/>` | :warning: Read-only<br/> (Prefer `<div></div>`) | :white_check_mark: Read-write | `div`, `span`, `p`, etc |
| Standard tag, self-closing, implicit | `<input>` | :white_check_mark: Read-write | :octagonal_sign: Error | `input`, `img`, `hr`, `br`, etc |
| Standard tag, self-closing, explicit | `<input/>` | :warning: Read-only<br/>(Prefer `<input>`) | :white_check_mark: Read-write | |
| Custom tag, text | `<custom>my text</custom>` | :white_check_mark: Read-write | :white_check_mark: Read-write | |
| Custom tag, empty-text | `<custom></custom>` | :white_check_mark: Read-write | :warning: Read-only<br/>(Prefer `<custom/>`) | |
| Custom tag, self-closing | `<custom/>` | :warning: Read-only<br/>(Prefer `<custom></custom>`) | :white_check_mark: Read-write | |
| Value-less attributes (custom or standard) | `<foo ng-list>` | :white_check_mark: Read-write | :octagonal_sign: Error | |
| Value for `disabled` | `<foo disabled="abcd">` | :octagonal_sign: Discard value | :white_check_mark: Read-write | |
(*I wanted to see this table because I had the nagging feeling that we were boxed in somehow -- and this does show that there are errors in either case. But these can be worked through...*)
I believe that running the auto-format (*at this moment*) would pose a few issues:
* For HTML mode (status-quo), it would discard the `disabled="..."` that appears in ~5 components. (*I suspect that this status-quo is actually a bug -- but only symptomatic when using `alterAngular`.*) We could update affected files. This may require changing some contracts.
* For XML mode, it would fail to parse `<input>`, `<foo ng-list>`, and similar. We could update these (`<input/>` and `<foo ng-list=",">`), though this would affect many more lines of code.
* In either mode, some files need to flip between `<foo></foo>` and `<foo/>`. Regex can control this a bit more tightly, though. We would need to choose which form we like.
There's also the possibility (out of left field) to replace the current parser or formatter or validator. But that feels like an even bigger rabbit hole.https://lab.civicrm.org/dev/core/-/issues/3925CORS support for CiviCRM urls2022-10-25T11:55:21ZMichael McAndrewCORS support for CiviCRM urlsI wrote an extension that allows you to add CORS headers to CiviCRM responses: https://github.com/civicrm/civicrm-core/pull/24767.
All comments, ideas and feedback gratefully received, as always :slightly_smiling_face:.I wrote an extension that allows you to add CORS headers to CiviCRM responses: https://github.com/civicrm/civicrm-core/pull/24767.
All comments, ideas and feedback gratefully received, as always :slightly_smiling_face:.https://lab.civicrm.org/dev/core/-/issues/3928Additional languages 'en_IN', 'en_SA', en_NZ, en_SG2022-10-20T07:16:47ZeileenAdditional languages 'en_IN', 'en_SA', en_NZ, en_SGCurrently in our database we have languages for
![image](/uploads/bdc8b1342d339599954b3c4f77ba00c3/image.png)
I think we should add additional variants for English where we know that English is the or an official language - as they ha...Currently in our database we have languages for
![image](/uploads/bdc8b1342d339599954b3c4f77ba00c3/image.png)
I think we should add additional variants for English where we know that English is the or an official language - as they have formatting associated with them - e.g both NZ & India use dd/mm/yyyy as the date format.
We could likely only do on new installs
I'm not imagining being incredibly thorough but these ones popped uphttps://lab.civicrm.org/dev/core/-/issues/3930Managed entity: creating custom fields with same label causes fatal error bec...2022-10-24T09:18:16ZherbdoolManaged entity: creating custom fields with same label causes fatal error because of db index.Overview
----------------------------------------
It seems a bit over the top to have fatal errors when custom field labels are not unique. This would (likely) only appear if someone had a managed entity for the custom field. Or if they...Overview
----------------------------------------
It seems a bit over the top to have fatal errors when custom field labels are not unique. This would (likely) only appear if someone had a managed entity for the custom field. Or if they were using the API directly.
Example use-case
----------------------------------------
1. Create a custom group. Add a custom field.
2. Go to API4, choose CustomField, create. Create a new custom field with the same label.
Current behaviour
----------------------------------------
> DB Error: already exists
Proposed behaviour
----------------------------------------
Remove the index from the database for the label. The custom field index on the name should be enough. The UI will still have validation to prevent duplicate labels, but that's necessary because there's no way to specify the label and name separately in the UI.https://lab.civicrm.org/dev/core/-/issues/3934Settings - Deprecate "serialize" flag2022-12-16T01:05:12ZtottenSettings - Deprecate "serialize" flag# Background: Basic settings
Settings are "key-value" pairs. The "value" part allows type `mixed` (*ie `string` or `int` or `float` or `array` or whatever*).
Support for `mixed` is consistent between `Civi::settings()` and `$civicrm_se...# Background: Basic settings
Settings are "key-value" pairs. The "value" part allows type `mixed` (*ie `string` or `int` or `float` or `array` or whatever*).
Support for `mixed` is consistent between `Civi::settings()` and `$civicrm_setting`.
```php
// Consistent access to a scalar
Civi::setting()->set('my_string', 'abc');
$civicrm_setting['domain']['my_string'] = 'abc';
assert Civi::setting()->get('my_string') === 'abc';
```
```php
// Consistent access to an array
Civi::setting()->set('my_array', [1,2,3]);
$civicrm_setting['domain']['my_array'] = [1,2,3];
assert Civi::setting()->get('my_string') === [1,2,3];
```
Under the hood, values are stored in MySQL using `serialize()`. You can see this in both recent and ancient code:
* 5.54's `setDb()`: https://github.com/civicrm/civicrm-core/blob/5.54/Civi/Core/SettingsBag.php#L413
* 5.54's `loadValues()`: https://github.com/civicrm/civicrm-core/blob/5.54/Civi/Core/SettingsBag.php#L137
* 4.3's `setItem()`: https://github.com/civicrm/civicrm-core/blob/4.3/CRM/Core/BAO/Setting.php#L325
* 4.3's `getItems()`: https://github.com/civicrm/civicrm-core/blob/4.3/CRM/Core/BAO/Setting.php#L216
# Background: `serialize` metadata
There is an additional option in `*.setting.php` metadata called `serialize`. Superficially, this resembles `serialize` metadata for DAO fields.
This option is defined for 8 settings: `contact_view_options`, `contact_edit_options`, `advanced_search_options`, `user_dashboard_options`, `address_options`, `contact_autocomplete_options`, `contact_reference_options`, and `quicksearch_options`.
In all cases, the setting has a serialization format of `SERIALIZE_SEPARATOR_BOOKEND`.
APIv4's `Setting` is a wrapper for `Civi::settings()`. Notably, it interprets the `serialize` metadata and applies an (additional/secondary) serialization.
# What's good about this?
It makes it easier for REST/AJAX agents to read and write those 8 historical settings (*the pre-existing settings with the annoying BOOKEND format*).
At the time of introduction, it probably had no effects on compatibility and required no upgrade steps.
# What's bad about this?
It encourages people to add unnecessary serialization layers. If you are writing a _new_ setting with structured data, then you'll see the `serialize` metadata and think: "I should add that!" Here are some docs that suggest `JSON` for a setting:
https://docs.civicrm.org/dev/en/latest/framework/setting/#settings-definition
But you shouldn't do that on a new setting! Here's why:
* __It adds complexity -- double-encoding.__ Every read and write of the data goes through both `serialize()` and some additional serialization method.
* __It adds complexity -- conflicted APIs.__ The setting can be viewed through various programmatic interfaces (e.g. local PHP array `$civicrm_setting`; local PHP method `Civi::setting()->get()`; e.g. remote REST method `Setting.get`). Some APIs present an `array` -- but others present an encoded string (like `^1^2^3^`). There is no decent reason why `$civicrm_setting`, `Civi::setting()`, and `Setting.get` should present it differently.
* __It distracts from the real data-modeling problem of serialized fields.__ We generally do simple settings (strings/ints) because they're easier. Complex settings (like `mailing_backend`) are possible, but we don't do them as much - because the tooling/validation/etc is not as good. Intuitively, complex settings could work better if we added more metadata. _But this is the wrong metadata._ Switching `mailing_backend` between `serialize`/JSON/CSV/YAML/etc doesn't make this any better. What you need is a sharper description of the data inside of that serialized blob.
* __It becomes overall harder to change.__ Perhaps settings should be stored with `json_encode()` instead of `serialize()`? `serialize()` has additional kinds of vulnerabilities. JSON has better support in MySQL. Plot out the path for how to migrate the entirety of `civicrm_setting.value` -- and compare that to switching serialization on a setting-by-setting basis. It's easier to change if individual settings are _not_ committed to specific serializers.
# What to do
Simplest thing:
* Keep the mechanism
* Present it as deprecatedhttps://lab.civicrm.org/dev/core/-/issues/3940Method and Tracking not set from API4 for Subscription History when creating ...2022-10-31T17:07:22ZlarsssandergreenMethod and Tracking not set from API4 for Subscription History when creating or updating GroupContactMethod and Tracking can be specified when creating or updating a GroupContact in API4 (with a default Method of API). However, these are always set as null for the Subscription History that is created.
I've been doing some debugging to ...Method and Tracking can be specified when creating or updating a GroupContact in API4 (with a default Method of API). However, these are always set as null for the Subscription History that is created.
I've been doing some debugging to follow these variables through the process and can see where this is going wrong, but am not clear on how this should actually work. The Subscription History is created by a [hook_post](https://github.com/civicrm/civicrm-core/blob/902fc38d0d6cd9d09ee02f30cc6621747151a06a/CRM/Contact/BAO/GroupContact.php#L39). The hook isn't even trying to set the Method and Tracking, nor does it have access to these variables (they are there when the GroupContact is created, but they aren't used for that).
I'm not clear on how these variables are supposed to get from the API call to the hook here. Happy to implement a fix, but not sure I understand how this is supposed to work or if we need to use an entirely different approach. I will also do the same with the delete action, which [I'm adding Subscription History to](https://github.com/civicrm/civicrm-core/pull/24804), so I'm looking for an approach which will also work for delete.https://lab.civicrm.org/dev/core/-/issues/3943Searching for Current Member = No returns contacts who are current members if...2022-10-28T20:05:12ZlarsssandergreenSearching for Current Member = No returns contacts who are current members if they also have a non-current membershipIf you search in Advanced Search or Find Memberships with Current Member = No, you will get all contacts who have a membership which is not current. This is not the same as saying they aren't a current member, because the contact could h...If you search in Advanced Search or Find Memberships with Current Member = No, you will get all contacts who have a membership which is not current. This is not the same as saying they aren't a current member, because the contact could have another membership which is current. This is probably quite common as many contacts will buy a new membership instead of renewing their past membership.
In the case that a contact has, for example, one expired membership and one current membership, they will be found when searching for Current Member = No and also for Current Member = Yes. Confusing!