CiviCRM Core issueshttps://lab.civicrm.org/dev/core/-/issues2024-03-18T15:35:22Zhttps://lab.civicrm.org/dev/core/-/issues/5086Ability to export FormBuilder forms from the UI2024-03-18T15:35:22ZherbdoolAbility to export FormBuilder forms from the UIWe've got an export link for SearchKit, but if a saved search is wrapped up in a FormBuilder form there's no easy way to also export those files. I can imagine a modal with the text from the forms files (`*.html`, `*.json`) and a link to...We've got an export link for SearchKit, but if a saved search is wrapped up in a FormBuilder form there's no easy way to also export those files. I can imagine a modal with the text from the forms files (`*.html`, `*.json`) and a link to copy it. Or even ability to save the files as a zip file.
And I suppose we'd need an import link as well to import on a different site (similar to SearchKit's link).https://lab.civicrm.org/dev/core/-/issues/5073Standalone default folder structure2024-03-29T07:36:10ZufundoStandalone default folder structureSome wise suggestions from @artfulrobot for clearer folder naming / out-of-the-box security:
> ```
> - civicrm-standalone-X.Y.Z.zip
> - index.php
> - .htaccess
> - robots.txt ➌
> - data/
> - data/ext/....Some wise suggestions from @artfulrobot for clearer folder naming / out-of-the-box security:
> ```
> - civicrm-standalone-X.Y.Z.zip
> - index.php
> - .htaccess
> - robots.txt ➌
> - data/
> - data/ext/.htaccess
> - data/public/.htaccess ➊
> - data/.private/.htaccess ➋
> - core/<ALL-THE-CODES>
> ```
>
> 1. use 'public' not 'upload'. It partners well with 'private', and 'upload' is such a daft relative-to-what? term (not everything you upload through a browser ought to be in a public dir). I think the whole 'persist' 'contribute' etc. is a right mess - or at least I don't understand the logic if there is logic, though last time I looked at it the logic was that at some point in history the first thing to allow uploads was civi contribute so ...
>
> 2. use a dot before _private/_. It's very common, and easy, to ban http access to all 'dot files'. So this gives an extra _likely_ shield against the "oh, I didn't realise nginx ignored .htaccess files" users. Just feels safer, if we're focussing on making this easy.
>
> 3. In terms of sensible defaults, I feel we should have a robots.txt that does its best to ban crawlers, especially AI ones, on everything except specific paths (e.g. event pages). It's one thing to have someone tell you you've accidentally exposed data and someone saw it, it's another to find that your [exposed data now lives in an LLM](https://arstechnica.com/information-technology/2022/09/artist-finds-private-medical-record-photos-in-popular-ai-training-data-set/) training set, ready to be given to anyone with a particular prompt.
Maybe to the nginx point, we could add `nginx-civicrm-site.conf.sample` in the root?
I think the principles for the public / private upload folder names apply to the composer template and the tarball.ufundoufundohttps://lab.civicrm.org/dev/core/-/issues/5060Automated reminder to people that didn't subcribe to groups they applied subs...2024-03-14T22:45:20Znikola.mladenovicAutomated reminder to people that didn't subcribe to groups they applied subscription forOverview
----------------------------------------
Subscribing for newsletter usually ends up as a one way street, if they want to have subscribers and be under wing of GDPR. If the user doesn't confirm their subscription, they would eith...Overview
----------------------------------------
Subscribing for newsletter usually ends up as a one way street, if they want to have subscribers and be under wing of GDPR. If the user doesn't confirm their subscription, they would either have to find original email or resubscribe if they cant find it, but usually users just forget they did subscribe.
Recommended improvement was discussed on Mattermost to check if such feature existed in civicrm and few users did raise concern and that they would like to have this feature request:
[Mattermost chat](https://chat.civicrm.org/civicrm/pl/bg6x9u7qmf8fdkwn8tz4o9ft1r )
Current behaviour
----------------------------------------
Subscribing to newsletter only sends email as user applies for newsletter
Proposed behaviour
----------------------------------------
Scheduled job for example, which would check if there are any users which are pending for some groups, check the date they applied and then send another nudge for user to join in. This feature should be limited to only once per fresh request for all pending users. Proposal for after how much this automated schedueld job would contact users should be a week, 2 weeks, month, 2 months, 3 months.
Comments
----------------------------------------
_Anything else you would like the reviewer to note._https://lab.civicrm.org/dev/core/-/issues/5057Make descriptions visible in drop down2024-03-22T18:22:38ZyashodhaMake descriptions visible in drop downAll the select2 drop-down are mostly option values which do have a description field in the database.
It will be useful to use this for display in select2 drop-downs.
We already show description for entities like events. It will be help...All the select2 drop-down are mostly option values which do have a description field in the database.
It will be useful to use this for display in select2 drop-downs.
We already show description for entities like events. It will be helpful to choose options for the user(if options do indeed explanation and are configured as such)https://lab.civicrm.org/dev/core/-/issues/5053Formbuilder/ event tab: Add the possibility to create or select a location2024-03-07T13:34:54ZNadaillacFormbuilder/ event tab: Add the possibility to create or select a locationIn the tab event, there should be the possibility to select a location or create a location, and in that case set all location fields. Similarly to the current backend form.
This enhancement could be funded.
Discussed with @colemanwIn the tab event, there should be the possibility to select a location or create a location, and in that case set all location fields. Similarly to the current backend form.
This enhancement could be funded.
Discussed with @colemanwhttps://lab.civicrm.org/dev/core/-/issues/5049Ability to add custom css class and/or ID to any formbuilder element2024-03-12T19:08:51ZnoahAbility to add custom css class and/or ID to any formbuilder elementOverview
----------------------------------------
In FormBuilder, the user should be able to add an arbitrary css class or id to any container, fieldset, field, button etc.
Based on discussion among @colemanw @JonGold @shaneonabike @tot...Overview
----------------------------------------
In FormBuilder, the user should be able to add an arbitrary css class or id to any container, fieldset, field, button etc.
Based on discussion among @colemanw @JonGold @shaneonabike @totten at the Montreal sprint.colemanwcolemanwhttps://lab.civicrm.org/dev/core/-/issues/5043Ability to set font type, font size and line spacing as default2024-02-29T16:43:58Zaydunsaidan.saunders@squiffle.ukAbility to set font type, font size and line spacing as default(From a user at a visibility charity.)
When you create a new activity, the text box for the activity defaults to sand-serif and 13 point.
It would be beneficial to have the ability to set a default font type and size, along with defaul...(From a user at a visibility charity.)
When you create a new activity, the text box for the activity defaults to sand-serif and 13 point.
It would be beneficial to have the ability to set a default font type and size, along with default line spacing, for all text boxes.
For example, we mandate Arial, 14 pt and 1.5 line spacing for electronic communications. That’s what we would like to have a standard on our version of CiviCRM. Having the ability to set these as default on CiviCRM saves workers from having to manually change font type and size when using the activity details box.
I don’t think there is any way to set line spacing in Civi. This would be a much-needed addition, if possible.https://lab.civicrm.org/dev/core/-/issues/5015'Clean-up Temporary Data and Files' doesn't empty the tplCache by default2024-02-23T15:18:23ZAndrew West'Clean-up Temporary Data and Files' doesn't empty the tplCache by defaultThe 'Clean-up temporary data and files' scheduled job [doesn't clean the tpl directory by default](https://github.com/civicrm/civicrm-core/blob/06540d39dc2cedcadb19be5c351ef831a9133534/api/v3/Job.php#L631). Should it? We see this balloon...The 'Clean-up temporary data and files' scheduled job [doesn't clean the tpl directory by default](https://github.com/civicrm/civicrm-core/blob/06540d39dc2cedcadb19be5c351ef831a9133534/api/v3/Job.php#L631). Should it? We see this ballooning to tens of gigabytes when sending large mailings.
Workaround is to add the parameter, but it seems like this might catch some people out.
![image](/uploads/85b4275fb08070a8995fe094e32d466e/image.png)https://lab.civicrm.org/dev/core/-/issues/5008[Enhancement] ContributionRecur edit activity should include more details2024-03-05T15:29:46ZElliott Eggleston[Enhancement] ContributionRecur edit activity should include more detailsOverview
----------------------------------------
When one edits a recurring contribution an activity is saved saying which user edited it. If the amount or the number of installments has been changed, that is included in the activity de...Overview
----------------------------------------
When one edits a recurring contribution an activity is saved saying which user edited it. If the amount or the number of installments has been changed, that is included in the activity details. However, no other changes are included (and the ContributionRecur row changes from log_contribution_recur are not in the contact change log), so it's impossible to tell in the UI if someone changed e.g. the next scheduled contribution date or the cycle day.
Example use-case
----------------------------------------
1. Edit a ContributionRecur
2. Look at the activity tab, and see all the changes you made
Current behaviour
----------------------------------------
Only changes to number of installments and amount are listed in the activity details
Proposed behaviour
----------------------------------------
Changes to other subscription properties are also listed in the activity details5.72.0https://lab.civicrm.org/dev/core/-/issues/5000SearchKit: Allow formatting of the totals in the footer2024-02-14T10:06:39Zsimon.hermannSearchKit: Allow formatting of the totals in the footerSearchKit allows to create a footer row, where the total of a column can be displayed. However, this total does not allow for any formatting or reproduces the formatting of the columns it sums.
It would be great if there would be either ...SearchKit allows to create a footer row, where the total of a column can be displayed. However, this total does not allow for any formatting or reproduces the formatting of the columns it sums.
It would be great if there would be either an option to choose a formatting (decimal separators, thousand separators, currencies if applicable etc) or if the total would reflect the formatting of the column it sums.https://lab.civicrm.org/dev/core/-/issues/4999Imagine a world without CodeGen2024-03-05T04:32:51ZcolemanwImagine a world without CodeGenCurrently we use `CRM_Core_CodeGen` to take our schema/xml files and generate DAO.php, install.sql and uninstall.sql files, which have to be periodically regenerated. This is a minor inconvenience for a core developer, a potential gotcha...Currently we use `CRM_Core_CodeGen` to take our schema/xml files and generate DAO.php, install.sql and uninstall.sql files, which have to be periodically regenerated. This is a minor inconvenience for a core developer, a potential gotcha for an extension developer, and a major coordination difficulty across *all* extensions in `universe` (whenever any aspect of the generated code needs to change).
But what if we didn't have to generate those files? What if we could read schema information directly from the xml (or potentially a different source).
Current Structure:
-----
**Key:** ✍️ Handwritten file | 📠 Generated file
| File | Purpose | In Core | In Extensions |
| --- | --- | --- | --- |
| ✍️ `schema/xml` | Canonical declaration of entity + all metadata | Run `setup.sh -g` | Run `civix generate:entity-boilerplate` |
| 📠 Install sql | Add schema tables | `civicrm.mysql` | `auto_install.sql` |
| 📠 Uninstall sql | Drop schema tables | `civicrm_drop.mysql` | `auto_uninstall.sql` |
| 📠 Entity.php | Declare entity's existence | `AllCoreTables.data.php` | `*.entityType.php` + `entity-types-php` mixin |
| 📠 `CRM/Core/I18n/SchemaStructure.php` | Lists localizable table columns | Seems kinda redundant with other metadata? | Doesn't exist |
| ✍️ `CRM_Core_DAO` | Base class for all generated DAOs | All DAOs extend this class | Extension DAOs also extend core class (makes change-management across `universe` difficult) |
| 📠 `CRM_*_DAO_*` | Generated from the xml file | Must be generated | Must be generated |
The generated DAO file (including stuff it inherits from `CRM_Core_DAO`) serves a variety of purposes:
1. OO class that allows a database table to be used like a php object, e.g.
```
$contact = new CRM_Contact_DAO_Contact();
$contact->first_name = 'Bob';
$contact->save();
```
This is a neat idea and probably ahead of its time, but is basically deprecated now, mostly because, in a relational database, having access to only one table at a time isn't very useful. If it weren't for legacy core & extension code still using this pattern, we could drop it in favor of more robust tools like APIv4.
2. Static methods like `fields()` and `indices()` which return the data from the xml file in php format.
3. Localizes strings, because CodeGen wraps titles and labels in `ts()`.
4. A bunch of other random static methods (e.g. `disableFullGroupByMode()`) which seem like they'd be better-placed in a separate utility class.
New Structure
----------
If we no longer want to generate files and just read from a canonical source, then the main question is, "what should be the canonical source of entity metadata?"
1. **Stick with XML:** Keep the existing xml files but delete the generated stuff. Parse the xml at runtime to get that data.
- Pro: It's already there, no rewrites needed.
- Con: Poor DX (developers don't generally enjoy writing XML files).
- Con: It's very slow (the slowest by far of all the options) so heavy caching would be needed.
- Con: Without generating php files, another method of i18n string extraction would be needed ([such as this](https://github.com/civicrm/civistrings/pull/17)).
2. **DAO Files:** Delete the `schema/xml` files and run everything from the generated DAO files, which going forward will be hand-edited instead of regenerated.
- Pro: DAO files are already there.
- Con: Also poor DX (the boilerplate in those files would not be fun to write/edit by hand).
3. **Somewhere Else:** Move schema info to e.g. json files or better-structured PHP files.
- Pro: DX and performance could be optimized.
- Con: XML files must be rewritten (could be scripted).
- Con: Migration management for core and extensions.
Supporting Dynamic/Virtual Entities
--------
It's also worth keeping in mind that there are now several types of entities that are dynamic & share a DAO:
- Multi-record custom fields
- ECK entities
- SearchKit materialized displays
The DAO structure doesn't cope with this very well, as the assumption has always been 1-1-1 between table, entity & DAO class. But while we're restructuring things let's avoid adding more code that makes this assumption. An ideal DAO from the POV of virtual entities would be an object that takes entity name in its constructor & initializes itself with the appropriately corresponding tablename, fields, and other metadata.https://lab.civicrm.org/dev/core/-/issues/4995Standalone - JWT for password resets2024-03-07T22:30:13ZufundoStandalone - JWT for password resetsSwitch Standaloneusers password reset mechanism to use JWT, instead of custom format.
PR from @pfigel is here: https://github.com/civicrm/civicrm-core/pull/28505Switch Standaloneusers password reset mechanism to use JWT, instead of custom format.
PR from @pfigel is here: https://github.com/civicrm/civicrm-core/pull/28505Patrick Figelpfigel@greenpeace.orgPatrick Figelpfigel@greenpeace.orghttps://lab.civicrm.org/dev/core/-/issues/4992Standalone - front-end theming2024-02-14T10:01:23ZufundoStandalone - front-end theming@artfulrobot suggests: provide a template for front facing pages. People can edit that file
(manually) to suit. It should not get overwritten on upgrade.@artfulrobot suggests: provide a template for front facing pages. People can edit that file
(manually) to suit. It should not get overwritten on upgrade.RichRichhttps://lab.civicrm.org/dev/core/-/issues/4991Standalone SSO - third-party authentication for single sign on, such as Hybri...2024-02-14T10:01:04ZufundoStandalone SSO - third-party authentication for single sign on, such as HybridAuth or LDAP or SAMLWould be great to build out the standalone users functionality.
Either in Standaloneusers extension or a separate one?Would be great to build out the standalone users functionality.
Either in Standaloneusers extension or a separate one?ufundoufundohttps://lab.civicrm.org/dev/core/-/issues/4982Form builder - Extension provided entity settings2024-02-14T09:53:44ZseamusleeForm builder - Extension provided entity settingsJMA is in the progress of working on a geographically targeted petition extension
At the moment we have some additional settings that we get a user to set when creating the petition page in Form Builder. These get injected by the alter ...JMA is in the progress of working on a geographically targeted petition extension
At the moment we have some additional settings that we get a user to set when creating the petition page in Form Builder. These get injected by the alter angular hook https://lab.jmaconsulting.biz/extensions/geotargetedpetitions/-/blob/main/geotargetedpetitions.php?ref_type=heads#L76 however this means that even on non petition related form builder forms they will show up.
This is about seeing if there is a better way to achieve this and or see if we can get them put onto the specific settings page for say GeotargetedPetition Entity.
@JoeMurray @Edselopez @colemanwhttps://lab.civicrm.org/dev/core/-/issues/4920Support the import of all API4 entities2024-02-20T09:29:00ZMichael McAndrewSupport the import of all API4 entitiesIt would be good to support importing of all APIv4 entities in core.
I found four extensions with a reasonable number of downloads that implement improvements to importing:
* https://civicrm.org/extensions/api-csv-import-gui by @eileen...It would be good to support importing of all APIv4 entities in core.
I found four extensions with a reasonable number of downloads that implement improvements to importing:
* https://civicrm.org/extensions/api-csv-import-gui by @eileen
* https://civicrm.org/node/5879 (csv import helper) by @artfulrobot
* https://civicrm.org/extensions/advanced-import by @bgm
* https://civicrm.org/extensions/option-value-importer by@sluc23
But most (if not all?) of these extensions are built on API3.
It would be great to have something built with API4 since this would allow import of any new 'API4 first' entities that are created in core and in extensions (including new entities that have been created with ECK @jensschuppe).
So I am wondering if there are any other initiatives to improve import in the works? And if anyone has the appetite for creating an import core extension that is powered by APIv4 with the long term aim of replacing and enhancing the core import functionality? Perhaps starting with one of the existing ones as a base.https://lab.civicrm.org/dev/core/-/issues/4907Event full inconsistencies2024-01-15T11:35:24ZeileenEvent full inconsistenciesThe way EventFull is calculated in apiv4 is not consistent with elsewhere
apiv4 returns a field `'remaining_participants'` which is
event.max_participants - COUNT(participant rows for the event where contact is not deleted, participant ...The way EventFull is calculated in apiv4 is not consistent with elsewhere
apiv4 returns a field `'remaining_participants'` which is
event.max_participants - COUNT(participant rows for the event where contact is not deleted, participant is not test and participant status is_counted)
By contrast apiv3 returns the contents of `CRM_Event_BAO_Participant::eventFull()` which also takes into account the participant role and other aspects of the status
`eventFull()` has some funky parameters - there are 12 calls to it
| caller | returnEmptySeats |includeWaitingList|returnWaitingCount|considerTestParticipant|onlyPositiveStatuses|
| ------ | ------ |------ |------ |------ |------ |
| apiv3-event | TRUE |TRUE|FALSE|FALSE|FALSE|
| Participant::eventFullMessage | FALSE |FALSE|FALSE|FALSE|FALSE|
| Participant::eventFullMessage(2) | FALSE |TRUE|TRUE|FALSE|FALSE|
|ParticipantStatusType::process|TRUE|FALSE|FALSE|FALSE|
|ParticipantTest|FALSE|TRUE|FALSE|FALSE|FALSE|
|Registration::preProcess|FALSE|if-event-has-waitlist|FALSE|FALSE|FALSE|
|Registration::preProcess(2)|TRUE|if-event-has-waitlist|FALSE|FALSE|FALSE|
|Event_Confirm::formRule|FALSE|if-event-has-waitlist|FALSE|FALSE|FALSE|
|ParticipantConfirm|TRUE|FALSE|TRUE|FALSE|TRUE|
|Register::preProcess|FALSE|if-event-has-waitlist|FALSE|FALSE|FALSE|
|EventInfo::run|FALSE|if-event-has-waitlist|FALSE|FALSE|FALSE|
|event-cart|TRUE|TRUE|FALSE|FALSE|FALSE|
Note that this issue somewhat relates/ explains
https://lab.civicrm.org/dev/event/-/issues/23https://lab.civicrm.org/dev/core/-/issues/4899Modelling many to many relationships with Entity reference, SearchKit, FormBu...2024-01-17T18:18:18ZMichael McAndrewModelling many to many relationships with Entity reference, SearchKit, FormBuilder, ECK, etc.Creating this issue as a way to keep track of different bits of functionality that can be used together to model many to many relationships in CiviCRM as when you put this all together, I think it is quite a game changer for data modelli...Creating this issue as a way to keep track of different bits of functionality that can be used together to model many to many relationships in CiviCRM as when you put this all together, I think it is quite a game changer for data modelling in CiviCRM :grinning:
Might make sense to document this at some point soon, and it would be good to collect feedback on how people are finding this functionality, ideas for improvement, etc.
Also, if you have any budget that you would like to put towards this work: to improve it or build out more features, etc. please get in contact with @colemanw or me :heart:.
* Entity Reference fields - allows you to reference other entities in custom data fields effectively creating **one to many** relationships.
* Multivalue custom data sets - [now available to all entities](https://github.com/civicrm/civicrm-core/pull/27549) allowing you to model **many to many** relationships _with additional meta data_ about the relationship
* Searchkit support for [joining via EntityRef fields in multivalue custom data](https://github.com/civicrm/civicrm-core/pull/28721) - allows you to create searches that span many to many joins
* FormBuilder support - allowing for editing of many to many relationships in entities (could probably do with some improvement)
* [ECK](https://github.com/systopia/de.systopia.eck) which allows people to make arbitrary new entities that can be joined via these relationshipshttps://lab.civicrm.org/dev/core/-/issues/4883[5.68.1] pre-minified source-less file: "bower_components/es-module-shims/dis...2024-01-29T10:13:51ZDmitry Smirnov[5.68.1] pre-minified source-less file: "bower_components/es-module-shims/dist/es-module-shims.js"`bower_components/es-module-shims/dist/es-module-shims.js` is bundled in pre-minified source-less form, which is bad practice.
Please bundle original un-compressed/un-obfuscated source, readable and modifiable by humans.
It looks like ...`bower_components/es-module-shims/dist/es-module-shims.js` is bundled in pre-minified source-less form, which is bad practice.
Please bundle original un-compressed/un-obfuscated source, readable and modifiable by humans.
It looks like bundling the following file should suffice: https://github.com/guybedford/es-module-shims/blob/1.7.2/src/es-module-shims.js
Thanks.5.70.0https://lab.civicrm.org/dev/core/-/issues/4875Better tracking of Scheduled Reminders including Reporting2024-02-21T19:56:17ZshaneonabikeBetter tracking of Scheduled Reminders including Reporting## Overview
_It has come up quite a few times with several clients of ours, whereby they want to see what Scheduled messages went out, if they bounced, and other details. In a few cases, the Scheduled Reminders were not configured prope...## Overview
_It has come up quite a few times with several clients of ours, whereby they want to see what Scheduled messages went out, if they bounced, and other details. In a few cases, the Scheduled Reminders were not configured properly by the client, which is understandable but that's another UI issue._
_So I think it could be really helpful to provide this type of tracking, including click-through, FFWD, etc for Scheduled Reminders like regular mailings._
## Example use-case
1. Set up a Scheduled Reminder for an Event or Membership Renewal
2. Go to Reports to view the progress of Scheduled Reminders
3. Filter by click-throughs, bounce, etc.
## Current behaviour
_Same as above. Presently, the only workaround in some cases like Scheduled Reminders is to create a "fake" mailing that is sent out to one person. Then copying the code generated into the Scheduled Reminder allows a person to actually track the Scheduled Reminder information like you would with a regular mailing because it is associated to our "fake" mailing. But this is super not sustainable or even doable for Event Reminders._
## Proposed behaviour
_Basically, I think it could be extremely useful to integrate the same functionality for Mailing tracking (including bounce) to Scheduled Reminders._