Development issueshttps://lab.civicrm.org/groups/dev/-/issues2023-10-13T21:11:08Zhttps://lab.civicrm.org/dev/core/-/issues/2486Apiv4 entity parity2023-10-13T21:11:08ZeileenApiv4 entity parity## Here are the entities in APIv3 & not v4
_In some cases by design - others we should maybe add..._
# Prioritised
- [ ] Attachment.php - `get` [has been added to support viewing attachments in SearchKit](https://github.com/civicrm/civ...## Here are the entities in APIv3 & not v4
_In some cases by design - others we should maybe add..._
# Prioritised
- [ ] Attachment.php - `get` [has been added to support viewing attachments in SearchKit](https://github.com/civicrm/civicrm-core/pull/27379) but `create` is still missing
- [ ] Extension.php - various install, uninstall etc functions (support for some actions already implemented)
# Partially done - lets resolve
- [ ] MembershipLog.php https://github.com/civicrm/civicrm-core/pull/21106
# Standard crud - note that doing these involves checking all pseudoconstants are correctly defined
- [ ] SmsProvider.php
- [ ] Premium.php
- [ ] RecurringEntity.php
# Mailing related - in scope
- [ ] MailingAB.php
- [ ] MailingComponent.php
- [ ] MailingContact.php
- [ ] MailingEventResubscribe.php
- [ ] MailingRecipients.php
# From here on down we are out of scope for 'entity parity'
## Order related - needed when we get to afform payment forms - out of scope for entity parity
- [ ] Order.php
- [ ] Payment.php
## Varied utils - out of scope for 'entity parity' - ad hoc
- [ ] SystemLog.php
- [ ] User.php
- [ ] Logging.php
## Migration doubtful - to be done ad hoc if we decide to
- [ ] ReportTemplate.php
- [ ] ParticipantPayment.php - likely should not be brought over
- [ ] CxnApp.php
- [ ] Cxn.php
- [ ] Profile.php
## Do not migrate
- ~~ActivityType.php (by design)~~
- ~~CustomSearch.php (by design)~~
- ~~Constant.php (by design)~~
- ~~SurveyRespondant.php(by design)~~
- ~~MembershipPayment.php (by design - use Line Item)~~
## Done
- [x] ~~AclRole.php~~ ACLEntityRole https://github.com/civicrm/civicrm-core/pull/20474
- [x] Batch.php https://github.com/civicrm/civicrm-core/pull/19931
- [x] CaseContact.php https://github.com/civicrm/civicrm-core/pull/19907
- [x] Case.php https://github.com/civicrm/civicrm-core/pull/19907
- [x] CaseType.php
- [x] ContributionProduct.php https://lab.civicrm.org/dev/core/-/issues/2683
- [x] Dedupe.php
- [x] EntityBatch.php https://lab.civicrm.org/dev/core/-/issues/2682
- [x] EntityFinancialAccount.php - https://github.com/civicrm/civicrm-core/pull/19927
- [x] EntityFinancialTrxn.php - https://github.com/civicrm/civicrm-core/pull/19932
- [x] ~~Exception.php~~ DedupeException.php https://github.com/civicrm/civicrm-core/pull/20466
- [x] File.php (part of attachment) https://github.com/civicrm/civicrm-core/pull/21158
- [x] FinancialItem.php - https://github.com/civicrm/civicrm-core/pull/20433
- [x] Job.php (crud portion)
- [x] Job.php
- [x] JobLog.php
- [x] Mailing.php - there are a bunch of related entities - see further down)
- [x] MailingEventConfirm.php
- [x] MailingEventQueue.php
- [x] MailingEventSubscribe.php
- [x] MailingEventUnsubscribe.php
- [x] MailingGroup.php
- [x] MailingJob.php
- [x] Membership.php - see https://lab.civicrm.org/dev/core/-/issues/2634 https://github.com/civicrm/civicrm-core/pull/21106
- [x] MembershipBlock.php https://github.com/civicrm/civicrm-core/pull/21106
- [x] MembershipStatus.php https://github.com/civicrm/civicrm-core/pull/21106
- [x] ParticipantStatusType.php
- [x] PaymentToken.php
- [x] Pcp.php
- [x] PrintLabel.php https://github.com/civicrm/civicrm-core/pull/21500
- [x] Product.php - since 5.41
- [x] ReportInstance.php
- [x] ~~Rule.php~~ DedupeRule.php https://github.com/civicrm/civicrm-core/pull/20466
- [x] ~~RuleGroup.php~~ DedupeRuleGroup.php https://github.com/civicrm/civicrm-core/pull/20466
- [x] Survey.php https://github.com/civicrm/civicrm-core/pull/21355
- [x] WordReplacement.php https://github.com/civicrm/civicrm-core/pull/20559https://lab.civicrm.org/dev/core/-/issues/2484Create apiv4 payment api2023-07-13T08:07:37ZeileenCreate apiv4 payment apiIn order to be able to search for payments we need to be able to search for payment rows (rows with is_payment = TRUE) in the financial_trxn table and for the entity bridge to be able to find it's way from there to the contribution table...In order to be able to search for payments we need to be able to search for payment rows (rows with is_payment = TRUE) in the financial_trxn table and for the entity bridge to be able to find it's way from there to the contribution table.
This gives us the ability to say find all contributions with a payment with a payment instrument of 'Check' or find the contribution where the payment's trxn value is y
The scope I would be looking for is
1) can use any of the fields in financial_trxn as a filter
2) can logically link to the contribution (probably through bridge entities)
3) forces 'is_payment' as a filter (only gets payments)
4) the create action calls Payment.create
5) the update & delete actions are blocked
This effectively gives us parity with the v3 api albeit with more options provided through the apiv4 layer itself.
Note that I think this is likely blocked on other feature requests which @mattwire will add notes onhttps://lab.civicrm.org/dev/core/-/issues/2481Discussion drop support for php 7.2 and mysql 5.62023-01-04T18:01:36ZeileenDiscussion drop support for php 7.2 and mysql 5.6We have been dropping support for EOL php & mysql on ESR releases because that gives sites the option of delaying for another 6 months for just $120. So far we are not aware of any sites opting for the ESR for that reason but we are crea...We have been dropping support for EOL php & mysql on ESR releases because that gives sites the option of delaying for another 6 months for just $120. So far we are not aware of any sites opting for the ESR for that reason but we are creating the option.
With 5.39 coming up we should consider whether we want to drop support for
php 7.2 (EOL Dec 2020)
mysql 5.6 (EOL Feb 2021)
Note that historically supporting older versions of mysql has been low effort although some features are not supported. However, it might be holding us back from using new features
Supporting older versions of php has historically been more problematic as it has been at the expense of supporting newer versions. In particular we often find packages don't support latest and EOL versions.
So dropping support for older php versions is more helpful than older mysql versions.
Last ESR we considered dropping support for php 7.1 and mysql 5.6. We did the former and not the latter. At the time the stats were:
mysql 5.6 : 805
php 7.1 : 658.
Maria DB 10.1 : 943
Currently they are
mysql 5.6 : 603
php 7.2 : 2846
Maria DB 10.1 : 526
Based on those numbers I think it's clearly too early for php 7.2 and I would be OK targetting both for the NEXT ESR (5.45)https://lab.civicrm.org/dev/translation/-/issues/67Canonize API for storing translated data2021-08-12T19:25:30ZtottenCanonize API for storing translated data# Goal
Enable richer user experiences which incorporate data-translation. Specifically, provide a CRUD API for administrative applications that need to read/write alternate versions of a string in the database.
# Background
* This is ...# Goal
Enable richer user experiences which incorporate data-translation. Specifically, provide a CRUD API for administrative applications that need to read/write alternate versions of a string in the database.
# Background
* This is most immediately motivated by https://lab.civicrm.org/dev/mail/-/issues/83, which aims to improve the process+experience of drafting+testing workflow templates. For this case, the string that is being edited (ie `civicrm_msg_template.msg_html`) is a relatively rich piece of content (with HTML tags, tokens, Smarty expressions - which in turn may vary based on the context for which the template will be used). The richness of the text implies that one should have more features available (token-pickers, syntax-highlighting, ad nauseum). Editing a translation of this content in a generic textbox (as with multilingual UI, Transifex UI, or POEdit) would be difficult and error-prone.
* This is intended as a step in support of https://lab.civicrm.org/community/feature-request/-/issues/26, which is a broad effort (initiated by @ayduns @BjoernE) to re-conceive how the multilingual subsystem works. TLDR: Current multilingual requires significant MySQL schema manipulation. This works for 1-3 languages but does not scale to 10 languages. Resolving it requires changes in the storage/lifecycle of translated data.
* Inspired by this discussion, Eileen wrote a proof-of-concept extension https://github.com/eileenmcnaughton/civi-data-translate. The scope of `civi-data-translate` mostly matches the scope of this filing, but not quite perfectly. It matches insofar as it introduces an APIv4 interface and a MySQL table for strings. It diverges insofar as it specifically touches on `MessageTemplate`. (The work for `MessageTemplate` is left as a separate matter.) Its biggest obstacle is dependency-hell: it requires a skilled administrator to maintain a deployment, which disincentivizes development and usage.
# Approaches
Working within the limits of available code and capacity, it appears feasible to adapt `civi-data-translate` to this purpose. Either:
1. Move its APIv4 interface and data-storage to core-proper, or...
2. Move its APIv4 interface and data-storage to core-extension.
# Comments
* Having an API to edit the strings would be meaningless if we did not have a data-store.
* There is a performance question about using MySQL for a string table. (Most FOSS applications use `gettext` MO files which are optimized for fast lookup of static strings. This is how Civi handles translation of its numerous app-strings.) In prior discussions with @BjoernE @ayduns etal, we identified this balance:
* There is a difference between *administration* (browsing/editing strings) and *runtime lookup* (substituting 1000 strings during a page-load).
* For administration, there is no question about whether the performance of a MySQL string-table would be acceptable. It would be. In fact, many different tools/workflows/stores can be acceptable.
* The performance question is relevant to *runtime lookup of heavily used strings*. The performance question is not necessarily closed, and it depends on other variables (*the #data-strings, the use-case, the hardware, etc*).
* If one does need to optimize lookup, the best known approach is to compile to gettext. To wit: Read strings from whatever source is handy, aggregate them, and [write them](https://github.com/pear/File_Gettext/blob/master/File/Gettext/MO.php) to a cache folder in `*.mo` format. (You can see [de.systopia.l10nmo](https://github.com/systopia/de.systopia.l10nmo) as a foray into this approach of blending/merging string sources.)
* I was worried about proposing this - specifically, worried that it might conflict with a more optimized dataflow. However, on reflection, I think it is complementary progress. Suppose you wanted to patch `l10nmo` to include a feed of strings provided by web-based administrators. If each web UI stored strings differently, then you'd probably give up. But if they use the same (shared/documented) string API, then it's easier to pull from there.
* (*I mention this as a hypothetical. In practice, some things like dev/mail#83 can be achieved without this level of optimization. The upshot is that we can bite off a chunk of work here on the API/storage side and make some incremental progress.*)https://lab.civicrm.org/dev/core/-/issues/2469Export of custom multiple choice options to CSV files limited to 32 characters2023-07-18T16:44:17ZbkeevilExport of custom multiple choice options to CSV files limited to 32 charactersA user is trying to export data from an advanced query on custom fields to a CSV file using the "Export contacts" action. This fails with a backtrace which is attached below. The error message is:
```
#6 /var/www/libertarian/vendor/pear...A user is trying to export data from an advanced query on custom fields to a CSV file using the "Export contacts" action. This fails with a backtrace which is attached below. The error message is:
```
#6 /var/www/libertarian/vendor/pear/db/DB/mysqli.php(936): DB_common->raiseError(-1, NULL, NULL, "\nINSERT INTO civicrm_tmp_d_export_1d387c75256673080ad64346b3c095c7 (`id`, `l...", "1406 ** Data too long for column 'custom_49' at row 13")
```
The `custom_49` field referred to in the error is a select list with a set of multiple choice options, some of which are up to 55 characters long.
The bug is that the temporary table that is being constructed to service the export to CSV is constructing the column as VARCHAR(32) while the text value of the multiple choice options in civicrm_option_value is a VARCHAR(512)
### Details
If I select the schema of the table `civicrm_tmp_d_export_1d387c75256673080ad64346b3c095c7` it reports the table structure as:
```
CREATE TABLE `civicrm_tmp_d_export_c3a74c8fb66b720f55e2661dab58716b` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`last_name` varchar(64) COLLATE utf8_unicode_ci DEFAULT NULL,
`first_name` varchar(64) COLLATE utf8_unicode_ci DEFAULT NULL,
`nick_name` varchar(128) COLLATE utf8_unicode_ci DEFAULT NULL,
`street_address` varchar(96) COLLATE utf8_unicode_ci DEFAULT NULL,
`city` varchar(64) COLLATE utf8_unicode_ci DEFAULT NULL,
`email` varchar(254) COLLATE utf8_unicode_ci DEFAULT NULL,
`phone` varchar(32) COLLATE utf8_unicode_ci DEFAULT NULL,
`sort_name` varchar(128) COLLATE utf8_unicode_ci DEFAULT NULL,
`custom_61` varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL,
`custom_50` varchar(32) COLLATE utf8_unicode_ci DEFAULT NULL,
`custom_51` varchar(16) COLLATE utf8_unicode_ci DEFAULT NULL,
`custom_68` varchar(32) COLLATE utf8_unicode_ci DEFAULT NULL,
`custom_58` varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL,
`custom_49` varchar(32) COLLATE utf8_unicode_ci DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `index_street_address` (`street_address`)
) ENGINE=InnoDB AUTO_INCREMENT=14 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci
```
Indeed it is trying to insert the text value of an option into a temporary table and several of those text options are over 32 characters in length.
```
SELECT label,CHAR_LENGTH(label) as length FROM civicrm.civicrm_option_value WHERE (option_group_id=103) AND (CHAR_LENGTH(label) > 32)
```
returns
```
| label | length |
|---------------------------------------------------------|--------|
| 003 - Aurora–Oak Ridges–Richmond Hill | 37 |
| 005 - Barrie–Springwater–Oro-Medonte | 36 |
| 035 - Haliburton–Kawartha Lakes–Brock | 37 |
| 039 - Hamilton West–Ancaster–Dundas | 35 |
| 040 - Hastings–Lennox and Addington | 35 |
| 052 - Leeds–Grenville–Thousand Islands and Rideau Lakes | 55 |
| 061 - Mississauga East–Cooksville | 33 |
| 073 - Northumberland–Peterborough South | 39 |
| 102 - Stormont–Dundas–South Glengarry | 37 |
```
The full backtrace may be relevant:
```
backTrace
#0 /var/www/libertarian/vendor/civicrm/civicrm-core/CRM/Core/Error.php(148): CRM_Core_Error::backtrace()
#1 /var/www/libertarian/vendor/pear/pear-core-minimal/src/PEAR.php(944): CRM_Core_Error::handle(Object(DB_Error))
#2 /var/www/libertarian/vendor/pear/db/DB.php(997): PEAR_Error->__construct("DB Error: unknown error", -1, 16, (Array:2), "\nINSERT INTO civicrm_tmp_d_export_1d387c75256673080ad64346b3c095c7 (`id`, `l...")
#3 /var/www/libertarian/vendor/pear/pear-core-minimal/src/PEAR.php(575): DB_Error->__construct(-1, 16, (Array:2), "\nINSERT INTO civicrm_tmp_d_export_1d387c75256673080ad64346b3c095c7 (`id`, `l...")
#4 /var/www/libertarian/vendor/pear/pear-core-minimal/src/PEAR.php(223): PEAR->_raiseError(Object(DB_mysqli), NULL, -1, 16, (Array:2), "\nINSERT INTO civicrm_tmp_d_export_1d387c75256673080ad64346b3c095c7 (`id`, `l...", "DB_Error", TRUE)
#5 /var/www/libertarian/vendor/pear/db/DB/common.php(1928): PEAR->__call("raiseError", (Array:7))
#6 /var/www/libertarian/vendor/pear/db/DB/mysqli.php(936): DB_common->raiseError(-1, NULL, NULL, "\nINSERT INTO civicrm_tmp_d_export_1d387c75256673080ad64346b3c095c7 (`id`, `l...", "1406 ** Data too long for column 'custom_49' at row 13")
#7 /var/www/libertarian/vendor/pear/db/DB/mysqli.php(406): DB_mysqli->mysqliRaiseError()
#8 /var/www/libertarian/vendor/pear/db/DB/common.php(1234): DB_mysqli->simpleQuery("\nINSERT INTO civicrm_tmp_d_export_1d387c75256673080ad64346b3c095c7 (`id`, `l...")
#9 /var/www/libertarian/vendor/civicrm/civicrm-packages/DB/DataObject.php(2696): DB_common->query("\nINSERT INTO civicrm_tmp_d_export_1d387c75256673080ad64346b3c095c7 (`id`, `l...")
#10 /var/www/libertarian/vendor/civicrm/civicrm-packages/DB/DataObject.php(1829): DB_DataObject->_query("\nINSERT INTO civicrm_tmp_d_export_1d387c75256673080ad64346b3c095c7 (`id`, `l...")
#11 /var/www/libertarian/vendor/civicrm/civicrm-core/CRM/Core/DAO.php(457): DB_DataObject->query("\nINSERT INTO civicrm_tmp_d_export_1d387c75256673080ad64346b3c095c7 (`id`, `l...")
#12 /var/www/libertarian/vendor/civicrm/civicrm-core/CRM/Core/DAO.php(1563): CRM_Core_DAO->query("\nINSERT INTO civicrm_tmp_d_export_1d387c75256673080ad64346b3c095c7 (`id`, `l...", TRUE)
#13 /var/www/libertarian/vendor/civicrm/civicrm-core/CRM/Export/BAO/Export.php(365): CRM_Core_DAO::executeQuery("\nINSERT INTO civicrm_tmp_d_export_1d387c75256673080ad64346b3c095c7 (`id`, `l...")
#14 /var/www/libertarian/vendor/civicrm/civicrm-core/CRM/Export/BAO/Export.php(199): CRM_Export_BAO_Export::writeDetailsToTable(Object(CRM_Export_BAO_ExportProcessor), (Array:100), (Array:14))
#15 /var/www/libertarian/vendor/civicrm/civicrm-core/CRM/Export/Form/Map.php(142): CRM_Export_BAO_Export::exportComponents(TRUE, (Array:118), (Array:8), "`sort_name` asc", (Array:14), NULL, 1, " contact_a.id IN ( 1546,1415,641,27,1688,1504,1701,42,650,626,56,1509,1537,15...", "civicrm_contact", 0, 0, (Array:11), "AND")
#16 /var/www/libertarian/vendor/civicrm/civicrm-core/CRM/Core/Form.php(513): CRM_Export_Form_Map->postProcess()
#17 /var/www/libertarian/vendor/civicrm/civicrm-core/CRM/Core/StateMachine.php(144): CRM_Core_Form->mainProcess()
#18 /var/www/libertarian/vendor/civicrm/civicrm-core/CRM/Core/QuickForm/Action/Next.php(43): CRM_Core_StateMachine->perform(Object(CRM_Contact_Export_Form_Map), "next", "Next")
#19 /var/www/libertarian/vendor/civicrm/civicrm-packages/HTML/QuickForm/Controller.php(203): CRM_Core_QuickForm_Action_Next->perform(Object(CRM_Contact_Export_Form_Map), "next")
#20 /var/www/libertarian/vendor/civicrm/civicrm-packages/HTML/QuickForm/Page.php(103): HTML_QuickForm_Controller->handle(Object(CRM_Contact_Export_Form_Map), "next")
#21 /var/www/libertarian/vendor/civicrm/civicrm-core/CRM/Core/Controller.php(347): HTML_QuickForm_Page->handle("next")
#22 /var/www/libertarian/vendor/civicrm/civicrm-core/CRM/Core/Invoke.php(312): CRM_Core_Controller->run((Array:4), (Array:0))
#23 /var/www/libertarian/vendor/civicrm/civicrm-core/CRM/Core/Invoke.php(68): CRM_Core_Invoke::runItem((Array:13))
#24 /var/www/libertarian/vendor/civicrm/civicrm-core/CRM/Core/Invoke.php(36): CRM_Core_Invoke::_invoke((Array:4))
#25 /var/www/libertarian/web/modules/contrib/civicrm/src/Civicrm.php(88): CRM_Core_Invoke::invoke((Array:4))
#26 /var/www/libertarian/web/modules/contrib/civicrm/src/Controller/CivicrmController.php(80): Drupal\civicrm\Civicrm->invoke((Array:4))
#27 [internal function](): Drupal\civicrm\Controller\CivicrmController->main((Array:4), "")
#28 /var/www/libertarian/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(123): call_user_func_array((Array:2), (Array:2))
#29 /var/www/libertarian/web/core/lib/Drupal/Core/Render/Renderer.php(573): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}()
#30 /var/www/libertarian/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(124): Drupal\Core\Render\Renderer->executeInRenderContext(Object(Drupal\Core\Render\RenderContext), Object(Closure))
#31 /var/www/libertarian/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(97): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->wrapControllerExecutionInRenderContext((Array:2), (Array:2))
#32 /var/www/libertarian/vendor/symfony/http-kernel/HttpKernel.php(151): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}()
#33 /var/www/libertarian/vendor/symfony/http-kernel/HttpKernel.php(68): Symfony\Component\HttpKernel\HttpKernel->handleRaw(Object(Symfony\Component\HttpFoundation\Request), 1)
#34 /var/www/libertarian/web/core/lib/Drupal/Core/StackMiddleware/Session.php(57): Symfony\Component\HttpKernel\HttpKernel->handle(Object(Symfony\Component\HttpFoundation\Request), 1, TRUE)
#35 /var/www/libertarian/web/core/lib/Drupal/Core/StackMiddleware/KernelPreHandle.php(47): Drupal\Core\StackMiddleware\Session->handle(Object(Symfony\Component\HttpFoundation\Request), 1, TRUE)
#36 /var/www/libertarian/web/core/modules/page_cache/src/StackMiddleware/PageCache.php(106): Drupal\Core\StackMiddleware\KernelPreHandle->handle(Object(Symfony\Component\HttpFoundation\Request), 1, TRUE)
#37 /var/www/libertarian/web/core/modules/page_cache/src/StackMiddleware/PageCache.php(85): Drupal\page_cache\StackMiddleware\PageCache->pass(Object(Symfony\Component\HttpFoundation\Request), 1, TRUE)
#38 /var/www/libertarian/web/core/lib/Drupal/Core/StackMiddleware/ReverseProxyMiddleware.php(47): Drupal\page_cache\StackMiddleware\PageCache->handle(Object(Symfony\Component\HttpFoundation\Request), 1, TRUE)
#39 /var/www/libertarian/web/core/lib/Drupal/Core/StackMiddleware/NegotiationMiddleware.php(52): Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, TRUE)
#40 /var/www/libertarian/vendor/stack/builder/src/Stack/StackedHttpKernel.php(23): Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, TRUE)
#41 /var/www/libertarian/web/core/lib/Drupal/Core/DrupalKernel.php(708): Stack\StackedHttpKernel->handle(Object(Symfony\Component\HttpFoundation\Request), 1, TRUE)
#42 /var/www/libertarian/web/index.php(19): Drupal\Core\DrupalKernel->handle(Object(Symfony\Component\HttpFoundation\Request))
#43 {main}
```https://lab.civicrm.org/dev/core/-/issues/2467[Meta] XLTS Angular JS longer-term-support2022-12-20T18:24:57Zhomotechsual[Meta] XLTS Angular JS longer-term-support@JoeMurray raised in chat that [AngularJS goes EoL in December](https://chat.civicrm.org/civicrm/pl/cm6jqmcex3n39ekf3zzx4tiree) there is a company [XLTS](https://xlts.dev/angularjs) providing extended LTS support through 2026. I've send ...@JoeMurray raised in chat that [AngularJS goes EoL in December](https://chat.civicrm.org/civicrm/pl/cm6jqmcex3n39ekf3zzx4tiree) there is a company [XLTS](https://xlts.dev/angularjs) providing extended LTS support through 2026. I've send them an enquiry asking how much they are likely to look to charge given the open-source nature of CiviCRM to see if, perhaps, they'd be willing to "donate" the XLTS to us.
We lose nothing by asking!JoeMurrayJoeMurrayhttps://lab.civicrm.org/dev/core/-/issues/2435Save fields button on export screen is not permissioned2023-05-24T06:21:51ZandyburnsSave fields button on export screen is not permissionedI cannot find in the code where the _Save Fields_ button is and therefore what permission is needed to save export mappings.
![image](/uploads/e9be7ed19215554edc7e6397dde46518/image.png)
Currently there is no core permission on who ca...I cannot find in the code where the _Save Fields_ button is and therefore what permission is needed to save export mappings.
![image](/uploads/e9be7ed19215554edc7e6397dde46518/image.png)
Currently there is no core permission on who can export. There is [this extension](https://github.com/progressivetech/net.ourpowerbase.exportpermission/blob/master/README.md) that hides the seach action of export but does not effect the actual export mappings screen.
Currently, non-admins try to save a field mapping and it just spins causing user confusion. What permission is currently needed to save field mappings? I think the logical solution is to get a permission added on saving field mappings button so users that don't have that permission do not see it.https://lab.civicrm.org/dev/financial/-/issues/168Preparing PayPal & 3DS2023-09-15T02:08:57Zjoshjosh@civicrm.orgPreparing PayPal & 3DSIn a recent discussion with PayPal we learned about changes coming to their product line as a result of current 3DS adoption in the UK, EU and expected adoption in the future. This appears to be most relevant to merchants using PayPal Pr...In a recent discussion with PayPal we learned about changes coming to their product line as a result of current 3DS adoption in the UK, EU and expected adoption in the future. This appears to be most relevant to merchants using PayPal Pro.
Currently, merchants in the UK and EU using PayPal Pro will experience an increase in declines as a result of 3DS enforcement. While the absolute number of CiviCRM users in both areas is low to modest, there are several rather large users, so the payment volume is sizable.
PayPal Pro does not support current 3DS parameters (version 1.0) and will not support future versions (version 2.0 arrives in October iirc). In essence, PayPal Pro does not support 3DS. In fact, they are phasing it out in favor of PayPal Commerce (venmo, apple pay, etc.), which is 3DS compliant.
PayPal expects 3DS enforcement to arrive in more countries such as CA and AU next, followed (eventually, ahem) by the US.
PayPal products that do not support direct card payment, i.e. Express and Standard, are not affected.
PayPal is advising partners to notify their merchants of the impending change and is recommending that partners build integrations with PayPal Commerce (or braintree, though they recommend Commerce for nonprofit organizations).https://lab.civicrm.org/dev/wordpress/-/issues/92Feature Request, CiviCRM Mailing, Public Page (ie. the page you use to read t...2023-02-09T16:50:04Zjustinfreeman (Agileware)Feature Request, CiviCRM Mailing, Public Page (ie. the page you use to read the email on-line) entirely bypass WordPress pages and are therefore rendered without website title, meta tags which negatively impacts sharing on social networksCurrently the CiviCRM Mailing, Public Page (ie. the page you use to read the email on-line) entirely bypass WordPress pages and are therefore rendered without website title, meta tags which negatively impacts sharing on social networks. ...Currently the CiviCRM Mailing, Public Page (ie. the page you use to read the email on-line) entirely bypass WordPress pages and are therefore rendered without website title, meta tags which negatively impacts sharing on social networks. See for example, https://civicrm.org/civicrm/mailing/view?reset=1&id=1686
If you share this page on a social network, it is expected by end users that you would see:
1. Organisation logo or at least a feature image
2. Short summary
3. Title of the newsletter
What you in fact see is very different. See for example https://developers.facebook.com/tools/debug/ using https://civicrm.org/civicrm/mailing/view?reset=1&id=1686
![screencapture-developers-facebook-tools-debug-2021-02-25-08_59_54](/uploads/d47e952a224080e34ad7d2ff08d99b3f/screencapture-developers-facebook-tools-debug-2021-02-25-08_59_54.png)
This is increasingly becoming a problem as CiviCRM sites re-share newsletters that they have recently sent out to their social networks.
The feature request is to provide a **Setting** which defines the **WordPress Page to be used to display the CiviCRM Mailing, Public Page** (ie. the page you use to read the email on-line). This is the same concept of the CiviCRM base page. On the CiviCRM Mailing page, a new shortcode would be present which is then used to render the public mailing.
The advantage of using a shortcode is that it then gives the website builder the ability to control how the page is presented, what metatags are used on that page and the template to be used for that page.
This is similar to the recent work to improve CiviCRM shortcode handling. See https://github.com/civicrm/civicrm-wordpress/pull/239 and https://lab.civicrm.org/dev/wordpress/-/issues/90
Ping @haystack and @kcristiano
Agileware Ref: CIVICRM-1662https://lab.civicrm.org/dev/core/-/issues/2414Deductible value treated as non-deductible if a non-deductible selection made...2023-11-23T07:09:32Zfreeform.stephDeductible value treated as non-deductible if a non-deductible selection made in the same contributionOriginally reported here: https://civicrm.stackexchange.com/questions/35284/non-deductible-value-incorrect-if-you-choose-a-nondeductible-membership-type-and
Possibly related to: https://lab.civicrm.org/dev/core/-/issues/1083
If you add...Originally reported here: https://civicrm.stackexchange.com/questions/35284/non-deductible-value-incorrect-if-you-choose-a-nondeductible-membership-type-and
Possibly related to: https://lab.civicrm.org/dev/core/-/issues/1083
If you add a donation field (financial type 'donation' which is deductible) to a membership price set that includes membership types that are nondeductible, and then use that price set for a membership contribution form, the non-deductible value on the contribution is saved as membership fee + donation amount instead of just the membership fee portion of the contribution. This means the non-deductible value can not be trusted.
We use the CDN Tax Receipts extension so to get around the issue we wiped out the nondeductible value from all relevant contributions and let the extension + a custom extension calculate the correct values.
How to replicate:
- add a test organization and test individual
- create a non-deductible financial type and a deductible financial type
- create 2 new membership types for your test organization with a fee and set one to the deductible type, and one to the non-deductible type
- create a price set with 2 fields:
- radio option with the 2 membership types
- radio option with a couple of donation values to choose from, financial type 'donation'
- create a new contribution form, default financial type your custom deductible financial type, and enable your price set on the memberships tab
- now use your new form to create a membership for your test individual and make sure to select the nondeductible membership type & a donation value.
- go to the individual's profile and pull-up the contribution on the contributions tab
To me, it would make sense if the non-deductible value included the donation amount if the contribution status were anything other than 'complete' (pending check for example, because we wouldn't want to issue a tax receipt for monies not receive), but once that status is set to complete then the nondeductible value should respect the financial types of the individual line items.![3X9oo](/uploads/71f07a5b7390c453e28a7ba9f94c54b6/3X9oo.png)seamusleeseamusleehttps://lab.civicrm.org/dev/financial/-/issues/167Document Support for CiviCRM from payment processors2024-03-07T16:39:04ZJoeMurrayDocument Support for CiviCRM from payment processorsDocument appropriately the support that CiviCRM receives from a variety of payment processors.
- [ ] Payment processing feature page that cites the sponsoring payment processors
- [ ] Individual payment processor pages similar to partn...Document appropriately the support that CiviCRM receives from a variety of payment processors.
- [ ] Payment processing feature page that cites the sponsoring payment processors
- [ ] Individual payment processor pages similar to partner detail pages
- [ ] Update the extension readme.md on extensions in gitlab
- [ ] Update extension pages on c.o (for as long as these will exist)
Josh has started on first two tasks as of Feb 22, 2021.
----
Original description
Document appropriately (after determining what that means ;) ) the support that CiviCRM receives from a variety of payment processors.
This might mean putting something into the info.xml or README.md of each extension, as well as adding something somewhere on c.o.
See https://github.com/agileware/cf-stripe/issues/1
@mattwire any sense of Stripe's compensation arrangements being different in Australia compared to other countries? I like the policy thrust of @justinfreeman 's comment but don't believe it is industry practice.joshjosh@civicrm.orgjoshjosh@civicrm.orghttps://lab.civicrm.org/dev/core/-/issues/2410Image custom field type2023-01-30T20:19:36ZMichael McAndrewImage custom field typeOverview
----------------------------------------
We want to add a custom field type of image that can be used to store images as part of a CiviCRM entity.
In terms of how this might be different to the file field type, I think the main...Overview
----------------------------------------
We want to add a custom field type of image that can be used to store images as part of a CiviCRM entity.
In terms of how this might be different to the file field type, I think the main consideration is how these fields would be displayed (resizing, thumbnails, etc.)
Example use-case
----------------------------------------
1. Store (one or more) images related to a contact as part of the contact record.
2. *** feel free to add more***
Current behaviour
----------------------------------------
At the moment, there is a single core field for a contact image (e.g. a headshot of an indiviual or a company logo). No other images can be added. Images can be added as files but are not displayed as one would expect an image to be displayed.
Proposed behaviour
----------------------------------------
People should be able to define image fields and upload images to them. The original should be retained and resized images should also be created (e.g. for thumbnails).
(Wireframes, mockups and more thinking required.)
Comments
----------------------------------------
Some relevant content (previous workarounds, etc.):
- https://drupal.stackexchange.com/questions/220599/how-to-add-image-field-in-civicrm
- https://civicrm.stackexchange.com/questions/15900/how-to-add-image-field-in-civicrmhttps://lab.civicrm.org/dev/core/-/issues/2396Translation support for Afforms & Managed Search Displays2024-02-07T17:45:35ZseamusleeTranslation support for Afforms & Managed Search DisplaysAs per https://lab.civicrm.org/extensions/afform/-/issues/22 and https://lab.civicrm.org/extensions/afform/-/issues/8 looks like there needs to be some improvements to support multilingual sites in AfformAs per https://lab.civicrm.org/extensions/afform/-/issues/22 and https://lab.civicrm.org/extensions/afform/-/issues/8 looks like there needs to be some improvements to support multilingual sites in Afformhttps://lab.civicrm.org/dev/core/-/issues/2382Allow disabling of confirmation page for paid events2022-10-16T01:24:38ZlarsssandergreenAllow disabling of confirmation page for paid eventsCurrently, a paid event must have a confirmation page. In simple paid event registration cases, it would be sometimes be preferable to skip the confirmation page (and potential drop-off of registrants or confusion from those who don't re...Currently, a paid event must have a confirmation page. In simple paid event registration cases, it would be sometimes be preferable to skip the confirmation page (and potential drop-off of registrants or confusion from those who don't realize that the confirmation page wasn't the end of the process) and simply register the participant directly.
Note that you can skip the confirmation page on a contribution page and this is a very similar situation (people are making a selection and filling in their payment details). We find our supporters are likely to be confused by multiple, verbose pages and the confirmation page definitely adds complexity that isn't needed in some cases. Are there any downsides to giving users the option to set up events without a confirmation page?https://lab.civicrm.org/dev/financial/-/issues/165Specify text for backend recurring receipts, also cc's and bcc's2022-12-19T03:45:27ZJoeMurraySpecify text for backend recurring receipts, also cc's and bcc'sIf a front end recurring donation is made, then the page's receipt text is added to every recurring receipt (from https://github.com/civicrm/civicrm-core/blob/master/xml/schema/Contribute/ContributionPage.xml#L326).
If a back end recurr...If a front end recurring donation is made, then the page's receipt text is added to every recurring receipt (from https://github.com/civicrm/civicrm-core/blob/master/xml/schema/Contribute/ContributionPage.xml#L326).
If a back end recurring donation is made, then the message template Contributions - Receipt (on-line) doesn't output anything from
```
{if $receipt_text}
<p>{$receipt_text|htmlize}</p>
{/if}
```
I think it it would make sense to have this additional text that is added at the top of the receipt specified for backend recurring contributions in core. It would involve a schema change to add a receipt_text field in civicrm_contributioin_recur after https://github.com/civicrm/civicrm-core/blob/master/xml/schema/Contribute/ContributionRecur.xml#L432 , and change the template above to use it.
(Could this be done in an extension, apart from the question of whether it is functionality that completes a feature? The hook at https://docs.civicrm.org/dev/en/latest/hooks/hook_civicrm_alterMailContent/ allows the text of a template to be altered. But there is no tooling for modifying a specific part of a template similar to how we've come to use js to modify a page's DOM rather than override its tpl. As a result, it's quite difficult to manage changes to the contents of core templates from extensions as I don't think there is a good way to 'modify its DOM' prior to sending. I suppose a regex every time a receipt is sent to insert the desired text is possible if a bit ugly.)
Simiilarly, there is no ability to cc or bcc these receipts for backend recurring contributions like there is for frontend ones. I'd add fields for this into civicrm_contribution_recur similar to those from civicrm_contribution_page.
I don't believe the is_email_receipt field on civicrm_contribution_recur is exposed on the backend contribution form (anymore). I think it could make sense to add it and the new receipt_text field just under the Receipt Date field on https://dmaster.demo.civicrm.org/civicrm/contact/view/contribution?reset=1&action=add&context=standalone&mode=live. A better approach if we are going to expose them, I think, is to hide these fields in a receipt fieldset, perhaps above Additional Details.
It's likely that many folks would prefer to just specify this once for all backend recurring donations, as it will include text similar to that on the manage contribution page settings for all front end recurring donations through that page. An alternative to adding these fields to the schema for civicrm_contribution_recur, and exposing them for each backend contribution, would be to make them into settings that could be set at Administer > CiviContribute > CiviContribute Component Settings.
Thoughts? Upvotes or even downvotes appreciated if you don't have time for a comment.
Cheers.seamusleeseamusleehttps://lab.civicrm.org/dev/core/-/issues/2357CiviCRM Dedupe Rules, default rules are incorrectly labelled as "and" when ea...2023-03-18T04:18:24Zjustinfreeman (Agileware)CiviCRM Dedupe Rules, default rules are incorrectly labelled as "and" when each rule is actually "or" eg. "Name and Email" should be "Name OR Email". Use of Name is ambiguous. Add "Nickname" to default Organisation dedupe rulesCiviCRM Dedupe Rules, default rules are incorrectly labelled as "and" when each rule is actually "or" eg. "Name and Email" should be "Name OR Email". Use of Name is ambiguous. Add "Nickname" to default Organisation dedupe rules.
Additio...CiviCRM Dedupe Rules, default rules are incorrectly labelled as "and" when each rule is actually "or" eg. "Name and Email" should be "Name OR Email". Use of Name is ambiguous. Add "Nickname" to default Organisation dedupe rules.
Additionally, Name is ambiguous in the context of Household and Organisation Contacts. Should be Household Name and Organisation Name. Organisations for example have Organisation Name and Nickname.
Suggested changes:
1. Change label to use "or" to match the dedupe criteria when or logic applies.
2. Remove ambiguity and be explicit about Name, ie. Household Name, Organisation Name and for Individuals: First Name and Last Name.
3. Add Nickname field to default Organisation dedupe rules.
![Screenshot_20210204_090434](/uploads/0c5b3b687825535aea7e8f0fdc814bf7/Screenshot_20210204_090434.png)
Agileware Ref: CIVICRM-1660https://lab.civicrm.org/dev/drupal/-/issues/155Installer does not recognize https urls when installing and setting civicrm.s...2021-02-04T20:02:47ZHeneryHInstaller does not recognize https urls when installing and setting civicrm.settings.php variablesTwo instances of url are in the settings file and when I install on an https site I need to edit the file manually to change the http to https.
Can the installer detect this and set the proper URL?Two instances of url are in the settings file and when I install on an https site I need to edit the file manually to change the http to https.
Can the installer detect this and set the proper URL?https://lab.civicrm.org/dev/core/-/issues/2342Search builder (NOT search kit) - Contribution search uses deprecated functio...2023-06-19T14:08:48ZDaveDSearch builder (NOT search kit) - Contribution search uses deprecated function CRM_Core_BAO_Query::getFieldName1. Search - Search Builder (not Search kit)
1. Do a contribution search. Let's say for Financial Type = Donation.
1. User deprecated function: Deprecated function CRM_Core_BAO_Query::getFieldName, use These parameters should be standardi...1. Search - Search Builder (not Search kit)
1. Do a contribution search. Let's say for Financial Type = Donation.
1. User deprecated function: Deprecated function CRM_Core_BAO_Query::getFieldName, use These parameters should be standardised before we get here. in CRM_Core_Error::deprecatedWarning() (line 1053 of /srv/buildkit/build/dmaster/web/sites/all/modules/civicrm/CRM/Core/Error.php).
1. The way deprecatedWarning works it loses the backtrace, but if you trace it it looks like this:
```
2 ...\CRM\Core\BAO\Query.php(97): CRM_Core_Error::deprecatedFunctionWarning("These parameters should be standardised before we get here")
3 ...\CRM\Contribute\BAO\Query.php(140): CRM_Core_BAO_Query::getFieldName((Array:5))
4 ...\CRM\Contribute\BAO\Query.php(114): CRM_Contribute_BAO_Query::whereClauseSingle((Array:5), Object(CRM_Contact_BAO_Query))
5 ...\CRM\Core\Component.php(284): CRM_Contribute_BAO_Query::where(Object(CRM_Contact_BAO_Query))
6 ...\CRM\Contact\BAO\Query.php(2061): CRM_Core_Component::alterQuery(Object(CRM_Contact_BAO_Query), "where")
7 \CRM\Contact\BAO\Query.php(574): CRM_Contact_BAO_Query->whereClause(FALSE)
8 ...\CRM\Contact\BAO\Query.php(523): CRM_Contact_BAO_Query->initialize(NULL)
9 ...\CRM\Contact\Selector.php(218): CRM_Contact_BAO_Query->__construct((Array:1), (Array:4), (Array:114), FALSE, FALSE, 1, FALSE, FALSE, FALSE, NULL, (Array:3))
10 ...\CRM\Contact\Form\Search.php(829): CRM_Contact_Selector->__construct(NULL, (Array:8), (Array:1), (Array:4), 8192, FALSE, FALSE, "builder", (Array:11))
11 ...\CRM\Contact\Form\Search\Builder.php(399): CRM_Contact_Form_Search->postProcess()
12 ...\CRM\Core\Form.php(513): CRM_Contact_Form_Search_Builder->postProcess()
```https://lab.civicrm.org/dev/translation/-/issues/63Italian translation of Household Member relationship doesn't specify which si...2021-01-29T10:58:50ZDaveDItalian translation of Household Member relationship doesn't specify which side the relationship is onIn english it's "Household Member of" to mean the individual is a member of a household (A to B), and "Household Member is" to mean the household has the individual as a member (B to A).
In italian it's "Membro del nucleo familiare" on ...In english it's "Household Member of" to mean the individual is a member of a household (A to B), and "Household Member is" to mean the household has the individual as a member (B to A).
In italian it's "Membro del nucleo familiare" on both sides, so it's not clear which way is which.
https://lab.civicrm.org/dev/translation/-/blob/14722d7fbae68eaea4214aa0f6966d635c1f4d91/po/it_IT/common-base.po#L25802-25808
Came up while looking at other stuff. I don't know what the right translation should be.https://lab.civicrm.org/dev/core/-/issues/2317CiviMail, Tracked Opens and Unique Opens should also be counted when click-th...2023-06-17T00:00:49Zjustinfreeman (Agileware)CiviMail, Tracked Opens and Unique Opens should also be counted when click-through events occurs to workaround issues where the tracked open, pixel is blocked by the email clientCiviMail, Tracked Opens and Unique Opens should also be counted when click-through events occurs to workaround issues where the tracked open, pixel is blocked by the email client. Thus enabling some statistics to be captured and avoid si...CiviMail, Tracked Opens and Unique Opens should also be counted when click-through events occurs to workaround issues where the tracked open, pixel is blocked by the email client. Thus enabling some statistics to be captured and avoid situations where Tracked Opens and Unique Opens have a statistic of 0 but all the other metrics have non-zero numbers.
PS. I had always thought that this was already the case, but discovered otherwise whilst investigating a support request. Quite surprised.
This would also require a documentation change.
Agileware Ref: CIVICRM-1649