Development issueshttps://lab.civicrm.org/groups/dev/-/issues2024-03-08T05:07:35Zhttps://lab.civicrm.org/dev/core/-/issues/3089Meta: Create list of items for moving a core component to an extension2024-03-08T05:07:35ZDaveDMeta: Create list of items for moving a core component to an extensionIt's never really been fully done before. Some of the Grant things are likely to come up again if it's not documented. Partial list just quickly compiled from tickets. Doesn't include the things that did work (which would still need doin...It's never really been fully done before. Some of the Grant things are likely to come up again if it's not documented. Partial list just quickly compiled from tickets. Doesn't include the things that did work (which would still need doing), and might be missing some PRs that had no tickets.
* https://lab.civicrm.org/dev/core/-/issues/3069
* https://lab.civicrm.org/dev/core/-/issues/3076
* https://lab.civicrm.org/dev/core/-/issues/3056
* https://lab.civicrm.org/dev/core/-/issues/3057
* https://lab.civicrm.org/dev/core/-/issues/3087
* https://lab.civicrm.org/dev/core/-/issues/3093
* https://lab.civicrm.org/dev/core/-/issues/3100
* https://github.com/civicrm/civicrm-buildkit/pull/677
* https://lab.civicrm.org/dev/core/-/issues/3101
* https://github.com/colemanw/webform_civicrm/pull/719
* extendedreports
* https://github.com/eileenmcnaughton/nz.co.fuzion.extendedreport/commit/6184de98c5b9f40475ab4164c993b19d68561240
* https://github.com/eileenmcnaughton/nz.co.fuzion.extendedreport/pull/503
* https://github.com/eileenmcnaughton/nz.co.fuzion.extendedreport/pull/506
* civicrm_entity
* https://github.com/eileenmcnaughton/civicrm_entity/pull/364
* an unresolved cache(?) issue
* https://github.com/twomice/com.joineryhq.jsumfields/pull/23
* https://github.com/civicrm/civicrm-drupal/pull/653
* https://github.com/civicrm/civicrm-drupal/pull/656
* https://github.com/civicrm/civicrm-core/pull/22905
* https://lab.civicrm.org/dev/core/-/issues/3118
* https://lab.civicrm.org/dev/core/-/issues/3119
* https://lab.civicrm.org/dev/core/-/issues/3159
* https://github.com/civicrm/civicrm-core/pull/23116
* https://github.com/civicrm/civicrm-core/pull/23118
* https://github.com/civicrm/civicrm-core/pull/23115
* https://lab.civicrm.org/dev/core/-/issues/3161
* https://github.com/civicrm/civicrm-core/pull/23336
* https://lab.civicrm.org/dev/core/-/issues/3485
* https://lab.civicrm.org/dev/core/-/issues/3492
* https://lab.civicrm.org/dev/core/-/issues/3503
* https://github.com/civicrm/civicrm-core/pull/24191
* https://github.com/civicrm/civicrm-core/pull/26118
Round 2:
* https://github.com/civicrm/civicrm-core/pull/26497
* https://github.com/civicrm/civicrm-core/pull/26499
Late to the party:
* https://lab.civicrm.org/dev/core/-/issues/5075https://lab.civicrm.org/dev/user-interface/-/issues/46Modernise ship-with-civi theme2023-09-20T15:55:57ZeileenModernise ship-with-civi themeI've opened this issue to see if we can agree on a way to improve the theme that ships with core (or to ship an additional theme with core) without the discussion spinning out into the technical sprawl that always seems to paralyse us on...I've opened this issue to see if we can agree on a way to improve the theme that ships with core (or to ship an additional theme with core) without the discussion spinning out into the technical sprawl that always seems to paralyse us on this issue...
**Problem statement**
CiviCRM ships with a theme looks dated and is off-putting to new adopters- some specific criticisms
- Colours are …. Beige. Current fashion would seem to be more white space (also drab in bootstrap)
- Button styling seems dated
- Some people seem to prefer side tabs - not sure if this is consensus
Goal of issue/discussion
- Find some achievable improvements
- Don’t get bogged down on solving everything
Potential solutions
- Improve the Greenwich theme - possibly as a paralell theme - addressing the most egregious issues (e.g just swapping colours / using more white makes it look more modern)
- Add an existing theme to core (Shoreditch, Aah, Finsbury park, Christian’s theme). Note that if we do this
1) it will mean that the theme becomes part of core codebase & would be maintained as such, with a priority place on maintainability and ensuring not too much css is downloaded (which might not always be in line with the designer’s vision).
2) bringing an existing theme into core would require the mainintainer agreeing to their theme being forked into a core extension & the core extension being maintained according to core maintenance priorities & principles. This may not be something current theme maintainers want as some design elements are likely to be sacrificed in the pursuit of maintainability / compatibility.
2) Anything that ships in core must work on all CMS and have acceptable page load speed for anonymouse users.
- Build up a minimal theme - ie what is the min theming we need to do to make it ‘load’ & move the rest of the css to greenwich (this is assuming a minimal theme would be better….)
Note that this ticket https://lab.civicrm.org/dev/user-interface/-/issues/33 covers previously discussion. Those discussions focussed on making it easier for themers to theme CiviCRM whereas the focus on this is what can we do to make the CiviCRM that ships with core look better.https://lab.civicrm.org/dev/core/-/issues/3083getNonDeductibleAmountFromPriceSet should take into account the financial typ...2023-12-05T21:36:19ZherbdoolgetNonDeductibleAmountFromPriceSet should take into account the financial type of the price field optionOverview
----------------------------------------
`getNonDeductibleAmountFromPriceSet()` gets called in a couple places to calculate the non-deductible amount of contributions. While the calling method will take into account the financi...Overview
----------------------------------------
`getNonDeductibleAmountFromPriceSet()` gets called in a couple places to calculate the non-deductible amount of contributions. While the calling method will take into account the financial type of the contribution when assigning the non-deductible amount, `getNonDeductibleAmountFromPriceSet()` does *not* take into account the financial type of the price set option. I believe it should be.
Reproduction steps
----------------------------------------
1. Create price set and add two fields: one with financial type Donation (deductible) = `$10` and other Event Fee (non-deductible) = `$20`. Make the Event Fee have a random non-deductible amount of say `$2.50`.
2. Add a contribution in the backend to a contact and choose this new price set.
3. Look at the resulting contribution.
Current behaviour
----------------------------------------
The contribution has non-deductible = `$2.50` for a `$30` contribution.
Expected behaviour
----------------------------------------
I expect the non-deducible to be `$20`, the full price of the Event Fee. The contribution should be setting the non-deductible to be equal that of the merchandise price field, regardless of the actual non-deductible amount for that field. If a price field is non-deductible then it is 100% non-deductible based on the financial type.https://lab.civicrm.org/dev/backdrop/-/issues/5Cron for multisite does not work2024-02-02T18:04:20ZcatorghansCron for multisite does not workWhen doing http cron (with .htaccess changes, see https://github.com/backdrop/backdrop-issues/issues/3903 ) or cli cron, on a multsite install, it uses the civicrm.settings.php from the root and not from the sites/[site]/ dir.
It looks ...When doing http cron (with .htaccess changes, see https://github.com/backdrop/backdrop-issues/issues/3903 ) or cli cron, on a multsite install, it uses the civicrm.settings.php from the root and not from the sites/[site]/ dir.
It looks like it does not expect a civicrm.settings.php in CMSroot if you have a multiinstall.
But to my understanding that should be allowed (and even normal)
In my case I have
[cmsroot]/civicrm.settings.php for the default site and also
[cmsroot]/sites/[site]/civicrm.settings.php for the 2nd domain.
When I look at civicrm.config.php it can not work.
When it finds a civicrm.settings.php in cmsroot it will not look further.https://lab.civicrm.org/dev/core/-/issues/3070Placeholders in forms2023-12-06T15:27:43ZshaneonabikePlaceholders in formsOverview
----------------------------------------
_Please describe your improvement in detail._
Example use-case
----------------------------------------
1. View a Contribution form
1. Fields such as _First name_ do not have a placehold...Overview
----------------------------------------
_Please describe your improvement in detail._
Example use-case
----------------------------------------
1. View a Contribution form
1. Fields such as _First name_ do not have a placeholder and only a label
Current behaviour
----------------------------------------
I think that in general providing placeholders is a good practice for all fields for the following reasons:
+ More accessible
+ Simplifies design (hidden labels) reduces the size of forms
Proposed behaviour
----------------------------------------
I was thinking there could be two ways to make this happen to start.
+ Provide placeholders that match the labels (Phase 1)
+ Provide ability on all fields with text entry to have a customizable placeholder (Phase 2)
Comments
----------------------------------------
Would there be any problem with me writing a patch that would allow the creation of placeholders for text holders that presently uses the label and copies that into a placeholder field?https://lab.civicrm.org/dev/core/-/issues/3068PDF filename on Pay later contributions is receipt.pdf2023-11-23T07:47:01Zluke.stewartPDF filename on Pay later contributions is receipt.pdfOverview
----------------------------------------
Currently if attach PDF copies to receipts is enabled and a contact completes a "pay later" contribution an email is sent with a pdf named 'receipt.pdf'
As a quick fix it is trivial to...Overview
----------------------------------------
Currently if attach PDF copies to receipts is enabled and a contact completes a "pay later" contribution an email is sent with a pdf named 'receipt.pdf'
As a quick fix it is trivial to detect in CRM/Contribute/BAO/ContributionPage::sendMail whether the payment is pay later or not and use this to change the PDF name.
We also need to check other workflows that trigger pdfs receipts.
These look to be CRM/Contribute/BAO/ContributionSoft.php which should be a similar fix to above - and also consider workflows for resending "receipts" where this is hardcoded as well.
Long term this value wouldn't be hard coded in this file but exposed through the UI as either a global setting - or potentially having a per contribution page override.
Additionally as it is currently this is not a translateable string either.
Reproduction steps
----------------------------------------
1. Enable "Attach PDF copy to receipts" via Admin -> System Settings -> Misc
1. Create a contribution page with a pay later and payment processor (dummy processor is fine) and add a $1 payment.
1. Under the Receipt options -> "Email Receipt to Contributor?" set true.
Current behaviour
----------------------------------------
When submitting the contribution form with "Pay Later" the email has "Invoice" in the subject - but the attachment filename is "receipt.pdf"
Expected behaviour
----------------------------------------
When submitting the contribution form with "Pay Later" the email has "Invoice" in the subject - but the attachment filename is "receipt.pdf"
Environment information
----------------------------------------
* __CiviCRM:__ _Master ( and Current Stable 5.46.2.)https://lab.civicrm.org/dev/core/-/issues/3067Improving performance of retreiving display names for multivalued custom data2022-11-25T15:39:20ZMichael McAndrewImproving performance of retreiving display names for multivalued custom data.Have been taking a look at https://github.com/civicrm/civicrm-core/pull/20764/files#diff-36ab608da0b5718996afd18c10a8c12653c2a1675e81fd3c4dc3e40c2ebb25d2R1040.
My hunch is that for a query that returns 50,000 rows, the 'group_concat fi....Have been taking a look at https://github.com/civicrm/civicrm-core/pull/20764/files#diff-36ab608da0b5718996afd18c10a8c12653c2a1675e81fd3c4dc3e40c2ebb25d2R1040.
My hunch is that for a query that returns 50,000 rows, the 'group_concat find_in_set' subquery is executed 50,000 times even if pagination is turned on since the limit is applied after it does all the subqueries
If you turn that subquery into a 'superquery', i.e. you do the 'group_concat find_in_set' hydration after the limit, it imrpoves the performance a lot.
I did a proof of concept below. For a query with 500 results it went from 37 seconds to 1.3 seconds.
And for 34,000 it went from never completing to 1.0 seconds.
Not sure how easy it would be to acheive this within the constraints of searchkit @colemanw but it would be great to get your initial thoughts on this.
Proof of concept changes to the query below...
Changing this query:
```sql
SELECT `a`.`id`, (
SELECT GROUP_CONCAT(
`display_name`
ORDER BY FIND_IN_SET(`civicrm_contact`.`id`, REPLACE(`Case_of_abuse_details_1`.`perpetrator_143`, '�', ','))
SEPARATOR '�'
)
FROM `civicrm_contact`
WHERE FIND_IN_SET(`civicrm_contact`.`id`, REPLACE(`Case_of_abuse_details_1`.`perpetrator_143`, '�', ','))
) AS `Case_of_abuse_details.Perpetrator.display_name`
FROM civicrm_activity a
LEFT JOIN `civicrm_value_case_of_abuse_68` `Case_of_abuse_details_1` ON a.id = Case_of_abuse_details_1.entity_id
WHERE ((`a`.`activity_type_id` IS NULL OR (`a`.`activity_type_id` IN (1, 2, 3, 4, 5, 9, 12, 13, 14, 15, 16, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 40, 44, 45, 48, 49, 50, 51, 52, 53, 55, 57, 59, 61, 63, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74)))) AND (`a`.`activity_type_id` = "68") AND (`a`.`is_test` = "0") AND (`a`.`is_deleted` = "0")
GROUP BY `a`.`id`
LIMIT 50
OFFSET 0
```
into this query
```sql
SELECT id,(SELECT GROUP_CONCAT(
`display_name`
ORDER BY FIND_IN_SET(`civicrm_contact`.`id`, REPLACE(`perpetrator_143`, '�', ','))
SEPARATOR '�'
)
FROM `civicrm_contact`
WHERE FIND_IN_SET(`civicrm_contact`.`id`, REPLACE(`perpetrator_143`, '�', ','))
) AS `Case_of_abuse_details.Perpetrator.display_name` FROM (SELECT `a`.`id`, `Case_of_abuse_details_1`.`perpetrator_143`
FROM civicrm_activity a
LEFT JOIN `civicrm_value_case_of_abuse_68` `Case_of_abuse_details_1` ON a.id = Case_of_abuse_details_1.entity_id
WHERE ((`a`.`activity_type_id` IS NULL OR (`a`.`activity_type_id` IN (1, 2, 3, 4, 5, 9, 12, 13, 14, 15, 16, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 40, 44, 45, 48, 49, 50, 51, 52, 53, 55, 57, 59, 61, 63, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74)))) AND (`a`.`activity_type_id` = "68") AND (`a`.`is_test` = "0") AND (`a`.`is_deleted` = "0")
GROUP BY `a`.`id`
LIMIT 50
OFFSET 0) AS alias
```https://lab.civicrm.org/dev/user-interface/-/issues/45Introduce a way to link event participants from the associated booking.2023-06-20T22:57:12ZBradley TaylorIntroduce a way to link event participants from the associated booking.When an event booking is made, the booking is stored across two primary entities:
- One or more participants
- One contribution
When you view an event participant, the list of associated payments is listed at the bottom of that screen:...When an event booking is made, the booking is stored across two primary entities:
- One or more participants
- One contribution
When you view an event participant, the list of associated payments is listed at the bottom of that screen:
![Screenshot_2022-02-04_at_08.46.02](/uploads/e8b76fabe84694810861ace5ecfef8b9/Screenshot_2022-02-04_at_08.46.02.png)
However, the reverse is not true. There is no way to easily get to the participants, when viewing a contriubtion:
![Screenshot_2022-02-04_at_08.46.41](/uploads/a1e19956ed2bda850fea737ca2eca33e/Screenshot_2022-02-04_at_08.46.41.png)
This can make it harder for administrators to see what a contribution has been used to purchase, and harder to see what additional participants the primary booker booked.
I've not looked into the feasability of either approach, but I can see a couple of obvious ways of fitting this into the UI.
**Option 1.** Link each event line item in the line items table to the relevant participant:
![Screenshot_2022-02-04_at_08.48.17](/uploads/cba5c2fb9239c82ecedaefe02adaa31d/Screenshot_2022-02-04_at_08.48.17.png)
(I'm not sure how this option would work with the line item editor extension)
**Option 2.** Introduce a new table of participants:
![Screenshot_2022-02-04_at_08.53.53](/uploads/c3565698dff7fae04900f47226526643/Screenshot_2022-02-04_at_08.53.53.png)
Similar thinking could be employed to link to the associated membership to a membership payment, but I thought I'd get the conversation going in relation to event bookings first.https://lab.civicrm.org/dev/core/-/issues/3050Error thrown when 'remove'ing from CiviContribute Batch2022-11-02T00:48:39ZmarshError thrown when 'remove'ing from CiviContribute BatchOverview
----------------------------------------
Civi throws an error when working with Accounting Batches. It's kind of hard to pin down - the front end gives the default / standard / very helpful
No response from the server. Check yo...Overview
----------------------------------------
Civi throws an error when working with Accounting Batches. It's kind of hard to pin down - the front end gives the default / standard / very helpful
No response from the server. Check your internet connection and try reloading the page.
Reproduction steps
----------------------------------------
- add a new accounting Batch (Contributions > Accounting Batches > New Batch)
- Assign a contribution to the batch
- try to remove the contribution from the batch
- at this point the message is displayed:
message, although ConfigAndLog does give a bit more, i.e.:
$Fatal Error Details = array:3 [
"message" => "Cannot delete EntityBatch with no id."
"code" => null
"exception" => CRM_Core_Exception {#2039
-errorData: array:1 [
"error_code" => 0
]
#cause: null
-_trace: null
#message: "Cannot delete EntityBatch with no id."
#code: 0
#file: "/var/www/html/drupal7/sites/all/modules/civicrm/CRM/Core/DAO.php"
#line: 958
trace: {
/var/www/html/drupal7/sites/all/modules/civicrm/CRM/Core/DAO.php:958 {
› if (empty($record['id'])) {
› throw new CRM_Core_Exception("Cannot delete {$entityName} with no id.");
› }
}
/var/www/html/drupal7/sites/all/modules/civicrm/CRM/Batch/BAO/EntityBatch.php:39 { …}
/var/www/html/drupal7/sites/all/modules/civicrm/CRM/Financial/Page/AJAX.php:207 { …}
/var/www/html/drupal7/sites/all/modules/civicrm/CRM/Utils/REST.php:266 { …}
/var/www/html/drupal7/sites/all/modules/civicrm/CRM/Utils/REST.php:550 { …}
/var/www/html/drupal7/sites/all/modules/civicrm/CRM/Core/Invoke.php:279 { …}
/var/www/html/drupal7/sites/all/modules/civicrm/CRM/Core/Invoke.php:69 { …}
/var/www/html/drupal7/sites/all/modules/civicrm/CRM/Core/Invoke.php:36 { …}
/var/www/html/drupal7/sites/all/modules/civicrm/drupal/civicrm.module:458 { …}
/var/www/html/drupal7/includes/menu.inc:527 { …}
/var/www/html/drupal7/index.php:21 { …}
}
}
]
This has alr3eady been [raised on stackechange](https://civicrm.stackexchange.com/questions/41126/entitybatch-with-no-id-error-when-remobing-items-from-accounting-batches?noredirect=1#comment48386_41126)
Current behaviour
----------------------------------------
![image](/uploads/f3431d6145caa54bbd5c96dec58ac3cf/image.png)
![image](/uploads/4dae3c200fc6696f0ecd6f5b25fd6747/image.png)
Expected behaviour
----------------------------------------
The item is removed from the batch
Environment information
----------------------------------------
- reproduced on dmaster
- using Chromium locally on a Debian boxhttps://lab.civicrm.org/dev/translation/-/issues/74ts() - Allow symbolic placeholders2022-01-27T13:24:34Ztottents() - Allow symbolic placeholdersOverview
---------
When writing translatable strings, placeholders must be numeric.
This leads to some quirky strings - eg if you're outputting a title and subtitle ("Some Chapter: Some Section"), the only way to encode that is `ts('%1...Overview
---------
When writing translatable strings, placeholders must be numeric.
This leads to some quirky strings - eg if you're outputting a title and subtitle ("Some Chapter: Some Section"), the only way to encode that is `ts('%1: %2')`. This string should be translatable because the layout is language dependent (eg LTR/RTL; eg punctuation-preferences), but the exported string will become meaningless when presented in Transifex (or similar). And it may be confused with other/similar strings.
Current behavior
-----------------
The function `ts(string $str, array $params)` accepts the following `$params`:
* Functional params (`count`, `plural`, `context`, `domain`, `escape`, `skip_translation`)
* Substitution params (numeric keys; `0`, `1`, `2`, ...)
As in:
```php
echo ts('Hello, %1!', [
1 => $name,
]);
echo '<li>' . ts('%1: %2', [
1 => $title,
2 => $subtitle,
]) . '</li>';
echo ts('The directory %1 is not writable. Please check your file permissions.', [
'plural' => 'The following directories are not writable: %1. Please check your file permissions.',
'count' => count($notWritableDirs),
1 => implode(', ', $notWritableDirs),
]),
```
```smarty
{ts 1=Bob}Hello %1!{/ts}
<li>{ts 1="Title" 2="Subtitle"}%1: %2{/ts}</li>
```
~~Proposed behavior (draft 1; symbol-prefix)~~
-----------------
Any `$params` which begin with a symbol (`%`, `_`, `{`, `@`, etc) will be used for variable substitution.
The earlier examples remain valid. Additionally, these examples are also valid:
```php
echo ts('Hello, %name!', [
'%name' => $name,
]);
echo '<li>' . ts('%title: %subtitle', [
'%title' => $title,
'%subtitle' => $subtitle],
) . '</li>';
echo ts('The directory %dir is not writable. Please check your file permissions.', [
'plural' => 'The following directories are not writable: %dir. Please check your file permissions.',
'count' => count($notWritableDirs),
'%dir' => implode(', ', $notWritableDirs),
]),
```
```smarty
{ts _NAME_=Bob}Hello _NAME_!{/ts}
<li>{ts _title_="Foo" _subtitle_="Bar"}_title_: _subtitle_{/ts}</li>
```
Proposed behavior (draft 2; capitalization-based)
-----------------
Any `$params` which begin with a capital letter (`[A-Z]`) will be used for variable substitution.
```php
echo ts('Hello, %NAME!', [
'NAME' => $name,
]);
echo '<li>' . ts('%TITLE: %SUBTITLE', [
'TITLE' => $title,
'SUBTITLE' => $subtitle],
) . '</li>';
echo ts('The directory %DIR is not writable. Please check your file permissions.', [
'plural' => 'The following directories are not writable: %DIR. Please check your file permissions.',
'count' => count($notWritableDirs),
'DIR' => implode(', ', $notWritableDirs),
]),
```
```smarty
{ts NAME=Bob}Hello %NAME!{/ts}
<li>{ts TITLE="Foo" SUBTITLE="Bar"}%TITLE: %SUBTITLE{/ts}</li>
```
Comments
---------
Either draft (using symbol-prefix or using uppercase) should avoid conflicts with existing strings+params. (Existing params do not begin with symbols and they are not uppercase.)
A couple factors in choosing a symbol:
* Using `%` is most consistent with current `ts()` style.
* `ts()` is used in multiple programming languages. Each has different compatibility/constraints.
* Pure PHP and pure JS are fairly flexible (`%FOO`, `{{FOO}}`, `_FOO_`, etc)
* Smarty-HTML (`{ts KEY=VALUE}...{/ts}`) is pretty limited. The bottleneck is how it parses the `KEY`.
* Angular-HTML (`{{ts('...')}}`) is in the middle. It's OK for `{{ts('%FOO')}}`, `{{ts('_FOO_')}}`, but I wouldn't count on `{{ts('{{FOO}}')}}`.
* If you want one symbol that works everywhere, `_` is probably it.
* Being flexible about symbol may be good for compatibility in the most contexts.
I suppose that the uppercase approach constrains style a bit, but maybe that's a good thing - ie you get more consistency in how `ts()` looks across environments.
Questions
---------
Are there any constraints in other layers -- eg Transifex, civistrings -- which would prevent this?
As far as I know, the only constraints which require using numeric-keys are baked into `ts()` itself.https://lab.civicrm.org/dev/core/-/issues/3040UI allows creation of unresolvable failure for status check "Relationship Typ...2023-11-16T16:09:55ZAllenShawUI allows creation of unresolvable failure for status check "Relationship Type Internal Name Duplicates"Admittedly, "unresolvable" is a bit of an overstatement, but you can create this situation in the UI, and then the UI provides no way to resolve it (it is resolvable via the API).
**To reproduce:**
1. Visit https://dmaster.demo.civicrm....Admittedly, "unresolvable" is a bit of an overstatement, but you can create this situation in the UI, and then the UI provides no way to resolve it (it is resolvable via the API).
**To reproduce:**
1. Visit https://dmaster.demo.civicrm.org/civicrm/admin/reltype?reset=1
1. Create a new relationship type with Label A "Foobar" and Label B "Employee Of"
1. Visit https://dmaster.demo.civicrm.org/civicrm/a/#/status and observe error: "Relationship Type Internal Name Duplicates;
Relationship type Employee of has the same internal machine name as another type.": ![error](/uploads/6d2eb70ff9aac5ce16ca948faee6d077/error.png)
1. Visit https://dmaster.demo.civicrm.org/civicrm/admin/reltype?reset=1 and edit the offending relationship (created in step 1); you can edit the labels but not the name. Best you can do is change "Employee of" to something else, like "Foobar of".
1. Visit https://dmaster.demo.civicrm.org/civicrm/a/#/status and observe that the error persists, exactly the same as in Step 3.
There's no way to change the name in the UI; you must use the API to change it.
**Proposed solution:**
* Upon creating a relationship type (other than reserved types created upon installation), use centralized logic to derive unique name from label; i.e., opt-in to the new behavior provided by https://github.com/civicrm/civicrm-core/pull/22508.
Thoughts?https://lab.civicrm.org/dev/core/-/issues/30315.44+ Unsubscribe from Smart Group form generates excessive process load2023-01-31T13:58:20Ztommybobo5.44+ Unsubscribe from Smart Group form generates excessive process loadOverview
----------------------------------------
The recent change to email unsubscribe from Smart Group is overloading mysql. [This change ](https://github.com/civicrm/civicrm-core/pull/21176) by @mattwire is providing a check to see w...Overview
----------------------------------------
The recent change to email unsubscribe from Smart Group is overloading mysql. [This change ](https://github.com/civicrm/civicrm-core/pull/21176) by @mattwire is providing a check to see which smart group an unsubscribing user is in before unsubscribing them. On a large mailing, site with a lot of smart groups, or even one complex smart this code is a time bomb. Especially as we see a rise in automated link checking bots on large email providers.
Current behavior
----------------------------------------
When a email recipient clicks unsubscribe all smart groups are regenerated. When CRM_Contact_BAO_GroupContactCache is called all smart groups are regenerated regardless of the setting of cache time. Also this is running even if the email doesn't contain any smart groups. Then the cache table is Left Joined onto the query for a search. Which also can be refreshed by another user clicking the link in the 5-10 second process time of a large site with large smart groups.
This is an incredibly high process load before any action is actually taken in order to display a group title on page that 9 times out of 10 is a bot not an end user.
Proposed behavior
----------------------------------------
Members do not need to see the Group labels from which they are unsubscribing, and if they are unsubscribing from this Email list then it is reasonable to mark them as unsubscribed from a smart group used in the mailing even if they are not in it. If they in the future become a member of the smart group, they probably don't want email about it.
It is reasonable to run this process AFTER the unsubscribe button is clicked by the user. Running it on page load is just wasting a ton resources on bot traffic.
Comments
----------------------------------------
@mattwire 's solution is reasonable for many, but it just isn't scaling well on larger civicrm installs. We have either reverted the code or switched to Opt Out of all emails for those larger clients, but those are temporary solutions.https://lab.civicrm.org/dev/core/-/issues/3027If you don't have permission to add contacts don't show ways to add contacts2022-11-15T17:49:39ZandyburnsIf you don't have permission to add contacts don't show ways to add contactsIn various places in Civi, you can either search for or add a contact when adding a contribution, participant, activity, etc. This works if you have the `add contacts` permission but not if you don't. The error is misleading too.
![add_...In various places in Civi, you can either search for or add a contact when adding a contribution, participant, activity, etc. This works if you have the `add contacts` permission but not if you don't. The error is misleading too.
![add_contact_form](/uploads/7d307dea4e7643e33db812398bc745e4/add_contact_form.png)
https://lab.civicrm.org/dev/core/-/blob/master/js/Common.js#L811https://lab.civicrm.org/dev/drupal/-/issues/173Leaking cacheability metadata2022-02-09T22:36:58ZfkohrtLeaking cacheability metadataAfter installing CiviCRM 5.45 on a fresh Drupal 9.3.2 running on Ubuntu 20.04.3 LTS, the [SAML Authentication](https://www.drupal.org/project/samlauth) module issues a warning after every successful authentication event:
> While process...After installing CiviCRM 5.45 on a fresh Drupal 9.3.2 running on Ubuntu 20.04.3 LTS, the [SAML Authentication](https://www.drupal.org/project/samlauth) module issues a warning after every successful authentication event:
> While processing SAML authentication response, code leaked cacheability metadata. This indicates a bug somewhere (but it is hard to pinpoint where): if the same code is called in other scenarios too, it may cause fatal crashes, or bloat the render cache unnecessarily. Please investigate. Metadata: i:6;:O:37:"Drupal\Core\Render\BubbleableMetadata":4:{s:16:"*cacheContexts";a:0:{}s:12:"*cacheTags";a:0:{}s:14:"*cacheMaxAge";i:-1;s:14:"*attachments";a:0:{}}
One of the maintainers of the `samlauth` module explains in a corresponding ticket:
> the samlauth module isn't leaking metadata. It's detecting that some other code in your website, which was executed during the login process, is leaking metadata and should be fixed.
>
> — roderik, “[Code leaked cacheability metadata](https://www.drupal.org/project/samlauth/issues/3232577)”, [comment #4](https://www.drupal.org/project/samlauth/issues/3232577#comment-14354950)
Before installing CiviCRM, no warnings appeared in the log, therefore I'd say the appearance of the warnings somehow correspond to the fact that CiviCRM has been installed: CiviCRM code for Drupal might leak cacheability metadata.https://lab.civicrm.org/dev/core/-/issues/3014Proposal: Explore adding hooks to allow plugins to track database queries, e....2023-04-23T20:13:36ZBradley TaylorProposal: Explore adding hooks to allow plugins to track database queries, e.g. to enable integration with plugins like Query MonitorOne of my favourite plugins in the WordPress space is [Query Monitor](https://wordpress.org/plugins/query-monitor/). Among other things it allows you to monitor at a glance what database calls have been made for a specific request, makin...One of my favourite plugins in the WordPress space is [Query Monitor](https://wordpress.org/plugins/query-monitor/). Among other things it allows you to monitor at a glance what database calls have been made for a specific request, making it easy to spot inefiencient code and queries.
Currently CiviCRM does not have anything analogous to this. The [CiviCRM docs recommend using the MySQL query log](https://docs.civicrm.org/dev/en/latest/tools/debugging/#viewing-a-query-log-from-mysql), but this isn't easy to view at a glance, and it doesn't make it easy to correlate queries to sepecific web requests. CiviCRM's database queries don't show in Query Monitor because CiviCRM does not use WordPress' `WPDB` class. Therefore, I've been looking to see if it would be possible to extend Query Monitor to know about CiviCRM's database queries. Its reasonably easy to make this work if you patch the `CRM_Core_DAO::query` method to the following:
```
public function query($query, $i18nRewrite = TRUE) {
// rewrite queries that should use $dbLocale-based views for multi-language installs
global $wpdb, $dbLocale, $_DB_DATAOBJECT;
if ( defined( 'SAVEQUERIES' ) && SAVEQUERIES ) {
$wpdb->timer_start();
}
if (empty($_DB_DATAOBJECT['CONNECTIONS'][$this->_database_dsn_md5])) {
// Will force connection to be populated per CRM-20541.
new CRM_Core_DAO();
}
$conn = &$_DB_DATAOBJECT['CONNECTIONS'][$this->_database_dsn_md5];
$orig_options = $conn->options;
$this->_setDBOptions($this->_options);
if ($i18nRewrite and $dbLocale) {
$query = CRM_Core_I18n_Schema::rewriteQuery($query);
}
$ret = parent::query($query);
if ( defined( 'SAVEQUERIES' ) && SAVEQUERIES ) {
$wpdb->num_queries++;
$wpdb->queries[] = array(
$query,
$wpdb->timer_stop(),
$wpdb->get_caller(),
$wpdb->time_start,
array(),
'trace' => new QM_Backtrace( array(
'ignore_frames' => 1,
)),
'result' => is_countable($ret) ? count($ret) : 1
);
}
$this->_setDBOptions($orig_options);
return $ret;
}
```
Esentially the `$wpdb->queries` array is being populated with knowledge of the CiviCRM database queries, which query monitor can then read.
With this in place you can start to see some interesting insights. For example, we can see that the new mailing screen contains a number of database queries that are reapeated 40 times each, which is probably ripe for optimisation:
![Screenshot_2022-01-02_at_14.13.41](/uploads/117036cbf7cb970e93c7399eeb436f64/Screenshot_2022-01-02_at_14.13.41.png)
The WordPress admin bar shows the total count of queries (243) combined from both CiviCRM and WordPress.
Hacking the core files works for my purposes, but obviously doesn't allow for this to be packaged into a distributable plugin. Therefore, I think it'd be cool if there were a generic hook at the end of `CRM_Core_DAO::query` which could be used to save the query information in a format ameanable to Query Monitor. Alternatively CiviCRM could copy WordPress idea of the `$wpdb->queries` array, creating an array of queries which plugins like Query Monitor could make use of (in WordPress this is behind a flag so there is no performance hit when not in a development environment).
I've not given too much thought to exactly how this hook should look, but before I take this further (e.g. to the Pull Request state) it'd be good to get some views from the community:
- Is this something others in the community would find useful if this were distributed as a plugin?
- Does anyone know of plugins similar to Query Monitor in the Drupal or Joomla space which may benifit from this type of solution?
- Would people find this sort of hook useful for other use cases - for example, for writing debugging tools native to CiviCRM.
Keen to get others views on this one. I'll be interested to know what people think.https://lab.civicrm.org/dev/core/-/issues/3010authx: Users that are disabled/blocked in the cms can still log in, even if t...2023-11-07T10:19:00ZDaveDauthx: Users that are disabled/blocked in the cms can still log in, even if the `XXX_user` setting is set to "require"At https://docs.civicrm.org/dev/en/latest/framework/authx/#settings you have a choice for whether the cms user needs to exist in order to log in, but even if you set it to "require", it will let a disabled/blocked user log in. My expecta...At https://docs.civicrm.org/dev/en/latest/framework/authx/#settings you have a choice for whether the cms user needs to exist in order to log in, but even if you set it to "require", it will let a disabled/blocked user log in. My expectation is that it would work the same way as regular logins if I choose "require" as the policy.
Some cms's may also allow other ways to block users, such as throttling or temporary IP bans. I would argue it should respect those too - i.e. if the cms doesn't allow login then authx shouldn't (when using "require").https://lab.civicrm.org/dev/core/-/issues/3008Load create cms user for checksum contact without uf_id2023-11-19T20:45:34ZjitendraLoad create cms user for checksum contact without uf_idTo replicate -
- Create a profile with Account creation required = 1.
- Add this profile to contribution or event page.
- Create a contact A without a CMS user record.
- Load the contribution page with checksum of contact A.
- The profi...To replicate -
- Create a profile with Account creation required = 1.
- Add this profile to contribution or event page.
- Create a contact A without a CMS user record.
- Load the contribution page with checksum of contact A.
- The profile wont load the username field even when the checksum contact does not have a cms user attached to it.
![image](/uploads/c6af3e369abe278101f21e831471ea98/image.png)https://lab.civicrm.org/dev/core/-/issues/2994Support oEmbed for external facing pages2024-03-21T18:50:44ZJoeMurraySupport oEmbed for external facing pagesAn important evolution of CiviCRM is to better support remote sites exposing CiviCRM forms and content.
Many platforms including WordPress and Drupal core since 8.6 support oEmbed, which allows a site to pull in content from a different...An important evolution of CiviCRM is to better support remote sites exposing CiviCRM forms and content.
Many platforms including WordPress and Drupal core since 8.6 support oEmbed, which allows a site to pull in content from a different site that exposes its content as oEmbed.
There is a good list of oEmbed providers, and CiviCRM should aspire to join it: https://oembed.com/providers.json
We should expose the following 'pages' as oEmbed content, using appropriate standard markup for use in WordPress and Drupal 8.6+ sites, and support the workflow for confirmation, thank you, and search results as appropriate:
1. Phase 1
1. Contribution page
2. Event info page
3. Event registration page
2. Phase 2
1. Something like a profile create/edit page that does not involve a payment but allows submission of a form to create a contact and activity and perhaps more (eg parent-child relationship)
2. Search Kit page
3. Petition page
3. Phase 3
1. Personal Campaign Page
2. Pages from extensions, eg Grant Application Page
3. CiviSurvey pages
Note: I haven't done the research to understand the details of oEmbed and potential challenges. I think this could be done through extension(s).
I'm posting this in order to generate discussion for a possible Roadmap item.https://lab.civicrm.org/dev/user-interface/-/issues/44Custom Field Groups Screen - Flip "Preview" and "Settings"2021-12-26T21:28:17ZJonGoldCustom Field Groups Screen - Flip "Preview" and "Settings"This is an "is it everyone or just me" situation - and I think we need to ask folks who are newer to CiviCRM. But personally, I don't use the "Preview" link on the Custom Field Groups, but I *do* use "Settings". It's pretty quick to fl...This is an "is it everyone or just me" situation - and I think we need to ask folks who are newer to CiviCRM. But personally, I don't use the "Preview" link on the Custom Field Groups, but I *do* use "Settings". It's pretty quick to flip them, so mainly looking for UX feedback.
@bgm Do you have the ability to grep the web server logs for Spark over a long enough time period to see which link is used more by Spark users?
More broadly - I think it's worth discussing Spark as a platform for collecting anonymized user behavior data (with consent and transparency obviously).JonGoldJonGoldhttps://lab.civicrm.org/dev/core/-/issues/2991Proposal: rework name fields to make name entity2023-12-13T09:20:59ZAndie HuntProposal: rework name fields to make name entity## Overview
A recent [discussion with Guy](https://chat.civicrm.org/civicrm/pl/wwqowz635fbpifio7on4e8r8dw) and #2883, plus the ongoing mayhem of organization names and personal experience with a variety of great and terrible name handli...## Overview
A recent [discussion with Guy](https://chat.civicrm.org/civicrm/pl/wwqowz635fbpifio7on4e8r8dw) and #2883, plus the ongoing mayhem of organization names and personal experience with a variety of great and terrible name handling systems, have led me to think that the solution to names isn't creating more or better-labeled fields for more types of names but rather to make name its own entity akin to phone, email, address, website, etc.
Once name is an entity, each contact could have multiple names, one primary, but with a bunch of possible types:
- legal name
- trade name
- nickname
- former name
- name in other language/culture
- name in an external system
- common mistakes
There could also be a bunch of name renderings - ways the name is displayed or inserted:
- sort name
- display name
- name for greetings
- name for listings
The latter could generate the existing greetings and potentially others. In particular, I think there's a real gap for how you insert someone's name in emails where you don't want "Dear", "Hi", or any other greeting as such.
Finally, the duplicate matching process could look for any of the possible names, cutting down on duplicates.
## Example use-case
### Organization has multiple names
An organization has their corporate name, colloquial name, and the name of their main product. These all refer to the same contact, but employees might fill the current employer field as one of the various ways. As an admin, you want to avoid having to merge them all the time. As a routine user, you will want to know all the various ways an organization is termed. As a bookkeeper, you will want to send invoices, checks, receipts, etc. with the correct entity name.
In this case, there are three name entities:
| Organization name | Name type | Is primary |
| ------------------------ | --------- | ---------- |
| Widget Productions | General | true |
| Widget Productions, Inc. | Legal | false |
| WidgetPro | General | false |
| Widgetizer | Product | false |
There would be a couple of name renderings:
| Rendering type | Value |
| -------------- | ------------------------ |
| Display Name | Widget Productions |
| Sort Name | Widget Productions |
| Billing name | Widget Productions, Inc. |
### Minister has variety of prefixes
Someone's name is styled differently when it is listed versus when they are addressed. In this case, a minister is styled "The Reverend Sally Jones" or "Rev. Sally Jones" when listed somewhere but her salutation is "Ms. Jones".
I think she'd have only one entry as a name entity but several renderings:
| Prefix | Title | First name | Middle Name | Last name | Suffix | Name type | Is primary |
|--------|--------------|------------|-------------|-----------|--------|-----------|------------|
| Ms. | The Reverend | Sally | Lynn | Jones | *null* | General | true |
These values are going to depend upon your sitewide preferences and choices for this contact, but a reasonable default might produce the following:
| Rendering type | Value |
|-----------------|--------------------------|
| Display Name | Sally Lynn Jones |
| Sort Name | Jones, Sally Lynn |
| Addressee | The Reverend Sally Jones |
| Email greeting | Dear Sally |
| Postal greeting | Dear Ms. Jones |
| Short name | Sally |
### An organization has a different name in another language
This should be self-explanatory:
| Organization name | Name type | Is primary |
|--------------------------|-----------|------------|
| Médecins Sans Frontières | General | true |
| Doctors Without Borders | Localized | false |
| Rendering type | Value |
|----------------|--------------------------|
| Display Name | Médecins Sans Frontières |
| Sort Name | Médecins Sans Frontières |
| English Name | Doctors Without Borders |
### Someone's family name comes before their given name
I'm on the fence about how this should be handled, but all I know is that we currently handle it terribly (except maybe in translations that assume it?) One option might be to call the fields first, middle, and last names according to position (*Ai* being the family name, but it comes first):
| Prefix | Title | First name | Middle Name | Last name | Suffix | Name type | Is primary |
|--------|--------|------------|-------------|-----------|--------|-----------|------------|
| Mr. | *null* | Ai | *null* | Weiwei | *null* | General | true |
Alternatively, we could name the fields according to function:
| Prefix | Title | Given name | Additional name | Surname | Suffix | Name type | Is primary |
|--------|--------|------------|-----------------|---------|--------|-----------|------------|
| Mr. | *null* | Weiwei | *null* | Ai | *null* | General | true |
In either case, the renderings should be like
| Rendering type | Value |
|-----------------|---------------|
| Display Name | Ai Weiwei |
| Sort Name | Ai Weiwei |
| Addressee | Mr. Ai Weiwei |
| Email greeting | Dear Weiwei |
| Postal greeting | Dear Mr. Ai |
| Short name | Weiwei |
### Clinic needs to retain legal name for insurance billing
Someone's real name doesn't match their legal name, let alone their nickname. While most organizations wouldn't want or need to record a contact's dead name, a clinic might need it for billing correspondence. In this case, a patient is named Stephen, or Steve for short, but his legal name is Barbara.
| Prefix | Title | First name | Middle Name | Last name | Suffix | Name type | Is primary |
|--------|--------|------------|-------------|-----------|--------|-----------|------------|
| Mr. | *null* | Stephen | Wayne | Smith | *null* | General | true |
| Mr. | *null* | Steve | *null* | Smith | *null* | Nickname | false |
| *null* | *null* | Barbara | Wayne | Smith | *null* | Legal | false |
All of the name renderings would use the primary name or nickname, but the clinic could add an additional name rendering for how they refer to patients when billing to the insurance company.
| Rendering type | Value |
|-----------------|----------------------|
| Display Name | Stephen Wayne Smith |
| Sort Name | Smith, Stephen Wayne |
| Addressee | Mr. Stephen Smith |
| Email greeting | Dear Steve |
| Postal greeting | Dear Mr. Smith |
| Short name | Steve |
| Insurance name | Barbara Smith |
### Household name possibilities
The current household name field befuddles lots of people and, in my opinion, makes households less attractive to use. If a household can have multiple names, it might be more useful.
You might use a Household name field that has everyone all listed out, or you could use the Last name field.
| Household name | Last name | Name type | Is primary |
|-------------------------------------|----------------|-----------|------------|
| Katherine Williams and Kendra Green | *null* | General | true |
| *null* | Williams-Green | General | false |
| Rendering type | Value |
|-----------------|------------------------------------------|
| Display Name | Katherine Williams and Kendra Green |
| Sort Name | Katherine Williams and Kendra Green |
| Addressee | The Williams-Green Household |
| Email greeting | Dear Katherine Williams and Kendra Green |
| Postal greeting | Dear Katherine Williams and Kendra Green |
Or if everyone shares a last name, a single name entity could do it all:
| Given name | Surname | Name type | Is primary |
|---------------|---------|-----------|------------|
| Jack and Jill | Hill | General | true |
(If we don't feel the need to stick with the legacy name field names, there's really no reason that "Given name" couldn't replace Organization name and Household name, too, so this illustrates that.)
| Rendering type | Value |
|-----------------|-------------------------|
| Display Name | Jack and Jill Hill |
| Sort Name | Hill, Jack and Jill |
| Addressee | The Hill Household |
| Email greeting | Dear Jack and Jill |
| Postal greeting | Dear Jack and Jill Hill |
## Current behavior
Right now, each of these situations is pretty annoying. It might require a separate custom field, located away from the name fields. It might make it difficult to automatically and accurately generate greetings for many people.
Currently, there are fields in the `civicrm_contact` table for each name part, plus nickname and legal name.
The only name renderings are display name and sort name (set sitewide), and addressee and email and postal greetings (site-wide presets and defaults, with custom options).
There's no provision for additional renderings or any way to do fallbacks like pick a nickname and fall back to a first name.
There's a limited number of alternative names for a contact.
Alternative names are not compared against primary names when deduping.
## Proposed behavior
### Schema
The name fields would be gone from the contact entity, and a new `civicrm_contact_name` entity is added with one of the following sets of fields:
| Traditional | Radical |
|-------------------|-------------------|
| id | id |
| contact_id | contact_id |
| contact_name_type | contact_name_type |
| is_primary | is_primary |
| first_name | given_name |
| middle_name | additional_name |
| last_name | surname |
| prefix_id | prefix_id |
| suffix_id | suffix_id |
| formal_title | formal_title |
| household_name | |
| organization_name | |
The `contact_name_type` would have reserved options General, Nickname, Legal, and Localized, but more could be added.
A new `civicrm_contact_rendering` entity is added with the following set of fields:
| Fields |
|-------------------------------|
| id |
| contact_id |
| contact_rendering_type |
| contact_rendering_template_id |
| value |
The `contact_rendering_type` would have reserved options Display name, Sort name, Addressee, Email greeting, Postal greeting, and Short name, but more could be added.
The `contact_rendering_template_id` would be akin to the existing `addressee_id`, `email_greeting_id`, and `postal_greeting_id`, while `value` would be the actual generated or custom value.
### Upgrade
On upgrade, the `civicrm_contact_name` would be populated with the main name fields, with nickname and legal name as additional rows. On upgrade, the `civicrm_contact_rendering` would be populated with rows for each contact's display name, sort name, addressee, and greetings.
### Contact rendering templates
The contact rendering templates would have the option to specify a contact name type to provide the default value, with a fallback to the primary contact name. For example, email greeting might default to "Dear {nickname.first_name}", where it picks the value of `first_name` from a contact's name entry with the name type of Nickname, and if that isn't available, it picks the primary name entry's value for `first_name`.
### Dedupe
When matching contacts on any of the name fields, the dedupe process would compare against all rows in the `civicrm_contact_name` field, regardless of `contact_name_type` or if it's the primary name.
## Comments
This is all very early in the process: I think it's important to do something like this, but I don't have my heart on the specific mechanisms here. I think it would need to be evaluated for performance in a variety of ways, but I don't expect it would have too huge of an impact.