Development issueshttps://lab.civicrm.org/groups/dev/-/issues2024-02-08T10:45:03Zhttps://lab.civicrm.org/dev/core/-/issues/4940Proposal to Change the Invoice date in the default Invoice message template t...2024-02-08T10:45:03ZeileenProposal to Change the Invoice date in the default Invoice message template to contribution.receive_dateThis spins off one discussion topic from https://lab.civicrm.org/dev/core/-/issues/1403 -
As part of our general effort to make the WorkflowMessage templates operate independently of the quickform layer (ie be available as actions from ...This spins off one discussion topic from https://lab.civicrm.org/dev/core/-/issues/1403 -
As part of our general effort to make the WorkflowMessage templates operate independently of the quickform layer (ie be available as actions from SearchKit etc) we should clarify which value is used for the invoice_date in the default message template that we ship from \`{$invoice_date}\`
```html
{domain.now|crmDate:"Full"}
or
{contribution.receive_date|crmDate:"Full"}
```
If we change the default it will affect new installs and un-customised templates. I'm not proposing any push upgrade on the customised templates at this stage so for most invoice-using sites this will be no change at this point. However, I am going to try to 'complete' the variable to token changes in this template so that anyone who is updating their customised template or installing a new site is using the tokens (& hence will be able to benefit from message previewability via MessageAdmin now & sending & rendering from outside QF if they install a relevant extension /when we add to core).
Potentially if we want to make it clear to people that they can change it we can use one of
```html
{domain.now|crmDate:"Full"}{* Code comment: - You can replace the domain.now token with this to show the contribution date instead {contribution.receive_date|crmDate:"Full"} *}
OR
{contribution.receive_date|crmDate:"Full"}{* Code comment: - You can replace the domain.now token with this to show the current date instead with {domain.now|crmDate:"Full"} *}
```
Note that the assumption in this gitlab is that any change would be opt in for existing sites (since basically everyone who uses invoices puts a logo in there). There is a proposal at https://github.com/civicrm/civicrm-core/pull/28630 that would force a change on everyone. I'm not convinced we should do a force update because it is pretty clear from the code & code history that the use of now was a deliberate attempt to meet a use case. However I have added an option to the emjoi poll below to allow people to vote for a forced update - basically a string_replace on upgrade.
EMOJI POLL Options
1) :boom: Change to {domain.now|crmDate:"Full"}
2) :man_dancing_tone2: Change to {contribution.receive_date|crmDate:"Full"}
3) :avocado: Change to 1 with additional in-code comments per above
4) :bellhop: Change to 2 with additional in code comments per above
5) :pick: Do 2 with a forced upgrade to replace {$invoice_date} with the above with {contribution.receive_date|crmDate:"Full"}https://lab.civicrm.org/dev/core/-/issues/4934Standalone: messages from Civi don't appear as expected2024-03-11T18:54:28ZAndy ClarkStandalone: messages from Civi don't appear as expectedMessages from Civi don't appear as expected e.g. when testing outbound email, check the mail() button and click 'Send and Save email' and no message will appear. This is 100% reproducible. Sometimes messages will appear a couple of scre...Messages from Civi don't appear as expected e.g. when testing outbound email, check the mail() button and click 'Send and Save email' and no message will appear. This is 100% reproducible. Sometimes messages will appear a couple of screens after they have been created, but not always. Using the 5.69.2 version of Standalone.RichRichhttps://lab.civicrm.org/dev/core/-/issues/4933Standalone: Cannot update default organisation to update it after Standalone ...2024-01-27T16:03:23ZAndy ClarkStandalone: Cannot update default organisation to update it after Standalone installAfter Standalone installation, it's require to update the default organisation to give it a name etc. However this crashes with the attached error. This is using the released 5.69.2 version of Standalone![screenshot_20240124_170636](/u...After Standalone installation, it's require to update the default organisation to give it a name etc. However this crashes with the attached error. This is using the released 5.69.2 version of Standalone![screenshot_20240124_170636](/uploads/c59bc5d9505b56cdb8764ffb36f0ad53/screenshot_20240124_170636.png)https://lab.civicrm.org/dev/coworker/-/issues/13Logging inadequate2024-01-25T14:19:24ZeileenLogging inadequateOur syslog is full of
Jan 23 22:03:34 coworker[1015]: #033[31m[2024-01-23T22:03:34.987520+00:00] CiviQueueWatcher.ERROR: Failed to read queues!#033[0m
The actual logged exception includes the exception but because this does not reach s...Our syslog is full of
Jan 23 22:03:34 coworker[1015]: #033[31m[2024-01-23T22:03:34.987520+00:00] CiviQueueWatcher.ERROR: Failed to read queues!#033[0m
The actual logged exception includes the exception but because this does not reach syslog because without logging to a file it winds up adding the ColorFormatter with no context or extra
https://lab.civicrm.org/dev/coworker/-/blob/master/src/Command/ConfigurationTrait.php#L168
The easiest fix would be to just add to the format line - eg
`$consoleFormatter = new ColoredLineFormatter(NULL, '[%datetime%] %channel%.%level_name%: %message%');`
`$consoleFormatter = new ColoredLineFormatter(NULL, '[%datetime%] %channel%.%level_name%: %message% %context% %extra%');`
or bring it into line with the LineFormatter used in file mode - ie pass in NULL & we get
` public const SIMPLE_FORMAT = "[%datetime%] %channel%.%level_name%: %message% %context% %extra%\n";`
There are various ways we can make this configurable but I lean towards passing in NULL for the format & using the defaults. As we don't have obvious other users it seems hard to make a case for a lot of work on configurability & when there is an obvious advantage of just using the package default over the existing default that feels like the easy / sane fix.
Related: - we have a lot of noise in our syslog from these two
Jan 23 12:45:17 coworker[1448965]: #033[31m[2024-01-23T12:45:17.882285+00:00] Pipe[ctl#1].ERROR: Received unexpected response line: "{"jsonrpc":"2.0","result":[],"id":667944}"#033[0m
Jan 23 12:45:19 coworker[1448965]: #033[31m[2024-01-23T12:45:19.882223+00:00] Pipe[ctl#1].ERROR: Received unexpected response line: "civicrm.wmf.INFO: Early return as queue is backed up. 529 contributions in the last 5 minutes is greater than the threshold of 500"#033[0m
In the former case I think the issue is that it should be a notice not an error - this seems to be what happens when coworker respawns as part of it's normal operation
In the latter case the issue is not that it gets logged - but that it gets logged every second. We might be able to figure out how to do something to pause the queue in this case but it would also be every second when the queue processing is in maintenance mode I think. Perhaps that is fine since that is a more unusual eventhttps://lab.civicrm.org/dev/core/-/issues/4929SK based on DB Entity includes contacts from the trash even when not specifie...2024-02-23T15:31:51ZlottieSK based on DB Entity includes contacts from the trash even when not specified as criteria## Overview
_When DB Entity is used as the 'search for' entity, it includes contacts from the trash even if not specified in the criteria of the original SK where the DB Entity display was constructed._
_This is the original post in _[...## Overview
_When DB Entity is used as the 'search for' entity, it includes contacts from the trash even if not specified in the criteria of the original SK where the DB Entity display was constructed._
_This is the original post in _[_Mattermost_](https://chat.civicrm.org/civicrm/pl/nji7p7prsid7dkf6p7ha3qksha)_._
## Reproduction steps
1. To test this you **_need to have some deleted contacts in your trash_**.
2. **Add a new** **SK** and do not change anything in the compose window (entity Contacts, everything default).
3. **Add a DB Entity display** to the SK you created in step 1 and preview it
4. **Save the SK** and **clear the cache** (I'm finding a have to clear the cache a 2-3 times)
5. **Add a new SK**
6. As the first **search for** entity, **_choose the DB Entity_** you created in step 2 and view the results
## Current behaviour
_The results returned for the child SK based on the DB Entity are higher than the results returned for the parent (first) SK. And the table created for the SK DB Entity the same number as shown in child SK. Note the result count in each of the following images. I have two contacts in the trash._
The SK created in step 1 (the parent for the DB Entity display - _Basic Contact Search_) has 201 results.
![sk-basic-contact-search_1.png](/uploads/c39cf6e5f33df445605656635569d70c/sk-basic-contact-search_1.png)
The SK where the DB Entity (_Basic Contact Search Child)_ created in step 1 is used as the **Search for** entity has 203 results.
![sk-basic-contact-search-child-based-on-db-entity_2.png](/uploads/7b7b5f3cc961fd0d3338fd06ea7726fa/sk-basic-contact-search-child-based-on-db-entity_2.png)
If I add the following where clause to the parent SK (_Basic Contact Search)_:
``WHERE (`a`.`is_deleted` = "0")``
![sk-basic-contact-search-addwhere-trash.png](/uploads/2a894589478f15154cebea240eaccbad/sk-basic-contact-search-addwhere-trash.png){width=491 height=178}
I get the correct result in my SK (_Basic Contact Search Child_) that is based on the DB Entity. The 201 results matches the original SK (_Basic Contact Search_).
![sk-basic-contact-search-child-updated.png](/uploads/ea5ebf91d8dc36c941cfa90a6409c411/sk-basic-contact-search-child-updated.png){width=525 height=272}
## Expected behaviour
_One should not be required to add the where clause to exclude the contacts in the trash. It seems as though that should be the default behaviour._
## Environment information
* **Browser:** _Firefox 59.0.1/Chrome 78.0.3904/Safari 13_
* **CiviCRM:** _5.69.2_
* **PHP:** _8.0_
* **CMS:** _Drupal 7.99_
* **Database:** _MariaDB 10.6_
* **Web Server:** _Apache 2.4_colemanwcolemanwhttps://lab.civicrm.org/dev/core/-/issues/4925Using crm_optgroup for headers in select lists causes the first option to be ...2024-01-27T23:04:48ZJamesStephensUsing crm_optgroup for headers in select lists causes the first option to be selected by default when the list is single selectOverview
----------------------------------------
I noticed this issue yesterday and tested it on https://demo.circle-interactive.co.uk/ to make sure it wasn't just my installation. If I use crm_optgroup to define headers in a select lis...Overview
----------------------------------------
I noticed this issue yesterday and tested it on https://demo.circle-interactive.co.uk/ to make sure it wasn't just my installation. If I use crm_optgroup to define headers in a select list of options in a custom field and that list is single select it doesn't work as expected. Rather than showing the "- Select [field label] -" it defaults to the first item under the crm_optgroup. This results in users accidentally submitting that if they are entering data in other fields.
Reproduction steps
----------------------------------------
1. Create a custom select field that will be single select
2. Add a crm_optgroup to the select options with some options below the optgroup
Current behaviour
----------------------------------------
The first option in the topmost crm_optgroup is selected as the default
Expected behaviour
----------------------------------------
It should show "- select [field label] -" instead in the field so that users can leave it blank when needed.
Environment information
----------------------------------------
Tested this in a couple of CiviCRM environments. Specific versions tested were 5.66.2 and 5.69.2
Tested in Firefox and Safari
Comments
----------------------------------------
Removing the optgroups (or making the list multiselect) corrects the problem.https://lab.civicrm.org/dev/core/-/issues/4924CiviReport: Contribution Detail Report Joining Incorrectly on civicrm_note2024-03-01T20:08:18Zcrawford.morganCiviReport: Contribution Detail Report Joining Incorrectly on civicrm_noteOverview
----------------------------------------
When 'Contribution Note' is selected as a column in a Contribution Detail report, the join is treated as required and all contributions in the results without notes are filtered out.
Thi...Overview
----------------------------------------
When 'Contribution Note' is selected as a column in a Contribution Detail report, the join is treated as required and all contributions in the results without notes are filtered out.
This issue occurs when using the 'Contribution Detail' report, 'Extended Report - Contributions' report, and 'Extended Report - Bookkeeping with extra fields' report.
Thankfully, reports created in SearchKit function properly (all contributions are returned in the results) and contribution search exports are also functioning properly (including 'Contribution Note' in the exported fields does not affect the records exported).
Reproduction steps
----------------------------------------
1. example.com/civicrm/report/contribute/detail?reset=1
2. View results with Columns > Contribution Note toggled off
3. Toggle Contribution Note on, run report again, note that fewer records are returned
Environment information
----------------------------------------
* CiviCRM: 5.69.2
* PHP:8.1.27
* Drupal: 7.99
* MySQL: 8.0.33
Comments
----------------------------------------
This only recently became an issue after making a minor upgrade to 5.69.2https://lab.civicrm.org/dev/core/-/issues/4921Add Soft Credit Recipient's and Contributor's Postal Address fields to includ...2024-01-29T10:13:28ZyashodhaAdd Soft Credit Recipient's and Contributor's Postal Address fields to include in the Soft Credit Report.Add Soft Credit Recipient's and Contributor's Postal Address fields to include in the Soft Credit Report. We already have phone/email having address would be helpful.Add Soft Credit Recipient's and Contributor's Postal Address fields to include in the Soft Credit Report. We already have phone/email having address would be helpful.https://lab.civicrm.org/dev/core/-/issues/4919Update CiviCRM on Drupal 10: Deprecation Notices regarding TOGoS2024-01-25T14:10:46ZDetlev SieberUpdate CiviCRM on Drupal 10: Deprecation Notices regarding TOGoS## Overview
During the updating process of CiviCRM with Drupal 10.2.2, Composer shows several Deprecation Notices regarding the package "TOGoS"
## Reproduction steps
1. Update CiviCRM on Drupal 10.2.2 using Composer
2. Console output ...## Overview
During the updating process of CiviCRM with Drupal 10.2.2, Composer shows several Deprecation Notices regarding the package "TOGoS"
## Reproduction steps
1. Update CiviCRM on Drupal 10.2.2 using Composer
2. Console output includes numerous Depreation Notices regarding TOGos
## Current behaviour
See:
`Deprecation Notice: Creation of dynamic property TOGoS_GitIgnore_FileFinder::$ruleset is deprecated in /usr/www/users/crmeay6f/crm.adb.de/vendor/togos/gitignore/src/main/php/TOGoS/GitIgnore/FileFinder.php:7`
... and another \~ 100 lines like this one
## Expected behaviour
There should be no deprecation notices displayed
## Environment information
* **CiviCRM:** update to 5.69.2
* **PHP:** _8.2.14_
* **CMS:** _Drupal 10.2.2_
##https://lab.civicrm.org/dev/core/-/issues/4917Contribution Radio Buttons Incorrectly add other Amount2024-01-28T22:13:07ZtreseroContribution Radio Buttons Incorrectly add other Amount## Overview
With the focus fix for Contribution (Main.tpl), there is still an issue with the Other Amount being added to the original, default amount.
The Other Amount should not be adding to the original, unchecked amount.
## Reprodu...## Overview
With the focus fix for Contribution (Main.tpl), there is still an issue with the Other Amount being added to the original, default amount.
The Other Amount should not be adding to the original, unchecked amount.
## Reproduction steps
1. Click on your public facing Contribute page, civicrm/contribute/transact/?reset=1&id=1
2. Clicked in Other Amount text box and added $20
3. This added $20 to the default amount resulting in $70 for the amount, instead of $20
## Current behaviour
## Expected behaviour
This should clear the initial amount and only show the Other Amount, and move focus to the Other Amount button.
## Environment information
* **CiviCRM:** _5.69.2_
* **PHP:** _8.1_
* **CMS:** _Wordpress_
## Comments
![Screenshot 2024-01-16 161324.png](/uploads/68f7a46cac54e4256937d6d472a868c2/Screenshot_2024-01-16_161324.png)5.70.0https://lab.civicrm.org/dev/core/-/issues/4916Invisible notes2024-02-23T15:31:02ZBohdanDmytryshynInvisible notesOverview
----------------------------------------
On install or update CiviCRM, null-valued notes creating.
After researching, I can assume that the problem is a misassignment of keys.
![image](/uploads/9f4fb6a13d4cf2460a5a5c87d238b3d6/...Overview
----------------------------------------
On install or update CiviCRM, null-valued notes creating.
After researching, I can assume that the problem is a misassignment of keys.
![image](/uploads/9f4fb6a13d4cf2460a5a5c87d238b3d6/image.png)
Environment information
----------------------------------------
* __CiviCRM:__ _Master/5.67.1_
* __PHP:__ _/7.1/8.1_
* __CMS:__ Joomla 4.4.2/WordPress 6.4.2/..._https://lab.civicrm.org/dev/core/-/issues/4908Integrate Angular calendar for better usability2024-01-15T11:36:56ZshaneonabikeIntegrate Angular calendar for better usabilityI don't mind the existing calendar date picker, but I was wondering if there are other solutions that might work a bit better for usability. I came across this [Angular version](https://laros.io/using-angular-material-calendar-with-date-...I don't mind the existing calendar date picker, but I was wondering if there are other solutions that might work a bit better for usability. I came across this [Angular version](https://laros.io/using-angular-material-calendar-with-date-ranges-and-range-presets) (maybe there is an alternative)
Since we are already starting to use a lot of Angular I thought it could be something to consider. There is even a date range option, which could be interesting for other areas.
You can spin a demo on this stackblitz https://stackblitz.com/edit/angular-material-calendar-with-date-ranges-and-presets?file=package.json
What I like about it:
* When choosing a different year it automatically prompts to choose the month which is fairly logical and usable
* There is the possibility to set ranges (future integration with Events?)
* There is the option to have quick options like Last 30 days, Last 12 months, etc.
![Angular Material Calendar with Date Range and Range Presets](https://laros.io/images/angular-material-calendar-date-range.png)
Anyway it was just something I wanted to raise but it's not a high priority but could be a great enhancement.https://lab.civicrm.org/dev/core/-/issues/4907Event full inconsistencies2024-01-15T11:35:24ZeileenEvent full inconsistenciesThe way EventFull is calculated in apiv4 is not consistent with elsewhere
apiv4 returns a field `'remaining_participants'` which is
event.max_participants - COUNT(participant rows for the event where contact is not deleted, participant ...The way EventFull is calculated in apiv4 is not consistent with elsewhere
apiv4 returns a field `'remaining_participants'` which is
event.max_participants - COUNT(participant rows for the event where contact is not deleted, participant is not test and participant status is_counted)
By contrast apiv3 returns the contents of `CRM_Event_BAO_Participant::eventFull()` which also takes into account the participant role and other aspects of the status
`eventFull()` has some funky parameters - there are 12 calls to it
| caller | returnEmptySeats |includeWaitingList|returnWaitingCount|considerTestParticipant|onlyPositiveStatuses|
| ------ | ------ |------ |------ |------ |------ |
| apiv3-event | TRUE |TRUE|FALSE|FALSE|FALSE|
| Participant::eventFullMessage | FALSE |FALSE|FALSE|FALSE|FALSE|
| Participant::eventFullMessage(2) | FALSE |TRUE|TRUE|FALSE|FALSE|
|ParticipantStatusType::process|TRUE|FALSE|FALSE|FALSE|
|ParticipantTest|FALSE|TRUE|FALSE|FALSE|FALSE|
|Registration::preProcess|FALSE|if-event-has-waitlist|FALSE|FALSE|FALSE|
|Registration::preProcess(2)|TRUE|if-event-has-waitlist|FALSE|FALSE|FALSE|
|Event_Confirm::formRule|FALSE|if-event-has-waitlist|FALSE|FALSE|FALSE|
|ParticipantConfirm|TRUE|FALSE|TRUE|FALSE|TRUE|
|Register::preProcess|FALSE|if-event-has-waitlist|FALSE|FALSE|FALSE|
|EventInfo::run|FALSE|if-event-has-waitlist|FALSE|FALSE|FALSE|
|event-cart|TRUE|TRUE|FALSE|FALSE|FALSE|
Note that this issue somewhat relates/ explains
https://lab.civicrm.org/dev/event/-/issues/23https://lab.civicrm.org/dev/core/-/issues/4906UI to make multivalue custom field sets on any entity2024-01-17T18:16:59ZMichael McAndrewUI to make multivalue custom field sets on any entityFollowing https://github.com/civicrm/civicrm-core/pull/27549 it appears that the ability to make custom data multi-value is missing from the create custom data set UI
![image](/uploads/e6c5ab54c7c80e33b34c206c41c1339a/image.png)
The ab...Following https://github.com/civicrm/civicrm-core/pull/27549 it appears that the ability to make custom data multi-value is missing from the create custom data set UI
![image](/uploads/e6c5ab54c7c80e33b34c206c41c1339a/image.png)
The above screenshot shows the form with the missing field. I would expect all entities to now show the 'Does this Custom Field Set allow multiple records?' checkbox as per the screenshot below (taken when the entity was set to individual).
![image](/uploads/38e5b73c0ebf6f44638c36b8bf1e262e/image.png)
@colemanw - you might have some thoughts on this. Guessing it might have something to do with the ' (in theory)' ending to https://github.com/civicrm/civicrm-core/pull/27549https://lab.civicrm.org/dev/core/-/issues/4902Unnecessary Event Breadcrumb When Editing Event2024-02-01T10:45:52ZthemakUnnecessary Event Breadcrumb When Editing EventWhen editing an event - the breadcrumbs that display are:
CiviCRM > CiviEvent Dashboard > Manage Events > Manage Events
The first 3 make sense, but the last one actually links to the edit event page that you are editing. While I have se...When editing an event - the breadcrumbs that display are:
CiviCRM > CiviEvent Dashboard > Manage Events > Manage Events
The first 3 make sense, but the last one actually links to the edit event page that you are editing. While I have seen breadcrumbs include the current page you are on, usually they are not linked and sometimes greyed out. To stay consistent with other parts of civi, I would remove the last breadcrumb. If we decide to keep for some reason - at very least it should say Configure Event - not Manage Event.
5.69.1 - also replicated in 5.71.alpha1
![Screenshot_2024-01-10_at_12.52.30_PM](/uploads/d6048cb7a5167aab82e3dca9c2aba45d/Screenshot_2024-01-10_at_12.52.30_PM.png)https://lab.civicrm.org/dev/core/-/issues/4900Crash when disabling an extension that provides an entity that is referenced ...2024-01-15T11:40:21ZufundoCrash when disabling an extension that provides an entity that is referenced in a custom fieldReproduction steps
----------------------------------------
1. Enable an extension which adds a new entity
1. Add a Custom Field through the UI which is an EntityReference to the entity from the extension
1. Disable the extension
2. Cras...Reproduction steps
----------------------------------------
1. Enable an extension which adds a new entity
1. Add a Custom Field through the UI which is an EntityReference to the entity from the extension
1. Disable the extension
2. Crash ensues
Current behaviour
----------------------------------------
Every Civi page crashes with
```
The website encountered an unexpected error. Please try again later.
TypeError: CRM_Core_DAO_AllCoreTables::getTableForEntityName(): Return value must be of type string, null returned in CRM_Core_DAO_AllCoreTables::getTableForEntityName() (line 365 of /var/www/html/vendor/civicrm/civicrm-core/CRM/Core/DAO/AllCoreTables.php).
Civi\Api4\Service\Schema\SchemaMapBuilder::getTableName('MyCustomEntity') (Line: 149)
Civi\Api4\Service\Schema\SchemaMapBuilder->addCustomFields(Object, Object, 'Contact') (Line: 74)
Civi\Api4\Service\Schema\SchemaMapBuilder->loadTables(Object) (Line: 52)
Civi\Api4\Service\Schema\SchemaMapBuilder->build() (Line: 309)
Civi\Api4\Utils\CoreUtil::getSchemaMap() (Line: 773)
Civi\Api4\Query\Api4SelectQuery->autoJoinFK('dashboard_contact.id') (Line: 183)
Civi\Api4\Query\Api4SelectQuery->buildSelectClause() (Line: 79)
Civi\Api4\Query\Api4Query->getSql() (Line: 90)
Civi\Api4\Query\Api4Query->getResults() (Line: 106)
Civi\Api4\Query\Api4SelectQuery->run() (Line: 107)
Civi\Api4\Generic\DAOGetAction->getObjects(Object) (Line: 94)
Civi\Api4\Generic\DAOGetAction->_run(Object) (Line: 72)
Civi\Api4\Provider\ActionObjectProvider->invoke(Object) (Line: 156)
Civi\API\Kernel->runRequest(Object) (Line: 256)
Civi\Api4\Generic\AbstractAction->execute() (Line: 91)
civicrm_api4('Dashboard', 'get', Array) (Line: 69)
CRM_Core_BAO_Dashboard::getContactDashlets() (Line: 46)
CRM_Contact_Page_DashBoard->run(Array, NULL) (Line: 322)
CRM_Core_Invoke::runItem(Array) (Line: 69)
CRM_Core_Invoke::_invoke(Array) (Line: 36)
CRM_Core_Invoke::invoke(Array) (Line: 88)
Drupal\civicrm\Civicrm->invoke(Array) (Line: 83)
Drupal\civicrm\Controller\CivicrmController->main(Array, '')
call_user_func_array(Array, Array) (Line: 123)
Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}() (Line: 592)
Drupal\Core\Render\Renderer->executeInRenderContext(Object, Object) (Line: 124)
Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->wrapControllerExecutionInRenderContext(Array, Array) (Line: 97)
Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}() (Line: 181)
Symfony\Component\HttpKernel\HttpKernel->handleRaw(Object, 1) (Line: 76)
Symfony\Component\HttpKernel\HttpKernel->handle(Object, 1, 1) (Line: 58)
Drupal\Core\StackMiddleware\Session->handle(Object, 1, 1) (Line: 48)
Drupal\Core\StackMiddleware\KernelPreHandle->handle(Object, 1, 1) (Line: 106)
Drupal\page_cache\StackMiddleware\PageCache->pass(Object, 1, 1) (Line: 85)
Drupal\page_cache\StackMiddleware\PageCache->handle(Object, 1, 1) (Line: 48)
Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object, 1, 1) (Line: 51)
Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object, 1, 1) (Line: 51)
Drupal\Core\StackMiddleware\StackedHttpKernel->handle(Object, 1, 1) (Line: 704)
Drupal\Core\DrupalKernel->handle(Object) (Line: 19)
```
Expected behaviour
----------------------------------------
I think it would be good to check for any custom fields referencing entities that are about to disappear before disabling the extension, and then:
a) disable those custom fields
OR
b) prevent you disabling the extension
Comments
----------------------------------------
I wonder if some very diligent extension authors may already handle this case in the extension disable logic and/or maybe there should be an option c) resolve the issue in some custom way specified by the extension?
Or maybe the `SchemaMapBuilder` could fail less catastrophically?
Only tested on D10.https://lab.civicrm.org/dev/core/-/issues/4891[PHP 8.1] Weight notices @ trash contact folder2024-02-09T20:39:06Zjofranzfranz@systopia.de[PHP 8.1] Weight notices @ trash contact folderOverview
----------------------------------------
_Weight notices @ trash contact folder_
Reproduction steps
----------------------------------------
1. Delete a contact in a non-permanent way
2. Go to: https://dmaster.demo.civicrm.org/...Overview
----------------------------------------
_Weight notices @ trash contact folder_
Reproduction steps
----------------------------------------
1. Delete a contact in a non-permanent way
2. Go to: https://dmaster.demo.civicrm.org/civicrm/contact/search/advanced (user/pass: demo demo)
3. Select: "Search in Trash
(deleted contacts)" (Important!!!)
4. Hit "Search"
Current behaviour
----------------------------------------
Many weight notices:
```
Warning: Undefined array key "weight" in CRM_Core_Action::{closure}() (line 318 of /srv/buildkit/build/dmaster/web/sites/all/modules/civicrm/CRM/Core/Action.php).
Warning: Undefined array key "weight" in CRM_Core_Action::{closure}() (line 318 of /srv/buildkit/build/dmaster/web/sites/all/modules/civicrm/CRM/Core/Action.php).
Warning: Undefined array key "weight" in CRM_Core_Action::{closure}() (line 318 of /srv/buildkit/build/dmaster/web/sites/all/modules/civicrm/CRM/Core/Action.php).
Warning: Undefined array key "weight" in CRM_Core_Action::{closure}() (line 318 of /srv/buildkit/build/dmaster/web/sites/all/modules/civicrm/CRM/Core/Action.php).
```
Expected behaviour
----------------------------------------
No notices.
Environment information
----------------------------------------
* __Browser:__ _Firefox_
* __CiviCRM:__ _5.68.1 (guess also below and above)_
* __PHP:__ _8.1+_
* __CMS:__ _Drupal10_
* __Database:__ _WhateverDB_
* __Web Server:__ _An amazing one even!_
----------------------------------------
_systopia reference: 23444_eileeneileenhttps://lab.civicrm.org/dev/core/-/issues/4890CiviCRM could not create trigger2024-02-01T10:38:38ZktatgenhorstCiviCRM could not create triggerOverview
----------------------------------------
_Please describe your problem or bug in detail._
Installation of version 5.69.0 on a fresh Ubuntu install with WordPress.
1. I used this link to setup Wordpress on Ubuntu:
https://ub...Overview
----------------------------------------
_Please describe your problem or bug in detail._
Installation of version 5.69.0 on a fresh Ubuntu install with WordPress.
1. I used this link to setup Wordpress on Ubuntu:
https://ubuntu.com/tutorials/install-and-configure-wordpress#1-overview
2. I followed this set of instructions to install CiviCRM:
https://docs.civicrm.org/installation/en/latest/wordpress/
3. When I go to the installation/setup wizard I add the credentials for the database I created (separate from WP database and the user has grant all and specifically grant trigger on civicrm.*). I did destroy the VM and build a second time with same steps and results.
_If you have already posted on https://civicrm.stackexchange.com or https://chat.civicrm.org, please include the link to that conversation._
I have not posted there, but I see older posts regarding this on previous versions. None of those offer a valid solution for this present issue,
Reproduction steps
----------------------------------------
1. Click on **Contacts -> New Individual**.
1. Entered **First Name** and **Last Name** and clicked **Save**.
1. Got an error "**Fatal error: DB error**".
Current behaviour
The setup stops after DB connect and reports unable to create triggers.
----------------------------------------
_What happens currently. Please provide error messages, screenshots or gifs ([LICEcap](http://www.cockos.com/licecap/), [SilentCast](https://github.com/colinkeenan/silentcast)) where appropriate._
```
TIP: The best way to convey an error message is to copy it in here and use
three backtick ` symbols. You may edit the message to remove private
information (like passwords). The backticks will help to preserve any
special characters or spaces.
```
Expected behaviour
----------------------------------------
_What should happen._
I should pass through the setup screen and have a functioning CiviCRM install.
Environment information
----------------------------------------
<!-- Some of the items below may not be relevant for every bug - if in doubt please include more information than you think is neccessary. -->
Ubuntu 20.04
MySqlD 8.0.35
Php 8.2.4
Apache2 2.4.41
CiviCRM 5.69.0
* __Browser:__ _Firefox 59.0.1/Chrome 78.0.3904/Safari 13_
* __CiviCRM:__ _Master/5.20.0/5.19.1/5.18.2/..._ <!-- If this problem relates to an upgrade, then specify both old and new versions -->
* __PHP:__ _7.0/7.1/7.2/7.3/...__
* __CMS:__ _Backdrop 1.5/Drupal 7.30/Joomla 3.3/WordPress 4.5/..._
* __Database:__ _MySQL 5.7.7/MariaDB 10.4/..._
* __Web Server:__ _Apache 2.4/Nginx 1.16/..._
Comments
----------------------------------------
_Anything else you would like the reviewer to note._
Thank you,
Karl Tatgenhorsthttps://lab.civicrm.org/dev/core/-/issues/4887Discussion / clarification on Api4 permissions2024-01-05T13:49:04ZRichDiscussion / clarification on Api4 permissionsI've been trying to [grok](https://en.wikipedia.org/wiki/Grok) this while working on Standalone, and it's raised a few questions for me. It looks like there are several nuances/quirks/gotchas. This is my attempt to document these. I'd re...I've been trying to [grok](https://en.wikipedia.org/wiki/Grok) this while working on Standalone, and it's raised a few questions for me. It looks like there are several nuances/quirks/gotchas. This is my attempt to document these. I'd really appreciate anyone chipping in to improve this understanding. I'll close this issue off once I understand things, hopefully after having opened other issues/PRs including for the docs.
See: [Developer docs on security](https://docs.civicrm.org/dev/en/latest/security/permissions/#apiv4)
API access is split between Coarse and Fine grained levels.
- Coarse is: `action::permissions()`, `api::authorize`, `AuthorizeEvent`,
`PermissionCheckSubscriber`...
- Fine is: `CoreUtil::checkAccessRecord`, `_checkAccess`, `AuthorizeRecordEvent`
**Coarse** level control is at the entity.action level only is handled by the
`API\Kernel->authorize()` method, which dispatches an `AuthorizeEvent` over
`civi.api.authorize`.
- `Api4/Event/Subscriber/PermissionCheckSubscriber` subscribes `W_LATE` and
authorizes the event: where checkPermissions is false or where the action is
getLinks, or if `$apiRequest->isAuthorized()` returns true. This generally is
handled by code in `AbstractAction` which finds the permissions needed
(`getPermissions()`) finds a match for the action name in the array returned
by the Api Action classes’ `permissions()` method) and checks them with
`\CRM_Core_Permission::check();`
Gotcha: the `save` action, if unspecified in the action's `permissions()`
method gets defaults from `create` (if specified) rather than `default`.
- there are other listeners too, (e.g. `AdhocProvider` (api4) and
`API/Subscriber/PermissionCheck` which is mostly api3 but it also authorizes
requests for api4 where `checkPermissions` has been set false, duplicating the
same logic above.)
**Fine** level control (at least for DAO entites) is at an Action-and-Record
level control (termed ACLs) rely on a single method on the entity's **BAO**:
public static function _checkAccess(string $entityName, string $action, array $record, ?int $userID): bool
- These methods are used by `\Civi\Api4\Utils\CoreUtil::checkAccessRecord`
- This only calls the BAO's `_checkAccess()` for actions that do not extend
`AbstractGetAction`; intended to be write actions, but could be others.
When it is called it must return TRUE for OK. A FALSE value will cause an
exception.
- `_checkAccess` is **not** called if a listener to
`\Civi\Api4\Event\AuthorizeRecordEvent` authorizes or explicitly declares
the event unauthorized.
- `_checkAccess` must only check data as follows: entity and action name and nothing else from `$apiRequest`. Data in `$record` (see below for what this contains). It may identify the record's primary key ID and fetch data from the database to consider, too.
- `checkAccessRecord` is used:
- `AbstractUpdateAction::_run` calls it per row to be updated
- `DAODeleteAction::_run` calls it per row.
- `BasicBatchAction::_run` calls it per row.
- `AbstractCreateAction::validateValues` calls it. (before `ValidateValuesEvent`)
### `$record` for Delete
This is `[$idFieldName => $id]` only.
### `$record` for Create
validateValues calls `_checkAccess` with data that has been pre processed via: *formatWriteValues » fillDefaults*
### `$record` for Update
- formatWriteValues
- some special code to convert passing a primary key value in values to a WHERE
clause, and to rule out the possiblity of changing an existing primary key
- For single row updates:
- populate only the row's id field (from the where)
- calls `_checkAccess` with the id field + input (after formatWriteValues)
- For batch updates:
- getBatchRecords
- each row, we take the input (after formatWriteValues) and fill from
existing row data. Then call `_checkAccess`.
- *then* call validateValues() which dispatches a ValidateValuesEvent
over `civi.api4.validate`
### `$record` for Save
- Loops rows
- applies defaults passed in to this specific row
- `formatWriteValues`
- `matchExisting` (presumably sets PKs?)
- if a create, fills defaults
- calls `validateValues` which loops records and for each calls
`checkAccessDelegated` with either 'create' or 'update'
- `checkAccessDelegated` creates a new API request of the same type and calls
`authorize`, as a way to check permissions for the entity.action pair.
(inside loop!)
- it then calls `_checkAccess` for the record with this new empty api request
and the record as a separate array. i.e. the `$apiRequest` handled by
`_checkAccess` only contains the entity name and action name.
## Questions
- There's a couple of places where I think we could reduce work that is
unnecessarily repeated in a loop.
- `AbstractSaveAction::validateValues` calls `checkAccessDelegated` which in
turn calls `authorize()` on a blank API call every time. The answer to this
won't change during the loop; there may be one answer for 'create' and one
for 'update'.
- `Generic\BasicBatchAction` (minor): move if(checkPermissions) to outside
the loop and cache the logged in user outside the loop too.
- `_checkAccess` is only called for actions that don't extend the
AbstractGetAction. This could potentially be confusing for other reporting
actions that don't extend this (maybe they should?). I've seen quite a few
examples where specific cases return FALSE and at the end there's a default
return TRUE. I'm not suggesting there's a security problem, but this doesn't
feel "security-first", i.e. it's a blanket allow with some exceptions rather
than a blanket deny with allow being explicitly checked.
- It feels weird that we start from an entity-action-specific class, then call
a wider entity-specific method, passing in the name of the action, meaning that
the logic then needs to disaggregate the action-specifics. It also feels weird
that `_checkAccess` is in the BAO, though it's only called by Api4 code.
Wouldn't it be better to have `_checkAccess` in the action classes?
- `$record` passed as part of update actions is inconsistent. It contains full original
data with updated field values replaced for *batch* updates, but only the
updated field values and primary key for *single* updates. Later,
validateValues is called with a `CRM_Utils_LazyArray` that would *reload* this
data (in the case of a batch update). The `ValidateValuesEvent->records` *only*
contains the updated values. So three possible different sets of data.
- A comment in the `ValidateValuesEvent` notes that it's expensive to lookup the
original values (because it would be one query per record), so avoid if possible.
The lazy array returns only the update values as 'new', not the merged array
(this is the same as in its records property). There is duplication for an
*update action* since these are already present on the event under 'records'
and are shared between all rows. However for a *save action*, there could be
per-row specific updates.
Q. where we've loaded the data (e.g. batch update) couldn't we make that available
to save the lazy function doing a one-query-per-row?RichRichhttps://lab.civicrm.org/dev/release/-/issues/235.68.1 failed to upgrade DB from 5.53.0: DB Error: constraint violation2024-01-15T11:30:51ZDmitry Smirnov5.68.1 failed to upgrade DB from 5.53.0: DB Error: constraint violationAfter successfully upgrading our staging CiviCRM from 5.53.0 to 5.68.1, upgrade of production instance failed as follows:
```
[Error: Backfill civicrm_mailing_event_queue (6 => 500005)]
Error Field Error Value
Type DB_Error
Code -3
Mess...After successfully upgrading our staging CiviCRM from 5.53.0 to 5.68.1, upgrade of production instance failed as follows:
```
[Error: Backfill civicrm_mailing_event_queue (6 => 500005)]
Error Field Error Value
Type DB_Error
Code -3
Message DB Error: constraint violation
Mode 16
UserInfo UPDATE civicrm_mailing_event_queue q INNER JOIN civicrm_mailing_job job ON job.id = q.job_id SET q.mailing_id = job.mailing_id, q.is_test=job.is_test WHERE q.id >= 6 AND q.id <= 500005 AND q.mailing_id IS NULL [nativecode=1452 ** Cannot add or update a child row: a foreign key constraint fails (`wordpress-civicrm`.`civicrm_mailing_event_queue`, CONSTRAINT `civicrm_mailing_event_queue_ibfk_1` FOREIGN KEY (`mailing_id`) REFERENCES `civicrm_mailing` (`id`) ON DELETE SET NULL)]
DebugInfo UPDATE civicrm_mailing_event_queue q INNER JOIN civicrm_mailing_job job ON job.id = q.job_id SET q.mailing_id = job.mailing_id, q.is_test=job.is_test WHERE q.id >= 6 AND q.id <= 500005 AND q.mailing_id IS NULL [nativecode=1452 ** Cannot add or update a child row: a foreign key constraint fails (`wordpress-civicrm`.`civicrm_mailing_event_queue`, CONSTRAINT `civicrm_mailing_event_queue_ibfk_1` FOREIGN KEY (`mailing_id`) REFERENCES `civicrm_mailing` (`id`) ON DELETE SET NULL)]
Civi\Core\Exception\DBQueryException: DB Error: constraint violation in /usr/share/php/PEAR.php on line 944
- DB_Error: DB Error: constraint violation in unknown on line unknown
```
That's on PHP-8.2, Debian 12 "Bookworm", Mariadb 10.9.3.
Please advise.