Development issueshttps://lab.civicrm.org/groups/dev/-/issues2024-02-27T01:10:05Zhttps://lab.civicrm.org/dev/core/-/issues/3314Contribution pages with memberships don't validate correctly when using "quic...2024-02-27T01:10:05ZJonGoldContribution pages with memberships don't validate correctly when using "quick config" price fieldsIn `CRM_Contribute_Form_Contribution_Main::formRule`, there are multiple references to `$self->_useForMember`. `$self->_useForMember` is only `TRUE` if the price set is a "quick config" price set (and is explicitly used as such in the t...In `CRM_Contribute_Form_Contribution_Main::formRule`, there are multiple references to `$self->_useForMember`. `$self->_useForMember` is only `TRUE` if the price set is a "quick config" price set (and is explicitly used as such in the template). However, during `formRule` validation, it's used as a proxy for "this contribution page uses memberships". @mattwire correctly identified the problem on the confirmation class in [PR 15094](https://github.com/civicrm/civicrm-core/pull/15094).
As a result, several membership-specific validations are bypassed.
I attempted to move his methods to the shared `ContributionBase` class and make them protected, but I ran into deeper validation issues. I can't justify further work on this, but wanted to document for future folks who bang their heads against this.https://lab.civicrm.org/dev/core/-/issues/3313Unable to save a membership with a text price field2023-12-19T22:18:09ZDaveDUnable to save a membership with a text price fieldSame problem in 5.28 and master and can reproduce on dmaster.demo.civicrm.org. Or maybe I'm missing something.
1. Create a price set used for memberships.
1. Set financial type to member dues.
1. Add one text field (the default input fi...Same problem in 5.28 and master and can reproduce on dmaster.demo.civicrm.org. Or maybe I'm missing something.
1. Create a price set used for memberships.
1. Set financial type to member dues.
1. Add one text field (the default input field type).
1. Amount doesn't matter - put in some number.
1. Leave other fields at defaults.
1. Create a new membership.
1. Choose the price set.
1. Put in quantity 1.
1. Can leave the rest at defaults.
1. Click Save.
1. ERROR: Select at least one membership option.
It works with a radio field.https://lab.civicrm.org/dev/core/-/issues/3311Proposal - store metadata on membership renewal on line item2023-06-18T00:43:47ZeileenProposal - store metadata on membership renewal on line item~~UPDATE - discussion on this is converging on adding a new json field 'metdata' to the line item table, permitting data related to that line item to be stored at point of order, and used when confirming. (UPDATE - this is not really the...~~UPDATE - discussion on this is converging on adding a new json field 'metdata' to the line item table, permitting data related to that line item to be stored at point of order, and used when confirming. (UPDATE - this is not really the convergence now - proposal updated below)~~
We have a few issues open ( https://lab.civicrm.org/dev/membership/-/issues/28 , https://lab.civicrm.org/dev/core/-/issues/1402 , https://lab.civicrm.org/dev/financial/-/issues/53 ) that are to my mind blocked by us not having a data model that holds the user's intent when renewing. The focus of this gl is the data model and exposing it via the back office renewal form & doing what is needed in the apis.
**The problem**
When renewing an expired membership the back office user has a fairly blunt way of influencing the calculated dates - they can set the 'renewal date'. However, if the payment is not made in the same form submission that intent is not captured anywhere and cannot be respected when completing the transaction. Likewise the receipt text and number of renewal terms are not captured.
**The proposal data model**
1) add membership_num_terms to civicrm_line_item table.
2) add 'payment_completion_metadata' to the ~~civicrm_line_item~~ civicrm_contribution table for data that is only used in the order->payment flow. Limit what can be stored here to tested values used in that flow - likely to be the message details and effective date the renewal form uses but doesn't sotre.
~~1. New field added to civicrm_line_item called 'metadata' which stores information required to complete the membership renewal - e.g. the field might store ``json_encode(['start_date' => '2010-01-01', 'end_date' => '2020-01-01', 'membership_status_id' => 'Current', 'membership_num_terms' => 2']~~
~~- note this is the new proposal - but I'm wondering what happens if it's renewed in the meantime & then paid~~
~~- New status 'Pending renewal' used when there is a need to record the dates (is_current_member = 0, is_reserved = 1, is_admin = 1 too I think)~~
~~- When a contribution linked to a membership in 'Pending renewal' is completed the status is updated but the dates are not changed~~
~~- If a membership has 'Pending renewal' status and status_override_end_date is set then any processing of that membership after that date would result in it being reverted to previous dates and status - this information should be available in the membership_log table.~~
**Proposed form exposure**
No proposed form changes - this is just storing data already submitted in the renewal form.
~~- the logic could be used outside of existing core forms but within core I think this is something we are exposing to the back-office user doing a renewal - they already have considerable ability to edit dates. So I think this is a form layer thing not some site-wide setting.
- when renewing an expired membership the renewal form would present the user with what will happen to the start date, end date etc if payment is received on that day and offer a checkbox to specify dates - if that is checked then date fields would be exposed an on save the Pending Renewal status would be used with the selected dates.
- potentially the user can also enter the date the 'offer' is available until - ie status_override_end_date~~
**Edge cases**
1. Multiple terms
We would add an extra field membership_num_terms to the line items column
~~https://lab.civicrm.org/dev/membership/-/issues/28 throws up the edge case that more than one renewal term might be selected. Both @jptillman and I assumed that the right way to represent that would be setting qty = n and then picking that when confirming. @andrewhunt didn't think that was the right assumption https://github.com/civicrm/civicrm-core/pull/18618. If we come down on NOT setting qty then we ALSO need a way to record intent for non-expired renewals - this could be an extra membership_num_terms column on the line_item table. (I'm still inclined to my original assumption but more points of view needed).~~
2. About to expire rather than actually expired
Out of scope for now - just do what the form currently does
~~I think we would still manage this in the form ie - "the membership is due to to expire in 4 days, if payment is received after that then .... ' and present the same options as for an actually expired membership~~https://lab.civicrm.org/dev/core/-/issues/3301Contrast ratios2024-02-28T11:51:33ZJoeMurrayContrast ratiosWCAG 2.0 specifies various minimum contrast ratios between foreground and background for different sizes of fonts, and for bolded and non-bolded fonts [https://www.w3.org/TR/WCAG20-TECHS/G18.html](https://www.w3.org/TR/WCAG20-TECHS/G18.h...WCAG 2.0 specifies various minimum contrast ratios between foreground and background for different sizes of fonts, and for bolded and non-bolded fonts [https://www.w3.org/TR/WCAG20-TECHS/G18.html](https://www.w3.org/TR/WCAG20-TECHS/G18.html).
> ...Priority 2 (AA), and Priority 3 (AAA). Whether or not you need to reach an AAA conformance depends on your target audience. Read more about this [on the w3 site](http://www.w3.org/TR/2005/WD-WCAG20-20051123/intro.html#conformance). For large text (over 18 points) the contrast ratio for AA is 3:1 and for AAA 5:1. For small text it's 5:1 for AA and 7:1 for AAA. ([http://www.colorsontheweb.com/Color-Tools/Color-Contrast-Analyzer](http://www.colorsontheweb.com/Color-Tools/Color-Contrast-Analyzer))
More exact ratios and translations of font sizes to px directly from the WCAG first mentioned for bold and non-bold text are:
* **BOLD and < 14pt (18.5px):** 7:1 contrast ratio
* Not bold and < 18pt (24px): 7:1 contrast ratio
*
* **BOLD and >= 14pt (18.5px):** 4.5:1 contrast ratio
* Not bold and >= 18pt (24px): 4.5:1 contrast ratio
Using the contrast analyzer provided on the page above, the current Shoreditch contrast ratios are not sufficient in some contexts.
For example, on pages like New Contact and New Activity we get the following ratios:
Blue font #2786c2 on tan #efefe5 in 13px: 3.44:1 (below 5:1)
Dimmed grey #999999 on offwhite #fff in 13px: 2.85:1 (below 5:1)
![2018-12-04_09-16-41](/uploads/efe32b9def1089f434b0cf1c11ef1b6e/2018-12-04_09-16-41.png)
![2018-12-04_09-33-52](/uploads/171119a46775a677bd44664497507def/2018-12-04_09-33-52.png)
![2018-12-04_09-51-12](/uploads/1a9006ebddb3d210f2b326d41d29f8cc/2018-12-04_09-51-12.png)
We should meet a 5:1 ratio for all text < 18pt.https://lab.civicrm.org/dev/core/-/issues/3292Upgrade select2 to stable version 4.0.4 and fix compatibility issues in Civi2024-02-23T17:58:01ZMonish DebUpgrade select2 to stable version 4.0.4 and fix compatibility issues in CiviCurrently, Civi is using select2 3.5 which we need to upgrade to latest stable 4.0.4 (https://github.com/select2/select2/releases) and also resolve compatibility issues with CiviCRM.
Major issues:
1. Auto appending values doesn't work a...Currently, Civi is using select2 3.5 which we need to upgrade to latest stable 4.0.4 (https://github.com/select2/select2/releases) and also resolve compatibility issues with CiviCRM.
Major issues:
1. Auto appending values doesn't work anymore
Error: Uncaught TypeError: Cannot read property 'prop' of undefined
LOC - https://github.com/civicrm/civicrm-core/blob/master/js/crm.searchForm.js#L9
Reference - https://select2.org/upgrading/migrating-from-35#select2-val
2. Placeholder broken
Error: Uncaught TypeError: Cannot read property 'id' of undefined
As per 3.5 - placeholderOption is used to set the placeholder value for select2 widget
As per 4.0.4 - placeholder option can now accept placeholder value both in string and object
Reference - https://select2.org/upgrading/migrating-from-35#more-flexible-placeholders
3. EntityRef select2 fields are broken
Error: Uncaught Error: No select2/compat/initSelection
As per 4.0.4 - Removed the requirement of 'initSelection'
Reference - https://select2.org/upgrading/migrating-from-35#removed-the-requirement-of-initselection
4. Navigation menu style is not applied
Error: Uncaught TypeError: Cannot read property 'prop' of undefined
As per 4.0.4 - $("select").val("1").trigger("change"); // instead of $("select").select2("val", "1");
Reference - https://select2.org/upgrading/migrating-from-35#select2-val
5. Broken entityRef field
There are multiple issues with an entityRef field which are listed below:
1. Placeholder doesn't show
2. Unable to populate data via REST API link
3. Create contact dialog box doesn't render below the search field
4. Escape text doesn't show up
6. Action-menu select2 doesn't work on selecting record(s) in a search list
It throws an error `Uncaught TypeError: Cannot read property 'apply' of undefined` after Search and also action-menu field doesn't behave on selecting a record.
7. The native crm CSS styling doesn't apply on select2/4.0+ widget
Minor issues
1. Multi select2 widget doesn't show downarrow placeholder icon
2. select2 css style aren't applied after upgradehttps://lab.civicrm.org/dev/core/-/issues/3218CSV exports does not respect localization configuration2023-10-19T14:38:19ZmarcusmCSV exports does not respect localization configurationWhen creating CSV exports from searches the e. g. money or date fields are not exported with the configured localization formats.
Configure language: German, decimal separator: ",", thousands sep.: "."
The export looks like this:
```...When creating CSV exports from searches the e. g. money or date fields are not exported with the configured localization formats.
Configure language: German, decimal separator: ",", thousands sep.: "."
The export looks like this:
```
| Nettobetrag |
|-------------|
| 135.00 |
| 20.00 |
| 50.00 |
| 1,130.00 |
should be
| Nettobetrag |
|-------------|
| 135,00 |
| 20,00 |
| 50,00 |
| 1.130,00 |
```
and an example for a date field:
```
| Geburtsdatum |
|--------------|
| 1942-05-02 |
| 1983-06-01 |
should be
| Geburtsdatum |
|--------------|
| 02.05.1942 |
| 01.06.1983 |
```
The export was created with a contribution search.https://lab.civicrm.org/dev/core/-/issues/3204Add recurring contributions to contribution reports2022-09-16T21:11:08ZlarsssandergreenAdd recurring contributions to contribution reportsIt would be very convenient to be able to do filter by recurring contributions for reports, add recurring contributions as a column or even group by.
[Here's a PR that adds this for the Contribution Summary report.](https://github.com/c...It would be very convenient to be able to do filter by recurring contributions for reports, add recurring contributions as a column or even group by.
[Here's a PR that adds this for the Contribution Summary report.](https://github.com/civicrm/civicrm-core/pull/20168) I will also implement in other relevant reports once this has been reviewed.https://lab.civicrm.org/dev/core/-/issues/3191SearchKit not functional on Joomla 42023-05-24T05:20:40Zjoshjosh@civicrm.orgSearchKit not functional on Joomla 4The past two release of CiviCRM have resulted in SearchKit not functioning on Joomla 4. When browsing to SearchKit, the user is presented with what appears to be unrendered code:
![Screenshot_2022-02-21_12.13.32](/uploads/d52471e665bbfc...The past two release of CiviCRM have resulted in SearchKit not functioning on Joomla 4. When browsing to SearchKit, the user is presented with what appears to be unrendered code:
![Screenshot_2022-02-21_12.13.32](/uploads/d52471e665bbfcaf82f934243c6c9c19/Screenshot_2022-02-21_12.13.32.jpg)
Console presents the following error:
```
Error: [$injector:unpr] http://errors.angularjs.org/1.8.2/$injector/unpr?p0=savedSearchesProvider%20%3C-%20savedSearches%20%3C-%20searchList
at angular.min.js:7:168
at angular.min.js:46:468
at Object.d [as get] (angular.min.js:44:197)
at angular.min.js:47:29
at d (angular.min.js:44:197)
at e (angular.min.js:44:438)
at Object.instantiate (angular.min.js:45:333)
at angular.min.js:99:267
at Object.link (angular-modules.d0a616a75686afef23d3772fcee92a7f.js:5648:217)
at angular.min.js:17:134 '<div ng-view="" class="ng-scope">'
(anonymous) @ angular.js:15697
```
All other aspects of CiviCRM appear to be functional. The issue can be reproduced at https://cividemo.com and on fresh installs of just Joomla and CiviCRM (no extensions installed on either systems).Monish DebMonish Debhttps://lab.civicrm.org/dev/core/-/issues/3185Tags for attachments are not properly assigned to the attachment2023-10-07T21:15:50ZDaveDTags for attachments are not properly assigned to the attachmentI'm not sure if this is recent or where it's happening. What happens is that civicrm_entity_tag.entity_id is one higher than the appropriate id from civicrm_file, e.g.
civicrm_file:
| id | file_type_id | mime_type | uri ...I'm not sure if this is recent or where it's happening. What happens is that civicrm_entity_tag.entity_id is one higher than the appropriate id from civicrm_file, e.g.
civicrm_file:
| id | file_type_id | mime_type | uri |
|------|--------------|-------------------|-------------------------------------------------|
|__28__| NULL | text/plain | abc_aef1644a7b96451b6c15b7e34b862f5d.txt |
civicrm_entity_tag:
| id | entity_table | entity_id | tag_id |
|----|------------------|---------------|--------|
| 57 | civicrm_file | --> __29__ <--| 31 |
1. Create a tag set for attachments
1. Create some tags for the set
1. Create an activity
1. In the attachments section add a file and choose a tag
1. When you go back to view or edit the activity, the tag isn't displayed. Check the db and you'll see the entity_id doesn't match up.
1. Manually edit the entity_id in the db and then go back to view the activity. Now the tag is displayed.https://lab.civicrm.org/dev/translation/-/issues/75Translatable fields within Extension table2022-04-21T20:39:46ZseamusleeTranslatable fields within Extension tableAt present the CiviCRM i18n widget code and also the i10n schema code depend on the functions in the schema structure file returning an array of fields or html widgets. The Schema Structure file does not contain any information about tab...At present the CiviCRM i18n widget code and also the i10n schema code depend on the functions in the schema structure file returning an array of fields or html widgets. The Schema Structure file does not contain any information about tables other than core tables so for example in the JMA Grant Applications extension if we wanted to make the title or the introductory text field translatable we would have to hack the schema structure file.
I would like to propose that we create 3 new Events or similar to handle altering the fields, indices and widgets functions in the schema structure file.https://lab.civicrm.org/dev/drupal/-/issues/177Cannot resolve path using "cms.root.url"2022-04-29T09:11:35ZalmadorxCannot resolve path using "cms.root.url"Hi! I'm having the issue with
```
In Paths.php line 140:
Cannot resolve path using "cms.root.url"
```
I'm using Drupal 9 and the last version of CiviCRM
I've tried the solution from here
([regression `cv` fails on CiviCRM 5.15.0](htt...Hi! I'm having the issue with
```
In Paths.php line 140:
Cannot resolve path using "cms.root.url"
```
I'm using Drupal 9 and the last version of CiviCRM
I've tried the solution from here
([regression `cv` fails on CiviCRM 5.15.0](https://lab.civicrm.org/dev/drupal/-/issues/75))
\vendor\civicrm\civicrm-core\CRM\Utils\System\Drupal8.php:
` public function getCurrentLanguage() {
// Drupal might not be bootstrapped if being called by the REST API.
if (!class_exists('Drupal') || !\Drupal::hasContainer()) {
return NULL;`
I've replaced _return NULL;_ with _return $url;_ but that doesn't solved the issue.https://lab.civicrm.org/dev/core/-/issues/3154Custom tokens not working in Scheduled Reminders2022-08-25T21:35:03ZmartyCustom tokens not working in Scheduled RemindersOverview
----------------------------------------
Custom tokens are not evaluated for Scheduled Reminders when initiated by Cron Job. The tokens are evaluated properly when the Scheduled Reminders job is run manually using the Execute No...Overview
----------------------------------------
Custom tokens are not evaluated for Scheduled Reminders when initiated by Cron Job. The tokens are evaluated properly when the Scheduled Reminders job is run manually using the Execute Now option.
Reproduction steps
----------------------------------------
1. Create a custom token using hook_civicrm_container() and implement the civi.token.list and civi.token.eval event listeners.
1. Add a new Scheduled Reminder (I'm using membership end date) and include the custom token in the email message.
1. Create a Cron Job to run civicrm/bin/cron.php periodically (I run every 15 minutes).
1. Enable the Send Scheduled Reminders job and set to run Always.
1. Trigger the reminder appropriately (I create a new membership and set the end date to trigger).
1. Note the custom token is __not__ included in the resulting email after the cron run.
1. Now trigger a new reminder and click Execute Now on the Scheduled Reminders job (before the next cron run).
1. Note the custom token __is evaluated properly__ and included in the resulting email message.
Current behaviour
----------------------------------------
Custom token not included in Scheduled Reminder email when initiated by Cron Job
Expected behaviour
----------------------------------------
Custom token should be included in Scheduled Reminder email when initiated by Cron Job.
Environment information
----------------------------------------
* __CiviCRM:__ _5.47.2_
* __PHP:__ _7.4.28_
* __CMS:__ _WordPress 5.9.2_
* __Database:__ _MySQL_
* __Web Server:__ _Apache_https://lab.civicrm.org/dev/core/-/issues/3152Data stored in universal time does not handle DST consistently2023-11-30T05:00:20ZtottenData stored in universal time does not handle DST consistently[[_TOC_]]
Overview
----------------------------------------
CiviCRM has a number of `TIMESTAMP` columns -- these are stored in universal time (UTC) and displayed in the user's timezone. However, there is a subtle error in handling Dayl...[[_TOC_]]
Overview
----------------------------------------
CiviCRM has a number of `TIMESTAMP` columns -- these are stored in universal time (UTC) and displayed in the user's timezone. However, there is a subtle error in handling Daylight Savings Time (DST): if the current-date and the target-date sit on different sides of the DST-switch, then the time may present as +/- 1 hour.
This bug was one of the major subissues identified in https://lab.civicrm.org/dev/core/-/issues/2122. Although that particular feature was rolled-back/deferred from 5.47, the DST bug still exists -- it's just less obvious.
Example use-case
----------------------------------------
1. Create a contact record.
2. View the contact record. Note the creation time (`civicrm_contact.created_date`).
3. Change the system clock - set to a date where DST differs (eg if today is March 30, then go to December 5).
4. View the contact record. Note the creation time (`civicrm_contact.created_date`).
Current behavior
----------------------------------------
The displayed value of `civicrm_contact.created_date` _appears_ to change by +/- 1 hour, depending on when you view it.
> (Viewed on Mar 31, 2022)
>
> ![Screen_Shot_2022-03-31_at_12.40.53_AM](/uploads/8a2f5a0e4fb6ec884aede78e30d31b1a/Screen_Shot_2022-03-31_at_12.40.53_AM.png)
> (Viewed on Dec 5, 2022)
>
> ![Screen_Shot_2022-12-05_at_12.44.57_AM](/uploads/2df387488ee029a949010da163bf2ea8/Screen_Shot_2022-12-05_at_12.44.57_AM.png)
Why? CiviCRM sends a note to MySQL about the current user's timezone (`SET time_zone = '...'`). However, it doesn't identify the timezone effectively. It gives [the current numeric offset (at the moment of viewing)](https://github.com/civicrm/civicrm-core/blob/5.47.3/CRM/Utils/System/Base.php#L758-L762) - but (in locales with DST) the offsets fluctuate over time.
(_Ex: On Mar 31, the offset in California is `-0700`. Under current/long-standing law, the offset will be `-0800` on Dec 5. Of course, the US Congress is reconsidering this law... so we don't really know what the offset will be!_)
Proposed behavior 1: Fix MySQL timezones
----------------------------------------
CiviCRM should send the timezone as a symbolic name, such as `Europe/Helsinki`, `America/Los_Angeles`, or `Australia/Sydney`. These symbolic-names have an underlying database which allows them adjust automatically based on DST-rules/target-dates/current-law. On the surface, the fix is extremely simple:
```diff
diff --git a/CRM/Utils/System/Base.php b/CRM/Utils/System/Base.php
index a4660834c5..8e40f6da35 100644
--- a/CRM/Utils/System/Base.php
+++ b/CRM/Utils/System/Base.php
@@ -755,10 +755,9 @@ abstract class CRM_Utils_System_Base {
* Set timezone in mysql so that timestamp fields show the correct time.
*/
public function setMySQLTimeZone() {
- $timeZoneOffset = $this->getTimeZoneOffset();
- if ($timeZoneOffset) {
- $sql = "SET time_zone = '$timeZoneOffset'";
- CRM_Core_DAO::executequery($sql);
+ $timeZone = $this->getTimeZoneString();
+ if ($timeZone) {
+ CRM_Core_DAO::executequery('SET time_zone = %1', [1 => [$timeZone, 'String']]);
}
}
```
There are a couple of catches.
* __Timezone rules change__ (occasionally). Any software that supports timezones ultimately needs a _data feed_ with current rules. The good news: IANA publishes a free/open feed (https://www.iana.org/time-zones; aka `tzdata`; aka `zoneinfo`), most Linux/Unix distros have this feed, and MySQL can read it (`mysql_tzinfo_to_sql`). It usually requires one command (which could run during system-config, system-startup, and/or cron):
```bash
mysql_tzinfo_to_sql /usr/share/zoneinfo | mysql mysql
```
The problem: we have no measures for (a) how many CiviCRM deployments actually subscribe to this feed and (b) how many could subscribe, if they chose to.
* __Timezone names may be inconsistent__ (occasionally). For example, in different contexts, it's been fashionable to refer to California's timezone as `America/Los_Angeles`, `US/Pacific`, and `PST8PDT`. (The current+official fashion is `America/Los_Angeles` - the others are deprecated.) However, since Civi integrates with various layers (different CMSs; PHP APIs; MySQL APIs), there are edge-case where the layers may choose different names. (*I'm not super-concerned, but we should raise sensible warnings when names are invalid or mismatched.*)
The central issue is - how to cope when data isn't available? This comes to mind:
* (Status check) If the active TZ (`getTimeZoneString()`) has a deprecated name (eg `PST8PDT`) or an offset (eg `-0700`), show a warning.
* (Status check) If the active TZ (`getTimeZoneString()`) isn't supported by MySQL, show a warning.
* (Runtime) If the active TZ (`getTimeZoneString()`) isn't supported by MySQL, fallback to sending offset.
Proposed behavior 2: Change format. Use only PHP TZs.
----------------------------------------
(_This expands on one of @haystack's suggestions in dev/core#2122._)
If you assume that MySQL time services aren't available - what else would you do? You could use PHP time services.
The astute observer will note the status-quo (using both PHP and MySQL time-services) creates two points-of-failure. If either PHP _or_ MySQL has bad/incomplete/old timezone data, then you'll get mis-calculations _somewhere_. Consolidating on PHP time-services would reduce the #dependencies.
Both Drupal and WordPress take this approach. (I suspect this is extremely useful for maximizing compatibility with heterogeneous web-hosts.) They each do it a bit differently, but some central concepts are the same:
* In Drupal, PHP processes read+write temporal data in universal time -- as an `INT` (Unix-style, seconds-since-epoch).
* In WordPress, PHP processes read+write temporal data in universal time -- as a `DATETIME` with a `_gmt` suffix (eg `post_modified_gmt`).
* Hypothetically, you could hardcode MySQL to `SET time_zone='+0:00'`. PHP processes would read+write temporal data in universal time -- as a `TIMESTAMP`.
In all those cases, the onus is on the PHP devs to convert to/from universal-time when implementing functionality (eg "find records from March 1 - March 15" or "find records from this afternoon" or "extract the hour:minute component").
But there is a catch here: Civi already relies on several MySQL time-services. The schema works a certain way; the reports/searches/UIs/APIs expect the schema to work a certain way; etc.
The central issue is - how do you manage/QA all the changes (in schema+logic) required to change time-service?
Comments
----------------------------------------
* I haven't tested, but I'm fairly certain there will be another manifestation in CiviMail scheduling. Ex:
* You live in a timezone where DST changes on March 16.
* On March 10, you schedule a mail-blast for 2:00pm on March 20. It stores the schedule with the wrong offset.
* When March 20 comes, the mailing actually goes out at 3:00pm (or maybe 1:00pm).https://lab.civicrm.org/dev/core/-/issues/3148Dedupe with multi-select custom fields can trigger IDS2023-03-18T04:20:40ZJonGoldDedupe with multi-select custom fields can trigger IDSWhen deduping contacts that have multi-select custom fields, and selecting to move the custom fields to the new contact, the IDS is triggered.
### Steps to replicate
* Create a custom field that allows saving multiple values (e.g. a che...When deduping contacts that have multi-select custom fields, and selecting to move the custom fields to the new contact, the IDS is triggered.
### Steps to replicate
* Create a custom field that allows saving multiple values (e.g. a checkbox). Note that you need several of these to trigger the "kick" on the IDS (3, I think).
* Create two contacts that are duplicates.
* Find and merge the records.
### Expected result
Contacts merged successfully.
### Actual result
"Your activity is a bit suspicious, hence aborting"
The issue is the POST request, which is passing arguments like `move_custom_12` with the `VALUE_SEPARATOR` control character. This triggers the IDS filter labeled "Detects nullbytes and other dangerous characters".
I'm really not certain what the correct answer is here - I can exempt users from the IDS, and maybe that's the solution to pursue, but it seems like there should be another solution available. Is it possible to exempt certain paths from the IDS, or use an alternate set of rules for a certain path?
Keyword: Intrusion Detection Systemhttps://lab.civicrm.org/dev/core/-/issues/3145CiviCase: if contact is an Organization or Household, cannot change Case Coor...2023-11-23T13:15:30ZAllenShawCiviCase: if contact is an Organization or Household, cannot change Case Coordinator (or, must edit Case Coordinator relationship type)# To reproduced on https://dmaster.demo.civicrm.org/ as of today ("Powered by CiviCRM 5.49.alpha1"):
1. Observe that the Homeless Services Coordinator role is defined as in a default installation (contact a type and contact b type are b...# To reproduced on https://dmaster.demo.civicrm.org/ as of today ("Powered by CiviCRM 5.49.alpha1"):
1. Observe that the Homeless Services Coordinator role is defined as in a default installation (contact a type and contact b type are both 'individual')
2. Create a new "Housing Support" case with an Organization contact for the Client.
3. Open Manage Case page for this case
4. Under Roles accordion, click the pencil icon to edit the "Homeless Services Coordinator is (Case Manager)" role
5. Observe pop-up overlay "Reassign Homeless Services Coordinator" where you would select an individual for this role
5. At this point:
* Expected behavior: This pop-up contains a contact reference field allowing me to type a contact name or email address
* Actual behavior: This pop-up may be displayed partially outside of the viewport, with no way to scroll to see all of it; in any case, you can see -- if you're able to use Developer Tools to make the thing display within the viewport -- that the expected contact reference field is just a plain text field; also the Save and Cancel buttons are non-functional, and the only thing I can do is to click the X control to close the pop-up:
![popup](/uploads/92a2aa68c67897f1da5b8ee4ee3e2a7d/popup.png)
# Workaround
1. Edit the Homeless Services Coordinator role to allow Organizations in the Contact A position.
2. Repease the repro steps above and observe expected behavior.
# Other thoughts:
* Testing on client sites indicates his is not limited to the "Homeless Services Coordinator" relationship type; instead, it's happening for any relationship type which is confiugured as the type for the Case Coordinator role.
* Not sure about how best to design a fix from the UX perspective. Prevent case creation? Warn in System Status?https://lab.civicrm.org/dev/core/-/issues/3139Badgelayouts cannot be edited with PHP warning2022-09-01T13:49:35ZBradley TaylorBadgelayouts cannot be edited with PHP warning_Reproduced on dmaster and locally on WordPress_
**Steps to reproduce**
1. Navigate to "Administer CiviCRM", "Event Name Badge Layouts".
2. Create a new name badge
3. Edit the newly created name badge.
**Expected outcode**
The edit scr..._Reproduced on dmaster and locally on WordPress_
**Steps to reproduce**
1. Navigate to "Administer CiviCRM", "Event Name Badge Layouts".
2. Create a new name badge
3. Edit the newly created name badge.
**Expected outcode**
The edit screen should be pre-filled with the values entered initially.
**Actual outcome**
Each field is blank, a PHP warning is shown:
![Screenshot_2022-03-27_at_10.18.33](/uploads/105dc6046cbb9b569207500da40cf77f/Screenshot_2022-03-27_at_10.18.33.png)
**Technical explanation**
The bug was introduced in https://github.com/civicrm/civicrm-core/commit/873bfeb503caa413f17460dbe450b74fac3d6dbf.
The commit above added a new tokens:
```
'{event.start_date|crmDate:"%B %E%f"}' => ts('Event Start Date'),
'{event.end_date|crmDate:"%B %E%f"}' => ts('Event End Date'),`
```
The data for badge layouts is stored as encoded JSON. This means that the quote marks in these two tokens are being wrapped in double-quotes for the string, causing something like `"{event.start_date|crmDate:"%B %E%f"}"`. As such the JSON is not valid and cannot be `json_decode`ed.
The actual fix could be straightforward enough: Switch the tokens to use single instead of double quotes. However, I'm not sure what the correct solution is for any broken JSON which is now stored in CiviCRM databases. Some sort of upgrade script might be required to find/replace the known broken JSON.
Pinging @eileen who did a lot of work on tokens last year.https://lab.civicrm.org/dev/core/-/issues/3137FormBuilder - setting a field as 'required' does not implement any validation...2023-02-28T19:52:15ZUpperholmeFormBuilder - setting a field as 'required' does not implement any validation checkCreating a submission form using Form Builder, it is possible to set any given field to be 'Required'.
I'm assuming that by doing this the form could not then be submitted if the required field is empty. A validation error should be res...Creating a submission form using Form Builder, it is possible to set any given field to be 'Required'.
I'm assuming that by doing this the form could not then be submitted if the required field is empty. A validation error should be resented assisting the user to complete the required field/s.
In practice the form can be submitted with no alert or apparent validation check taking place.
Using CiviCRM 5.47.2 with Wordpress.Kurund JalmiKurund Jalmihttps://lab.civicrm.org/dev/wordpress/-/issues/120Recurring contribution page on frontend tries to access wp-admin.2023-11-23T07:47:01ZdandrzejewskiRecurring contribution page on frontend tries to access wp-admin.I've got a user dashboard that includes recurring contributions. A recurring contribution has two links, cancel and view. When clicking the "view" link, the user is directed to the following URL:
https://example.org/civi/contact/view/...I've got a user dashboard that includes recurring contributions. A recurring contribution has two links, cancel and view. When clicking the "view" link, the user is directed to the following URL:
https://example.org/civi/contact/view/contributionrecur/?reset=1&id=2&cid=66&context=dashboard
This page contains two pieces, the details on the recurring contribution and the "related contributions" - i.e. the previous iterations of the recurring contribution.
However, the second part of the page only works when users have the "access the civicrm backend and API" permission, and I believe that's because the second half is retrieved using this URL:
https://example.org/wp-admin/admin.php?page=CiviCRM&q=civicrm%2Fcontribute%2Fcontributionrecur-payments&reset=1&id=2&cid=66&snippet=json
If I remove that permission, the user gets an error.
Additionally, and I can open a separate bug for this if it is in fact a bug, there's a "done" button on the contribution detail page. This attempts to take the user back to the civicrm "main" dashboard (on the front-end) rather than the user dashboard where we came from.
I can provide any other details, logs, etc as needed.https://lab.civicrm.org/dev/core/-/issues/3130Lack of hooks to detect when an Attachment is deleted2023-12-07T08:27:58ZhaystackLack of hooks to detect when an Attachment is deletedOverview
----------------------------------------
At present, when an Attachment is deleted via the CiviCRM UI (e.g. on "Edit Activity" dialogs/screens) only two CiviCRM hooks fire: the `civi.dao.preDelete` and `civi.dao.postDelete` Symf...Overview
----------------------------------------
At present, when an Attachment is deleted via the CiviCRM UI (e.g. on "Edit Activity" dialogs/screens) only two CiviCRM hooks fire: the `civi.dao.preDelete` and `civi.dao.postDelete` Symfony events triggered by `CRM_Core_BAO_EntityTag::del()`. All other deletions are done via direct SQL queries.
It would be great to:
* Have `hook_civicrm_pre` and `hook_civicrm_post` fire
* Have some way of retrieving the Attachment data prior to deletion
Reproduction steps
----------------------------------------
1. Set up CiviCRM to log all hooks
1. Click the "bin/trash" icon next to an Attachment (as per the screenshot below)
1. Inspect logged callbacks
![attachments-accordion](/uploads/9abded9e0e7f9cc602213b8b20de345d/attachments-accordion.png)
Current behaviour
----------------------------------------
The following callbacks are the only ones that will be seen (data trimmed for concision)
#### `civi.dao.preDelete`
```
[event] => Civi\Core\DAO\Event\PreDelete Object
(
[object] => CRM_Core_BAO_EntityTag Object
(
[id] =>
[entity_table] => civicrm_file
[entity_id] => 6
[tag_id] =>
)
)
```
#### `civi.dao.postDelete`
```
[event] => Civi\Core\DAO\Event\PostDelete Object
(
[object] => CRM_Core_BAO_EntityTag Object
(
[id] =>
[entity_table] => civicrm_file
[entity_id] => 6
[tag_id] =>
)
[result] => 0
)
```
It is possible to use the `entity_id` to retrieve the "File" data via the CiviCRM API. However, because of the order of deletions in [the `deleteEntityFile()` method](https://github.com/civicrm/civicrm-core/blob/6769e8bf8556701b81f914d30fc1bb913f8ed2ce/CRM/Core/BAO/File.php#L250), it is _not_ possible to retrieve the compound "Attachment" data via the API (because the "Entity File" data has already been deleted) and it is therefore not possible to find out which Entity/Entities the File was attached to.
It should also be noted that when the "Delete All Attachment(s)" checkbox is used, the `civi.dao.preDelete` and `civi.dao.postDelete` Symfony events fire *before* `hook_civicrm_pre` is fired for the Activity. To me, it would make more sense for `hook_civicrm_pre` to fire *before* the File deletion process is initiated so that `Activity.pre` and `Activity.post` "wrap" the entire process.
Expected behaviour
----------------------------------------
I would expect to be able to receive callbacks from `hook_civicrm_pre` and `hook_civicrm_post`, where `$op = "delete"` and `$objectName = "File"` along with some more contextual data about the "File" and the "Attachment" being deleted.
Comments
----------------------------------------
I'm going to open a PR that addresses the issue of retrieving the "Attachment" data via the API when using the Symfony hook, since that's pretty straightforward.
I'd appreciate any guidance from those more familiar with this class on the most sensible place(s) to add hooks during the deletion process - or whether deletion should more properly be done via `BAO` objects instead of direct queries.https://lab.civicrm.org/dev/core/-/issues/3128Cases dashlet sometimes gives error when sort by subject2023-12-04T13:49:19ZDaveDCases dashlet sometimes gives error when sort by subjectPulling this out from https://lab.civicrm.org/dev/core/-/issues/1624 as a separate issue.
_Sometimes_, sorting by the subject on a cases dashlet will give a datatables error. The actual error if you look at the network response is `Colu...Pulling this out from https://lab.civicrm.org/dev/core/-/issues/1624 as a separate issue.
_Sometimes_, sorting by the subject on a cases dashlet will give a datatables error. The actual error if you look at the network response is `Column 'subject' in order clause is ambiguous`.
```
Error Field Error Value
Type DB_Error
Code -1
Message DB Error: unknown error
Mode 16
UserInfo SELECT civicrm_case.id as case_id, civicrm_case.subject as case_subject, civicrm_contact.id as contact_id, civicrm_contact.sort_name as sort_name, civicrm_phone.phone as phone, civicrm_contact.contact_type as contact_type, civicrm_contact.contact_sub_type as contact_sub_type, t_act.activity_type_id as activity_type_id, civicrm_case.case_type_id as case_type_id, civicrm_case.status_id as case_status_id, t_act.status_id as status_id, civicrm_case.start_date as case_start_date, GROUP_CONCAT(DISTINCT IF(case_relationship.contact_id_b = 2, case_relation_type.label_a_b, case_relation_type.label_b_a) SEPARATOR ', ') as case_role, t_act.activity_date_time as activity_date_time, t_act.id as activity_id, case_status.label AS case_status, civicrm_case_type.title AS case_type FROM civicrm_case INNER JOIN civicrm_case_contact ON civicrm_case.id = civicrm_case_contact.case_id INNER JOIN civicrm_contact ON civicrm_case_contact.contact_id = civicrm_contact.id LEFT JOIN civicrm_case_type ON civicrm_case.case_type_id = civicrm_case_type.id LEFT JOIN civicrm_option_group option_group_case_status ON ( option_group_case_status.name = 'case_status' ) LEFT JOIN civicrm_option_value case_status ON ( civicrm_case.status_id = case_status.value AND option_group_case_status.id = case_status.option_group_id ) LEFT JOIN civicrm_case_activity ca4 ON civicrm_case.id = ca4.case_id LEFT JOIN civicrm_activity t_act ON t_act.id = ca4.activity_id AND t_act.is_current_revision = 1 LEFT JOIN civicrm_phone ON civicrm_phone.contact_id = civicrm_contact.id AND civicrm_phone.is_primary = 1 LEFT JOIN civicrm_relationship case_relationship ON ((case_relationship.contact_id_a = civicrm_case_contact.contact_id AND case_relationship.contact_id_b = 2) OR (case_relationship.contact_id_b = civicrm_case_contact.contact_id AND case_relationship.contact_id_a = 2)) AND case_relationship.is_active AND case_relationship.case_id = civicrm_case.id LEFT JOIN civicrm_relationship_type case_relation_type ON case_relation_type.id = case_relationship.relationship_type_id AND case_relation_type.id = case_relationship.relationship_type_id WHERE (1) AND civicrm_case.is_deleted = 0 AND civicrm_contact.is_deleted <> 1 AND (case_relationship.contact_id_b = 2 OR case_relationship.contact_id_a = 2) AND case_relationship.is_active AND civicrm_case.status_id IN (1,3) GROUP BY case_id ORDER BY subject asc LIMIT 0, 10 [nativecode=1052 ** Column 'subject' in order clause is ambiguous]
DebugInfo SELECT civicrm_case.id as case_id, civicrm_case.subject as case_subject, civicrm_contact.id as contact_id, civicrm_contact.sort_name as sort_name, civicrm_phone.phone as phone, civicrm_contact.contact_type as contact_type, civicrm_contact.contact_sub_type as contact_sub_type, t_act.activity_type_id as activity_type_id, civicrm_case.case_type_id as case_type_id, civicrm_case.status_id as case_status_id, t_act.status_id as status_id, civicrm_case.start_date as case_start_date, GROUP_CONCAT(DISTINCT IF(case_relationship.contact_id_b = 2, case_relation_type.label_a_b, case_relation_type.label_b_a) SEPARATOR ', ') as case_role, t_act.activity_date_time as activity_date_time, t_act.id as activity_id, case_status.label AS case_status, civicrm_case_type.title AS case_type FROM civicrm_case INNER JOIN civicrm_case_contact ON civicrm_case.id = civicrm_case_contact.case_id INNER JOIN civicrm_contact ON civicrm_case_contact.contact_id = civicrm_contact.id LEFT JOIN civicrm_case_type ON civicrm_case.case_type_id = civicrm_case_type.id LEFT JOIN civicrm_option_group option_group_case_status ON ( option_group_case_status.name = 'case_status' ) LEFT JOIN civicrm_option_value case_status ON ( civicrm_case.status_id = case_status.value AND option_group_case_status.id = case_status.option_group_id ) LEFT JOIN civicrm_case_activity ca4 ON civicrm_case.id = ca4.case_id LEFT JOIN civicrm_activity t_act ON t_act.id = ca4.activity_id AND t_act.is_current_revision = 1 LEFT JOIN civicrm_phone ON civicrm_phone.contact_id = civicrm_contact.id AND civicrm_phone.is_primary = 1 LEFT JOIN civicrm_relationship case_relationship ON ((case_relationship.contact_id_a = civicrm_case_contact.contact_id AND case_relationship.contact_id_b = 2) OR (case_relationship.contact_id_b = civicrm_case_contact.contact_id AND case_relationship.contact_id_a = 2)) AND case_relationship.is_active AND case_relationship.case_id = civicrm_case.id LEFT JOIN civicrm_relationship_type case_relation_type ON case_relation_type.id = case_relationship.relationship_type_id AND case_relation_type.id = case_relationship.relationship_type_id WHERE (1) AND civicrm_case.is_deleted = 0 AND civicrm_contact.is_deleted <> 1 AND (case_relationship.contact_id_b = 2 OR case_relationship.contact_id_a = 2) AND case_relationship.is_active AND civicrm_case.status_id IN (1,3) GROUP BY case_id ORDER BY subject asc LIMIT 0, 10 [nativecode=1052 ** Column 'subject' in order clause is ambiguous]
PEAR_Exception: DB Error: unknown error in ...\sites\all\modules\civicrm\vendor\pear\pear-core-minimal\src\PEAR.php on line 922
- DB_Error: DB Error: unknown error in unknown on line unknown
Exception trace
# Function Location
0 CRM_Core_Error::exceptionHandler(Object(DB_Error)) ...\sites\all\modules\civicrm\vendor\pear\pear-core-minimal\src\PEAR.php:922
1 PEAR_Error->__construct('DB Error: unknow…', -1, 16, Array, 'SELECT civicrm_c…') ...\sites\all\modules\civicrm\vendor\pear\db\DB.php:997
2 DB_Error->__construct(-1, 16, Array, 'SELECT civicrm_c…') ...\sites\all\modules\civicrm\vendor\pear\pear-core-minimal\src\PEAR.php:575
3 PEAR::_raiseError(Object(DB_mysqli), null, -1, 16, Array, 'SELECT civicrm_c…', 'DB_Error', true) ...\sites\all\modules\civicrm\vendor\pear\pear-core-minimal\src\PEAR.php:223
4 PEAR->__call('raiseError', Array) ...\sites\all\modules\civicrm\vendor\pear\db\DB\common.php:1928
5 DB_common->raiseError(-1, null, null, 'SELECT civicrm_c…', '1052 ** Column '…') ...\sites\all\modules\civicrm\vendor\pear\db\DB\mysqli.php:936
6 DB_mysqli->mysqliRaiseError() ...\sites\all\modules\civicrm\vendor\pear\db\DB\mysqli.php:406
7 DB_mysqli->simpleQuery('SELECT civicrm_c…') ...\sites\all\modules\civicrm\vendor\pear\db\DB\common.php:1234
8 DB_common->query('SELECT civicrm_c…') ...\sites\all\modules\civicrm\packages\DB\DataObject.php:2696
9 DB_DataObject->_query('SELECT civicrm_c…') ...\sites\all\modules\civicrm\packages\DB\DataObject.php:1829
10 DB_DataObject->query('SELECT civicrm_c…') ...\sites\all\modules\civicrm\CRM\Core\DAO.php:472
11 CRM_Core_DAO->query('SELECT civicrm_c…', true) ...\sites\all\modules\civicrm\CRM\Core\DAO.php:1637
12 CRM_Core_DAO::executeQuery('SELECT civicrm_c…') ...\sites\all\modules\civicrm\CRM\Case\BAO\Case.php:583
13 CRM_Case_BAO_Case::getCases(false, Array) ...\sites\all\modules\civicrm\CRM\Case\Page\AJAX.php:186
14 CRM_Case_Page_AJAX::getCases() ...\sites\all\modules\civicrm\CRM\Core\Invoke.php:285
15 CRM_Core_Invoke::runItem(Array) ...\sites\all\modules\civicrm\CRM\Core\Invoke.php:69
16 CRM_Core_Invoke::_invoke(Array) ...\sites\all\modules\civicrm\CRM\Core\Invoke.php:36
17 CRM_Core_Invoke::invoke(Array) ...\sites\all\modules\civicrm\drupal\civicrm.module:471
18 civicrm_invoke('ajax', 'get-cases') ...\includes\menu.inc:527
19 menu_execute_active_handler() ...\index.php:21
20 {main}
Sorry, due to an error, we are unable to fulfill your request at the moment. You may want to contact your administrator or service provider with more details about what action you were performing when this occurred.
DB Error: unknown error
```