Development issueshttps://lab.civicrm.org/groups/dev/-/issues2022-09-03T05:03:50Zhttps://lab.civicrm.org/dev/core/-/issues/427Let CiviCRM accept unix domain sockets for the database URL.2022-09-03T05:03:50ZatheiaLet CiviCRM accept unix domain sockets for the database URL.Currently the database address input of the onboarding page expects an address of the format of $host:$port number.
The issue is that some databases may be set up instead to communicate through a unix domain socket.
A solution would be...Currently the database address input of the onboarding page expects an address of the format of $host:$port number.
The issue is that some databases may be set up instead to communicate through a unix domain socket.
A solution would be to allow for additional address parsing that would recognise the use of a socket instead of a port number.
This address format is supported by mysqli_connect already, so we just need to adapt our handling of the input field. The solution would allow the user to specify the address something like unix:///var/run/mysql-socket.
Best wishes,
Alexhttps://lab.civicrm.org/dev/core/-/issues/428Fatal error in Activity Details report when Sorting uses Section Header2018-11-09T22:11:46ZAllenShawFatal error in Activity Details report when Sorting uses Section Header**Steps to reproduce on dmaster:**
1. Enable debug/backtrace options for better error reporting (this is not required to achieve the relevant fatal error).
1. Open the Activity Details report at http://drupal-demo.amp.l/civicrm/report...**Steps to reproduce on dmaster:**
1. Enable debug/backtrace options for better error reporting (this is not required to achieve the relevant fatal error).
1. Open the Activity Details report at http://drupal-demo.amp.l/civicrm/report/instance/3?reset=1&output=criteria
1. Open the Sort tab and enable the 'Section Header / group by' option for any sort-by column; click *Refresh results* to reload the report with this configuration change.
1. Observe a fatal error page with the following Error Details:
```
Database Error Code: Table 'dmasterciv_jqcej.Array' doesn't exist, 1146
Additional Details:
Array
(
[callback] => Array
(
[0] => CRM_Core_Error
[1] => handle
)
[code] => -18
[message] => DB Error: no such table
[mode] => 16
[debug_info] => SELECT ANY_VALUE(ifnull(civicrm_activity_activity_date_time_date, '')) as civicrm_activity_activity_date_time_date, ANY_VALUE(ifnull(civicrm_contact_contact_assignee, '')) as civicrm_contact_contact_assignee, ANY_VALUE(ifnull(civicrm_contact_contact_target, '')) as civicrm_contact_contact_target, ANY_VALUE(ifnull(civicrm_contact_contact_source_id, '')) as civicrm_contact_contact_source_id, ANY_VALUE(ifnull(civicrm_contact_contact_assignee_id, '')) as civicrm_contact_contact_assignee_id, ANY_VALUE(ifnull(civicrm_contact_contact_target_id, '')) as civicrm_contact_contact_target_id, ANY_VALUE(ifnull(civicrm_activity_id, '')) as civicrm_activity_id, ANY_VALUE(ifnull(civicrm_activity_source_record_id, '')) as civicrm_activity_source_record_id, ANY_VALUE(ifnull(civicrm_activity_activity_type_id, '')) as civicrm_activity_activity_type_id, ANY_VALUE(ifnull(civicrm_activity_activity_subject, '')) as civicrm_activity_activity_subject, ANY_VALUE(ifnull(civicrm_activity_activity_date_time, '')) as civicrm_activity_activity_date_time, ANY_VALUE(ifnull(civicrm_activity_status_id, '')) as civicrm_activity_status_id , count(DISTINCT civicrm_activity_id) as ct from Array group by civicrm_activity_activity_date_time_date [nativecode=1146 ** Table 'dmasterciv_jqcej.Array' doesn't exist]
[type] => DB_Error
[user_info] => SELECT ANY_VALUE(ifnull(civicrm_activity_activity_date_time_date, '')) as civicrm_activity_activity_date_time_date, ANY_VALUE(ifnull(civicrm_contact_contact_assignee, '')) as civicrm_contact_contact_assignee, ANY_VALUE(ifnull(civicrm_contact_contact_target, '')) as civicrm_contact_contact_target, ANY_VALUE(ifnull(civicrm_contact_contact_source_id, '')) as civicrm_contact_contact_source_id, ANY_VALUE(ifnull(civicrm_contact_contact_assignee_id, '')) as civicrm_contact_contact_assignee_id, ANY_VALUE(ifnull(civicrm_contact_contact_target_id, '')) as civicrm_contact_contact_target_id, ANY_VALUE(ifnull(civicrm_activity_id, '')) as civicrm_activity_id, ANY_VALUE(ifnull(civicrm_activity_source_record_id, '')) as civicrm_activity_source_record_id, ANY_VALUE(ifnull(civicrm_activity_activity_type_id, '')) as civicrm_activity_activity_type_id, ANY_VALUE(ifnull(civicrm_activity_activity_subject, '')) as civicrm_activity_activity_subject, ANY_VALUE(ifnull(civicrm_activity_activity_date_time, '')) as civicrm_activity_activity_date_time, ANY_VALUE(ifnull(civicrm_activity_status_id, '')) as civicrm_activity_status_id , count(DISTINCT civicrm_activity_id) as ct from Array group by civicrm_activity_activity_date_time_date [nativecode=1146 ** Table 'dmasterciv_jqcej.Array' doesn't exist]
[to_string] => [db_error: message="DB Error: no such table" code=-18 mode=callback callback=CRM_Core_Error::handle prefix="" info="SELECT ANY_VALUE(ifnull(civicrm_activity_activity_date_time_date, '')) as civicrm_activity_activity_date_time_date, ANY_VALUE(ifnull(civicrm_contact_contact_assignee, '')) as civicrm_contact_contact_assignee, ANY_VALUE(ifnull(civicrm_contact_contact_target, '')) as civicrm_contact_contact_target, ANY_VALUE(ifnull(civicrm_contact_contact_source_id, '')) as civicrm_contact_contact_source_id, ANY_VALUE(ifnull(civicrm_contact_contact_assignee_id, '')) as civicrm_contact_contact_assignee_id, ANY_VALUE(ifnull(civicrm_contact_contact_target_id, '')) as civicrm_contact_contact_target_id, ANY_VALUE(ifnull(civicrm_activity_id, '')) as civicrm_activity_id, ANY_VALUE(ifnull(civicrm_activity_source_record_id, '')) as civicrm_activity_source_record_id, ANY_VALUE(ifnull(civicrm_activity_activity_type_id, '')) as civicrm_activity_activity_type_id, ANY_VALUE(ifnull(civicrm_activity_activity_subject, '')) as civicrm_activity_activity_subject, ANY_VALUE(ifnull(civicrm_activity_activity_date_time, '')) as civicrm_activity_activity_date_time, ANY_VALUE(ifnull(civicrm_activity_status_id, '')) as civicrm_activity_status_id , count(DISTINCT civicrm_activity_id) as ct from Array group by civicrm_activity_activity_date_time_date [nativecode=1146 ** Table 'dmasterciv_jqcej.Array' doesn't exist]"]
)
```
5. Also observe this PHP Notice: `Array to string conversion in CRM_Report_Form_Activity->sectionTotals() (line 1083 of /srv/buildkit/build/dmaster/sites/all/modules/civicrm/CRM/Report/Form/Activity.php).`
**Best guess what's going on:**
The activity Details report assumes the name of a temporary table is stored in `$this->temporaryTables['activity_temp_table']`, but that's really an array. The table name is actually stored in `$this->temporaryTables['activity_temp_table']['name']`.5.7AllenShawAllenShawhttps://lab.civicrm.org/dev/core/-/issues/429Dedupe ignores Phone-Work-Mobile field2022-08-31T05:03:19ZkenDedupe ignores Phone-Work-Mobile fieldCRM_Dedupe_Finder uses a dedupe rule to find contacts who match the supplied fields (eg, when registering participants). This failed in the case when one of the supplied fields was a Phone.
This occurs when you register for an event whe...CRM_Dedupe_Finder uses a dedupe rule to find contacts who match the supplied fields (eg, when registering participants). This failed in the case when one of the supplied fields was a Phone.
This occurs when you register for an event where ...
- The dedupe rule includes a Phone to get a match
- The fields on the register page include a Phone that is not Primary (eg Work Mobile)
- The registration data should match an existing contact if the phone number is used in the dedupe rule
However, a new contact is created.
What is required is to modify CRM_Dedupe_Finder::formatParams() to support dedupe rules that include non-Primary 'phone' fields. This was being blocked because profile fields named 'phone-N-N' were not being recognised as 'phone' fields. The regex that parses the field name recognised 'phone-Primary' and 'phone-Primary-2' and 'phone-2' as 'phone' fields. I extended the regex to recognise 'phone-6-2' as a 'phone' field.https://lab.civicrm.org/dev/core/-/issues/430Child groups of ACL groups cause ACL to fail2022-12-12T05:03:30ZRichChild groups of ACL groups cause ACL to fail## Problem
If you have group A which is used to identify contacts that an ACL grants permissions to, and you add group B as a child group of A, the ACLs fail to allow you to see the contacts in group A.
Seems that [this line](https://l...## Problem
If you have group A which is used to identify contacts that an ACL grants permissions to, and you add group B as a child group of A, the ACLs fail to allow you to see the contacts in group A.
Seems that [this line](https://lab.civicrm.org/dev/core/blob/master/CRM/ACL/BAO/ACL.php#L785) (and again in line 804) is assuming a table alias stored in a PHP serialized field to look like `civicrm_group_contact-N` where `N` is Group A's ID, but actually the alias will be `civicrm_group_contact-N,M` where M is the group B's ID.
Presumably the aliases get less predictable if a group has multiple children/grandchildren.
As far as I have understood, the code is trying to standardise the groups into two buckets: one for smart groups and one for normal groups.
I think the logic should be that groups both A and B should be included in the collected group IDs, since if you're in the child group, you're considered to be in the parent group - I *think*. Although this does mean that if someone has status "Removed" in the parent group (the one that defines which contacts the ACL grants access to), but "Added" in the child group, then the Added will win; access will be granted. So it could get more complicated.
Or the desired behaviour could simply be "no, you have to be actually added to group A since that is the only group explicitly mentioned in the ACL, and that status of group B should not have any bearing on the situation".
## Workaround
To fix this for my client I had to:
1. Edit group B: Remove group A as the parent for group B.
2. Edit group A and just hit Save. This step is necessary to ensure the whereTables and selectTables are updated.
3. Instruct them not to add child groups of ACL groups.
## Discuss
What's the desired behaviour? Is looping through the generated table aliases the most reliable way to implement it?https://lab.civicrm.org/dev/core/-/issues/339Support utf8mb4 so notes can save Emojis and other good things2021-12-16T21:04:12ZgharrisSupport utf8mb4 so notes can save Emojis and other good things(Joe is highjacking this bug report issue to deal with general support for utf8mb4 as suggested by @gharris at https://lab.civicrm.org/dev/core/issues/392#note_8614)
As part of moving to support MySQL8, we should plan on better support ...(Joe is highjacking this bug report issue to deal with general support for utf8mb4 as suggested by @gharris at https://lab.civicrm.org/dev/core/issues/392#note_8614)
As part of moving to support MySQL8, we should plan on better support for utf8mb4 charset.
This issue was originally about bug with emojis that will be solved by this. It needs planning and agreement on approach as well as eventual development.
Now is a good time to consider shifting from charset utf8 (which only supports 3 bytes and doesn't fully implement the UTF8 standard https://medium.com/@adamhooper/in-mysql-never-use-utf8-use-utf8mb4-11761243e434) to the new default charset of utf8mb4, and at same time change the collation from utf8_unicode_ci to the new default of utf8mb4_0900_ai_ci. This will allow new emoticons as well as more languages and their characters to be properly stored in CiviCRM. My sense is that this is not difficult to code in GenCode. However, extensions that define fields in other collations and then compare them to core fields will likely need to be modified. Maybe an extension that changes the charset and collation of all core fields, and that hooks into the dynamic creation of fields for custom fields, is a way to start down this path.
---------- Original description below
Apparently Notes don't like emoji characters. Kind of funny that watching the logs via SSH the emoji renders, but it won't save. After only removing the emoji characters from the text, the note saved correctly.
Characters that were removed:
😳❤️ ❤️ 😜
```
Here's a backtrace:
Aug 17 13:15:39 [info] $backTrace = #0 /basedir/civicrm/civicrm/CRM/Core/Error.php(232): CRM_Core_Error::backtrace("backTrace", TRUE)
#1 /basedir/civicrm/civicrm/packages/PEAR.php(921): CRM_Core_Error::handle(Object(DB_Error))
#2 /basedir/civicrm/civicrm/packages/DB.php(985): PEAR_Error->__construct("DB Error: unknown error", -1, 16, (Array:2), "INSERT INTO civicrm_note (entity_table , entity_id , note , contact_id , modi...")
#3 /basedir/civicrm/civicrm/packages/PEAR.php(575): DB_Error->__construct(-1, 16, (Array:2), "INSERT INTO civicrm_note (entity_table , entity_id , note , contact_id , modi...")
#4 /basedir/civicrm/civicrm/packages/PEAR.php(223): PEAR->_raiseError(Object(DB_mysqli), NULL, -1, 16, (Array:2), "INSERT INTO civicrm_note (entity_table , entity_id , note , contact_id , modi...", "DB_Error", TRUE)
#5 /basedir/civicrm/civicrm/packages/DB/common.php(1907): PEAR->__call("raiseError", (Array:7))
#6 /basedir/civicrm/civicrm/packages/DB/mysqli.php(933): DB_common->raiseError(-1, NULL, NULL, "INSERT INTO civicrm_note (entity_table , entity_id , note , contact_id , modi...", "1366 ** Incorrect string value: '\xF0\x9F\x98\xB3\xE2\x9D...' for column 'not...")
#7 /basedir/civicrm/civicrm/packages/DB/mysqli.php(403): DB_mysqli->mysqliRaiseError()
#8 /basedir/civicrm/civicrm/packages/DB/common.php(1216): DB_mysqli->simpleQuery("INSERT INTO civicrm_note (entity_table , entity_id , note , contact_id , modi...")
#9 /basedir/civicrm/civicrm/packages/DB/DataObject.php(2415): DB_common->query("INSERT INTO civicrm_note (entity_table , entity_id , note , contact_id , modi...")
#10 /basedir/civicrm/civicrm/packages/DB/DataObject.php(1040): DB_DataObject->_query("INSERT INTO civicrm_note (entity_table , entity_id , note , contact_id , modi...")
#11 /basedir/civicrm/civicrm/CRM/Core/DAO.php(571): DB_DataObject->insert()
#12 /basedir/civicrm/civicrm/CRM/Core/BAO/Note.php(173): CRM_Core_DAO->save()
#13 /basedir/civicrm/civicrm/CRM/Note/Form/Note.php(194): CRM_Core_BAO_Note::add((Array:26), (Array:0))
#14 /basedir/civicrm/civicrm/CRM/Core/Form.php(489): CRM_Note_Form_Note->postProcess()
#15 /basedir/civicrm/civicrm/CRM/Core/QuickForm/Action/Upload.php(169): CRM_Core_Form->mainProcess()
#16 /basedir/civicrm/civicrm/CRM/Core/QuickForm/Action/Upload.php(136): CRM_Core_QuickForm_Action_Upload->realPerform(Object(CRM_Note_Form_Note), "upload")
#17 /basedir/civicrm/civicrm/packages/HTML/QuickForm/Controller.php(203): CRM_Core_QuickForm_Action_Upload->perform(Object(CRM_Note_Form_Note), "upload")
#18 /basedir/civicrm/civicrm/packages/HTML/QuickForm/Page.php(103): HTML_QuickForm_Controller->handle(Object(CRM_Note_Form_Note), "upload")
#19 /basedir/civicrm/civicrm/CRM/Core/Controller.php(351): HTML_QuickForm_Page->handle("upload")
#20 /basedir/civicrm/civicrm/CRM/Contact/Page/View/Note.php(182): CRM_Core_Controller->run()
#21 /basedir/civicrm/civicrm/CRM/Contact/Page/View/Note.php(225): CRM_Contact_Page_View_Note->edit()
#22 /basedir/civicrm/civicrm/CRM/Core/Invoke.php(309): CRM_Contact_Page_View_Note->run((Array:4), NULL)
#23 /basedir/civicrm/civicrm/CRM/Core/Invoke.php(84): CRM_Core_Invoke::runItem((Array:14))
#24 /basedir/civicrm/civicrm/CRM/Core/Invoke.php(52): CRM_Core_Invoke::_invoke((Array:4))
#25 /basedir/civicrm/civicrm.php(1246): CRM_Core_Invoke::invoke((Array:4))
#26 /wp-base-dir/wp-includes/class-wp-hook.php(286): CiviCRM_For_WordPress->invoke("")
#27 /wp-base-dir/wp-includes/class-wp-hook.php(310): WP_Hook->apply_filters("", (Array:1))
#28 /wp-base-dir/wp-includes/plugin.php(453): WP_Hook->do_action((Array:1))
#29 /wp-base-dir/wp-admin/admin.php(224): do_action("toplevel_page_CiviCRM")
#30 {main}
```
Edit: Reordered emojis to match the way they appeared in original log file.
Summary of current state (edited Oct. 2, 2018):
Proposed ideas:
I. Create Extension for changing the way database definitions are handled in order to fit the LExIM approach
1. Create blog post and extension developer heads up materials based on proposed changes
2. Create new global variables that reside in civicrm.settings.php, thus creating new defaults for new installations
a. dbChar=utf8mb4 - To define the character set used by the installation
b. dbColl=utf8mb4_unicode_520_ci - To define the collation used by the installation
c. dbEngine=InnoDB - To define whether the table is innodb or myisam or whatever comes next
3. Define new global variables that can replace some of code (I believe that JoeMurray has expressed concerns with this step. If I've misread which concerns are being raised, please redirect.)
a. dbEngineCharColl - a concatenation that would end up looking like "ENGINE=InnoDB DEFAULT CHARACTER SET utf8 COLLATE utf8mb4_unicode_520_ci"
b. dbCharColl - another common concatenation that looks like it is reused multiple times throughout the code
4. During bootstrap, check if the new variables are set, and if they are empty, fill them with the current defaults of utf8 and utf8_unicode_ci, this keeps current installs from breaking on upgrade
5. Replace all instances of database definitions with the appropriate variables (I think this is a set of five SED commands that I haven't built yet!)
II. Create an extension to upgrade existing installs to the new defaults, similar to Eileen's innodbtriggers extension, perhaps utilizing something similar to the script posted above, though it should probably utilize SQL commands directly, rather than a bash script
1. Check current installed version of MySQL to see if utf8mb4_0900_ai_ci is available and act accordingly.
a. For MySQL 5.7 and below the current preferred collation seems to be utf8mb4_unicode_520_ci.
b. For MySQL 8.0 the current preferred collation seems to be utf8mb4_0900_ai_ci.
2. Convert existing database to the determined collation above
a. There's a bash script below to get started (https://lab.civicrm.org/dev/core/issues/339#note_8461)
b. Is this done via PHP script or via MySQL commands?
c. Should this conversion be limited to only the core fields as to not break existing extensions?
3. Change the value of civicrm.settings.php dbChar and dbColl variables to match above via PHP script.
4. Create a warning in System Status if my.cnf defaults do not match dbEngine, dbChar, and dbColl variables
5. Create a hook into post event on creation of custom fields (https://docs.civicrm.org/dev/en/latest/hooks/hook_civicrm_post/) to change the charset and collation of tables and fields created to support newly defined custom fields.
III. Figure out what to do about other Extensions that have hard coded collation/charsets
1. Create helpful documentation for converting code to the Database variable model
2. Create template-able MySQL conversion scripts to be included in the update. Or can this be done with the hook?
IV. Rejoice in the sound of crickets instead of phone calls!https://lab.civicrm.org/dev/core/-/issues/431notice errors on dmaster on create case2018-12-07T19:06:00ZJoeMurraynotice errors on dmaster on create caseCreated a Housing support case on dmaster just now and got:
Notice: Undefined variable: url in CRM_Activity_Selector_Activity::actionLinks() (line 225 of /srv/buildkit/build/dmaster/sites/all/modules/civicrm/CRM/Activity/Selector/Activit...Created a Housing support case on dmaster just now and got:
Notice: Undefined variable: url in CRM_Activity_Selector_Activity::actionLinks() (line 225 of /srv/buildkit/build/dmaster/sites/all/modules/civicrm/CRM/Activity/Selector/Activity.php).
Notice: Undefined variable: qsView in CRM_Activity_Selector_Activity::actionLinks() (line 226 of /srv/buildkit/build/dmaster/sites/all/modules/civicrm/CRM/Activity/Selector/Activity.php).
Notice: Undefined variable: url in CRM_Activity_Selector_Activity::actionLinks() (line 225 of /srv/buildkit/build/dmaster/sites/all/modules/civicrm/CRM/Activity/Selector/Activity.php).
Notice: Undefined variable: qsView in CRM_Activity_Selector_Activity::actionLinks() (line 226 of /srv/buildkit/build/dmaster/sites/all/modules/civicrm/CRM/Activity/Selector/Activity.php).
Notice: Undefined variable: url in CRM_Activity_Selector_Activity::actionLinks() (line 225 of /srv/buildkit/build/dmaster/sites/all/modules/civicrm/CRM/Activity/Selector/Activity.php).
Notice: Undefined variable: qsView in CRM_Activity_Selector_Activity::actionLinks() (line 226 of /srv/buildkit/build/dmaster/sites/all/modules/civicrm/CRM/Activity/Selector/Activity.php).https://lab.civicrm.org/dev/core/-/issues/432Event repeats miss one out incorrectly2022-08-31T05:03:20ZAndy ClarkEvent repeats miss one out incorrectlyUser created an event (a nightshelter) spanning 2 days - started 9:30pm ends 7:30am next day. Then they repeated the event for 2 months, correctly stating they wanted the first repeated event to start on the day the first event finished....User created an event (a nightshelter) spanning 2 days - started 9:30pm ends 7:30am next day. Then they repeated the event for 2 months, correctly stating they wanted the first repeated event to start on the day the first event finished. e.g first event started Oct 10 and ended Oct 11 so first repeated event was to start Oct 11. However the repeated events missed one event - the one on the day the first event finished. The rest were repeated correctly. This is reproducible on the demo systems - screenshot shows that the event for Oct 11 to 12 is missing. ![screenshot_20181010_191112](/uploads/d20ff3033fb18ec6d73750420c444f06/screenshot_20181010_191112.png)https://lab.civicrm.org/dev/financial/-/issues/112Add FormRule to force user to remove "&" from both "Billing First Name" and "...2019-12-19T08:39:04ZKarinGAdd FormRule to force user to remove "&" from both "Billing First Name" and "Billing Last Name" fields**Issue:**
First Name field can be filled out as e.g. "Jon & Mary";
That "&" can cause issues with Payment Processors when it's passed on as part of the Cardholder Name - resulting in rejecting properly entered Credit Card information....**Issue:**
First Name field can be filled out as e.g. "Jon & Mary";
That "&" can cause issues with Payment Processors when it's passed on as part of the Cardholder Name - resulting in rejecting properly entered Credit Card information.
Credit Card Payments with iATS Payments will get a REJ:23 failure code when the Cardholder name contains a "&" - I did some research to see if this is iATS Payments' specific, but I don't think it is.
I found e.g. CartPay -> also has error codes ERROR: 54043: "The Billing Firstname contains invalid characters."
I also found SagePay has such error codes - for every single text field actually:
```
"5037 : The Delivery Address contains invalid characters.","5037 : The Delivery Address contains invalid characters."
"5038 : The Delivery Phone contains invalid characters.","5038 : The Delivery Phone contains invalid characters."
"5039 : The Delivery City contains invalid characters.","5039 : The Delivery City contains invalid characters."
"5040 : The Billing Surname contains invalid characters.","5040 : The Billing Surname contains invalid characters."
"5041 : The Billing Firstname contains invalid characters.","5041 : The Billing Firstname contains invalid characters."
"5042 : The Billing Address1 contains invalid characters.","5042 : The Billing Address1 contains invalid characters."
"5043 : The Billing Address2 contains invalid characters.","5043 : The Billing Address2 contains invalid characters."
"5044 : The Billing City contains invalid characters.","5044 : The Billing City contains invalid characters."
"5045 : The Billing Phone contains invalid characters.","5045 : The Billing Phone contains invalid characters."
"5046 : The Delivery Surname contains invalid characters.","5046 : The Delivery Surname contains invalid characters."
"5047 : The Delivery Firstname contains invalid characters.","5047 : The Delivery Firstname contains invalid characters."
"5048 : The Delivery Address1 contains invalid characters.","5048 : The Delivery Address1 contains invalid characters."
"5049 : The Delivery Address2 contains invalid characters.","5049 : The Delivery Address2 contains invalid characters."
"5050 : The Billing Address contains invalid characters.","5050 : The Billing Address contains invalid characters."
"5051 : The Contact Number contains invalid characters.","5051 : The Contact Number contains invalid characters."
"5052 : The Customer Name contains invalid characters.","5052 : The Customer Name contains invalid characters."
"5053 : The Email Message contains invalid characters.","5053 : The Email Message contains invalid characters."
"5054 : The Cardholder Name contains invalid characters.","5054 : The Cardholder Name contains invalid characters."
"5055 : A Postcode field contains invalid characters.","5055 : A Postcode field contains invalid characters."
"5056 : The Contact Fax contains invalid characters.","5056 : The Contact Fax contains invalid characters."
```
This Image illustrates -> issue (on the left) -> no issue (on the right). Of course the checkbox for My billing address is the same as above is what copies the First Name and Last Name fields down into the Cardholder name bits. The form rule can look for "&" and then tell the user to remove it. Note the user can still leave "Mark & Karin" in the Name and Address Profile. But we would prevent it from going into the Billing details.
![image](/uploads/bd1131f6790be6d085fe3962d4017ffa/image.png)https://lab.civicrm.org/dev/core/-/issues/433Queries combining multiple text labels using REGEXP do not escape regular exp...2019-12-15T05:25:58ZAgilewareQueries combining multiple text labels using REGEXP do not escape regular expression metacharactersMost obvious example of this is the "Fee Level" options for events.
We have a site that had set their price set option like:
Price set option name - $120 (GST incl)
This fails during search, either because of the characters after the ...Most obvious example of this is the "Fee Level" options for events.
We have a site that had set their price set option like:
Price set option name - $120 (GST incl)
This fails during search, either because of the characters after the $ end-of-string match, or because (GST incl) is interpreted as a sub-expression rather than matching the parentheses when it's selected as a fee level to search for.
Currently we're working on a solution using preg_quote.
Agileware Ref: CIVICRM-9905.13.0https://lab.civicrm.org/dev/core/-/issues/434Scheduled Reminder Error On Using Absolute Date With Repeat2019-01-08T07:47:10ZvarshithScheduled Reminder Error On Using Absolute Date With RepeatOverview
--------
When adding a scheduled reminder with an 'absolute_date' initially and then enabling 'Repeat'and adding conditions results in an SQL error when executed.
Technical Details
-----------------
A function in RecipientBuild...Overview
--------
When adding a scheduled reminder with an 'absolute_date' initially and then enabling 'Repeat'and adding conditions results in an SQL error when executed.
Technical Details
-----------------
A function in RecipientBuilder.php, 'prepareRepetitionEndFilter' expects a date field which would be empty when using an 'absolute_date'. This causes the SQL query to be constructed with an empty string inside thus creating a syntax error.5.11https://lab.civicrm.org/dev/core/-/issues/435Improve selection criteria for non-primary mobile numbers in SMS mailings2022-09-02T05:03:35ZJKingsnorthImprove selection criteria for non-primary mobile numbers in SMS mailingsFollowing on from: https://lab.civicrm.org/dev/core/issues/384
It is now possible to send an SMS to someone (direct, or as a mass SMS) when they have their mobile number as their 'non-primary' phone.
However, where someone has multiple...Following on from: https://lab.civicrm.org/dev/core/issues/384
It is now possible to send an SMS to someone (direct, or as a mass SMS) when they have their mobile number as their 'non-primary' phone.
However, where someone has multiple non-primary mobile types the selection criteria is not that smart. eg:
Primary Home Phone 01
Main Mobile 02
Other Mobile 03
In the case above the last one in the list (other mobile) is selected (depending on the other they were created in). This is **sub-optimal**!
One improvement would be to implement some kind of smart ordering. There are several different approaches to this. The email marketing tools handles this in the recipients configuration box with a couple of drop-downs:
![image](/uploads/6ae06c638762b1064a3f3d8fa5181858/image.png)
We could implement something similar here - or maybe allow administrators to 'rank' location types, so that the order can be customised per site. This 'custom location order' could then be applied to email marketing and mass SMS. The complication being that the 'primary' phone should still be preferred, if it is of the type 'mobile'.
Would be great to hear peoples' thoughts/suggestions on this.https://lab.civicrm.org/dev/core/-/issues/436API Get action lookups for alphanumeric custom fields with option values with...2018-10-24T22:17:40ZjackrabbithannaAPI Get action lookups for alphanumeric custom fields with option values with uppercase characters fails to find contactsReplicable on https://dmaster.demo.civicrm.org
Create a custom field on a contact that is Alphanumeric and uses a Multi-Select widget (may be a condition for other custom field types / widgets as well)
Add a few options, but make at le...Replicable on https://dmaster.demo.civicrm.org
Create a custom field on a contact that is Alphanumeric and uses a Multi-Select widget (may be a condition for other custom field types / widgets as well)
Add a few options, but make at least one of the option values contain at least one uppercase character.
Update a contact, give the custom field value one of the options that has an uppercase character in the value.
Use the API anywhere, or use the API explorer, and do a Contact, get call, where custom_[id] = the option with the uppercase character.
No contacts will be returned.
Here where the custom field query is built, the param value for the custom field is made lowercase:
https://github.com/civicrm/civicrm-core/blob/master/CRM/Core/BAO/CustomQuery.php#L338
In this file
https://github.com/civicrm/civicrm-core/blob/master/CRM/Contact/BAO/Query.php#L5687
The operator RLIKE is updated to include the BINARY operator option
Which makes the clause case sensitive...
Introduced in this commit:
https://github.com/civicrm/civicrm-core/commit/a047568869892f9af65002c1709af1931f5f4b0d
This PR: https://github.com/civicrm/civicrm-core/pull/12364
So it looks like a fix for the search builder regex search, has broken the API Contact Get for custom fields with option values with uppercase characters.
This could affect other things?
I'm not sure how to make these two things gel, but surely querying via the CiviCRM API should work, or take precedence.
I'm happy to make a PR, but I'm not sure what is the best thing to do...
Do we really need to make the option value lowercase when building the custom field query where clause?5.8https://lab.civicrm.org/dev/core/-/issues/437Default limit for dedupe screen setting does not behave as expected - remove?2021-04-25T16:14:10ZlarsssandergreenDefault limit for dedupe screen setting does not behave as expected - remove?The "Default limit for dedupe screen" setting does not behave as expected. It limits the duplicate search to the first N contacts in the database, ignoring the selected group. There is no way to dedupe the contacts beyond the limit. This...The "Default limit for dedupe screen" setting does not behave as expected. It limits the duplicate search to the first N contacts in the database, ignoring the selected group. There is no way to dedupe the contacts beyond the limit. This seems to be the desired behaviour when it was set up in 2016:
https://issues.civicrm.org/jira/browse/CRM-18958
https://github.com/civicrm/civicrm-core/pull/8456
But this makes little sense from a user's perspective. I can't think of a reasonable use case for the function as is; limiting by group makes much more sense than by the first N contacts. As it stands, this is confusing as the user expects that if there is a limit, there would be some way to check the next set of contacts (and that the limit would be applied within the selected group).
Should the limit setting be removed until the system is changed? Or is there a use case for this setting and how can we make the explanatory text reflect that?https://lab.civicrm.org/dev/core/-/issues/438Regression? Basic Fiscal relative date filters gone missing2021-02-22T06:26:28Zmagnolia61Regression? Basic Fiscal relative date filters gone missingNot sure when this happened but seems like a regression.
Two of the built-in relative date filters for Fiscal year are gone missing from the UI.
They are still present in the option group and seem to be present in CRM/Utils/Date.php
For ...Not sure when this happened but seems like a regression.
Two of the built-in relative date filters for Fiscal year are gone missing from the UI.
They are still present in the option group and seem to be present in CRM/Utils/Date.php
For us this breaks quite a few smart groups and reports.
"This Fiscal year" and Previous "Fiscal Year" are gone now. Doublechecked in the sandbox.
![missing_thisfiscalyear](/uploads/b292f8be15569768c25131ef1c6e243f/missing_thisfiscalyear.png)https://lab.civicrm.org/dev/core/-/issues/440Proposal - Extension tpls should support the tpl override feature, as support...2020-09-17T09:48:38Zjustinfreeman (Agileware)Proposal - Extension tpls should support the tpl override feature, as supported by CiviCRM core tpls**Current feature:**
Currently, an Extension can ship with tpls but these cannot be overridden which is contrary to the method of overriding CiviCRM core tpls using the **custom templates directory**.
**Proposed change:**
Extension tp...**Current feature:**
Currently, an Extension can ship with tpls but these cannot be overridden which is contrary to the method of overriding CiviCRM core tpls using the **custom templates directory**.
**Proposed change:**
Extension tpls should support the tpl override feature, as supported by CiviCRM core tpls.
So a developer can customise an Extension tpl on a per site basis by adding the tpl file to the correct location in the **custom templates directory**.
Agileware Ref: CIVICRM-994https://lab.civicrm.org/dev/core/-/issues/441Proposal - Add Timezone support for all Date/Time fields in CiviCRM2023-07-02T19:22:33Zjustinfreeman (Agileware)Proposal - Add Timezone support for all Date/Time fields in CiviCRMAdd Timezone support for all Date/Time fields in CiviCRM. Currently, CiviCRM does not have any timezone handling for date/time fields - timezone is not stored with the value.
The first change is to start storing a timezone with the date...Add Timezone support for all Date/Time fields in CiviCRM. Currently, CiviCRM does not have any timezone handling for date/time fields - timezone is not stored with the value.
The first change is to start storing a timezone with the date/time field. After this is implemented then other changes can follow which deal with the display of the date/time field in the local time zone or showing the timezone for the value.
**Proposed Change**
Have ability to set the timezone for a date/time field and store that value with the field.
Default timezone should set using the CMS timezone default and/or a new setting on the Localisation Settings page.
Agileware Ref: CIVICRM-995https://lab.civicrm.org/dev/core/-/issues/442Misc form issues2018-12-08T21:12:48ZAkA84Misc form issuesFound a handful of form-related issues that are pretty quick to fix, just wanted to have the green light on them before opening the PR. All screenshots are taken from https://dmaster.demo.civicrm.org (`v5.8.alpha1`)
## Wrong layout in "...Found a handful of form-related issues that are pretty quick to fix, just wanted to have the green light on them before opening the PR. All screenshots are taken from https://dmaster.demo.civicrm.org (`v5.8.alpha1`)
## Wrong layout in "Tags and Groups" accordion of "New Individual" form
The "Tag(s)" label+select group is missing a `<br>`
*Before*
![tags-before](/uploads/69da499d7792714de2dcf417981cfab7/tags-before.png)
*After*
![tags-after](/uploads/1fc640e4a276e0e0e6b7d70917d7337a/tags-after.png)
## Missing class on text fields in "Dedupe exceptions"
The input fields are missing the standard `crm-form-text class
*Before*
![dedupe-before](/uploads/0861f7c3c98f5b0a8a722b98a858fe92/dedupe-before.png)
*After*
![dedupe-after](/uploads/ca670378bf85ed8b3b8d8caba04764d4/dedupe-after.png)
## Wrong class on `<select>`s in "Submit Credit Card Contribution" modal
The <select> elements have the wrong class: `crm-form-date`
![contribute-select-before](/uploads/09ec405fac0bac56fe5c5cd10db1589f/contribute-select-before.png)
Applying the correct class, `crm-form-select` doesn't change much, but allows Shoreditch to style the selects correctly
![contribute-select-after](/uploads/3fd8125dcae0301d83381e329f543a3e/contribute-select-after.png)
## Textareas instead of text input in "Word Replacements" form
This one I'm not really sure is really a markup error, but it looks to me that instead of `<textarea>` elements, the form would benefit by using `input[type="text"]` fields instead (see how the checkboxes get a vertical alignment, for example)
*Before*
![replacement-before](/uploads/47d5a07307d2bb2d2a19850626ad8ee3/replacement-before.png)
*After*
![replacement-after](/uploads/362b5b8591a1171121562e31ca46dded/replacement-after.png)5.9https://lab.civicrm.org/dev/core/-/issues/443Fix Non-static method calls at CRM_Utils_Mail_Incoming class2018-10-24T20:56:26ZscardiniusFix Non-static method calls at CRM_Utils_Mail_Incoming classOn PHP7.1 class CRM_Utils_Mail_Incoming returns strict warnings due to inconsistency in function declarations
```
Strict warning: Non-static method CRM_Utils_Mail_Incoming::formatMailMultipart() should not be called statically w CRM_Uti...On PHP7.1 class CRM_Utils_Mail_Incoming returns strict warnings due to inconsistency in function declarations
```
Strict warning: Non-static method CRM_Utils_Mail_Incoming::formatMailMultipart() should not be called statically w CRM_Utils_Mail_Incoming::formatMailMultipart()(linia 83 z /srv/staging/ocal.stg.civicrm.pl/public/sites/all/modules/civicrm/CRM/Utils/Mail/Incoming.php).
```5.8https://lab.civicrm.org/dev/core/-/issues/444Updated alterReportVars or new hook for reports2023-08-31T08:51:04ZJoeMurrayUpdated alterReportVars or new hook for reportsThere is no way to currently change a core report to get it to expose custom fields for a component if it does not do it already. It's not unusual an unusual use case to combine creating a custom field with exposing it in a core report....There is no way to currently change a core report to get it to expose custom fields for a component if it does not do it already. It's not unusual an unusual use case to combine creating a custom field with exposing it in a core report. What's needed is to be able to use a hook to add values to $_customGroupExtends. This would also allow extensions to define their own custom fields and add them to a report.
The alterReportVar hook is one place where this functionality could be added https://docs.civicrm.org/dev/en/latest/hooks/hook_civicrm_alterReportVar/. The current three supported varTypes, columns, rows and sql, could be extended by a new one, customFields, or some similar name.
There has been criticism in the past of this hook. IIRC @eileen the concern was especially around how it was not well designed for the sql argument, but I don't recall the details. Perhaps it was that the hook was trying to do too many things and provided too broad a surface.
Rather than further complexifying this hook, an alternative approach would be to create a new one focussed exclusively on the issue of changing custom field exposed by a report. Perhaps it could be call alterReportCustomFields.
So we'd like feedback on
1. Should there be hook support for changing the custom fields exposed by reports?
2. If so, should we change alterReportVar or create a new hook, alterReportCustomFields?https://lab.civicrm.org/dev/core/-/issues/445Proposal - Custom Fields, Is this Field Searchable? option should be enabled ...2021-04-13T03:25:40Zjustinfreeman (Agileware)Proposal - Custom Fields, Is this Field Searchable? option should be enabled by default, currently it is disabled by defaultWhen creating Custom Fields the option **Is this Field Searchable?** is **disabled by default** and as a result any new Custom Field is not searchable. This is frustrating since there is rarely a case when you would not want a field to b...When creating Custom Fields the option **Is this Field Searchable?** is **disabled by default** and as a result any new Custom Field is not searchable. This is frustrating since there is rarely a case when you would not want a field to be available in search. For end users the experience of adding a Custom Field in CiviCRM is like this:
1. Add the Custom Field
1. Add some data to CiviCRM using the new Custom Field
1. Open Search and try to search using the new Custom Field, it is not shown
1. Confusion about why it is not shown in Search
1. Finally, navigate back to the Custom Field and enable the option, Is this Field Searchable?
1. Return to Search and bingo! There it is.
1. Repeat steps for every Custom Field created.
**Proposed Change**
The **Is this Field Searchable?** option should be **enabled by default**, making new Custom Fields available in Search when created - reducing potential confusion as noted above. The user would then have to explicitly disable this option to remove from search.
Agileware Ref: CIVICRM-964