CiviCRM Core issueshttps://lab.civicrm.org/dev/core/-/issues2018-05-13T07:14:33Zhttps://lab.civicrm.org/dev/core/-/issues/112CiviCRM locking may not account for multiple CiviCRM sites per server2018-05-13T07:14:33ZxurizaemonCiviCRM locking may not account for multiple CiviCRM sites per server* [MySQL's `GET_LOCK()` uses server-wide lock names.](https://dev.mysql.com/doc/refman/5.7/en/miscellaneous-functions.html#function_get-lock)
* [CiviCRM's lock name generation does not seem to include the current CiviCRM site / DB name /...* [MySQL's `GET_LOCK()` uses server-wide lock names.](https://dev.mysql.com/doc/refman/5.7/en/miscellaneous-functions.html#function_get-lock)
* [CiviCRM's lock name generation does not seem to include the current CiviCRM site / DB name / identifier](https://github.com/eileenmcnaughton/civicrm-core/blob/b551edf53eff0019a9daed7cc950cf86f70f2e6f/CRM/Core/Lock.php#L111)
* Therefore, multiple CiviCRM sites on the same DB server may compete for the same lock names.
https://lab.civicrm.org/dev/core/-/issues/3622[regression] CiviMail crashing on send instead of throwing exception when usi...2022-06-11T14:57:15ZJonGold[regression] CiviMail crashing on send instead of throwing exception when using Job.executeIn 5.27.0+, [PR #17277](https://github.com/civicrm/civicrm-core/pull/17277) is causing CiviMail to crash in the most common configuration on any SMTP error.
For example, if a contact has an invalid email address, then it could produce a...In 5.27.0+, [PR #17277](https://github.com/civicrm/civicrm-core/pull/17277) is causing CiviMail to crash in the most common configuration on any SMTP error.
For example, if a contact has an invalid email address, then it could produce an SMTP error - and cause a failure which prevents other messages from going out.
### Replication
* Create a contact with an email address that will generate an SMTP error. Screenshot below; you know best how to cause this problem in your own env :)
![Selection_998](/uploads/9670a781edd7e8838d9b4ae725f14212/Selection_998.png)
* Send a transactional email to this user. You should get an error like the one in the screenshot.
* View the Civi logs. You should have two new entries; `Ignoring exception thrown by nullHandler` and a backtrace.
* Now, create a CiviMail to a group containing this contact.
* If you send the mailing via `Job.execute`, you get the error but NOT the backtrace; any other recipients in the mailing will not get the mailing, it exits silently (there will be an error in the Scheduled Job log for `send_mailing`). The email is not placed on hold; CiviMail is now broken and won't send any new mailings, nor finish the existing mailing.
* Now, send the same mailing via `Job.send_mailing`. The bad email address is handled correctly (placed on hold, mailing continues, backtrace appears in log).
### Technical notes
* When you run `Job.execute`, `CRM_Core_JobManager` specifies a [custom fatal error handler](https://github.com/civicrm/civicrm-core/blob/master/CRM/Core/JobManager.php#L44).
* `CRM_Core_Error::nullHandler` calls `CRM_Core_Error::debug_log_message`.
* [PR #17277](https://github.com/civicrm/civicrm-core/pull/17277) adds a call to the custom fatal error handler in `CRM_Core_Error::debug_log_message`.
* Since `CRM_Core_JobManager::CRM_Core_JobManager_scheduledJobFatalErrorHandler` throws an exception, this causes the job to exit silently.
#### Correct backtrace
```
Jul 31 12:05:40 [error] Ignoring exception thrown by nullHandler: 10005, Failed to add recipient: onddddd@zabuntu.lan [SMTP: Invalid response code received from SMTP server while sending email. This is often caused by a misconfiguration in Outbound Email settings. Please verify the settings at Administer CiviCRM >> Global Settings >> Outbound Email (SMTP). (code: 550, response: 5.1.1 <onddddd@zabuntu.lan>: Recipient address rejected: User unknown in local recipient table)]
Jul 31 12:05:40 [debug] $backTrace = #0 /home/jon/local/civicrm-buildkit/build/dmaster/web/sites/all/modules/civicrm/CRM/Core/Error.php(963): CRM_Core_Error::backtrace("backTrace", TRUE)
#1 /home/jon/local/civicrm-buildkit/build/dmaster/web/sites/all/modules/civicrm/vendor/pear/pear-core-minimal/src/PEAR.php(922): CRM_Core_Error::nullHandler(Object(PEAR_Error))
#2 /home/jon/local/civicrm-buildkit/build/dmaster/web/sites/all/modules/civicrm/vendor/pear/pear-core-minimal/src/PEAR.php(577): PEAR_Error->__construct("Failed to add recipient: onddddd@zabuntu.lan [SMTP: Invalid response code rec...", 10005, 16, (Array:2), NULL)
#3 /home/jon/local/civicrm-buildkit/build/dmaster/web/sites/all/modules/civicrm/vendor/pear/pear-core-minimal/src/PEAR.php(236): PEAR::_raiseError(NULL, "Failed to add recipient: onddddd@zabuntu.lan [SMTP: Invalid response code rec...", 10005)
#4 /home/jon/local/civicrm-buildkit/build/dmaster/web/sites/all/modules/civicrm/vendor/pear/mail/Mail/smtp.php(325): PEAR::__callStatic("raiseError", (Array:2))
#5 /home/jon/local/civicrm-buildkit/build/dmaster/web/sites/all/modules/civicrm/vendor/pear/mail/Mail/smtp.php(258): Mail_smtp->send_or_fail((Array:1), (Array:10), NULL)
#6 /home/jon/local/civicrm-buildkit/build/dmaster/web/sites/all/modules/civicrm/CRM/Utils/Mail/FilteredPearMailer.php(68): Mail_smtp->send((Array:1), (Array:10), NULL)
#7 /home/jon/local/civicrm-buildkit/build/dmaster/web/sites/all/modules/civicrm/CRM/Utils/Mail.php(303): CRM_Utils_Mail_FilteredPearMailer->send((Array:1), (Array:10), NULL)
#8 /home/jon/local/civicrm-buildkit/build/dmaster/web/sites/all/modules/civicrm/CRM/Activity/BAO/Activity.php(1538): CRM_Utils_Mail::send((Array:10))
#9 /home/jon/local/civicrm-buildkit/build/dmaster/web/sites/all/modules/civicrm/CRM/Activity/BAO/Activity.php(1236): CRM_Activity_BAO_Activity::sendMessage("\"a a\" <adminnnnn@zabuntu.lan>", "202", "205", "test2", "", "", "onddddd@zabuntu.lan", 630, (Array:0), "", "")
#10 /home/jon/local/civicrm-buildkit/build/dmaster/web/sites/all/modules/civicrm/CRM/Contact/Form/Task/EmailTrait.php(431): CRM_Activity_BAO_Activity::sendEmail((Array:1), "test2", "", "", "onddddd@zabuntu.lan", "202", "\"a a\" <adminnnnn@zabuntu.lan>", (Array:0), "", "", (Array:1), "", (Array:0), NULL, NULL)
#11 /home/jon/local/civicrm-buildkit/build/dmaster/web/sites/all/modules/civicrm/CRM/Contact/Form/Task/EmailTrait.php(351): CRM_Contact_Form_Task_Email->submit((Array:19))
#12 /home/jon/local/civicrm-buildkit/build/dmaster/web/sites/all/modules/civicrm/CRM/Core/Form.php(505): CRM_Contact_Form_Task_Email->postProcess()
#13 /home/jon/local/civicrm-buildkit/build/dmaster/web/sites/all/modules/civicrm/CRM/Core/QuickForm/Action/Upload.php(152): CRM_Core_Form->mainProcess()
#14 /home/jon/local/civicrm-buildkit/build/dmaster/web/sites/all/modules/civicrm/CRM/Core/QuickForm/Action/Upload.php(119): CRM_Core_QuickForm_Action_Upload->realPerform(Object(CRM_Contact_Form_Task_Email), "upload")
#15 /home/jon/local/civicrm-buildkit/build/dmaster/web/sites/all/modules/civicrm/packages/HTML/QuickForm/Controller.php(203): CRM_Core_QuickForm_Action_Upload->perform(Object(CRM_Contact_Form_Task_Email), "upload")
#16 /home/jon/local/civicrm-buildkit/build/dmaster/web/sites/all/modules/civicrm/packages/HTML/QuickForm/Page.php(103): HTML_QuickForm_Controller->handle(Object(CRM_Contact_Form_Task_Email), "upload")
#17 /home/jon/local/civicrm-buildkit/build/dmaster/web/sites/all/modules/civicrm/CRM/Core/Controller.php(347): HTML_QuickForm_Page->handle("upload")
#18 /home/jon/local/civicrm-buildkit/build/dmaster/web/sites/all/modules/civicrm/CRM/Utils/Wrapper.php(98): CRM_Core_Controller->run()
#19 /home/jon/local/civicrm-buildkit/build/dmaster/web/sites/all/modules/civicrm/CRM/Core/Invoke.php(285): CRM_Utils_Wrapper->run("CRM_Contact_Form_Task_Email", "Activities", (Array:1))
#20 /home/jon/local/civicrm-buildkit/build/dmaster/web/sites/all/modules/civicrm/CRM/Core/Invoke.php(68): CRM_Core_Invoke::runItem((Array:14))
#21 /home/jon/local/civicrm-buildkit/build/dmaster/web/sites/all/modules/civicrm/CRM/Core/Invoke.php(36): CRM_Core_Invoke::_invoke((Array:4))
#22 /home/jon/local/civicrm-buildkit/build/dmaster/web/sites/all/modules/civicrm/drupal/civicrm.module(456): CRM_Core_Invoke::invoke((Array:4))
#23 /home/jon/local/civicrm-buildkit/build/dmaster/web/includes/menu.inc(527): civicrm_invoke("activity", "email", "add")
#24 /home/jon/local/civicrm-buildkit/build/dmaster/web/index.php(21): menu_execute_active_handler()
#25 {main}
```5.27.4https://lab.civicrm.org/dev/core/-/issues/113Add new api handler 'current_domain' along the lines of 'user_contact_id'2022-06-13T01:56:11ZeileenAdd new api handler 'current_domain' along the lines of 'user_contact_id'Several api accept domain_id & in almost all cases the value passed is the current domain id. I think we should be able to set that as a default for api like membership type, instead of requiring it to be passed in as is currently the ca...Several api accept domain_id & in almost all cases the value passed is the current domain id. I think we should be able to set that as a default for api like membership type, instead of requiring it to be passed in as is currently the case.
With this we could change
```
function _civicrm_api3_membership_type_create_spec(&$params) {
// todo could set default here probably
$params['domain_id']['api.required'] = 1;
```
to
```
function _civicrm_api3_membership_type_create_spec(&$params) {
$params['domain_id']['api.default'] = 'current_domain';
```
We have a similar thing in place for 'user_contact_id' here
https://github.com/eileenmcnaughton/civicrm-core/blob/fee14197b427c1781e369e5bfd36816afad6d7ee/api/v3/utils.php#L2086 and I feel we could expand that piece of code.
Pinging @totten because Tim re-wrote that earlier handling of user_contact_id & may have some points he wishes to interjecthttps://lab.civicrm.org/dev/core/-/issues/3581Email to activity processing: New feature to skip emails which do not have a ...2022-06-11T14:54:48ZJamie Novick - CompucoEmail to activity processing: New feature to skip emails which do not have a Case ID or Case token- As a CiviCRM administrator
- I would like to configure whether CiviCRM will process emails without a case ID (or case “token”) in the subject line
- so that I can ensure that emails which do not have a case ID are not filed on the con...- As a CiviCRM administrator
- I would like to configure whether CiviCRM will process emails without a case ID (or case “token”) in the subject line
- so that I can ensure that emails which do not have a case ID are not filed on the contact record outside the case by accident.
**How it works currently**
For those a little less familiar with email to activity processing:
CiviCRM will connect to a users MS exchange mailbox and create the following folder structure:
- Inbox
- /CiviCRM
- //CiviMail
- ///ignored
- ///processed
Notes:
- Users simply copy or move emails into the /civicrm folder in their inbox. CiviCRM has a scheduled job that can be configured to run periodically (say every hour) and poll the mailbox folder (Civimail) by IMAP in order to read and process the emails.
- By default CiviCRM will match any email from, to, cc fields to contacts in the CRM and file the email as an activity against those contacts (including recording any attachments as files).
- If however there is a case ID in the subject line (or a case ID "token"*) then CiviCRM will instead file the email straight onto the case itself. The format is: [case #1234] (see: https://issues.civicrm.org/jira/browse/CRM-21446)
- If the email is processed successfully it will be moved to the processed folder.
- If for any reason CiviCRM cannot file the email it will be moved to the ignored folder. This normally happens if the email address is invalid for some reason (please note: emails that are sent internally between staff on exchange server can sometimes have this problem as exchange doesn't always use the external email address but instead uses some local username/domain combination - this maybe something to test and see if it maybe a problem for NEU).
- Note: *When sending out emails from CiviCRM from CiviCase it appends a case ID token - which is a string of characters and not the exact Case ID. This is done to obscure the case ID number in the email. In effect you can have either this token or the case id in the subject line and CiviCRM will file the email correctly.
- More background: https://docs.civicrm.org/sysadmin/en/latest/setup/civimail/inbound/#autofiling-email-activities-via-emailprocessor
**Problem**
For multiple clients they want to use email to activity processing for their casework teams. However sometimes they forget to add the case ID to the subject line of the email and the system incorrectly files the email outside the case - which is a "security" risk as the emails can be sensitive.
As such they would like to be able to specify that the emails being filed from the casework team inbox will only be filed if there is a valid Case ID in the subject line and are skipped if not, and hence there is no risk of the email being filed outside of the case.
**Proposed improvement**
Approach:
When "Used For?" = Email to activity processing
Show an additional option:
- Skip emails which do not have a Case ID or Case token:
- Checkbox
- Help text:
- CiviCRM has functionality to file emails which contain the Case ID or Case Hash in the subject line in the format [case #1234] against a case record.
- Where the Case ID or Case Hash is not included CiviCRM will file the email against the contact record, by matching the email addresses on the email with any email addresses of Contact records in CiviCRM.
- Enabling this option will have CiviCRM skip any emails that do not have the Case ID or Case Hash so that the system will only process emails that can be placed on case records.
- Any emails that are not processed will be moved the ignored folder.
- Default null
- If checked:
- Emails which do not have a valid case ID or case token should be moved into the “ignored” folder. (See folders above) after processing and no Activity should be created.
Would be great to know if we can get the magical "concept approved" flag.
We need to work on this quite urgently so if there are no great concerns that would be much appreciated...5.31.0https://lab.civicrm.org/dev/core/-/issues/114Email-Invoices fails since 4.7.31 because of invalid From-Addresses2018-05-16T02:43:17Zthomas_SYSTOPIAEmail-Invoices fails since 4.7.31 because of invalid From-AddressesReproduce:
1. Enable Tax and Invoicing (in "CiviContribute Component Settings")
2. Setup a From-Email-Adress
3. Create a Contribution
4. Use "Email Invoice" to send an Invoice.
You get: "**Validation failed for: from-email-address**"
T...Reproduce:
1. Enable Tax and Invoicing (in "CiviContribute Component Settings")
2. Setup a From-Email-Adress
3. Create a Contribution
4. Use "Email Invoice" to send an Invoice.
You get: "**Validation failed for: from-email-address**"
The reason could be found in CRM/Contribute/Form/Task/Invoice.php[460:465]:
```
$fromEmail = CRM_Core_BAO_Email::getFromEmail();
// from email address
if (isset($params['from_email_address'])) \{
$fromEmailAddress = CRM_Utils_Array::value($params['from_email_address'], $fromEmail);
}
```
The values of $fromEmail are html-escaped versions of the email-adresses. They might be useful for Select-Options in HTML-Forms. But they won't work for real email-headers.5.2.0https://lab.civicrm.org/dev/core/-/issues/439Contact Reference Field Cutoff on Export2019-08-04T18:55:16ZhershelContact Reference Field Cutoff on ExportOn a brand new Drupal 7 / CiviCRM 5.6.0 install I created a custom field group for Grants and added one Contact Reference field and then created 2 grants. When I search for grants and export and try to export that Contact Reference field...On a brand new Drupal 7 / CiviCRM 5.6.0 install I created a custom field group for Grants and added one Contact Reference field and then created 2 grants. When I search for grants and export and try to export that Contact Reference field, the name of the Contact is limited to 16 characters.
Further testing indicates that the name of a Contact Reference field on a Contact record also is limited to 16 characters when exported.
First reported here: https://civicrm.stackexchange.com/questions/26835/grant-export-custom-field-cutoff5.17.0https://lab.civicrm.org/dev/core/-/issues/3550Email to activity processing: New feature to make the creation of new contact...2022-06-11T14:50:44ZJamie Novick - CompucoEmail to activity processing: New feature to make the creation of new contacts "optional"**User story:**
- As an administrator
- When filing inbound emails
- I would like to configure whether CiviCRM will create a new contact where a contact does not already exist within the system
- So that I can ensure that new contacts a...**User story:**
- As an administrator
- When filing inbound emails
- I would like to configure whether CiviCRM will create a new contact where a contact does not already exist within the system
- So that I can ensure that new contacts are not created in the system when filing an email.
- In the situation of a case email where the email has a valid Case ID or Case token in the subject line the email should still be filed to a case, even if no contacts are matched.
**How it works currently:**
Currently, when CiviCRM performs email to activity processing, Civi will search CiviCRM for a contact with the relevant email address in the From / To / CC fields and then assign the created activity accordingly.
When CiviCRM cannot find a contact with a relevant email address, it simply creates a new "stub" contact with the email address.
**The problem**
For multiple reasons this is might not be desirable behaviour:
This is a security risk, as in most cases CiviCRM email-activity processing works off a mailbox, that might need to be public. As such it is possible that an unscrupulous third party may chose to spam your mailbox and thus spam your CRM with unwanted emails and content. Whilst this can be mitigated by having your automated filing on a separate mailbox, it might be preferable to simply only file the emails for contacts who already exist in your system, and to selectively add the rest.
Also, from a GDPR perspective it may not be desirable to hold the details of the persons who sent an email, whilst the contents of the email might be important (i.e. the attachments) especially in a casework context.
**Proposed solution:**
Add an additional option to the email to activity processing configurations (civicrm/admin/mailSettings?action=add&reset=1)
- When "Used For?" = Email to activity processing
- Show an additional option:
- "Do not create new contacts when filing emails"
- Checkbox
- Default null
- Help:
- If this option is enabled, CiviCRM will not create new contacts when filing emails.
- The email should also always be filed as an activity.
If checked:
- When CiviCRM checks for a matching contact, if no matching contact is found it will not create one and the email is skipped
- unless the email has a valid Case ID or Case token in the subject line, in which it will still be filed against the case, but no contacts will be linked to the email.5.31.0https://lab.civicrm.org/dev/core/-/issues/115Proposal - create fields array standard & generic field template/s for it & f...2022-10-11T15:26:36ZeileenProposal - create fields array standard & generic field template/s for it & for fields, work towards generic entity form templateWe have a generalised issue where we want people to write extensions to alter CiviCRM but in many cases the way CiviCRM is structured makes that difficult and unmaintainable. As part of pushing people out to extensions we also need to wo...We have a generalised issue where we want people to write extensions to alter CiviCRM but in many cases the way CiviCRM is structured makes that difficult and unmaintainable. As part of pushing people out to extensions we also need to work to improve core code to support that purpose.
In the 5.x series we have made it easier for extension writers to store entity specific data by extending custom fields to (almost) all entities via the api*. However, it is still very difficult for extension writers to inject these fields, or other fields, into forms or to remove or re-order them. In general our approach has been to make things more metadata driven but we have not really standardised how to do this for templates. Conversely we already have examples in our code of templates that have been re-written to support output being modified by a php hook - (notably it is possible to alter the Billing fields on the payment form & the output columns in the contribution search from a php hook because metadata is assigned to the template & processed in the template) but we have not standardised those approaches into a recommendation.
To be clear - this proposal is NOT about going through all our forms and altering them to be more editable - it's about agreeing what we are working towards when we ARE editing forms. Currently it is possible to say
* "we are working towards replacing jcalendar.tpl with datepicker fields and when we touch date fields we should attempt to convert them"
* "we are working towards replacing non-metadata ways of adding fields to a form with addField and when we touch fields not added that way we should attempt to convert them"
Conversely we would say "if you are trying to fix the way a datefield is handling date defaults and the field is a jcalendar field we expect the fix to be converting it to a datepicker field rather than adding extra mangling for the legacy method".
In this case we would say
* "we are working towards assigning a standardised array of fields to a form & a tpl way of handling them and when we are working on a form we should seek to convert them"
(Generally we do require that PRs are improving the consistency & quality of our codebase & compliance with code-directions when we merge them)
I would note that complex forms like Contribution & Event forms might make poor candidates for tpl standardisation but a lot of our forms are simple settings forms or editing basic entities and these are good candidates for being generic & standardised. (This is allied with fixing the forms such that IF custom data is enabled on the entity THEN it can be altered via the form - which I will cover separately)
There are 2 open PRs which revolve around making tpls more generic, more metadata driven & more form-alterable
https://github.com/civicrm/civicrm-core/pull/12128/files#diff-1f76de158e02dfb7afa76303ef60e9ebR27
and https://github.com/civicrm/civicrm-core/pull/12078
The proposal would be that where it is possible to iterate through fields we assign them to the tpl in an array that looks like
```
$this-assign('entityFields', [
'field_1' => [
'name' => 'field_1',
'description' => ts("description to show below field"),
'help' => ['id' => 'id-field_1, 'file' => 'CRM/Contact/Form/Contact',
'is_add_translate_dialog' => 1,
],
'field_1' => ['name' => 'field_1'],
'money_field' => ['name' => 'money', 'formatter' => 'crmMoney'],
'weird_looking_field' => [
'name' => 'weird_looking_field',
'template' => CRM/Member/Form/MembershipType/weird_looking_field.tpl',
```
In practice some helpers would be involved in building the array so that fields like is_add_translate_dialog are derived from existing metadata. That would also potentially apply to keys like description, help & formatter. It is expected that if a field were to use a field specific template it would be under the path of the form.
The form template would then work towards looking more like
```
{foreach from=$entityFields item=fieldSpec}
{assign var=fieldName value=$fieldSpec.name}
<tr class="crm-{$entityInClassFormat}-form-block-{$fieldName}">
{include file="CRM/Core/Form/Field.tpl"}
</tr>
{/foreach}
```
With the tpl being
```
{if $fieldSpec.template}
{include file=$fieldSpec.template}
{else}
<td class="label">{$form.$fieldName.label}
{if $fieldSpec.help}{assign var=help value=$fieldSpec.help}{capture assign=helpFile}{if $fieldSpec.help}
{$fieldSpec.help}
{else}''{/if}
{/capture}{help id=$help.id file=$help.file}{/if}
{if $action == 2 && $fieldSpec.is_add_translate_dialog}{include file='CRM/Core/I18n/Dialog.tpl' table=$entityTable field=$fieldName id=$entityID}{/if}
</td>
<td>{if $form.$fieldName.html}{if $fieldSpec.formatter === 'crmMoney'}{$form.$fieldName.html|crmMoney}{else}{$form.$fieldName.html}{/if}{else}{$fieldSpec.place_holder}{/if}<br />
{if $fieldSpec.description}<span class="description">{$fieldSpec.description}</span>{/if}
</td>
{/if}
```
In some cases it might be too hard to make the form iterate through an array but individual fields could be converted - either in order to make that field hook alterable - (ie. the hook could then change the description metadata, or suppress a field) or as part of chipping away at the conversion
```
{assign var=fieldName value='my_converted_field'}
<tr class="crm-{$entityInClassFormat}-form-block-{$fieldName}">
{include file="CRM/Core/Form/Field.tpl"}
</tr>
```
* Note to enable custom data for a new type in an extension use code like the following
```
$optionValues = civicrm_api3('OptionValue', 'get', [
'option_group_id' => 'cg_extend_objects',
'name' => 'civicrm_relationship_type'
]);
if (!$optionValues['count']) {
civicrm_api3('OptionValue', 'create', [
'option_group_id' => 'cg_extend_objects',
'name' => 'civicrm_relationship_type',
'label' => ts('Relationship Type'),
'value' => 'RelationshipType',
]);
}
```https://lab.civicrm.org/dev/core/-/issues/3590Email to activity processing - creates duplicate contacts where verp is present2024-01-29T09:59:12ZeileenEmail to activity processing - creates duplicate contacts where verp is presentWhen contacts reply to a civimail from mail@domain.com they use the verp reply to - eg.
mail+r.2057.84206.03323c082c189038@domain.com
The address mail@domain.com exists in the db but a new contact is created rather than matching themWhen contacts reply to a civimail from mail@domain.com they use the verp reply to - eg.
mail+r.2057.84206.03323c082c189038@domain.com
The address mail@domain.com exists in the db but a new contact is created rather than matching themhttps://lab.civicrm.org/dev/core/-/issues/116Search builder searches on primary addresses are producing unexpected results2018-05-21T11:34:30ZAndryg8Search builder searches on primary addresses are producing unexpected resultsLooking at civicrm.demo.civihosting.com v5.0.0. as well as our v5.2beta1 dev site, I am getting weird results for various search builder searches:
Contacts > Country > Primary is empty - returns many addresses, many of which have countr...Looking at civicrm.demo.civihosting.com v5.0.0. as well as our v5.2beta1 dev site, I am getting weird results for various search builder searches:
Contacts > Country > Primary is empty - returns many addresses, many of which have country field AND are set to primary
Contacts > Country > Home is empty - seems to behave correctly.
Contacts > Country > Primary is NULL - same as above
Contacts > Country > Primary is regex - produces an error:
`
^united$ is not of the type Positive`
Hope that's useful info! Our 5.2Beta1 site has the fixes for https://lab.civicrm.org/dev/core/issues/1065.3.0Monish DebMonish Debhttps://lab.civicrm.org/dev/core/-/issues/3560CiviMail flagged URI_WP_HACKED_2 by SpamAssassin2024-01-29T09:58:35ZbrendanspaarCiviMail flagged URI_WP_HACKED_2 by SpamAssassinOverview
----------------------------------------
CiviCRM emails were reported as going to the Spam folder in Gmail. I did some digging with mail-tester.com and found that SpamAssassin is dinging my mailings -1.629 points for the code UR...Overview
----------------------------------------
CiviCRM emails were reported as going to the Spam folder in Gmail. I did some digging with mail-tester.com and found that SpamAssassin is dinging my mailings -1.629 points for the code URI_WP_HACKED_2.
I did some more digging and found this is likely caused by the WP REST API that was integrated into Civi in 5.25. I'm currently running 5.27.0.
Things I have done that have not resolved the issue:
- I have installed the cv-rest-mailing plugin located here: https://develop.tadpole.cc/plugins/cv-rest-mailing
- I have also ran my site against MalCare which returned a Clean rating.
- I have disabled the plugin WP Mail SMTP.
- I have disabled tracing of opens and click throughs.
I have confirmed that this is occurring on 3 of my clients sites all running Civi 5.27.0 with Mosaico.
I have confirmed this on sites that are running gSuite as their email provider as well as sites that are running a local mail exchanger.
The following two Stack Exchange questions are open with no resolution. I confirmed with Heather O. that she is still facing difficulty.
- https://civicrm.stackexchange.com/questions/35064/civimail-tracked-links-in-wordpress-site-are-marked-down-by-mail-tester?r=SearchResults&s=3|14.7395
- https://civicrm.stackexchange.com/questions/36244/spamassassin-marking-mailings-as-compromised-wordpress-site?r=SearchResults&s=1|42.5634
Reproduction steps
----------------------------------------
1. Create a new group called SpamTesting
2. Create a new individual named Spam Tester
3. Add Spam Tester to the Group SpamTesting
4. Give the individual Spam Tester the email address corresponding to https://www.mail-tester.com/
5. Create a new mailing or reuse an existing mailing.
6. Add SpamTesting as a recipient group.
7. Send the mailing immediately
8. Click check your score on Mail-Tester.com
Current behaviour
----------------------------------------
Many CiviCRM mailings are going to spam after upgrading to 5.27.0. Please see attached image.
![KrY6G_1_](/uploads/9e3ce80bac221077adffbf7380dd8434/KrY6G_1_.png)
Expected behaviour
----------------------------------------
Emails should not be getting flagged as a hacked WordPress site.
Environment information
----------------------------------------
<!-- Some of the items below may not be relevant for every bug - if in doubt please include more information than you think is neccessary. -->
* __Mail Server__ : gSuite
* __Browser:__ Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.105 Safari/537.36
* __CiviCRM:__ _Master/5.27.1
* __PHP:__ 7.3.14
* __CMS:__ WordPress 5.4.2
* __Database:__ _MySQL 8.0.18
* __Web Server:__ _Apache 2.4.41
Comments
----------------------------------------https://lab.civicrm.org/dev/core/-/issues/117Remove usage of each() This is deprecated in php7.22018-05-28T02:48:08ZseamusleeRemove usage of each() This is deprecated in php7.2The following parts of the code still used each() which is an older format of foreach
```
CRM/Activity/BAO/Query.php: while (list(, $k) = each($value)) {
CRM/Activity/BAO/Query.php: while (list($k, $v) = each($value)...The following parts of the code still used each() which is an older format of foreach
```
CRM/Activity/BAO/Query.php: while (list(, $k) = each($value)) {
CRM/Activity/BAO/Query.php: while (list($k, $v) = each($value)) {
CRM/Case/XMLProcessor/Report.php: while (list($gid, $group_values) = each($groupTree)) {
CRM/Case/XMLProcessor/Report.php: while (list($id, $field_values) = each($group_values['fields'])) {
CRM/Event/Form/Participant.php: while (list($k, $dupeCheckContactId) = each($this->_contactIds)) {
CRM/Member/Form/MembershipView.php: $dir = each($relDirection);
CRM/Member/BAO/Membership.php: while (($available > 0) && ($params = each($queue))) {
CRM/Pledge/BAO/Pledge.php: list($field, $value) = each($startDate);
CRM/Pledge/BAO/PledgeBlock.php: list($field, $value) = each($date);
CRM/Report/Form/Contribute/Recur.php: while (list(, $suffix) = each($date_suffixes)) {
CRM/Report/Form/ActivitySummary.php: while (list(, $suffix) = each($date_suffixes)) {
CRM/Utils/Migrate/Import.php: while (list($group_id, $fields) = each($fields_indexed_by_group_id)) {
CRM/Utils/Migrate/Import.php: while (list(, $customFieldXML) = each($fields)) {
CRM/Utils/Token.php: while (list($key) = each($greetingTokens)) {
```
ping @eileen @monish.deb5.3.0https://lab.civicrm.org/dev/core/-/issues/3592(MOVED) [Feature] Please give us the option to disable subtotals for Soft Cre...2022-06-11T14:55:06Zswebervna(MOVED) [Feature] Please give us the option to disable subtotals for Soft Credit Contribution ReportsIssue moved to: https://lab.civicrm.org/dev/report/-/issues/51Issue moved to: https://lab.civicrm.org/dev/report/-/issues/51https://lab.civicrm.org/dev/core/-/issues/118Fix where count() is used on an object that isn't an array nor implements Cou...2018-12-19T14:11:24ZseamusleeFix where count() is used on an object that isn't an array nor implements Countable for php7.2 (tested instances)```
<error type="PHPUnit_Framework_Error_Warning">CRM_Contribute_Import_Parser_ContributionTest::testImportParserWithSoftCreditsByExternalIdentifier with data set #0 ('.')
count(): Parameter must be an array or an object that...```
<error type="PHPUnit_Framework_Error_Warning">CRM_Contribute_Import_Parser_ContributionTest::testImportParserWithSoftCreditsByExternalIdentifier with data set #0 ('.')
count(): Parameter must be an array or an object that implements Countable
/home/seamus/buildkit/build/47.demo/sites/all/modules/civicrm/tests/phpunit/CRM/Contribute/Import/Parser/ContributionTest.php:67
/home/seamus/buildkit/build/47.demo/sites/all/modules/civicrm/tests/phpunit/CiviTest/CiviUnitTestCase.php:182
/home/seamus/buildkit/bin/phpunit5:598
</error>
</testcase>
<testcase name="testImportParserWithSoftCreditsByExternalIdentifier with data set #1" assertions="8" time="0.685860">
<error type="PHPUnit_Framework_Error_Warning">CRM_Contribute_Import_Parser_ContributionTest::testImportParserWithSoftCreditsByExternalIdentifier with data set #1 (',')
count(): Parameter must be an array or an object that implements Countable
/home/seamus/buildkit/build/47.demo/sites/all/modules/civicrm/tests/phpunit/CRM/Contribute/Import/Parser/ContributionTest.php:67
/home/seamus/buildkit/build/47.demo/sites/all/modules/civicrm/tests/phpunit/CiviTest/CiviUnitTestCase.php:182
/home/seamus/buildkit/bin/phpunit5:598
</error>
<error type="PHPUnit_Framework_Error_Warning">CRM_Pledge_BAO_PledgePaymentTest::testRetrieveZeroPledeID
count(): Parameter must be an array or an object that implements Countable
/home/seamus/buildkit/build/47.demo/sites/all/modules/civicrm/tests/phpunit/CRM/Pledge/BAO/PledgePaymentTest.php:100
/home/seamus/buildkit/build/47.demo/sites/all/modules/civicrm/tests/phpunit/CiviTest/CiviUnitTestCase.php:182
/home/seamus/buildkit/bin/phpunit5:598
</error>
<error type="PHPUnit_Framework_Error_Warning">CRM_Pledge_BAO_PledgePaymentTest::testRetrieveStringPledgeID
count(): Parameter must be an array or an object that implements Countable
/home/seamus/buildkit/build/47.demo/sites/all/modules/civicrm/tests/phpunit/CRM/Pledge/BAO/PledgePaymentTest.php:113
/home/seamus/buildkit/build/47.demo/sites/all/modules/civicrm/tests/phpunit/CiviTest/CiviUnitTestCase.php:182
/home/seamus/buildkit/bin/phpunit5:598
</error>
<error type="PHPUnit_Framework_Error_Warning">CRM_Pledge_BAO_PledgePaymentTest::testRetrieveKnownPledgeID
count(): Parameter must be an array or an object that implements Countable
/home/seamus/buildkit/build/47.demo/sites/all/modules/civicrm/tests/phpunit/CRM/Pledge/BAO/PledgePaymentTest.php:127
/home/seamus/buildkit/build/47.demo/sites/all/modules/civicrm/tests/phpunit/CiviTest/CiviUnitTestCase.php:182
/home/seamus/buildkit/bin/phpunit5:598
</error>
<error type="PHPUnit_Framework_Error_Warning">CRM_Pledge_BAO_PledgePaymentTest::testDeletePledgePaymentsNormal
count(): Parameter must be an array or an object that implements Countable
/home/seamus/buildkit/build/47.demo/sites/all/modules/civicrm/tests/phpunit/CRM/Pledge/BAO/PledgePaymentTest.php:137
/home/seamus/buildkit/build/47.demo/sites/all/modules/civicrm/tests/phpunit/CiviTest/CiviUnitTestCase.php:182
/home/seamus/buildkit/bin/phpunit5:598
</error>
<error type="PHPUnit_Framework_Error_Warning">CRM_Pledge_BAO_PledgePaymentTest::testDeletePledgePaymentsNullId
count(): Parameter must be an array or an object that implements Countable
/home/seamus/buildkit/build/47.demo/sites/all/modules/civicrm/tests/phpunit/CRM/Pledge/BAO/PledgePaymentTest.php:160
/home/seamus/buildkit/build/47.demo/sites/all/modules/civicrm/tests/phpunit/CiviTest/CiviUnitTestCase.php:182
/home/seamus/buildkit/bin/phpunit5:598
</error>
<error type="PHPUnit_Framework_Error_Warning">CRM_Pledge_BAO_PledgeTest::testRetrieveZeroPledeID
count(): Parameter must be an array or an object that implements Countable
/home/seamus/buildkit/build/47.demo/sites/all/modules/civicrm/tests/phpunit/CRM/Pledge/BAO/PledgeTest.php:106
/home/seamus/buildkit/build/47.demo/sites/all/modules/civicrm/tests/phpunit/CiviTest/CiviUnitTestCase.php:182
/home/seamus/buildkit/bin/phpunit5:598
</error>
</testcase>
<testcase name="testRetrieveStringPledgeID" class="CRM_Pledge_BAO_PledgeTest" file="/home/seamus/buildkit/build/47.demo/sites/all/modules/civicrm/tests/phpunit/CRM/Pledge/BAO/PledgeTest.php" line="112" assertions="1" time="0.143334">
<error type="PHPUnit_Framework_Error_Warning">CRM_Pledge_BAO_PledgeTest::testRetrieveStringPledgeID
count(): Parameter must be an array or an object that implements Countable
/home/seamus/buildkit/build/47.demo/sites/all/modules/civicrm/tests/phpunit/CRM/Pledge/BAO/PledgeTest.php:117
/home/seamus/buildkit/build/47.demo/sites/all/modules/civicrm/tests/phpunit/CiviTest/CiviUnitTestCase.php:182
/home/seamus/buildkit/bin/phpunit5:598
</error>
</testcase>
<testcase name="testRetrieveKnownPledgeID" class="CRM_Pledge_BAO_PledgeTest" file="/home/seamus/buildkit/build/47.demo/sites/all/modules/civicrm/tests/phpunit/CRM/Pledge/BAO/PledgeTest.php" line="123" assertions="1" time="0.165138">
<error type="PHPUnit_Framework_Error_Warning">CRM_Pledge_BAO_PledgeTest::testRetrieveKnownPledgeID
count(): Parameter must be an array or an object that implements Countable
/home/seamus/buildkit/build/47.demo/sites/all/modules/civicrm/tests/phpunit/CRM/Pledge/BAO/PledgeTest.php:147
/home/seamus/buildkit/build/47.demo/sites/all/modules/civicrm/tests/phpunit/CiviTest/CiviUnitTestCase.php:182
/home/seamus/buildkit/bin/phpunit5:598
</error>
</testcase>
<testcase name="testGetPledgeStartDate" class="CRM_Pledge_BAO_PledgeTest" file="/home/seamus/buildkit/build/47.demo/sites/all/modules/civicrm/tests/phpunit/CRM/Pledge/BAO/PledgeTest.php" line="153" assertions="1" time="0.149295">
<error type="PHPUnit_Framework_Error_Deprecated">CRM_Pledge_BAO_PledgeTest::testGetPledgeStartDate
The each() function is deprecated. This message will be suppressed on further calls
/home/seamus/buildkit/build/47.demo/sites/all/modules/civicrm/CRM/Pledge/BAO/Pledge.php:1203
/home/seamus/buildkit/build/47.demo/sites/all/modules/civicrm/tests/phpunit/CRM/Pledge/BAO/PledgeTest.php:163
/home/seamus/buildkit/build/47.demo/sites/all/modules/civicrm/tests/phpunit/CiviTest/CiviUnitTestCase.php:182
/home/seamus/buildkit/bin/phpunit5:598
</error>
```
ping @eileen @monish.deb5.4.0https://lab.civicrm.org/dev/core/-/issues/3604Can't save Message Template - authorization failed2022-06-11T14:55:44ZkenCan't save Message Template - authorization failedThis problem has been [reported on StackExchange](https://civicrm.stackexchange.com/questions/36233/problem-with-permissions-to-edit-message-templates) with version 5.27.2 and I have experienced it with 5.28.3.
Unless you have permissio...This problem has been [reported on StackExchange](https://civicrm.stackexchange.com/questions/36233/problem-with-permissions-to-edit-message-templates) with version 5.27.2 and I have experienced it with 5.28.3.
Unless you have permission "Administer CiviCRM" you can't edit a Message Template. This fails with an "Authorization failure" message.
I don't understand the code well enough to suggest a fix, but here's a hack to get around it.
The saving of the Message Template is done by this code on line 311 of CRM/Admin/Form/MessageTemplate.php
```
$messageTemplate = MessageTemplate::save()->setDefaults($params)->setRecords([['id' => $this->_id]])->execute()->first();
```
At the point when it comes to checking permissions, there are permissions on 'message_template' for 'get', 'create' and 'update' but not for 'save'. So the default permission is used (Administer CiviCRM).
The following patch to CRM/Core/Permission.php works around the problem ...
```
@@ -1483,6 +1483,7 @@ class CRM_Core_Permission {
$permissions['message_template'] = [
'get' => ['access CiviCRM'],
+ 'save' => [['edit message templates', 'edit user-driven message templates', 'edit system workflow message templates']],
'create' => [['edit message templates', 'edit user-driven message templates', 'edit system workflow message templates']],
'update' => [['edit message templates', 'edit user-driven message templates', 'edit system workflow message templates']],
];
```https://lab.civicrm.org/dev/core/-/issues/119Notice error2018-11-09T22:11:44ZJoeMurrayNotice errorhttp://dmaster.demo.civicrm.org/civicrm/contact/map/event?eid=3&reset=1
Notice: Undefined variable: is_public in CRM_Contact_Form_Task_Map_Event->preProcess() (line 54 of /srv/buildkit/build/dmaster/sites/all/modules/civicrm/CRM/Contact/...http://dmaster.demo.civicrm.org/civicrm/contact/map/event?eid=3&reset=1
Notice: Undefined variable: is_public in CRM_Contact_Form_Task_Map_Event->preProcess() (line 54 of /srv/buildkit/build/dmaster/sites/all/modules/civicrm/CRM/Contact/Form/Task/Map/Event.php).5.8Monish DebMonish Debhttps://lab.civicrm.org/dev/core/-/issues/3601MS Exchange - IMAP-XOAuth2 authentication fails2022-09-12T18:25:55ZtottenMS Exchange - IMAP-XOAuth2 authentication failsOverview
----------------------------------------
https://lab.civicrm.org/dev/core/-/issues/2141 introduced support for (1) obtaining OAuth2 tokens and (2) using the OAuth2 tokens for IMAP-XOAuth2.
In my test setup, I am able to obtai...Overview
----------------------------------------
https://lab.civicrm.org/dev/core/-/issues/2141 introduced support for (1) obtaining OAuth2 tokens and (2) using the OAuth2 tokens for IMAP-XOAuth2.
In my test setup, I am able to obtain a token from Microsoft which appears valid -- but their IMAP server is not accepting the token.
The "Critical Evaluation" below include a more thorough review of theories/data, but here's a high-level summary of debugging efforts:
| Provider | OAuth2 initiation | Use token for IMAP/XOAuth2 | Use token for HTTP/REST |
| -- | -- | -- | -- |
| Microsoft Exchange Online | :white_check_mark: | :stop_sign: fails, tho JWT has IMAP claims | :white_check_mark: |
| Google Mail | :white_check_mark: | :white_check_mark: | (not tested) |
Reproduction steps
----------------------------------------
Configure Civi's OAuth client as described in the draft documentation: https://lab.civicrm.org/documentation/docs/sysadmin/-/merge_requests/278
Recap:
1. Enable `oauth-client`
2. Register the OAuth details in Civi and in Azure Portal.
3. In Civi's "Admin => CiviMail => Mail Accounts", add a mail account
4. For the newly created account, choose "Save & Test".
Current behavior
----------------------------------------
The IMAP server responds with `NO AUTHENTICATE failed.` and `* BAD Command Error. 12`
In the GUI, it appears as:
![Screen_Shot_2020-11-06_at_5.10.49_PM](/uploads/dfa7fd59bd5be48743ef1be7132778c8/Screen_Shot_2020-11-06_at_5.10.49_PM.png)
On a network level, the interaction is:
```
<< * OK The Microsoft Exchange IMAP4 service is ready. [{_LONG_RANDOM_CODE_}]
>> A0001 AUTHENTICATE XOAUTH2 {_THE_XOAUTH2_TOKEN_}
<< A0001 NO AUTHENTICATE failed.
>>
<< * BAD Command Error. 12
>> A0002 LOGOUT
<< * BAD Command Error. 12
```
(To see this, I applied [a hack to log network I/O](https://gist.github.com/totten/05de1aaf6948513792a40fb6f1eafb26) and re-ran the check via CLI.)
Expected behavior
----------------------------------------
The IMAP test should succeed.
Environment information
----------------------------------------
* __Browser:__ _Firefox_
* __CiviCRM:__ _5.32.beta1_
* __PHP:__ _7.1_
* __CMS:__ _Drupal 7_
* __Database:__ _MySQL 5.6_
* __Web Server:__ _Apache 2.4_
Critical evaluation
----------------------------------------
Here are a few theories for where the problem may reside.
1. The problem is in the OAuth2 client.
1. The OAuth2 client is broken - it does not obtain a real token from Microsoft.
2. The OAuth2 client obtains a real token - but the token does not have the proper scopes.
3. The OAuth2 client obtains a valid token with proper scopes - but it doesn't speak IMAP-XOAuth2 correctly.
2. The problem is in the OAuth2 service provider (either the authorization-server or the resource-server/IMAP-server).
1. There is a problem in the account flags/configuration.
2. There is a bug in the OAuth2 service provider.
Let's consider each in turn to see if we can prove/disprove the theory:
## (Theory 1.1) The OAuth2 client is broken - it does not obtain a real token from Microsoft.
This explanation doesn't hold -- because the same token works for other scenarios.
* Navigate to "Admin => System Settings => OAuth => ms-exchange"
![Screen_Shot_2020-11-06_at_5.41.30_PM](/uploads/c7cd3286bdebf306c40466d3b22b7555/Screen_Shot_2020-11-06_at_5.41.30_PM.png)
* Click "Inspect"
* Find "Access Token: Raw". Copy the value.
* In CLI, make an HTTP request to MS with the same token:
```
$ curl 'https://graph.microsoft.com/v1.0/me' \
-H 'Accept: application/json' \
-H 'Authorization: Bearer <TOKEN_VALUE>'
{"@odata.context":"https://graph.microsoft.com/v1.0/$metadata#users/$entity","businessPhones":["XXX-XXX-XXXX"],"displayName":"My Name","givenName":"My","jobTitle":null,"mail":"myname@wariomail.onmicrosoft.com","mobilePhone":null,"officeLocation":null,"preferredLanguage":"en-US","surname":"Name","userPrincipalName":"myname@wariomail.onmicrosoft.com","id":"XXXXX-XXXX-XXXXX"}
```
* The response is well-formed and has the correct information.
## (Theory 1.2) The OAuth2 client obtains a real token - but the token does not have the proper scopes.
I can show that the client works according to spec.
* In the Microsoft documentation [Authenticate an IMAP, POP or SMTP connection using OAuth](https://docs.microsoft.com/en-us/exchange/client-developer/legacy-protocols/how-to-authenticate-an-imap-pop-smtp-application-by-using-oauth#configure-your-application), we can see the required scope is `IMAP.AccessAsUser.All`.
* In Azure Portal, navigate to "Azure Active Directory => App registrations => (my-app) => API permissions". Note the list of permissions includes IMAP:
![Screen_Shot_2020-11-06_at_6.06.48_PM](/uploads/739adc203b05b30fdeb3efd152b340f7/Screen_Shot_2020-11-06_at_6.06.48_PM.png)
* In CiviCRM, navigate to "Admin => System Settings => OAuth => ms-exchange"
* In the "Details" tab, we can see the list of scopes that requested during setup:
![Screen_Shot_2020-11-06_at_5.56.41_PM](/uploads/9f23d454175a2ed8d8b16da55b5a4bd1/Screen_Shot_2020-11-06_at_5.56.41_PM.png)
* In the "Client #X" tab, we can "Inspect" the token.
* Observe: The "Token Record" shows that the `raw` response from the OAuth2 service. It indicates approval for the necessary scopes:
![Screen_Shot_2020-11-06_at_5.58.47_PM](/uploads/3ebf06fe493d8aa83d22c25fc3ce3380/Screen_Shot_2020-11-06_at_5.58.47_PM.png)
* Observe: The "Access Token: JWT Payload" also reports these scopes:
```
"scp": "IMAP.AccessAsUser.All POP.AccessAsUser.All SMTP.Send User.Read profile openid email",
```
So, it appears that every component (from Azure app registration, to the client's request configuration, to the received token data, to the decoded JWT token data) reports `IMAP.AccessAsUser.All` as an included scope.
There is a subtle discrepancy in how the scope is labeled - sometimes, it's presented as the shorter `IMAP.AccessAsUser.All`; other times, it's the longer `https://outlook.office.com/IMAP.AccessAsUser.All`. To be sure, I've requested tokens with both notations -- and in both cases, the outcome is the same. (To wit: the JWT token has `IMAP.AccessAsUser.All` - but it does not work for IMAP access.)
## (Theory 1.3) The OAuth2 client obtains a valid token with proper scopes - but it doesn't speak IMAP-XOAuth2 correctly.
I've tested this theory a few ways:
1. Connect to another service (Google Mail) which uses the same protocol (IMAP-XOauth2). It works. :white_check_mark:
2. Hack the PHP IMAP library (`imap_transport.php`) to inspect the wire-communication. Manually decode the XOAuth2 statement to ensure that it follows [the XOAuth2 formula](https://docs.microsoft.com/en-us/exchange/client-developer/legacy-protocols/how-to-authenticate-an-imap-pop-smtp-application-by-using-oauth#sasl-xoauth2). :white_check_mark:
3. Write a new script with a different PHP IMAP library to try the same token. The alternate library works with Google Mail but not Microsoft Exchange.
## (Theory 2.1) There is a problem in the account flags/configuration.
As I mentioned above (theory 1.2), the "App registration" has all the scopes described in the docs
![Screen_Shot_2020-11-06_at_6.06.48_PM](/uploads/afe29d69f170150a89b98802fd5262b8/Screen_Shot_2020-11-06_at_6.06.48_PM.png)
In the "Exchange Administration Center", the account has IMAP enabled:
![Screen_Shot_2020-11-06_at_6.32.17_PM](/uploads/a5980fff21dcf85add1df96d03e1c9b2/Screen_Shot_2020-11-06_at_6.32.17_PM.png)
In the Exchange webmail UI, there is a settings page with a search-box. Search for "IMAP", and it shows:
![Screen_Shot_2020-11-06_at_6.27.06_PM](/uploads/a4ae9d1aebae308df98a41b3ac348379/Screen_Shot_2020-11-06_at_6.27.06_PM.png)
It's peculiar that searching for "IMAP" presents an inert radio-button for POP, but it does show connection details for IMAP.
TBH, Microsoft's backend here is labyrinthine, so it's hard to be sure that I've checked every possible permission. But I can't find anything else that would prevent IMAP access.
Perhaps someone with more experience in Microsoft's backend should try to run on a different account? Perhaps an account that has been demonstrated to work with IMAP?
## (Theory 2.2) There is a bug in the OAuth2 service provider.
I can't really prove or disprove this theory. I don't really want to believe this is the problem (*surely this stuff works for other people...*). OTOH, maybe it's actually quite rare for people to use IMAP+XOAuth2 with Exchange Online...5.32.0https://lab.civicrm.org/dev/core/-/issues/120Advanced Search - Contacts throws Fatal Error v 5.1.12018-05-17T02:11:00ZkcristianoAdvanced Search - Contacts throws Fatal Error v 5.1.1Advanced Search for Contacts White Screens - It works for Participants, Members, but not contacts.
Tested on official Download of CiviCRM 5.1.1.
Tested with:
Drupal 7
CiviCRM 5.1.1
php 7.0
MariaDB 10.1
WP 4.9.5
CiviCRM 5.1.1
php 7...Advanced Search for Contacts White Screens - It works for Participants, Members, but not contacts.
Tested on official Download of CiviCRM 5.1.1.
Tested with:
Drupal 7
CiviCRM 5.1.1
php 7.0
MariaDB 10.1
WP 4.9.5
CiviCRM 5.1.1
php 7.0
MariaDB 10.2
WP 4.9.5
CiviCRM 5.1.1
php 7.0
MariaDB 10.1
WP 4.9.5
CiviCRM 5.1.1
php 7.0
MySQL 5.6
All fail on a contact search with `PHP Fatal error: Uncaught Error: Call to undefined method CRM_Core_DAO::disableFullGroupByMode() in /home/ranger/public_html/wp-content/plugins/civicrm/civicrm/CRM/Contact/BAO/Query.php:4848 `
These commits are in 5.1.1 - https://github.com/civicrm/civicrm-core/pull/12074 https://github.com/civicrm/civicrm-core/pull/11996
but https://github.com/civicrm/civicrm-core/pull/12043 is not in 5.1.1
Will test adding https://github.com/civicrm/civicrm-core/pull/12043 and see if that fixes.5.1.0https://lab.civicrm.org/dev/core/-/issues/3551Clickthrough tracking breaks when a URL has a leading space2022-06-11T14:50:48ZJonGoldClickthrough tracking breaks when a URL has a leading spaceSteps to replicate:
* Start a new CiviMail mailing.
* Create a link where the URL has a leading whitespace, e.g.:
```
<a href=" https://civicrm.org">
```
* Send the mailing with clickthrough tracking enabled.
* Click the link.
##### Exp...Steps to replicate:
* Start a new CiviMail mailing.
* Create a link where the URL has a leading whitespace, e.g.:
```
<a href=" https://civicrm.org">
```
* Send the mailing with clickthrough tracking enabled.
* Click the link.
##### Expected Result
I arrive at https://civicrm.org.
##### Actual Result
I end up on the dashboard of my Civi install (or access denied, depending on whether I'm logged in).
Since leading whitespace in an `HREF` attribute is handled gracefully, I think CiviCRM should handle it identically to if clickthrough tracking was disabled.
This is just a matter of adding a `trim()`.5.33.0JonGoldJonGoldhttps://lab.civicrm.org/dev/core/-/issues/121Can't change 'Accept profile submissions from external sites' on Joomla2020-02-27T20:16:18Zaydunsaidan.saunders@squiffle.ukCan't change 'Accept profile submissions from external sites' on JoomlaJoomla 3.8.7 CiviCRM 5.1.1
Administration > System Settings > Misc accepts changes and 'Save' does not show any errors but the new value for 'Accept profile submissions from external sites' is not saved, so reloading the page shows the ...Joomla 3.8.7 CiviCRM 5.1.1
Administration > System Settings > Misc accepts changes and 'Save' does not show any errors but the new value for 'Accept profile submissions from external sites' is not saved, so reloading the page shows the original unmodified setting.
However, @kcristiano tested on Joomla successfullyhttps://lab.civicrm.org/dev/core/-/issues/3646Support tracking URLs with tokens in query strings2022-06-11T14:58:11ZlarsssandergreenSupport tracking URLs with tokens in query stringsURLs with tokens in mailings do not have click tracking. This is an important metric for fundraising (e.g. to see how many and even which contacts click through to contribution pages). @artfulrobot put together some code that worked well...URLs with tokens in mailings do not have click tracking. This is an important metric for fundraising (e.g. to see how many and even which contacts click through to contribution pages). @artfulrobot put together some code that worked well to track these clicks in Flexmailer, but that code was not merged before Flexmailer was merged into core.
https://github.com/civicrm/org.civicrm.flexmailer/issues/30
We used this without issue for four months and found it worked very well. @artfulrobot was using it since March. Would it be possible to merge this into core. Rich, would you be willing to copy your code to a patch against core?RichRichhttps://lab.civicrm.org/dev/core/-/issues/122Wrong Action Links Shown for Reserved and Locked Option Groups2018-05-19T06:07:03ZmichaelWrong Action Links Shown for Reserved and Locked Option GroupsIn #55 I hid the "delete" link for options in a locked option group. This has a bug as it will already not have the "delete" link if the option value is reserved:
```
if ($dao->is_reserved) {
$action = CRM_Core_Action::UPDATE;
}
```
...In #55 I hid the "delete" link for options in a locked option group. This has a bug as it will already not have the "delete" link if the option value is reserved:
```
if ($dao->is_reserved) {
$action = CRM_Core_Action::UPDATE;
}
```
So subtracting the `CRM_Core_Action::DELETE` will result in a negative value for `$action` and more links than expected will be shown.
This can be fixed by first checking if the `CRM_Core_Action::DELETE` flag is active before unsetting it.
---
Link to PR: https://github.com/civicrm/civicrm-core/pull/12154/files5.3.0https://lab.civicrm.org/dev/core/-/issues/123Import - Participant - Custom participant date fields are not formatted2022-06-11T16:02:19ZtschuettlerImport - Participant - Custom participant date fields are not formattedThe date fields are not converted from the import date format to the default date format and thus end up beeing imported with a date value of 0.
![image](/uploads/dda9843f54de137b5060553f4785655f/image.png)
Additionally 2 notice errors...The date fields are not converted from the import date format to the default date format and thus end up beeing imported with a date value of 0.
![image](/uploads/dda9843f54de137b5060553f4785655f/image.png)
Additionally 2 notice errors appear for each mapped custom particpant field in the import file:
>Notice: Undefined offset: 8 in CRM_Event_Import_Parser_Participant->import() (line 296 of /opt/buildkit/build/dmaster/sites/all/modules/civicrm/CRM/Event/Import/Parser/Participant.php).
>Notice: Undefined offset: 8 in CRM_Event_Import_Parser_Participant->import() (line 300 of /opt/buildkit/build/dmaster/sites/all/modules/civicrm/CRM/Event/Import/Parser/Participant.php).
Steps to reproduce:
1. Create a custom date field for participants
1. Import a participant with a non default date format: [participant_test.csv](/uploads/83f09e28f2435b819b41e120a9e28968/participant_test.csv)
See https://issues.civicrm.org/jira/browse/CRM-19386 for the same issue with activity imports.
A PR will be provided.5.3.0https://lab.civicrm.org/dev/core/-/issues/3613System Workflow Messages - Improve localization experience2024-01-14T14:55:57ZtottenSystem Workflow Messages - Improve localization experience## Background
Several CiviCRM components - such as CiviContribute, CiviCase, and CiviEvent - send automated email notifications. Many organizations find it useful to customize/tune these emails, and the emails may be administered in the...## Background
Several CiviCRM components - such as CiviContribute, CiviCase, and CiviEvent - send automated email notifications. Many organizations find it useful to customize/tune these emails, and the emails may be administered in the web UI ("Administer: Message Templates: System Workflow Messages").
In principle, this is a powerful tool in which an administrator may use Smarty templating to produce highly tuned messages -- and it requires only the ability to understand template notations (e.g. `{$contact.first_name}` or `{$event.start_date}`). However, in practice, this is often difficult -- real-world templates are long and complex, and they may be used with data of varying quality/character, and the workflow to test a template can be tedious.
## Goals
This epic (*overarching issue*) aims to improve the experience of maintaining system workflow templates. We are specifically funded to work on it from the following perspective: You have an organization which maintains multiple templates for use in multiple countries/locales -- periodically, as campaigns/operations are revised, you may need to engage with a few people to update+retest a few templates. We have three main themes to improve upon:
* __Drafting/Workflow__: Currently, Civi has exactly one variant/revision of a template, and it is always live. There is no way to develop a draft (new revisions) or to keep a record of past revisions.
* __~~Translation~~ Localization__: For any given message, you may need to compose different versions for different locales.
* __Previews/Samples__: Many templates are sensitive to fine-grained details. This is true in multiple ways, e.g. (1) the template adjusts based on *available data* ("Do we have a mailing address for this person? Do we have a full-name, or just an email? Is this one-off contribution or a recurring contribution?"), and (2) the template includes *precise codes* that must be well-formed (e.g. matching tags, matching quotes, using an email-friendly dialect of HTML/CSS).
## Related work
* https://lab.civicrm.org/extensions/msgtpltools
* https://lab.civicrm.org/community/feature-request/-/issues/26
* https://github.com/eileenmcnaughton/civi-data-translate
* https://lab.civicrm.org/extensions/l10nx
## Subtopics / Brainstorming
(Some of the descriptions here are terse - but they may get more explanation in the wireframe.)
* __Sample data presentation__: While composing a message, how do you show a preview based on sample data? Some ideas:
* __Edit-Preview Split-Panes__
* __Pro__: very amenable to live-update
* __Con__: needs wide screen (prob dialog). agreeable with O(10) sample records - but disagreeable with O(1000) records
* __Edit-Preview Accordion__
* __Pro__: fairly amenable to live-update. full width.
* __Con__: agreeable with O(10) sample records - but disagreeable with O(1000) records
* __Explore-Preview Panes in dialog__
* __Pro__: agreeable with O(1000) records
* __Con__: requires more navigating.
* __Comment__: live-update might be possible with separate popup - but more difficult
* __Comment__: might mitigate navigation work with a hotkey
* __Sample data source__: When viewing a preview based on some example record(s), where does the example come from?
* __Basic/prepackaged__: Define some JSON files, which are linked to the workflows.
* __Comment__: Requires metadata to say, "workflows $a+$b can use sample data $c+$d"
* __Pro__: Some templates have very nuanced purposes and may have special vars. This allows you to mock data that's attuned to edge-cases, and you can use these samples in any environment (dev-site/public-demo-site/live-site). No mixing-up real records and sample records. Can simulate complex data (e.g. "tpl for use-case with 2 contribution records")
* __Con__: More upfront work in making sample data. User cannot test against real data.
* __Comment__: Should have unit-test to ensure sample-data-structure stays in sync with real-data-structure
* __Data search__: Search for a live record (eg `Contribution` or `Membership`) to use as a sample.
* __Comment__: Requires metadata to say, "workflows $a+$b need data from entity $c"
* __Pro__: You can use real data, which can be subjectively attuned.
* __Con__: You have to find real data that's *useful*. May be awkward if you're trying to test edge-cases, unless you're willing to add mock/synthetic records to the DB. All mail-merge data must be selectable from one record (eg "Contribution #123"; eg no other inputs besides DB content).
* __Variants__: The UI for this can take on a few variations, eg
* In "Edit Msg Tpl", show a searchable tree. Filter by contact name and select the target record.
* In "Edit Msg Tpl", ask the user to enter a numeric ID.
* In "View Contact", add links/actions for generating previews.
* __Flagged records__: As an adminsitrator, find real records and save them in a list (e.g. "Record #$x is a sample for use in workflows $a, $b")
* __Comment__: Requires somewhere to store the list (e.g. custom-field or new-field or new-table)
* __Pro__: You can use real data, which can be subjectively attuned. The list of samples is curated/abbreviated.
* __Con__: The individual composing the template may not know how to flag/unflag sample records. Requires configuration (and extant samples) before you can use email preview.
* __Grain of ~~translation~~ localization__
* __Record level__: For each `(locale,workflow)`, create another `civicrm_msg_template` record.
* __Field level__: Each `(workflow)` only has one `civicrm_msg_template` record. Attached to that record are multiple translations (ex: `field=subject,locale=fr_FR,text="Bonjour"`).
* __String level__: Each `(workflow)` only has one `civicrm_msg_template` record, and that record has one `body_text`. However, the `body_text` uses translatable strings (ex: 'body_text={cts id=greeting}Greetings,{/cts}`)
* Trade-off
| Record-level | Field-level | String-level |
| -- | -- | -- |
| Each translation goes through a separate, well-delineated drafting workflow. | Translation workflow is batched (per template). All translations must be ready to activate together. | Orthogonal workflows for templates vs strings. |
| Message structure (paras/sections) is localized. | Message structure (paras/sections) is localized. | Message structure is uniform across locales. (Realistic? What abouts `<table>` LTR/RTL? |
| Cannot re-use translated snippets. | Cannot re-use translated snippets. | Re-use translated snippets. |
## Wireframes
There are a few different ways to approach this. To get a sense for the usability and work, we can post a few alternative wireframes. (*This section may be updated as more are added.*)
* __(0) Status quo__: [_0__Sys_Wf_Msgs](/uploads/ad96003d71133d298f95ce79902a7583/_0__Sys_Wf_Msgs.png), [_0__Edit_Msg_Tpl](/uploads/81bc234651182b65993bc28c8052b68b/_0__Edit_Msg_Tpl.png)
* __(1) Record-level translations w/previews and drafting workflow__: [_1Rec__Sys_Wf_Msgs](/uploads/532b842406c17c31be5ae9195ceb6aa8/_1Rec__Sys_Wf_Msgs.png), [_1Rec__Edit_Msg_Tpl](/uploads/13a1f717e829e3de480750c4893969a7/_1Rec__Edit_Msg_Tpl.png) (*Note: The blue-arcs indicate a couple variations on how to include the preview/sample functionality. This tracks close to status-quo, and incremental changes are highlighted with yellow-dots.*)
* __(2) Field-level translations__: [_2Fld__Sys_Wf_Msgs](/uploads/7e1f46e722d8afc706a92c56e295c55e/_2Fld__Sys_Wf_Msgs.png), [_2Fld__Edit_Msg_Tpl](/uploads/4bc932b7bb3f304853b0c107b98d41b5/_2Fld__Edit_Msg_Tpl.png) (*Note: This tracks close to status-quo, and incremental changes are highlighted with yellow-dots.*)
* __(3) String-level translations__: [_3Subfld__Sys_Wf_Msgs](/uploads/f52ffbb764917954790fbbd3ed807acf/_3Subfld__Sys_Wf_Msgs.png), [_3Subfld__Edit_Msg_Tpl](/uploads/b20bc83d302d55a485d975aff6ca70e1/_3Subfld__Edit_Msg_Tpl.png) (*Note: This tracks close to status-quo, and incremental changes are highlighted with yellow-dots.*)
* __(4) Record-level translations w/browse-edit tree + accordion-preview__: [_4Tree__Browse-Edit](/uploads/8cc47238a9d597f80950c1d98c0ea43a/_4Tree__Browse-Edit.png)
* __(5) Field-level translations w/browse-edit tree + accordion-preview__ [_5Tree__Browse-Edit](/uploads/aab231199d0b4217a2f44f9ad45aecb1/_5Tree__Browse-Edit.png)
* __(6) Field-level translations w/matrix, preview-dialog, and drafting workflow__: [_6Mat__Sys_Wf_Msgs](/uploads/3a32390f36f50d8e158e1155dc586969/_6Mat__Sys_Wf_Msgs.png), [_6Mat__Edit_Msg_Tpl](/uploads/353f97846fd9d2b1e037ec92e60101fc/_6Mat__Edit_Msg_Tpl.png) (*Note: This assumes that there is a separate string-storage service like [civi-data-translate](https://github.com/eileenmcnaughton/civi-data-translate), and it assumes that the `String` record has some workflow property like like `is_draft` or `revision`. So each string might have the properties `entity`,`entity_id`,`field`,`locale`,`text`,`is_draft`.*)
* __(7) As with 6, but with various refinements__: [_7Mat__Sys_Wf_Msgs](/uploads/73d37acab4307054b4fe4f3291ae7dcd/_7Mat__Sys_Wf_Msgs.png),
[_7Mat__Edit_Msg_Tpl](/uploads/4807652bd20bfb5bf0912bf24f05d444/_7Mat__Edit_Msg_Tpl.png)https://lab.civicrm.org/dev/core/-/issues/124Registration approval issues2018-06-17T23:18:36Zaydunsaidan.saunders@squiffle.ukRegistration approval issuesFor events requiring approval:
1) The confirmation screen generates a warning:
`Warning: A non-numeric value encountered in XXX/civicrm/CRM/Event/Form/Registration/Confirm.php on line 262`
2) The pre-approval confirmation and post-appr...For events requiring approval:
1) The confirmation screen generates a warning:
`Warning: A non-numeric value encountered in XXX/civicrm/CRM/Event/Form/Registration/Confirm.php on line 262`
2) The pre-approval confirmation and post-approval confirmation emails both have the subject 'Registration Confirmation' which is confusing to recipients.
3) The pre-approval confirmation mail includes a Fees section which shows fees of 0 since the fees have not been selected at this stage. So it would be better to remove the Fees section from the pre-approval confirmation (but not the post-approval confirmation)5.4.0aydunsaidan.saunders@squiffle.ukaydunsaidan.saunders@squiffle.ukhttps://lab.civicrm.org/dev/core/-/issues/3548(Deleted)2022-06-11T14:50:42Zswebervna(Deleted)Please disregard this, I posted it in the wrong area.Please disregard this, I posted it in the wrong area.https://lab.civicrm.org/dev/core/-/issues/23355.34 Upgrade fail when row format is not Dynamic2023-07-12T05:03:22Zeileen5.34 Upgrade fail when row format is not Dynamic [type] => DB_Error
[user_info] => ALTER TABLE `civicrm_action_schedule` ADD COLUMN `created_date` timestamp NULL DEFAULT CURRENT_TIMESTAMP COMMENT 'When was the schedule reminder created.' [nativecode=1118 ** Row size too large.... [type] => DB_Error
[user_info] => ALTER TABLE `civicrm_action_schedule` ADD COLUMN `created_date` timestamp NULL DEFAULT CURRENT_TIMESTAMP COMMENT 'When was the schedule reminder created.' [nativecode=1118 ** Row size too large. The maximum row size for the used table type, not counting BLOBs, is 8126. This includes storage overhead, check the manual. You have to change some columns to TEXT or BLOBs]
NOTE THIS WORKS
ALTER TABLE `civicrm_action_schedule` ADD COLUMN `created_date` timestamp NULL DEFAULT CURRENT_TIMESTAMP COMMENT 'When was the schedule reminder created.', ROW_FORMAT=DYNAMIC;https://lab.civicrm.org/dev/core/-/issues/125Invalid link to custom-fields documentation2018-12-15T22:50:50ZedgimarInvalid link to custom-fields documentationOn the `index.php?q=civicrm/admin/custom/group&reset=1` screen, the "learn more..." link appears to be broken -- this target of this link is currently https://docs.civicrm.org/user/en/latest/organising-your-data/custom-fields but should ...On the `index.php?q=civicrm/admin/custom/group&reset=1` screen, the "learn more..." link appears to be broken -- this target of this link is currently https://docs.civicrm.org/user/en/latest/organising-your-data/custom-fields but should instead be https://docs.civicrm.org/user/en/latest/organising-your-data/creating-custom-fields/.
Using CiviCRM 5.1.1.5.9https://lab.civicrm.org/dev/core/-/issues/3543Provide warning for mailings that are being edited to prevent Mailing not sav...2024-02-12T05:03:28ZlarsssandergreenProvide warning for mailings that are being edited to prevent Mailing not saved errors.In our org, we have multiple users who might need to edit the same mailing. Unfortunately, this creates the possibility that someone leaves the mailing open and someone else opens it and begins editing, leading to the "Mailing not saved....In our org, we have multiple users who might need to edit the same mailing. Unfortunately, this creates the possibility that someone leaves the mailing open and someone else opens it and begins editing, leading to the "Mailing not saved. Content may be out of date" error (and tears and hair loss due to lost changes). In our case, this is mostly one user putting together a mailing and another user editing it, fixing formatting errors, etc. It's very hard to train users who are used to Google Docs not to leave the tab open, leading to the need for rigid hand off procedures. We've been trying to get people trained for years, but this continues to be an issue that we waste a lot of time on.
Would it be feasible to add some kind of warning that would warn another user who wants to open another mailing that is already open for editing? I'm thinking just setting a flag on the mailing with a 30 minute expiration, which is updated every time the mailing or content is saved. The flag would be unset when the user exits the mailing. Mailings with the flag would have a warning on the draft mailings screen that would say "Someone else may be editing this mailing" or a warning before opening the mailing. This won't be a perfect solution, but I think it would prevent 95% of problems.
We're using Mosaico, but I think this would be implemented in core and apply to both kinds of mailings.https://lab.civicrm.org/dev/core/-/issues/126Soft Credit report redirect to Contribution Detail report doesn't filter cont...2018-05-21T12:01:59ZjrolfsSoft Credit report redirect to Contribution Detail report doesn't filter contact idIt looks like the names in the Soft Credit report no longer redirect to a Contribution Detail report that filters the results based on the contact's id. The same error occurs on dmaster.demo.civicrm.org.
![Snap_2018-05-17_at_16.53.45](/...It looks like the names in the Soft Credit report no longer redirect to a Contribution Detail report that filters the results based on the contact's id. The same error occurs on dmaster.demo.civicrm.org.
![Snap_2018-05-17_at_16.53.45](/uploads/229b8d3c6a85a76130928875c2616d94/Snap_2018-05-17_at_16.53.45.jpg)5.3.0https://lab.civicrm.org/dev/core/-/issues/3349Creating an event clears the cache2022-04-22T16:21:15ZjaapjansmaCreating an event clears the cacheWhen creating a new event or updating an existing the system will clear all caches in the background.
The line causing this is in the CRM_Event_BAO_Event::add
```php
public static function add(&$params) {
CRM_Utils_System::flush...When creating a new event or updating an existing the system will clear all caches in the background.
The line causing this is in the CRM_Event_BAO_Event::add
```php
public static function add(&$params) {
CRM_Utils_System::flushCache();
```
see: https://github.com/civicrm/civicrm-core/blame/master/CRM/Event/BAO/Event.php#L86
My question is why the caches need to be cleared when adding or updating an event? And if there is no reason is it safe to remove this line?
5.18.0https://lab.civicrm.org/dev/core/-/issues/3647Deleting C or Final mailing for A/B test makes report on A and B segments unv...2024-02-24T05:03:28ZlarsssandergreenDeleting C or Final mailing for A/B test makes report on A and B segments unviewableIf you create and A/B test, send the A & B segments and then delete the C or Final segment from Draft and Unscheduled Mailings, you can no longer view the mailing report for the A/B test. The mailing report URL just shows a blank page wi...If you create and A/B test, send the A & B segments and then delete the C or Final segment from Draft and Unscheduled Mailings, you can no longer view the mailing report for the A/B test. The mailing report URL just shows a blank page with header and footer.
I think it is probably not that uncommon an occurrence for someone to create an A/B test and later need to make a small change to the final version before sending it. That's easy enough to do by re-using one of the mailings and excluding the A and B mailings from your recipients. However, you will probably delete the C mailing from the unscheduled list at this point and then no longer be able to view the original A/B mailing report.
I'm testing on 5.28.4. This happens with both Mosaico and traditional mailings.https://lab.civicrm.org/dev/core/-/issues/127Smart Group incorrect counts2018-06-05T10:42:15Zaydunsaidan.saunders@squiffle.ukSmart Group incorrect countsI'm seeing some strange behaviour with smart groups where the number of group members is wrong. This is on 5.1.2 but has been seen previously.
1. On the Manage Groups page, the counts show initially as 'unknown'.
1. Clicking the 'Upd...I'm seeing some strange behaviour with smart groups where the number of group members is wrong. This is on 5.1.2 but has been seen previously.
1. On the Manage Groups page, the counts show initially as 'unknown'.
1. Clicking the 'Update Smart Group Counts' then updates one particular group to have a count of 113.
1. Clicking Contacts then takes to a list of contacts - 113 of them.
1. Clicking 'Edit Smart Group Search Criteria for ...' then shows 2 contacts - which is correct.
The smart group criteria are:
> Membership Type = Associate Member ...AND...
> Is a current member ...AND...
> Primary Members Only ...AND...
> Contacts with a Relationship Type of: 'Correspondent 1 of'
I am seeing similar differences between the reported numbers and the right results for other smart groups so it does not seem to be specific to those criteria. FWIW several of the other groups also report 113 members - wrongly.
If I create a mailing with the smart group it reports ~107 which would be about right if it really were 113 group members as a few are on hold. But it should just be 2!
If I go to basic Search and put the groupname in the group box, then it correctly returns 2 contacts.
* These results are as administrator, so it is not permission issues.
* PHP 7.1.16 (but seen when previously on 5.6)
* MySQL: Server version: 5.7.21-percona-sure2-log MySQL Community Server (GPL)
* `ONLY_FULL_GROUP_BY` is disabled
* Smart group cache timeout: 5
* Rebuild Smart Group Cache scheduled job: every 15 mins
https://lab.civicrm.org/dev/core/-/issues/3558Multilingual and Mailings: the interface only displays the groups selected in...2024-03-04T14:19:20ZbgmMultilingual and Mailings: the interface only displays the groups selected in a given languageThis is a weird bug:
* Enable multi-lingual, and enable a second language (ex: English & French)
* Create a new mailing while in English
* Add a group to the list of recipients.
* Save the draft
Now switch the interface language to the...This is a weird bug:
* Enable multi-lingual, and enable a second language (ex: English & French)
* Create a new mailing while in English
* Add a group to the list of recipients.
* Save the draft
Now switch the interface language to the other language (ex: French), and edit the mailing. Notice that the group has disappeared. It's not gone, but hidden.
In my case, the mailing was re-used from EN to FR, and since the groups were not visible, and there are different mailing groups for each language, the mailing ended up being sent to both language recipients.https://lab.civicrm.org/dev/core/-/issues/128Add deprecated warning helper function2018-05-28T02:48:08Zmattwiremjw@mjwconsult.co.ukAdd deprecated warning helper functionOverview
----------------------------------------
This adds a standard method to log a deprecated function warning.
PR: https://github.com/civicrm/civicrm-core/pull/12040
Before
----------------------------------------
Deprecated func...Overview
----------------------------------------
This adds a standard method to log a deprecated function warning.
PR: https://github.com/civicrm/civicrm-core/pull/12040
Before
----------------------------------------
Deprecated function warnings were inconsistent and you have to put too much detail in the log line.
After
----------------------------------------
One simple function that autogenerates the log message based on caller.
Comments from previous PR #12007
----------------------------------------
@totten:
> I think @totten tried to enhance deprecated output at some point
Probably this: https://chat.civicrm.org/civicrm/pl/cf1p7hgkkprp9kthkosq98qrer
@mattwire
> A standalone helper function is more pithy than the Civi::log...civi.tag=>deprecated.... Pithy is good.
From r-code perspective, I don't like having that helper in CRM_Utils_System. That class is too heavy already, and deprecatedFunctionWarning() is qualitatively different from every other function in the class.
Yes, I agree. I wasn't sure where to "dump" the function. Maybe `CRM/Core/Error/Log.php` would be a better place for it?
> My main hesitation with cf1p7hgkkprp9kthkosq98qrer was that findNonLogCaller() felt a little heavy-handed. It's nice how deprecatedFunctionWarning() doesn't have to dig as far back into the callstack.
> On the other hand, it's nice how cf1p7hgkkprp9kthkosq98qrer works with the existing coding convention.
We don't actually log deprecated very much so I don't think it's a big change to convention.5.3.0https://lab.civicrm.org/dev/core/-/issues/129Make Event Participant form use more generic custom data functions2022-08-11T05:03:23Zmattwiremjw@mjwconsult.co.ukMake Event Participant form use more generic custom data functionsUse more generic custom data functions for event forms. Waiting for generic custom data functions to be finalised.
Ref: https://github.com/civicrm/civicrm-core/pull/12119Use more generic custom data functions for event forms. Waiting for generic custom data functions to be finalised.
Ref: https://github.com/civicrm/civicrm-core/pull/12119https://lab.civicrm.org/dev/core/-/issues/130create_function is deprecated in php7.22018-05-19T12:57:28Zseamusleecreate_function is deprecated in php7.2the function create_function has been depreciated in php7.2 and throws a warning.the function create_function has been depreciated in php7.2 and throws a warning.seamusleeseamusleehttps://lab.civicrm.org/dev/core/-/issues/131Add in missing UK shire Monmouthshire2018-07-24T10:41:09ZseamusleeAdd in missing UK shire Monmouthshire5.3.0seamusleeseamusleehttps://lab.civicrm.org/dev/core/-/issues/132'Current Member' field in searches does not perform as expected2022-10-25T20:35:58Zeileen'Current Member' field in searches does not perform as expectedReplacing https://issues.civicrm.org/jira/browse/CRM-21403 with gitlab issue - much discussion over on the original proposed code change - https://github.com/civicrm/civicrm-core/pull/11968 and it's not clear what changes should be madeReplacing https://issues.civicrm.org/jira/browse/CRM-21403 with gitlab issue - much discussion over on the original proposed code change - https://github.com/civicrm/civicrm-core/pull/11968 and it's not clear what changes should be madehttps://lab.civicrm.org/dev/core/-/issues/133Reply-to field with empty string get saved in DB as NULL2018-06-18T20:41:13ZvarshithReply-to field with empty string get saved in DB as NULL**Issue:**
Somehow our client's mailing got saved with 'NULL' reply-to field and was causing problems.
On looking into it and trying to reproduce the issue, 'reply to' field can be NULL in the following case
- Add a new mailing (or reus...**Issue:**
Somehow our client's mailing got saved with 'NULL' reply-to field and was causing problems.
On looking into it and trying to reproduce the issue, 'reply to' field can be NULL in the following case
- Add a new mailing (or reuse one)
- Select a 'reply to' email (I had to enable "Enable Custom Reply-To" in 'civicrm/admin/mail')
- Fill in other fields as usual
- Under 'Responses' tab, check "Track Replies" checkbox. This brings up a warning message on the top right saying 'reply-to' has been cleared out (but still allows user to save mailing)
- Proceed to next step and save mailing
**Proposed Fix:**
I am not sure how to fix this as the user is expected to see the warning message and act accordingly. But I think we can check and not allow NULL values for reply-to in DB.
In CRM_Mailing_BAO_Mailing.php when adding a new mailing (add() method), there is a condition to check if 'reply-to' field is set and if it is not set, then 'from-email' is used.
here instead if using
if(\!isset($params['replyto_email']))...
we should use
if(empty($params['replyto_email']))...
so that empty string like '' will also be considered and reply-to could not be NULL5.4.0https://lab.civicrm.org/dev/core/-/issues/134Search Builder broken filter for Source Contact ID2018-06-08T04:12:28ZMonish DebSearch Builder broken filter for Source Contact IDSearch Builder lists the 'Source Contact ID' field but doesn't populate the information nor filter things appropriately. Use any filter in Search Builder with Activity >> Source Contact ID criteria, it will always show empty value below ...Search Builder lists the 'Source Contact ID' field but doesn't populate the information nor filter things appropriately. Use any filter in Search Builder with Activity >> Source Contact ID criteria, it will always show empty value below this column.
https://issues.civicrm.org/jira/secure/attachment/63475/Search%20Builder.pngMonish DebMonish Debhttps://lab.civicrm.org/dev/core/-/issues/135Non Numeric value encountered in CRM_Batch_Form_entryTest on PHP7.1 and 7.22018-11-04T20:33:27ZseamusleeNon Numeric value encountered in CRM_Batch_Form_entryTest on PHP7.1 and 7.2```
<error type="PHPUnit_Framework_Error_Notice">CRM_Batch_Form_EntryTest::testMembershipRenewalDates
A non well formed numeric value encountered
/home/seamus/buildkit/build/47.demo/sites/all/modules/civicrm/CRM/Price/BAO/LineItem.php:...```
<error type="PHPUnit_Framework_Error_Notice">CRM_Batch_Form_EntryTest::testMembershipRenewalDates
A non well formed numeric value encountered
/home/seamus/buildkit/build/47.demo/sites/all/modules/civicrm/CRM/Price/BAO/LineItem.php:374
/home/seamus/buildkit/build/47.demo/sites/all/modules/civicrm/CRM/Price/BAO/PriceSet.php:747
/home/seamus/buildkit/build/47.demo/sites/all/modules/civicrm/CRM/Batch/Form/Entry.php:761
/home/seamus/buildkit/build/47.demo/sites/all/modules/civicrm/CRM/Batch/Form/Entry.php:922
/home/seamus/buildkit/build/47.demo/sites/all/modules/civicrm/tests/phpunit/CRM/Batch/Form/EntryTest.php:256
/home/seamus/buildkit/build/47.demo/sites/all/modules/civicrm/tests/phpunit/CiviTest/CiviUnitTestCase.php:192
/home/seamus/buildkit/bin/phpunit5:598
</error>
```
The relevant code is here https://github.com/civicrm/civicrm-core/blob/master/CRM/Price/BAO/LineItem.php#L374 i'm guessing its most likely going to be $qty that is NULL maybe5.3.0https://lab.civicrm.org/dev/core/-/issues/136Search Preferences fails to save when "InnoDB Full Text Search" is set to yes2022-08-12T05:03:23ZjitendraSearch Preferences fails to save when "InnoDB Full Text Search" is set to yesReplicated on dmaster.
- Navigate to http://dmaster.demo.civicrm.org/civicrm/admin/setting/search?reset=1.
- Try to save the form after enabling `InnoDB Full Text Search`.Replicated on dmaster.
- Navigate to http://dmaster.demo.civicrm.org/civicrm/admin/setting/search?reset=1.
- Try to save the form after enabling `InnoDB Full Text Search`.jitendrajitendrahttps://lab.civicrm.org/dev/core/-/issues/137Possible regression on case search - placeholder2018-05-28T02:48:05ZeileenPossible regression on case search - placeholderMentioned here -
https://github.com/civicrm/civicrm-core/pull/11759#pullrequestreview-122555717
Opened this so we don;'t lose track of it
@monish.debMentioned here -
https://github.com/civicrm/civicrm-core/pull/11759#pullrequestreview-122555717
Opened this so we don;'t lose track of it
@monish.deb5.2.0https://lab.civicrm.org/dev/core/-/issues/138API Participant getcount limits at 252018-05-25T07:43:44ZpbatroffAPI Participant getcount limits at 25When making an API call on Participant with action 'getcount', the result is limited to 25 Participants, unless the option for result limit `'options' => array('limit' => 0),` is added.
This is easily reproduce able with a default inst...When making an API call on Participant with action 'getcount', the result is limited to 25 Participants, unless the option for result limit `'options' => array('limit' => 0),` is added.
This is easily reproduce able with a default installation and the example data. Select Participant, getcount and all Events. The API result is 25, but when the option is set it is 50. The API behaves correctly with Contacts - e.g. getcount Contacts (type Individuals) results in 161 contacts with the example data.
Tested with 4.7.18, and the newest 5.1.2.
I think the getcount action should behave consistently, and not be limited to 25.https://lab.civicrm.org/dev/core/-/issues/139Contribution Details Report throws DB error When trying to filter by soft_cre...2018-05-30T02:26:23ZyashodhaContribution Details Report throws DB error When trying to filter by soft_credit_type but not including soft creditsSteps to replicate:
Go to Contribution Details Report
In filters section, select Contribution OR Soft Credit? Is equal to Soft Credits Only and Soft Credit Type Is Solicited
When you run the report, you get following DB error:
`
Databas...Steps to replicate:
Go to Contribution Details Report
In filters section, select Contribution OR Soft Credit? Is equal to Soft Credits Only and Soft Credit Type Is Solicited
When you run the report, you get following DB error:
`
Database Error Code: Unknown column 'contribution_soft_civireport.soft_credit_type_id' in 'where clause', 1054
Additional Details:
Array
(
[callback] => Array
(
[0] => CRM_Core_Error
[1] => handle
)
[code] => -19
[message] => DB Error: no such field
[mode] => 16
[debug_info] => CREATE TEMPORARY TABLE civireport_contribution_detail_temp1 DEFAULT CHARACTER SET utf8 COLLATE utf8_unicode_ci AS SELECT SQL_CALC_FOUND_ROWS GROUP_CONCAT(DISTINCT contact_civireport.sort_name) as civicrm_contact_sort_name, GROUP_CONCAT(DISTINCT contact_civireport.id) as civicrm_contact_id, GROUP_CONCAT(DISTINCT email_civireport.email) as civicrm_email_email, GROUP_CONCAT(DISTINCT phone_civireport.phone) as civicrm_phone_phone, (contribution_civireport.id) as civicrm_contribution_contribution_id, GROUP_CONCAT(DISTINCT contribution_civireport.financial_type_id) as civicrm_contribution_financial_type_id, GROUP_CONCAT(DISTINCT contribution_civireport.currency) as civicrm_contribution_currency, GROUP_CONCAT(DISTINCT contribution_civireport.receive_date) as civicrm_contribution_receive_date, sum(contribution_civireport.total_amount) as civicrm_contribution_total_amount_sum, GROUP_CONCAT(DISTINCT 'Contribution') as civicrm_contribution_contribution_or_soft, GROUP_CONCAT(DISTINCT address_civireport.country_id) as civicrm_address_country_id FROM civicrm_contact contact_civireport
INNER JOIN civicrm_contribution contribution_civireport
ON contact_civireport.id = contribution_civireport.contact_id
AND contribution_civireport.is_test = 0
LEFT JOIN civicrm_phone phone_civireport
ON contact_civireport.id = phone_civireport.contact_id AND
phone_civireport.is_primary = 1
LEFT JOIN civicrm_address address_civireport
ON (contact_civireport.id =
address_civireport.contact_id) AND
address_civireport.is_primary = 1
LEFT JOIN civicrm_email email_civireport
ON (contact_civireport.id = email_civireport.contact_id AND
email_civireport.is_primary = 1) WHERE ( contact_civireport.is_deleted = 0 ) AND (1) AND ( contribution_civireport.contribution_status_id IN (1) ) AND ( ( contribution_soft_civireport.soft_credit_type_id IN ( '3') ) ) GROUP BY contribution_civireport.id ORDER BY civicrm_contact_sort_name ASC LIMIT 0, 50 [nativecode=1054 ** Unknown column 'contribution_soft_civireport.soft_credit_type_id' in 'where clause']
[type] => DB_Error
[user_info] => CREATE TEMPORARY TABLE civireport_contribution_detail_temp1 DEFAULT CHARACTER SET utf8 COLLATE utf8_unicode_ci AS SELECT SQL_CALC_FOUND_ROWS GROUP_CONCAT(DISTINCT contact_civireport.sort_name) as civicrm_contact_sort_name, GROUP_CONCAT(DISTINCT contact_civireport.id) as civicrm_contact_id, GROUP_CONCAT(DISTINCT email_civireport.email) as civicrm_email_email, GROUP_CONCAT(DISTINCT phone_civireport.phone) as civicrm_phone_phone, (contribution_civireport.id) as civicrm_contribution_contribution_id, GROUP_CONCAT(DISTINCT contribution_civireport.financial_type_id) as civicrm_contribution_financial_type_id, GROUP_CONCAT(DISTINCT contribution_civireport.currency) as civicrm_contribution_currency, GROUP_CONCAT(DISTINCT contribution_civireport.receive_date) as civicrm_contribution_receive_date, sum(contribution_civireport.total_amount) as civicrm_contribution_total_amount_sum, GROUP_CONCAT(DISTINCT 'Contribution') as civicrm_contribution_contribution_or_soft, GROUP_CONCAT(DISTINCT address_civireport.country_id) as civicrm_address_country_id FROM civicrm_contact contact_civireport
INNER JOIN civicrm_contribution contribution_civireport
ON contact_civireport.id = contribution_civireport.contact_id
AND contribution_civireport.is_test = 0
LEFT JOIN civicrm_phone phone_civireport
ON contact_civireport.id = phone_civireport.contact_id AND
phone_civireport.is_primary = 1
LEFT JOIN civicrm_address address_civireport
ON (contact_civireport.id =
address_civireport.contact_id) AND
address_civireport.is_primary = 1
LEFT JOIN civicrm_email email_civireport
ON (contact_civireport.id = email_civireport.contact_id AND
email_civireport.is_primary = 1) WHERE ( contact_civireport.is_deleted = 0 ) AND (1) AND ( contribution_civireport.contribution_status_id IN (1) ) AND ( ( contribution_soft_civireport.soft_credit_type_id IN ( '3') ) ) GROUP BY contribution_civireport.id ORDER BY civicrm_contact_sort_name ASC LIMIT 0, 50 [nativecode=1054 ** Unknown column 'contribution_soft_civireport.soft_credit_type_id' in 'where clause']
[to_string] => [db_error: message="DB Error: no such field" code=-19 mode=callback callback=CRM_Core_Error::handle prefix="" info="CREATE TEMPORARY TABLE civireport_contribution_detail_temp1 DEFAULT CHARACTER SET utf8 COLLATE utf8_unicode_ci AS SELECT SQL_CALC_FOUND_ROWS GROUP_CONCAT(DISTINCT contact_civireport.sort_name) as civicrm_contact_sort_name, GROUP_CONCAT(DISTINCT contact_civireport.id) as civicrm_contact_id, GROUP_CONCAT(DISTINCT email_civireport.email) as civicrm_email_email, GROUP_CONCAT(DISTINCT phone_civireport.phone) as civicrm_phone_phone, (contribution_civireport.id) as civicrm_contribution_contribution_id, GROUP_CONCAT(DISTINCT contribution_civireport.financial_type_id) as civicrm_contribution_financial_type_id, GROUP_CONCAT(DISTINCT contribution_civireport.currency) as civicrm_contribution_currency, GROUP_CONCAT(DISTINCT contribution_civireport.receive_date) as civicrm_contribution_receive_date, sum(contribution_civireport.total_amount) as civicrm_contribution_total_amount_sum, GROUP_CONCAT(DISTINCT 'Contribution') as civicrm_contribution_contribution_or_soft, GROUP_CONCAT(DISTINCT address_civireport.country_id) as civicrm_address_country_id FROM civicrm_contact contact_civireport
INNER JOIN civicrm_contribution contribution_civireport
ON contact_civireport.id = contribution_civireport.contact_id
AND contribution_civireport.is_test = 0
LEFT JOIN civicrm_phone phone_civireport
ON contact_civireport.id = phone_civireport.contact_id AND
phone_civireport.is_primary = 1
LEFT JOIN civicrm_address address_civireport
ON (contact_civireport.id =
address_civireport.contact_id) AND
address_civireport.is_primary = 1
LEFT JOIN civicrm_email email_civireport
ON (contact_civireport.id = email_civireport.contact_id AND
email_civireport.is_primary = 1) WHERE ( contact_civireport.is_deleted = 0 ) AND (1) AND ( contribution_civireport.contribution_status_id IN (1) ) AND ( ( contribution_soft_civireport.soft_credit_type_id IN ( '3') ) ) GROUP BY contribution_civireport.id ORDER BY civicrm_contact_sort_name ASC LIMIT 0, 50 [nativecode=1054 ** Unknown column 'contribution_soft_civireport.soft_credit_type_id' in 'where clause']"]
)
`
I was able to replicate this on dmaster as well5.3.0yashodhayashodhahttps://lab.civicrm.org/dev/core/-/issues/140Add missing pseudoconstant for option_group_id in CustomField2018-11-04T20:40:17ZMichael McAndrewAdd missing pseudoconstant for option_group_id in CustomFieldPseudoconstant tag was missing from the XML definition, meaning that one could not create custom fields and specify their select options without knowing the numerical ID.Pseudoconstant tag was missing from the XML definition, meaning that one could not create custom fields and specify their select options without knowing the numerical ID.5.4.0https://lab.civicrm.org/dev/core/-/issues/141Custom groups with different names but same title would not be saved2018-07-14T01:25:44ZjaapjansmaCustom groups with different names but same title would not be savedI have created two custom groups from within my extension:
* Name: group_areas, Title: Areas
* Name: contact_areas, Title: Areas
When I now edit the custom group it will fail because both custom groups have the same title but a differe...I have created two custom groups from within my extension:
* Name: group_areas, Title: Areas
* Name: contact_areas, Title: Areas
When I now edit the custom group it will fail because both custom groups have the same title but a different name.
Is this correct behaviour?
See this stack exchange question: https://civicrm.stackexchange.com/questions/25102/https-civicrm-org-extensions-areas-causing-error/25104#25104
Tested on CiviCRM version 5.0.0 and CiviCRM version 5.1.aplha15.5.0https://lab.civicrm.org/dev/core/-/issues/142States and Counties don't chain in Search Builder2022-07-01T20:25:48ZMonish DebStates and Counties don't chain in Search BuilderMany states and counties have common names. Maryland could be in Liberia or the United States. There are 3 states named "Distrito Federal". Within the US, there are 18 counties named Montgomery.
The Search Builder state and county field...Many states and counties have common names. Maryland could be in Liberia or the United States. There are 3 states named "Distrito Federal". Within the US, there are 18 counties named Montgomery.
The Search Builder state and county fields don't filter with reference to the country or state fields that are within the same AND grouping. The state field displays the states from the countries that have states enabled, and the county field displays all counties in alphabetical order.
This presents two problems:
1. If you have multiple options with the same name, you have no idea which is which. Is "Distrito Federal" in Brazil, Mexico, or Venezuela? Which of the 31 "Washington" counties is the right one? There's no way to input the ID anymore if you happen to know.
2. If you are looking for a state that's not in one of the countries you have set for "available states and provinces", you're out of luck.
Two proposed alternatives:
1. Make the states drop-down depend upon a country of the same location type, within the same "AND" grouping. Do the same for the counties with regard to the states. This would require some kind of parameter to be sent to the address/getoptions API to filter the list, but it would make them work in the same way as elsewhere in CiviCRM.
2. Have a new context for address/getoptions that prefixes the state name with the country and prefixes the county name with the state. (We can probably safely assume that the same county/state combinations appear in multiple countries--no Montgomery County, Maryland, Liberia.) This means that a user doesn't need to add redundant criteria to the search.
For the second option, that doesn't solve the problem of picking states in other countries, but maybe a checkbox could alternate between showing all countries' states (maybe setting the context as "get" instead of "search").
JIRA ticket : https://issues.civicrm.org/jira/browse/CRM-17572Monish DebMonish Debhttps://lab.civicrm.org/dev/core/-/issues/143Contact 'World Region' Field not functioning properly in Search Builder2018-05-28T02:48:08ZMonish DebContact 'World Region' Field not functioning properly in Search BuilderTo reproduce:
'Search' -> 'Search Builder'.
Search for 'Contacts' -> 'World Region' -> '='.
Upon doing the above, the user is presented with a text box instead of the expected dropdown <select> menu.
If the user proceeds to use the une...To reproduce:
'Search' -> 'Search Builder'.
Search for 'Contacts' -> 'World Region' -> '='.
Upon doing the above, the user is presented with a text box instead of the expected dropdown <select> menu.
If the user proceeds to use the unexpected textbox and manually type in the World Region they are searching for, they get an error of the following kind:
Europe and Central Asia is not of the type Positive
This has been reproduced successfully on the test site at http://wpmaster.demo.civicrm.org
JIRA - https://issues.civicrm.org/jira/browse/CRM-203955.3.0Monish DebMonish Debhttps://lab.civicrm.org/dev/core/-/issues/144getCustomFieldID switch to API, add caching, add full string return option2018-06-17T23:18:27Zmattwiremjw@mjwconsult.co.ukgetCustomFieldID switch to API, add caching, add full string return optionIn most of my extensions I implement something like this: https://github.com/mattwire/uk.co.mjwconsult.recurmaster/blob/master/CRM/Recurmaster/Utils.php#L14
Because I want the lookup to be cached so I can use it multiple times without a...In most of my extensions I implement something like this: https://github.com/mattwire/uk.co.mjwconsult.recurmaster/blob/master/CRM/Recurmaster/Utils.php#L14
Because I want the lookup to be cached so I can use it multiple times without a performance issue (eg. within hooks). It is also useful to be able to return the full string equivalent (ie. custom_123 instead of 123) as it can then be used directly in parameter arrays like `$key = getCustomFieldID()` without doing: `$key = 'custom_' . getCustomFieldID()`
This is a proposal to update the core function getCustomFieldID so that it is more useful. It should also help towards https://lab.civicrm.org/dev/core/issues/1095.4.0https://lab.civicrm.org/dev/core/-/issues/145CRM-20637 extend expired price fields in backend2022-08-13T05:03:47ZeileenCRM-20637 extend expired price fields in backendPlace to agree correct approach per discussions on
https://github.com/civicrm/civicrm-core/pull/10423 and https://github.com/civicrm/civicrm-core/pull/10804Place to agree correct approach per discussions on
https://github.com/civicrm/civicrm-core/pull/10423 and https://github.com/civicrm/civicrm-core/pull/10804https://lab.civicrm.org/dev/core/-/issues/146Searchbuilder crash in 5.1.02018-05-29T07:55:33ZAndy ClarkSearchbuilder crash in 5.1.0I built a search using Searchbuilder with 2 search lines. The first selected a group, the second selected a contact ID that was not equal to a certain value.
I tried 2 different 5.1.0 systems that I maintain, and also the demo system a...I built a search using Searchbuilder with 2 search lines. The first selected a group, the second selected a contact ID that was not equal to a certain value.
I tried 2 different 5.1.0 systems that I maintain, and also the demo system at https://drupal.demo.civicrmfr.org - all crashed the same.
Traceback is attached here... [searchbulder.txt](/uploads/5c43ccec61b75f0a46ce71e63baf909e/searchbulder.txt)https://lab.civicrm.org/dev/core/-/issues/2068Address greenwich for d8 on upgrade2020-10-22T21:41:28ZeileenAddress greenwich for d8 on upgradeOutstanding issues getting greenwich theme extension to be installed on upgrade for d8 sitesOutstanding issues getting greenwich theme extension to be installed on upgrade for d8 siteshttps://lab.civicrm.org/dev/core/-/issues/2012Document impact of css / html buttons merged to 5.31 that may require action ...2021-03-01T13:07:40ZeileenDocument impact of css / html buttons merged to 5.31 that may require action by integrators(from @andrewhunt in chat) PR #18410 just merged to master that changes how nearly all buttons in CiviCRM are formed. The good news is that buttons will look better, be easier to theme, and be handled consistently with things we make loo...(from @andrewhunt in chat) PR #18410 just merged to master that changes how nearly all buttons in CiviCRM are formed. The good news is that buttons will look better, be easier to theme, and be handled consistently with things we make look like buttons. The bad news is that you may need to update your extensions to accommodate the change. You can read more details in the PR and the two earlier ones it referred to (changes were originally to come in 5.29, but it got deferred to early in the 5.31 cycle so there's time for everyone to accommodate them).
The most important change is that
```<input type="submit">``` and ```<input type="button"> ```
are replaced with
```<button type="submit">``` and ```<button type="button">``` across the board.
There are three main ways you may need to accommodate this:
Check your CSS-style selectors. If you have something like ```$("input#mybutton").click()```, it won't work anymore. Same with ```.crm-button input``` --the <button> elements themselves are the wrapper now, so all those ```<span class="crm-button">``` elements are toast. To handle the old and new ways, you might look for specific IDs, classes, names, and/or types rather than just switching input to button.
Check how you add buttons. The helper ```CRM_Core_Form::addButtons()``` has been updated to work the new way, so if that's all you do, you're safe. However, if you do something like
PHP
```$form->addElement('submit', 'submit_button', ts('Submit'));```
You'll want to replace it with something like
PHP
```
$form->addElement(
'xbutton',
'submit_button',
ts('Submit'),
[
'type' => 'submit',
'class' => 'crm-button',
]
);
```
In your template, you won't need to add a wrapper, but if you don't give the <button> element the crm-button class, it won't look like the others. Please note that xbutton is not a typo. Quickform, in it's wisdom, uses button for creating <input type="button"> elements. You must use xbutton to create an actual <button>.
Check how you identify the button that was clicked. The text on an <input> button is the value attribute, and if that button was clicked, the button's name comes through on the submitted params. However, a <button> element wraps around the button text and may have no value. If there's no value, you won't see a value for it in the params. Instead of looking for this, you might consider using $form->controller->getButtonName() to identify the button that was clicked. That said, core now gives most buttons a value of 1 because there were too many instances where the code looked for the value in the params.
E.g. of code to change
![Screen_Shot_2020-09-11_at_10.48.04_AM](/uploads/d0ac6d6e54d53e2ba1afc04c5419da0e/Screen_Shot_2020-09-11_at_10.48.04_AM.png)
e.g of backward compatible code change
![Screen_Shot_2020-09-11_at_10.58.14_AM](/uploads/6d8ed3f6461d3a5b872d2eedca377851/Screen_Shot_2020-09-11_at_10.58.14_AM.png)
https://gitlab.com/physician-health-program-of-bc/saveandnewcopy/-/commit/441a165ea1debf15c335f291d3251aecc68dc5f2https://lab.civicrm.org/dev/core/-/issues/147One of parameters is not of the type MysqlColumnNameOrAlias when using Non-AS...2019-04-29T20:20:11ZMonish DebOne of parameters is not of the type MysqlColumnNameOrAlias when using Non-ASCII display names.To reproduce, change Display Name of a location type (http://dmaster.demo.civicrm.org/civicrm/admin/locationType) to some Non-ASCII text (or just install civicrm on some Non-ASCII language, in my case Russian). When go to Search Builder ...To reproduce, change Display Name of a location type (http://dmaster.demo.civicrm.org/civicrm/admin/locationType) to some Non-ASCII text (or just install civicrm on some Non-ASCII language, in my case Russian). When go to Search Builder (http://dmaster.demo.civicrm.org/civicrm/contact/search/builder) and try to search contacts with phone, or email, or address of that location type.
Get an error message:
Sorry but we are not able to provide this at the moment.
One of parameters (value: Дом-email) is not of the type MysqlColumnNameOrAlias
Ref : https://issues.civicrm.org/jira/browse/CRM-19089Monish DebMonish Debhttps://lab.civicrm.org/dev/core/-/issues/148Activity Separation field changes the default behaviour2022-08-13T05:03:48ZyashodhaActivity Separation field changes the default behaviourSome of the users without realizing might select the first option that is *Create separate activities for each contact* which might confuse them since that's not how it behaved earlier.Some of the users without realizing might select the first option that is *Create separate activities for each contact* which might confuse them since that's not how it behaved earlier.yashodhayashodhahttps://lab.civicrm.org/dev/core/-/issues/149Fatal Error on customvalue get api2018-06-01T12:12:09ZjitendraFatal Error on customvalue get apiReplicated on dmaster http://dmaster.demo.civicrm.org/civicrm/api#explorer
Entity = `CustomValue` Action = `get`
Fields to return => `custom_1`
Parameter => `entity_id = <any contact id>`
$result = civicrm_api3('CustomValue', 'get'...Replicated on dmaster http://dmaster.demo.civicrm.org/civicrm/api#explorer
Entity = `CustomValue` Action = `get`
Fields to return => `custom_1`
Parameter => `entity_id = <any contact id>`
$result = civicrm_api3('CustomValue', 'get', [
'sequential' => 1,
'return' => "custom_1",
'entity_id' => 202,
]];
Result -
"is_error": 1,
"error_message": "A fatal error was triggered: is not of type String"jitendrajitendrahttps://lab.civicrm.org/dev/core/-/issues/150Chain select for country/state in Search Builder does not stay within OR grou...2018-07-17T06:54:34ZAndie HuntChain select for country/state in Search Builder does not stay within OR groupingsdev/core#142 introduces a chained select for picking states from countries and picking counties from states. The changes there correctly limit the chaining to be within the same location type. However, picking a country will modify *al...dev/core#142 introduces a chained select for picking states from countries and picking counties from states. The changes there correctly limit the chaining to be within the same location type. However, picking a country will modify *all* state/province fields of that location type, even if they're in different "Also include contacts where" sections.
To recreate, let's say you are planning an event in Detroit:
1. Search for contacts in Michigan:
```
Contacts | Country | Home | = | United States
Contacts | State | Home | = | Michigan
```
2. Search for contacts in Ontario by clicking "Also include contacts where" and setting up:
```
Contacts | Country | Home | = | Canada
Contacts | State | Home | = | Ontario
```
Notice that the state dropdown where Michigan had been selected now has nothing selected and offers only Canadian provinces.
The same thing happens if you have a state field on its own (without country) and you pick a country in another `OR` grouping.5.5.0Monish DebMonish Debhttps://lab.civicrm.org/dev/core/-/issues/151Action to Update Recurring Contributions From Membership View is Never Shown2018-06-20T20:41:20ZCamilo RodríguezAction to Update Recurring Contributions From Membership View is Never Shown## Overview
The table added to view recurring contributions form the membership details view (https://lab.civicrm.org/dev/core/issues/38) was never showing the 'Edit' action. The table shown when looking at recurring contributions from w...## Overview
The table added to view recurring contributions form the membership details view (https://lab.civicrm.org/dev/core/issues/38) was never showing the 'Edit' action. The table shown when looking at recurring contributions from within a membership should behave exactly as the table shown on a contact's **Contributions** tab.
## How it Works Currently
The table added to view recurring contributions form the membership details view was never showing the 'Edit' action, even though the user had all necessary permissions.
## How it Should Work
If the 'Edit' action is shown on the list of recurring contributions of a contact on **Contributions** tab, it should also be available on the list shown within a membership.
## Comments
Related to change introduced in #385.4.0https://lab.civicrm.org/dev/core/-/issues/152Remove AdvMulti-Select custom field type2018-07-18T20:03:11ZcolemanwRemove AdvMulti-Select custom field type![Screenshot_from_2018-05-29_15_19_57](/uploads/ec6cb2f79769ef926d2d2deff6b8e1c8/Screenshot_from_2018-05-29_15_19_57.png)
Since 2014 this widget has been deprecated and this notice has displayed whenever someone tries to select it.
I th...![Screenshot_from_2018-05-29_15_19_57](/uploads/ec6cb2f79769ef926d2d2deff6b8e1c8/Screenshot_from_2018-05-29_15_19_57.png)
Since 2014 this widget has been deprecated and this notice has displayed whenever someone tries to select it.
I think it's time to remove it alltogether.https://lab.civicrm.org/dev/core/-/issues/153Pending Pay Later /w Custom payment method2020-11-19T07:07:10ZADG CreativePending Pay Later /w Custom payment methodCiviCRM 5.1.2
Joomla 3.8.7
(Replicated on civicrm.demo.civihosting.com - CiviCRM 5.0.0 - Drupal)
Create a custom Payment Method (ie. Undetermined), with Financial Account = Accounts Receivable
On the administrative side, register contac...CiviCRM 5.1.2
Joomla 3.8.7
(Replicated on civicrm.demo.civihosting.com - CiviCRM 5.0.0 - Drupal)
Create a custom Payment Method (ie. Undetermined), with Financial Account = Accounts Receivable
On the administrative side, register contact for an event with Payment Method=Undetermined, Payment Status=Pending
Go to the Contribution tab, the contribution is Pending Pay Later
Do Record Payment, Payment Method=Check, Add Check #, Payment Status=Complete, (Full Amount Paid)
The Contribution will be marked with a Payment Status of Partially Paid
Admin can edit the contribution and mark it as Payment Status = Complete, but...
Expand the payment information, Record Payment links are still visible, but will throw an error if clicked
civicrm_contribution.payment_instrument_id = Undetermined (Although payment method was set to Check)
civicrm_financial_trxn.from_financial_account_id = Accounts Receivable
civicrm_financial_trxn.to_financial_account_id = Accounts Receivable
Contribution not listed in Bookkeeping Report
Also cannot find contribution by Check# in Contributions->Find Contributions (Not sure if related or separate issue)
![CompletedMarkedasPartiallyPaid](/uploads/26c971d0956ac4c6abd40cb6933fd917/CompletedMarkedasPartiallyPaid.jpg)5.9https://lab.civicrm.org/dev/core/-/issues/154Can't edit related records when current employer has a pending membership2018-06-06T08:42:12ZUpperholmeCan't edit related records when current employer has a pending membershipI originally posted this in https://lab.civicrm.org/dev/core/issues/14 and Eileen suggested it might be a separate issue.
Using Civi 5.1 and D7. An org record exists and has a pending membership (created through a contribution page and ...I originally posted this in https://lab.civicrm.org/dev/core/issues/14 and Eileen suggested it might be a separate issue.
Using Civi 5.1 and D7. An org record exists and has a pending membership (created through a contribution page and opted to pay later). the membership is configured such that individuals linked via 'employed by' inherit the membership.
There are several related individual records, with the org as current employer. In anticipation of the payment being made and the membership going live, we want to edit the individual records to add further detail (just stuff like Job title, etc.) When trying to save any edits, we get the error message:
"The membership cannot be saved because the status cannot be calculated for start_date: null end_date null join_date null as at 2018-05-25 10:40:37"
The only workaround seems to be to edit the membership record and give it a start date and then override the status of the membership so that it remains pending.5.3.0https://lab.civicrm.org/dev/core/-/issues/155Improvements and bugfixes to Option Groups UI2018-12-15T23:10:58Zmattwiremjw@mjwconsult.co.ukImprovements and bugfixes to Option Groups UI**Bugfix:**
Allow saving of new option value with value = 0.
PR (**MERGED**): https://github.com/civicrm/civicrm-core/pull/12229
**Additional fields on OptionGroup list**
This adds Enabled/Reserved columns to list and puts buttons at t...**Bugfix:**
Allow saving of new option value with value = 0.
PR (**MERGED**): https://github.com/civicrm/civicrm-core/pull/12229
**Additional fields on OptionGroup list**
This adds Enabled/Reserved columns to list and puts buttons at the top as well.
PR: https://github.com/civicrm/civicrm-core/pull/12233
![localhost_8000_civicrm_admin_options_reset_1](/uploads/e742132345fd2c7119f1fe3996a6a1d9/localhost_8000_civicrm_admin_options_reset_1.png)
**Resolve issues with use of "is_reserved" database field**
As the is_reserved flag is not being shown in the UI, and is not editable from the UI it has not been used for anything in core even though it is there.
A common use-case is adding an option group and then adding a set of custom fields that use that option group (eg. for surveys) but that is currently not possible via the UI because the query that looks for available option groups is looking ONLY for option groups that are already linked to custom fields - a catch 22!
PR: https://github.com/civicrm/civicrm-core/pull/12235
![customfieldoptiongroup](/uploads/85ef8c8ecf9826f94acedda058a7aa14/customfieldoptiongroup.png)5.5.0https://lab.civicrm.org/dev/core/-/issues/156Only set defaults when creating a custom field (not when editing one)2020-11-05T22:07:08ZMichael McAndrewOnly set defaults when creating a custom field (not when editing one)Before
----------------------------------------
When editing a custom field via the API, any values that were not supplied would revert to the default
After
----------------------------------------
When editing current values are loa...Before
----------------------------------------
When editing a custom field via the API, any values that were not supplied would revert to the default
After
----------------------------------------
When editing current values are loaded via the ->find() method.
See test for more details5.3.0https://lab.civicrm.org/dev/core/-/issues/157Do not delete indexes if they are needed for foreign keys2019-02-24T21:11:32ZMichael McAndrewDo not delete indexes if they are needed for foreign keysBefore
------
When changing a custom field from searchable to not searchable, CiviCRM tries to remove the index. This causes a fatal error when the column is also a foreign key to another table because the index is needed for the foreig...Before
------
When changing a custom field from searchable to not searchable, CiviCRM tries to remove the index. This causes a fatal error when the column is also a foreign key to another table because the index is needed for the foreign key.
After
-----
We ensure that the column is not an FK before removing the index
Technical details
-----------------
I added a test to CustomFieldTest which exercises this functionality5.3.0https://lab.civicrm.org/dev/core/-/issues/158intra-rc regression group contacts list and export limits results to 500 records2018-06-15T09:05:26Zlcdwebintra-rc regression group contacts list and export limits results to 500 recordsTo reproduce:
* create a group and add > 500 contacts to it
* go to the manage groups page and click contacts for that group to get the listing
* select all records and choose the export contacts action
* only 500 records are select...To reproduce:
* create a group and add > 500 contacts to it
* go to the manage groups page and click contacts for that group to get the listing
* select all records and choose the export contacts action
* only 500 records are selected for export
This appears to be a regression caused by this PR: https://github.com/civicrm/civicrm-core/pull/12110
Specifically the changes to Select.php -- haven't yet figured out where the breakdown is. What's interesting is that if you search for the group contacts via advanced search, the export works as expected. It's only when triggering after the manage groups > contacts link that the export list is truncated.5.2.0https://lab.civicrm.org/dev/core/-/issues/159Include Premium in List of Contributions2022-08-14T05:03:43ZCamilo RodríguezInclude Premium in List of Contributions## Overview
List of contributions shown on recurring contribution detail view doesn't show the column for **Premium**, which is visible on Contact's contributions tab.
## How it Works Currently
1. Go to a contact
2. Click contribution...## Overview
List of contributions shown on recurring contribution detail view doesn't show the column for **Premium**, which is visible on Contact's contributions tab.
## How it Works Currently
1. Go to a contact
2. Click contributions tab
3. Column **Premium** is shown on contributions list
4. Scroll down to list of recurring contributions
5. Click on **View** action
6. List of contributions does not show **Premium** column
## How it Should Work
1. Go to a contact
2. Click contributions tab
3. Column **Premium** is shown on contributions list
4. Scroll down to list of recurring contributions
5. Click on **View** action
6. List of contributions shows **Premium** column
## Comments
Related to change implemented on #40https://lab.civicrm.org/dev/core/-/issues/160Import Progress bar polls server too often2018-06-05T04:12:17ZseamusleeImport Progress bar polls server too oftenThe import progress bar polls the server every 5 milliseconds for progress rather 5 seconds which is what i think the expectation was meant to beThe import progress bar polls the server every 5 milliseconds for progress rather 5 seconds which is what i think the expectation was meant to be5.3.0seamusleeseamusleehttps://lab.civicrm.org/dev/core/-/issues/161Adding preSave_table_name hook2023-05-20T05:03:22Zomar_compucorpAdding preSave_table_name hookCurrently there is only **postSave_table_name** hook that run after a write operation on a core table that has an associated DAO. But in some cases it is useful to have a hook that run before the write operation.
An example whwre such ...Currently there is only **postSave_table_name** hook that run after a write operation on a core table that has an associated DAO. But in some cases it is useful to have a hook that run before the write operation.
An example whwre such hook is useful is when you cancel a membership contribution then the membership status will also change to **Cancelled**, but the cancellation of the membership does not trigger **hook_civicrm_pre** nor **hook_civicrm_post** hooks because this is how civicrm change the membership status when you cancel the membership :
https://github.com/civicrm/civicrm-core/blob/cbaa9a31f02afda8d2e19610da823fd0c4d28a16/CRM/Contribute/BAO/Contribution.php#L1049-L1052
hence that **hook_civicrm_pre** and **hook_civicrm_post** are invoked in the BAO class create method so doing anything using DAO directly as in the example above won't invoke any of these hooks and if you are planning to do any operation before editing the membership table in the case above then you are out of luck.
A new hook similar to **postSave_table_name** hook with the name **preSave_table_name** should be added to solve the issue mentioned issue.https://lab.civicrm.org/dev/core/-/issues/162Use checksum to access user dashboard2019-02-14T17:34:43ZjitendraUse checksum to access user dashboardSE - https://civicrm.stackexchange.com/questions/8619/using-checksums-to-access-the-civicrm-user-dashboardSE - https://civicrm.stackexchange.com/questions/8619/using-checksums-to-access-the-civicrm-user-dashboard5.3.0jitendrajitendrahttps://lab.civicrm.org/dev/core/-/issues/163Disabled Groups used for mailings block access to mailing reports2018-10-14T00:59:59ZseamusleeDisabled Groups used for mailings block access to mailing reportsWhen a group that was used to send a mailing is disabled, that blocks access to the past mailing as the mailingACLIDs only checks for enabled groupsWhen a group that was used to send a mailing is disabled, that blocks access to the past mailing as the mailingACLIDs only checks for enabled groups5.3.0seamusleeseamusleehttps://lab.civicrm.org/dev/core/-/issues/164scheduled reminders: form validations when repetition enabled2018-06-06T14:41:34Zlcdwebscheduled reminders: form validations when repetition enabledin the scheduled reminders interface, if you enable repetition, the every X until Y fields are required -- the job will fail with SQL errors if the fields are left empty. but currently, the fields are not required by the UI. we should ad...in the scheduled reminders interface, if you enable repetition, the every X until Y fields are required -- the job will fail with SQL errors if the fields are left empty. but currently, the fields are not required by the UI. we should add form validations to require values in those fields.lcdweblcdwebhttps://lab.civicrm.org/dev/core/-/issues/165Bad link in civicrm/case/activity/view breadcrumb2018-06-08T06:11:50ZmikantchapBad link in civicrm/case/activity/view breadcrumbDisable popups in display preferences.
Find a case eg civicrm/case/search?reset=1.
'Manage' the case to open it.
Click on the activity Subject, goes to eg civicrm/case/activity/view?cid=203&aid=642
The 'Case Activity' breadcrumb link...Disable popups in display preferences.
Find a case eg civicrm/case/search?reset=1.
'Manage' the case to open it.
Click on the activity Subject, goes to eg civicrm/case/activity/view?cid=203&aid=642
The 'Case Activity' breadcrumb links to civicrm/case/activity?reset=1 which gives an error when you try to use it:
Sorry, due to an error, we are unable to fulfill your request at the moment. You may want to contact your administrator or service provider with more details about what action you were performing when this occurred. required params missing.
Can reproduce this on http://dmaster.demo.civicrm.orghttps://lab.civicrm.org/dev/core/-/issues/166updateAllMemberships in memberschip BAO crasches out of memory on large numbe...2022-08-14T05:03:44ZJoostupdateAllMemberships in memberschip BAO crasches out of memory on large numbers of membersWe have a database with about 1.3 million memberships. the scheduled task to update membership statuses crashes with an out of memory error.We have a database with about 1.3 million memberships. the scheduled task to update membership statuses crashes with an out of memory error.https://lab.civicrm.org/dev/core/-/issues/167Google geocoding - what response should there be to their new terms of service?2022-08-15T14:20:01ZeileenGoogle geocoding - what response should there be to their new terms of service?Per this
https://github.com/eileenmcnaughton/org.wikimedia.geocoder/issues/2
"Google's new Terms of Service coming into effect on June 11th prohibit "stor[ing] Google Maps Content for more than 30 days". "Google Maps Content" is explic...Per this
https://github.com/eileenmcnaughton/org.wikimedia.geocoder/issues/2
"Google's new Terms of Service coming into effect on June 11th prohibit "stor[ing] Google Maps Content for more than 30 days". "Google Maps Content" is explicitly defined to include geocoding results."
One of my hopes with the geocoder extension was to take a step towards getting geocoders out of core & put the responsibility on the sites using the software to understand their own terms of service (this is part of the message on the install screen "You are responsible for understanding the geocoder terms of service and ensuring you are working within them."). I was also intending to gently move people to open data solutions rather than proprietary ones.
In theory sites using geocoding in CiviCRM are complying with their terms of service using the tools CiviCRM provides.
However, perhaps the new terms of service force our hand to stop providing geocoding with Google as a service? If that is the case we would need to remove google geocoding from any releases published on the CiviCRM site (including 4.6 since that is published on the main downloads) and also consider whether to remove it from the extension.https://lab.civicrm.org/dev/core/-/issues/168If scheduled jobs are run individually nothing is logged to civicrm job log2022-08-15T05:03:21ZlolcodeIf scheduled jobs are run individually nothing is logged to civicrm job logWhen running scheduled jobs individually to have greater control over the timing or parameters of those jobs then nothing is logged to the JobLog at least for certain jobs like process_mailing. This leaves the user to implement something...When running scheduled jobs individually to have greater control over the timing or parameters of those jobs then nothing is logged to the JobLog at least for certain jobs like process_mailing. This leaves the user to implement something else for logging.
I don't agree with this approach because it will push organisations to use custom solutions but it should at least be a documented limitation.
https://docs.civicrm.org/user/en/latest/initial-set-up/scheduled-jobs/#specific-jobshttps://lab.civicrm.org/dev/core/-/issues/169scheduled reminder: activity reminder with choose recipients option limits to...2022-08-27T05:03:19Zlcdwebscheduled reminder: activity reminder with choose recipients option limits to activity targetsTo reproduce:
1. create a scheduled reminder of type activity with a type/status as desired
2. in the recipients field, select "choose recipient(s)" and select a contact. for the purpose of testing, choose a delivery date of today
3. ...To reproduce:
1. create a scheduled reminder of type activity with a type/status as desired
2. in the recipients field, select "choose recipient(s)" and select a contact. for the purpose of testing, choose a delivery date of today
3. create an activity matching the type/status and dated today. the target should be someone other than the contact selected in 2
4. trigger the reminder
No reminder is sent. The selected contact does not receive the email and no activity is created. The issue is that the query used to retrieve matching activities applies the recipient contact filter to the table joined for activity targets. So the reminder is not sent unless the recipient is also a target. It effectively applies the selection as a limiting clause instead of allowing the message to be sent to anyone.lcdweblcdwebhttps://lab.civicrm.org/dev/core/-/issues/170contribution detail report errors when soft credits column is exposed2022-04-22T15:54:10Zlcdwebcontribution detail report errors when soft credits column is exposedto reproduce:
1. create a new contrib details report (or open existing)
2. select Soft Credits column
3. run the report
this errors with a "no such field" message.to reproduce:
1. create a new contrib details report (or open existing)
2. select Soft Credits column
3. run the report
this errors with a "no such field" message.5.2.1https://lab.civicrm.org/dev/core/-/issues/171Personal information can be leaked by omitting capture token in mailing2023-09-20T05:03:28ZIan KellingPersonal information can be leaked by omitting capture token in mailingI've tested this on 5.0. I don't see any changelog entry that would make
this not happen in newer versions.
I'm sure there is a more minimal repro if I understood the root cause.
Repro:
Create a group with 3 contacts, I'll call them ...I've tested this on 5.0. I don't see any changelog entry that would make
this not happen in newer versions.
I'm sure there is a more minimal repro if I understood the root cause.
Repro:
Create a group with 3 contacts, I'll call them contact1 contact2 contact3.
Do a mailing to that group, where the html and plain text email version is this:
{capture assign=addressee}{contact.addressee}{/capture}
Dear {if $addressee }{ $addressee }{else}Supporter{/if},
{domain.address}
{action.optOutUrl}
Have contact3 opt out using the link in the email.
Reuse that mailing to create a new one. Name it different, and change
just the text version of the email to be this:
Dear {if $addressee }{ $addressee }{else}Supporter{/if},
{domain.address}
{action.optOutUrl}
Expected result:
Text version of second email says "Dear Supporter" to two contacts.
Actual result:
contact1 gets an email saying "Dear Supporter". contact2 gets an email
saying "Dear contact1".
Now contact1's name and the fact that they are on this list has been
revealed to contact2. This is private information which should not
have been revealed to contact2.
This should be considered a very high priority issue, because in a
larger group, almost all contacts would receive the private information
of someone other than themselves, which is a serious data breach /
privacy problem.
If this bug is confirmed, as part of the fix I recommend creating a
tool which can analyze civi users past mailings to see if any of them
were affected by this bug.https://lab.civicrm.org/dev/core/-/issues/172Word replacements not working on Joomla2020-02-27T20:22:05Zaydunsaidan.saunders@squiffle.ukWord replacements not working on JoomlaOn a Joomla 3.8.8 system with CiviCRM 5.1.2 Word Replacements have no effect.
This was working when the system was on 4.6 before upgrading. Word Replacements are working on a 5.1.2 system with Drupal so it may be specific to Joomla.On a Joomla 3.8.8 system with CiviCRM 5.1.2 Word Replacements have no effect.
This was working when the system was on 4.6 before upgrading. Word Replacements are working on a 5.1.2 system with Drupal so it may be specific to Joomla.https://lab.civicrm.org/dev/core/-/issues/173Missing email on contribution thank you page when using pay later and a profi...2022-12-13T05:03:44ZAlanDixonMissing email on contribution thank you page when using pay later and a profile that includes an email addressAs per https://civicrm.stackexchange.com/questions/23034/email-address-is-missing-on-thank-you-page-where-it-says-an-email-confirmation reported by @petednz
1. It seems like the _emailExists property is setup to track when the email is...As per https://civicrm.stackexchange.com/questions/23034/email-address-is-missing-on-thank-you-page-where-it-says-an-email-confirmation reported by @petednz
1. It seems like the _emailExists property is setup to track when the email is included in a profile (and therefore not at it's usual key), and I notice that that property is handled on the Confirm page but not on the Thankyou page.
2. The email address does show up on the page within the profile display, so it's available.
So we just need to get it assigned to the email variable in the smarty, I think the preprocess method is where it should be done. I tried using some code similar to the stuff on the Confirm page, but it didn't work, I'm kind of making it up ...https://lab.civicrm.org/dev/core/-/issues/174Consistently use swappable cache interfaces2023-12-20T06:26:25ZtottenConsistently use swappable cache interfaces# Background
CiviCRM caches were originally (and are primarily) stored in the table `civicrm_cache`. This design is great for small sites (easy to deploy on server with less software to administer). However, for larger sites that get va...# Background
CiviCRM caches were originally (and are primarily) stored in the table `civicrm_cache`. This design is great for small sites (easy to deploy on server with less software to administer). However, for larger sites that get value from using multiple MySQL servers, it's not so great: cache data (by its nature) is frequently written+read, but it's expensive to write cache data when there are multiple MySQL servers... and the overhead doesn't give us much benefit because the data is fundamentally expendable. A lighter system (like Memcache/Redis) would be more generally appropriate.
You might recall that CiviCRM's cache subsystem can optionally incorporate Memcache/Redis (since roughly 4.3ish?). This appears as `CRM_Utils_Cache_*`, and it started some great improvements (e.g. using faster cache providers; and swapping cache providers). Notably, the functionality uses Memcache/Redis as an *extra cache tier* (i.e. when reading a cache, it first checks memcache; then it falls back to checking `civicrm_cache` in MySQL). This helps *read performance* (because your cache-hit usually returns faster), but it doesn't help *write performance* (because you need to propagate cache-writes to every tier). Using *only* Memcache/Redis (as a single tier -- without MySQL) would be even better for performance.
I don't believe that the status quo reflects a technical necessity. (If you can handle deployment/configuration of Memcache/Redis a front-tier... then you can probably handle it as the main tier.) Rather, it reflects a development practicality: *the app was originally written to only use `civicrm_cache`*. Consequently, there are a handful of oddball use-cases/code-paths which directly access `civicrm_cache` or `CRM_Core_BAO_Cache`, and it would've been harder to do anything if it had required finding+fixing all of them.
# Scope
The general scope of this ticket is to hunt down the oddball use-cases/code-paths which directly reference `civicrm_cache` or `CRM_Core_BAO_Cache`. I expect the project to produce a handful of distinct PRs. To avoid redundantly documenting this, I'm aiming for each PR to have a more concrete discussion about the use-cases/code-paths being cleaned up.
# PR List
* [12330 - CRM_Utils_Cache_Redis::flush() - Respect prefixes](https://github.com/civicrm/civicrm-core/pull/12330) (m)
* [12331 - CRM_Utils_Cache - Always use a prefix. Standardize delimiter](https://github.com/civicrm/civicrm-core/pull/12331) (m)
* [12336 - systemStatusCheckResult - Migrate from settings to cache](https://github.com/civicrm/civicrm-core/pull/12336) (m)
* [12342 - Caching - Comply with PSR-16 interfaces](https://github.com/civicrm/civicrm-core/pull/12342) (m)
* [215 - Import PSR-16 compliance test](https://github.com/civicrm/civicrm-packages/pull/215) (m)
* [12348 - Cache-keys for CRM_Utils_Cache_* should avoid reserved chars](https://github.com/civicrm/civicrm-core/pull/12348) (m)
* ~~[12350 - civicrm_cache - Allow storage of binary data](https://github.com/civicrm/civicrm-core/pull/12350)~~
* [12354 - civicrm_cache - Allow storage of binary data](https://github.com/civicrm/civicrm-core/pull/12354) (m)
* [~~12360~~ - Full PSR-16 compliance for ...](https://github.com/civicrm/civicrm-core/pull/12360) (*depends: ~~12342, 12348, 12354~~*). Cherry-picked to form smaller PRs:
* [12376 - Add various utilities to support PSR-16](https://github.com/civicrm/civicrm-core/pull/12376) (m)
* [12378 - ArrayCache, Redis](https://github.com/civicrm/civicrm-core/pull/12378) (m)
* [12379 - SqlGroup](https://github.com/civicrm/civicrm-core/pull/12379) (m)
* [12380 - Memcache(d)](https://github.com/civicrm/civicrm-core/pull/12380) (m)
* [12381 - APCcache](https://github.com/civicrm/civicrm-core/pull/12381) (m)
* [12389 - Add test to prevent hidden interactions among caches](https://github.com/civicrm/civicrm-core/pull/12389) (m)
* [12362 - Forms/Sessions - Store state via Civi::cache('session')](https://github.com/civicrm/civicrm-core/pull/12362) (*depends: 12360*)
* [12368 - ~~civicrm_cache - Finish transition from DATETIME to TIMESTAMP](https://github.com/civicrm/civicrm-core/pull/12368)~~ (m)
See also
--------
* dev/core#217 - Allow replacement of PrevNextCache implementation (for search screens)
* dev/core#284 - Aggressive cache clearing significantly increases test time5.9tottentottenhttps://lab.civicrm.org/dev/core/-/issues/175Smart group involving relationship type filter display incorrect results.2018-11-04T20:48:08ZjitendraSmart group involving relationship type filter display incorrect results.replicated on dmaster -
- Advanced Search -> Relationship tab -> type = `Employer of` (or any `b_a` reln type) -> Search (results all organization contacts).
- Proceed and add all of them to a smart group.
- When you click on `Done` aft...replicated on dmaster -
- Advanced Search -> Relationship tab -> type = `Employer of` (or any `b_a` reln type) -> Search (results all organization contacts).
- Proceed and add all of them to a smart group.
- When you click on `Done` after creating smart group, you are navigated to advanced search result with the contacts belonging to this group. All contacts are incorrectly displayed.
Another way to replicate -
- Advanced Search -> Relationship tab -> type = `Employer of` (or any `b_a` reln type) -> Search (results all organization contacts).
- Click on Search button again. All contacts are replaced with `Employee of` relationship5.4.0jitendrajitendrahttps://lab.civicrm.org/dev/core/-/issues/176Odd / Even street number sorting column missing from Reports2018-07-06T22:11:10Zjoels341Odd / Even street number sorting column missing from ReportsWhen trying to make a walk list in reports (odd/even street numbers), there is no option to sort by odd or even. I am using Civicrm in English on Joomla.
The French demo on Drupal has "Numéro de rue pair/impair" in the "column" dropdown...When trying to make a walk list in reports (odd/even street numbers), there is no option to sort by odd or even. I am using Civicrm in English on Joomla.
The French demo on Drupal has "Numéro de rue pair/impair" in the "column" dropdown, but I do not see it in the English version.![street-number](/uploads/f74556e4b99a646ee3aa1c9fdaacd96c/street-number.jpg)5.5.0Monish DebMonish Debhttps://lab.civicrm.org/dev/core/-/issues/177Redis driver - Reports incorrect value for cache-miss2018-06-17T23:17:42ZtottenRedis driver - Reports incorrect value for cache-miss`CRM_Utils_Cache_Interface::get()` specifies that the return value for a non-existent cache-key is `NULL`. But `CRM_Utils_Cache_Redis::get()` returns `FALSE`. This makes it impossible to detect a cache-miss.`CRM_Utils_Cache_Interface::get()` specifies that the return value for a non-existent cache-key is `NULL`. But `CRM_Utils_Cache_Redis::get()` returns `FALSE`. This makes it impossible to detect a cache-miss.5.4.0https://lab.civicrm.org/dev/core/-/issues/178Redis driver - Error messages are invisible2018-06-14T20:51:19ZtottenRedis driver - Error messages are invisibleIf the Redis server returns an error message (e.g. due to malformed requests or credentials), the substance of this message is invisible to the site administrator.If the Redis server returns an error message (e.g. due to malformed requests or credentials), the substance of this message is invisible to the site administrator.https://lab.civicrm.org/dev/core/-/issues/179Redis driver - Allow anonymous connections2018-06-15T21:04:39ZtottenRedis driver - Allow anonymous connectionsThe Redis driver *always* sends a password. That's well and good in production. However, if you're doing local development, the dev instance of redis may be configured to allow local-only, unauthenticated connections -- for which no pass...The Redis driver *always* sends a password. That's well and good in production. However, if you're doing local development, the dev instance of redis may be configured to allow local-only, unauthenticated connections -- for which no password is available.https://lab.civicrm.org/dev/core/-/issues/180Deleting custom values and custom_hook parameter $entity_id empty2018-06-13T20:41:11ZjaapjansmaDeleting custom values and custom_hook parameter $entity_id emptyIssue occured in CiviRules (https://github.com/CiviCooP/org.civicoop.civirules/issues/208)
The problem is that as soon as a record is deleted from a multi value custom group the hook civicrm_custom is called but `$entityId` parameter is...Issue occured in CiviRules (https://github.com/CiviCooP/org.civicoop.civirules/issues/208)
The problem is that as soon as a record is deleted from a multi value custom group the hook civicrm_custom is called but `$entityId` parameter is empty. This causes several errors in CiviRules as there is a trigger Custom Data Changed which assumes at least a contact id (which is this case the `$entityId`).
To reproduce:
create a custom data set for contacts that supports multiple records
create a rule that triggers on custom data changing
add a record to that custom data set for a contact
delete the record
observe the logs
This returns the error: "Expected one Contact but found 25"
Traced to CRM/CivirulesPostTrigger/ContactCustomDataChanged.php where there is an api call to contacts getsingle. The issue is that when the custom data record is deleted, there is no remaining entity_id with which to retrieve the contact, and so no id is passed to the getsingle call.https://lab.civicrm.org/dev/core/-/issues/356Grant export shows IDs instead of labels.2023-03-20T05:03:17ZyashodhaGrant export shows IDs instead of labels.Grant export shows IDs instead of labels.
Note : It doesn't work for *Select Fields to Export* but works for *Export PRIMARY fields*.Grant export shows IDs instead of labels.
Note : It doesn't work for *Select Fields to Export* but works for *Export PRIMARY fields*.yashodhayashodhahttps://lab.civicrm.org/dev/core/-/issues/181Export table field sizes are inadequate2020-09-18T04:30:33ZAlanDixonExport table field sizes are inadequateA client modified the default options for gender to include some values longer than 16 characters.
The export code tries to put them in a varchar(16) size field and it breaks.
The code seems to think that an integer field (CRM_Utils_Ty...A client modified the default options for gender to include some values longer than 16 characters.
The export code tries to put them in a varchar(16) size field and it breaks.
The code seems to think that an integer field (CRM_Utils_Type::T_INT) should be fine with a varchar(16) to hold it (https://github.com/civicrm/civicrm-core/blob/master/CRM/Export/BAO/Export.php, line 1123), but the thing is that it's putting the value, not the key as stored in the civicrm_contact table, so there's no telling how big a field you might want there. [As of July 30, 2018, the code is now in the ExportProcessor.php class pulled out in recent work by Eileen.]
I notice that there's some code to put in exceptions for prefix_id and suffix_id to use varchar(255), which seems like a short-sighted hack since it's going to potentially affect all the integer fields (at least, those that are storing keys into option groups).
So - either all integer fields should use varchar(255), or if this is just another exception, it can be added to the list (the vaguely named lookUp variable ...). Here's where those exceptions were added, it seems like not enough consideration was given to the actual cause of the problem: https://issues.civicrm.org/jira/browse/CRM-15877
Happy to put together a PR if no one has any additional insight.https://lab.civicrm.org/dev/core/-/issues/182Proposal - add testSetup hook2023-09-19T05:03:18ZeileenProposal - add testSetup hookI am having a problem with test interoperability. I have an extension that other extensions can hook into (by virtue of having an api with a particular action).
The issue appears in unit tests. On the main extension test it will call t...I am having a problem with test interoperability. I have an extension that other extensions can hook into (by virtue of having an api with a particular action).
The issue appears in unit tests. On the main extension test it will call the code in the other extension as it would in production. However, there are some specific set up actions required for testing on other extension that happen in it's own tests but not, currently, in the main extension's test runs.
My best idea for this is to have a hook that sits in core called civicrm_testSetup & which could be called from the setup function in a unit test.
I did think about using the existing Config hook - but we run these tests under the Drupal CMS rather than under the Unit tests CMS so I'm not sure how we would identify it's a test. There is also something a bit odd about putting test concerns in the main config function.
I also thought about just putting the hook in my extension - but it seems that I would quickly want to add it to more than one extension - hence I think core maybe makes sensehttps://lab.civicrm.org/dev/core/-/issues/183Temporary tables should follow consistent naming convention2020-08-31T02:03:19ZtottenTemporary tables should follow consistent naming convention#tldr
- check https://lab.civicrm.org/dev/core/issues/183#note_5717 for how to use the new syntax
# Context
For `epic:ro-db`, we aim to allow CiviCRM PHP workers to connect to read-only MySQL slaves and run searches+reports. These of...#tldr
- check https://lab.civicrm.org/dev/core/issues/183#note_5717 for how to use the new syntax
# Context
For `epic:ro-db`, we aim to allow CiviCRM PHP workers to connect to read-only MySQL slaves and run searches+reports. These often require temporary tables.
By default, if you have replicated MySQL configuration, MySQL replicates TEMPORARY tables. However, there isn't much point in doing this because TEMPORARY tables are session-scoped. (If there's a fault, you can't recover access to them.) One can disable replication by setting an option like `--replicate-wild-ignore-table=<wildcard>`. See also:
https://dev.mysql.com/doc/refman/5.7/en/replication-features-temptables.html
> A recommended practice when using statement-based or mixed-format replication is to designate a prefix for exclusive use in naming temporary tables that you do not want replicated...
If you search the CiviCRM source tree for temp tables, you can find a variety of table name formats... such as `civicrm_temp_exampledata` or `civicrm_exampledata_temp_{$rand}` or `X_exampledata` or just `exampledata`. Fortunately, these are amenable to change: they’re easy to find with “grep”, and (by virtue of being temporary tables) they’re only referenced a couple times. However, there are a large number of them, and (after renaming a temp-table) you should `r-run` the affected code to ensure it works.
Note: The benefit of the change will likely accrue on systems which use a replicated master/slave topology on MySQL v5.6+. However, we don't need to test that specific configuration. Our focus here is just on the naming.
# Goal
Enforce a common naming convention for temporary tables:
* `civicrm_tmp_e_%` matches any true temp table (i.e. "temporary-during-this-connection"; CREATE TEMPORARY TABLE). The "e" stands for "ephemeral".
* `civicrm_tmp_d_%` matches any quasi-temp table (i.e. "temporary-for-a-few-minutes-or-hours"; CREATE TABLE). The "d" stands for "durable".
* `civicrm_tmp_%` matches any temp table, whether ephemeral or durable.
This arrangement will give the MySQL DBA some decent ways to scope their policies.
To generate a table name, one can hard-code the prefix or use a helper. Following [12311](https://github.com/civicrm/civicrm-core/pull/12311), you can use the helper `CRM_Utils_SQL_TempTable`, e.g.
![Screen_Shot_2018-06-15_at_10.24.08_AM](/uploads/c7b670a73c0d1eafd464a1e65ddde0bd/Screen_Shot_2018-06-15_at_10.24.08_AM.png)
The general goal is update calls to `createTempTableName()` (or to manually-constructed table-names) with the newer convention, and then re-test anything that have used it.
Changing the prefix on temp tables should usually be pretty safe/portable, provided that we do some basic searching/testing after each item is renamed, i.e.
* Search code-base for anything that might have used the temporary-table name.
* Run a use-case or two that involves actually creating the temporary table.
# Audit list
Just to help track progress, here's a list of files with tell-tale signs that they may be using temporary (ephemeral or durable) tables. Here's the process for auditing each file:
1. Search for the file for any references to the target phrase (`CREATE TEMPORARY TABLE` or `CREATE TABLE` or `createTempTableName`).
2. Determine if the file is creating a temporary (ephemeral or durable) table. Check if the table name matches either `civicrm_tmp_e_%` or `civicrm_tmp_d_%`.
1. If it doesn't really create a temp-table, we're done.
2. If it creates a compliant temp-table, we're done.
3. If it creates a non-compliant temp-table, then we need to patch.
3. Figure out use-case(s) which hit that temp-table.
1. To see if you've found a real use-case, it helps to hack the code in question and provoke an error. (eg `throw new Exception('Oooga booga')`)
2. (NB: we're being optimistic that if one use-case works for the table that they all will. We're expecting failures to come out in the wash.)
4. Change the temp-table name. Search+update anything that references the same temp table. Verify the use-case still works as before.
5. Submit the patch. (If it's a new PR, include the link.)
6. Check the item as done. (The PR doesn't have to be merged -- we just to ensure that this item isn't re-examined.)
Files:
(note some of these have been fixed without being checked off)
* Files matching `CREATE TEMPORARY TABLE`
* [x] `CRM/Activity/BAO/Activity.php`
* [x] `CRM/Campaign/BAO/Query.php`
* [x] `CRM/Campaign/Form/Task/Interview.php`
* [x] `CRM/Contact/BAO/GroupContactCache.php`
* [x] `CRM/Contact/BAO/Query.php`
* [x] `CRM/Contact/Form/Search/Custom/ContribSYBNT.php`
* [x] `CRM/Contact/Form/Search/Custom/DateAdded.php`
* [x] `CRM/Contact/Form/Search/Custom/FullText.php`
* [x] `CRM/Contact/Form/Search/Custom/Group.php`
* [x] `CRM/Contact/Form/Search/Custom/PriceSet.php`
* [x] `CRM/Contact/Form/Search/Custom/RandomSegment.php`
* [x] `CRM/Contact/Import/Form/DataSource.php`
* [x] `CRM/Contribute/BAO/Query.php`
* [x] `CRM/Dedupe/BAO/QueryBuilder/IndividualUnsupervised.php`
* [x] `CRM/Dedupe/BAO/RuleGroup.php`
* [x] `CRM/Logging/ReportSummary.php`
* [x] `CRM/Mailing/BAO/Mailing.php`
* [x] `CRM/Mailing/BAO/Recipients.php`
* [x] `CRM/Mailing/Event/BAO/Delivered.php`
* [x] `CRM/Report/Form/Activity.php`
* [x] `CRM/Report/Form/ActivitySummary.php`
* [x] `CRM/Report/Form/Contribute/Bookkeeping.php`
* [x] `CRM/Report/Form/Contribute/Detail.php`
* [x] `CRM/Report/Form/Contribute/Lybunt.php`
* [x] `CRM/Report/Form/Contribute/Repeat.php`
* [x] `CRM/Report/Form/Member/ContributionDetail.php`
* [x] `CRM/Report/Form.php`
* [x] `CRM/Upgrade/Incremental/php/FourThree.php`
* [x] `CRM/Upgrade/Incremental/php/FourTwo.php`
* [x] `CRM/Utils/SQL/TempTable.php`
* [x] `Civi/Install/Requirements.php`
* [x] `tests/phpunit/api/v3/MailingTest.php`
* [x] `install/index.php`
* Files matching `CREATE TABLE`
* [x] `CRM/Contact/Form/Task.php`
* [x] `CRM/Contact/Import/Form/DataSource.php`
* [x] `CRM/Contact/Import/ImportJob.php`
* [x] `CRM/Core/BAO/SchemaHandler.php`
* [x] `CRM/Core/DAO.php`
* [x] `CRM/Export/BAO/Export.php`
* [x] `CRM/Import/DataSource/CSV.php`
* [x] `CRM/Logging/Schema.php`
* [x] `CRM/Upgrade/Incremental/php/FourFour.php`
* [x] `CRM/Upgrade/Incremental/php/FourSeven.php`
* [x] `CRM/Upgrade/Incremental/sql/4.2.alpha1.mysql.tpl`
* [x] `CRM/Upgrade/Incremental/sql/4.2.beta1.mysql.tpl`
* [x] `CRM/Upgrade/Incremental/sql/4.2.beta6.mysql.tpl`
* [x] `CRM/Upgrade/Incremental/sql/4.3.alpha1.mysql.tpl`
* [x] `CRM/Upgrade/Incremental/sql/4.4.alpha1.mysql.tpl`
* [x] `CRM/Upgrade/Incremental/sql/4.5.alpha1.mysql.tpl`
* [x] `CRM/Upgrade/Incremental/sql/4.6.3.mysql.tpl`
* [x] `CRM/Upgrade/Incremental/sql/4.6.5.mysql.tpl`
* [x] `CRM/Upgrade/Incremental/sql/4.6.alpha1.mysql.tpl`
* [x] `CRM/Upgrade/Incremental/sql/4.7.alpha1.mysql.tpl`
* [x] `CRM/Utils/SQL/TempTable.php`
* [x] `CRM/Utils/String.php`
* [x] `Civi/Core/InstallationCanary.php`
* [x] `Civi/Install/Requirements.php`
* [x] `Civi/Test/CiviEnvBuilder.php`
* [x] `tests/phpunit/api/v3/CustomApiTest.php`
* [x] `tests/phpunit/api/v3/dataset/uf_group_contact_activity_26.xml`
* [x] `tests/phpunit/api/v3/LoggingTest.php`
* [x] `tests/phpunit/CRM/Contact/Import/Form/MapFieldTest.php`
* [x] `tests/phpunit/CRM/Core/BAO/SchemaHandlerTest.php`
* [x] `tests/phpunit/CRM/Core/DAOTest.php`
* [x] `tests/phpunit/CRM/Logging/LoggingTest.php`
* [x] `tests/phpunit/CRM/Logging/SchemaTest.php`
* [x] `tests/phpunit/CRM/Utils/QueryFormatterTest.php`
* [x] `install/index.php`
* Files matching `createTempTableName`
* [x] `CRM/Campaign/BAO/Query.php`
* [x] `CRM/Campaign/Form/Task/Interview.php`
* [x] `CRM/Contact/BAO/Query.php`
* [x] `CRM/Core/DAO.php`
* [x] `CRM/Report/Form/ActivitySummary.php`
* [x] `CRM/Report/Form/Contribute/Bookkeeping.php`
* [x] `CRM/Upgrade/Incremental/php/FourThree.php`
* [x] `CRM/Upgrade/Incremental/php/FourTwo.php`
* [x] `tests/phpunit/CRM/Core/DAOTest.php`
# Audit Mentality
The deliverables are patches -- hopefully mostly simple patches -- but this is a bit misleading. If we just wanted simple patches to some files, there are faster ways to edit the files. The main work in here is the audit -- ie when we look at a file, figure out how it's used. That's why we have to do searching+testing as part of the update.tottentottenhttps://lab.civicrm.org/dev/core/-/issues/184Event registration transfer - wrong fees shown2023-02-20T05:04:08Zaydunsaidan.saunders@squiffle.ukEvent registration transfer - wrong fees shownWhen an event registration is transferred, the fees are doubled on the original registration record.
For example, on dmaster.demo.civicrm.org I transferred the event registration from Shauna Barkley to Betty Adams.
The $50 contribution ...When an event registration is transferred, the fees are doubled on the original registration record.
For example, on dmaster.demo.civicrm.org I transferred the event registration from Shauna Barkley to Betty Adams.
The $50 contribution stays on Shauna's record, there is no additional contribution on Betty's record and the event registration for Betty is correct.
But the transferred event registration for Shauna shows Total Fees of $100 - with an outstanding balance of $50:
![Screenshot_from_2018-06-14_20-57-53](/uploads/e71937197c3642e3c10fb70fea1d111f/Screenshot_from_2018-06-14_20-57-53.png)
If this just affect the registration that this was transferred from then it is fairly minor, but it might point to a more serious issue in the way the transfer is being done.https://lab.civicrm.org/dev/core/-/issues/185“Print selected rows” option missing in Event Participant listing dropdown ac...2018-10-30T09:51:44ZPradeep Nayakpradpnayak@gmail.com“Print selected rows” option missing in Event Participant listing dropdown actionshttps://civicrm.stackexchange.com/questions/25363/print-selected-rows-option-missing-in-event-participant-listing-dropdown-actio/25365#25365https://civicrm.stackexchange.com/questions/25363/print-selected-rows-option-missing-in-event-participant-listing-dropdown-actio/25365#25365https://lab.civicrm.org/dev/core/-/issues/186Consider phasing out various custom-searches2022-08-16T05:03:35ZtottenConsider phasing out various custom-searchesThere are several custom-searches. I'm sometimes confused by the behavior with them -- and generally feel suspicious because (a) they don't have much test coverage and (b) they involve lots of edge-cases/combinations and (c) they're not ...There are several custom-searches. I'm sometimes confused by the behavior with them -- and generally feel suspicious because (a) they don't have much test coverage and (b) they involve lots of edge-cases/combinations and (c) they're not very visible to users (ie not much foot-traffic), (d) I sort think they were written quickly a long time ago, and (e) there's overlap with reporting.
It would be useful to re-evaluate each of them and perhaps phase-out some. For example, a phase-out might look like this:
1. Create a new extension
2. Move the search(es) to the extension
3. Bundle the extension with the main tarball
4. Disable by default
5. Add an upgrade message telling people how to get them back
6. Test the upgrade/transition process to ensure it works
NOTE: Take care when disabling/re-enabling. There might be references to the searches in various places (e.g. smart-group definition; custom navigation), and you want the old IDs/URLs to continue work.https://lab.civicrm.org/dev/core/-/issues/187Fix typo in CRM_Grant_Form_Task that prevents retrieving session key from URL2018-06-21T21:41:15Zmattwiremjw@mjwconsult.co.ukFix typo in CRM_Grant_Form_Task that prevents retrieving session key from URLOverview
----------------------------------------
Identified during investigation for dev/core#158
Before
----------------------------------------
Cannot retrieve session key for Grant form task.
After
---------------------------------...Overview
----------------------------------------
Identified during investigation for dev/core#158
Before
----------------------------------------
Cannot retrieve session key for Grant form task.
After
----------------------------------------
Session key for Grant form task is retrieved just like all other tasks.
Technical Details
----------------------------------------
The wrong object was being used in static context ($this instead of $form).5.4.0https://lab.civicrm.org/dev/core/-/issues/188Fix Floating Point Precision Comparison Exception on Order Creation2018-07-15T23:12:07ZCamilo RodríguezFix Floating Point Precision Comparison Exception on Order Creation## Overview
When adding Orders, a check is done by CiviCRM core when the contribution is being created to make sure the sum of line item totals is equal to the contribution total amount. This comparison, however, is done comparing floati...## Overview
When adding Orders, a check is done by CiviCRM core when the contribution is being created to make sure the sum of line item totals is equal to the contribution total amount. This comparison, however, is done comparing floating point values, which may fail falsely, under some circumstances.
This is the line where the comparison is done:
https://github.com/civicrm/civicrm-core/blob/master/CRM/Contribute/BAO/Contribution.php#L5011
This problem is documented in PHP official documentation, and is actually related to the way floating point values are modeled in programming languages in general.
http://php.net/manual/en/language.types.float.php
http://floating-point-gui.de/basic/
This has to be taken into account to make sure the comparison doesn't fail when it shouldn't.
## How it Works Currently
1. Use API to try to create Order with given sample data.
2. Exception is thrown, saying "Line item total doesn't match with total amount".
## How it Should Work
1. Use API to try to create Order with given sample data.
2. Order is created without throwing exception.
## Sample Data
```
data ={
'contact_id': 1051,
'payment_instrument_id': 6,
'line_items': [{
'line_item': {
'0': {
'price_field_value_id': 56,
'price_field_id': 33,
'entity_id': 646,
'tax_amount': 0,
'line_total': 14.85,
'label': 4,
'entity_table': 'civicrm_membership',
'unit_price': 14.85,
'qty': 1}}}, {
'line_item': {
'1': {
'price_field_value_id': 55,
'price_field_id': 32,
'entity_id': 648,
'tax_amount': 0,
'line_total': 1.66,
'label': 50,
'entity_table': 'civicrm_membership',
'unit_price': 1.66,
'qty': 1}}}, {
'line_item': {
'2': {
'price_field_value_id': 49,
'price_field_id': 26,
'entity_id': 647,
'tax_amount': 0,
'line_total': 0.16,
'label': 47,
'entity_table': 'civicrm_membership',
'unit_price': 0.16,
'qty': 1}}}],
'total_amount': 16.67,
'financial_type_id': 2,
'fee_amount': 0,
'payment_processor_id': 5,
'receive_date': '2017-10-16',
'contribution_status_id': 1
}
```
## Failing API Call Example
Calling API using cv:
```
echo '{"contact_id":45,"payment_instrument_id":6,"line_items":[{"line_item":{"6":{"price_field_value_id":52,"price_field_id":29,"tax_amount":0,"line_total":14.85,"financial_type_id":2,"qty":1,"entity_table":"civicrm_membership","unit_price":14.85,"label":5}}},{"line_item":{"7":{"price_field_value_id":53,"price_field_id":30,"tax_amount":0,"line_total":1.66,"financial_type_id":2,"qty":1,"entity_table":"civicrm_membership","unit_price":1.66,"label":49}}},{"line_item":{"8":{"price_field_value_id":49,"price_field_id":26,"tax_amount":0,"line_total":0.16,"financial_type_id":2,"qty":1,"entity_table":"civicrm_membership","unit_price":0.16,"label":47}}}],"tax_amount":0,"total_amount":16.67,"contribution_recur_id":733,"financial_type_id":2,"fee_amount":0,"payment_processor_id":5,"receive_date":"2017-12-15","contribution_status_id":1}' | cv api Order.create --in=json
```5.5.0https://lab.civicrm.org/dev/core/-/issues/189Event Fees not loading when registering in back-end2018-06-20T03:00:42ZPradeep Nayakpradpnayak@gmail.comEvent Fees not loading when registering in back-endRecently upgraded a site to 5.2.1 in WordPress 4.9.6. When the user tries to register a event participant in the back-end the fee selections price set does not load after the event is selected. All works fine on front end pages and if yo...Recently upgraded a site to 5.2.1 in WordPress 4.9.6. When the user tries to register a event participant in the back-end the fee selections price set does not load after the event is selected. All works fine on front end pages and if you edit an existing registration you are able to change the fee selections.
Checked another site with the same setup and the results are the same.
The results are the same if you try from the contact record either using the register button or the submit CC payment with registration button.
From apache error log
[Tue Jun 19 00:21:46.010459 2018] [:error] [pid 10807] [client 127.0.0.1:59798] PHP Fatal error: Smarty error: [in CRM/Event/Form/EventFees.tpl line 27]: syntax error: expecting '=' after attribute name 'neq' (Smarty_Compiler.class.php, line 1558) in /var/www/html/staging.org/wp-content/plugins/civicrm/civicrm/packages/Smarty/Smarty.class.php on line 1100, referer: http://localhost/staging.org/wp-admin/admin.php?page=CiviCRM&q=civicrm%2Fparticipant%2Fadd&reset=1&action=add&context=standalone5.2.2https://lab.civicrm.org/dev/core/-/issues/190custom data with multiple records profile: after submission, profile view sho...2019-02-24T19:02:48Zlcdwebcustom data with multiple records profile: after submission, profile view shows first record instead of most recentTo reproduce:
* create a custom data set with multiple records enabled. create a field or two.
* create a profile that includes the fields and the email field (to enable duplicate matching). make sure "update contact" is selected in the...To reproduce:
* create a custom data set with multiple records enabled. create a field or two.
* create a profile that includes the fields and the email field (to enable duplicate matching). make sure "update contact" is selected in the advanced setting.
* open the profile in create view
* submit the form with values
* go back and submit the form a second time with different values
* notice that after submitting the second time, the profile view displays values from the first time you submitted the form, not the most recent5.12.0lcdweblcdweb