Development issueshttps://lab.civicrm.org/groups/dev/-/issues2022-06-11T14:57:32Zhttps://lab.civicrm.org/dev/core/-/issues/3629Extraneous space in From address causes on-hold set on all recipients2022-06-11T14:57:32ZBobSExtraneous space in From address causes on-hold set on all recipientsIf a CiviMail email is sent with a From address having the form<br>
`"Name" <me@example.com>` (note two spaces before '<')<br>
then:
1. No emails are sent
2. **All recipients are marked On-Hold!**
Occurs on CiviCRM 4.7.26.
*Edit*: Al...If a CiviMail email is sent with a From address having the form<br>
`"Name" <me@example.com>` (note two spaces before '<')<br>
then:
1. No emails are sent
2. **All recipients are marked On-Hold!**
Occurs on CiviCRM 4.7.26.
*Edit*: Also occurs on 5.2.1
Could not reproduce on demo site as sending email is not enabled there.
The following three CMS log entries are created for each recipient:
1:<pre>Ignoring exception thrown by nullHandler: , Validation failed for: "(INVALID)" <(INVALID)></pre>
2:<pre>$backTrace = #0 /var/www/clients/client1/web3/web/drupal/sites/all/modules/civicrm/CRM/Core/Error.php(959): CRM_Core_Error::backtrace("backTrace", TRUE) #1 /var/www/clients/client1/web3/web/drupal/sites/all/modules/civicrm/packages/PEAR.php(921): CRM_Core_Error::nullHandler(Object(PEAR_Error)) #2 /var/www/clients/client1/web3/web/drupal/sites/all/modules/civicrm/packages/PEAR.php(577): PEAR_Error->__construct("Validation failed for: \"(INVALID)\" <(INVALID)>", NULL, 16, (Array:2), NULL) #3 /var/www/clients/client1/web3/web/drupal/sites/all/modules/civicrm/packages/PEAR.php(236): PEAR::_raiseError(NULL, "Validation failed for: \"(INVALID)\" <(INVALID)>") #4 /var/www/clients/client1/web3/web/drupal/sites/all/modules/civicrm/packages/Mail/RFC822.php(209): PEAR::__callStatic("raiseError", (Array:1)) #5 /var/www/clients/client1/web3/web/drupal/sites/all/modules/civicrm/packages/Mail.php(191): Mail_RFC822->parseAddressList((Array:2), "localhost", FALSE) #6 /var/www/clients/client1/web3/web/drupal/sites/all/modules/civicrm/packages/Mail/mail.php(151): Mail->prepareHeaders((Array:10)) #7 ...</pre>
3:<pre>
Notice: Only variables should be passed by reference in CRM_Mailing_BAO_MailingJob->deliverGroup() (line 721 of /var/www/clients/client1/web3/web/drupal/sites/all/modules/civicrm/CRM/Mailing/BAO/MailingJob.php).
</pre>5.5.0https://lab.civicrm.org/dev/core/-/issues/3634When using a smart group as a mailing list, users who unsubscribe from the sm...2022-06-11T14:57:45Ztom.mWhen using a smart group as a mailing list, users who unsubscribe from the smart group are still included in the mailing.It looks like in CRM/MailingBAO/Mailing.php, is not taking into account those who have been removed from the smart group. Civi version 5.3.
```
if (count($includeSmartGroupIDs)) {
$query = CRM_Utils_SQL_Select::from($contact)
...It looks like in CRM/MailingBAO/Mailing.php, is not taking into account those who have been removed from the smart group. Civi version 5.3.
```
if (count($includeSmartGroupIDs)) {
$query = CRM_Utils_SQL_Select::from($contact)
->select("$contact.id as contact_id, $entityTable.id as $entityColumn")
->join($entityTable, " INNER JOIN $entityTable ON $entityTable.contact_id = $contact.id ")
->join('gc', " INNER JOIN civicrm_group_contact_cache gc ON $contact.id = gc.contact_id ")
->join('mg', " INNER JOIN civicrm_mailing_group mg ON gc.group_id = mg.entity_id AND mg.search_id IS NULL ")
->join('temp', " LEFT JOIN $excludeTempTablename temp ON $contact.id = temp.contact_id ")
->where('gc.group_id IN (#groups)')
->merge($criteria)
->replaceInto($includedTempTablename, array('contact_id', $entityColumn))
->param('#groups', $includeSmartGroupIDs)
->param('#mailingID', $mailingID)
->execute();
}
```https://lab.civicrm.org/dev/core/-/issues/3636CiviCRM, Find Mailings. If there are no results, the same message "There are ...2022-06-11T14:57:49Zjustinfreeman (Agileware)CiviCRM, Find Mailings. If there are no results, the same message "There are no Unscheduled Mailings" is shownCiviCRM, Find Mailings. If there are no results, the same message "There are no Unscheduled Mailings" is shown regardless of the search criteria. If a different mailing status was used for search, it does not make sense to refer to "Unsc...CiviCRM, Find Mailings. If there are no results, the same message "There are no Unscheduled Mailings" is shown regardless of the search criteria. If a different mailing status was used for search, it does not make sense to refer to "Unscheduled Mailings".
This text should be changed to something else like: "There are no mailings matching this criteria".
Using CiviCRM 5.7.2
Agileware Ref: CIVICRM-1121
![Find_Mailings-2](/uploads/34e6b1996338721a1a2a333a3a16cd2a/Find_Mailings-2.png)https://lab.civicrm.org/dev/core/-/issues/3637Google G Suite: Sending via smtp.gmail.com will alter the From Address2022-06-11T14:57:51ZkenGoogle G Suite: Sending via smtp.gmail.com will alter the From AddressIf you're using Google G Suite, send email via smtp-relay.gmail.com rather than smtp.gmail.com.
If you use smtp.gmail.com then the FROM header in the email will be a mashup of the From Address and the SMTP Username.
For example ...
* ...If you're using Google G Suite, send email via smtp-relay.gmail.com rather than smtp.gmail.com.
If you use smtp.gmail.com then the FROM header in the email will be a mashup of the From Address and the SMTP Username.
For example ...
* Say my SMTP Username is set in _Settings - Outbound Mail_ to **smtp@example.com**
* Say I use the following _From Address_ in my CiviMail: **My Name <my.name@example.com>**
* The FROM header in the email will be **From: My Name <smtp@example.com>**
Using smtp-relay.gmail.com does what you expect.https://lab.civicrm.org/dev/core/-/issues/3640Most recent contact note exposed in event confirmation emails.2022-06-11T14:57:56ZBobSMost recent contact note exposed in event confirmation emails.* Create an event where the on-line registration profile contain a Contacts:Note field.
* Contact leaves that field blank during form submission.
* Result: Contact receives a confirmation email showing the most recent note on that contac...* Create an event where the on-line registration profile contain a Contacts:Note field.
* Contact leaves that field blank during form submission.
* Result: Contact receives a confirmation email showing the most recent note on that contact.
Not sure if this behavior is intentional, but I consider it problematic.
- a) Showing a contact a note he might have submitted years ago is rather confusing.
- b) Backend users may have an expectation of confidentiality when they add notes to a contact record, believing that info will never be exposed to the contact.
Confirmed on CiviCRM 5.13.4, 5.15.1.https://lab.civicrm.org/dev/core/-/issues/3644Delivery summary doesn't show2022-06-11T14:58:02ZLoganBearDelivery summary doesn't showUpdated to CiviCRM 5.19 - ran one of our biweekly mailings last night and I wanted to check the open count. But Civi says that the mailing isn't completed,![Screen_Shot_2019-11-08_at_10.23.56_AM](/uploads/9723b265e4f977ffebce0033575d5291...Updated to CiviCRM 5.19 - ran one of our biweekly mailings last night and I wanted to check the open count. But Civi says that the mailing isn't completed,![Screen_Shot_2019-11-08_at_10.23.56_AM](/uploads/9723b265e4f977ffebce0033575d5291/Screen_Shot_2019-11-08_at_10.23.56_AM.png)
but the summary shows it's completed.![Screen_Shot_2019-11-08_at_10.23.41_AM](/uploads/24352601bef0fce245cc4198826cb937/Screen_Shot_2019-11-08_at_10.23.41_AM.png)5.19.1https://lab.civicrm.org/dev/core/-/issues/3646Support tracking URLs with tokens in query strings2022-06-11T14:58:11ZlarsssandergreenSupport tracking URLs with tokens in query stringsURLs with tokens in mailings do not have click tracking. This is an important metric for fundraising (e.g. to see how many and even which contacts click through to contribution pages). @artfulrobot put together some code that worked well...URLs with tokens in mailings do not have click tracking. This is an important metric for fundraising (e.g. to see how many and even which contacts click through to contribution pages). @artfulrobot put together some code that worked well to track these clicks in Flexmailer, but that code was not merged before Flexmailer was merged into core.
https://github.com/civicrm/org.civicrm.flexmailer/issues/30
We used this without issue for four months and found it worked very well. @artfulrobot was using it since March. Would it be possible to merge this into core. Rich, would you be willing to copy your code to a patch against core?RichRichhttps://lab.civicrm.org/dev/core/-/issues/3649Token renders in message preview and test message, not in final sent message2022-06-11T14:58:16ZChrisHardieToken renders in message preview and test message, not in final sent messageI'm on CiviCRM 5.35.1 with WordPress 5.7. Today I used CiviMail for the first time to send a mailing to a group of contacts. I used some tokens (custom contact fields) that rendered fine in my "preview HTML" view and in the generated mes...I'm on CiviCRM 5.35.1 with WordPress 5.7. Today I used CiviMail for the first time to send a mailing to a group of contacts. I used some tokens (custom contact fields) that rendered fine in my "preview HTML" view and in the generated message when I sent a test email to myself. However, in the final sent message that went out to recipients, they were not rendered and instead displayed as `{contact.custom_10}` and `{contact.custom_9}`. Other tokens were rendered just fine. The only thing that seems different about these two is that they are custom fields, and that they were in a bolded line of text.
Is this something I've missed in our configuration? Let me know if you need any other information. Thank you.https://lab.civicrm.org/dev/core/-/issues/3485Installer - CiviGrant option no longer works2022-06-11T15:47:08ZtottenInstaller - CiviGrant option no longer worksOverview
----------------------------------------
When performing a web-based installation on Drupal or WordPress, it presents a list of components that you can toggle. One option no longer works correctly.
Reproduction steps
---------...Overview
----------------------------------------
When performing a web-based installation on Drupal or WordPress, it presents a list of components that you can toggle. One option no longer works correctly.
Reproduction steps
----------------------------------------
1. Create an empty D7/BD/WP site
2. Enable CiviCRM
3. In the status alert, click the link to `civicrm/setup`
4. Enable the CiviGrant option
5. Proceed with install
Current behaviour
----------------------------------------
CiviGrant is not activated
Expected behaviour
----------------------------------------
Activate CiviGrant extension
Or: Don't present the option
Environment information
----------------------------------------
Hydra site for WP: http://site-list.test-1.civicrm.org:8001/?filter=hydra-* (http://hydra-wp.test-1.civicrm.org:8002/)
Comments
----------------------------------------
I noticed this while trying the 5.50 installer, but it probably regressed circa 5.48.
The installer currently presents a list of components (https://github.com/civicrm/civicrm-core/blob/master/setup/plugins/blocks/components.civi-setup.php) and populates `$e->getModel()->components`.
To configurably toggle extensions during install, it needs to update `$e->getModel()->extensions`.
https://github.com/civicrm/civicrm-core/blob/master/setup/src/Setup/Model.php#L42-L455.51.0https://lab.civicrm.org/dev/core/-/issues/2141OAuth2 administration (email focus)2022-06-11T16:02:07ZtottenOAuth2 administration (email focus)# Background
CiviCRM integrates with various third-party services. When submitting read/write requests to contemporary web APIs, it is quite common to authenticate via OAuth - esp OAuth2 (https://tools.ietf.org/html/rfc6749). For this u...# Background
CiviCRM integrates with various third-party services. When submitting read/write requests to contemporary web APIs, it is quite common to authenticate via OAuth - esp OAuth2 (https://tools.ietf.org/html/rfc6749). For this use-case, we consider CiviCRM (the web-app) acting a client to some service-provider. Two important workflows are:
* _Client registration_: This is a manual, non-standardized process. The site admin logs into the Google/Microsoft/Facebook/Twitter/etc administrative system and generates a `client_id` and `client_secret`. These values must be copied into the web-app (CiviCRM) configuration.
* _Authorization Code Grant_: This process obtains permissions for a specific user/resource/service. It involves (a) performing a semi-manual workflow to confirm permissions and (b) periodically requesting access tokens.
For CiviCRM integrations (core or contrib) that wish to speak to other systems, this creates a recurring administrative problem:
* (Developer) You need to define several artifacts (docs, settings, routes) for managing `client_id`, `client_secret`, and each of the authorizations/grants.
* (Administrator) You need to examine the separate artifacts (docs, settings, etc) required by each integration, and there will likely be arbitrary differences.
This proposal arises from https://lab.civicrm.org/dev/mail/-/issues/59 and parallel issues for Microsoft's email service. When we need examples to make this more concrete, we'll refer to IMAP and cloud email.
# Goal
Define a general OAuth2 administration mechanism in CiviCRM.
To keep the scope limited, the initial concern is managing credentials for *general system-level integrations* (e.g. `Administer => System Settings => Credentials`). For example, one might use this to:
* Connect CiviMail with a cloud mail service
* Connect CiviContribute with a cloud accounting system
This does not address the per-user authorizations. For example, if you wished to allow each staff member or peer-to-peer fundraiser to link their own email account (for use with sending one-on-one emails), then that would be a different endeavor. (I believe it is a *superset* of the endeavor described here, and the "Add-ons/Strech-goals/Future-directions" touches on this.)
# Model
We track information in three layers:
1. _Provider (Client-Type)_: Suppose, for example, that many CiviCRM deployments connect to services provided by Google and Microsoft. Anytime you connect to Google, it should go to a particular URL; similarly, anytime you connect to Microsoft, it goes a particular URL. You need store a list of these common URLs. For each provider, you'd expect a few key properties like:
* `authorize_url`
* `token_url`
* `default_scopes`
* `provider_class` (string; ex: in `theleague/oauth2-client`, it has `\League\OAuth2\Client\Provider\GenericProvider` for generic use; but [alternate classes](https://oauth2-client.thephpleague.com/providers/thirdparty/) can provide extra options+data)
2. _Client_: When a CiviCRM site (`example.com`) connects to a provider, it must identify itself. Key properties:
* `client_id` (unique id; functions a username when contacting the OAuth2 service)
* `client_secret` (secret credential; functions a password when contacting the OAuth2 service)
3. _Account Authorization_: When a CiviCRM site is configured as a client, it may not have authorization to any particular resources/accounts/services. To acquire access, you need the user ("resource owner") to open the web-app and go through some redirects/authorization screens.
# Ex: IMAP: UI Mockup
The current IMAP configuration screen accepts a username+password. Instead, one needs to use credentials that are stored in an OAuth2 subsystem. Compare:
* With username+password: [New_Mail_Account_Orig](/uploads/1153ffb46c67689bcf85e6e51cb0f42e/New_Mail_Account_Orig.png)
* With choice of username+password or OAuth credentials: [New_Mail_Account_UA](/uploads/99d771dffb7120528d7ed7f9854c766c/New_Mail_Account_UA.png) ~~[New_Mail_Account_XOAuth2](/uploads/69de67526f7cbb02ded8118eac2cab57/New_Mail_Account_XOAuth2.png)~~
The "Manage Credentials" screen gives the Civi administrator the ability to configure the client identity and the account authorizations. The layout might be:
* Tabular layout, akin to many of Civi's other administrative screens -- e.g. [Manage_Credentials_Table](/uploads/a6a30561dd60a3f7854440b8e9a797a5/Manage_Credentials_Table.png)
* Bespoke layout, with more key details on the admin screen -- e.g. [Manage_Credentials_Sections](/uploads/df793faa4104bbde61220cd8ee044820/Manage_Credentials_Sections.png)
# Ex: IMAP: General tasks
A loose/general list of tasks:
* _Add OAuth data structures_:
* _Client types_: Define a data-structure (e.g. JSON file, hooks) for programmatically defining client types.
* (Should this have a SQL storage option? IMHO, no, that's an extremely niche proposition. To require that the administrator to first create the "Client Type" in the web UI seems like an exercise in software brutalism.)
* _Clients_: Define a data-structure for client_ids, client_secrets, etc. These details are necessarily different for each deployment, so it's necessary to store them somewhere in the admin's purview -- ie SQL and/or `civicrm.settings.php`.
* _Account Authorizations and Tokens_: Define a data-structure for account authorizations and their tokens. These must be stored in SQL.
* _Update IMAP/POP data-structure_:
* Store a reference to OAuth
* _Implement UI_: Like in the mockups above:
* Implement the "Manage credentials" screen and "Add account" flow (incl [thephpleague/oauth2-client](https://github.com/thephpleague/oauth2-client))
* Update the "New Mail Account" screen
* _Update IMAP/POP driver_: The current IMAP/pop driver uses username/password. If a specific mailbox uses IMAP
* _Documentation_:
* Developer: How to define or edit a client-type
* Developer: How to send a web-service request with stored credentials and Guzzle
* Sysadmin: How to manage clients/connections
# Add-ons / Stretch goals / Future directions
Here are few topics for potential exploration:
* _E2E Testing_: Setup a test-case which makes real IMAP+OAuth2 connections
* _Reuse client-type/provider metadata_: The list of client-types is, essentially, metadata. The `oauthd` project has a ton of metadata stored as JSON files. Consider using their metadata. https://github.com/oauth-io/oauthd/tree/master/providers
* _Aggregator support_: This issue is oriented toward singular/dedicated deployments. However, if you have multi-tenant/multi-site deployments, then deploying OAuth2 is more onerous. (*To wit: do you register every site with its own client_ids -- which is a lot of manual setup -- or do you copy/share the IDs -- which is less secure and may not be supported by all providers*). An OAuth aggregator allows your various tenants/subsites to use the same client ids more securely. Generally, the goal would be to integrate the list of "Clients" in Civi with the aggregator.
* (The oauthd project linked above is an aggregator. It seems it isn't being updated, but it does have a lot of forks. Curious.)
* _In-situ authorization_ or _Wizards_: The "Manage Credentials" screen is disjoint from the "New Mail Account" screen. With in-situ authorization or wizards, you could handle the authorization in-siteu (e.g. in "New Mail Account" or in CiviMail).
* _Authorization (Meta)Access Controls_: This issue is oriented toward a global/shared service (like the CiviCase IMAP checker), but OAuth can also be used for per-user services. Example: Suppose staffer Alice wants to send a mail-merge document to Google Cloud Print. This probably requires OAuth authorization. Civi can store that credential and re-use it whenever Alice wants to print. But if Bob wants to print, then he needs to go through a different authorization (because he uses different printers).https://lab.civicrm.org/dev/core/-/issues/123Import - Participant - Custom participant date fields are not formatted2022-06-11T16:02:19ZtschuettlerImport - Participant - Custom participant date fields are not formattedThe date fields are not converted from the import date format to the default date format and thus end up beeing imported with a date value of 0.
![image](/uploads/dda9843f54de137b5060553f4785655f/image.png)
Additionally 2 notice errors...The date fields are not converted from the import date format to the default date format and thus end up beeing imported with a date value of 0.
![image](/uploads/dda9843f54de137b5060553f4785655f/image.png)
Additionally 2 notice errors appear for each mapped custom particpant field in the import file:
>Notice: Undefined offset: 8 in CRM_Event_Import_Parser_Participant->import() (line 296 of /opt/buildkit/build/dmaster/sites/all/modules/civicrm/CRM/Event/Import/Parser/Participant.php).
>Notice: Undefined offset: 8 in CRM_Event_Import_Parser_Participant->import() (line 300 of /opt/buildkit/build/dmaster/sites/all/modules/civicrm/CRM/Event/Import/Parser/Participant.php).
Steps to reproduce:
1. Create a custom date field for participants
1. Import a participant with a non default date format: [participant_test.csv](/uploads/83f09e28f2435b819b41e120a9e28968/participant_test.csv)
See https://issues.civicrm.org/jira/browse/CRM-19386 for the same issue with activity imports.
A PR will be provided.5.3.0https://lab.civicrm.org/dev/core/-/issues/1234Disabling a relationship type used in a case role causes the label to be blan...2022-06-11T16:02:31ZDaveDDisabling a relationship type used in a case role causes the label to be blank on the case type edit screen![screenshot](/uploads/9090c9fc22c472e52ba729a067b2f4ef/screenshot.png)
I know there's differing opinions on what a disabled option value means but probably everyone would agree that a blank row seems wrong. Interestingly everything sti...![screenshot](/uploads/9090c9fc22c472e52ba729a067b2f4ef/screenshot.png)
I know there's differing opinions on what a disabled option value means but probably everyone would agree that a blank row seems wrong. Interestingly everything still seems to work and it lets you save. And when you view an existing case it still shows the previously assigned role correctly (in my opinion). But where this came up was while working on #1046 I'm noticing that the existing functions only return active option values, and that's fine sometimes but that isn't what you want when converting values/names to labels or doing comparisons against name, which is also what's happening here. TBD.5.20.0https://lab.civicrm.org/dev/core/-/issues/236MCrypt extension is abandonware and dependency on it should be removed2022-06-11T16:02:31ZAgilewareMCrypt extension is abandonware and dependency on it should be removedPer the deprecation notes for PHP7.1 - http://php.net/manual/en/migration71.deprecated.php, The MCrypt extension has not been maintained for a long time, and is actually removed from the core distribution to PECL in 7.2.
Apart from inst...Per the deprecation notes for PHP7.1 - http://php.net/manual/en/migration71.deprecated.php, The MCrypt extension has not been maintained for a long time, and is actually removed from the core distribution to PECL in 7.2.
Apart from installing PECL packages not being achievable in many hosting environments, use of an abandonware encryption extension is not great from a confidence perspective. Parts of the system that are currently dependent on MCrypt ought to be retooled to use OpenSSL functions, per the PHP project's recommendations.https://lab.civicrm.org/dev/core/-/issues/3534Simplify the settings for filesystem paths2022-06-11T16:35:32ZxurizaemonSimplify the settings for filesystem paths* [Hosting environments like WPEngine don't permit (extra) .php files located in the webroot (apparently)](https://civicrm.stackexchange.com/questions/15942/is-it-possible-in-civicrm-to-change-extension-of-cached-php-files-from-php-to-s)...* [Hosting environments like WPEngine don't permit (extra) .php files located in the webroot (apparently)](https://civicrm.stackexchange.com/questions/15942/is-it-possible-in-civicrm-to-change-extension-of-cached-php-files-from-php-to-s)
* Caching generated content there means potential for info leakage or execution via input variables
* Filesystem caching can be inefficient compared to other caches
* Herb has done work to get Smarty caching via Redis?
* https://www.drupal.org/node/2570335
* https://forum.civicrm.org/index.php?topic=35125.0;nowap
* https://issues.civicrm.org/jira/browse/CRM-17401
* https://github.com/freeform/civicrm-drupal-pantheon/blob/4.6.x/patches/smarty-redis-civi-cache.patch
* would Smarty 2 choke if we just set the path to redis:// ?
* .php extension is [tacked on here](https://github.com/civicrm/civicrm-packages/blob/master/Smarty/Smarty.class.php#L1512), may be other places to change thishttps://lab.civicrm.org/dev/drupal/-/issues/167Drupal service 'site.path' is deprecated in drupal 9 and will be removed in d...2022-06-11T19:26:30ZDaveDDrupal service 'site.path' is deprecated in drupal 9 and will be removed in drupal 10In https://github.com/civicrm/civicrm-core/blob/master/setup/src/Setup/DrupalUtil.php#L23 it calls `\Drupal::service('site.path');`.
But I'm wondering why that function getDrupalSiteDir() even exists and why it's not part of CRM_Utils_S...In https://github.com/civicrm/civicrm-core/blob/master/setup/src/Setup/DrupalUtil.php#L23 it calls `\Drupal::service('site.path');`.
But I'm wondering why that function getDrupalSiteDir() even exists and why it's not part of CRM_Utils_System_X. Maybe a timing issue since civi isn't installed yet at that point?
Anyway the relevant drupal issue is https://www.drupal.org/project/drupal/issues/3074585#comment-13220446.
Because of the (bleep) `@trigger_error` usage that drupal uses though it only comes up when running unit tests in drupal's own testing environment (until drupal 10 when it will hard fail, effectively nulling the effect of outputting a deprecation for most people).5.52.0https://lab.civicrm.org/dev/core/-/issues/2325Activity import ignores time component of activity_date_time if the import fi...2022-06-11T19:27:55ZDaveDActivity import ignores time component of activity_date_time if the import file includes seconds and always ignores time for custom date fieldse.g. for activity_date_tinme 2021-01-02 10:11 imports correctly, but 2021-01-02 10:11:12 imports as 2021-01-02 00:00.
In master but also goes back to at least 5.24.
For custom fields of type date it seems to always ignore time.e.g. for activity_date_tinme 2021-01-02 10:11 imports correctly, but 2021-01-02 10:11:12 imports as 2021-01-02 00:00.
In master but also goes back to at least 5.24.
For custom fields of type date it seems to always ignore time.5.51.0https://lab.civicrm.org/dev/core/-/issues/35125.51beta1: TypeError on CRM_Contact_Import_Parser_Contact::checkStatesForCoun...2022-06-11T20:27:42ZJonGold5.51beta1: TypeError on CRM_Contact_Import_Parser_Contact::checkStatesForCountry()Another TypeError, this time on `CRM_Contact_Import_Parser_Contact::checkStatesForCountry()`
Sidenote - I'm hoping standalone CiviCRM comes to fruition because it would greatly simplify the process of using static analysis (e.g. PhpStan...Another TypeError, this time on `CRM_Contact_Import_Parser_Contact::checkStatesForCountry()`
Sidenote - I'm hoping standalone CiviCRM comes to fruition because it would greatly simplify the process of using static analysis (e.g. PhpStan) on Civi, which would catch type errors.
```
811744 10/Jun 11:23 php Error TypeError: Argument 2 passed to CRM_Contact_Import_Parser_Contact::checkStatesForCountry() must be of the type array, null given, called in
/home/jon/local/mysite/web/vendor/civicrm/civicrm-core/CRM/Contact/Import/Parser/Contact.php on line 1794 in CRM_Contact_Import_Parser_Contact->checkStatesForCountry() (line 1854 of
/home/jon/local/mysite/web/vendor/civicrm/civicrm-core/CRM/Contact/Import/Parser/Contact.php) #0
/home/jon/local/mysite/web/vendor/civicrm/civicrm-core/CRM/Contact/Import/Parser/Contact.php(1794): CRM_Contact_Import_Parser_Contact->checkStatesForCountry(1228, NULL)
#1 /home/jon/local/mysite/web/vendor/civicrm/civicrm-core/CRM/Contact/Import/Parser/Contact.php(1828): CRM_Contact_Import_Parser_Contact->tryToResolveStateProvince('invalid_import_...', '')
#2 /home/jon/local/mysite/web/vendor/civicrm/civicrm-core/CRM/Contact/Import/Parser/Contact.php(1449): CRM_Contact_Import_Parser_Contact->fillStateProvince(Array)
#3 /home/jon/local/mysite/web/vendor/civicrm/civicrm-core/CRM/Import/Parser.php(1665): CRM_Contact_Import_Parser_Contact->getMappedRow(Array)
#4 /home/jon/local/mysite/web/vendor/civicrm/civicrm-core/CRM/Import/Parser.php(1645): CRM_Import_Parser->validateValues(Array)
#5 /home/jon/local/mysite/web/vendor/civicrm/civicrm-core/CRM/Contact/Import/Form/MapField.php(450): CRM_Import_Parser->validate()
#6 /home/jon/local/mysite/web/vendor/civicrm/civicrm-core/CRM/Contact/Import/Form/MapField.php(372): CRM_Contact_Import_Form_MapField->submit(Array)
#7 /home/jon/local/mysite/web/vendor/civicrm/civicrm-core/CRM/Core/Form.php(573): CRM_Contact_Import_Form_MapField->postProcess()
#8 /home/jon/local/mysite/web/vendor/civicrm/civicrm-core/CRM/Core/StateMachine.php(144): CRM_Core_Form->mainProcess()
#9 /home/jon/local/mysite/web/vendor/civicrm/civicrm-core/CRM/Core/QuickForm/Action/Next.php(43): CRM_Core_StateMachine->perform(Object(CRM_Contact_Import_Form_MapField), 'next', 'Next')
#10 /home/jon/local/mysite/web/vendor/civicrm/civicrm-packages/HTML/QuickForm/Controller.php(203): CRM_Core_QuickForm_Action_Next->perform(Object(CRM_Contact_Import_Form_MapField), 'next')
#11 /home/jon/local/mysite/web/vendor/civicrm/civicrm-packages/HTML/QuickForm/Page.php(103): HTML_QuickForm_Controller->handle(Object(CRM_Contact_Import_Form_MapField), 'next')
#12 /home/jon/local/mysite/web/vendor/civicrm/civicrm-core/CRM/Core/Controller.php(355): HTML_QuickForm_Page->handle('next')
#13 /home/jon/local/mysite/web/vendor/civicrm/civicrm-core/CRM/Core/Invoke.php(319): CRM_Core_Controller->run(Array, NULL)
#14 /home/jon/local/mysite/web/vendor/civicrm/civicrm-core/CRM/Core/Invoke.php(69): CRM_Core_Invoke::runItem(Array)
#15 /home/jon/local/mysite/web/vendor/civicrm/civicrm-core/CRM/Core/Invoke.php(36): CRM_Core_Invoke::_invoke(Array)
#16 /home/jon/local/mysite/web/modules/contrib/civicrm/src/Civicrm.php(88): CRM_Core_Invoke::invoke(Array)
#17 /home/jon/local/mysite/web/modules/contrib/civicrm/src/Controller/CivicrmController.php(80): Drupal\civicrm\Civicrm->invoke(Array)
#18 [internal function]: Drupal\civicrm\Controller\CivicrmController->main(Array, '')
#19 /home/jon/local/mysite/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(123): call_user_func_array(Array, Array)
#20 /home/jon/local/mysite/web/core/lib/Drupal/Core/Render/Renderer.php(564): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}()
#21 /home/jon/local/mysite/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(124):
Drupal\Core\Render\Renderer->executeInRenderContext(Object(Drupal\Core\Render\RenderContext), Object(Closure))
#22 /home/jon/local/mysite/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(97):
Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->wrapControllerExecutionInRenderContext(Array, Array)
#23 /home/jon/local/mysite/web/vendor/symfony/http-kernel/HttpKernel.php(158): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}()
#24 /home/jon/local/mysite/web/vendor/symfony/http-kernel/HttpKernel.php(80): Symfony\Component\HttpKernel\HttpKernel->handleRaw(Object(Symfony\Component\HttpFoundation\Request), 1)
#25 /home/jon/local/mysite/web/core/lib/Drupal/Core/StackMiddleware/Session.php(58): Symfony\Component\HttpKernel\HttpKernel->handle(Object(Symfony\Component\HttpFoundation\Request), 1,
true)
#26 /home/jon/local/mysite/web/core/lib/Drupal/Core/StackMiddleware/KernelPreHandle.php(48): Drupal\Core\StackMiddleware\Session->handle(Object(Symfony\Component\HttpFoundation\Request), 1,
true)
#27 /home/jon/local/mysite/web/core/modules/page_cache/src/StackMiddleware/PageCache.php(106):
Drupal\Core\StackMiddleware\KernelPreHandle->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#28 /home/jon/local/mysite/web/core/modules/page_cache/src/StackMiddleware/PageCache.php(85):
Drupal\page_cache\StackMiddleware\PageCache->pass(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#29 /home/jon/local/mysite/web/core/lib/Drupal/Core/StackMiddleware/ReverseProxyMiddleware.php(48):
Drupal\page_cache\StackMiddleware\PageCache->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#30 /home/jon/local/mysite/web/core/lib/Drupal/Core/StackMiddleware/NegotiationMiddleware.php(51):
Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#31 /home/jon/local/mysite/web/vendor/stack/builder/src/Stack/StackedHttpKernel.php(23):
Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#32 /home/jon/local/mysite/web/core/lib/Drupal/Core/DrupalKernel.php(708): Stack\StackedHttpKernel->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#33 /home/jon/local/mysite/web/index.php(19): Drupal\Core\DrupalKernel->handle(Object(Symfony\Component\HttpFoundation\Request))
#34 {main}.
```5.51.0https://lab.civicrm.org/dev/core/-/issues/2345User improvement, CiviCRM Participant Import, users can be misled into thinki...2022-06-11T20:45:45Zjustinfreeman (Agileware)User improvement, CiviCRM Participant Import, users can be misled into thinking Participant Status ID field is mandatory because it is shown in red with asteriskAs reported by CiviCRM users. When using the CiviCRM Participant Import, users can be misled into thinking Participant Status ID field is mandatory because it is shown in red with asterisk. However, the requirement is for either Particip...As reported by CiviCRM users. When using the CiviCRM Participant Import, users can be misled into thinking Participant Status ID field is mandatory because it is shown in red with asterisk. However, the requirement is for either Participant Status OR Participant Status ID to be present in the import.
Suggested user improvement is to do the following:
1. Not use RED as the colour or asterisks in the drop-down options as this denotes mandatory field in other parts of the CRM, eg. RED asterisk. Recommend using bold and black text, no asterisk.
2. Provide user help note / explanatory text on the page to detail the mandatory fields required for the import.
We can submit a PR for this user improvement, just posting here for feedback and other suggestions.
![Screenshot_20210201_100120](/uploads/14bb0d1b3b5dba8931c2530b1328bd3f/Screenshot_20210201_100120.png)
Agileware Ref: CIVICRM-16555.51.0justinfreeman (Agileware)justinfreeman (Agileware)https://lab.civicrm.org/dev/core/-/issues/2858Importing soft credit payments by email address fails with confusing message ...2022-06-11T23:42:51ZjhungerfordImporting soft credit payments by email address fails with confusing message if dedupe rule requires extra fieldsOverview
----------------------------------------
Importing soft credit payments (matched to existing contacts by email address) fails if the relevant Unsupervised dedupe rule requires any extra fields. The error message in this case is:...Overview
----------------------------------------
Importing soft credit payments (matched to existing contacts by email address) fails if the relevant Unsupervised dedupe rule requires any extra fields. The error message in this case is:
`Invalid email address(doesn't exist) email@example.com. Row was skipped`
The proposed change passes any required fields to the dedupe check if they are available in the import, but does not amend the error message if it fails.
Example use-case
----------------------------------------
- Add to the Unsupervised rule so it requires First and Last names as well as email address
- Import soft credit payments where the Fundraiser ID is set by External Identifier (or Contact ID) and the Donor is matched by email, First Name, and Last Name
Current behaviour
----------------------------------------
Currently each row in such an import will fail with this message:
`Invalid email address(doesn't exist) email@example.com. Row was skipped`
Proposed behaviour
----------------------------------------
If the required fields are available in the import data, they could be passed to the dedupe function. This is what I've done here (on version 5.39.2):
https://github.com/AsylumSeekersCentre/civicrm-core/commit/c924fb95a2e311a442656bf65c23523091d351da
If the required fields are not available, it would be better if the error message pointed the user in the right direction. Currently it says that the email address does not exist, but in the case of our import, all of the email addresses did exist for contacts of the appropriate type. I haven't addressed this part of the issue with the code above.
Comments
----------------------------------------
I think the patch above is not the whole solution. That code pathway is used when there is no External Identifier or Contact ID but there is an email address. It then calls the Unsupervised dedupe rule, which may not refer to the email address at all (though the default does). The change I've proposed accommodates extra fields added to the rule, but doesn't consider the possible removal of the email address from the rule.5.51.0https://lab.civicrm.org/dev/core/-/issues/3135Regression - dedupe redirects to wrong place in 5.48rc2022-06-12T00:47:07ZeileenRegression - dedupe redirects to wrong place in 5.48rcWith [this commit](https://github.com/civicrm/civicrm-core/pull/22768/commits/a6f2a80ffbddfc0788199ec824b8aff46ccd1542) the user is redirected to the wrong place after merging 2 contacts
The correct flow is that
1) user does a search f...With [this commit](https://github.com/civicrm/civicrm-core/pull/22768/commits/a6f2a80ffbddfc0788199ec824b8aff46ccd1542) the user is redirected to the wrong place after merging 2 contacts
The correct flow is that
1) user does a search from basic search, advanced search etc
2) user selects 2 contacts & chooses the action to merge them
3) user submits the screen to merge them
4) user redirected to the single resulting contact
The regression is that in 4 the user is redirected to the search results again. This has been reported as a regression/ problematic change by users
@colemanw this is the regression I commented in github5.48.0