CiviCRM Core issueshttps://lab.civicrm.org/dev/core/-/issues2021-03-27T15:11:01Zhttps://lab.civicrm.org/dev/core/-/issues/1750View sent emails in activities2021-03-27T15:11:01ZshitijgView sent emails in activitiesOverview
----------------------------------------
As it stands currently, a CiviCRM user can send an email to a contact via a few methods. The user can also use tokens which personalise the content per contact they are sending an email t...Overview
----------------------------------------
As it stands currently, a CiviCRM user can send an email to a contact via a few methods. The user can also use tokens which personalise the content per contact they are sending an email too.
On submission, an activity is created for this. However, the activity only stores the tokens and not the actual content sent to a contact.
The behaviour for how the content of the email when we use the “send email” activity functionality is different to that from mass mailings with respect to tokens, and any used tokens aren’t replaced in the resultant stored body of the content in the activity with the actual content.
To see the content of the sent email (rather than just “token” itself) is important to users as
- They should be able to see what was sent to a contact at any given time and not just the tokens
- information might change with time i.e. the data in the field may change so the user will not know what was actually sent to the contact
Current behaviour
----------------------------------------
A mailing can be sent to the user in multiple ways in CiviCRM eg. bulk mailings, through send email activity on the contact record, inside a case, scheduled reminders, and through extensions like CiviRules.
However, when we use the send email activity form, any used tokens aren’t replaced in the resultant stored body of the content in the activity eg
![image1](/uploads/7419b37d976e1603ae2ad05d1c8619f2/image1.png)
This is happening due to an issue in the following create email activity functionality:
https://github.com/civicrm/civicrm-core/blob/master/CRM/Contact/Form/Task/EmailCommon.php#L467-L483
This is used:
1. When creating an activity from the contact record: This can be done by clicking on the ‘Send email’ action which triggers the form above
2. From within a case: The same form is used when creating a ‘Send email’ activity from within a case
Proposed behaviour
----------------------------------------
The created activity should store (in the content) the "resultant value" of any "tokens" with the content they were actually sent as to the recipient rather than just the {token} field name itself.
Technical implementation suggestions:
- The email activity is created before the token replacements is done here: https://github.com/civicrm/civicrm-core/blob/master/CRM/Activity/BAO/Activity.php#L1122.
- Token replacements are done here: https://github.com/civicrm/civicrm-core/blob/master/CRM/Activity/BAO/Activity.php#L1193-L1216
- Change will be to move the activity creation to just before the email is sent, after token replacements are done to somewhere like here: https://github.com/civicrm/civicrm-core/blob/master/CRM/Activity/BAO/Activity.php#L1227
with the following code:
$activityID = self::createEmailActivity($userID, $tokenSubject, $tokenHtml, $tokenText, $additionalDetails, $campaignId, $attachments);
We have tested this on our local and are happy to submit a core PR if the concept is approved.5.36.0https://lab.civicrm.org/dev/core/-/issues/3753Proposal: Allow negative rules for ACLs2023-09-08T10:52:34ZTobias Voigttobias.voigt@civiservice.deProposal: Allow negative rules for ACLs**PROBLEM:**
When defining ACL rules, it is only possible to 'allow' certain behaviour but not to 'disallow' it. This makes it tedious to define a set of rules that restricts access to e.g. groups of contacts or sets of custom fields - ...**PROBLEM:**
When defining ACL rules, it is only possible to 'allow' certain behaviour but not to 'disallow' it. This makes it tedious to define a set of rules that restricts access to e.g. groups of contacts or sets of custom fields - especially if you have many (groups or sets of custom fields).
**USE CASE:**
In one of our client's system, we have many groups and many sets of custom fields. What I want to achieve is:
1. define a privileged group / role that has exclusive access to **one** set of custom fields
2. allow access to all other sets of custom fields for all users
3. prevent 'normal' users from giving themselves access to the exclusive information by adding themselves to the privileged group
Because I can only define 'positive' ACL rules, here's what I have to do:
- turn off the WP permission for custom fields for the relevant roles (to be able to define my rules via CiviCRM ACLs)
- allow access to all sets of custom fields for 'everyone' (excluding the one I want to restrict access for) - **by creating a rule for each set of custom fields**
- define a rule to allow access to the restricted set of custom fields to the privileged role
Depending on the number of sets of custom fields, this can be a tedious process. Not only that - if I create a new set of custom fields at a later time, I have to remember to create a new ACL rule for that as well.
Now for the third task (preventing users from adding themselves to the privileged group). To achive this I furthermore have to:
- turn off the WP permissions for viewing and editing all contacts for the relevant roles
- allow access to all groups for 'everyone' (excluding the one I want to restrict access for) - **by creating a rule for each contact group**
- define a rule to allow access to the restricted contact group only for the privileged role
Again: With about 50+ groups, that's a tedious task. And again: I have to remember to create a new ACL rule each time I add a new group.
All in all I could end up with a multitude of rules (in our case 70+) that are really hard to maintain.
**PROPOSAL:**
This task could be so much easier, if it was possible to define 'negative' ACL rules. If this was possible, I could:
- turn off the relevant WP permissions
- allow access to all sets of custom fields for everyone (1 rule)
- allow access to all contact groups for everyone (1 rule)
- disallow access to the restricted set of custom fields for everyone (1 rule)
- disallow access to the restricted contact group for everyone (1 rule)
- allow access to the restricted set of custom fields for the privileged role (1 rule)
- allow access to the restricted contact group for the privileged role (1 rule)
As is implied above, **there would have to be some form of priorisation** of rules to make this work.
This way I would end up with only 6 rules and would not have to remember to create new rules whenever I create a new set of custom fields or a new contact group.5.64.0https://lab.civicrm.org/dev/core/-/issues/3835Proposal: Deprecate profile standalone listing mode2023-11-30T17:03:40ZJonGoldProposal: Deprecate profile standalone listing modeProfile standalone listing code is deeply entangled with the query BAOs, and produces weird and wonderful bugs like #3834.
I don't think they're widely used in modern CiviCRM, they seem to have been a stopgap before Views integration, a...Profile standalone listing code is deeply entangled with the query BAOs, and produces weird and wonderful bugs like #3834.
I don't think they're widely used in modern CiviCRM, they seem to have been a stopgap before Views integration, and later for non-Drupal users to get a Views-esque experience. But Search Kit has effectively replaced them - they have feature parity and then some.
I think we can improve the query code if we no longer needed to worry about standalone listing mode.
Proposal:
* Remove the ability to create standalone listings on new installations of CiviCRM.
* Create a check for profiles using standalone listing mode, warning admins if they use it.
We can deprecate this over a long period of time, so the impact should be gentle.
If this proposal is approved, we may also want to consider a function that converts profile listings to SK searches. This will have added utility as we deprecate other uses of profiles (e.g. once SK replaces Advanced Search, Search Profiles will go away).https://lab.civicrm.org/dev/core/-/issues/2640User experience improvement, hide by default the Latitude and Longitude fiel...2023-08-23T00:28:55Zjustinfreeman (Agileware)User experience improvement, hide by default the Latitude and Longitude fields, users can enable display if neededWe have found that users are somewhat confused by the default display of the Latitude and Longitude fields in the Address section for Contacts. This is a proposal to change the default so that the Latitude and Longitude fields are not di...We have found that users are somewhat confused by the default display of the Latitude and Longitude fields in the Address section for Contacts. This is a proposal to change the default so that the Latitude and Longitude fields are not displayed by default. This would affect both new sites and existing sites that rely on the default settings and require an upgrade message to notify existing sites of this change.
The change is to de-select the Latitude and Longitude fields in the Address Settings. CiviCRM > Administer CiviCRM > Address Settings
![Screenshot_20210608_091407](/uploads/ca7ad12a7aa0e9472d92c40b41b6f485/Screenshot_20210608_091407.png)
This change was initially discussed in MM and support indicated, see https://chat.civicrm.org/civicrm/channels/product-roadmap/pzm9gqjy4jdeznennqi6kuhceo
Agileware will provide a PR at some stage.
Agileware Ref: CIVIUX-1415.66.0https://lab.civicrm.org/dev/core/-/issues/2243Add created_date column to the civicrm_note table2021-03-28T16:44:34ZjasonmAdd created_date column to the civicrm_note tableOverview
----------------------------------------
After merging contacts, we noticed that when viewing the notes of the merged contact, the notes merged from the duplicated contact display the timestamp of when the merge happened. It wou...Overview
----------------------------------------
After merging contacts, we noticed that when viewing the notes of the merged contact, the notes merged from the duplicated contact display the timestamp of when the merge happened. It would be helpful for note records to store both the created date and the last modified date and to display both dates in the notes list view at /civicrm/contact/view?reset=1&cid=xxxxxx
Example use-case
----------------------------------------
The user wants to see the date that a note was created in addition to when a note was last modified.
Current behaviour
----------------------------------------
Currently the civicrm_note table has only a modified_date field so the date a note is created is lost upon any update.
Proposed behaviour
----------------------------------------
Add a created_date column to the civicrm_note table and display both created and modified dates at /civicrm/contact/view?reset=1&cid=xxxxxx
![screenshot-a](/uploads/99cabcdde44fa725841c2d5de5afd28a/screenshot-a.png)
![screenshot-b](/uploads/003327183320ec3af790840cbe835393/screenshot-b.png)
![screenshot-c](/uploads/d8ecdc89bb06f23037df82ea81e4974e/screenshot-c.png)5.37.0https://lab.civicrm.org/dev/core/-/issues/2122Account for time zone on event registration pages2023-11-05T14:13:53ZwmortadaAccount for time zone on event registration pagesOverview
----------------------------------------
There doesn't appear to be a way to account for the time zone on an event registration page.
This is less of an issue for a local event, but is quite a problem for a global online event....Overview
----------------------------------------
There doesn't appear to be a way to account for the time zone on an event registration page.
This is less of an issue for a local event, but is quite a problem for a global online event.
At a minimum I would expect there to be an option to set the time zone for the event and for the time zone to display on the event registration page. (Ideally you'd want the page to show the time in the user's time zone.)
Current behaviour
----------------------------------------
There is no option to set the time zone for an event. (I presume that the system time zone is used but I'm not sure.)
![image](/uploads/6727ccb71e4b3129120a3275240c56cf/image.png)
The time zone isn't displayed on the event registration page.
![image](/uploads/a16d1c595ab9020e2d9e9c2d668e114c/image.png)
Proposed behaviour
----------------------------------------
There is an option to set the time zone when creating an event. This should default to the user's time zone.
The time zone is displayed on the event registration page.
Comments
----------------------------------------
I became aware of this when trying to register for this event: https://civicrm.org/civicrm/event/info?reset=1&id=1553
Probably relates to https://lab.civicrm.org/dev/core/-/issues/441
Agileware Ref: CIVICRM-404 (feature not found)https://lab.civicrm.org/dev/core/-/issues/392Support MySQL 8.0 now that it is GA2021-04-01T01:52:03ZJoeMurraySupport MySQL 8.0 now that it is GA# Overview
MySQL 8.0 went General Availability on April 19, 2018. CiviCRM's aim is to support the latest GA for required infrastructure (MySQL, PHP) as soon as feasible.
In reviewing removed features that cause incompatibilities (htt...# Overview
MySQL 8.0 went General Availability on April 19, 2018. CiviCRM's aim is to support the latest GA for required infrastructure (MySQL, PHP) as soon as feasible.
In reviewing removed features that cause incompatibilities (https://dev.mysql.com/doc/refman/8.0/en/upgrading-from-previous-series.html) I noticed a couple of issues that definitely require changes, one that might cause syntax errors that are easily fixed when found, and a fourth that we may want to change to improve our functionality and get current.
This is a meta issue for tracking what's needed for MySQL8. I have added a MySQL8 flag to lab for the moment to help in tracking. We can delete that flag after this issue is deemed complete.
# Related issue: Upgrade test infrastructure to enable testing against a version on the edge of being officially supported: https://lab.civicrm.org/dev/core/issues/1142
# Issue: Field Names now Reserved Words https://lab.civicrm.org/dev/core/issues/1143
# Testing issue: dependence on PHP version for MySQL authentication ~~https://lab.civicrm.org/dev/core/issues/1144~~
# Document dealing with changes to MySQL user password library https://lab.civicrm.org/dev/core/issues/1145
# Possible issue: Sort order on GROUP BY clauses
~~I don't think we have any code that tries to specify sort order on GROUP BY fields, but if so we will need to change to add an ORDER BY clause with the sort order. I think we should just try running tests on MySQL 8.0 and using it manually to determine locations in code with this issue, and play whack-a-mole on anything missed.~~
# Nice to have: Upgrade our text fields to support all utf8 characters https://lab.civicrm.org/dev/core/issues/339https://lab.civicrm.org/dev/core/-/issues/4899Modelling many to many relationships with Entity reference, SearchKit, FormBu...2024-01-17T18:18:18ZMichael McAndrewModelling many to many relationships with Entity reference, SearchKit, FormBuilder, ECK, etc.Creating this issue as a way to keep track of different bits of functionality that can be used together to model many to many relationships in CiviCRM as when you put this all together, I think it is quite a game changer for data modelli...Creating this issue as a way to keep track of different bits of functionality that can be used together to model many to many relationships in CiviCRM as when you put this all together, I think it is quite a game changer for data modelling in CiviCRM :grinning:
Might make sense to document this at some point soon, and it would be good to collect feedback on how people are finding this functionality, ideas for improvement, etc.
Also, if you have any budget that you would like to put towards this work: to improve it or build out more features, etc. please get in contact with @colemanw or me :heart:.
* Entity Reference fields - allows you to reference other entities in custom data fields effectively creating **one to many** relationships.
* Multivalue custom data sets - [now available to all entities](https://github.com/civicrm/civicrm-core/pull/27549) allowing you to model **many to many** relationships _with additional meta data_ about the relationship
* Searchkit support for [joining via EntityRef fields in multivalue custom data](https://github.com/civicrm/civicrm-core/pull/28721) - allows you to create searches that span many to many joins
* FormBuilder support - allowing for editing of many to many relationships in entities (could probably do with some improvement)
* [ECK](https://github.com/systopia/de.systopia.eck) which allows people to make arbitrary new entities that can be joined via these relationshipshttps://lab.civicrm.org/dev/core/-/issues/4539Very slow query when sending mailings via CiviMail2023-09-12T05:08:13ZwmortadaVery slow query when sending mailings via CiviMailOverview
----------------------------------------
We have come across an issue where the database connection was timing out due to slow, long-running queries when the Send Scheduled Mailings job was running. We became aware of this when...Overview
----------------------------------------
We have come across an issue where the database connection was timing out due to slow, long-running queries when the Send Scheduled Mailings job was running. We became aware of this when we set a maximum execution time limit of 3 minutes for MySQL queries. Mailings for one of our clients were not being sent because the database connection was timing out.
As a workaround, we have increased the maximum execution time to 10 minutes and reduced the Mailer Batch Limit and Mailer Job Size to 1,000. However, we'd like to fix the underlying issue that the queries are running so slowly.
When sending a mailing CiviCRM runs a query of the form:
```sql
SELECT r.contact_id, r.email_id, r.phone_id
FROM civicrm_mailing_recipients r
INNER JOIN civicrm_contact c on
(c.id = r.contact_id
AND c.is_deleted = 0
AND c.is_deceased = 0
AND c.do_not_email = 0
AND c.is_opt_out = 0
)
INNER JOIN civicrm_email e ON (r.email_id = e.id AND e.on_hold = 0)
WHERE r.mailing_id = 3271
LIMIT 18000, 1000
```
There are no indices for `is_deceased`, `do_not_email` or `is_opt_out` so this query is very slow. (There is an index for `is_deleted`. There should be an index for `is_deceased` but this appears to be missing from this site.)
We have tested adding these indices in a development environment to see what difference it would make and this increase the speed of the query by a factor of one thousand:
- 20,000 rows with no indices = 80 seconds
- 20,000 rows with indices = 0.085 seconds
Reproduction steps
----------------------------------------
This is likely to only affect sites with a large number of contacts and large number of rows in the `civicrm_mailing_recipients` table (in this case 46,000 contacts and 9 million rows respectively).
1. Set a maximum execution time for MySQL of 3 minutes (say)
2. Create a mailing to a large group of contacts (>20,000)
3. Schedule the mailing to send
4. Send Scheduled Mailings job fails with error: "Finished execution of Send Scheduled Mailings with result: Failure, Error message: DB Error: unknown error"
Current behaviour
----------------------------------------
No indices ~~`is_deceased`~~, `do_not_email` or `is_opt_out` in `civicrm_contact` so queries run very slowly.
Expected behaviour
----------------------------------------
Indices for the above fields so that the query runs quickly.
Environment information
----------------------------------------
* __CiviCRM:__ _5.62_
* __PHP:__ _8.1__
* __CMS:__ _N/A_
* __Database:__ _MySQL 5.7.43_
Comments
----------------------------------------
There are other speed improvements in #4045 but these relate to the speed of the user interface rather than the speed of sending emails.
Perhaps this can also help to [reduce our collective CO2 emissions](https://civicrm.org/blog/systopia/beginning-green-how-can-we-make-civicrm-more-sustainable)?https://lab.civicrm.org/dev/core/-/issues/4516Rethinking 20 minutes QF session timeouts2023-10-05T15:27:38ZJonGoldRethinking 20 minutes QF session timeoutsThe 20 minute timeout has been a part of Civi for as long as I can remember. It's also caused lost donations, frustrated applicants, and so on. Reading an [article about why short session timeouts aren't helpful](https://www.sjoerdlang...The 20 minute timeout has been a part of Civi for as long as I can remember. It's also caused lost donations, frustrated applicants, and so on. Reading an [article about why short session timeouts aren't helpful](https://www.sjoerdlangkemper.nl/2023/08/16/session-timeout/) made me think about this for Civi.
* What is the original reason? Is it still valid?
* If we lengthen the timeout, do we need to make special arrangements around event registration when there's a maximum number of seats available?https://lab.civicrm.org/dev/core/-/issues/3096Marking historical contact data - add on hold, hold date, reset date columns ...2023-01-06T22:02:03ZandyburnsMarking historical contact data - add on hold, hold date, reset date columns to civicrm_address and civicrm_phone to match civicrm_emailCurrently only civicrm_email has a mechanism to mark a bad email. This data model should be mirrored over to civicrm_address and civicrm_phone. We currently use the lightweight "location type" approach where we have an Invalid location t...Currently only civicrm_email has a mechanism to mark a bad email. This data model should be mirrored over to civicrm_address and civicrm_phone. We currently use the lightweight "location type" approach where we have an Invalid location type to designate a bad email/phone/address. However, it is limiting as you can only have 1 address per location type for a contact.
This was previously brought up here but was not pursued: https://issues.civicrm.org/jira/browse/CRM-13777.
The rationale is storing older data on these 3 entities helps prevent the creation of duplicates when bringing external sources of data (that may be older as well) upon import and therefore need to remain in the respective db table and not only sent to a log table.
The limitation of 1 address per location type would not apply to on hold addresses, therefore creating a history of addresses.
Setting to on hold would automatically set the hold date.
On hold entities would be shown in a "Former Communications Data" tab much like this extension (https://civicrm.org/extensions/former-communication-data) to prevent cluttering the Contact Summary Screen by only querying the entities that are on hold (and conversely, the contact's summary screen would only show those not on hold) and showing them in descending On Hold date order. Allow filtering by email/phone/address.
On hold entities would not be able to used throughout CiviCRM. E.g. a on hold mobile phone would not show Outbound SMS action, an on hold address would be added as a criteria to this checkbox upon export:
![image](/uploads/d481ef9470febc1620b895af2fb1baff/image.png)
There is obviously overlap with what detailed logging does and this so I'd greatly appreciate anyone's thoughts on how to approach / refine this feature request. We are willing to fund a solution.
Ref:
- https://civicrm.stackexchange.com/questions/681/what-are-best-practices-for-bad-postal-addresses
- https://civicrm.stackexchange.com/questions/20584/need-historical-address-informationhttps://lab.civicrm.org/dev/core/-/issues/2994Support oEmbed for external facing pages2024-03-21T18:50:44ZJoeMurraySupport oEmbed for external facing pagesAn important evolution of CiviCRM is to better support remote sites exposing CiviCRM forms and content.
Many platforms including WordPress and Drupal core since 8.6 support oEmbed, which allows a site to pull in content from a different...An important evolution of CiviCRM is to better support remote sites exposing CiviCRM forms and content.
Many platforms including WordPress and Drupal core since 8.6 support oEmbed, which allows a site to pull in content from a different site that exposes its content as oEmbed.
There is a good list of oEmbed providers, and CiviCRM should aspire to join it: https://oembed.com/providers.json
We should expose the following 'pages' as oEmbed content, using appropriate standard markup for use in WordPress and Drupal 8.6+ sites, and support the workflow for confirmation, thank you, and search results as appropriate:
1. Phase 1
1. Contribution page
2. Event info page
3. Event registration page
2. Phase 2
1. Something like a profile create/edit page that does not involve a payment but allows submission of a form to create a contact and activity and perhaps more (eg parent-child relationship)
2. Search Kit page
3. Petition page
3. Phase 3
1. Personal Campaign Page
2. Pages from extensions, eg Grant Application Page
3. CiviSurvey pages
Note: I haven't done the research to understand the details of oEmbed and potential challenges. I think this could be done through extension(s).
I'm posting this in order to generate discussion for a possible Roadmap item.https://lab.civicrm.org/dev/core/-/issues/2765Revisit the decision to enforce the CMS new user account option, allow CiviCR...2023-09-26T05:03:25Zjustinfreeman (Agileware)Revisit the decision to enforce the CMS new user account option, allow CiviCRM to create CMS user accounts when CMS has public registrations disabledRaising this a request to revisit the decision [originating in 2009 (CRM-4036)](https://issues.civicrm.org/jira/browse/CRM-4036.html) to enforce the CMS new user account option and instead **allow CiviCRM to create CMS user accounts when...Raising this a request to revisit the decision [originating in 2009 (CRM-4036)](https://issues.civicrm.org/jira/browse/CRM-4036.html) to enforce the CMS new user account option and instead **allow CiviCRM to create CMS user accounts when CMS has public registrations disabled**.
There are three key reasons to have this functionality:
1. Public registrations on websites attract spamming and therefore require moderation and anti-spam systems to be in place.
2. For member-only websites, the method to because a member is using the membership sign-up form (CiviCRM) and when the membership fee is paid, that's when the corresponding CMS user account should be created. And not by any other means.
3. With user account creation disabled, a separate and in some cases manual process needs to be performed to create the corresponding user account for members - which is a waste of effort.
With the current logic where CiviCRM uses the CMS configuration, the site must be open for public user registrations to support CiviCRM registering a member user account.
The change is relatively simple and would be removing the check of the user registration option for each supported CMS. Thus CiviCRM Profiles can be used to create CMS accounts.
See also related, _User Profiles, the User account registration option is displayed even when the CMS has the "User can register" option disabled_ - https://lab.civicrm.org/dev/user-interface/-/issues/41
Agileware Ref: CIVICRM-1809https://lab.civicrm.org/dev/core/-/issues/2241Don't check for .git in the isDevelopment() function2021-01-11T22:35:28ZDaveDDon't check for .git in the isDevelopment() functionThe function isDevelopment() checks if there's a .git folder present. The function controls for example mysql strict mode and whether deprecations throw errors (more about deprecations in ticket dev/core#2240). But it's not crazy for a l...The function isDevelopment() checks if there's a .git folder present. The function controls for example mysql strict mode and whether deprecations throw errors (more about deprecations in ticket dev/core#2240). But it's not crazy for a live site to have a .git folder.
Previous discussion at https://github.com/civicrm/civicrm-core/pull/17276. Where it stalled was not really on whether using .git is wrong, it was on what it should do instead. Summary of alternates:
1. Use the Environment setting at System Settings - Debugging.
* There may be some differences between what that setting is actually for and what this function gets used for.
* Also switching to this setting has some side effects which aren't blockers but would need to be dealt with (test suite, buildkit defaults).
1. Instead of "is dev", have a way to tell "for testing" and "not for testing".
* CIVICRM_UF(?)
1. Use whatever the symfony environment is set to.
@mattwire @AlanDixon @MikeyMJCO @artfulrobot5.35.0https://lab.civicrm.org/dev/core/-/issues/4909Split 'Edit all contacts' permission2024-03-06T10:57:35ZGuillaumeSorelSplit 'Edit all contacts' permissionI jumped into a user case where some admin should be able to edit contacts they're allowed to manage but shouldn't be able to manage tags. After looking inside the documentation, stackexchange and the ACL I discovered that the 'Edit all ...I jumped into a user case where some admin should be able to edit contacts they're allowed to manage but shouldn't be able to manage tags. After looking inside the documentation, stackexchange and the ACL I discovered that the 'Edit all contacts' embeds several sub-permissions like editing the contacts information and also tags or groups. It's everything or nothing within this permission.
I'm not talking about the capability to manage Tags as entity (which as a separate permission) but to manage the use of them.
I think it could be interesting to split this permission with a 'edit all contacts' on the one hand and create a new 'edit contact tags / groups' on the other hand.https://lab.civicrm.org/dev/core/-/issues/4350Proposal: Smart Group Profiler to help identify slow smart groups2023-11-02T14:29:14ZlarsssandergreenProposal: Smart Group Profiler to help identify slow smart groupsSlow smart groups are a problem that I'm sure we've all experienced. It's difficult for users to guess which of their potentially many smart groups is exceptionally slow or if they just have too many, each taking a little time. @JonGold ...Slow smart groups are a problem that I'm sure we've all experienced. It's difficult for users to guess which of their potentially many smart groups is exceptionally slow or if they just have too many, each taking a little time. @JonGold has a [profiling script](https://gist.github.com/MegaphoneJon/59089c1e155645a438af41051bca7ce7), but that's not accessible to many. I'm sure many orgs would be happy to cut or re-work their slow smart groups if it meant they could use the listing accordion on individual contacts, etc — but they just don't know which ones they need to cut!
What I propose is a Smart Group Profiler in Admin that lets admin users easily check their smart groups in the UI. Here's a quick sketch:
- The user can select groups to profile or leave the select empty to profile all (active, non-hidden) smart groups
- The user can select to run once or average up to three times, to reduce the potential for random variation
- The profiler queues CRM_Contact_BAO_GroupContactCache::add for each group, in random order
- Once the queue finishes, the user gets a table with average time and links to edit the Smart Group criteria for each
I'd be happy to do this as an extension, but it also seems like something that address a core problem, so I think it makes sense to have in core.5.67.0https://lab.civicrm.org/dev/core/-/issues/3648SMS limit is hardcoded at 460 but should be changeable somewhere2024-01-29T09:44:22ZjasonobrownSMS limit is hardcoded at 460 but should be changeable somewhereTwillio (and probably other sms providers) allows SMS messages up to 1600 characters, but Civi has the limit hardcoded at 460. I've verified that by simply going to CRM/SMS/Provider.php and changing MAX_SMS_CHAR from 460 to 1600 the sys...Twillio (and probably other sms providers) allows SMS messages up to 1600 characters, but Civi has the limit hardcoded at 460. I've verified that by simply going to CRM/SMS/Provider.php and changing MAX_SMS_CHAR from 460 to 1600 the system now accepts (and sends) messages up to 1600 characters. It seems logical that the character limit should able to be adjusted and stored somewhere rather than set as a static value in the code. We are currently using the larger limit, and would really like to see this implemented asap so that we can stop manually updating the code each time we update civi.https://lab.civicrm.org/dev/core/-/issues/3628Add pre/post hook for CRM_Mailing_BAO_MailingJob2022-06-15T11:19:25ZMonish DebAdd pre/post hook for CRM_Mailing_BAO_MailingJobThis ticket is about adding pre, post hook to MailingJob and it involve:
1. Call the pre/post hook inside `CRM_Mailing_BAO_MailingJob::create()`
2. Add `CRM_Mailing_BAO_MailingJob::deleteMailingJob` to delete Mailing Jobs and add pre/pos...This ticket is about adding pre, post hook to MailingJob and it involve:
1. Call the pre/post hook inside `CRM_Mailing_BAO_MailingJob::create()`
2. Add `CRM_Mailing_BAO_MailingJob::deleteMailingJob` to delete Mailing Jobs and add pre/post delete hook
3. Replace all the DAO call with corresponding create and delete fn in the codebase.Monish DebMonish Debhttps://lab.civicrm.org/dev/core/-/issues/3180Import contributions, memberships etc - usefulness of contact type...2023-01-31T04:38:32ZeileenImport contributions, memberships etc - usefulness of contact type...When importing contributions, memberships etc there is an option to select the contact type.
On digging into the code I think it only makes sense to specify contact type if
1) you are matching by email and
2) you specifically want to on...When importing contributions, memberships etc there is an option to select the contact type.
On digging into the code I think it only makes sense to specify contact type if
1) you are matching by email and
2) you specifically want to only import to a contact of a particular type.
If you are giving the contact_id or external_identifier it makes sense that we make to whatever type we find (this also saves from having to do multiple uploads if there is a mix of types).
However, I can see that with email it makes sense that there are both organizations and individuals with that email and you want it to choose the organization.
I propose that we make the field optional and ONLY validate for emails (if set)
![image](/uploads/67ada1ec273c176a6f974cab6d846bf9/image.png)https://lab.civicrm.org/dev/core/-/issues/3014Proposal: Explore adding hooks to allow plugins to track database queries, e....2023-04-23T20:13:36ZBradley TaylorProposal: Explore adding hooks to allow plugins to track database queries, e.g. to enable integration with plugins like Query MonitorOne of my favourite plugins in the WordPress space is [Query Monitor](https://wordpress.org/plugins/query-monitor/). Among other things it allows you to monitor at a glance what database calls have been made for a specific request, makin...One of my favourite plugins in the WordPress space is [Query Monitor](https://wordpress.org/plugins/query-monitor/). Among other things it allows you to monitor at a glance what database calls have been made for a specific request, making it easy to spot inefiencient code and queries.
Currently CiviCRM does not have anything analogous to this. The [CiviCRM docs recommend using the MySQL query log](https://docs.civicrm.org/dev/en/latest/tools/debugging/#viewing-a-query-log-from-mysql), but this isn't easy to view at a glance, and it doesn't make it easy to correlate queries to sepecific web requests. CiviCRM's database queries don't show in Query Monitor because CiviCRM does not use WordPress' `WPDB` class. Therefore, I've been looking to see if it would be possible to extend Query Monitor to know about CiviCRM's database queries. Its reasonably easy to make this work if you patch the `CRM_Core_DAO::query` method to the following:
```
public function query($query, $i18nRewrite = TRUE) {
// rewrite queries that should use $dbLocale-based views for multi-language installs
global $wpdb, $dbLocale, $_DB_DATAOBJECT;
if ( defined( 'SAVEQUERIES' ) && SAVEQUERIES ) {
$wpdb->timer_start();
}
if (empty($_DB_DATAOBJECT['CONNECTIONS'][$this->_database_dsn_md5])) {
// Will force connection to be populated per CRM-20541.
new CRM_Core_DAO();
}
$conn = &$_DB_DATAOBJECT['CONNECTIONS'][$this->_database_dsn_md5];
$orig_options = $conn->options;
$this->_setDBOptions($this->_options);
if ($i18nRewrite and $dbLocale) {
$query = CRM_Core_I18n_Schema::rewriteQuery($query);
}
$ret = parent::query($query);
if ( defined( 'SAVEQUERIES' ) && SAVEQUERIES ) {
$wpdb->num_queries++;
$wpdb->queries[] = array(
$query,
$wpdb->timer_stop(),
$wpdb->get_caller(),
$wpdb->time_start,
array(),
'trace' => new QM_Backtrace( array(
'ignore_frames' => 1,
)),
'result' => is_countable($ret) ? count($ret) : 1
);
}
$this->_setDBOptions($orig_options);
return $ret;
}
```
Esentially the `$wpdb->queries` array is being populated with knowledge of the CiviCRM database queries, which query monitor can then read.
With this in place you can start to see some interesting insights. For example, we can see that the new mailing screen contains a number of database queries that are reapeated 40 times each, which is probably ripe for optimisation:
![Screenshot_2022-01-02_at_14.13.41](/uploads/117036cbf7cb970e93c7399eeb436f64/Screenshot_2022-01-02_at_14.13.41.png)
The WordPress admin bar shows the total count of queries (243) combined from both CiviCRM and WordPress.
Hacking the core files works for my purposes, but obviously doesn't allow for this to be packaged into a distributable plugin. Therefore, I think it'd be cool if there were a generic hook at the end of `CRM_Core_DAO::query` which could be used to save the query information in a format ameanable to Query Monitor. Alternatively CiviCRM could copy WordPress idea of the `$wpdb->queries` array, creating an array of queries which plugins like Query Monitor could make use of (in WordPress this is behind a flag so there is no performance hit when not in a development environment).
I've not given too much thought to exactly how this hook should look, but before I take this further (e.g. to the Pull Request state) it'd be good to get some views from the community:
- Is this something others in the community would find useful if this were distributed as a plugin?
- Does anyone know of plugins similar to Query Monitor in the Drupal or Joomla space which may benifit from this type of solution?
- Would people find this sort of hook useful for other use cases - for example, for writing debugging tools native to CiviCRM.
Keen to get others views on this one. I'll be interested to know what people think.