CiviCRM Core issueshttps://lab.civicrm.org/dev/core/-/issues2018-11-13T03:11:11Zhttps://lab.civicrm.org/dev/core/-/issues/503Editing Rich Text custom field on relationship non-functional2018-11-13T03:11:11ZmclarkeEditing Rich Text custom field on relationship non-functional## Summary
Editing a Note/RichTextEditor custom field on a relationship causes the HTML tags to be encoded into the content. Each time you edit the field the tags get literally added to the content, piling up.
## Steps to Reproduce
- ad...## Summary
Editing a Note/RichTextEditor custom field on a relationship causes the HTML tags to be encoded into the content. Each time you edit the field the tags get literally added to the content, piling up.
## Steps to Reproduce
- add a custom field set on relationships
- add a custom field to the set of type Note->Rich Text Editor
- add a relevant relationship to a contact, put in any content to the rich field, save
- re-edit the relationship
- note that the paragraph tags are rendered into the rich editor as literals
- rinse, repeat to see the tags continue to pile up
[Screenshot_2018-11-02_14.15.10](/uploads/1e3e5bc6158cf32813ce73dade0b25a8/Screenshot_2018-11-02_14.15.10.png)5.9https://lab.civicrm.org/dev/core/-/issues/498Undefined index in mailing report for mailing_modified_date; and room for ref...2018-12-14T23:17:06ZJKingsnorthUndefined index in mailing report for mailing_modified_date; and room for refactoringNotice: Undefined property: CRM_Mailing_BAO_Mailing::$mailing_modified_date in CRM_Mailing_BAO_Mailing::report() (line 1821 of /home/developer/buildkit/build/civicrmdev/sites/all/modules/civicrm/CRM/Mailing/BAO/Mailing.php).
Appears on ...Notice: Undefined property: CRM_Mailing_BAO_Mailing::$mailing_modified_date in CRM_Mailing_BAO_Mailing::report() (line 1821 of /home/developer/buildkit/build/civicrmdev/sites/all/modules/civicrm/CRM/Mailing/BAO/Mailing.php).
Appears on pages like: /civicrm/mailing/report?mid=7&reset=1
This is because the DAO key for the field is 'mailing_modified_date' - but the name is 'modified_date'.
And the way the field values are assigned to the object does not account for this difference.
This is a very minor issue as the modified date doesn't actually get used in the report!
We should probably refactor the query/code to only load the fields we're actually interested in; and get rid of the raw SQL in the file at the same time: https://github.com/civicrm/civicrm-core/blob/f87c35c7308fc6876e1a4fea8ac3ea908d3d9453/CRM/Mailing/BAO/Mailing.php#L18075.9https://lab.civicrm.org/dev/core/-/issues/467Case Start Date and End Date don't work properly in Batch Update2018-11-15T11:18:29ZRayWrightCase Start Date and End Date don't work properly in Batch UpdateWhen trying to batch update a case start date or end date you get the following error: Notice: Undefined index: formatType in CRM_Utils_Date::addDateMetadataToField() (line 1878 of /srv/buildkit/build/dmaster/sites/all/modules/civicrm/C...When trying to batch update a case start date or end date you get the following error: Notice: Undefined index: formatType in CRM_Utils_Date::addDateMetadataToField() (line 1878 of /srv/buildkit/build/dmaster/sites/all/modules/civicrm/CRM/Utils/Date.php).
This results in
Start Date: the current date being filled into all values
End Date: if it's not already filled in, you can only select today's date.5.9https://lab.civicrm.org/dev/core/-/issues/3725.6.alpha1 notice2018-11-11T20:41:11ZJoeMurray5.6.alpha1 noticeCreate simple Meeting activity with minimal options. View activity:
Notice: Undefined index: details in CRM_Activity_Form_Activity->preProcess() (line 526 of /home/git-civicrm/civicrm-core/CRM/Activity/Form/Activity.php).
http://cividemo...Create simple Meeting activity with minimal options. View activity:
Notice: Undefined index: details in CRM_Activity_Form_Activity->preProcess() (line 526 of /home/git-civicrm/civicrm-core/CRM/Activity/Form/Activity.php).
http://cividemo.jmaconsulting.biz/civicrm/activity/add?reset=1&atype=1&action=update&reset=1&id=720&cid=203&context=activity5.9Monish DebMonish Debhttps://lab.civicrm.org/dev/core/-/issues/321Prevent duplicate entries in civicrm_entity_file2018-11-17T02:41:11ZMonish DebPrevent duplicate entries in civicrm_entity_fileSteps to replicate:
1. Enable CiviCase
2. Create a new Case.
3. Edit any Case Activity >> add an attachment >> Save
4. Reopen the Edit Activity form and simply save.
DB result: There will duplicate entries in civicrm_entity_fileSteps to replicate:
1. Enable CiviCase
2. Create a new Case.
3. Edit any Case Activity >> add an attachment >> Save
4. Reopen the Edit Activity form and simply save.
DB result: There will duplicate entries in civicrm_entity_file5.9Monish DebMonish Debhttps://lab.civicrm.org/dev/core/-/issues/217Allow replacement of PrevNextCache implementation (for search screens)2019-01-10T10:07:11ZtottenAllow replacement of PrevNextCache implementation (for search screens)This is a major sub-task of dev/core#174 (*Consistently use swappable cache interfaces*).
# Background
The PrevNextCache is used to temporarily save the results of a query. It is specifically used in the following scenarios:
* Contact...This is a major sub-task of dev/core#174 (*Consistently use swappable cache interfaces*).
# Background
The PrevNextCache is used to temporarily save the results of a query. It is specifically used in the following scenarios:
* Contact searches (Related -- other derived searches and the "View Contact" result screen)
* Dedupe/merge
For this issue, our aim is specifically to allow the contact-searches to retain their results via Redis/Memcache (instead of MySQL).
# Touch Points
We want to make it possible to swap the prev-next implementation, but the functionality of `CRM_Core_BAO_PrevNextCache` is a bit more sophisticated than `CRM_Utils_Cache_Interface`. So, first, we need to understand the places which touch the prev-next cache (*before patching*).
__Q1: Where does it `INSERT` into `civicrm_prevnext_cache`?__
* `CRM_Contact_Selector::fillupPrevNextCache`
* `CRM_Campaign_Selector_Search::buildPrevNextCache`
__Q2: Where does it `SELECT` from `civicrm_prevnext_cache`?__
* `CRM_Contact_BAO_Query::getCachedContacts`
__Q3: What are all the functions supported by the BAO? When are they relevant__
* (Group A) Relevant to contact searches (and maybe also dedupe/merge)
* `public static function markSelection($cacheKey, $action = 'unselect', $cIds = NULL, $entity_table = 'civicrm_contact') {`
* `public static function getSelection($cacheKey, $action = 'get', $entity_table = 'civicrm_contact') {`
* `public static function getPositions($cacheKey, $id1, $id2, &$mergeId = NULL, $join = NULL, $where = NULL, $flip = FALSE) {`
* `public static function deleteItem($id = NULL, $cacheKey = NULL, $entityTable = 'civicrm_contact') {`
* `public static function getSelectedContacts() {`
* `public static function getCount($cacheKey, $join = NULL, $where = NULL, $op = "=", `$params = array()) {
* (Group B) Only relevant to dedupe/merge
* `public static function flipPair(array $prevNextId, $onlySelected) {`
* `public static function deletePair($id1, $id2, $cacheKey = NULL, $isViceVersa = FALSE, $entityTable = 'civicrm_contact') {`
* `public static function markConflict($id1, $id2, $cacheKey, $conflicts) {`
* `public static function setItem($values) {`
* `public static function refillCache($rgid, $gid, $cacheKeyString, $criteria, $checkPermissions, $searchLimit = 0) {`
* `public static function retrieve($cacheKey, $join = NULL, $whereClause = NULL, $offset = 0, $rowCount = 0, $select = array(),...)`
* (Group C) Special, internal, or unused
* `public static function cleanupCache() {`
* `public static function is_serialized($string) {`
* `public static function buildSelectedContactPager(&$form, &$params) {`
# General Approach
* Define a service `Civi::service('prevnext')` which satisfies an interface `CRM_Core_PrevNextCache_Interface`.
* Create two implementations of the service:
* `CRM_Core_PrevNextCache_Sql` (which is pretty thin -- it generally delegates to the existing BAO code)
* ~~`CRM_Core_PrevNextCache_Memory` (which uses a memory-backed cache)~~
* `CRM_Core_PrevNextCache_Redis` (which uses Redis for a memory-backed cache)
* The scope of the interface would include:
* Functionality needed by the inserters (`CRM_Contact_Selector::fillupPrevNextCache` and `CRM_Campaign_Selector_Search::buildPrevNextCache`)
* Functionality needed by the reader (`CRM_Contact_BAO_Query::getCachedContacts`)
* Functionality from the BAO that's relevant to contact-searches (eg `markSelection` but not `deletePair`)
* In the contact-search use-cases, change to use the service instead of the BAO.
* Ex: `CRM_Core_BAO_PrevNextCache::markSelection(...)` => `Civi::service('prevnext')->markSelection(...)`
*Note*: I initially considered a different relationship (where `CRM_Core_BAO_PrevNextCache` remains the canonical/front-facing interface and `CRM_Core_PrevNextCache_Interface` remains hidden behind the BAO). However, given that dedupe/merge needs to continue using the existing SQL implementation, it felt like that would get messier; it seemed cleaner to split them apart more clearly ("use-case (1) always invokes code (A), and use-case (2) always invokes code (B)" -- i.e. "all contact-search code invokes `Civi::service('prevnext')`, and all dedupe code invokes PrevNext BAO").
![Screen_Shot_2018-06-28_at_6.55.29_PM](/uploads/7fc6852ecefe6e5638e0b3a21fb37e44/Screen_Shot_2018-06-28_at_6.55.29_PM.png)
# PR List
* [12377 - General/overall patchset](https://github.com/civicrm/civicrm-core/pull/12377)
* [12438 - CRM_Contact_Selector::getRows() - Use generator instead of DAO loop](https://github.com/civicrm/civicrm-core/pull/12438)
* [12528 - Add skeletal PrevNextCache service ](https://github.com/civicrm/civicrm-core/pull/12528)
* [12543 - PrevNext - Probe for best available implementation (memory-backed or SQL-backed)](https://github.com/civicrm/civicrm-core/pull/12543)
* [12544 - CRM_Utils_Cache_Redis::connect() - Allow pooling connections](https://github.com/civicrm/civicrm-core/pull/12544)
* [12545 - PrevNext - Define and use fillWithSql()/fillWithArray()](https://github.com/civicrm/civicrm-core/pull/12545)
* [12556 - Migrate selection methods](https://github.com/civicrm/civicrm-core/pull/12556)
* [12558 - Allow swapping getPositions (etal) for contact-search](https://github.com/civicrm/civicrm-core/pull/12558)
* [12663 - Use more consistent cache-keys while adjusting filters](https://github.com/civicrm/civicrm-core/pull/12663)
* [12664 - Remove references to entity_table and entity_id2 from service. Add test.](https://github.com/civicrm/civicrm-core/pull/12664)
* [12665 - Implement Redis. Decouple Query::getCachedContacts()](https://github.com/civicrm/civicrm-core/pull/12665)
# Test Scenarios
The prev-next cache has evolved a number of conditionals to handle edge-cases in which it's used. We should be testing these in a targeted/incremental way during development. Once we have a fairly complete PR, we should do a final/comprehensive pass through these use-cases.
* __Quick Search Box / Advanced Search / Search Builder__: This is the main section of the code that we're targetting for revision.
* Execute a search with multiple pages of results
* Execute a search with one page of results
* Drill-down and view a contact in the search results. Navigate using "Previous" button, "Next" button, and the breadcrumb "Search Results"
* [CRM-9096](https://issues.civicrm.org/jira/browse/CRM-9096) Create a profile. Mark some fields as visible/enabled for listings. Run an "Advanced Search" and use this profile. Re-sort results.
* Select a handful of contacts on different pages. Execute a task (e.g. "Export") and ensure that it has the appropriate contacts.
* Select all contacts. Execute a task (e.g. "Export") and ensure that it has the appropriate contacts.
* __Custom Search__: Most of the code is shared with above, but a few lines are different.
* Run a few custom searches. Use various sort/filter options.
* __CiviCampaign__: This component has logic to fill the cache directly -- which doesn't use the same code-paths.
* Create a survey. Reserve respondents. Interview. Release. (When browsing respondents, try to sort.)
* __CiviRules__: This component uses some of the code from contact-search, and there's a comment about special case.
* Browse the list of available rules.
* __Dedupe/Merge__: In principle, this code isn't supposed to be touched/changed by our PR, and there is test coverage over this functionality. OTOH, if one made a mistake (inappropriately changing shared code), there could be a regression. So do a couple tests with this anyway.5.9tottentottenhttps://lab.civicrm.org/dev/core/-/issues/174Consistently use swappable cache interfaces2023-12-20T06:26:25ZtottenConsistently use swappable cache interfaces# Background
CiviCRM caches were originally (and are primarily) stored in the table `civicrm_cache`. This design is great for small sites (easy to deploy on server with less software to administer). However, for larger sites that get va...# Background
CiviCRM caches were originally (and are primarily) stored in the table `civicrm_cache`. This design is great for small sites (easy to deploy on server with less software to administer). However, for larger sites that get value from using multiple MySQL servers, it's not so great: cache data (by its nature) is frequently written+read, but it's expensive to write cache data when there are multiple MySQL servers... and the overhead doesn't give us much benefit because the data is fundamentally expendable. A lighter system (like Memcache/Redis) would be more generally appropriate.
You might recall that CiviCRM's cache subsystem can optionally incorporate Memcache/Redis (since roughly 4.3ish?). This appears as `CRM_Utils_Cache_*`, and it started some great improvements (e.g. using faster cache providers; and swapping cache providers). Notably, the functionality uses Memcache/Redis as an *extra cache tier* (i.e. when reading a cache, it first checks memcache; then it falls back to checking `civicrm_cache` in MySQL). This helps *read performance* (because your cache-hit usually returns faster), but it doesn't help *write performance* (because you need to propagate cache-writes to every tier). Using *only* Memcache/Redis (as a single tier -- without MySQL) would be even better for performance.
I don't believe that the status quo reflects a technical necessity. (If you can handle deployment/configuration of Memcache/Redis a front-tier... then you can probably handle it as the main tier.) Rather, it reflects a development practicality: *the app was originally written to only use `civicrm_cache`*. Consequently, there are a handful of oddball use-cases/code-paths which directly access `civicrm_cache` or `CRM_Core_BAO_Cache`, and it would've been harder to do anything if it had required finding+fixing all of them.
# Scope
The general scope of this ticket is to hunt down the oddball use-cases/code-paths which directly reference `civicrm_cache` or `CRM_Core_BAO_Cache`. I expect the project to produce a handful of distinct PRs. To avoid redundantly documenting this, I'm aiming for each PR to have a more concrete discussion about the use-cases/code-paths being cleaned up.
# PR List
* [12330 - CRM_Utils_Cache_Redis::flush() - Respect prefixes](https://github.com/civicrm/civicrm-core/pull/12330) (m)
* [12331 - CRM_Utils_Cache - Always use a prefix. Standardize delimiter](https://github.com/civicrm/civicrm-core/pull/12331) (m)
* [12336 - systemStatusCheckResult - Migrate from settings to cache](https://github.com/civicrm/civicrm-core/pull/12336) (m)
* [12342 - Caching - Comply with PSR-16 interfaces](https://github.com/civicrm/civicrm-core/pull/12342) (m)
* [215 - Import PSR-16 compliance test](https://github.com/civicrm/civicrm-packages/pull/215) (m)
* [12348 - Cache-keys for CRM_Utils_Cache_* should avoid reserved chars](https://github.com/civicrm/civicrm-core/pull/12348) (m)
* ~~[12350 - civicrm_cache - Allow storage of binary data](https://github.com/civicrm/civicrm-core/pull/12350)~~
* [12354 - civicrm_cache - Allow storage of binary data](https://github.com/civicrm/civicrm-core/pull/12354) (m)
* [~~12360~~ - Full PSR-16 compliance for ...](https://github.com/civicrm/civicrm-core/pull/12360) (*depends: ~~12342, 12348, 12354~~*). Cherry-picked to form smaller PRs:
* [12376 - Add various utilities to support PSR-16](https://github.com/civicrm/civicrm-core/pull/12376) (m)
* [12378 - ArrayCache, Redis](https://github.com/civicrm/civicrm-core/pull/12378) (m)
* [12379 - SqlGroup](https://github.com/civicrm/civicrm-core/pull/12379) (m)
* [12380 - Memcache(d)](https://github.com/civicrm/civicrm-core/pull/12380) (m)
* [12381 - APCcache](https://github.com/civicrm/civicrm-core/pull/12381) (m)
* [12389 - Add test to prevent hidden interactions among caches](https://github.com/civicrm/civicrm-core/pull/12389) (m)
* [12362 - Forms/Sessions - Store state via Civi::cache('session')](https://github.com/civicrm/civicrm-core/pull/12362) (*depends: 12360*)
* [12368 - ~~civicrm_cache - Finish transition from DATETIME to TIMESTAMP](https://github.com/civicrm/civicrm-core/pull/12368)~~ (m)
See also
--------
* dev/core#217 - Allow replacement of PrevNextCache implementation (for search screens)
* dev/core#284 - Aggressive cache clearing significantly increases test time5.9tottentottenhttps://lab.civicrm.org/dev/core/-/issues/153Pending Pay Later /w Custom payment method2020-11-19T07:07:10ZADG CreativePending Pay Later /w Custom payment methodCiviCRM 5.1.2
Joomla 3.8.7
(Replicated on civicrm.demo.civihosting.com - CiviCRM 5.0.0 - Drupal)
Create a custom Payment Method (ie. Undetermined), with Financial Account = Accounts Receivable
On the administrative side, register contac...CiviCRM 5.1.2
Joomla 3.8.7
(Replicated on civicrm.demo.civihosting.com - CiviCRM 5.0.0 - Drupal)
Create a custom Payment Method (ie. Undetermined), with Financial Account = Accounts Receivable
On the administrative side, register contact for an event with Payment Method=Undetermined, Payment Status=Pending
Go to the Contribution tab, the contribution is Pending Pay Later
Do Record Payment, Payment Method=Check, Add Check #, Payment Status=Complete, (Full Amount Paid)
The Contribution will be marked with a Payment Status of Partially Paid
Admin can edit the contribution and mark it as Payment Status = Complete, but...
Expand the payment information, Record Payment links are still visible, but will throw an error if clicked
civicrm_contribution.payment_instrument_id = Undetermined (Although payment method was set to Check)
civicrm_financial_trxn.from_financial_account_id = Accounts Receivable
civicrm_financial_trxn.to_financial_account_id = Accounts Receivable
Contribution not listed in Bookkeeping Report
Also cannot find contribution by Check# in Contributions->Find Contributions (Not sure if related or separate issue)
![CompletedMarkedasPartiallyPaid](/uploads/26c971d0956ac4c6abd40cb6933fd917/CompletedMarkedasPartiallyPaid.jpg)5.9https://lab.civicrm.org/dev/core/-/issues/125Invalid link to custom-fields documentation2018-12-15T22:50:50ZedgimarInvalid link to custom-fields documentationOn the `index.php?q=civicrm/admin/custom/group&reset=1` screen, the "learn more..." link appears to be broken -- this target of this link is currently https://docs.civicrm.org/user/en/latest/organising-your-data/custom-fields but should ...On the `index.php?q=civicrm/admin/custom/group&reset=1` screen, the "learn more..." link appears to be broken -- this target of this link is currently https://docs.civicrm.org/user/en/latest/organising-your-data/custom-fields but should instead be https://docs.civicrm.org/user/en/latest/organising-your-data/creating-custom-fields/.
Using CiviCRM 5.1.1.5.9https://lab.civicrm.org/dev/core/-/issues/11Email - send now error screen should display earlier2023-06-23T17:54:21ZJchesterEmail - send now error screen should display earlierIn Advanced Search if you are displaying results as contacts and then:
* select more than 50 of the results
* select **Email - send now (to 50 or less)**
you see the Error message telling you that you can't email more than 50 contac...In Advanced Search if you are displaying results as contacts and then:
* select more than 50 of the results
* select **Email - send now (to 50 or less)**
you see the Error message telling you that you can't email more than 50 contacts this way and the process stops
However, if you display the results as contributions or as memberships then
* select more than 50 of the results
* select **Email - send now (to 50 or less)**
you are taken to the email set up page. It is not until you click on Send Email on that page that you see the Error message telling you that you can't email more than 50 contacts this way and the process is stopped.
It would be a better user experience if the error message always appeared when **Email - send now (to 50 or less)** was selected.5.9