Development issueshttps://lab.civicrm.org/groups/dev/-/issues2020-01-13T09:38:31Zhttps://lab.civicrm.org/dev/financial/-/issues/78Order api should probably create an invoice ID2020-01-13T09:38:31ZeileenOrder api should probably create an invoice IDCurrently contribute.transact api does this -it makes sense to do it at the order level - for create.Currently contribute.transact api does this -it makes sense to do it at the order level - for create.https://lab.civicrm.org/dev/financial/-/issues/76Metaissue for ensuring our preferred order->pay->createPayment flow is used ...2023-11-23T06:59:31ZeileenMetaissue for ensuring our preferred order->pay->createPayment flow is used everywhere.We have long had the principle (since 4.6 was pre-alpha) that we want to work towards the following flow at a code level.
1. Create a pending Order using Order.create api
1. Optionally process a payment against that order using PaymentP...We have long had the principle (since 4.6 was pre-alpha) that we want to work towards the following flow at a code level.
1. Create a pending Order using Order.create api
1. Optionally process a payment against that order using PaymentProcessor.pay api
1. Create a payment record (either from above or just recording an offline payment) using Payment.create api
The expectation is that the code correctly creates a pending contribution (order) with all the related entities when Order.create is called.
When a payment is added the related entities are updated & if appropriate notification emails are sent out.
(Note an earlier iteration of this was to create a pending contribution always & then update it with contribution.completetransaction - this maps to the above as the Order.create creates the pending contribution and Payment.create calls contribution.completetransaction when the payment completes the order).
This issue is a metaissue to link to & categorise all the things that need to be done to reach this. 'High' Priority are any issues that stand before people adopting the recommended flow.
| Item | Status| Priority| Blockers|Notes|
| ------ | ------ |------ | ------ |------ |
| Document order api / Pseudoentity | In progress|High| - |At the [sprint this was started](https://docs.civicrm.org/dev/en/latest/financial/OrderAPI/)[Current tweak](https://github.com/civicrm/civicrm-dev-docs/pull/704)|
|~~Deprecate creating orders with non-pending status~~|Done|High||||
|~~Order.create should not require total amount~~|Done|High|||https://lab.civicrm.org/dev/financial/issues/73|
|Identify any issues that would block someone moving off a non-preferred flow to our preferred flow|Started|High||This is really what this issue is about|
| Update payment instrument id when adding a payment to a pending order|To discuss| High| Discussion| https://lab.civicrm.org/dev/financial/issues/81 and possibly https://lab.civicrm.org/dev/financial/issues/47|
|~~Support invoice_id in PaymentProcessor.pay~~|done|high||https://lab.civicrm.org/dev/financial/issues/77|
| ~~Support chaining Payment.create from Order~~|PR|High||Needed to ease adaption to us deprecating creating Orders as non-pending [pr](https://github.com/civicrm/civicrm-core/pull/15548)|
|Fix identified core bug on ParticipantPayment creation|To do|High|Tests not yet passing| https://lab.civicrm.org/dev/financial/issues/74|
|~~Make contribution_id mandatory for PaymentProcessor.pay~~|Done|High||[PR](https://github.com/civicrm/civicrm-core/pull/15477) - since we have agreed for policy (not technical) reasons to do this we should try to get a test & merged|
| ~~Document moving off transact api~~| Done | Medium||[PR here](https://github.com/civicrm/civicrm-dev-docs/pull/705)|
| Consider generating default invoice ID in Order.create|To do|Medium|Discussion|https://lab.civicrm.org/dev/financial/issues/78|
|~~Add getters & setters to Payment Class~~|Done - could do more maybe| Medium||https://lab.civicrm.org/dev/financial/issues/82|
| ~~Deprecate transact api from explorer~~ | Done |Medium||https://lab.civicrm.org/dev/financial/issues/79|
|~~Deprecate & remove calls to CRM_Contribute_BAO_Contribution::addPayments~~|To do|Low||This is part of consolidating all payment creation on Payment.create|
|Fix all remaining places in core to create a pending contribution/order & then complete|Needs chipping away at - tacking events at the moment|In progress|Low|https://lab.civicrm.org/dev/financial/issues/53|
| ~~Noisily deprecate transact api~~ |Done| low||https://lab.civicrm.org/dev/financial/issues/80|https://lab.civicrm.org/dev/core/-/issues/1328Escape-on-Input => Escape-on-Output2022-09-27T23:42:48ZtottenEscape-on-Input => Escape-on-Output# Introduction
In a multi-tier application like CiviCRM, a piece of data (such as a `first_name`) may be passed through several systems and formats (MySQL/PHP/Firefox/etc; SQL/HTML/URL/JSON/CSV/PDF/etc). In each system, this piece of d...# Introduction
In a multi-tier application like CiviCRM, a piece of data (such as a `first_name`) may be passed through several systems and formats (MySQL/PHP/Firefox/etc; SQL/HTML/URL/JSON/CSV/PDF/etc). In each system, this piece of data may be encoded in a different way. The differences are sometimes imperceptible to a casual reader (ex: "banana" looks the same in SQL, HTML, URL, JSON, and CSV), but mistakes and confusion about encoding are still problematic. The impact ranges from quirky (odd characters appearing occasionally in unexpected places) to malicious (security vulnerabilities).
CiviCRM has a long-standing technical debt with respect to encoding. This debt is not specifically a bug, feature, or vulnerability in itself. (If you know a specific piece of data that is misencoded, then that is often a security problem that should be [reported to the security team](https://civicrm.org/security) for discrete resolution.) Rather, this debt is *an unusual convention* ("Escape-on-Input") which has *unintuitive consequences* and thereby invites bugs.
See also: [Dev Guide: Sanitize inputs or outputs?](https://docs.civicrm.org/dev/en/latest/security/#strategy)
At the recent Barcelona sprint, there was a partial meeting of the security team, and attendees agreed that this was the root issue behind multiple problems and merited special attention. Unfortunately, this convention is deeply embedded. Changing it is difficult and risky, so it has lingered for several years without substantive improvement, and (with its current framing/visibility) would remain indefinitely. The aim of this filing is to have a public discussion/record which works out the what/how/why of a (possible) cleanup.
# Definitions
* __Escape-on-Input (aka Esc-In)__: An architectural style in which data is taken as input and immediately escaped; it is then written to disk in escaped (HTML-esque) format. Consequently, subsequent searches+reads yield HTML-esque strings. These can be outputted literally on HTML pages without any extra processing, but (for other media) they require double processing (decode/re-encode).
* __Escape-on-Output (aka Esc-Out)__: An architectural style in which data is is taken as input and stored in its canonical format. When the data is presented as output, it is escaped in a way that matches the current output-format.
* __Canonical (String format)__: An encoding in which strings look... like they should. For example, if a user types `first_name` of `:<`, then the string is `:<`
* __HTML-esque (String format)__: An encoding in which key HTML characters are escaped as entities. For example, canonical string `:<` becomes HTML-esque string `:<`. In HTML-esque, one does not intend to convey HTML tags.
* __O(1) Plan/Task__: A plan/task in which the number of tasks is fixed. (Ex: To change behavior of all "quickforms", you might patch+test the base class `HTML_Quickform` one time.) It may strictly be 1 or 2 or 3 steps... but the number of steps does *not* depend on how many screens/use-cases are impacted.
* __O(n) Plan/Task__: A plan/task in which the number of tasks depends on how many screens/use-cases are impacted. (Ex: To change behavior of all "quickforms", you might patch+test *every subclass*.)
* __Smarty Filter/Autoescape__: A Smarty filter allows one to programmatically change the behavior/content of all Smarty templates. For example, you can write a Smarty prefilter to automatically add HTML "escape" commands to all output.
* __Linter__: A programmer tool which scans source code and checks for compliance. For example, a linter could have the rule "all Smarty variable include the `|escape` or `|noescape` modifier."
# Safe and unsafe situations
The status quo is peculiar. It is loosely safe in some common situations, but it's surprisingly unsafe in other situations. Consider this table:
<table>
<thead>
<tr><th></th><th>Read/write data via SQL/DAO/BAO</th><th>Read/write data via APIv3/v4</th></tr>
<thead>
<tbody>
<tr><th>Process inputs+outputs via HTML_Quickform+Smarty (HTML-only)</th><td>OK</td><td>Caution</td></tr>
<tr><th>Process inputs+outputs via most frameworks (Drupal Form API, AngularJS, DOM API, PHPExcel, etc)</th><td>Caution</td><td>OK</td></tr>
<tr><th>Interchange data between SQL/DAO/BAO and APIv3/v4</th><td>Caution</td><td>Caution</td></tr>
</tbody>
</table>
In "OK" situations, you can take a string and send it to the screen -- and there's a high probability it will end-up encoded correctly without any thinking/effort.
In "Caution" situations, you should treat most strings with suspicion -- many would merit extra encoding/decoding steps.
The general aim of this cleanup issue is to be "OK" on the entire grid.
# Relevant background/reading
* (Jan 2010) [CRM-5667](https://issues.civicrm.org/jira/browse/CRM-5667): Introduction of "Escape-on-Input"
* (Dec 2012) [svn log](https://github.com/civicrm/civicrm-svn/commits/v4.2/CRM/Core/HTMLInputCoder.php): Mitigation for APIv3 consumers
* (Dec 2012) [CRM-11532](https://issues.civicrm.org/jira/browse/CRM-11532): First filing about the problematic status-quo. (Shorter)
* (Apr 2018, et al) [security/core#6](https://lab.civicrm.org/security/core/issues/6): Second filing about the problematic status-quo. (Longer)
* (2018) [Dev Guide: Sanitize inputs or outputs?](https://docs.civicrm.org/dev/en/latest/security/#strategy): Reference material for developers
* (Oct 2019) [POC: Auto-escaping in Smarty v2](https://gist.github.com/totten/5b30e10a21fe626a43a30e21ded26fc4)
* (Oct 2019) [Report: Categorize DB fields by escaping implication](https://gist.github.com/totten/ec78dcb869aa7cbbc95c5f273f7ea30d)
* (Oct 2019) [Report: List of Smarty expressions](https://gist.github.com/totten/9c7176bbdfc63f88d423f026359f50fc)
# Candidate Approaches
* __Do Nothing__:
* __How__: Kick off your shoes. Pull up Netflix.
* __Pro__: Less work
* __Con__: Encourages developers to write buggy code. Feels embarrassing whenever they realize this is a thing.
* __Epic w/Smarty Filter__:
* __Pitch__: Smarty does all HTML-escaping automatically and by default.
* __How__:
* Remove Esc-In policy (`HTMLInputCoder`)
* Write a generic upgrade step to recode all stored strings.
* To restore output-coding, add a Smarty prefilter to auto-escape output.
* __Pro__: Cleans up the database.
* __Con__: The main problem is Smarty templates are not homogeneous - *most* generate HTML, but *some* generate other formats. They *often* pass-through data from the database, but they *also* display data which is computed and/or loaded from a different source. Consequently, there is an O(n) task of re-testing all screens/fields and patching a large number of exceptions.
* __Con__: Additionally, there's the `r-tech` issue - even if we correctly change every screen/field/line in the main application, there is an open-set of third-party integrations. If these use SQL/DAO/BAO and work correctly in the current system, then they'll likely break as soon as the epic change is made, which will create stress for upgrade-scheduling.
* __Con__: Smarty is difficult to write unit tests for.
* __Epic w/DAO Filter__:
* __Pitch__: DAO does all HTML-escaping automatically and by default.
* __How__:
* Remove Esc-In policy (`HTMLInputCoder`)
* Write a generic upgrade step to recode all stored strings.
* To restore output-coding, update DAO/BAO with an optional escaping mode. (Take the current skiplist - everything else defaults to autoescaping.)
* __Pro__: Cleans up the database.
* __Pro__: Looks like an O(1) plan.
* __Pro__: Restoring output coding can be done at a single point - no need to fix existing templates.
* __Pro__: Output coding can be unit-tested.
* __Con__: Only addresses DAO-based query-building (`$d=new DAO_Foo(); $d->bar=123; $d->fetch()`). Does not address `executeQuery('SELECT foo AS bar...')`, `executeQuery('UPDATE foo SET bar...')`, or other query-building. This seems to be left as O(n) task?
* __Question__: Do we get read/write consistency?
* __Wait by Extension Approach__:
* __Pitch__: Grow new UIs on top of APIv3/v4. Obsolete everything all UIs that rely on DAO/BAO/SQL.
* __How__: Do nothing specifically for this problem. Instead, wait for extensions like Afform or Data Processor to gradually replace the HTML_Quickform/Smarty UIs. Once there's a critical mass, remove all old UIs and do an "Epic" switch.
* __Pro__: Don't have to do anything now! Creates pressure to do other useful changes.
* __Con__: You may be waiting a long time.
* __Incremental, Per-Component/Per-Field__:
* __Pitch__: Convert fields incrementally
* __How__: Make a list of all relevant DB fields (~500). In the first month, take a subset (such as "all CiviEvent fields") and transition/test the subset. In the second month, do another subset of fields (such as "all CiviCampaign fields").
* __Pro__: This allows tasks to spread out over time with more manageable/affordable chunks.
* __Con__: This still has the `r-tech` issue for third-party integrations. It creates a long-term drip-drip of compatibility breaks -- every month, some subset of fields have their data-format changed. This will make it quite challenging for extensions/integrations to keep in sync.
* __Comment__: Bespoke screens (e.g. "New participant") are amenable to simple grep+divide+conquer, but we might get boxed-in with how to update metadata-driven screens which tend to treat all strings the same way (e.g. import/export).
* __Incremental, Setting Plus Smarty Linting__:
* __Pitch__: Add toggle to allow old+new storage-formats. Incrementally enhance code to make new-storage-format work.
* __How__:
* Define a new setting to indicate whether strings are stored as "html-esque" (initial-default; long-term-deprecate) or "canonical" (initial-experimental; long-term-goal).
* Define a migration script - which can be run whenever one changes the setting.
* Update Smarty/Quickform/API to check the setting and filter accordingly. Define corresponding Smarty filter. (Naive example: `{$records[$cid]->display_name|crmEscape:'Contact.display_name'}` means "escape this data based on the escaping-rules for the `Contact` field `display_name`)
* Write a linter to report on which Smarty templates use the filter. Over time (month-by-month), incrementally update Smarty templates to use the new filters and re-test those screens.
* __Pro__: This allows tasks to be spread out over time with more manageable/affordable chunks. The linter provides a guide for helping us track "cleared ground".
* __Pro__: Site-implementers and extension/integration-authors will have greater flexibility on scheduling the change - they don't have to update all codes and all databases at the same time.
* __Con__: Requires exposing some new filters/functions/settings.
* __Incremental, Setting Plus DAO Filter__:
* __How__:
* Define a new setting to indicate whether strings are stored as "html-esque" (initial-default; long-term-deprecate) or "canonical" (initial-experimental; long-term-goal).
* Define a migration script - which can be run whenever one changes the setting.
* Update DAO/API to check the setting and filter accordingly.
* __Pro__: At a glance, this looks like an O(1) plan.
* __Pro__: Site-implementers and extension/integration-authors will have greater flexibility on scheduling the change - they don't have to update all codes and all databases at the same time.
* __Con__: Only addresses DAO-based query-building (`$d=new DAO_Foo(); $d->bar=123; $d->fetch()`). Does not address literal queries (`executeQuery('SELECT foo AS bar...')`, `executeQuery('UPDATE foo SET bar...')`) or other forms of query-building (`CRM_Utils_SQL`/APIv4/adhoc PHP). This seems to be left as O(n) task?
* __Question__: Do we get read/write consistency?
* __Incremental, SQL Views with Deprecation Alerts__:
* __Pitch__: Support both formats in MySQL *at the same time*. Borrow the idea of wide-spread VIEWs from multilingual.
* __How__:
* Every entity (eg `Contact`) should have a concrete TABLE and a derived VIEW (eg `civicrm_contact` and `civicrm_contact_new`) which present the same data in different formats (old/HTML-esque and new/canonical).
* `executeQuery()` generates a deprecation warning whenever there's a query that hits old-format table. The warning says something like "`civicrm_contact_old` is deprecated. Please use `civicrm_contact_new` instead. Be sure to update the escaping on any consumers."
* At the mid-point during transition, swap the SQL schema (eg the old-format is given by SQL VIEW and the new-format is given by SQL TABLE).
* Eventually, remove the old-format VIEWs completely.
* __Pro__: Existing (unpatched) consumers in any medium continue to work throughout the transition. But they start to emit warnings immediately.
* __Pro__: Extension/integration-authors (and site-admins/upgraders) have greater flexibility on scheduling changes - they don't have to update all codes and all databases at the same time.
* __Con__: Query performance on the filtered VIEW should be slower than the canonical TABLE. During the transitional process, queries will flop-flop around between faster/slower.https://lab.civicrm.org/dev/core/-/issues/1321group.get API (v3- but lets fix for v4) fails to find groups of one group_typ...2023-08-04T20:49:54ZRichgroup.get API (v3- but lets fix for v4) fails to find groups of one group_type if group has multiple typesWe used (5.15) to be able to filter for mailing groups with the API and now we can't. I'm not sure when this came in.
e.g. on a buildkit at 58e7f004e2
```
cv api Group.get return='id,title,group_type'
```
Returns
```
{
"is_error":...We used (5.15) to be able to filter for mailing groups with the API and now we can't. I'm not sure when this came in.
e.g. on a buildkit at 58e7f004e2
```
cv api Group.get return='id,title,group_type'
```
Returns
```
{
"is_error": 0,
"version": 3,
"count": 4,
"values": {
"1": {
"id": "1",
"title": "Administrators",
"group_type": [
"1"
]
},
"2": {
"id": "2",
"title": "Newsletter Subscribers",
"group_type": [
"1",
"2"
]
},
"3": {
"id": "3",
"title": "Summer Program Volunteers",
"group_type": [
"1",
"2"
]
},
"4": {
"id": "4",
"title": "Advisory Board",
"group_type": [
"1",
"2"
]
}
}
}
```
But
```
cv api Group.get return='id,title,group_type' group_type='Mailing List'
cv api Group.get return='id,title,group_type' group_type=2
```
returns none.
<del>On 5.15 Mailing groups are returned properly.</del>https://lab.civicrm.org/dev/financial/-/issues/75on order.get, line item has obsoleted contribution_type_id2019-10-19T22:49:23ZJoeMurrayon order.get, line item has obsoleted contribution_type_idThe line item table does not have a contribution_type_id. But when the results of a contribution.get are displayed, the line item fields includes both financial_type_id and contribution_type_id. This seems like an historical artifact tha...The line item table does not have a contribution_type_id. But when the results of a contribution.get are displayed, the line item fields includes both financial_type_id and contribution_type_id. This seems like an historical artifact that should be removed.https://lab.civicrm.org/dev/user-interface/-/issues/9Can't add / edit Name Badge Template Options2019-10-08T15:58:20ZHeatherOliverCan't add / edit Name Badge Template OptionsThrough the UI, you can add / edit Mailing Label format options, but not options for Event Labels. This means unless you are using one of the four pre-configured options, you need to go to the SQL to change between Mailing Label and Name...Through the UI, you can add / edit Mailing Label format options, but not options for Event Labels. This means unless you are using one of the four pre-configured options, you need to go to the SQL to change between Mailing Label and Name badge to test and use.
/civicrm/admin/labelFormats?reset=1
![mailing-labels](/uploads/9b6e6dcd428fd100daddb4aacf6aefeb/mailing-labels.PNG)https://lab.civicrm.org/dev/financial/-/issues/74Order.create - registering multiple Participants (participantPayment records ...2020-03-04T23:10:14ZAndrei Mondocandreimondoc@gmail.comOrder.create - registering multiple Participants (participantPayment records not created properly)Creating an Order to register two (or more) participants creates two (or more) `ParticipantPayment` records, one for each participant line item, registering two participants via an Event page creates only one `ParticipantPayment` record ...Creating an Order to register two (or more) participants creates two (or more) `ParticipantPayment` records, one for each participant line item, registering two participants via an Event page creates only one `ParticipantPayment` record regardless of the number of participants registered.
Which is the correct one/what is expected behaviour?
### Steps to reproduce
Call Order.create with parameters similar to:
``` php
$params = [
'total_amount' => 100.00,
'financial_type_id' => 4,
'contribution_status_id' => 'Pending',
'payment_instrument_id' => 1,
'currency' => 'USD',
'source' => 'Multiple participants (Order.create)',
'contact_id' => 2,
'line_items' => [
1 => [
'line_item' => [
0 => [
'price_field_id' => 3,
'label' => 'Tickets',
'financial_type_id' => 4,
'price_field_value_id' => 3,
'qty' => 1,
'field_title' => '1 Ticket',
'unit_price' => 50.000000000,
'line_total' => 50,
'entity_table' => 'civicrm_participant',
],
],
'params' => [
'contact_id' => 2,
'event_id' => 1,
'role_id' => 1,
'source' => 'Multiple participants (Order.create)',
'price_set_id' => 7,
'fee_level' => '1 Ticket',
'fee_amount' => 50,
],
],
2 => [
'line_item' => [
0 => [
'price_field_id' => 3,
'label' => 'Tickets',
'financial_type_id' => 4,
'price_field_value_id' => 3,
'qty' => 1,
'field_title' => '1 Ticket',
'unit_price' => 50.000000000,
'line_total' => 50,
'entity_table' => 'civicrm_participant',
],
],
'params' => [
'contact_id' => 5,
'event_id' => 1,
'role_id' => 1,
'source' => 'Multiple participants (Order.create)',
'price_set_id' => 7,
'fee_level' => '1 Ticket',
'fee_amount' => 50,
],
],
],
];
$order = civicrm_api3('Order', 'create', $params);
```
This results in two ParticipantPayment records:
``` json
// civicrm_participant_payment
[
{
"id": 5,
"participant_id": 5,
"contribution_id": 18
},
{
"id": 6,
"participant_id": 6,
"contribution_id": 18
}
]
```
Registering two or more participants via an Event page, always creates one ParticipantPayment regardless of the number of participants registered.
``` json
// civicrm_participant_payment
{
"id": 7,
"participant_id": 7,
"contribution_id": 19
}
```https://lab.civicrm.org/dev/user-interface/-/issues/8Create New Donation Form Option with Better Design2022-11-08T22:36:19ZroshaniCreate New Donation Form Option with Better Design[Improvements_to_CiviCRM_Donation_Forms.docx](/uploads/13d08f094f6ecde376bf84304daaec6e/Improvements_to_CiviCRM_Donation_Forms.docx)
Improvements to CiviCRM Donation Forms
1. Convert Donation Amounts location and styling
- Change the d...[Improvements_to_CiviCRM_Donation_Forms.docx](/uploads/13d08f094f6ecde376bf84304daaec6e/Improvements_to_CiviCRM_Donation_Forms.docx)
Improvements to CiviCRM Donation Forms
1. Convert Donation Amounts location and styling
- Change the donation amount radio buttons from radio buttons to buttons with donation amounts
- Move the donation amounts below One-Time and Monthly buttons
https://ccrjustice.org/donate?track1=website&track2=wheader&track3=2016-DEC-16
Give a donation in the Amount of:
$100.00
$250.00
$500.00
$1,000.00
$5,000.00
Other Amount
2. One-Time & Monthly Donation Buttons
1. Add One-Time & Monthly Buttons above the donation amounts
Current CiviCRM Example:
Currently there is a checkbox for creating a recurring donation. Also note that donation amounts appear as radio buttons above the recurring options.
https://ccrjustice.org/donate?track1=website&track2=wheader&track3=2016-DEC-16
I want to join the CCR Justice Sustainers and contribute this amount every month
Monthly giving provides reliable funding which allows CCR to plan, leverage and allocate resources in a way that means more hope for our clients, more support for movements, more justice and accountability.
Examples:
https://secure.humanesociety.org/site/Donation2;jsessionid=00000000.app325b https://action.aclu.org/give/now
2. If someone selects Monthly, it will change the donation amounts shown.
3. Default will be One-Time donation button option.https://lab.civicrm.org/dev/user-interface/-/issues/7Ability to exclude activities that appear on the actions button on a contact ...2019-10-07T13:29:15ZHeatherOliverAbility to exclude activities that appear on the actions button on a contact recordAll activity types that are added to CiviCRM are included in the Actions button when you view a contact record. For sites with a lot of activities (particularly those which are used in connection with Drupal webforms andnot normally trig...All activity types that are added to CiviCRM are included in the Actions button when you view a contact record. For sites with a lot of activities (particularly those which are used in connection with Drupal webforms andnot normally triggered in CiviCRM directly) this becomes a bit frustrating to manage.
Adding / filtering activities in other places uses the look up function, meaning the the length of the list in those places isn't really an issue.
See two attached images as an example. This isn't even all the activities for this site.
![image-1](/uploads/9b605ca4b8c666484a20ffb6be744651/image-1.PNG)![image-2](/uploads/56b30fe891eeaa9bbab310c74328cfb5/image-2.PNG)https://lab.civicrm.org/dev/user-interface/-/issues/4Enhancement - Quick search contact lookup2019-10-07T13:26:00ZrebeccatregennaEnhancement - Quick search contact lookupAllow quick search to be used with full names rather than by leading last name; e.g. allow users to search for Oliver Gibson.Allow quick search to be used with full names rather than by leading last name; e.g. allow users to search for Oliver Gibson.https://lab.civicrm.org/dev/user-interface/-/issues/3Support Menu Addition - Bug Reporting2022-01-04T20:14:54ZrebeccatregennaSupport Menu Addition - Bug ReportingAdd menu option under 'Support' titled 'Report an issue' linking to existing 'View issues and report bugs'Add menu option under 'Support' titled 'Report an issue' linking to existing 'View issues and report bugs'https://lab.civicrm.org/dev/drupal/-/issues/92Drupal8 - Solarium/Solarium 5.13 breaks Civicrm2020-10-24T21:12:08ZRar9Drupal8 - Solarium/Solarium 5.13 breaks CivicrmIf Solarium/Solarium 5.13 is used all Drupal 8 breaks
I was forced to apply composer require symfony/event-dispatcher:"4.3.4 as 3.4.99"
But this gives
PHP Fatal error: Uncaught Error: Class 'Symfony\Component\EventDispatcher\Contai...If Solarium/Solarium 5.13 is used all Drupal 8 breaks
I was forced to apply composer require symfony/event-dispatcher:"4.3.4 as 3.4.99"
But this gives
PHP Fatal error: Uncaught Error: Class 'Symfony\Component\EventDispatcher\ContainerAwareEventDispatcher' not found in /var/www/vhosts/XXX/vendor/civicrm/civicrm-core/Civi/Core/CiviEventDispatcher.php:18
![image](/uploads/cfcf9db6c5faf6acecd610b4b2933197/image.png)
If civicrm is disabled site works again.https://lab.civicrm.org/dev/core/-/issues/1275Batch update for cases gives a warning and then fatal error if only one case ...2022-09-20T11:04:16ZDaveDBatch update for cases gives a warning and then fatal error if only one case is in the listI also see this on dmaster.demo, and I'm still trying to see the exact triggers to reproduce and when it started, but what seems to be a consistent one is:
1. First set up a profile that has a case field in it, e.g. a profile that just ...I also see this on dmaster.demo, and I'm still trying to see the exact triggers to reproduce and when it started, but what seems to be a consistent one is:
1. First set up a profile that has a case field in it, e.g. a profile that just has case status.
2. Do a find cases that results in only one search result.
3. Select it and choose update multiple cases from the actions dropdown.
4. You'll get a warning like `Notice: Undefined property: CRM_Core_DAO::$case_id in CRM_Core_Form_Task::preProcessCommon() (line 139 of /srv/buildkit/build/dmaster/web/sites/all/modules/civicrm/CRM/Core/Form/Task.php)`.
5. If you continue and select your profile, you then get a fatal error where it's obvious the problem is that it wasn't able to obtain the case id (the `IN ()` part):
```
[message] => DB Error: syntax error
[mode] => 16
[debug_info] =>
SELECT contact.id as contactId, civicrm_case.id as componentId, contact.sort_name as sort_name, email as email
FROM civicrm_case as civicrm_case
INNER JOIN civicrm_case_contact ccs ON (ccs.case_id = civicrm_case.id)
INNER JOIN civicrm_contact contact ON ( contact.id = ccs.contact_id ) LEFT JOIN civicrm_email email ON ( contact.id = email.contact_id AND email.is_primary = 1 )
WHERE civicrm_case.id IN ()
GROUP BY civicrm_case.id, contact.id
```https://lab.civicrm.org/dev/drupal/-/issues/88CRM/Utils/System/Drupal8.php autoload.php path issue 5.16.32020-04-22T14:54:19ZndavisCRM/Utils/System/Drupal8.php autoload.php path issue 5.16.3line 419.
Relies on cmsRootPath
Standard drupal installation cms path (cmsRoot) is now [drupal root]/web.
Drupal8.php now looks for autoload as **[cms.root]//autoload.php** (line 419)
Unless you symlink autoload.php from ../vendor/au...line 419.
Relies on cmsRootPath
Standard drupal installation cms path (cmsRoot) is now [drupal root]/web.
Drupal8.php now looks for autoload as **[cms.root]//autoload.php** (line 419)
Unless you symlink autoload.php from ../vendor/autoload.php this fails to find autoload.php.
Suggest an if block that first tries `require_once "autoload.php";` and if that fails (vendor directory is not in php's include_path) *then* look for it in a better place than cmsRoot. As well [cms.root] has a trailing slash so the forward slash in the `$autoloader = require_once $root."/autoload.php"` is unnecessary.
vendor is not supposed to be accessible in the web root, so for anyone who's vendor directory is not the web root, and is in a directory one directory level up (out of web root) Drupal8.php won't find autoload.php.
I'm not sure of the Drupal8.php's history, but changes related to the path to autoload.php between 5.15.1 and 5.16.3 and broke things.
If you change the [cms.root] so Drupal8.php can find autoload.php the rest of civi breaks. If I set this path properly in Drupal8.php everything works.https://lab.civicrm.org/dev/drupal/-/issues/86Group Role sync not mapping users but duplicating users2019-09-13T07:05:05ZacaselliGroup Role sync not mapping users but duplicating usersHi,
The group role sync module doesn't seem to work properly anymore. When I create a new user and I log in for the first time, instead of mapping the Drupal user id with the civicrm user id, the module maps the drupal user id to a new ...Hi,
The group role sync module doesn't seem to work properly anymore. When I create a new user and I log in for the first time, instead of mapping the Drupal user id with the civicrm user id, the module maps the drupal user id to a new user that has everything blank apart from the email.
I checked in the database, and in the table uf_match to confirm and here is (I think) what the module does.
When a user logs in, the checking of the existing user doesn't work anymore, so a new user is created with only the email field and that new user civicrm id is mapped with the drupal id in the uf_match table.
This is causing a lot of problems in our website. I updated to the latest version, 5.17.3 but that didn't fix it either. I noticed this error since version 5.11https://lab.civicrm.org/dev/joomla/-/issues/21cron.php broken since Joomla 3.8 in some PHP environments2021-02-26T08:34:13ZAndrew Thompsoncron.php broken since Joomla 3.8 in some PHP environmentsThis issue was mentioned in https://issues.civicrm.org/jira/browse/CRM-21203 and the [associated PR](https://github.com/civicrm/civicrm-core/pull/11609), which eventually solved problems for cli.php. But there is a session issue that has...This issue was mentioned in https://issues.civicrm.org/jira/browse/CRM-21203 and the [associated PR](https://github.com/civicrm/civicrm-core/pull/11609), which eventually solved problems for cli.php. But there is a session issue that hasn't gone away for cron.php so I am reporting that here.
Depending on PHP error reporting settings (and PHP version possibly, certainly it happens on PHP 7+), this warning will be given:
```
Warning: ini_set(): A session is active. You cannot change the session module's ini settings at this time in /var/www/html/membership/libraries/joomla/session/handler/joomla.php on line 48
```
Which may also cause this one:
```
Warning: session_cache_limiter(): Cannot change cache limiter when headers already sent in /var/www/html/membership/libraries/joomla/session/handler/native.php on line 235
```
Followed by this error:
```
Error: Failed to start application: Failed to start the session because headers have already been sent by "/var/www/html/membership/libraries/joomla/session/handler/joomla.php" at line 48.
```
The cron.php problem started in Joomla 3.8 [due to a change that Joomla made](https://github.com/joomla/joomla-cms/commit/5b92dc69b39240debc69fb90c7f8d71f060631e8#diff-bf82c85e1f29a328e842c4a195392b10) to `JSessionHandlerJoomla::__construct()` in `libraries/joomla/session/handler/joomla.php`.
It could be fixed by Joomla, however I have given up on that. There was an abandoned [Joomla PR #15742](https://github.com/joomla/joomla-cms/pull/15742), superseded by [PR #21260](https://github.com/joomla/joomla-cms/pull/21260), which is also languishing. Anyway we might be better off fixing this on the CiviCRM side.
This is a similar (but separate) case to [https://github.com/civicrm/civicrm-core/pull/14074](https://github.com/civicrm/civicrm-core/pull/14074), where `extern/rest.php` called `session_start()` before bootstrapping the CMS - that issue referred to Drupal though I am sure that it applied to Joomla also (don't know about Wordpress). Removing session_start() was the solution for that REST session problem.https://lab.civicrm.org/dev/joomla/-/issues/16[Joomla 4.0] Warnings when CiviCRM is uninstalled2022-08-12T05:14:12ZAndrew Thompson[Joomla 4.0] Warnings when CiviCRM is uninstalledTwo PHP warnings are displayed when uninstalling CiviCRM from Joomla 4.0 alpha 11:
```
Warning: require_once(CRM/Utils/String.php): failed to open stream: No such file or directory in /var/www/html/j4/administrator/components/com_civic...Two PHP warnings are displayed when uninstalling CiviCRM from Joomla 4.0 alpha 11:
```
Warning: require_once(CRM/Utils/String.php): failed to open stream: No such file or directory in /var/www/html/j4/administrator/components/com_civicrm/script.civicrm.php on line 226
```
```
Fatal error: require_once(): Failed opening required 'CRM/Utils/String.php' (include_path='.:/usr/share/pear:/usr/share/php') in /var/www/html/j4/administrator/components/com_civicrm/script.civicrm.php on line 226
```Joomla 4 Integrationhttps://lab.civicrm.org/dev/core/-/issues/1197Participants counted even when marked as "0" on price set line item2023-06-18T00:47:30ZphilmorbruParticipants counted even when marked as "0" on price set line item[This issue was originally reported in Jira as [CRM-20784](https://issues.civicrm.org/jira/browse/CRM-20784). Bringing it here to make sure it doesn't get lost as it continues to be a bug in 5.13.5]
When creating a price set, you can se...[This issue was originally reported in Jira as [CRM-20784](https://issues.civicrm.org/jira/browse/CRM-20784). Bringing it here to make sure it doesn't get lost as it continues to be a bug in 5.13.5]
When creating a price set, you can set the number of participants who should be counted for that price option. In this use case, I was setting up a price option for a table (with a participant count of 8) and for a table guest (with a participant count of 0). This way, the person purchasing the table could choose whether or not to register the actual names and emails of the additional table guests without paying more or having them count toward the total number. When testing, the table price option worked as expected, counting 8 participants, but the table guest still counted as 1. If there's an option to set the participant count to 0, then it should not calculate accordingly in the Actual Participant Count.https://lab.civicrm.org/dev/core/-/issues/1191CiviCRM reaching MySQL join limit2023-11-26T16:41:52ZJGauntCiviCRM reaching MySQL join limitI've seen a similar error on a few websites where there is a lot of custom data which can throw up a few issues and it was mentioned I should write up an issue here to see whether it is a common issue in the community.
All current versi...I've seen a similar error on a few websites where there is a lot of custom data which can throw up a few issues and it was mentioned I should write up an issue here to see whether it is a common issue in the community.
All current versions of MySQL and MariaDB have a join limit of 61 and Civi needs to join a lot of tables together (dependant on how much custom data you have). This can cause problems when you have a site with a lot of custom data sets as this join limit is reached which prevents Civi from working as expected.
An example of this would be yesterday where I tried to load up a contact's activities but, because of the amount of custom data sets on the site, I was given an AJAX error and could not view the contact's activities.
After doing a bit of debugging, I found out the reason was because the join limit of 61 had been reached.
I managed to resolve the issue by disabling some unused custom data sets but I am wondering if this is quite a common issue with some bigger sites.
My thinking is that as time goes on, the database will naturally get bigger as more is added to it and the issue will come up more and more.
Looking forward to hearing some thoughts and suggestions on this!https://lab.civicrm.org/dev/drupal/-/issues/81Drupal8: Large export results in logout2019-11-03T16:19:27ZJonGoldDrupal8: Large export results in logoutWhen you attempt a large export of CiviCRM data (~63K contacts, ~15 columns), Symfony kills the session. This is relatively recent behavior (didn't happen in April, for instance). Here are the watchdog logs:
Here are those stack traces,...When you attempt a large export of CiviCRM data (~63K contacts, ~15 columns), Symfony kills the session. This is relatively recent behavior (didn't happen in April, for instance). Here are the watchdog logs:
Here are those stack traces, more readable:
```
Warning: session_start(): Failed to decode session object. Session has been destroyed in Symfony\Component\HttpFoundation\Session\Storage\NativeSessionStorage->start() (line 149 of /var/www/crm.agbu.org/vendor/symfony/http-foundation/Session/Storage/NativeSessionStorage.php)
#0 /var/www/crm.agbu.org/web/core/includes/bootstrap.inc(587): _drupal_error_handler_real(2, 'session_start()...', '/var/www/crm.ag...', 149, Array)
#1 [internal function]: _drupal_error_handler(2, 'session_start()...', '/var/www/crm.ag...', 149, Array)
#2 /var/www/crm.agbu.org/vendor/symfony/http-foundation/Session/Storage/NativeSessionStorage.php(149): session_start()
#3 /var/www/crm.agbu.org/web/core/lib/Drupal/Core/Session/SessionManager.php(164): Symfony\Component\HttpFoundation\Session\Storage\NativeSessionStorage->start()
#4 /var/www/crm.agbu.org/web/core/lib/Drupal/Core/Session/SessionManager.php(118): Drupal\Core\Session\SessionManager->startNow()
#5 /var/www/crm.agbu.org/vendor/symfony/http-foundation/Session/Session.php(57): Drupal\Core\Session\SessionManager->start()
#6 /var/www/crm.agbu.org/web/core/lib/Drupal/Core/StackMiddleware/Session.php(53): Symfony\Component\HttpFoundation\Session\Session->start()
#7 /var/www/crm.agbu.org/web/core/lib/Drupal/Core/StackMiddleware/KernelPreHandle.php(47): Drupal\Core\StackMiddleware\Session->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#8 /var/www/crm.agbu.org/web/core/modules/page_cache/src/StackMiddleware/PageCache.php(106): Drupal\Core\StackMiddleware\KernelPreHandle->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#9 /var/www/crm.agbu.org/web/core/modules/page_cache/src/StackMiddleware/PageCache.php(85): Drupal\page_cache\StackMiddleware\PageCache->pass(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#10 /var/www/crm.agbu.org/web/core/lib/Drupal/Core/StackMiddleware/ReverseProxyMiddleware.php(47): Drupal\page_cache\StackMiddleware\PageCache->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#11 /var/www/crm.agbu.org/web/core/lib/Drupal/Core/StackMiddleware/NegotiationMiddleware.php(52): Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#12 /var/www/crm.agbu.org/vendor/stack/builder/src/Stack/StackedHttpKernel.php(23): Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#13 /var/www/crm.agbu.org/web/core/lib/Drupal/Core/DrupalKernel.php(693): Stack\StackedHttpKernel->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#14 /var/www/crm.agbu.org/web/index.php(19): Drupal\Core\DrupalKernel->handle(Object(Symfony\Component\HttpFoundation\Request))
#15 {main}.
```
```
RuntimeException: Failed to start the session in Symfony\Component\HttpFoundation\Session\Storage\NativeSessionStorage->start() (line 150 of /var/www/crm.agbu.org/vendor/symfony/http-foundation/Session/Storage/NativeSessionStorage.php)
#0 /var/www/crm.agbu.org/web/core/lib/Drupal/Core/Session/SessionManager.php(164): Symfony\Component\HttpFoundation\Session\Storage\NativeSessionStorage->start()
#1 /var/www/crm.agbu.org/web/core/lib/Drupal/Core/Session/SessionManager.php(118): Drupal\Core\Session\SessionManager->startNow()
#2 /var/www/crm.agbu.org/vendor/symfony/http-foundation/Session/Session.php(57): Drupal\Core\Session\SessionManager->start()
#3 /var/www/crm.agbu.org/web/core/lib/Drupal/Core/StackMiddleware/Session.php(53): Symfony\Component\HttpFoundation\Session\Session->start()
#4 /var/www/crm.agbu.org/web/core/lib/Drupal/Core/StackMiddleware/KernelPreHandle.php(47): Drupal\Core\StackMiddleware\Session->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#5 /var/www/crm.agbu.org/web/core/modules/page_cache/src/StackMiddleware/PageCache.php(106): Drupal\Core\StackMiddleware\KernelPreHandle->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#6 /var/www/crm.agbu.org/web/core/modules/page_cache/src/StackMiddleware/PageCache.php(85): Drupal\page_cache\StackMiddleware\PageCache->pass(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#7 /var/www/crm.agbu.org/web/core/lib/Drupal/Core/StackMiddleware/ReverseProxyMiddleware.php(47): Drupal\page_cache\StackMiddleware\PageCache->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#8 /var/www/crm.agbu.org/web/core/lib/Drupal/Core/StackMiddleware/NegotiationMiddleware.php(52): Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#9 /var/www/crm.agbu.org/vendor/stack/builder/src/Stack/StackedHttpKernel.php(23): Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#10 /var/www/crm.agbu.org/web/core/lib/Drupal/Core/DrupalKernel.php(693): Stack\StackedHttpKernel->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#11 /var/www/crm.agbu.org/web/index.php(19): Drupal\Core\DrupalKernel->handle(Object(Symfony\Component\HttpFoundation\Request))
#12 {main}.
```
The first issue is a warning that the session is too large, and so it destroys the session (which causes a logout). The second issue is of "error" severity saying the session can't be found.
Here's someone else having a similar issue in Symfony (not Drupal8): https://github.com/symfony/symfony/issues/26623tottentotten